Load balancing & Reverse Proxies

[Mahngiel]

I have a web application that's getting a substantial number of concurrent users.  Occasionally, when there are around 2k users the site chokes out and 502's.  When this happens, the CPU (~18%) and RAM (~40%) are relatively low, so I assume this is due to network congestion. 

 

I'm trying to gather as much information as I can before I commit to any solution, but I believe the best approach to a solution revolves around load balancing through a reverse proxy.  I'm hoping there is some experience on these boards with this and am looking for some approach advice and common pitfalls - as well as some suggested hardware specs.

 

I assume for a proxy to balance across two servers, I would need three servers:  The proxy and two cloned production servers.  Some outstanding questions I have are:

 

In regards to a proxy, what is more important: The proxy or the server specs behind it?  I this this is a obvious rhetorical question, but I feel it needs to be asked.  Currently, the site is running on a 3ghz i5 with 4gb of ram served over a 1gbit pipe.

 

In regards to cloning the servers, is there a standard method for the master to copy down to the slave?  My best thought on this would be to set up an scp script, but I've read twitter uses bittorrent, so there has got to be a plethora of ways.

 

Any input on what I have or have not mentioned in regards to balancing and reversing would be appreciated.

 

[Christian F.]

Adding a reverse proxy might not help things, depending upon what exactly is causing the 502 error. What you really need to do is to investigate this further, to accurately determine what causes the 502 and why. Once you've done that, you'll be better equipped to deal with it in the correct manner.

One of the things that you can do, is to monitor the traffic going in and out from your web server, and see if you can't capture the packets that cause the 502 error. If you do, it should be quite easy to track the situation that caused them to be sent. Checking the error logs for any clues is a good thing to do as well, might find some really good clues in there. Same with reproducing the error locally, by using a load-tester in your development environment. (If unable to replicate it, you know it's caused by a condition unique to the production server.)

Profiling your code isn't a bad thing to do either, might find some good clues there. At the very least you'll know how your code performs, which is always a good thing.