Load Balancing and Server Architectures

Status
Not open for further replies.
H

HairyHun

Masturbating Bandit
Oct 11, 2007
576
2
0
Hello,

I've been reading about load balancing, redundant servers, and the works and it's a fascinating field. Fascinating but also confusing.

I'd like to hear from any one who had experiences with it, and maybe short case examples.

In my case, I got a site that will provide templated website for people.

basically people...
  • Open account
  • Customize template look
  • Enter inventory

A few questions I got:

How do you know:
  • How to split it up? (Just multiple web servers, or have database servers too, Maybe dns or other? )
  • Which you need more ( do you need more webservers than database servers, or the other way, how many )
  • What kind of load balancing material and how to configure them
  • Backup & recovery policies?
  • Support
  • Cost Vs Service?


Also what possible road maps ( what parts you start by doubling, etc)
and how do you it's the right time to double or multiple a certain part of the system ( diagnostic tools )

Thanks
HH
 


Thats a pretty loaded question and the answers could go on for pages and pages. But here is how most LAMP applications go. Scaling something like Java can be different.

1. You can start with a single server. But design your application so that you can split it up into multiple servers.

2. For most LAMP applications and forums, all of the interactive data is kept in a MySQL database. You may allow your users to upload static content (images, css files etc)

3. Session data should be stored in the MySQL database so the same user's session can be supported across multiple web front-end servers

4. Once you outgrow 1 server, move to a web front-end server plus a database back-end server. Generally, you'll want to invest more in the database back-end server since it is your site's "brain".

5. The MySQL server will need to be optimized heavily. Your queries will need to be optimized. Most of the processing load should now be on the web front-end.

6. At this point it should be easy to add more web front-end servers for both: A) additional processing power and B) redundancy in case one front-end server is down.

7. A front-end load balancer can be used to distribute the load of HTTP requests across your web front-end servers. Load Balancers generally are much more reliable than single web servers because they have relatively low power needs and no hard disk. Depending on budget, two load-balancers can be deployed in a High Availability config. This means they share the same IP. If LB A goes down, LB B immediately picks up the IP address so you barely notice any interruption.

8. Load balancers can also offload SSL processing if your app is SSL heavy (such as ecommerce). This works well because you can get load balancers with special SSL hardware that handles the encryption in ASIC, instead of burning up normal CPU cycles.

9. Once your web front-end has grown to several servers you can setup a standby database back-end server and replicate data to it (MySQL has replication functions built in).

10. For sites with large databases (like large forums), another database box can be configured and optimized just for search functions. This offloads search functions from your main database server. Search is generally less of a priority than the essential Read/Write functions on your main database server.

11. If your users are uploading static content, you will want to use network storage or SAN between your web front-end servers so that an upload on one box is seen on the others instantly. Depending on how much static content you're serving, you may setup separate web front-ends that only handle static content, and optimize them for that (such as lighttpd).


So, in summary, there isn't a 'one size fits all' plan. Every component of the design is based on the particular site's budget, load, functions, uptime goals, etc. Note that multiple servers can be used for the purposes of both distributing a high load and also offering maximum uptime. Some clusters may be put together just for distributing a high load but still have single points of failure resulting overall in no more reliability than a single server small website.

You will need competent system administrators to monitor and tune the boxes, and to determine when you should upgrade. Some hosting companies provide load balancing hardware and support it for you. Generally if you want a hosting company to take care of ALL of this for you AND do it right, it will be more expensive than you think simply because of the engineering hours that go into the design, setup, administration and monitoring.
 
HairyHun said:
How do you know:
  • How to split it up? (Just multiple web servers, or have database servers too, Maybe dns or other? )
  • Which you need more ( do you need more webservers than database servers, or the other way, how many )
  • What kind of load balancing material and how to configure them
  • Backup & recovery policies?
  • Support
  • Cost Vs Service?
Click to expand...


It depends on your app. Is it read or write intensive? How busy do your web servers get? Is the goal load balancing or HA? It really comes down to "What's the current bottleneck?"

Speaking on the front end side there are several ways to distribute load across front ends, from round-robin DNS (don't knock it, it works) to load balancers, proxies, and geo-based redirection. On the database layer you can add read slaves, partition your data set, denormalize, etc.

It all comes down to the app though.

I gave a presentation to my local Linux user's group on LAMP tuning and my experiences scaling at b5media
http://www.muug.mb.ca/meetings/lamp_tuning_seanwalberg.pdf

The format is largely based on what LiveJournal did:

Index of /words

Sean
 
Short answer: If you're hitting 100K pageviews/hour, then you need load balancing. If you're site is brand new, don't worry bout it bro!


When you grow to the point where dual SQL servers are required, then a load balanced cluster should be looked at.

Look at F5 BIG-IP products.

Keep in mind you'll spend ~$45K for a fully redundant and clustered system.
 
bb_wolfe said:
Keep in mind you'll spend ~$45K for a fully redundant and clustered system.
Click to expand...

While F5 is good gear, if all you're doing is l3/4 balancing, you're wasting your money. You're also making a lot of assumptions about the application to say you should throw a pair of F5's in front of them. Without getting too far down into it, you can have a fully redundant and clustered system for the cost of two servers, as there are many HA solutions out there for Windows and Linux that are free and perform just as well at that scale as a hardware solution. (And, failing that, there are a lot of cheaper load balancers out there that will do the job)


That said, I do agree with the first statement to not worry about it now. Scaling your site should be a problem you're looking forward to. It means you've got traffic. Concentrate on other things, fix your scaling problems later. Read through Getting Real, a look at building a business around a web app by the guys that created Basecamp.

Sean
 
I read the essays from the signal37 guys. Very nice.


So at first we'll go with a 2nd server. One to run things and another one to test the updates before transfering it and possible backup.

I was looking at the config.

Our first one is a
  • quad 2.4 core 2
  • 2 Gig ram
  • 1 Tb HD
  • and 2 g traffic.

Seems ok, except the ram seems a little low for a server.

I would think get the first server with a nice config, possibly max out the ram and other such things. So you maximize how long you can last on that first step before going into multi.

Should I consider RAID at this point? they offer 1 SW, 1 HW, 5 HW and 10 HW. (each with their requirement of additional drives, etc)
Port is 10mb. I can get 100mb for 5$ more per month.

HH
 
I configure all of my servers with RAID. Minimum of RAID1 for low-end servers. Software RAID is ok for low-end boes. For higher performance boxes, go with RAID10 in hardware for redundancy and performance.
 
SeanW said:
While F5 is good gear, if all you're doing is l3/4 balancing, you're wasting your money. You're also making a lot of assumptions about the application to say you should throw a pair of F5's in front of them. Without getting too far down into it, you can have a fully redundant and clustered system for the cost of two servers, as there are many HA solutions out there for Windows and Linux that are free and perform just as well at that scale as a hardware solution. (And, failing that, there are a lot of cheaper load balancers out there that will do the job)

Sean
Click to expand...


No, you need to worry about scale in your architecture now, unless you'd like to end up like Twitter. They are the perfect example of what happens when you don't think about scalability up front.

Yes, you can build a site this way, but you can't have a fully reliable site with just two servers. In this description one of those servers has to handle the clustering management, and that means you are still in a situation where if you lose that machine you are offline. Basically, you're suggesting to go from one configuration to another where there is a single point of failure.

You should also ask whether you're just doing "dumb" load balancing or trying to ensure that you can scale, secure your application, and be able to futureproof your setup. Clustering software addresses only one thing: dumb load balancing. It can't help with security, or make servers more efficient, or offload SSL, or improve performance. And the clustering software requires resources, which means the machine on which it's running isn't going to perform as well as the other, unburdened server.

Yes, if you're just trying to do L3/L4 load balancing, a BIG-IP may be more than you need - unless you are hoping/planning on needing a lot more capacity in the future. Then you do need something more than clustering software, and you certainly need to consider something a lot more intelligent than just a dumb load balancer.

A KEMP or CoyotePoint or even Zeus (ON A THIRD MACHINE) would be more acceptable than clustering software for reliability and an affordable alternative at this stage. It would additionally let you architect for a more robust infrastructure later, when you grow beyond the capacity of even two servers and an SMB load balancer like KEMP or CoyotePoint. That means less work when you have to change things around later to support more traffic.
 
lmacvittie said:
No, you need to worry about scale in your architecture now, unless you'd like to end up like Twitter. They are the perfect example of what happens when you don't think about scalability up front.

Yes, you can build a site this way, but you can't have a fully reliable site with just two servers. In this description one of those servers has to handle the clustering management, and that means you are still in a situation where if you lose that machine you are offline. Basically, you're suggesting to go from one configuration to another where there is a single point of failure.

You should also ask whether you're just doing "dumb" load balancing or trying to ensure that you can scale, secure your application, and be able to futureproof your setup. Clustering software addresses only one thing: dumb load balancing. It can't help with security, or make servers more efficient, or offload SSL, or improve performance. And the clustering software requires resources, which means the machine on which it's running isn't going to perform as well as the other, unburdened server.

Yes, if you're just trying to do L3/L4 load balancing, a BIG-IP may be more than you need - unless you are hoping/planning on needing a lot more capacity in the future. Then you do need something more than clustering software, and you certainly need to consider something a lot more intelligent than just a dumb load balancer.

A KEMP or CoyotePoint or even Zeus (ON A THIRD MACHINE) would be more acceptable than clustering software for reliability and an affordable alternative at this stage. It would additionally let you architect for a more robust infrastructure later, when you grow beyond the capacity of even two servers and an SMB load balancer like KEMP or CoyotePoint. That means less work when you have to change things around later to support more traffic.
Click to expand...


Until you reach your bottlenecks, you won't know where they are. What are you going to optimize?

Until you have users on your system actually using them, you don't know where the load will come from.

I'm not saying don't think about scaling, I'm just saying that it's not the most important thing at this point. No point spending all your time trying to optimize when you have no idea what to optimize for, and sacrificing your time that could better be spent developing your business.

Sean
 
I think the OP is biting off more than they can chew. If they have the kind of traffic that requires a load balancer as opposed to a faster server, then we're the last people they'd be talking to.
 
The ghetto solution, use UltraDNS to load balance via DNS (it's very very cheap), then setup Panda (free) to load balance between your internal web-servers.

For easy setup, have one or two Master SQL servers and a scalable setup of Slaves MYSQL servers.

Obviously for web-servers, which are your front-end, you can just keep adding those also.

How many of each you have as people have said, greatly depends on the type of traffic ( is it a DB transaction heavy site and light on client side front end or vice versa).

You can even forgo the UltraDNS part and just use Panda until you are ready. So have Panda Load Balancing + N Webservers + Memcached + 1 Master SQL + 1 Slave SQL. Where N is a function of your traffic.

BTW, great posts SeanW and Sloop !!
 
Last edited:
SeanW said:
Speak for yourself, I do this for a living ;)

Sean
Click to expand...



As do I, on a much larger scale than 99.9% of users on this forum. Just last night we had over 10 million pageviews. Of course, the VM host manager fucked I/O up and our performance was shit.

Which is why I'm just finishing my 10th beer since 2am! Of course after last night I don't care to see a fucking cluster/load balancer/ssl accelerator ever again.
 
We now discussed what are the things you can do and when.

My next question: how do you know WHEN is when?

What are the tools for diagnostic? What are the symptoms of imminent problems? What about favorite tools for these?

HH
 
HairyHun said:
My next question: how do you know WHEN is when?
Click to expand...

Make sure to monitor page load times. cURL, IOS SAA, or third party utilities work well. You can also link your request rate to the number of servers to get an idea of front end load.

I do think you're getting ahead of yourself though. First get customers and traffic. Then, measure everything. Use that data to help you see where your bottlenecks are, and then work on those.

Edit: I did an article on this using cURL: Expose Web Performance Problems using RRDTool

Sean
 
SeanW said:
Make sure to monitor page load times. cURL, IOS SAA, or third party utilities work well. You can also link your request rate to the number of servers to get an idea of front end load.

I do think you're getting ahead of yourself though. First get customers and traffic. Then, measure everything. Use that data to help you see where your bottlenecks are, and then work on those.

Edit: I did an article on this using cURL: Expose Web Performance Problems using RRDTool

Sean
Click to expand...

Pretty much the 99.999% answer to the "when" question.

Only thing I would add is as your budget allows.
 
SeanW said:
I gave a presentation to my local Linux user's group on LAMP tuning and my experiences scaling at b5media
http://www.muug.mb.ca/meetings/lamp_tuning_seanwalberg.pdf
Click to expand...



Very nice presentation ( although some parts were too technical for me ).

I saw that at the end you were looking for a server ninja.

How do you find such a server ninja. I mean if you need a ninja because you're not technical. How do you know if its a good ninja, if you are non-technical? Catch 22.

Can you hire a good one part time? ( or is it the kind of thing that hard to do part time ) and what is good rates for that?

Yes I am probably getting ahead of myself. But I like to get an idea of what's beyond the horizon as I take the first steps. This is also a field that is completely new to me, so this way I have time to read up on some of the things. :D

Thanks all for your great contributions so far!

HH
 
HairyHun said:
How do you find such a server ninja. I mean if you need a ninja because you're not technical. How do you know if its a good ninja, if you are non-technical? Catch 22.
Click to expand...

I got into the role because the owner was a friend of mine, and I would give him a hand with his server migrations and operations. As the company grew and took funding, they brought me in as a contractor. This was over the span of about 2 years. My full time replacement was found through a recruiter.

So, look to your personal network. Somewhere must be a friend or two who knows something about servers / networking (and can learn the rest), and can help you out until you have the problem of needing a full time person.

As for rates it will depend a lot. Hell, I did it for free for a while because the owner was an old friend and I found the work fun.

As an aside, I remember listening to a podcast (might have been Shoemoney's) where he interviewed the lead architect for Digg. He started out the same way, helping out as a friend, eventually growing into a full time role.

Sean
 
Status
Not open for further replies.
Top Bottom