Linux virtual server cluster

Why have a Linux virtual server cluster

The idea is simple:

• The user looks up your website.
• The user is connected to the LVS load-balancing computer.
• The connection is forwarded one of the real servers on the local area network.
• A service like the Apache web server or a FTP client on the real server handles the request.
• The user is presented the requested content quickly even though the site is heavily used.
• ???
• Profit !!!

There are many benefits of having a cluster handling incoming requests. The requests are just forwarded, so any kind of computer and OS can be used to serve the real requests. Computers can be added and removed from the cluster real-time, making it simple to minimize downtime during upgrades and more computers can be added easily as your traffic grows.

 What you need

The Linux Virtual Server Project modules where added to version 2.4.28 of the Linux kernel. It is also available in the 2.6.x kernel, but there are many problems reported with using LVS on 2.6 kernels. LVS is known to be rock solid stable on 2.4.x kernels. Patches are available for previous versions of the 2.4.x kernel from the official LVS website.

2.1. Needed kernel configuration

You can configure your kernel by running ''make menuconfig''. Go to the Networking options sub-menu and then the sub-menu IP: Virtual Server Configuration.

2.4.2x Linux Virtual Server kernel configuration

Networking options -> IP: Virtual Server Configuration

[Y] virtual server support (EXPERIMENTAL)

[ ] IP virtual server debugging

(16) IPVS connection table size (the Nth power of 2)

--- IPVS scheduler x

[M] round-robin scheduling

[M] weighted round-robin scheduling

[M] least-connection scheduling

[M] weighted least-connection scheduling

[M] locality-based least-connection scheduling

[M] locality-based least-connection with replication scheduling

[M] destination hashing scheduling

[M] source hashing scheduling

[M] shortest expected delay scheduling

[M] never queue scheduling

--- IPVS application helper

[M] FTP protocol helper

Save your new kernel configuration and rebuild your kernel:

make dep && make clean && make bzImage && make modules && make modules_install

Update your boot manager and your LVS kernel is ready to be used.

2.2. Software: ipvsadm and iptables

Gentoo: emerge ipvsadm iptables
You must turn on IP forwarding on to use the LVS modules. You must turn it on by adding net.ipv4.ip_forward = 1 to your /etc/sysctl.conf.
You are now ready to start iptables.

• Fedora users: service iptables start
• Gentoo users: /etc/init.d/iptables start

Now you must set your iptables firewall rules. For LVS, you must enable IP masquerading.
If your eth0 is connected to the Real Internet and eth1 is connected to your local area network then you can turn on masquerading with the following commands:

• iptables -t nat -P POSTROUTING DROP
• iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

There are many good scripts for creating firewalls available. Gentoo users can save their rules (/etc/init.d/iptables save) so they are applied the next time iptables is started.

Create a server cluster and choose algorytm module

The most common choices are the wlc (weighted least-connection) and wrr (weighted round robin) scheduling algorithms.
This is how simple it is to create a virtual server using weighted least-connection for HTTP port number 80 (replace 66.199.240.50 with the IP you want the virtual server on):
ipvsadm -A -t 66.199.240.50:80 -s wlc

Redirect the incoming traffic

You can explain to the LVS where to send the incoming requests with ipvsadm -a -t Real-IP:port -r Internal-IP:port -m. You can set the server weight (load) with the -w switch. The default is 1, 2 would mean the server gets twice the normal load. Examples:

ipvsadm -a -t 66.199.240.50:80 -r 10.0.0.100:80 -m

ipvsadm -a -t 66.199.240.50:80 -r 10.0.0.101:80 -m -w 2

The single point of failure

Your LVS will be a single point of failure. You can setup multiple A (ipv4) and AAAA (ipv6) records for your domain. This is a good and simple way of doing basic load-balancing. If you have real amounts of traffic then you may want two or three virtual servers load balanced by the DNS records, all connected to different clusters.

Not as easy as it sounds..?

Congratulations, you have the knowledge needed to make a cluster.
Here are some important things you must consider: All the real servers connected to the virtual server must present the content you have created to drive the the visiting consumers into profit-generating behavior. If you have a web forum or a wiki then you are required to setup database clusters and so on to ensure the consumers are driven into purchasing behavior.
If you are mostly serving static files for example to serve an anonymous ftp site with Linux files then it is very easy to synchronize the servers using rsync. It is simple to setup a virtual ftp service with weighted round robin scheduling with the boxes on internal IPs 10.0.0.100, 10.0.0.101 and 10.0.0.102:

  RealIp=66.199.240.50
  ipvsadm -A -t $RealIp:21 -s wlc
  ipvsadm -a -t $RealIp:21 -r 10.0.0.100:21 -m -w 2
  ipvsadm -a -t $RealIp:21 -r 10.0.0.101:21 -m
  ipvsadm -a -t $RealIp:21 -r 10.0.0.102:21 -m