A really common issue when people start to look towards scaling horizontally (adding on additional web/app servers) is session persistence.
Rackspace Cloud Load Balancers as a Service offers session persistence for HTTP (Port 80) traffic. This is done by the LB injecting a cookie into the response that specifies a node. The next time the user requests a page, they send the cookie and the load balancer reads it then directs traffic to the correct node.
This Session Persistence does NOT work on HTTPS (Port 443) because the LB is not able to terminate SSL. This means that the LB has no way to read the cookie being sent by the browser to achieve persistent sessions (and for that matter, no way to inject the cookie either).
Even if you are just load balancing port 80 traffic, what happens if you want to change or modify some code on a node? If you pull it out of rotation, it will go into a draining state where existing sessions can still connect; Not exactly a fast solution.
The solution to Load Balancing HTTPS or simply to load balancing without having to worry about session persistence at the LB is to store your sessions somewhere else. But where? You can store them in a Database if you want, but more than likely your database is busy enough as it is. A better solution would be to store the sessions on a separate memcache server.
For the uninitiated, memcache was originally created by livejournal. What it does is fairly simple: Gives you control over a certain amount of memory on the server so that you can store anything you want in there. This allows you to retrieve it much faster than if you had to read from disk. You can store DB query results, pages, or practically anything. We are going to store sessions.
This is assuming a brand new install of CentOS 5.5 from Rackspace Cloud Servers. First, let’s setup the memcache server.
You will need the EPEL repo, so run this to install it:
rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-4.noarch.rpm
Now that the EPEL repo is available, you can use yum to install it (while we are doing this, might as well install vim and tcpdump so we can watch it working)
yum install memcached tcpdump vim-enhanced
Next up, we need to setup the very simply config file that memcache uses. It will be at /etc/sysconfig/memcached. Here is mine:
PORT="11211" USER="memcached" MAXCONN="1024" CACHESIZE="1500" OPTIONS="-l 10.3.3.3"
The variables are pretty straight forward: Port (default for memcache is 11211) User, Maximum Connections, Cache Size (How much memory in MB you are allowing memcache access to), Options (I specified -l for listen and told it to listen on the private IP address only. This would be the IP of the Memcache server, NOT the web server(s))
Next up, start the daemon and make it start on boot.
/etc/init.d/memcached start && chkconfig memcached on
Next up, secure the memcache server. We don’t want to allow just anyone on the private network access to memcache. A rule set like this should do. Note that 10.1.1.1 and 10.2.2.2 would represent my Web Server Private IPs. If you don’t know the difference between -A and -I read up on it Here
iptables -I INPUT -p tcp --dport 11211 -s 10.1.1.1 -j ACCEPT iptables -I INPUT -p tcp --dport 11211 -s 10.2.2.2 -j ACCEPT iptables -A INPUT -p tcp --dport 11211 -j DROP
That’s it for the memcache server. Now all you have to do is setup the web server to write the sessions to the correct place.
The Web Server
Again, this is assuming a stock Centos 5.5 server from Rackspace, so we have to install what we need.
yum install httpd php php-pecl-memcache vim-enhanced
Now you can test that php has the memcache module loaded in.
Look for memcache; it should be there.
Start apache and make it start on boot
/etc/init.d/httpd start && chkconfig httpd on
Open up iptables on the websevrer:
iptables -I INPUT -p tcp --dport 80 -j ACCEPT
Now we need to edit the php configuration file to tell it that we want to save sessions to memcache, and where our memcache server is. This file will be at /etc/php.ini.
Look for these 2 lines:
session.save_handler = files session.save_path = "/var/lib/php/session"
You will change these to (where 10.3.3.3 is your memcache servers IP address:
session.save_handler = memcache session.save_path = "tcp://10.3.3.3:11211"
This simply tells php to write sessions to memcache and gives the address and port of the memcache server.
Now restart apache so that this will take effect:
All that is left to do now is test it. I created a file named sessiontest.php in /var/www/html that contains:
<?php session_start(); $_SESSION['test']="This is my Session!"; ?>
Start up a tcpdump on the memcaceh server listening for 11211:
tcpdump -i eth1 port 11211
Then access your test page at http://YourIpAddress/sessiontest.php. You will see that there is activity on the memcache server when the page is activated. If you really want to see the test in action, start up a 2nd web server with the exact same configuration, but change the script to:
<?php session_start(); echo $_SESSION['test']; ?>
You will see it echo out the session that was created on the 1st web server.
That about covers it, Leave a message if you have any questions!