Varnish running on VPS with cPanel - Speed up Wordpress, Joomla, Drupal and More

The information in this article is targeted to Sysadmin and Developers who are very familiar with doing things at the command line and understand how things like Varnish and Apache work.  ALWAYS do a backup of your system or do a test implementation on another server before you start messing with production.  This information is provided as-is and you accept all risks when installing any of these applications.  This article is for informational purposes only.

I have been working with Varnish a number of years now.  It's truly an amazing open source product for web acceleration.

I want to start this blog entry with my opinion on Varnish and Apache before I get into the details of how I made it all work together.  Sure, there is lightspeed, there is nginx and a dozen other web servers out there that all claim to be the fastest, work with Varnish, etc.  If you are one of those people that feel you'll get better performance without using Apache then go for it.  I personally am very familiar with Apache.  I use Apache every day and my preference is to use that since it is a mature product with a huge amount of support and options.   The others while new and fast have their own share of trouble.  That being said my approach here was to improve a web server with Varnish and leave Apache in place to do what it has been doing very well for a very long time.  This article is not pro-Apache it is simply an illustration on how a minimal amount of work can get you huge performance gains on modest hardware.

So let's talk about the modest hardware I have here.  I have a customer who has a dedicated VPS and I provide the support and management of the hosting.

The hardware used:

• 8 Cores - Intel(R) Xeon(R) CPU L5506 @ 2.13GHz
• The hypervisor is Parallels so the Kernel and other OS components are in shared memory not specifically the memory of my VPS.  This allows you to have more ram available.
• 4GB Ram available to VPS (originally 2GB)
• 120GB Hard Disk

This is some very modest hardware here.    It is a reasonably sized server but by no means a 64 core box with dedicated 64 gigs of ram.  What you have here is about $150/mo in hosting when you add in the other bells and whistles.  So it is very affordable. 

The software used (all on the same VPS):

• MySQL 5.1
• PHP 5.4.x
• Apache 2.2
• varnishd (varnish-3.0.5 revision 1a89b1f)
• WHM 11.42.1 with cPanel 
• CENTOS 5.9 i686 virtuozzo
• Wordpress MU

This modest hardware was starting to run out of memory at the 2GB configuration so the customer agreed to an upgrade and an installation of Varnish.  They had reservations about varnish simply because they had tried some plugin modules for WordPress and they had mixed results.  Varnish I know is very configurable. 

So my first attempt was to get Varnish installed and get a port running varnish to test.  I ran into all sorts of trouble because of the way cPanel hijacks everything.  The customer uses cPanel because they integrate with their API for creating accounts for their customers.  So cPanel had to stay. 

So I did some Google searches and came up with a company called Unixy who has a cPanel varnish plug in.  It's not free but it is close.  The module will cost you $17.89 quarterly or you can pay for an entire year for $71.89.  It's worth every penny given the time it will save you from a Sysadmin standpoint.  Basically $6/mo which I rolled into the new hosting fee as part of the upgrade. 

You can see from the Cacti graph above almost exactly when I put Varnish into place.  Instead of all these spikes for Apache workers (and consequentially load) is now replaced with a very even and consistent amount of work.  This is because once the Varnish cache had warmed the load for a lot of the response to the web browsers were coming from Varnish and not Apache.   In fact the Hit/Miss ratio was around 40%-50%.  This means literally half the traffic is now coming form the cache. 

So lets talk about how we got here.  I think this should save anyone trying to do the same thing a ton of time.  Unlike a lot of the other tutorials out there this tutorial focuses on the Application itself (Wordpress) instead of focusing on just installing Varnish on a single server.  

You will need to get your site ready for all this and I suggest you take the time to do right.  This will give you the largest increase in performance.  

MySQL

We start with  MySQL since it does the most heavy lifting on web applications these days.  If you have not gone through your /etc/my.cnf I suggest you take some time and do it.  Anyone reading this article should know how to do this manually however Percona has a MySQL Configuration Tool which can get you on the right path.  Setup MySQL so it is performing well.  My configuration is set to use around 1.6GB of memory so it can cache queries that run often.

Apache & PHP

Here is another thing you want to do beforehand.  I use FastCGI on this installation.  It is a good balance of security and speed.  It also supports opcode cache which helps speed things up. You can configure all of this within EasyApache in WHM.  With this and the MySQL changes I was able to speed up the site itself by roughly 20%.  When you take the time and target the application with specific MySQL and Apache settings you can easily gain performance.

Application Cleanup

This is critical as well.  Have the developer review the logs on the website and see what errors are popping up.  If you have none thats great, otherwise fix the errors.  They don't help performance and it eats up disk IO writing to the logs.  Disable warnings if you must but fix the errors. 

Once you have your system tweaked it is time to throw it into overdrive.  This is where Varnish comes into play.  SSH into your server and lets get started. 

sudo yum install varnish

This will get varnish installed on your system.  It wont actually do anything just yet but you will have it installed. 

Next, go download the Unixy Varnish Plugin for cPanel.  Note they have ones for other web servers such as Nginx.  We will only deal with the Apache plugin.  We are also only using the one for WHM which controls the entire server.  No end user cPanel features are enabled.  Users mucking around with Varnish sounds like a truly bad idea. 

Once you have downloaded it, unzip and expand the tar.  Then go into the directory where you expanded the tar to and you'll see a installvarnish script.  This is actually a python script.  Execute it with:

sudo python installvarnish

Now it will check dependencies and install a huge amount of .vcl files for varnish.   When I checked out how many configuration files were out there after this application did its thing I had new respect for how much time and effort went into the building of this.  The $6/mo is a steal for the time and effort you will save with this.

Note, once you perform this process Varnish will be running and active on your system.  It will NOT however cache anything and it performs in a pass through mode doing nothing at all to your system.

Once it is installed successfully, you'll need to go into WHM and set it up.  You can find it by searching for varnish in WHM.

If you are this far it's time to get this baby running.  I'll now go through the simple configuration values that will get your server some huge performance gains.

The advanced configuration will allow you to setup just about everything related to how varnish is going to perform.  You only need to setup a few things to get started.  You can see in the image below that I have configured some conservative settings.  You can increase these over time for better performance but you'll want to get started with some low values.   

The hints are pretty useful here.  I have the cache TTL set to an hour, I have the file cache set to 4 hours.  Ultimately you want to put some of these values for days if you can.  One setting here that gains huge performance is the Memory Cache.  I have 512MB assigned.  I plan to up this to 1GB.  This is where varnish gains speed since it has the objects in memory not on the disk.  

From this page you can also opt-out based on certain URLs.  For example you do not want your admin panel cached.  You can exclude wp-admin and any time that is on the URL, Varnish will not cache it.  You can even exclude an entire host should you have a site that caching is causing problems with.

You need to go here and turn on the Malloc setting if you want to use the varnish memory.  This will then tell varnish to use memory vs. file cache for performance.  Make absolutely sure that the memory you allocate is always available.  If its not then you'll likely have Varnish, MySQL or Apache crash on you.

Once that all is done go here and enable Varnish.  This will turn on the caching portion and will begin using the values you specified.

This is where the fun begins.  You can now start monitoring varnish by using the Varnish Stats option for the plugin.  The key is the number of hits/misses that you see.  Over time you should start to see hits go up and misses should grow but not as fast.

Browse the site, use the admin, check to see that changes are being reflected on the site.

A few tips on using Varnish long term.

• Version all your dynamic content, e.g. images, CSS, JS, etc.  This way you never have an old version stuck in cache. 
• Use a CSS and JS Minifier.  Generally speaking they will change the filename each time they detect a change and this works the same as file versioning. 
• Never upload a replacement image.  Change the filename, use myimage-1.png or what not.  This will prevent you from having varnish cache the wrong image for hours or days.  
• Flush cache in off hours.  If you need to flush your varnish cache remember that when you do the site is going to suddenly get a huge burst of traffic to Apache to rebuild that cache.  
• Monitor everything.  Keep track with tools like Cacti to see what performance improvements you made. 
• cPanel users are going to get some free bandwidth as part of this.  The reason being is that PHP/Apache is not being called and thus the cPanel hooks to track traffic are not there.  So cache items don't count towards the overall statistics for a user.  However YOUR traffic will be counted.  Service providers meter your data usage differently.  So make sure you're no where near your bandwidth cap before you implement this.

Overall this is a rather simple setup for WHM/cPanel users.  The benefit is that you have a lot of control over your high traffic sites. 

Did you find this article helpful?  If so please leave a comment I'd love to hear your success using Varnish with cPanel.