ThorneLabs

Setup Varnish 4 on CentOS 6 as a Caching Server and Load Balancer

• Updated March 17, 2019


In a previous post, I described how to setup Varnish 2.1.5 on CentOS 6 as a caching server and load balancer. After setting this up, I realized MP4 files were not streaming and large downloads were taking too long to begin.

Varnish 4 is the latest major release of Varnish and has provided fixes to all of these problems.

This post will be very similar to the previous post, but will provide the new VCL syntax that applies to Varnish 4.

Once again, this post assumes you already have two or more web servers running and a new third server running CentOS 6 to install Varnish on.

Install Varnish

To get the latest version of Varnish, add their yum repository:

cat << EOF >> /etc/yum.repos.d/varnish.repo
[varnish]
name=Varnish for Enterprise Linux 6
baseurl=https://repo.varnish-cache.org/redhat/varnish-4.0/el6/
enabled=1
gpgkey=https://repo.varnish-cache.org/GPG-key.txt
gpgcheck=1
EOF

Now, install Varnish:

yum install -y varnish

Configure Varnish

There are two configuration files to edit, /etc/sysconfig/varnish and /etc/varnish/default.vcl.

Assuming you want Varnish to run on port 80, open /etc/sysconfig/varnish and change VARNISH_LISTEN_PORT to the following:

VARNISH_LISTEN_PORT=80

Next, open /etc/varnish/default.vcl and use the following configuration. Be sure to replace PUBLIC_IP_ADDRESS with the IP address of your web servers.

vcl 4.0;

import directors;

backend web1 {
    .host = "PUBLIC_IP_ADDRESS";
    .port = "80";
    .probe = {
        .url = "/";
        .interval = 5s;
        .timeout = 1s;
        .window = 5;
        .threshold = 3;
    }
}

backend web2 {
    .host = "PUBLIC_IP_ADDRESS";
    .port = "80";
    .probe = {
        .url = "/";
        .interval = 5s;
        .timeout = 1s;
        .window = 5;
        .threshold = 3;
    }
}

sub vcl_init {
    new apache = directors.round_robin();
    apache.add_backend(web1);
    apache.add_backend(web2);
}

sub vcl_recv {
    set req.backend_hint = apache.backend();

    unset req.http.Cookie;

    # Do not cache listed file extensions
    if (req.url ~ "\.(zip|sql|tar|gz|tgz|bzip2|bz2|mp3|mp4|m4a|flv|ogg|swf|aiff|exe|dmg|iso|box|qcow2)") {
        set req.http.X-Cacheable = "NO:nocache file";
        return (pass);
    }
}

sub vcl_backend_response {
    # Set cached objects to expire after 1 hour instead of the default 120 seconds.
    set beresp.ttl = 1h;
}

sub vcl_deliver {
    if (obj.hits > 0) {
           set resp.http.X-Cache = "HIT";
    } else {
           set resp.http.X-Cache = "MISS";
    }
}

Enable and Start the varnish Service

Enable the varnish service on boot:

chkconfig varnish on

Finally, start the varnish service:

service varnish start

Verify Varnish is Working

Assuming everything is working properly and you have changed your website’s DNS records to point to the new Varnish server, you should be able to browse to any of your websites in a web browser without issue.

If you have not yet changed your website’s DNS records to point to the new Varnish server, you can still verify Varnish is working by running command curl -i -H 'Host: example.com' varnish.example.com | less (example.com is your website’s domain name and varnish.example.com is the hostname of the new Varnish server).

At the top of the output should be a header called X-Cache. The value of this header will probably be MISS. If you run the curl command again, the value should now be HIT, because Varnish cached the web page after you ran the first curl command.

Once you change your website’s DNS records to point to the new Varnish server, you can run a similar curl command, curl -i http://example.com | less, to verify Varnish is working.

In addition, you can monitor what Varnish is doing by running varnishlog, varnishhist, varnishstat, and varnishtop on the server running the varnish service.

Troubleshooting

If you have problems starting the varnish service, you will find there isn’t much information to troubleshoot with. I ran into a situation where service varnish start simply failed with no debug output.

Output from /var/log/messages only showed No VCL available. In addition, running varnishd -d also only showed No VCL available. Not very useful.

Luckily, you can get much more useful debugging information by running varnishd -C -f /etc/varnish/default.vcl. This actually told me where my syntax error was.

References

If you found this post useful and would like to help support this site - and get something for yourself - sign up for any of the services listed below through the provided affiliate links. I will receive a referral payment from any of the services you sign-up for.

Get faster shipping and more with Amazon Prime: About to order something from Amazon but want to get more value out of the money you would normally pay for shipping? Sign-up for a free 30-day trial of Amazon Prime to get free two-day shipping, access to thousands of movies and TV shows, and more.

Thanks for reading and take care.