Ubuntu 16 and Varnish 4 with Drupal

Geoff Stratton Networking, Systems Administration

This article describes how to install Varnish 4 with Drupal 7 on Ubuntu 16.04. The procedure is similar to installing Varnish 3 in my Drupal recipe, but the update to Varnish 4 requires a few changes. These are described below.

Installing Varnish 4

root@ubuntu:/# apt-cache madison varnish
   varnish |    4.1.1-1 | http://us.archive.ubuntu.com/ubuntu xenial/universe amd64 Packages

Run apt-get install varnish. Then ensure your /etc/default/varnish has the correct startup settings:

# Should we start varnishd at boot?  Set to "no" to disable.
START=yes

## Alternative 2, Configuration with VCL
#
# Listen on port 6081, administration on localhost:6082, and forward to
# one content server selected by the vcl file, based on the request.
#
DAEMON_OPTS="-a :6081 
             -T localhost:6082 
             -f /etc/varnish/default.vcl 
             -S /etc/varnish/secret 
             -s malloc,256m"

Configure Your Web Server

We’re using Nginx here, but the ports and concepts are identical for Apache virtual hosts. In your /etc/nginx/sites-available/mysite.com file, ensure Nginx is listening on 8080:

server {
        listen 127.0.0.1:8080 default_server;
     
        root /path/to/drupal;
        etc.

And if you’re running SSL on your Drupal site, ensure your /etc/nginx/sites-available/ssl-mysite.com is correct too:

server {
        listen 443 ssl default;
        server_name mysite.com;

        ssl_certificate /etc/ssl/certs/secret.pem;
        ssl_certificate_key /etc/ssl/private/secret.key;

        location / {
            # Pass the request on to Varnish.
            proxy_pass  http://127.0.0.1;

            # Pass some headers to the downstream server, so it can identify the host.
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

            # Tell any web apps like Drupal that the session is HTTPS.
            proxy_set_header X-Forwarded-Proto https;

            proxy_redirect     off;
        }
}

Varnish doesn’t do SSL, so we’ve set up Nginx to accept incoming requests on 443, decrypt them, and pass them to Varnish, which (if necessary) will then pass back to Nginx for processing. An Nginx->Varnish->Nginx sandwich.

Set up Varnish for Drupal

Edit your /etc/varnish/default.vcl file:

vcl 4.0;

#acl upstream_proxy {
#  "127.0.0.1";
#}

# Default backend definition. Set this to point to your content server.
backend default {
  .host = "127.0.0.1";
  .port = "8080";
}

acl purge {
  "localhost";
  "127.0.0.1";
}

sub vcl_recv {
  # Do not cache these paths.
  if (req.url ~ "^/status.php$" ||
      req.url ~ "^/update.php" ||
      req.url ~ "^/install.php" ||
      req.url ~ "^/apc.php$" ||
      req.url ~ "^/admin" ||
      req.url ~ "^/admin/.*$" ||
      req.url ~ "^/user" ||
      req.url ~ "^/user/.*$" ||
      req.url ~ "^/users/.*$" ||
      req.url ~ "^/info/.*$" ||
      req.url ~ "^/flag/.*$" ||
      req.url ~ "^.*/ajax/.*$" ||
      req.url ~ "^.*/ahah/.*$" ||
      req.url ~ "^/system/files/.*$") {

    return (pass);
  }

  # Always cache the following file types for all users. This list of extensions
  # appears twice, once here and again in vcl_fetch so make sure you edit both
  # and keep them equal.
  if (req.url ~ "(?i).(pdf|asc|dat|txt|doc|xls|ppt|tgz|csv|png|gif|jpeg|jpg|ico|swf|css|js)(?.*)?$") {
    unset req.http.Cookie;
  }

  # Remove all cookies that Drupal doesn't need to know about. We explicitly
  # list the ones that Drupal does need, the SESS and NO_CACHE. If, after
  # running this code we find that either of these two cookies remains, we
  # will pass as the page cannot be cached.
  if (req.http.Cookie) {
    # 1. Append a semi-colon to the front of the cookie string.
    # 2. Remove all spaces that appear after semi-colons.
    # 3. Match the cookies we want to keep, adding the space we removed
    #    previously back. (1) is first matching group in the regsuball.
    # 4. Remove all other cookies, identifying them by the fact that they have
    #    no space after the preceding semi-colon.
    # 5. Remove all spaces and semi-colons from the beginning and end of the
    #    cookie string.
    set req.http.Cookie = ";" + req.http.Cookie;
    set req.http.Cookie = regsuball(req.http.Cookie, "; +", ";");
    set req.http.Cookie = regsuball(req.http.Cookie, ";(SESS[a-z0-9]+|SSESS[a-z0-9]+|NO_CACHE)=", "; 1=");
    set req.http.Cookie = regsuball(req.http.Cookie, ";[^ ][^;]*", "");
    set req.http.Cookie = regsuball(req.http.Cookie, "^[; ]+|[; ]+$", "");

    if (req.http.Cookie == "") {
      # If there are no remaining cookies, remove the cookie header. If there
      # aren't any cookie headers, Varnish's default behavior will be to cache
      # the page.
      unset req.http.Cookie;
    }
    else {
      # If there is any cookies left (a session or NO_CACHE cookie), do not
      # cache the page. Pass it on to Apache directly.
      return (pass);
    }
  }
}

sub vcl_recv {
  # Check the incoming request type is "PURGE", not "GET" or "POST".
  if (req.method == "PURGE") {
    # Check if the IP is allowed.
    if (!client.ip ~ purge) {
      # Return error code 405 (Forbidden) when not.
      return (synth(405, "Not allowed."));
    }
    return (purge);
  }
}

Set Up Drupal

Install the Drupal Varnish module. On your Varnish module configuration page in Drupal’s admin interface, pick “4.x” for your Varnish version and copy the contents of /etc/varnish/secret into the “Varnish Control Key” field. Finally, ensure that the Varnish Control Terminal field is pointing to your administration port specified in /etc/default/varnish (127.0.0.1:6082). After saving settings, the config page should tell you that Varnish is running.

Finally, ensure that Varnish is set to run on boot:

root@ubuntu:/# ln -s /lib/systemd/system/varnish.service /etc/systemd/system/varnish.service
root@ubuntu:/# systemctl start varnish

systemctl list-units --type=service --all should show varnish.service, varnishlog.service, and varnishncsa.service all loaded/active.

Final Checks

root@ubuntu:/# netstat -ntlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 127.0.0.1:3306          0.0.0.0:*               LISTEN      1463/mysqld     
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      5657/varnishd   
tcp        0      0 127.0.0.1:8080          0.0.0.0:*               LISTEN      5492/nginx -g daemo
tcp        0      0 0.0.0.0:443             0.0.0.0:*               LISTEN      5492/nginx -g daemo
tcp        0      0 0.0.0.0:22044           0.0.0.0:*               LISTEN      1151/sshd       
tcp        0      0 127.0.0.1:6082          0.0.0.0:*               LISTEN      5657/varnishd   
tcp6       0      0 :::80                   :::*                    LISTEN      5657/varnishd   
tcp6       0      0 :::22044                :::*                    LISTEN      1151/sshd       
tcp6       0      0 ::1:6082                :::*                    LISTEN      5657/varnishd

HTTP headers will show the presence of Varnish 4:

$ curl -I http://mysite.com/
HTTP/1.1 200 OK
Server: nginx/1.10.0 (Ubuntu)
Date: Tue, 29 Nov 2016 16:05:04 GMT
Content-Type: text/html; charset=utf-8
Expires: Sun, 19 Nov 1978 05:00:00 GMT
Cache-Control: no-cache, must-revalidate
X-Content-Type-Options: nosniff
Content-Language: en
X-Frame-Options: SAMEORIGIN
X-UA-Compatible: IE=edge,chrome=1
X-Generator: Drupal 7 (http://drupal.org)
Link: </node/354>; rel="shortlink",</node/354>; rel="canonical"
X-Varnish: 230275
Age: 0
Via: 1.1 varnish-v4
Accept-Ranges: bytes
Connection: keep-alive

Loading

Leave a Reply

Your email address will not be published. Required fields are marked *