Downloads

Every file from the lesson. Change www.example.com, the certificate paths and /var/www/magento to your own before you use them.

How it fits together

nginx keeps the certificate and hands every request to Varnish. Varnish answers from memory, or asks the second nginx server on port 8080, which runs Magento. Varnish listens on localhost only, so nobody reaches it from outside, and port 80 just redirects to HTTPS.

1. Measure first

Time to first byte of a category page, three times, then a small load test with ab (from apache2-utils): a thousand requests, twenty at a time.

for i in 1 2 3; do curl -sk -o /dev/null -w 'TTFB %{time_starttransfer}s\n' https://www.example.com/women/tops-women.html; done

ab -q -n 1000 -c 20 https://www.example.com/women/tops-women.html | grep -E 'Requests per second|Time per request:.*mean\)'

2. Install Varnish 8

curl -s https://packagecloud.io/install/repositories/varnishcache/varnish80/script.deb.sh | bash
apt install -y varnish && varnishd -V

3. nginx and HTTPS

Replace the site config. Three servers: port 80 redirects, port 443 keeps HTTPS and passes requests to Varnish, and 127.0.0.1:8080 runs Magento with the nginx.conf.sample that ships with it.

upstream fastcgi_backend { server unix:/run/php/magento.sock; }
server {
    listen 80;
    server_name www.example.com;
    return 301 https://$host$request_uri;
}
server {
    listen 443 ssl;
    http2 on;
    server_name www.example.com;
    ssl_certificate     /etc/nginx/ssl/www.crt;
    ssl_certificate_key /etc/nginx/ssl/www.key;
    location / {
        proxy_pass http://127.0.0.1:6081;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-Proto https;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_buffer_size 128k; proxy_buffers 4 256k; proxy_busy_buffers_size 256k;
        proxy_read_timeout 600s;
    }
}
server {
    listen 127.0.0.1:8080;
    server_name www.example.com;
    set $MAGE_ROOT /var/www/magento;
    include /var/www/magento/nginx.conf.sample;
}

Save it as /etc/nginx/conf.d/magento.conf. The certificate here is a self-signed test certificate; on a real domain see Let's Encrypt.

nginx -t && systemctl reload nginx

4. Varnish settings

A systemd override: Varnish on 127.0.0.1:6081, 1 GB of memory for the cache, and bigger header limits. Magento sends long X-Magento-Tags headers; without http_resp_hdr_len and http_resp_size big category pages end in a 503.

[Service]
ExecStart=
ExecStart=/usr/sbin/varnishd -a 127.0.0.1:6081 -f /etc/varnish/default.vcl -P %t/%N/varnishd.pid \
  -s malloc,1g -p http_resp_hdr_len=65536 -p http_resp_size=98304

Save it as /etc/systemd/system/varnish.service.d/override.conf.

systemctl daemon-reload && systemctl restart varnish && ss -ltn | grep -E ':(443|6081|8080) '

5. Magento admin

Stores › Configuration › Advanced › System › Full Page Cache:

Save Config, then System › Cache Management › Flush Magento Cache. The same from the command line:

cd /var/www/magento
sudo -u magento bin/magento config:set system/full_page_cache/caching_application 2
sudo -u magento bin/magento config:set system/full_page_cache/varnish/access_list 127.0.0.1
sudo -u magento bin/magento config:set system/full_page_cache/varnish/backend_host 127.0.0.1
sudo -u magento bin/magento config:set system/full_page_cache/varnish/backend_port 8080
sudo -u magento bin/magento cache:flush

6. VCL and purges

varnish:vcl:generate does not read the admin values, so pass the same ones. Then tell Magento where to send purge requests.

cd /var/www/magento
sudo -u magento bin/magento varnish:vcl:generate --export-version=7 \
  --access-list=127.0.0.1 --backend-host=127.0.0.1 --backend-port=8080 > /etc/varnish/default.vcl

systemctl reload varnish
sudo -u magento bin/magento setup:config:set --http-cache-hosts=127.0.0.1:6081 -n

7. Test

The first request is a MISS, the second a HIT.

curl -skI https://www.example.com/women/tops-women.html | grep -iE '^HTTP|x-magento-cache-debug'
curl -skI https://www.example.com/women/tops-women.html | grep -iE '^HTTP|x-magento-cache-debug'

Built-in cache vs Varnish

Measured in the video, same store, same sample data, same server size, warm cache in both cases:

Why Varnish

• Pages from memory in milliseconds. PHP and the database aren’t touched for cached pages.
• Many more visitors on the same server. Campaigns and traffic spikes don’t take the store down.
• Precise purges and grace mode. Magento purges only the pages that changed, by cache tags; visitors still get a page while Varnish refreshes it.
• PHP stays free for what can’t be cached: the cart, the checkout and the admin. Varnish is what Adobe recommends for production.

Let’s Encrypt on a real domain

Let port 80 answer the certificate challenge and redirect everything else, then get the certificate with the webroot method. Renewals keep working with this config.

upstream fastcgi_backend { server unix:/run/php/magento.sock; }
server {
    listen 80;
    server_name www.example.com;
    location /.well-known/acme-challenge/ { root /var/www/letsencrypt; }
    location / { return 301 https://$host$request_uri; }
}
server {
    listen 443 ssl;
    http2 on;
    server_name www.example.com;
    ssl_certificate     /etc/letsencrypt/live/www.example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/www.example.com/privkey.pem;
    location / {
        proxy_pass http://127.0.0.1:6081;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-Proto https;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_buffer_size 128k; proxy_buffers 4 256k; proxy_busy_buffers_size 256k;
        proxy_read_timeout 600s;
    }
}
server {
    listen 127.0.0.1:8080;
    server_name www.example.com;
    set $MAGE_ROOT /var/www/magento;
    include /var/www/magento/nginx.conf.sample;
}

Use this instead of magento.conf once the certificate exists; until then keep the test certificate lines.

apt install -y certbot
mkdir -p /var/www/letsencrypt
certbot certonly --webroot -w /var/www/letsencrypt -d www.example.com
nginx -t && systemctl reload nginx

Troubleshooting