Infiniroot Blog: We sometimes write, too.

Of course we cannot always share details about our work with customers, but nevertheless it is nice to show our technical achievements and share some solutions.

Important Nginx configuration changes after upgrading Ubuntu from 16.04 to 20.04

Published on March 4th 2021


Ubuntu 16.04 (Xenial) will be end of life in a bit more than a month. That means: Upgrade to a newer release. Right now there are 18.04 (Bionic) and 20.04 (Focal) available as LTS versions.

Depending on the installed packages and applications an upgrade can be as easy as flipping a switch. But newer versions often mean newer configurations. 

After upgrading an Nginx web server from Ubuntu 16.04 (with Nginx 1.10.0) to 18.04 and then on to 20.04, the following (kind of breaking) changes were seen.

Nginx on Ubuntu 18.04: Dynamic module loading

With the upgrade from Ubuntu 16.04 to 18.04, a newer Nginx version (1.14.0) was installed. During the upgrade, apt nicely asked whether or not to replace the existing main configuration /etc/nginx/nginx.conf.

Nginx upgrade on Ubuntu 16.04

By using the D option (show differences) the configuration changes are nicely shown. There is one change in particular that has a large effect on Nginx: An additional include in the global section (before events):

Nginx upgrade on Ubuntu 16.04: show differences in config

If the difference of the new config is not too big, I suggest to approve the changes from the new version with "Y".
But if there are too many changes from your own modifications, I suggest to use the "Z" option and manually add the include line in /etc/nginx/nginx.conf.

ck@ubuntu:~$ head -n 4 /etc/nginx/nginx.conf
user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

Apt should keep your old config file as /etc/nginx/nginx.conf.dpkg-old so you can do a proper comparison after the package upgrade.

The reason for this change is that Nginx is now using dynamic module loading instead of "hard-compiled" modules. This can be nicely seen when looking at the installed Nginx package(s) on a 16.04:

ck@ubuntu1604:~$ dpkg -l|grep nginx
ii  nginx-common                          1.10.3-0ubuntu0.16.04.4                    all          small, powerful, scalable web/proxy server - common files
ii  nginx-extras                          1.10.3-0ubuntu0.16.04.4                    amd64        nginx web/proxy server (extended version)

Compared with an Ubuntu 18.04, there are now a lot of Nginx packages:

ck@ubuntu1804:~$ dpkg -l|grep nginx
ii  libnginx-mod-http-auth-pam             1.14.0-0ubuntu1.7                               amd64        PAM authentication module for Nginx
ii  libnginx-mod-http-cache-purge          1.14.0-0ubuntu1.7                               amd64        Purge content from Nginx caches
ii  libnginx-mod-http-dav-ext              1.14.0-0ubuntu1.7                               amd64        WebDAV missing commands support for Nginx
ii  libnginx-mod-http-echo                 1.14.0-0ubuntu1.7                               amd64        Bring echo and more shell style goodies to Nginx
ii  libnginx-mod-http-fancyindex           1.14.0-0ubuntu1.7                               amd64        Fancy indexes module for the Nginx
ii  libnginx-mod-http-geoip                1.14.0-0ubuntu1.7                               amd64        GeoIP HTTP module for Nginx
ii  libnginx-mod-http-headers-more-filter  1.14.0-0ubuntu1.7                               amd64        Set and clear input and output headers for Nginx
ii  libnginx-mod-http-image-filter         1.14.0-0ubuntu1.7                               amd64        HTTP image filter module for Nginx
ii  libnginx-mod-http-lua                  1.14.0-0ubuntu1.7                               amd64        Lua module for Nginx
ii  libnginx-mod-http-ndk                  1.14.0-0ubuntu1.7                               amd64        Nginx Development Kit module
ii  libnginx-mod-http-perl                 1.14.0-0ubuntu1.7                               amd64        Perl module for Nginx
ii  libnginx-mod-http-subs-filter          1.14.0-0ubuntu1.7                               amd64        Substitution filter module for Nginx
ii  libnginx-mod-http-uploadprogress       1.14.0-0ubuntu1.7                               amd64        Upload progress system for Nginx
ii  libnginx-mod-http-upstream-fair        1.14.0-0ubuntu1.7                               amd64        Nginx Upstream Fair Proxy Load Balancer
ii  libnginx-mod-http-xslt-filter          1.14.0-0ubuntu1.7                               amd64        XSLT Transformation module for Nginx
ii  libnginx-mod-mail                      1.14.0-0ubuntu1.7                               amd64        Mail module for Nginx
ii  libnginx-mod-nchan                     1.14.0-0ubuntu1.7                               amd64        Fast, flexible pub/sub server for Nginx
ii  libnginx-mod-stream                    1.14.0-0ubuntu1.7                               amd64        Stream module for Nginx
ii  nginx-common                           1.14.0-0ubuntu1.7                               all          small, powerful, scalable web/proxy server - common files
ii  nginx-extras                           1.14.0-0ubuntu1.7                               amd64        nginx web/proxy server (extended version)

Each module is now packages in its own deb package.

Without the new include line in /etc/nginx/nginx.conf dynamic loading of these modules will not work. For example if you want to use the geoip module somewhere in your Nginx configs, Nginx will fail with an error as it cannot find the geoip module.

Nginx on Ubuntu 20.04: New SSL enabling and TLSv1.3

With the upgrade from Ubuntu 18.04 to Ubuntu 20.04, Nginx is upgraded to 1.18.0. There are no breaking changes in the main configuration file /etc/nginx/nginx.conf this time, however when using the configuration check (nginx -t) you are likely to spot deprecation warnings:

root@ubuntu2004:~# nginx -t
nginx: [warn] the "ssl" directive is deprecated, use the "listen ... ssl" directive instead in /etc/nginx/sites-enabled/api.example.com.conf:22
nginx: [warn] the "ssl" directive is deprecated, use the "listen ... ssl" directive instead in /etc/nginx/sites-enabled/api.example2.com.conf:22
nginx: [warn] the "ssl" directive is deprecated, use the "listen ... ssl" directive instead in /etc/nginx/sites-enabled/api.example3.com.conf:22
nginx: [warn] the "ssl" directive is deprecated, use the "listen ... ssl" directive instead in /etc/nginx/sites-enabled/api.example4.com.conf:19
nginx: [warn] the "ssl" directive is deprecated, use the "listen ... ssl" directive instead in /etc/nginx/sites-enabled/backend.example.com.conf:38
nginx: [warn] the "ssl" directive is deprecated, use the "listen ... ssl" directive instead in /etc/nginx/sites-enabled/backend.example2.com.conf:59

The reason for this is that the syntax has slightly changed. Even though a deprecation warning is more of a hint that in future versions this way of configuration will break, it's a good moment to adapt the Nginx configs now.

Basically what changes is the way SSL is enabled. In the past with older Nginx versions, this was often done the following way:

root@ubuntu2004:~# cat /etc/nginx/sites-enabled/api.example.com.conf
server {
  listen 443;
  server_name api.example.com;
  access_log /var/log/nginx/api.example.com.access.log include_ssl_params;
  error_log /var/log/nginx/api.example.com.error.log;

  ssl on;
  ssl_certificate /etc/nginx/ssl.crt/wildcard.example.com.crt;
  ssl_certificate_key /etc/nginx/ssl.key/wildcard.example.com.key;
[...]

However the new Nginx version prefers (and in future wants) that ssl enabling happens with the listen parameter:

root@ubuntu2004:~# cat /etc/nginx/sites-enabled/api.example.com.conf
server {
  listen 443 ssl;
  server_name api.example.com;
  access_log /var/log/nginx/api.example.com.access.log include_ssl_params;
  error_log /var/log/nginx/api.example.com.error.log;

  ssl_certificate /etc/nginx/ssl.crt/wildcard.example.com.crt;
  ssl_certificate_key /etc/nginx/ssl.key/wildcard.example.com.key;
[...]

As this change does not require a major configuration rewrite, it can be automated using sed.

Important note: The following sed is used in /etc/nginx/sites-available as in all my Nginx setups, the config files in /etc/nginx/sites-enabled are symlinks to the actual configuration files in /etc/nginx/sites-available!

root@ubuntu2004:/etc/nginx/sites-available# sed -i "/ssl on;/d" *.conf
root@ubuntu2004:/etc/nginx/sites-available# sed -i "s/listen 443;/listen 443 ssl;/" *.conf

After this change, nginx -t should not show any deprecation warnings anymore. At least not related to the SSL enabler.

Another change after this Nginx upgrade is the added TLSv1.3 version in the ssl_protocols in /etc/nginx/nginx.conf:

Added TLSv1.3 in Nginx after upgrade to Ubuntu 20.04

ck@ubuntu2004:~$ cat /etc/nginx/nginx.conf | grep ssl_protocols
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3; # Dropping SSLv3, ref: POODLE

These protocols are defined by the default configuration file. Of course TLSv1 and TLSv1.1 should be disabled nowadays.