How to Build and Install Nginx kTLS from Source on Ubuntu 22.04

This toturial will guide you through the 4 detailed steps to install Nginx and create nginx sites that support SSL with kTLS and TLSv1.3 on Ubuntu 22.04 server.

Introduction

Nginx is one of the most popular advanced load balancer, web Server and reverse proxy in the world and is responsible for serving some largest and high-traffic websites.

One of its great features is light weight. Because Nginx consumes less computing resource and can handle a high volume of connections, it is commonly used as a reverse proxy and load balancer to manage incoming traffic and distribute it to slower upstream servers, anything from microservices to heavy Odoo ERP Online.

Why kTLS?

Transport Layer Security (TLS) is an extremely popular cryptography protocol. Implementing TLS in the kernel (kTLS) improves performance by significantly reducing the need for copying operations between user space and the kernel.

By combining kTLS and sendfile(), data is encrypted directly in kernel space before being passed to the network stack for transmission. This eliminates the need to copy data into user space to be encrypted by TLS libraries and then back into kernel space for transmission. kTLS also enables offload of TLS processing to hardware, including offload of TLS symmetric crypto processing to network devices.

Why TLSv1.3?

TLS 1.3 is the latest version of the TLS protocol, which is used by HTTPS and other network protocols for encryption, is the modern version of SSL. TLS 1.3 dropped support for older, less secure cryptographic features, and it sped up TLS handshakes, among other improvements.

Executive Summary

There are two methods of installing NGINX on Ubuntu 22.04:

Using operating system’s package manager. For example, apt in Ubuntu 22.04: apt install nginx
Building and installing Nginx from source code.

In this tutorial, the method we choose is building and installing from source as we need to customize the installation for kTLS. After completion of the steps in this tutorial, we expect to achieve the followings:

NginX is built and installed from source with kTLS module
A sample Nginx site will be deployed with a domain name and has SSL Certificate with TLSv1.3 supported using kTLS.
HTTP/2 is enabled for better performance

Prerequisites

Before you begin this tutorial, you should have meet the following prerequisites:

You should have a Ubuntu 22.04 server. I recommend you to get one from DigitalOcean if you don’t have one yet.
You have setup your server with a regular, non-root user with sudo privileges configured on your server.
You should also have a public IP assigned to the server.
You already have a domain name (e.g. your_domain.com) pointed to the IP address mentioned above.

Step 1 – Nginx Server Preparation

Loading kTLS in the Kernel

As we are implementing Nginx with kTLS support, we need to load the kTLS module in the Kernel by running the following command:

sudo modprobe tls

Please note that if you are using a linux container such as LXD / LCX whose kernel is shared by the host, the command above should be run on the host instead.

Install Dependant Packages for Nginx Installation

As usual, we need to update the system first.

sudo apt update; sudo apt dist-upgrade

Now, install software and packages that are required by Nginx:

sudo apt install unzip build-essential \
libpcre3 libpcre2-dev libpcre3-dev zlib1g \
zlib1g-dev libssl-dev libgd-dev libxml2 \
libxml2-dev uuid-dev libmaxminddb-dev \
libgeoip-dev libxslt1-dev

Prepare Nginx User and Group

We need a group named nginx whose credentials will be used by nginx worker processes.

sudo addgroup nginx

We also need to have an unprivileged user named nginx that belongs to the group nginx mentioned above, whose credentials will be used by Nginx worker processes.

The user will have the /var/www as its home directory.

sudo adduser nginx --system \
--home=/var/www --disabled-login \
--disabled-password --ingroup nginx

Create Nginx Working Directories

Firstly, create HTTP client body temporary directory:

sudo mkdir -p /var/lib/nginx/body

Secondly, create HTTP proxy temporary directory

sudo mkdir /var/lib/nginx/proxy

And now, create HTTP FastCGI temporary directory

sudo mkdir /var/lib/nginx/fastcgi

Then, create HTTP uWSGI temporary directory

sudo mkdir /var/lib/nginx/uwsgi

After that, create HTTP SCGI (Simple Common Gateway Interface) temporary directory

sudo mkdir /var/lib/nginx/scgi

And then, create Nginx modules directory

sudo mkdir -p /usr/lib/nginx/modules

After that, create the directory /etc/nginx/sites-available to store Nginx site configuration files:

sudo mkdir /etc/nginx/sites-available

Then create the directory /etc/nginx/sites-enabled to store symbolic links to the files inside the sites-available that we want Nginx to load.

sudo mkdir /etc/nginx/sites-enabled

Finally, create SSL certificate directory:

sudo mkdir -p /etc/nginx/ssl

Step 2 – Compile and Install Nginx

Let’s start with downloading Nginx source code then compile and install it.

Download Nginx Source Code

kTLS support was added in Nginx Version 1.21.4 (mainline version). At the time of writing this post (Oct., 2022), the latest stable version is 1.22.1. We’re going to download this version using wget:

wget https://nginx.org/download/nginx-1.22.1.tar.gz

After you finish the download, you can run the following command to extract the download source code archive to the current directory:

tar xvfz nginx-1.22.1.tar.gz

Download Extra Nginx Modules

This step is optional if you don’t have a need to use GeoIP2. But it’s good to have it if you are deploying Nginx as proxy for your Odoo Installation.

wget https://github.com/leev/ngx_http_geoip2_module/archive/refs/tags/3.4.zip

unzip 3.4.zip

mv ngx_http_geoip2_module-3.4 /usr/lib/nginx/modules/

Configure Nginx compilation

Firstly, you need to change the current directory to the Nginx source code directory using the below command:

cd nginx-1.22.1

Then, you can run the following command to see configuration options before compiling Nginx:

./configure --help

Or, read the full nginx configure parameters for details.

Now, we can start to configure nginx compilation using most popular compile-time options by running the command below. Please note that the option --with-openssl-opt=enable-ktls is required to have kTLS support. If you also want to support weak cipher for earlier TLS version (e.g. TLSv1.2, the option should be --with-openssl-opt='enable-ktls enable-weak-ssl-ciphers'. Here is the configuration options I usually use for building Nginx as reverse proxy for Odoo Installation on Ubuntu 22.04.

./configure \
--prefix=/usr/local/nginx \
--sbin-path=/usr/sbin/nginx \
--conf-path=/etc/nginx/nginx.conf \
--http-log-path=/var/log/nginx/access.log \
--error-log-path=/var/log/nginx/error.log \
--http-client-body-temp-path=/var/lib/nginx/body \
--http-fastcgi-temp-path=/var/lib/nginx/fastcgi \
--http-proxy-temp-path=/var/lib/nginx/proxy \
--http-scgi-temp-path=/var/lib/nginx/scgi \
--http-uwsgi-temp-path=/var/lib/nginx/uwsgi \
--lock-path=/var/lock/nginx.lock \
--pid-path=/run/nginx.pid \
--user=nginx \
--group=nginx \
--with-pcre-jit \
--with-http_ssl_module \
--with-http_stub_status_module \
--with-http_realip_module \
--with-http_auth_request_module \
--with-http_addition_module \
--with-http_dav_module \
--with-http_geoip_module \
--with-http_gunzip_module \
--with-http_gzip_static_module \
--with-http_image_filter_module \
--with-http_v2_module \
--with-http_sub_module \
--with-http_xslt_module \
--with-stream \
--with-stream_ssl_module \
--with-file-aio \
--with-mail \
--with-mail_ssl_module \
--with-http_slice_module \
--with-debug \
--with-compat \
--with-threads \
--with-stream_ssl_preread_module \
--with-http_mp4_module \
--with-openssl-opt='enable-ktls enable-weak-ssl-ciphers' \
--add-module=/usr/lib/nginx/modules/ngx_http_geoip2_module-3.4 \
--with-cc-opt='-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2' \
--with-ld-opt='-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now -Wl,--as-needed -pie'

The result may look as below:

Configuration summary
  + using threads
  + using system PCRE library
  + using system OpenSSL library
  + using system zlib library

  nginx path prefix: "/usr/local/nginx"
  nginx binary file: "/usr/sbin/nginx"
  nginx modules path: "/usr/local/nginx/modules"
  nginx configuration prefix: "/etc/nginx"
  nginx configuration file: "/etc/nginx/nginx.conf"
  nginx pid file: "/run/nginx.pid"
  nginx error log file: "/var/log/nginx/error.log"
  nginx http access log file: "/var/log/nginx/access.log"
  nginx http client request body temporary files: "/var/lib/nginx/body"
  nginx http proxy temporary files: "/var/lib/nginx/proxy"
  nginx http fastcgi temporary files: "/var/lib/nginx/fastcgi"
  nginx http uwsgi temporary files: "/var/lib/nginx/uwsgi"
  nginx http scgi temporary files: "/var/lib/nginx/scgi"

Compile and Install Nginx

After you complete custom configuration, can can now compile NGINX source code by using the command below :

make

Install Nginx:

sudo make install

Verify if you have installed Nginx successfully by running the command below:

nginx -V

You should see results as below:

Create Nginx Configuration File

I’m using the text editor nano to edit the configration file /etc/nginx/nginx.conf:

sudo nano /etc/nginx/nginx.conf

Then replace the existing content with the following:

user nginx;
worker_processes auto;
pid /run/nginx.pid;

events {
        worker_connections 1024;
        # multi_accept on;
}

http {

        ##
        # Basic Settings
        ##

        sendfile on;
        tcp_nopush on;
        types_hash_max_size 2048;
        # server_tokens off;

        # server_names_hash_bucket_size 64;
        # server_name_in_redirect off;

        include /etc/nginx/mime.types;
        default_type application/octet-stream;

        ##
        # SSL Settings
        ##

        ssl_protocols TLSv1.2 TLSv1.3;
        ssl_prefer_server_ciphers on;

        ##
        # Logging Settings
        ##

        access_log /var/log/nginx/access.log;
        error_log /var/log/nginx/error.log;

        ##
        # Gzip Settings
        ##

        gzip on;

        # gzip_vary on;
        # gzip_proxied any;
        # gzip_comp_level 6;
        # gzip_buffers 16 8k;
        # gzip_http_version 1.1;
        # gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

        ##
        # Virtual Host Configs
        ##

        include /etc/nginx/conf.d/*.conf;
        include /etc/nginx/sites-enabled/*;
}

After that, you can save the Nginx configuration file by pressing Ctrl+X and input y when being asked then press Enter.

IMPORTANT: You must have the directive sendfile on; in the http block to support kTLS.

Start Nginx

Now, you are almost done and can start Nginx by using this command:

sudo nginx

And then you can run the command below to check if Nginx worker is running in the background:

sudo ps aux | grep nginx

You should see the output as below:

Stop Nginx

To stop Nginx, run the below command:

sudo nginx -s stop

Step 3- Run Nginx as a Service / Daemon

Running an NGINX service (a.k.a daemon) will not only allow us to manage starting, stopping and reloading NGINX in a more standardized way but also make starting NGINX on boot much simpler.

Create Nginx Unit File

We’re going to use the text editor nano to create the file:

sudo nano /lib/systemd/system/nginx.service

Now, you can copy and paste the following to the nano UI:

# Stop dance for nginx
# =======================
#
# ExecStop sends SIGSTOP (graceful stop) to the nginx process.
# If, after 5s (--retry QUIT/5) nginx is still running, systemd takes control
# and sends SIGTERM (fast shutdown) to the main process.
# After another 5s (TimeoutStopSec=5), and if nginx is alive, systemd sends
# SIGKILL to all the remaining processes in the process group (KillMode=mixed).
#
# nginx signals reference doc:
# http://nginx.org/en/docs/control.html
#
[Unit]
Description=A high performance web server and a reverse proxy server
Documentation=man:nginx(8)
After=network.target nss-lookup.target

[Service]
Type=forking
PIDFile=/run/nginx.pid
ExecStartPre=/usr/sbin/nginx -t -q -g 'daemon on; master_process on;'
ExecStart=/usr/sbin/nginx -g 'daemon on; master_process on;'
ExecReload=/usr/sbin/nginx -g 'daemon on; master_process on;' -s reload
ExecStop=-/sbin/start-stop-daemon --quiet --stop --retry QUIT/5 --pidfile /run/nginx.pid
TimeoutStopSec=5
KillMode=mixed

[Install]
WantedBy=multi-user.target

Then save the file and exit.

Now, just run the following command to notify systemd that a new unit file exists:

sudo systemctl daemon-reload

Enable Nginx start-up on boot

sudo systemctl enable nginx

Working with Nginx Service

Start Nginx

sudo systemctl start nginx

Stop Nginx

sudo systemctl stop nginx

Checking Nginx service status

sudo systemctl status nginx

Step 4 – Create Nginx site with kTLS support

In this section, I will guide you to create a sample site that supports:

SSL Certificate. You can use Let’s Encrypt for free or buy from a public Certificate Authority.
kTLS: see Why kTLS
HTTP/2: HTTP/2 (originally named HTTP/2.0) is a major revision of the HTTP network protocol used by the World Wide Web. It was derived from the earlier experimental SPDY protocol, originally developed by Google

Create Nginx Site

Please make sure that:

You already have your domain name pointed to your Nginx server IP address;
You already have a SSL Certificate for the domain name and stored in /etc/nginx/ssl;

Create a sample HTML page:

sudo mkdir /var/www/html

Use nano to create the page

sudo nano /var/www/html/index.html

Copy and past the following content:

<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

After that, save the file and change the owner of the directory /var/www/html and its content to nginx:

sudo chown -hR nginx: /var/www/html

Now, create a site configuration file using nano as below:

sudo nano /etc/nginx/sites-available/your_domain.com.conf

Then copy and paste the following in the nano UI:

server {
        listen 443 ssl http2;
        server_name your_domain.com;

        error_log /var/log/nginx/your_domain_com_error.log debug;
        root /var/www/html;

        # Add index.php to the list if you are using PHP
        index index.html index.htm index.nginx-debian.html;

        ssl_certificate ssl/your_ssl_certificate.crt;
        ssl_certificate_key ssl/your_ssl_certificate.key;
        ssl_conf_command Options KTLS;
        ssl_protocols TLSv1.3;

        location / {
                try_files $uri $uri/ =404;
        }
}

Please note that:

We enabled debuging by specifying debug in the directive error_log so that we can have debug information later in the error log file /var/log/nginx/your_domain_com_error.log. After everything is fine, you should turn the debug off by removing the debug, especially in production environments. Debug logging incurs a performance penalty due to the large volume of write operations; also, debug logs can be huge and quickly exhaust available space on the disk partition.
You will have to change value of the directives ssl_certificate and ssl_certificate_key to match your SSL certificate location.
The directive ssl_conf_command Options KTLS; is to activate kTLS support for this site.

Now, create a symbolic link for the newly created file your_domain.com.conf so that Nginx can load it:

sudo ln -s /etc/nginx/sites-available/your_domain.com.conf /etc/nginx/sites-enabled/your_domain.com.conf

Then check if any error in Nginx configuration by running command:

sudo nginx -t

If no error, you should find something like this:

Now, restart Nginx for the new site to take effect:

sudo systemctl restart nginx

Then, open your web browser and input https://your_domain.com to see a page that looks like below:

Please note that the URL must be https. In the example above, I use a local domain with self-signed SSL certificate. In production environment, it should be your domain with a valid SSL.

Verify if kTLS is Enabled

To verify that NGINX is using kTLS, check for BIO_get_ktls_send() and SSL_sendfile() in the error log.

Run the below command to check if BIO_get_ktls_send is in the log:

sudo grep BIO /var/log/nginx/tuan_local_error.log

Then you should see something like below as results:

2022/10/26 03:40:31 [debug] 10662#10662: *3 BIO_get_ktls_send(): 1
2022/10/26 03:50:35 [debug] 10667#10667: *7 BIO_get_ktls_send(): 1

In a similar way, run the below command to check if SSL_sendfile is in the log:

sudo grep SSL_sendfile /var/log/nginx/tuan_local_error.log

Then the result would be:

2022/10/26 03:55:35 [debug] 274550#274550: *2 SSL_sendfile: 1048576
2021/11/10 03:56:49 [debug] 274550#274550: *3 SSL_sendfile: 1048576

Conclusions

Now, you have just finished to install Nginx on Ubuntu 22.04 with a sample site that support HTTP/2 and kTLS. To create more sites, just repeat the step 4.

David Tran

How to Build and Install Nginx kTLS from Source on Ubuntu 22.04

Introduction

Why kTLS?

Why TLSv1.3?

Executive Summary

Prerequisites

Step 1 – Nginx Server Preparation

Loading kTLS in the Kernel

Install Dependant Packages for Nginx Installation

Prepare Nginx User and Group

Create Nginx Working Directories

Step 2 – Compile and Install Nginx

Download Nginx Source Code

Download Extra Nginx Modules

Configure Nginx compilation

Compile and Install Nginx

Create Nginx Configuration File

Start Nginx

Stop Nginx

Step 3- Run Nginx as a Service / Daemon

Create Nginx Unit File

Enable Nginx start-up on boot

Working with Nginx Service

Step 4 – Create Nginx site with kTLS support

Create Nginx Site

Verify if kTLS is Enabled

Conclusions

Like this:

Related

Leave a Reply Cancel reply

Enterprise Digital Transformation, where our global future starts

Introduction

Why kTLS?

Why TLSv1.3?

Executive Summary

Prerequisites

Step 1 – Nginx Server Preparation

Loading kTLS in the Kernel

Install Dependant Packages for Nginx Installation

Prepare Nginx User and Group

Create Nginx Working Directories

Step 2 – Compile and Install Nginx

Download Nginx Source Code

Download Extra Nginx Modules

Configure Nginx compilation

Compile and Install Nginx

Create Nginx Configuration File

Start Nginx

Stop Nginx

Step 3- Run Nginx as a Service / Daemon

Create Nginx Unit File

Enable Nginx start-up on boot

Working with Nginx Service

Step 4 – Create Nginx site with kTLS support

Create Nginx Site

Verify if kTLS is Enabled

Conclusions

Share this:

Like this:

Related

Leave a Reply Cancel reply

Enterprise Digital Transformation, where our global future starts