Nginx as a reverse proxy for Nifi web UI and Kibana

Thu, 16 Jan 2020 22:22:51 -0400

Nginx can act as a application neutral proxy. One example is to front Nifi. The nifi default configuration provides an HTTP access point, specified in the following entries in nifi.properties:

nifi.web.http.host=192.168.133.5
nifi.web.http.port=8080

Nifi can provide secure port by commenting out the lines above and provide the followings:

nifi.web.https.host=192.168.133.5
nifi.web.https.port=8083

However, it does require configuring JKS keystore for Java, as well as authentication. Customers with existing AD servers are likely to require authentication via LDAP. While Nifi does support LDAP integration according to its administration guide. The configuration is quite involving. You need to configure the identity provider, as well as authorizes. I have personally spent a couple days on this without much progress. The information in the logging isn’t to the point. Restarting nifi also is a long process, making it painful to troubleshoot.

I then moved to Nginx (open-source) as an alternative and it is quite enlightening. I already knew that the SSL termination in nginx is super easy to configure. This time I learned that the opensource community even has a support for LDAP integration. Here is a diagram of how it works:

This approach is outlined in a blog post on Nginx website. The ldap-auth daemon is implemented in Python can can be wrapped up as a systemd service. Once a client sends a request in https, security layer is terminated in nginx, and an authentication request in http is sent to the ldap-auth daemon, which proxies converts the request into LDAP searches and proxies it over to customer’s Active Directory server, for authentication. Once authenticated, the http request can make to one of the backend container or server which hosts Nifi. Below is an example of how this can be configure on RedHat.

Install python3 and python-ldap

RedHat may have both python2 and python3 pre-installed, python2 being the default. We do not want to change the default because other applications such as yum still depends on python2 as of early 2020.

yum -y install python3
yum -y install gcc python3-devel openldap-devel
pip3 install python-ldap

Once python3 is installed, pip3 will be available and we use that to install python-ldap. This is a module in Python3 that will be used by the script that act as ldap daemon.

Configure ldap-auth daemon as systemd service

In the github project for ldap-auth, download nginx-ldap-auth-daemon.py to local location such as /usr/bin, then we create nginx-ldap-auth.service in /etc/systemd/system/ with the following content.

[Unit]
Description=LDAP authentication helper for Nginx
After=network.target network-online.target
 
[Service]
Type=simple
User=root
Group=root
WorkingDirectory=/var/run
ExecStart=/usr/bin/python3 /usr/bin/nginx-ldap-auth-daemon.py
KillMode=process
KillSignal=SIGINT
Restart=on-failure
 
[Install]
WantedBy=multi-user.target

Then, run the following command to load, start and check nginx-ldap-auth service.

systemctl reload-daemon
systemctl start nginx-ldap-auth
systemctl status nginx-ldap-auth

This service will be up and listening to port 8888 for http traffic.

Configure Nginx

Then configure nginx with the following entries in its default.conf file, typically located in /etc/nginx/conf.d.

upstream nifibackend {
 # default: round robin
 server container1.nifi.digihunch.com:8080;
 server container2.nifi.digihunch.com:8080;
}
proxy_cache_path cache/ keys_zone=auth_cache:10m;
# nifi proxy
server {
 listen 8083ssl;
 include /etc/nginx/ssl/default.conf;
 
 location / {
 auth_request /auth-proxy;
 
 proxy_pass http://nifibackend;
 proxy_set_header Host $host:$server_port;
 proxy_set_header X-ProxyScheme https;
 proxy_set_header X-ProxyHost $1;
 proxy_set_header X-ProxyPort 8083;
 proxy_set_header X-ProxyContextPath /;
 }
 
 location /auth-proxy {
 internal;
 proxy_pass http://127.0.0.1:8888;
 
 proxy_pass_request_body off;
 proxy_set_header Content-Length "";
 proxy_cache auth_cache;
 proxy_cache_valid 200 10m;
 
 proxy_cache_key "$http_authorization$cookie_nginxauth";
 proxy_set_header X-Ldap-URL "ldaps://ldap.digihunch.com:636";
 proxy_set_header X-Ldap-BaseDN "OU=Corporate User Accounts,DC=digihunch,DC=org";
 proxy_set_header X-Ldap-BindDN "CN=Digi Hunch Service Account,OU=Digi,OU=ServiceAccounts,OU=Digi,OU=Digi Applications,DC=digihunch,DC=org";
 proxy_set_header X-Ldap-BindPass "myownpasswordtricks";
 proxy_set_header X-CookieName "nginxauth";
 proxy_set_header Cookie nginxauth=$cookie_nginxauth;
 proxy_set_header X-Ldap_Starttls "true";
 proxy_set_header X-Ldap-Template "(&(sAMAccountName=%(username)s)(objectClass=organizationalPerson)(memberOf=CN=GH_SYSADMIN,OU=GHCO,OU=Groups,OU=Digi,OU=Digi Applications,DC=digihunch,DC=org))";
 }
}

We need the full distinguished name of bind user to get this to work. Once configured properly, and user attempts to connect through a browser, Nginx will pop up a prompt for username and password. The username will be plugged into the X-Ldap-Template for further queries. The same HTTP header also allows you to filter by membership that the user is associated with.

Virtual Host on Digi Hunch

Nginx as a reverse proxy for Nifi web UI and Kibana

Install python3 and python-ldap

Configure ldap-auth daemon as systemd service

Configure Nginx