ldap on Digi Hunch

SASL Authentication Mechanisms

Thu, 19 Mar 2020 22:53:00 -0400

Introduction

Authentication is used in many protocols (such as LDAP binding) and it usually involves sending password. Given the nature of authentication protocol, its traffic encryption is usually mandatory. Simple Authentication and Security Layer (SASL) is introduced to ensure the security during authentication. It is not a single protocol, but rather a framework for authentication and data security involving many protocols. The intent is to decouple authentication mechanisms from application protocols, thus allowing any authentication mechanism (under SASL) to be used in any application protocol (that supports SASL). Application protocols that support SASL typically can also be built on Transport Layer Security (TLS), whose latest versions (1.2 and 1.3) are considered more secure.

None (ANONYMOUS)

The server basically does not authenticate the client. The client connects to the server anonymously. Under SASL framework, this may also be referred to as ANONYMOUS mechanism.

Simple (PLAIN)

In simple authentication method the password is sent to server in the clear. This is subject to eavesdropping and is not secure. It is still surprisingly widespread in legacy configurations probably due to the simplicity of configuration. This option should not be available in cloud environment. Under SASL framework, this may also be referred to as PLAIN mechanism.

CRAM-MD5 and DIGEST-MD5

CRAM-MD5: described in RFC 2195, using HMAC-MD5 algorithm. In this challenge-response scheme based mechanism, the client’s password is protected during authentication, but the application session (e.g. LDAP) traffic is not encrypted. It includes random data from the server and is slightly better than Simple authentication. However, this authentication method is not recommended either.

DIGEST-MD5: described in RFC 2831. This is very similar to CRAM-MD5 but is is somewhat stronger because it includes random data from both the client and server. In addition, it also provides a provision to ensure connection integrity and confidentiality (a data security layer).

GSSAPI

Generic Security Service Application Program Interface (GSS-API) is an API specification for programs to access security services. GSS-API by itself does not provide any security. Instead, security-service vendors provide GSSAPI implementations – usually in the form of libraries installed with their security software. These libraries present a GSSAPI-compatible interface to application developers who can write their application to use only the vendor-independent GSSAPI.

Under SASL, the dominant GSSAPI mechanism implementation in use is Kerberos version 5. GSSAPI allows Kerberos implementations to be API compatible. In many contexts, GSSAPI simply implies Kerberos.

NTLM

NT LAN Manager (NTLM) is a challenge-response based Microsoft security protocols. It is implemented in a Security Support Provider (SSP), which combines the older LAN Manager authentication protocol, NTLMv1, NTLMv2 and NTLM2 Session protocols in a single package. Group policy manages whether these protocols are used or can be used. NTLM passwords are considered weak because they can be brute-forced very easily with modern hardware. It might still be enabled in server configuration as a backup mechanism to Kerberos.

TLS (EXTERNAL)

CRAM-MD5, DIGEST-MD5, GSSAPI, and NTLM are more commonly referred to as SASL mechanisms (in loose terms). These mechanisms allow for a secure password exchange without requiring TLS by trying to address the authentication traffic encryption problem at application layer. Using TLS this can also be address at transport layer. TLS can be used in combination with any of the mechanisms above but usually TLS/Simple mechanism is sufficient. In many occasions the mechanisms under SASL can be replaced by simple authentication encrypted with TLS. Under the SASL framework, this may also be referred to as EXTERNAL mechanism so TLS (in strict terms) is also considered a SASL mechanism.

Summary

This article outlined several authentication protocols under SASL. Since SASL is the framework that intends to govern all authentication protocols, the use case of these protocol can be widespread. For example, in LDAP you can find all of them. Here is a comparison across them:

	Password on wire	Session
Simple	Clear	No Encryption
SASL/CRAM-MD5	Encrypted	No Encryption
SASL/DIGEST-MD5	Encrypted	No Encryption
SASL/GSSAPI	Kerberos	Encryption
TLS:SIMPLE	Encrypted	Encrypted

Common SASL implementation includes Cyrus SASL and GNU SASL. There are also some API implementations that supports some of SASL mechanisms, such as SSPI.

Security Assertion Markup Language (SAML)

Sun, 08 Mar 2020 22:23:00 -0400

SAML is an XML-based standard for exchanging authentication and authorization data between IdP (identity provider) and service provider. We can compare SAML with LDAP (as authentication protocol) as both are to provide single-sign-on (SSO) feature.

LDAP is considered traditional configuration in on-premise operation for organizations. The configuration can be complex and administrators needs to complete significant work upfront. AD is notoriously hard to integrate into the cloud. On the other hand, LDAP gives the organization greater level of control over authentication and authorization due to its tighter integration with domain controller. It is prevalent in on-premise enterprise infrastructure and integrate well with OpenVPN, Jenkins, Docker, Kubernetes, etc.

On the other hand, SAML was created in early 2000s with the exclusive purpose of federating identities to web applications. The protocol was introduced assuming an IdP already exists in an organization. The SAML protocol doesn’t intend to replace the IdP, but rather use it to assert the validity of a user’s identity. This timed assertion (declaration that user’s identity is valid for a period of time) will be delivered to a service provider via secure XML exchange.

The benefit of SAML is that an on-premise identity typically stored in Active Directory (AD), could be extended to authenticate its users against web applications. ISVs (independent software vendor) can build web applications that integrates with on-premise AD server to achieve SSO feature. The AD server in this case provides IDaaS (Identity as a service) using its FS (Federation Service) module. Examples of web applications that support SAML integration include Confluence, Zendesk, Slack, Bombgar, etc. In the configuration, you typically need to specify who is the IdP (e.g. Microsoft AD, Okta, etc) and it’s SSO URL.

At a high level, an SSO process using SAML takes places in the following steps:

User tires to reach web application (service provider);
Web application redirects user browser to SSO URL;
User provide credential in the SSO URL;
IdP authenticates the user;
IdP produces SAML response to browser;
Browser passes the SAML response to service provider’s dedicated endpoint
Service provider permits user access to web application

Here is an example of SAML integration guide from an G Suite. The guide outlines how it works and the assertion requirements.

The Wikipedia page for SAML 2.0 provide an example of assertion message.

Lightweight Directory Access Protocol (LDAP)

Mon, 02 Mar 2020 21:11:00 -0400

Introduction

Originally LDAP only refers to the connectivity protocol to the directory server. This term is being used loosely today and it also refers to the actual directory service that supports and complies with LDAP. LDAP v3 is the current version developed in RFC 2251.

A directory is information about some set of entities such as people, organization, or stones. An example of directory would be /etc/passwd file in Linux. A directory server is simply an application with the main purpose of maintaining directories. Typically, the read traffic is high whereas write traffic is low. LDAP is a general-purpose directory server. It can store information about people, or cars, or rocks. You just need to define what a person’s entry looks like as well as what a rock’s entry looks like. The general architecture of LDAP provides the capability nedded for managing large amount of diverse directory entries.

An LDAP entry consists of DN (distinguished name) and attributes. An attribute may have one or more attribute names and they are defined in attribute definitions. Attribute names are not case-sensitive. An attribute may have one or more values if multiple values are allowed for that attribute. Attribute values may be case-sensitive depending on the definition.

A special attributed named objectclass attribute provides information about what type of record it is, and what attributes canbe given to the record. For example, the organization name (o) is required for any entry with an organization object class. While a record may have multiple object classes, one of these object classes must be the structural object class for the record. A structural object class determines what type of object the record is.

In addition to regular attributes, the directory server may also attach special operational attributes to an entry. Operational attributes are used by the directory server itself to store information about entries. Such attributes are not designed for use by end users, and are usually not returned during LDAP searches.

An LDAP schema defines types of records in a directory and how those records might relate to each other.

Information in an LDAP directory is organized into one or more hierarchies where, at the top of the hierarchy, there is a base entry, and other entries are organized in tree-like structures beneath the base entry. Each node on the hierarchy is an entry, with a DN and more than one attributes. This hierarchically organized collection of entries is called a directory information tree (DIT). In DIT, LDAP directories stores data in hierarchical relationships. The root entry sits at the top and subordinate entry is beneath that, which in turn may have its own subordinate entries. Each of these records has its own DN, and its own attributes. The DN of each entry is composed of two parts: the relative DN (RDN) and the full DN of the superior entry.

LDAP is nothing other than a special sort of database that organizes data into tree structures, like a file system hierarchy. This view is more easily seen by comparing an LDAP directory to a relational database system (RDB), where SQL is the protocol and RDBMS is the service. LDAP refers to both the protocol and the service.

OpenLDAP

A common LDAP implementation is openldap. OpenLDAP suite can be broken up into four components:

Servers: slapd (stand-alone LDAP Daemon) provides LDAP services.
Clients: ldapsearch is used to manipulate LDAP data
Utilities: support LDAP servers
Libraries: provide programming interfaces to LDAP

Installing OpenLDAP requires libldap-2.3-0, slapd, ldap-utiles packages. It is configured in /etc/ldap/. An HDB (hierarchical database) needs to be specified in the configuration.

To test as a client, the first thing that must happen is the client must authenticate to the server (via simple bind or SASL Bind). LDAP server verifies the identity, permission as well as password provided by the client.

LDAPTLS_REQCERT=never ldapsearch -x -o ldif-wrap=256 -H ldaps://ldap.digihunch:636/ -b "OU=Admin,OU=Service Department,DC=digihunch,DC=com" -D "gh\ldap-bind-user" -w 'S@f35+P@55w0rd' "(objectclass=user)" -s sub -d 9

The command above first sets client environment variable LDAPTLS_REQCERT to never, in case the client is being asked to provide certificate. Then the ldapsearch command performs the bind.

To search the directory, the client needs to provide the followings:

Base DN: where in the directory to start from
Scope: how deep in the tree to look
Attributes: what information to be retrieved per result
Filter: what to look for

Below is an example of ldapsearch (-b for Base DN, -s for Scope, -S for attributes, stdin for filter):

ldapsearch -x -o ldif-wrap=256 -H ldaps://ldap.digihunch:636/ -b "OU=Admin,OU=Service Department,DC=digihunch,DC=com" -D "gh\ldap-bind-user" -w 'S@f35+P@55w0rd' "(memberof=CN=Security-Admin,OU=Admin,OU=Service Department,DC=digihunch,DC=com)" -s sub -S name

Users with appropriate permissions may also other directory operations using ldapadd, ldapmodify, ldapdelete, ldapcompare, ldapmodrdn, ldappasswd, ldapwhoami, etc

Apart from those in Openldap toolkits, there are many other tools such as Apache Directory Studio that allows you to perform similar functionality with a user interface.

LDAP security

Historically LDAP servers listens to port 389 through which traffic is sent in clear text. This is a bad security practice known as “insecure bind”. To secure LDAP traffic, two prevalent approaches are Secure LDAP and StartTLS.

Secure LDAP was the original attempt to secure LDAP traffic as an addition to LDAP v2. It is also known as LDAPS, LDAP over TLS/SSL or LDAP channel binding (“channel binding” just refers to the establishment of encrypted channel following TLS handshake. It provides a facility to tie an authentication exchange to security services provided at a lower layer. Defined in RFC 5056). Secure LDAP operates on port 636 on the server side and TLS handshake must be established for traffic encryption. Client application usually need to import the certificate of LDAP server. As part of TLS 1.2 protocol, the server may also request client certificate during ServerHello message. The presence of CertificateRequest means the server either demands client certificate, or tries to get client certificate (i.e. TLSVerifyClient is set to demand or try, which is only visible on the server). If client cert is only attempted, the LDAP client may choose to ignore it. If client cert is demanded, then a two-way TLS authentication is required and thus the client must proof its identity to the server. This Secure LDAP configuration requires the server to listen to both 389 and 636 ports on the same server to support both secure and legacy applications, which is unnecessary. Secure LDAP therefore is not the preferred approach.

The standardized way of implementing SSL/TLS in LDAP v.3 is to use the StartTLS method. This method should be implemented whenever possible. If an AD server supports StartTLS, the client can start with a STARTTLS command to the server so that the server begins the TLS encryption process. In the binding phase, TLS handshake follows a LDAP_START_TLS_OID command through port 389.

Here’s the summary of the three LDAP configuration mode:

	Legacy	SecureLDAP (aka LDAPS, LDAP over TLS/SSL)	StartTLS
Listening port	389	636	389
Traffic Encrypted	No	Yes	Yes
Standard	Yes but this should always be avoided since it is insecure	Introduced in the time of LDAP v2, but the option is deprecated (although still supported) by RedHat	Introduced in LDAP v3. This may be left as the only valid option.

Note that one of the recent changes that drives may customer away from the legacy mode is the requirement for LDAP channel binding on Windows servers, with a target date of March 2020. Our current strategy at CS is to direct customer towards Secure LDAP as we do not support StartTLS yet and we know we do support LDAPS. Although Secure LDAP itself is somewhat legacy this would not hold long. According to this Wikipedia page:

The use of LDAP over SSL was common in LDAP Version 2 (LDAPv2) but it was never standardized in any formal specification. This usage has been deprecated along with LDAPv2, which was officially retired in 2003.

The trade off between StartTLS and TLS/SSL exists not only in LDAP protocol, but also in many other protocols such as SMTP (port 2525, 25, 587). StartTLS is also called Opportunistic TLS. The standard is in the relevant RFC documents.

Introduction to Active Directory (AD)

Fri, 28 Feb 2020 21:36:00 -0400

Workgroup, homegroup and Windows Domain

A workgroup is a group of computers on the same local network. A Windows computer not joined to a domain is part of a workgroup. In a workgroup, no computer has control over any other computer and it does not require a password. Any computer can join or leave a workgroup any time. Workgroup was previously for home file and printer sharing and Microsoft later introduced homegroup for more security. Compared to workgroup, all computers in a homegroup needs to be on the same home network (instead of local network). Homegroup is password protected. New computer needs to join homegroup by providing the password.

Windows domains (or domains for short) provide network administrators with a way to manage a large number of PCs and control them from one place and remotely. One or more servers — known as domain controllers — have control over the domain and the computers on it. Computers on a domain has to be on the same local network, either physically or over VPN. Centralized control is essential for corporate operation.

Administrators can join a Windows PC with professional or enterprise license to a domain. Once joined, the computer does not use its own local user accounts. When a user logs into a computer on that domain, the computer authenticates the user account name and password with the domain controller. Also, the computer cannot just leave the domain without administrator access.

Network administrators can change group policy settings on the domain controller. Each computer on the domain will get these settings from the domain controller and they’ll override any local settings users specify on their PCs. All the settings are controlled from a single place. This prevents from users from changing many system settings on a computer joined to a domain. The domain controller is in charge of what a user can do.

Apart from centralized administration, the benefit the users is that they can log in with the same username and password on any computer joined to the domain, if permission allows.

Domain Controller

In order to achieve centralized administration and log-in from any computer in the corporate world, a centralized service called domain controller is introduced. At a high level, a domain controller maintains a list for each of the followings:

Users and their passwords
Computers and their credentials

This is because in a domain, not only the users, but also the computers (workstations or servers) need to be authenticated. For example, when a Windows server boots up, it needs to log on to the domain with its own credential. This way we can control whether the server is allowed to query the domain for information about users. If it is allowed to query the domain, then we can determine whether the user is allowed to log on that server, and eventually, authenticate the user.

For a domain controller, it responsibility to credentials for users and computers, and respond to log in requests (authentication service) is a critical commitment in the enterprise environment. Domain controller is therefore commonly built with high availability and fault tolerance.

Administrators needs to add each new user to the user directory in domain controller. They also needs to register each new computer with the domain controller by joining them to the domain.

Joining a Domain

As explained earlier, joining a Window domain means register a computer in the domain so it has the permission to query the domain to validate users identity and permissions. Both Windows server and Linux server can join a domain. Windows servers usually provide a path through UI to join a domain and password is required.

To join a Linux (e.g. Redhat) server to a domain, we can use a tool called adcli. Here is a good example of using this command to join a domain.

Authentication

A Windows or Linux server in the domain needs to go to the domain controller to authenticate itself and the users. Authentication involves several protocols, including kerberos, NTLM, TLS/SSL and Digest, as part of an extensible architecture. In addition, some protocols are combined into authentication packages such as Negotiate and the Credential Security Support Provider.

The MIT Kerberos Documentation provides some tools (e.g. kinit, klist) to configure and troubleshoot Kerberos protocol.

Active Directory

Since Windows 2000, Active Directory is a complete redesign and re-branding of the entire Windows Domain system. The term Active Directory now refer to either the entire domain system, or the actual database that comprises the Windows Domain information or both.

All of the information that makes up an Active Directory is stored in an X.500 compatible database, typically replicated between domain controllers to ensure high availability and fault tolerance. X.500 is a set of network directory standards. A Windows Domain is a kind of network directory, hence the name Active Directory for its replacement.

Active Directory introduced one important new type of object and concept, Forests. An Active Directory Forest is kind of a list of lists, meaning, it is a collection of Domains that are all related to each other for both security and management purposes.

Here is more details about Active Directory.

Lightweight Directory Access Protocol

As mentioned above, X.500 is a series of computer networking standards covering electronic directory services. ISO incorporated it into OSI suite of protocols. The protocols defined by X.500 include DAP (Directory Access Protocol), DSP (Directory System Protocol), DISP (Directory Information Shadowing Protocol) and DOP (Directory Operational Bindings Management Protocol). DAP is a heavyweight protocol that operates over a full OSI protocol stack and requires a significant amount of computing resources. LDAP (Lightweight Directory Access Protocol), as its alternative, is designed to operate over TCP/IP and provides most of the functionality of DAP at a much lower cost.

Technically speaking, LDAP is a directory access protocol to an X.500 directory service. In early days, the typical architecture involves a proxy. Client connects to the proxy in LDAP and the proxy connects to X.500 server in DAP. Nowadays, it is common that LDAP is directly implemented in X.500 servers.

Because LDAP is based on a simpler subset of the standards contained within the X.500 standard, LDAP was sometimes called X.500-lite. While DAP and the other X.500 protocols can now use the TCP/IP networking stack, LDAP remains a popular directory access protocol.

Nginx as a reverse proxy for Nifi web UI and Kibana

Thu, 16 Jan 2020 22:22:51 -0400

Nginx can act as a application neutral proxy. One example is to front Nifi. The nifi default configuration provides an HTTP access point, specified in the following entries in nifi.properties:

nifi.web.http.host=192.168.133.5
nifi.web.http.port=8080

Nifi can provide secure port by commenting out the lines above and provide the followings:

nifi.web.https.host=192.168.133.5
nifi.web.https.port=8083

However, it does require configuring JKS keystore for Java, as well as authentication. Customers with existing AD servers are likely to require authentication via LDAP. While Nifi does support LDAP integration according to its administration guide. The configuration is quite involving. You need to configure the identity provider, as well as authorizes. I have personally spent a couple days on this without much progress. The information in the logging isn’t to the point. Restarting nifi also is a long process, making it painful to troubleshoot.

I then moved to Nginx (open-source) as an alternative and it is quite enlightening. I already knew that the SSL termination in nginx is super easy to configure. This time I learned that the opensource community even has a support for LDAP integration. Here is a diagram of how it works:

This approach is outlined in a blog post on Nginx website. The ldap-auth daemon is implemented in Python can can be wrapped up as a systemd service. Once a client sends a request in https, security layer is terminated in nginx, and an authentication request in http is sent to the ldap-auth daemon, which proxies converts the request into LDAP searches and proxies it over to customer’s Active Directory server, for authentication. Once authenticated, the http request can make to one of the backend container or server which hosts Nifi. Below is an example of how this can be configure on RedHat.

Install python3 and python-ldap

RedHat may have both python2 and python3 pre-installed, python2 being the default. We do not want to change the default because other applications such as yum still depends on python2 as of early 2020.

yum -y install python3
yum -y install gcc python3-devel openldap-devel
pip3 install python-ldap

Once python3 is installed, pip3 will be available and we use that to install python-ldap. This is a module in Python3 that will be used by the script that act as ldap daemon.

Configure ldap-auth daemon as systemd service

In the github project for ldap-auth, download nginx-ldap-auth-daemon.py to local location such as /usr/bin, then we create nginx-ldap-auth.service in /etc/systemd/system/ with the following content.

[Unit]
Description=LDAP authentication helper for Nginx
After=network.target network-online.target
 
[Service]
Type=simple
User=root
Group=root
WorkingDirectory=/var/run
ExecStart=/usr/bin/python3 /usr/bin/nginx-ldap-auth-daemon.py
KillMode=process
KillSignal=SIGINT
Restart=on-failure
 
[Install]
WantedBy=multi-user.target

Then, run the following command to load, start and check nginx-ldap-auth service.

systemctl reload-daemon
systemctl start nginx-ldap-auth
systemctl status nginx-ldap-auth

This service will be up and listening to port 8888 for http traffic.

Configure Nginx

Then configure nginx with the following entries in its default.conf file, typically located in /etc/nginx/conf.d.

upstream nifibackend {
 # default: round robin
 server container1.nifi.digihunch.com:8080;
 server container2.nifi.digihunch.com:8080;
}
proxy_cache_path cache/ keys_zone=auth_cache:10m;
# nifi proxy
server {
 listen 8083ssl;
 include /etc/nginx/ssl/default.conf;
 
 location / {
 auth_request /auth-proxy;
 
 proxy_pass http://nifibackend;
 proxy_set_header Host $host:$server_port;
 proxy_set_header X-ProxyScheme https;
 proxy_set_header X-ProxyHost $1;
 proxy_set_header X-ProxyPort 8083;
 proxy_set_header X-ProxyContextPath /;
 }
 
 location /auth-proxy {
 internal;
 proxy_pass http://127.0.0.1:8888;
 
 proxy_pass_request_body off;
 proxy_set_header Content-Length "";
 proxy_cache auth_cache;
 proxy_cache_valid 200 10m;
 
 proxy_cache_key "$http_authorization$cookie_nginxauth";
 proxy_set_header X-Ldap-URL "ldaps://ldap.digihunch.com:636";
 proxy_set_header X-Ldap-BaseDN "OU=Corporate User Accounts,DC=digihunch,DC=org";
 proxy_set_header X-Ldap-BindDN "CN=Digi Hunch Service Account,OU=Digi,OU=ServiceAccounts,OU=Digi,OU=Digi Applications,DC=digihunch,DC=org";
 proxy_set_header X-Ldap-BindPass "myownpasswordtricks";
 proxy_set_header X-CookieName "nginxauth";
 proxy_set_header Cookie nginxauth=$cookie_nginxauth;
 proxy_set_header X-Ldap_Starttls "true";
 proxy_set_header X-Ldap-Template "(&(sAMAccountName=%(username)s)(objectClass=organizationalPerson)(memberOf=CN=GH_SYSADMIN,OU=GHCO,OU=Groups,OU=Digi,OU=Digi Applications,DC=digihunch,DC=org))";
 }
}

We need the full distinguished name of bind user to get this to work. Once configured properly, and user attempts to connect through a browser, Nginx will pop up a prompt for username and password. The username will be plugged into the X-Ldap-Template for further queries. The same HTTP header also allows you to filter by membership that the user is associated with.