Web Authentication and HTTPS

Preparation

We need many of the installations from the LAMP and other documents, including In particular: We are going to create 3 empty directories in public_html to use for our experiments:
$ cd ~/public_html
$ mkdir protected1 protected2 protected3

Auth SASL Authentication

Apache supports a form of user-based authentication called basic authentication whereby a browser is prompted for a user name and password upon entry into some directory. Once validated, the validation persists (mostly) throughout the browser session.

Apache also provides many ways, via loadable modules, to perform this authentication. The one we're interested in here is the authentication module for SASL (Simple Authentication and Security Layer). The reason this is useful is that SASL acts, as indicated, as a layer between the Apache user and the actual authentication. SASL permits authentication via a number of standard mechanisms, making the whole scheme more flexible if we want to change authentication mechanisms.

Start with the installation:
$ sudo apt-get install sasl2-bin
An Ubuntu feature makes you have to edit the following file to actually enable SASL. Look for the line:

/etc/default/saslauthd
START=no
Change this to:
START=yes
The SASL installation creates a new "sasl" group. A second protection common to Ubuntu is to deny users access to the SASL service unless they belong to this group. Add yourself to it:
$ sudo adduser  sasl
$ groups
The groups command shows you what groups you belong to. Your presence in the sasl group has not been recognized yet, but you can do so with the newgrp command (the second call resets our working group back to the login group) After these statements observe "sasl" as one of your groups.
$ newgrp sasl
$ groups
To test SASL's effectiveness, first, restart the service:
$ sudo service saslauthd restart
Then run this (beware that your password is not hidden!)
$ testsaslauthd -u LOGIN -p password_for_LOGIN_on_MACHINE

Install and use the Apache authn-sasl module

The Apache module is installed and enabled like this:
$ sudo apt-get install libapache2-mod-authn-sasl
$ sudo a2enmod authn_sasl
Like you, Apache's user must belong to the sasl group:
$ sudo adduser www-data sasl
Then restart Apache:
$ sudo service apache2 restart
We're going to show how to password-protect the directory
~/public_html/protected1
using this Apache module. We're going to work with our local Apache configuration file. Add this content to the file:

/etc/apache2/local.conf
...
 
<Directory /home/LOGIN/public_html/protected1>
  AuthType Basic
  AuthName "Restricted"
  AuthBasicAuthoritative On
  AuthBasicProvider sasl
  AuthSaslPwcheckMethod saslauthd
  Require user LOGIN
  #Require valid-user
</Directory>
Reload Apache to pick up the changes:
$ sudo service apache2 reload

Testing

It was remarked about basic authentication that, in regular web browsers, the authentication persists throughout the entire browser session. The point is that there is no easy user-controllable way to remove the authentication without exiting the browser.

So bring up a separate browser to do the tests (Firefox or Chrome), opposite the one you're using to read these notes. Exit, then start over to clear the authentication memory for the next test.

First, view from your test browser:
http://localhost/~LOGIN/
You get a directory listing, but protected1 is not there. Why? Because Apache recognizes the browser's currently unauthenticated access to the directory and hides it from the listing.

Go explicitly to the protected URL:
http://localhost/~LOGIN/protected1
Now the Apache Basic authentication kicks in. Try these tests: We can also use links for testing. Try this:
$ links http://localhost/~LOGIN/protected1
Give incorrect, then correct login/password info to see what happens. The second time give incorrect login/password info; you will be repeatedly prompted — tab to Cancel to get out of the loop.

Site access by other users

Note the commented line in the config file:
<Directory ...>
  ...
  #Require valid-user
</Directory>
If this is put to use, then any valid system user can access this site after giving his/her credentials. There are other alternatives; for example:
  Require user1 user2

Specification using the .htaccess file

Although I don't recommend that you, a system admin, do this, there is an alternative way to create the same effect. Creating the following file containing the directives above, minus the start and end Directory tags:

~/public_html/protected1/.htaccess
AuthType Basic
AuthName "Restricted"
AuthBasicAuthoritative On
AuthBasicProvider sasl
AuthSaslPwcheckMethod saslauthd
Require user LOGIN
The advantage of .htaccess is that it is controlled by the non-administrative user (assuming that the Apache configuration permit these directives), and that changes go into effect without reloading.

The disadvantage of .htaccess is that it is slower because Apache processes directives in .htaccess at run time for each browser request whereas the same directives in .conf files are processed once, at startup-time.

Protect the /phpmyadmin website

You don't exactly want to allow just anyone to access phpMyAdmin site, and so a useful thing to do is restrict access only to users on the system. We can use either authn_sasld or authnz_external. The latter approach has us add this code:

/etc/apache2/local.conf
<Location /phpmyadmin>
  AuthType Basic
  AuthName "Restricted"
  AuthBasicAuthoritative On
  AuthBasicProvider sasl
  AuthSaslPwcheckMethod saslauthd
  Require valid-user
</Location>
select
After adding, restart Apache.

Exit and restart the test browser and then verify that access to
http://localhost/phpmyadmin
works as desired.

AuthNZ External Authentication

Install the Apache module by:
$ sudo apt-get install libapache2-mod-authnz-external
The installation also enables the Apache module (authnz_external). The point of using this module is that you can use an external program to perform the access authentication. Apache retrieves a username and password through the browser interaction and sends these on separate lines to the external program. The program decides whether to authenticate or not by exiting with 0, if OK, and non-zero if not.

We will first test authentication using the pwauth program provided by the auxiliary package of the same name which is also installled. Add this content to the file:

/etc/apache2/local.conf
...
 
DefineExternalAuth pwauth pipe /usr/sbin/pwauth
 
<Location /~LOGIN/protected2>
  AuthType Basic
  AuthName "Restricted"
  AuthBasicProvider external
  AuthExternal pwauth
  Require user LOGIN 
</Location>
 
and restart Apache:
$ sudo service apache2 restart
Note that we're using "Location" as an alternative to "Directory" in the configuration.

Try the following URL, both with your test browser and using links. Remember to exit and restart the test browser.
http://localhost/~LOGIN/protected2
We can verify what's happening by these tests (your machine password is entered in clear text):
--- successful ---
$ pwauth
LOGIN  
LOGIN_password_on_MACHINE 
$ echo $?
0
$ clear
--- unsuccessful ---
$ pwauth
LOGIN 
anything_but_the_right_password
$ echo $?
1

The program's exit status, "$?", is all that Apache uses to decide whether to authenticate or not.

AuthNZ External with Database

In this example we will use a MySQL database to hold (user,password) records. A Python program createauth.py, will be employed to add records to the database, and another, dbauth, will do the authentication.

We need a Python database package installed:
$ sudo apt-get install python-pymysql

Create the backend database

Create the following database setup file (say, in your home directory):

~/auth-setup.sql
create database if not exists auth;
create user if not exists authuser@localhost identified by 'authpass';
grant all on auth.* to authuser@localhost;
select
Create the database by running:
$ sudo -H mysql < ~/auth-setup.sql

Populate the database

Create a python program to initialize the database, creating a table holding the credentials for a single user with credentials:
myuser/myuserpass
This program can reside anywhere, and so we'll assume it's in your home directory.

~/create_auth.py
#!/usr/bin/env python
 
import pymysql, sys
import hashlib
 
database    = 'auth'
db_username = 'authuser'
db_password = 'authpass'
 
# make a connection
cx = pymysql.connect(
  host='localhost', db=database, user=db_username, passwd=db_password
)
 
# cursor() gets a blank "statement" from the connection 
cur = cx.cursor()
 
table = 'user'
 
sql = "drop table if exists {}".format(table)
 
cur.execute(sql)
 
sql = """create table {} (
           username varchar(255) not null primary key, 
           password char(64) not null
         )""".format(table)
 
cur.execute(sql)
 
sql = "insert into {} values(%s,%s)".format(table)
 
username = 'myuser'
password = 'myuserpass'
 
cur.execute( sql, [ username, hashlib.sha256(password).hexdigest() ] )
 
cx.commit()
select
Run it to make it take effect:
$ python ~/create_auth.py
Ignore any warning about unknown table (this must be an innocuous bug). The code is written to recreate the table each time, and so you can repeat it.

Verify the result by running:
$ sudo -H mysql auth
mysql> select * from user;
Password Information We are managing the password encryption external to the MySQL database. It could be done internally, but, as of MySQL verion 5.7.6 and beyond, the internal encryption has become vastly more complex. This scheme we're using is called a 256-bit hash digest. The encryption generated is always 64, 4-bit hex digits.

The external dbauth program

This python script will play the role of pwauth from the previous section. Create this file (as root):

/usr/local/bin/dbauth
#!/usr/bin/env python
 
import pymysql, sys
import hashlib
 
database    = 'auth'
db_username = 'authuser'
db_password = 'authpass'
 
# make a connection
cx = pymysql.connect(
  host='localhost', db=database, user=db_username, passwd=db_password
)
 
# get two lines from standard input, remove the trailing newline
username = sys.stdin.readline().rstrip('\n')
password = sys.stdin.readline().rstrip('\n')
 
# cursor() gets a blank "statement" from the connection 
cur = cx.cursor()
 
table = 'user'
 
# execute the sql statement looking form username/password match
sql = """select count(*) from {} 
         where username=%s and password=%s""".format(table)
 
cur.execute( sql, [ username, hashlib.sha256(password).hexdigest() ] )
 
rows = cur.fetchone()
 
# rows[0] = num matching rows, this is either 1 or 0 because of 
# username uniqueness constraint, 1 = success, 0 = failed
 
exit( 1 - rows[0] )
select
We want the program to be generally accessible, and so it resides in /usr/local/bin. You can create it directly as root, or create it as you and move it into place.

Set the permissions and ownership on it so that it usable only by Apache (and root, of course):
$ sudo chmod 750 /usr/local/bin/dbauth
$ sudo chown root:www-data /usr/local/bin/dbauth
$ ll /usr/local/bin/dbauth
-rwxr-x--- 1 root www-data ... /usr/local/bin/dbauth*
Run these shell-based tests to verify the intended behavior:
$ sudo dbauth
myuser
myuserpass
$ echo $?
0
$ sudo dbauth
xxx
yyy
$ echo $?
1

Create the Apache configuration

Edit your Apache /etc/apache2/local.conf file and add this code:

/etc/apache2/local.conf
...
 
DefineExternalAuth dbauth pipe /usr/local/bin/dbauth
 
<Directory /home/LOGIN/public_html/protected3>
  AuthType Basic
  AuthName "Restricted"
  AuthBasicProvider external
  AuthExternal dbauth
  require valid-user
</Directory>
Make it take effect by restarting Apache.

Test the site as we did before. Use both your test browser and links.
http://localhost/~/protected3
Remember to use the credentials myuser/myuserpass.

Further Apache access control

The access controls we've seen so far are
require all granted        (everyone)
require user ...           (list of users)
require valid-user         (any valid user)
But we can go further, specifying access from certain host names or ip addresses. Apache 2.4 access control directives are explained in the Apache documentation:
Access Control.
If we two such directives, the effect is the "or" of the two, i.e., a client accessing the site must satisfy at least one of the requirements. Let's reconsider the access to phpMyAdmin, adding another require directive:

/etc/apache2/local.conf
<Location /phpmyadmin>
  AuthType Basic
  AuthName "Restricted"
  AuthBasicAuthoritative On
  AuthBasicProvider sasl
  AuthSaslPwcheckMethod saslauthd
 
  require valid-user
 
  # add this line:
  require ip 127.0.0.1  ::1
</Location>
Reload Apache to have it take effect.

As it stands with this addition, satisfying any of the require statements will permit access. What this means is that

HTTPS

HTTPS (Secure HTTP) offers encrypted transmission of information on the web. It is based on encryption provided by the secure socket layer (SSL) protocol applied to the HTTP. Apache/SSL is called ModSSL because it is represents a module for SSL loaded into the Apache server.

Configure the default SSL site

$ sudo a2enmod ssl
$ sudo a2ensite default-ssl
$ sudo service apache2 reload
Ubuntu sets you up with a self-signed certificate. Look in the file
/etc/apache2/sites-enabled/default-ssl.conf
for the lines which identify the public certificate and private key as:
SSLCertificateFile    /etc/ssl/certs/ssl-cert-snakeoil.pem
SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key
Modern browsers are downright hostile about self-signed certificate and try to dissuade only the most resolute user from going ahead.

For example, try Firefox on the HTTPS home URL:
https://localhost
I get the frightening message:

Your connection is not secure

blah, blah, blah, ...

Of course, we know it is legitimate, so take a deep breath and work your way through the steps to confirm the security certificate. You can view the certificate you just permanently accepted by going to:
Edit ⇾ Preferences ⇾ Advanced ⇾ Certificates ⇾ View Certificates ⇾ Servers
Most likely it's at the bottom: "ubuntu-mate.home".

Unfortunately, to get a satisfying experience with HTTPS, you really need a real certificate, which is beyond the scope of this course. However, these self-signed ones are ultimately just as good as real ones if you can initially trust them.

Here are two alternatives to ensure the usage of HTTPS: disallow HTTP, and force redirection to HTTPS. Let's use this site as our example.
/MACHINE/~LOGIN/protected3
Currently access to this site is protected by basic authentication, but under normal circumstances this authentication information is transmitted in clear text from an external browser to the Apache server.

Disallow HTTP

The presence of the SSLRequireSSL directive will disallow http. Test it by adding the following line:

/etc/apache2/conf-available/local.conf
<Directory /home/LOGIN/public_html/protected3>
  ...
  SSLRequireSSL
</Directory>
Try it by adding it as indicated, restarting Apache and then checking out the outcome for
http://MACHINE/~LOGIN/protected3
You get:

Forbidden

You don't have permission to access /~LOGIN/protected3 on this server.

So explicitly tack on the https protocol:
https://MACHINE/~LOGIN/protected3
Remember, this takes the myuser/myuserpass credentials.

Redirect http to https

It's common to be more friendly and automatically redirect http requests to https when https is required. The change must be made in the Apache "http-site" file. At the top add this rule:

/etc/apache2/sites-enabled/000-default.conf
<VirtualHost *:80>
  Redirect permanent /~LOGIN/protected3  https://MACHINE/~LOGIN/protected3
  ...

Discussion

More information can be found at:
http://www.modssl.org/
HTTPS (as well as other secure protocols) is based on transmission of a X.509 certificate, commonly known as a digital certificate. The web server administrator creates a public/private key pair, sends the public key to a Certificate Authority (CA) which binds the public key to information about the web server (location, DNS name, etc.) The binding method hashes the subject information, encrypts it using the CA's private key and attaches the encryption along with information about the CA, encryption algorithm, etc. This total conglomeration of information is the certificate.

An established CA will charge you for such a certificate. The point is that the CA's information and public key is readily available to common browsers and so it can be validated that this certificate was really issued by the CA. Thus the client can be assured that the server's public key is valid.

If you use a self-signed certificate, it means that you are acting like a CA, using your own private key to create the certificate. The problem is that standard browsers won't recognize you as a legitimate CA and will give a security warning to the user. The problem is legitimate, because how does the browser actually know that the information contained within is valid if not signed by a known CA. This problem is referred to as the non-repudiation problem, namely, being uncertain that the public key received is the legitimate public key of the intended server.


© Robert M. Kline