Virtualization with KVM

Virtualization

The goal of virtualization is to provide a software platform consisting an instance of an operating system running as a virtual machine within the existing machine. A system which supports virtualization is called the host and a virtual machine is called a guest. There are a number of software tools called hypervisors, which serve as the virtualization software, including KVM, XEN, VmWare, VirtualBox, etc. Ubuntu focuses on the KVM (Kernel-based Virtual Machine) software which requires that the CPU has a hardware virtualization extension.

The main benefit for us is the ability to set up a "test bed" for trying out software and networking features without using other physical computers. One can use this virtualization to install multiple versions of Linux in order to evaluate and compare them. From a security perspective, we can use virtual machines as "attack targets" for security software testing. In an enterprise setting a virtual machine can be a lightweight specialized "server" which is optimized to perform one service task.

Enable Virtualization in the BIOS

This is how to do so for the Lab computers. Start by rebooting the machine. You have to catch it before it boots the operating system. Type F12 repeatedly as we did for the installation.
  1. Select: Enter Setup
  2. Go to the Advanced tab, select CPU Setup.
  3. Select Intel (R) Virtualization Technology [Disabled]. Enter
  4. Select Enabled. Enter.
  5. Type F10 to save and exit.

Software Installation

Install the necessary packages:
$ sudo apt-get install qemu qemu-kvm virt-manager virt-viewer libvirt-bin uvtool
As installer, you are added to the libvirtd group allowing you to use the virtualization tools as you without becoming root. You must log-out/log-in in order to pick up your membership in this group.

After logging in again, confirm:
$ groups
... libvirtd ...
Our guest machine's network will be attached to the virtual bridge interface virbr0, which has been created in the libvirt-bin installation. Check it out by:
$ ifconfig virbr0
We're going to use so-called cloud images obtained and manipulated by the uvtool package.
https://help.ubuntu.com/16.04/serverguide/cloud-images-and-uvtool.html
Download the cloud image (this takes some time, be patient):
$ uvt-simplestreams-libvirt sync release=xenial arch=amd64
This image is that of Ubuntu server without extraneous features needed to be on a real server OS.

Create a VM and access it

In order to be able to get into the virtual machines created from the cloud image, you need to have created your RSA key:
~/.ssh/id_rsa.pub
Then create the machine and start it running:
[MACHINE]$ uvt-kvm create vm --memory 256
Within a short time, you can access it by:
[MACHINE]$ uvt-kvm ssh vm --insecure
The login you're using is ubuntu. The VM creation aleady establishes key access to the guest using this login which you can check by:
ubuntu@vm:~$ cat .ssh/authorized_keys
In fact, by default, the only way you can externally access this VM is by public key. You can check that out by:
ubuntu@vm:~$ egrep ^.?Password /etc/ssh/sshd_config
PasswordAuthentication no
whereas if you go back to the host and do the same, you'll see
[MACHINE]$ egrep ^.?Password /etc/ssh/sshd_config
#PasswordAuthentication no
The significance is the password authentication is intentionally turned off in the guest machines created in this way.

RAM and Disk space

The "--memory 256" flag means give the VM 256M of RAM, which is subtracted from RAM available to the host when the guest is running. According to the Ubuntu docs, 192M of RAM is sufficient to run the server OS.

The default disk size is 8G, but the disk space allocation is only "as needed" up to 8G. The type of files created to represent virtual disks are called sparse files, consisting mostly of zeros. The unused space can be used for other operating system purposes. If you want a potentially larger VM disk, use the option:
$ uvt-kvm create ... --disk size     (size in gigabytes)
You can see the actual disk image files and their "virtual" sizes by running:
$ sudo ll -h /var/lib/uvtool/libvirt/images/
To see the "actual" sizes, use the du -h command:
$ sudo du -h /var/lib/uvtool/libvirt/images/*
For example, the Wordpress VM we're creating in the next section occupies about 900MB when installed.

Control by virsh

It is easy to stop/start machines from VMM, but you can also use the shell-based virsh tool to control the virtual machines from a non-GUI environment. For example, start our guest by:
$ virsh start vm
Get a list of running VM's by:
$ virsh list
Shut it down by:
$ virsh shutdown vm
virsh acts like a command shell in its own right if you activate it without parameters
$ virsh
virsh # help

Delete a VM

For proof of concept, this is how you delete a virtual machine created in this way:
$ uvt-kvm destroy vm
Doing this is WAY to easy, careful with this command!

Help!

If you somehow really screwed things up so that you cannot access your virtual machine, here is suggested fix:
$ sudo apt-get purge --auto-remove uvtool

Build a Wordpress Virtual Machine

This demonstration project illustrates several interesting new features: We'll call the dedicated VM wp. Once it is created, it's most convenient to leave two shells open, one on the host and other on the guest.
[MACHINE]$ ...
    [wp]$ ...

Create wp and update it

Follow the procedure we've laid out:
[MACHINE]$ uvt-kvm create wp --memory 256

[MACHINE]$ uvt-kvm ssh wp --insecure
[wp]$ sudo apt update
Note that user is ubuntu and that you didn't have to give the password for sudo usage, how is that? The gimmick is a special control file in the /etc/sudoers.d directory. Observe:
[wp]$ sudo su
[wp]# ls /etc/sudoers.d/
[wp]# cat /etc/sudoers.d/90-cloud-init-users
...
ubuntu ALL=(ALL) NOPASSWD:ALL
The uncommented line means that the ubuntu user can use sudo to run all commands without giving a password.

For the sake of expediency, we'll omit the OS upgrade.

Change networking on wp

We want to change the networking information, setting a static IP address.
[wp]$ sudo nano /etc/network/interfaces
The current file content is:
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# Source interfaces
# Please check /etc/network/interfaces.d before changing this file
# as interfaces may have been defined in /etc/network/interfaces.d
# See LP: #1262951
source /etc/network/interfaces.d/*.cfg
The main network interface is defined in the auxiliary file:

/etc/network/interfaces.d/50-cloud-init.cfg
auto lo
iface lo inet loopback
 
auto ens3
iface ens3 inet dhcp
Edit the main networking file. Comment out the source statement and add code which sets a static IP address

/etc/network/interfaces
...
#source /etc/network/interfaces.d/*.cfg
 
auto ens3
iface ens3 inet static
    address 192.168.122.11
    gateway 192.168.122.1
    netmask 255.255.255.0
    dns-nameservers 192.168.122.1
Reboot the virtual machine (it's quite fast):
[wp]$ sudo reboot
Test to ensure connectivity:
[MACHINE]$ ping 192.168.122.11
If OK, assign a name to this IP address on your host machine. Edit /etc/hosts, adding this line:

/etc/hosts (on MACHINE)
...
192.168.122.11    wp
...
Also, add another entry to ~/.ssh/config to set the user correctly for access to wp:

~/.ssh/config (on MACHINE)
...
host wp
  user ubuntu
Then test to confirm that you can enter the guest like this:
[MACHINE]$ ssh wp
[wp]$ 

Open MySQL on host to external network access

First, we have to allow MySQL on the host to be accessed externally, in particular, from the virtual machine. To do so edit the MySQL configuration file
/etc/mysql/mysql.conf.d/mysqld.cnf
Look for the line:
bind-address = 127.0.0.1
Comment it out:
#bind-address = 127.0.0.1
Then restart MySQL:
[MACHINE]$ sudo service mysql restart
Without this step, clients can only connect to the database from localhost.

Set up the wordpress database on host

Next, we want to create a dedicated database for wordpress. Again, we need user/password credentials. After installation, you do not need to remember these credentials.
Wordpress DB Password:
Create the following file (say in your home directory):

wordpress.sql
CREATE DATABASE wordpress;
GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,DROP,ALTER ON wordpress.*
TO wordpress IDENTIFIED BY 'WP_DB_PASS';
FLUSH PRIVILEGES;
select
Run it on the host:
[MACHINE]$ sudo -H mysql < wordpress.sql
Test access from the host machine and guest machines:
[MACHINE]$ mysql -u wordpress -pWP_DB_PASS wordpress
From the guest machine we use the -h (host) flag. Here we just need the MySQL client for testing only.
[wp]$ sudo apt-get install mysql-client
[wp]$ mysql -h 192.168.122.1 -u wordpress -pWP_DB_PASS wordpress

More info To better understand what is going on, query the mysql database:
[MACHINE]$ sudo -H mysql mysql
mysql> select user,host from user; 
mysql> select user,host,db from db;
Note the presence of the wildcard "%" in the host fields for both tables. It means that the wordpress user can access MySQL from any host, and that the wordpress database accessed by the wordpress user can be accessed from any host.

Install and configure Wordpress in the guest

[wp]$ sudo apt-get install wordpress
Once this is done, create an Apache configuration file defining the URL "/discussion" for the site implemented by wordpress:

/etc/apache2/conf-available/wordpress.conf (on wp)
Alias /discussion /usr/share/wordpress
<Directory /usr/share/wordpress>
  Options FollowSymLinks
  AllowOverride Limit Options FileInfo
  DirectoryIndex index.php
  Require all granted
</Directory>
<Directory /var/lib/wordpress/wp-content>
  Options FollowSymLinks
  Require all granted
</Directory>
select
Enable this .conf file:
[wp]$ sudo a2enconf wordpress
Create the following wordpress configuration file. Note the presence of the wordpress user/database password, WP_DB_PASS in clear text (common to CMS systems).

/etc/wordpress/config-wp.php (on wp)
<?php
define('DB_NAME', 'wordpress');
define('DB_USER', 'wordpress');
define('DB_PASSWORD', 'WP_DB_PASS');
define('DB_HOST', '192.168.122.1');
define('WP_CONTENT_DIR', '/usr/share/wordpress/wp-content');
select
Restart Apache:
[wp]$ sudo service apache2 restart
You can now "forget" the Wordpress DB Password with respect to this document.
Wordpress DB Password:

Configure the Wordpress site

Access and configure wordpress from the host vial the URL:
http://wp/discussion
Running the site the first time will ask to set up an administrative user and other parts. Like in Drupal, you need a password, so I would again recommend your machine's login password, since this is again encrypted in the wordpress database.
Site Tite


Username:


Password


Confirm password (if necessary):
Confirm usage of weak password    

Email

Click:

Finally, log in to confirm. We're not going to set up anything specific.

Make wp externally accessible

Enable mod_proxy_http on the host:
[MACHINE]$ sudo a2enmod proxy_http
We assume you've created a "local.conf" Apache config file and made it accessible in /etc/apache2 via a symbolic link; otherwise, set it up. Append this to the host's /etc/apache2/local.conf file:

/etc/apache2/local.conf (appended)
ProxyPreserveHost On
ProxyPass  /discussion   http://wp/discussion
select
Restart Apache on the host:
[MACHINE]$ sudo service apache2 restart
On the guest machine, wp, create another wordpress file specific to the new URL we want to apply. We can use config-wp.php as the basis and append to it:
[wp]$ sudo su
[wp]# cd /etc/wordpress
[wp]# cp config-wp.php config-MACHINE.php
[wp]# cat <<END >> config-MACHINE.php 
define('WP_HOME','http://MACHINE/discussion');
define('WP_SITEURL','http://MACHINE/discussion');
END
The modification adds two new configuration settings to force the new URL to be maintained through the site usage. Verify:
[wp]# cat config-MACHINE.php
Now you should be able to access wordpress using the URL:
http://MACHINE/discussion
Try accessing and logging into this site to verify that the base URL is maintained. This URL is also available externally from any taz client.

Tunnel Access

If you want to access this through the taz tunnel via:
http://localhost:2003/discussion
you have to create yet another wordpress configuration file:
[wp]$ sudo su
[wp]# cd /etc/wordpress
[wp]# cp config-wp.php config-localhost.php
[wp]# cat <<END >> config-localhost.php 
define('WP_HOME','http://localhost:2003/discussion');
define('WP_SITEURL','http://localhost:2003/discussion');
END
Then verify:
[wp]# cat config-localhost.php

Django blog app on a VM

Here is another example in which we are using as a basis, the Django site constructed in these two documents (you'd have to complete it):
Django
Django App Features
The variation is to run this Django site, slightly modified, on a virtual machine. The kinks of this construction are not entirely worked out.

Guest creation

Create a dedicated virtual machine and access it:
$ uvt-kvm create djsite --memory 256
$ uvt-kvm ssh djsite --insecure
Set a static IP address as before. Edit /etc/network/interfaces and make the IP have last octet 12.
...
#source /etc/network/interfaces.d/*.cfg

auto ens3
iface ens3 inet static
    address 192.168.122.12
    gateway 192.168.122.1
    netmask 255.255.255.0
    dns-nameservers 192.168.122.1
After completion reboot the VM.

Create an entry in the /etc/hosts file on the host machine:
192.168.122.12   djsite
Create an entry in your ~/.ssh/config file:
Host djsite
  user ubuntu
Maintain 2 shells, one for host, other for guest. Go into the guest like this:
[MACHINE]$ ssh djsite

Guest appsite installation

Update and do the package installations (we'll skip upgrade now to expedite things)
[djsite]$ sudo apt-get update
[djsite]$ sudo apt-get install \
  python-django sqlite3 python-sqlite apache2 links libapache2-mod-wsgi
Then do the website installation:
[djsite]$ django-admin startproject appsite  
[djsite]$ cd appsite
[djsite]$ python manage.py startapp blog
Edit the appsite/settings.py file on djsite, making these changes:
INSTALLED_APPS = (
  ...
  'blog',
)

DATABASES = {
    'default': {
        ...
        'NAME': os.path.join(BASE_DIR, 'database', 'db.sqlite3'),
    }
}

TIMEZONE = 'America/New_York'

USE_TZ = False

STATIC_URL = '/static/appsite/'

STATICFILES_DIRS = (
    '/var/www/html/static/appsite',
)
Send selected appsite files from host to the guest:
[MACHINE]$ cd ~/workspace/appsite
[MACHINE]$ wget ftp://ftp.cs.wcupa.edu/pub/rkline/linux/SEND2DJ
[MACHINE]$ chmod +x SEND2DJ
[MACHINE]$ ./SEND2DJ
Pitch for Program 1 If you have a working transfer program you could transfer the files as follows:
[MACHINE]$ transfer -uf TRANS.xml
using this config file:

TRANS.xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE sync SYSTEM "/usr/local/share/xml/declaration/transfer.dtd">
<sync src="~/workspace/appsite" dst="djsite:appsite">
<rules>
+ /appsite
+ /blog
+ /blog/templates
+ /blog/templates/*
+ /appsite/urls.py
+ /blog/urls.py
+ /blog/views.py
+ /blog/models.py
+ /make_entries.py
- *
</rules>
</sync>
The file you couldn't send in this manner is layout.css, which lies outside the ~/workspace/appsite directory.

On the guest, install the database. We'll use the default SQLite database, but we need to make some placement and permission changes.
--- in ~/appsite ----
[djsite]$ mkdir database
[djsite]$ chmod 777 database/
[djsite]$ touch db.sqlite3 
[djsite]$ chmod 666 db.sqlite3 
[djsite]$ mv db.sqlite3 database/
Now the actual installation:
[djsite]$ ./manage.py makemigrations
[djsite]$ ./manage.py migrate
[djsite]$ ./make_entries.py 
Move the stylesheet sent into place:
[djsite]$ sudo mkdir -p /var/www/html/static/appsite/css
[djsite]$ sudo mv layout.css /var/www/html/static/appsite/css

Test on development server

The /blog app should work now for the development server. In order to test this, use a second guest shell:
[MACHINE]$ ssh djsite
[djsite]$ cd appsite
[djsite]$ ./manage.py runserver
Test it from the first guest shell:
[djsite]$ links http://localhost:8000/blog
Once you've confirmed that it works, you can shut down the second guest shell:
Ctrl-C            (stop the development server)
[djsite]$ exit

Make the site availalble to the host

We want to run the site through Apache on the guest using ModWSGI. First add some necessary code into the wsgi.py file:

appsite/appsite/wsgi.py (on djsite)
...
 
# add these 3 lines just above the last line in the file:
import sys
from os.path import realpath, dirname
sys.path.append( dirname(dirname(realpath(__file__))) )
 
application = get_wsgi_application()
Edit the Apache http site file on djsite. Remove what's in there and make the content be this:

/etc/apache2/sites-available/000-default.conf (on djsite)
<VirtualHost *:80>
  WSGIScriptAlias / /home/ubuntu/appsite/appsite/wsgi.py
 
  Alias /static /var/www/html/static
 
  <Files wsgi.py>
    Require all granted
  </Files>
 
  ErrorLog ${APACHE_LOG_DIR}/error.log
  CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>
Restart Apache:
[djsite]$ sudo service apache2 restart
At this point you should be able to access the VM site from the host (in a normal browser):
http://djsite/blog
We'll stop here, because we already have a version of http://localhost/blog. We make note that this version could serve as a replacement for the /blog app already created.

Admin access

The djsite Apache host serves all of its apps, including the admin app, which could be usefule to modify the database from the host level.

First, give access to the admin static files (this is a bit of a hack):
[djsite]$ sudo ln -s \
  /usr/share/python-django-common/django/contrib/admin/static/admin \
  /var/www/html/static/appsite/
Create an admin account for yourself:
--- in ~/appsite ----
[djsite]$ ./manage.py createsuperuser
Finally, access the admin app with the credentials you just set:
http://djsite/admin
As before, to pick up access to the Entry model, you must modify the blog/admin.py file:

appsite/blog/admin.py
from django.contrib import admin
 
# Register your models here
 
from blog.models import Entry
admin.site.register(Entry)
Restart Apache before refreshing the browser:
$ sudo service apache2 reload


© Robert M. Kline