Webserver Speedups
— print (last updated: Jun 18, 2009) print

Select font size:
Personalize this online document by providing values relevant to you. Replace the generic LOGIN by the actual login:
   

Speedy/CGI

Speedy/CGI is a technique for speeding up CGI scripts by running them persistently. This is apparently the home page:
http://daemoninc.com/SpeedyCGI/
It is very easy to install and use.
$ sudo apt-get install speedy-cgi-perl
To use it in it simplest form, all you have to do is to call the speedy executable in a Perl/CGI script. Simply replace
#!/usr/bin/perl -w
by
#!/usr/bin/speedy

Test it

Assuming you have the Perl/CGI example installed in ~/public_html:
$ cd ~/public_html/perl-cgi
$ cp printenv.cgi printenv-speedy.cgi
Then edit printenv-speedy.cgi and change the shebang line to
#!/usr/bin/speedy
Verify that it works by:
http://localhost/~LOGIN/perl-cgi/printenv-speedy.cgi
It should look the same as the regular perl script. What is different is that there is a persistent process involved. To get an idea of what's going on do this:
$ ps ax | grep speedy_backend
You should see a representation of the script as a process.

Benchmark comparison

Compare the speedy performance with regular perl/CGI performance using the Apache Benchmark software tool, ab. For comparison sake, it's best to run each command in separate shells (not concurrently):
$ ab -n 2000 -c 100 http://localhost/~rkline/perl-cgi/printenv.cgi
versus:
$ ab -n 2000 -c 100 http://localhost/~rkline/perl-cgi/printenv-speedy.cgi
This is creating 2000 request with a concurrency level of 100 connections. You'll notice that the latter is faster and the statistics bear this out. Look at the number for:
Time per request:      ______ [ms] (mean, across all concurrent requests)
A discussion of the ab tool and other benchmarking tools can be found at this URL:
http://www.xenoclast.org/doc/benchmark/HTTP-benchmarking-HOWTO/node6.html
According to this document, the ab tool is not the most reliable performance measure, and the httperf tool is better.

Even faster

Speedy can be installed an run as an Apache module. Try this:
$ sudo apt-get install libapache2-mod-speedycgi
$ sudo /etc/init.d/apache2 reload
The installation already enables the speedcgi module. Look at the config file:
$ cat /etc/apache2/mods-enabled/speedycgi.conf
By default, it's expecting the ".speedy" suffix. Let's try it
$ cd ~/public_html/perl-cgi
$ cp printenv-speedy.cgi  printenv-speedy.speedy
Verify that it works by:
http://localhost/~LOGIN/perl-cgi/printenv-speedy.speedy
Then benchmark it:
$ ab -n 2000 -c 100 http://localhost/~rkline/perl-cgi/printenv-speedy.speedy
You'll see that this is significantly faster than either of the previous two versions. The main issue with this approach is there does not appear to be any way (at least, any easy way) to make this last version work in suexec mode.

Compressed JavaScript

One of the downsides of client-side (i.e., JavaScript) toolkits like jQuery is that browsers must download a significant amount of JavaScript code in order to use them. From the jQuery website, the JavaScript file is presented in two forms minified and uncompressed, e.g.:
jquery-1.3.1.min.js   54.0 KB   (minified)
jquery-1.3.1.js       114  KB   (uncompressed)
The minified form has exactly the same code, except that almost every possible whitespace character is removed, leaving it essentially unreadable. The reason the minified version is better to use is that, obviously, that it requires less bandwidth than the compressed version.

Even better is using an actual compression. It turns out that you usually can use gzip-based compression. A browser request will send to the server its capabilites in the HTTP request header, and in this header, indicate whether it can receive and process gzipped content. To see what a browser sends, install the Live HTTP headers plugin in Firefox by going to:
Tools Add-ons Get Add-ons
Enter "http headers" in the search box to bring up Live HTTP headers. Then install it, restart Firefox, and bring up Tools Live HTTP headers from the menu.

Assuming you have jQuery installed as indicated in the previous documents, active this URL:
http://localhost/javascript/jquery.js
Here are the headers (client & server) generated:
GET /javascript/jquery.js HTTP/1.1
Host: localhost
User-Agent: Mozilla/5.0 ...
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer: ...
Cookie: ...
Pragma: no-cache
Cache-Control: no-cache

HTTP/1.x 200 OK
Date: Wed, 17 Jun 2009 02:19:03 GMT
Server: Apache/2.2.11 ...
Last-Modified: Tue, 21 Apr 2009 11:27:58 GMT
Etag: "1e4944-dfa6-4680ef0586b80"
Accept-Ranges: bytes
Content-Length: 57254
Keep-Alive: timeout=15, max=100
Connection: Keep-Alive
Content-Type: application/javascript
The key client information is the gzip value within the in the Accept-Encoding key which indicates that the browser accepts gzip compression. We can see how much bandwidth we'll save by counting the number of bytes in the compressed form:
$ gzip -c /usr/local/share/javascript/jquery.js | wc -c
It should come out to around 19K bytes, one third of the minimified version and one sixth of the uncompressed version.

There appear to be a wide variety of ways to have the Apache server do the compression, but I've found this Php-based way to be simple and successful (it can even be used if you don't have root access). Edit the file /etc/apache2/conf.d/aliases.conf and add this to the end:
<Directory /usr/local/share/javascript>
  php_value output_handler ob_gzhandler
  AddHandler application/x-httpd-php .js
</Directory>
Then reload Apache. To see the effect, refresh the URL and look at HTTP headers:
http://localhost/javascript/jquery.js
In addition to the different appearance, note the change in the server-side response header which indicates that it sent the gzip-compressed data:
HTTP/1.x 200 OK
Date: ...
Server: ..
X-Powered-By: PHP/5.2.6-3ubuntu4.1
Content-Encoding: gzip
Vary: Accept-Encoding
Keep-Alive: timeout=15, max=100
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: text/html

Lighttpd Server

The Lighttpd (pron. lighty) HTTP server is an alternative to Apache. It claims to be designed for high performance. The home and documentation pages are these:
http://www.lighttpd.net/
http://redmine.lighttpd.net/wiki/lighttpd
To install in Ubuntu:
$ sudo apt-get install lighttpd
The configuration files are:
/etc/lighttpd/
/etc/lighttpd/lighttpd.conf          (main config file)
/etc/lighttpd/conf-enabled/*.conf    (extra config files)
The server will fail to start because of the port conflict with Apache on port 80. We want to run it as an alternative on port 81, so edit /etc/lighttpd/lighttpd.conf and look for the line which defines server.port and set it:
## bind to port (default: 80)
server.port = 81
Also, enable the necessary modules while we're at it. We'll soon configure the CGI module.
$ sudo lighty-enable-mod  cgi  userdir  fastcgi
$ sudo /etc/init.d/lighttpd start
Tentatively, try these URLs:
http://localhost:81/         and         http://localhost:81/~LOGIN/perl-cgi

Configuration

As in the case of Apache, we must do some configuration steps to get Lighttpd to work on Php and Perl/CGI scripts. Setting up Perl/CGI is easy; however, creating suexec capabilities is difficult, poorly documented, etc. You have to use the Apache suexec executable, /usr/lib/apache2/suexec, explicitly. which is also difficult due to the poor documentation and strong usage restrictions. The way it works out (for me) most easily is that a each user who wants to run CGI scripts in suexec mode must own a directory within in the server's document root. The directory we create for the user, although it could be named otherwise, uses their login name.

Here is how to proceed. You need two executables, one owned by root in a common location and the other in the special directory in the document root.
  1. Create a directory named by your login owned by you within the document root, /var/www. The easiest way is:
    $ mkdir LOGIN 
    $ sudo mv LOGIN /var/www
    
  2. Next, download the support files and run the installation script:
    $ wget ftp://ftp.cs.wcupa.edu/pub/rkline/gradlinux/suexec.zip
    $ unzip suexec.zip
    $ cd suexec 
    $ ./install.sh
    
  3. Finally, configure Lighttpd to call the wrapper program. Edit /etc/lighttpd/conf-enabled/10-cgi.conf, go to the end of the file and add one line:
    cgi.assign  = ( ".cgi" => "/usr/local/bin/cgi-wrapper",)
    
Replicating our javascript handling with compression is simple. Create a new file:
/etc/lighttpd/conf-enabled/aliases.conf 
Within it, add these line:
alias.url += ( "/javascript/" => "/usr/local/share/javascript/" )
alias.url += ( "/websvn/" => "/usr/share/websvn/" )

compress.filetype += ( "application/javascript" )
After all these steps, restart the server:
/etc/init.d/lighttpd restart

Tests

With exception of the ".speedy" scripts and wordpress, you should be able to run every web application simply by adding the ":81" port designation. The fastcgi module in Lighttpd automatically builds in support for Php/CGI, which it claims to be faster than using the Apache Php module.

Here are some tests:
  1. Regular CGI:
  2. Speedy CGI:
  3. JavaScript compression (open HTTP headers in Firefox):
  4. Authlog application:
  5. WebSVN:

Php Benchmark

An interesting benchmark comparison is the comparison of a Php-based web application:
$ ab -n 4000 -c 100 http://localhost/websvn
versus:
$ ab -n 4000 -c 100 http://localhost:81/websvn


© Robert M. Kline