Over the past half a year we worked on several systems for logistics/courier/delivery companies. It was quite fun as a big logistics and inventory control system was my very first project when I arrived to Asia 9 years ago. Last week we launched one of them. It was slightly different as most of our systems are used mainly internally but this one is also used by clients of the courier company to book deliveries. Each delivery requires printing of confirmation documents, internal routing documents, then signed and stamped documents are uploaded back to the system as a proof of delivery for clients. It has quite a number of other (technically) interesting features like integration with google maps to geo-tag the orders and notifying drivers in the area - I'll write about it soon.

As this courier company handles several hundreds of deliveries every day it requires quite a number of uploads/downloads of documents and images. The issue with uploads and downloads is that they will tie up rails processes for much longer than usual requests. The problem is that rails processes are quite expensive in terms of memory (around 70 - 100MB for simple operations). Do 20 concurrent uploads/downloads and all your rails processes and memory will get tied up - and quite wastefully as there is no 'dynamic' processing during upload or download. The process just sits there and waits for the file to get delivered. Luckily there's a simple way how to move this burden to Apache. This kind of upload/download processes will take only around 2MB on average and your rails processes are free to handle the dynamic requests.

To let apache handle file downloads use x_sendfile. All you need to do is install x_senfile module on apache and adjust you config a little. Here's the installation instructions for Ubuntu 8.04:

cd /usr/local/src
wget http://tn123.ath.cx/mod_xsendfile/mod_xsendfile.c
apxs2 -cia mod_xsendfile.c

Open apache config file and add the module line:

sudo vim /etc/apache2/apache2.conf

LoadModule xsendfile_module /usr/lib/apache2/modules/mod_xsendfile.so

Add the following 2 lines to your virtual host file

sudo vim /etc/apache2/sites-available/my_app

XSendfile on
XSendFileAllowAbove on

You will have to restart the apache and you should be done:

sudo /etc/init.d/apache2 restart

Now in your controller handling download add x_sendfile attribute and set it to true

 send_file("/home/test/big_file.zip",
           :filename => "big_file.zip",
           :type => "application/zip,
           :x_sendfile=> true)

The uploads are equally easy. You just need to apache modules libapreq2 and modporter.

wget http://mirror.nus.edu.sg/apache/httpd/libapreq/libapreq2-2.12.tar.gz
tar -xzf libapreq2-2.12.tar.gz
cd libapreq2-2.12
./configure --with-apache2-apxs=/usr/bin/apxs2
make
sudo make install

Now download and install modporter. Get it from the github. Either use the download link and unzip or just clone the repository

git clone git://github.com/actionrails/modporter.git
cd modporter

On Ubuntu I had to change reference to apxs in Rakefile as the in ubuntu it's apxs2.

vim Rakefile

Change the APXS line to the following:

APXS           = "apxs2"

No just run rake:

sudo rake

add the following to your apache2.conf

sudo vim /etc/apache2/apache2.conf

LoadModule apreq_module /usr/lib/apache2/modules/mod_apreq2.so
LoadModule porter_module /usr/lib/apache2/modules/mod_porter.so

To actually use it in your rails application you will need modporter plugin.

script/plugin install git://github.com/actionrails/modporter-plugin.git

Add the following to the application_controller.rb (application.rb before Rails 2.3)

class ApplicationController < ActionController::Base 
mod_porter_secret = "secret"
.
.

and just enable porter in you vhost file:

sudo vim /etc/apache2/sites-available/my_app

Porter On
PorterSharedSecret secret

And that's all now if you check the uploaded files they will be a ModPorter::UploadedFile Credit for this goes to Koz from theRailsWay.com - you can find more information here.

This second installment about installation of the new Apache 2.2.4 and it's configuration for production use with Ruby on Rails and Java. This installment discusses installation of Mongrel and it builds on installation of apache.

Mongrel Installation

Install mongrel and its supporting software. Make sure when prompted you select the most recent, non mswin32 option.

sudo gem install daemons gem_plugin mongrel mongrel_cluster --include-dependencies

Set up mongrel cluster

When deploying a mongrel cluster, none of the mongrel servers will be listening on a privileged port. This means that you don't have to run mongrel as root. I usually use just my normal non-root user to run the mongrel cluster. It also helps to avoid problems with file permissions.

For the purpose of this example we will just use a freshly minted rails app. You can adjust these instructions to work with your app, wherever you may have placed it. I usually put all the applications in /home/peter/rails/. So lets go set up our app and test that mongrel will serve it:

mkdir -p /home/peter/rails/mediacom/releases/
cd /home/peter/rails/mediacom/releases/
rails 1
cd ../
ln -s current releases/1
cd current
mongrel_rails start

You should now be able to see your application at http://host:3000/. If you can, you are good to go. Hit CTRL+C to stop the mongrel server. At a minimum the log directory of your app has to be writable by the peter user:

sudo chown -R peter:peter /home/peter/rails/mediacom/

With mongrel working and our webapp directory prepared we can proceed with the mongrel_cluster configuration step:

sudo mongrel_rails cluster::configure -e production \
   -p 8000 -N 3 -c /home/peter/rails/mediacom/current -a 127.0.0.1 \
   --user peter --group peter

This will write a configuration file in config/mongrel_cluster.yml. We have setup to run our cluster in production mode as the user mongrel and will start 3 mongrel servers listening on ports 8000, 8001, and 8002. Now, lets do a quick test of what we have setup so far:

sudo mongrel_rails cluster::start

Checking our host on ports 8000, 8001, and 8002 we should now be able to see our test application. We can stop all of those mongrels with

sudo mongrel_rails cluster::stop.

On Boot Initialization Setup

At this point, mongrel and mongrel_cluster are setup and working with our sample webapp. Ultimately, we want this cluster to start on boot. Fortunately, mongrel_cluster comes with an init script that we can just drop into place. All we need to do is put the configuration files in /etc/mongrel_cluster and take care of a few system tasks:

sudo mkdir /etc/mongrel_cluster
sudo ln -s /home/peter/rails/mediacom/current/config/mongrel_cluster.yml \
   /etc/mongrel_cluster/testapp.yml

The following step needs to be done only once and ofcourse you mongrel_cluster version may vary:

sudo cp \
   /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-0.2.1/resources/mongrel_cluster \
   /etc/init.d/
sudo chmod +x /etc/init.d/mongrel_cluster

Now we have a typical System V init script that will launch our mongrel cluster. Actually, this script will launch any cluster that has a configuration file in /etc/mongrel_cluster/. So when we run /etc/init.d/mongrel_cluster start it will start all clusters. Likewise for stop and restart. To install the script use:

sudo /usr/sbin/update-rc.d -f mongrel_cluster defaults

Change the shebang line (the first line) of the mongrel_cluster_ctl to:

#!/usr/local/bin ruby

Apache configuration

Probably the most difficult part. As the apache was compiled from source the configuration files are located quite nonstandardly under /usr/local/apache/conf. As I use the apache for hosting multiple applications (I really wonder how many mongrel instances can such slice host sustain :-) I create a separate folder for configuration files for each of them. This example uses mediacom application and will be accessible from virtual domain mediacom.nextlogic.net. Of course when setting up another application instead of mediacom in files and folder names use the appropriate name :-)

1) Create folder for configuration files

mkdir /usr/local/apache/conf/mediacom.conf
cd /usr/local/apache/conf/mediacom.conf

2) Create file mediacom.common

  ServerName mediacom.nextlogic.net
 DocumentRoot /home/peter/rails/mediacom/current/public

 
   Options FollowSymLinks
   AllowOverride None
   Order allow,deny
   Allow from all
 

 RewriteEngine On

 # Uncomment for rewrite debugging
 #RewriteLog logs/myapp_rewrite_log
 #RewriteLogLevel 9

 # Check for maintenance file and redirect all requests
 #  ( this is for use with Capistrano's disable_web task )
 RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f
 RewriteCond %{SCRIPT_FILENAME} !maintenance.html
 RewriteRule ^.*$ /system/maintenance.html [L]

 # Rewrite index to check for static
 RewriteRule ^/$ /index.html [QSA]

 # Rewrite to check for Rails cached page
 RewriteRule ^([^.]+)$ $1.html [QSA]

 # Redirect all non-static requests to cluster
 RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f
 RewriteRule ^/(.*)$ balancer://mongrel_cluster_mediacom%{REQUEST_URI} [P,QSA,L]

 # Deflate
 AddOutputFilterByType DEFLATE text/html text/plain text/css
 # ... text/xml application/xml application/xhtml+xml text/javascript
 BrowserMatch ^Mozilla/4 gzip-only-text/html
 BrowserMatch ^Mozilla/4.0[678] no-gzip
 BrowserMatch \bMSIE !no-gzip !gzip-only-text/html

 # Uncomment for deflate debugging
 #DeflateFilterNote Input input_info
 #DeflateFilterNote Output output_info
 #DeflateFilterNote Ratio ratio_info
 #LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate
 #CustomLog logs/myapp_deflate_log deflate

3) Create mediacom.conf

    Include conf/mediacom.conf/mediacom.common

 # This is required to convince Rails (via mod_proxy_balancer) that we're
 # actually using HTTPS.
 RequestHeader set X_FORWARDED_PROTO 'https'

   ErrorLog logs/mediacom_errors_log
   CustomLog logs/mediacom_log combined
 

4) Create mediacom.proxy_cluster.conf

    BalancerMember http://127.0.0.1:8000
   BalancerMember http://127.0.0.1:8001
   BalancerMember http://127.0.0.1:8002
 

Note; Of course the port numbers will differ (based on whatever you configured when setting up mongrel cluster.

5) Create mediacom.proxy_frontend.conf

Listen 8100

 
   SetHandler balancer-manager
   Deny from all
   Allow from peter.dyndns.org
 

Note
Of course the port number will have to be unique on the server. Allow access from your current location (or whatever the location you will need the access from)

6) Inform httpd.conf about your configuration files At the following to the very bottom of /usr/local/apache/conf/httpd.conf

Include conf/mediacom.conf/*

7) Restart apache and you should be good to go. Of course, you will have to make sure that the virtual host subdomain actually points to the server (i.e. that mediacom.nextlogic.net will actually point to your server - for testing just make an entry in /etc/hosts)

/usr/local/apache/bin/apachectl stop
/usr/local/apache/bin/apachectl start

8) Go to http://mediacom.nextlogic.net and you should be able to see your app :-)