Passenger for Nginx

I didn’t consider using Passenger when it was first released because it only worked with Apache and I prefer nginx. But now Passenger works with nginx! This is great news. Nginx and Passenger make for simple high-performance Rack and Rails web application deployment. I still like thin. But the simplicity of deploying with Passenger makes it a better solution for my needs. I’ve created a second webfaction-rails-environment installation script that implements Passenger. It’s available at GitHub.

The one potential downside of nginx+Passenger on WebFaction is that the reported RSS is greater than with monit+nginx+thin. In actuality, nginx+Passenger uses far less physical memory (especially with Ruby Enterprise Edition) than any other solution. The reported RSS is higher because of multiple counting. Separate application instances share memory that doesn’t change. If memory usage is metered strictly on the basis of RSS, then you’re not getting your money’s worth of memory using Passenger. “Private Dirty RSS” is a more accurate measure of memory usage. The README of webfaction-rails-environment shows you how to determine it. The difference can be substantial.

Rails Deployment Options

There are many ways to deploy a Rails application. Sorting them all out and finding the right solution for your situation can be confusing. In this post I list and briefly describe popular choices.

Ruby Virtual Machine

Most Ruby implementations can run Rails. Some, however, cannot. The ability to run Rails is a major achievement for alternative Ruby VMs. For detailed comparisons check out Antonio Cangiano’s blog. He does the Ruby shootout.

  • MRI
    • 1.8 standard
    • Ruby 1.8.6 is the recommended version to use with Rails
  • YARV
    • 1.9 standard
    • Significantly faster than MRI
    • Rails is not yet fully compatible with Ruby 1.9
    • Many gems are not compatible with Ruby 1.9
  • JRuby
    • Java implementation of Ruby
    • Runs Rails
  • Rubinius
    • “Ruby in Ruby”
    • Runs Rails
  • Ruby Enterprise Edition
    • From the creators of Passenger
    • Fork of MRI
    • 33% less memory consumption on average when used with Passenger
  • MagLev
    • Commercial
    • Pending release
    • Lots of promise in terms of performance and features, but won’t run Rails for some time

For critical production applications, there are really only two choices to consider. If you are using Passenger, use Ruby Enterprise Edition. Otherwise, use the standard Ruby VM. The other implementations are progressing rapidly and developers are working hard to make them production ready.

Server Configuration

This is an active area. Notable configurations include:

  • nginx + mongrel | thin | ebb | fuzed (yaws)
    • nginx is a powerful lightweight frontend server/reverse proxy/load balancer that can withstand a real pounding
    • mongrel is the veteran backend web server for Ruby on Rails
    • thin is an evented backend server that’s faster than mongrel and supports unix socket connections
    • ebb is an evented backend server written in C that’s faster than thin and also supports unix socket connections, but it uses more memory than thin while idling
    • fuzed allows Rails to be served up by yaws, a server written in Erlang that provides an unparalleled degree of concurrency
  • Apache + Passenger (mod_rails) + Ruby Enterprise Edition
    • New and exciting deployment option for Apache
    • Easy to setup
    • Deploying an app can be as simple is uploading your app
    • mod_rails and Ruby Enterprise Edition, both developed by Phusion, together provide a 33% lower memory footprint (for Rails) on average
    • Integrated monitoring and load balancing; monitors Rails processes and starts/kills them as necessary based on demand
  • LiteSpeed
    • Commercial
    • Relatively easy to setup
    • Better performance than most other solutions
    • Despite its qualities, not a popular choice

I only mention LiteSpeed because of its performance. Few people actually use it for serious Rails deployments. I omitted lighttpd from the list because nginx has stolen the show. Ancient solutions like fastcgi were also omitted.

I use nginx + thin. I have not transitioned to ebb because of higher memory consumption (at least at idle). I included the fuzed project in my list because I find yaws and Erlang fascinating. Yaws puts Apache to shame when it comes to concurrency. I’m not sure how polished the fuzed project is, but it could be a contender. It’s also good to see cooperation between Ruby and Erlang. Mongrel, thin, and ebb are all good options. It all depends on your needs and preferences.

I have not tried out Passenger. It is being touted as a breakthrough solution because of how simple it makes deployment. My first impression is that it is more for deployment novices and conventional shared hosting environments. With WebFaction, you have the freedom and ability to build your own app stack. I’ve made this a breeze with a shake and bake shell script. Nginx is a better frontend server than Apache in its ability to serve static pages and with regard to memory usage. Unfortunately, Passenger only works with Apache.

Load Balancing

Load balancing allows your applications to scale horizontally.

  • Hardware
    • For very large applications
    • Most advanced
    • Expensive
  • HAProxy
    • For large applications
    • Very advanced
    • Difficult to setup
  • nginx-upstream-fair
    • Third party module for nginx
    • Adds fair load balancing to nginx (replaces standard round-robin load balancing)
    • Very simple to setup
    • Small to large applications

I use the nginx-upstream-fair module for load balancing. Written by Grzegorz Nosek, the module works very well and is so easy to setup that there is no reason not to do so.

Monitoring

To make sure that your processes are behaving, you need a process monitor.

I use monit. I haven’t tried the god gem, but I’ve heard good things.

Ruby on Rails Stack on WebFaction

I’ve created a shell script to build a complete Ruby on Rails stack (application environment) on WebFaction. Although written for WebFaction users, the script is fairly generic, aside from a few minor details. All you have to do is edit a few variable assignments (install path, rails app name, and service ports) at the beginning of the script and execute. In less than 20 minutes, your rails app will be up and running with nginx reverse proxying (and fair load balancing) to a pair of thin servers and with monit keeping watch.

In case you’re unfamiliar with thin, it’s the likely successor to mongrel. It uses mongrel’s excellent http parser, provides various overall enhancements, and offers a number of features mongrel lacks. I specifically chose to use thin on WebFaction because of its support for unix socket listeners. For more technical information, see the comments in the script and the accompanying README.markdown file.

What you get:

  • Ruby
  • RubyGems
  • Gems: rails, merb, mongrel, mongrel_cluster, thin, capistrano, termios, ferret, acts_as_ferret, god, sqlite3-ruby, mysql, and typo
  • Git
  • Nginx (with nginx-upstream-fair third party module)
  • Monit
  • Startup scripts and working default configuration files for nginx and monit

UPDATE: New script with Passenger on nginx!

I will try to keep this script reasonably up to date at GitHub.

Rails and Git

My notes from Scott Chacon’s screencast. You should see it yourself. It is definitely worthwhile.

Instantiate a git repository with a newly minted rails app.

rails railsapp && cd railsapp
git init-db
touch .gitignore

Add the following lines to .gitignore:

config/database.yml
tmp/*
log/*

Add all the files to the repository and commit all.

git add .
git status # To check the status of the working copy.
git commit -a -m "Initial commit"
git log # To see the log for the repository.

Create a remote git repository from the one just created.

cd ..
git clone --bare railsapp/.git railsapp.git
scp -r railsapp.git username@remote-machine:/home/username/git-repos
cd railsapp
git remote add gitserver username@remote-machine:/home/username/git-repos/railsapp.git

Replace gitserver with a name you want to reference the remote machine by. Make sure that the path to the git binaries is defined in ~/.bashrc and not ~/.bash_profile because remote commands load the former and not the latter. Information about the remote is added in the git config.

git push gitserver # Push the code in local repository to gitserver

On the remote machine, in railsapp.git:

export GIT_DIR=.
git log

Branching and merging in git.

git branch -a # Show all git branches (including the remote machine).
git branch # Show all local git branches.
git checkout -b experimental # Create and switch to new branch "experimental".
git checkout master # Switch back to master branch.
git checkout experimental # Switch back to experimental branch.

To merge experimental into master:

git checkout master # Switch to the master branch as the working copy.
git pull . experimental # Does a fetch and then a merge; you could just merge.
git add filenameinconflict # Fix files in conflict and then do a git add.
git commit -a # After merging do a commit.

After merging the experimental branch into master, we’re finished with it so we can delete its identifier. The branch’s change history will still be there but the branch name is gone. To do so:

git branch -d experimental
git branch # See that the branch name is deleted.
gitk --all& # Visualize the change history using a TK GUI.

Database

database.yml

development:
  adapter: sqlite3
  database: db/development.sqlite3

test:
  adapter: sqlite3
  database: db/test.sqlite3

production:
  adapter: mysql
  encoding: utf8
  host: localhost
  database: production_db_name
  username: mysql_username
  password: mysql_password

Mongrel Cluster

mongrel_cluster.yml

---
user: user
group: user
environment: production
address: 127.0.0.1
port: 3000
servers: 2
cwd: /home/user/webapps/railsapp/current
log_file: log/mongrel.log
pid_file: tmp/pids/mongrel.pid

Capistrano

In railsapp, execute:

capify .

Capfile

load 'deploy' if respond_to?(:namespace) # cap2 differentiator
Dir['vendor/plugins/*/recipes/*.rb'].each { |plugin| load(plugin) }
load 'config/deploy'
load 'config/mongrel' # mongrel overrides

deploy.rb

set :application, "railsapp"
set :repository,  "user@webxx.webfaction.com:/home/user/git-repos/railsapp.git"
set :domain, "webxx.webfaction.com"
set :deploy_to, "/home/user/webapps/#{application}"
set :mongrel_conf, "#{current_path}/config/mongrel_cluster.yml"
set :scm, :git
set :deploy_via, :remote_cache
ssh_options[:paranoid] = false
set :user, "user"
set :runner, "user"
set :use_sudo, false
role :app, domain
role :web, domain
role :db,  domain, :primary => true

# If the production web server doesn't have access to your git server,
# add the following two lines.
set :deploy_via, :copy # instead of :remote_cache
set :git_shallow_clone, 1 # optional, but makes things faster

# moves over server config files after deploying the code
task :update_config, :roles => [:app] do
  run "cp -Rf #{shared_path}/config/* #{release_path}/config/"
end
after 'deploy:update_code', :update_config

mongrel.rb

# mongrel-based overrides of the default tasks

namespace :deploy do
  namespace :mongrel do
    [ :stop, :start, :restart ].each do |t|
      desc "#{t.to_s.capitalize} the mongrel appserver"
      task t, :roles => :app do
        #invoke_command checks the use_sudo variable to determine how to run the mongrel_rails command
        invoke_command "mongrel_rails cluster::#{t.to_s} -C #{mongrel_conf}", :via => run_method
      end
    end
  end

  desc "Custom restart task for mongrel cluster"
  task :restart, :roles => :app, :except => { :no_release => true } do
    deploy.mongrel.restart
  end

  desc "Custom start task for mongrel cluster"
  task :start, :roles => :app do
    deploy.mongrel.start
  end

  desc "Custom stop task for mongrel cluster"
  task :stop, :roles => :app do
    deploy.mongrel.stop
  end

end

mongrel_cluster with Nonconsecutive Ports

Need to run mongrel_cluster with nonconsecutive ports? No problem.

WebFaction assigns ports to its users through the control panel. The panel will not assign you consecutive ports. You can still use ports that aren’t assigned to you, but I assume that WebFaction does not want you to. Running mongrel_cluster without stretching or breaking the rules requires some effort. Unmodified, mongrel_cluster will only spawn mongrel_rails listeners on consecutive ports. Configuration is limited to specifying the first port and the number of instances. The situation with WebFaction requires a more precise way to configure ports. We will need to modify the mongrel_cluster code. All we need to do is add one line.

The file requiring modification is lib/mongrel_cluster/init.rb inside the mongrel_cluster gem directory. The easiest way to find and open this file for editing is to execute the following command:

nano `locate lib/mongrel_cluster/init.rb`

Locate the read_options method. In version 1.0.5, it should read:

def read_options
    @options = {
        "environment" => ENV['RAILS_ENV'] || "development",
        "port" => 3000,
        "pid_file" => "tmp/pids/mongrel.pid",
        "log_file" => "log/mongrel.log",
        "servers" => 2
    }
    conf = YAML.load_file(@config_file)
    @options.merge! conf if conf

    process_pid_file @options["pid_file"]
    process_log_file @options["log_file"]

    start_port = end_port = @only
    start_port ||=  @options["port"].to_i
    end_port ||=  start_port + @options["servers"] - 1
    @ports = (start_port..end_port).to_a
end

Add the following line to the end of the method:

@ports = @options["ports"] if @options["ports"] && !@only

The addition recognizes a parameter named ‘ports’ in config/mongrel_cluster.yml. Unless a single port is specified on the command line using the --only option, ‘ports’ will be respected over the ‘port’ and ‘servers’ parameters used to specify a continuous range. The ‘ports’ parameter should be accompanied by an array of integers in YAML format. An example mongrel_cluster.yml file that defines nonconsecutive ports follows:

---
cwd: /home/user/webapps/railsapp
environment: production
user: user
group: user
address: 127.0.0.1
log_file: log/mongrel.log
pid_file: tmp/pids/mongrel.pid
ports:
- 3333
- 3335
- 3359
- 3401

The one line addition does not give you the ability to define discontinuous ports on the command line. You must edit mongrel_cluster.yml to do so.