April 27, 2011

Setting up Passenger Standalone, RVM and launchd on OS X

Filed under: General Information,ResearchAndDevelopment — Ryan Wilcox @ 9:22 am

Installing Radiant with Passenger (Standalone), RVM and Launchd on OS X

Introduction – why Passenger Standalone

I like to deploy all my basic sites on the same host. For example, deploying a Radiant site along with a few small Rails sites on the same box. This keeps things easy for me, and no sense devoting a lot of resource if I know the site will be fairly small.

However, this poses some deployment challenges.

I want to use Passenger, like all the cool kids are doing. Just set up some Apache things and I’m ready to go.

I also always use RVM. It is so amazing being able to keep project gems separate from each other. This is especially important when you have multiple projects on the same machine.

This sounds like a dream team: use Apache/Passenger to host multiple Rails apps, with separate gemsets, end of story.

Except, the real world doesn’t work like that.

First, your rvm and Passenger must share a common Ruby. If they can’t, you must use Apache Reverse Proxy cleverness with other servers (like Passenger Standalone).

The easy way to avoid this mess is to make sure they match. We’re all using REE anyway, amiright?

Actually, usually not. If you’re on a shared host, or host multiple Ruby sites from one box, you don’t want to make everyone on that site use REE… even if it’s the best.

For example, what happens in a year when you’re working on a Rails 3 site in Ruby 1.9… you just forced yourself into yesterday’s version.

It is tempting to set up Passenger to use one Ruby (REE) and point Passenger to your gemsets. I really suggest you go to the extra work and set up this system instead

Setting up Apache as a Proxy

So, we want to use Apache as a front-end, and use reverse proxies to run all these sites on their own Passenger standalone servers.

In your Apache configuration (in /etc/apache2/sites, in a file that increments the leading count):

<VirtualHost: *:80>
  ServerName cms.example.com:80

  ProxyPass / localhost:3000/
  ProxyPassReverse / localhost:3000/
</VirtualHost: *:80>

Note: that trailing slash is important

If you have your Rails/Radiant site on the server, at this point you should be able to fire up Passenger (passenger start in your Radiant folder), visit your server name, and have it redirected.

This Passenger instance will run as long as you are connected, or as long as the machine stays up (whichever ends first). But that won’t do for a production webserver…

Making Your Passenger Get Up, And Stay Up (OS X)

Figuring out what command to give launchd

This goal is not as obvious as it might seem. We want a Passenger Standalone instance to be up all the time, including after machine restarts. So, how do we best do this?

In the past, for demonstration purposes, I’ve SSHed into a server, fired up GNU Screen, ran passenger, then detected the screen. GNU Screen will keep the SSH socket open (from the server’s perspective, anyway), and thus Passenger. This solution works, but it’s a hack.

Ideally I want to keep Passenger up without this hack, or other similar “poll based” monitoring methods.

Launchd to the rescue.

Here’s how to get that up:

  1. Create a rvm wrapper:

    $ rvm wrapper RUBY@GEMLIST radiant_site

    Will create ~/.rvm/bin/radiant_site_ruby

    So if you’re running ree with your gemset name “radiant_site”

    $ rvm wrapper ree@radiant_site radiant_site

    Which creates a Ruby named radiant_site_ruby.

  2. Find out where your passenger is:

    which passenger

Your command to start up your Rails site is an amalgamation of these three things and the path to your Rails root:

/Users/aias/.rvm/bin/radiant_site_ruby /Users/aias/.rvm/gems/ruby-1.8.7-p299@radiant_site/bin/passenger start PATH TO SITE -e production

Try this command on the command line before pouring it into a launchd script.

Pouring it into Launchd

Launchd has two interesting features which we’ll use here:

  1. KeepAlive: Launchd will make sure your progress stays up, restarting if neccisary
  2. UserName: run the command as a particular user

My Launchd file is here:

Save your version of this file to /Library/LaunchDaemons/ – you want this to launch even if your user is not logged in.

Load the file with: sudo launchctl load /Library/LaunchDaemons/FILE NAME

Launchd Debugging Notes

  1. launchd scripts must be owned by root:wheel. (Lingon does not save files this way)
  2. Use StandardErrorPath and StandardOutPath to debug these. Note that the files must exist first, and be writable by root. These are amazingly useful files to have to see what your job is going.
  3. Seriously, you do not want the -d flag for Passenger. (It makes the whole thing not work – launchd has rules about what you can and can’t do in KeepAlive/LaunchDaemon mode)

But what about Passenger Standalone on Linux?

I haven’t tried either of these out, but they look like they would fit the bill.

April 9, 2011

Installing Passenger Standalone — and solutions

Filed under: General Information,ResearchAndDevelopment — Ryan Wilcox @ 12:10 am

Intro: What is Passenger, and why I want to use it

I seriously want to love Passenger Standalone, I really do. Certain client projects of mine could use the increased speed, and I’d love to use Passenger Standalone for production deploys too (more on that in a future article).

But it seems like I’m running into every problem under the sun installing this thing. Here are my solutions to these problems.

Problems & Solutions

I ran passenger start and it downloads and compiles things, then fails with sh: line 0: cd: ext/libev/: No such file or directory

Thanks to this issue on the Passenger Google Code site I found out my CDPATH needed to contain “.”.


$ export CDPATH=.:$CDPATH and redo passenger start

I ran passenger start, it downloads and compiled everything, now I get: the following error during startup:
Unable to start the Phusion Passenger logging agent because its executable
(....passenger/standalone/3.0.6...-macosx-10.6/support/agents/PassengerLoggingAgent) doesn't exist

Thanks to this Google Group conversation the best way to deal with this issue is to:

  1. $ passenger package-runtime. This will create a passenger-standalone folder in your current directory, with a folder (3.0.6…macosx-10.6). This folder will have a support.tar.gz and a nginx- file.
  2. Note where it says the file is missing from (standalone/3.0.6…-macosx-10.6, in the case above), and go ~/.passenger/(that directory). Trash the support folder that’s there, and extract the support.tar.gz file (from step 1) in this directory


That took a lot more debugging than it should have (or, to be fair, than it has in the past). But I eventually got Passenger Standalone running. Now that yack is shaved, I hope this blog entry helps others with similar issues

February 20, 2011

Introducing Scrum Status

Filed under: General Information,ResearchAndDevelopment — Ryan Wilcox @ 3:07 pm

One of my projects has a problem: while we all try to be in the office for scrum, sometimes we can’t all be. Maybe one person is in the car and can’t talk, maybe another person is on the subway… maybe another person has the day off (but the team needs to know their status).

So I started working on a website to take care of these things, and used this team as guinea pigs. Today I announced this site to the world: Scrum Status

Now we have a centralized place to see the entire team’s statuses, easily. Likewise, it’s easy to make an entry for the day while on the go – scrum status works pretty well on iPhone (even though it hasn’t been specifically tweaked for it).

In the spirit of startup launches I’m launching early, to get feedback. Early testing with my team has ironed out the critical bugs, but now I’m announcing this to the world.

Scrum Status runs on Google App Engine, so I don’t have to worry about scaling issues: 1 person could hit the site, or 5,000 – Google App Engine should be able to handle it.

Interested in technical details? The entire project is open source, and the README has a lot of information about the tech involved: Scrum Status on Bitbucket.

July 5, 2010

Django app/sample

Filed under: General Information,ResearchAndDevelopment — Ryan Wilcox @ 11:56 pm

So I decided to switch things around and write a sample application in Django. I started a simple project management / bug tracker / projects-have-many-todo-items app.

Hopefully in the near future I can spend some more time on this example app, for example actually finishing up more than just the project level screens :)

Find it at (in a Mercurial repository on bitbucket:) wd_project_dashboard.

May 22, 2010

Turbogearn 2.1: installation, for the impatient, using virtualenv and pip

Filed under: ResearchAndDevelopment,Turbogears — Ryan Wilcox @ 9:33 am

I’m playing with Turbogears 2 more and more, and as part of this process I wanted to improve and codify my process for getting a new TG2 app up and running.

I also really like virtualenv, and am learning to love pip (mostly for the ease of requirements.txt).

I also know that the Installation for the Impatient make me, well, impatient. Ughh, 5 commands.

So I wanted to see if I could make this slightly better with pip, and I kind of did:

$ pip install -E tg2env -i http://www.turbogears.org/2.1/downloads/current/index/tg.devtools
$ source tg2env/bin/activate
$ paster quickstart my_wonderful_app

Two issues I found when trying to install TG2:

  • You want tg.devtools, not Turbogears2: tg.devtools includes the tools to make new projects, where as the Turbogears package just includes the code to run a Turbogears app.
  • Ultimately leaving out the -i part should work, but I ran into problems that way (had version conflicts with repoze.who), so ended up having to specify the package source with -i. It could be that these are solved by the time you read this.

or, to install the latest beta (at this point, 2.1b2)

$ pip install -E tg2env\
-e 'hg+http://bitbucket.org/turbogears/tg-dev/@2.1b2#egg=TurboGears2'\
-e 'hg+http://bitbucket.org/turbogears/tgdevtools-dev/@2.1b2#egg=tg.devtools'

Ok, it’s still a lot of commands, but it can be done.

Hope this helps enable great Turbogears apps!

April 19, 2010

Git Workflow

Filed under: ResearchAndDevelopment — Ryan Wilcox @ 11:00 pm

So I’m on a Rails project, we use Git for source control, and we have a team policy of good test coverage (validated by a CI server – specifically CruiseControl – running tests and alerting the team on a failed build).

Day to day we do work on feature branches which we later merge to the master branch when that sub-system (ticket) is complete.

My typical workflow is to start work on a ticket, make a number of intermediate commits, then merge back to master. However, my coworker was noting today that the intermediate commits mean that I have commits that could technically break the build – which is not an issue normally, but if we were to use git bisect to find a bug the failing tests could get in the way.

This is contrary to centralized version control systems, where you know every commit will pass tests, because you don’t want to break the build. But in Git the CI server only checks the latest commit.

I said that I saw two, potentially equally legitimate, git workflows:

  • The commit is the unit of work, by which you want to commit as often as possible to get those benefits, and you publish those intermediate commits (useful for history and code review!)
  • The Subsystem is the unit of work: commit as often as possible, but when you’re ready to publish git rebase -i so you don’t pollute history. (Of course, this only works if it’s just a local branch – doing a rebase -i to a remote branch is a bad idea).

I’d love to hear two cents from the various git gurus around the internets…

January 27, 2010

Why your syslog events don’t show up under Console.app

Filed under: ResearchAndDevelopment — Ryan Wilcox @ 12:21 am

Tonight I was debugging something for a client. Our Rails app uses SyslogLogger to log to syslog (instead of the files in log/*.log, like Rails is set up by default. We think there are some wins using syslog).

I was trying to debug why some logging statements would show up, and some didn’t. In fact, logger.info and logger.debug weren’t working for us. These correspond, by default, to syslog log levels info and debug (oddly enough).

This blog entry will explain how what settings you need to tweak to have Console.app display things at a higher log level. To see log level debug in Console.app. (Yes, this is the part of the article where I input search terms I used in trying to find this information, so some other sod can find it too.)

Syslog is one of those unix components that Apple pulled the guts out of and left a small shell that’s compatible with the rest of Unix. Domain of the Bored has an excellent series of articles on ASL. There’s also some really good documentation from an (I’m guessing) Apple engineer on the darwin-dev mailing list on ASL.

So great. ASL controls syslog. If ASL says something isn’t stored, syslog doesn’t ever see it – even if it was a log level syslog would normally care about.

We need to add an entry to asl.conf. man asl.conf gives us syntax, etc. All we want to do is capture all log levels for our app, rails_app_mine.

edit /etc/asl.conf with your favorite editor. Add the following line:

? [<= Level debug] [= Sender rails_app_mine] store
Sender is the syslog process. We set this in our Rails app by creating the SyslogLogger object like so:

config.logger = RAILS_DEFAULT_LOGGER = SyslogLogger.new("rails_app_mine")

Now syslog will see your lines, and you can configure syslog using normal syslog mechanisms (like sending your rails_app_mine messages to a syslog server, or syslog-ng, let's say).

Alternatively, in /etc/asl.conf you could modify the line

? [<= Level notice] store

to instead read

? [<= Level debug] store

but that would log ALL debug statements, including ones that we really don't care about. And seriously, there's too much spew in Console.app already.

Make this change and reload syslog by having launchd reload it (no, -HUP-ing it doesn't work):

sudo launchctl unload /System/Library/LaunchDaemons/com.apple.syslogd.plist; \
sleep 1;\
sudo launchctl load /System/Library/LaunchDaemons/com.apple.syslogd.plist

And you're done.

February 8, 2009

Installation Hints for ReviewBoard on Debian PPC (Sarge)

Filed under: General Information,ResearchAndDevelopment — Ryan Wilcox @ 12:12 am

So, after trying to install ReviewBoard on a virtual machine (and failing), I decided to install it on my real server, a machine running Debian PPC (sarge).

Eventually I got it up and running, but here are my notes:

I decided to run it off fastcgi (asking apt-get to install lighttpd pulled in tons of other software I didn’t want… and I didn’t want to screw around with setting up mod_python – although in retrospect maybe that was silly). You need to apt-get libapache2-mod-fastcgi. EXCEPT this doesn’t work by default: you need to set your /etc/apt/sources.list file to download “non-free” software too. So you lines will look something like deb http://debian.lcs.mit.edu/debian/ testing main non-free. What makes this confusing is that there is libapache2-mod-fastcgid, which is a different implementation of the fastcgi protocol… but not what you want at all. Anyway there’s more yack about sources.list here.

Also, if you’re having Apache2 listen on an oddball port for requests to your ReviewBoard, make sure your firewall lets those ports through :).

Make sure to uncomment the line in apache2.conf that Includes the files in sites-enabled!

I thought Everburning: Fighting Review Board was going to be helpful, but turns out it mostly works out of the box (without the stuff he does).

You might have to fiddle with settings_local.py’s FORCE_SCRIPT_NAME, and what the rewrite rule in the Review Board generated Apache2 config file says, to make them agree. This will be pretty obvious when the front review board page is ugly (because the CSS didn’t load), OR you’ll get an error about URLconf defined in djblets.util.rooturl, Django tried these. URL patterns...

I think that’s it. I’m pretty sure I easy_install ReviewBoard to get it installed, but I think there might be an apt-get package for it too.

So it wasn’t a bad experience, just one I was hoping to avoid with a virtual machine for it.. oh well I guess. Hope this helps someone else.

November 2, 2008

Rails migrations and model validations

Filed under: ResearchAndDevelopment — Ryan Wilcox @ 8:42 pm

So Rails has a feature called “migrations”. The idea here is that your database will change over time and you need a way to change this incrementally as the project changes and new requirements come up. So essentially your database moves through time.

Rails also has a feature where you can define validations for a field at the model level (aka: application level, vs database level, validations): make sure it is defined, is in a certain range, etc. These are declarative statements, meaning that they are set up when the class is defined and ran when an instance is created.

In certain situations, however, there comes an issue with this. For example, if you’re using Cruise Control to do integration tests, you might find that you’re migrating down to the first migration and then migrating all the way back up to current. Which is fine and a good practice, except for one thing: your model validations don’t move through time with your migrations. This is no problem… unless you’re trying to create or modify records using these model objects in your migrations. Then you have a paradox: the model tries to validate a field, but was given no value because that field doesn’t exist yet.

Lets say in migration 2 we are creating vendors and populating some default ones:

class CreateVendors < ActiveRecord::Migration def self.up create_table :vendors do |t| t.string :name t.string :email end a = Vendor.new(:name => "Fred Flinstone", :email => "fred@example.com")

b = Vendor.new(:name => "Barney Ruble", :email => "barney@example.com")


def self.down
drop_table :vendors

But we”ve been developing the app for some time, and in migration 20 we add a rating field to the Vendor table, as well as the following validation to the model:

validates_presence_of :rating, :on => :create, :message => "can't be blank"

Now when we migrate down to 0 and migrate back up, we get an error in migration 2:

$ rake db:migrate
(in .... )
== 2 CreateVendors: migrating =======================================
rake aborted!
Validation failed: Rating can't be blank

(See full trace by running task with --trace)

Oops! We’re validating something that doesn’t exist yet!!

Rails saves the day again, because each validation_ declaration has an :if parameter, which should be a Symbol or Procedure to run. If this symbol or procedure method returns true, then the validation is run.

So our problem can be solved by changing the model validation to read:

validates_presence_of :rating, :on => :create, :message => "can't be blank",
:if => Proc.new { |record| record.respond_to? :rating }

responds_to asks the Rails model: “Do you have a method named ‘rating’?”. Because Rails magically creates methods for every column an entity has in the database, Rails will either have that method (because it has been defined in the database), or not (because no such column exists). So we only run the validation if we have the column, avoiding the mess with trying to validate data that doesn’t make sense (yet).

March 21, 2008

Batching Requests to Amazon Web Services via pyaws

Filed under: ResearchAndDevelopment — Ryan Wilcox @ 4:49 pm

PyAWS (originally py-Amazon by Mark Pilgrim), is a neat, simple, Python API for getting requests from Amazon Web Services.

But it doesn’t let you do any batching: allowed by the Amazon Web Services API to reduce requests.

THe other day I wrote py_aws batch: which uses PyAWS as a springboard for batch requests. Which I think is really neat, so I’m sharing it with the community at large.

So here it is: pyaws_batch.py (Now Updated with right link!)

« Previous PageNext Page »