February 10, 2014

My Base Rails Setup, 2013 Edition

Filed under: ResearchAndDevelopment — Ryan Wilcox @ 10:32 am

In 2010 ago I wrote My Base Rails Setup.

I looked at it again and it looks pretty dated. The Rails community changes a lot in just 3 years.

Here are the tools I run into with frequency on Rails projects, and some notes where that has changed from the 2010 tools of choice.

Over the last 3 years I’ve done full time, then part time, consulting, mostly in Rails. The advantage here is that I see a lot of Rails apps written by other people. Occasionally I also get to start a Rails app from scratch.

So, my choices tend to be relatively conservative: tools that are either so common that I pick them because “everyone” uses them, or tools that introducing some potentially odd tool to the Rails app is negated by the clear win it has over the “competition” (be that existing gems or roll-your-own.

I’m very thankful that I’ve gotten to see a lot of Rails apps, of all sizes, in my nearly 5 years doing Ruby on Rails.

Reviewing Old Picks

Looking back at my tools from 2010:

  • will_paginate: I mostly see and use Katamari these days
  • formtastic: I’m still a big Formtastic fan, but I mostly am brought into already existing projects, and the court of public opinion has chosen simple_form.
  • DataMapper: in the dozens and dozens of Rails projects I’ve been on I’ve seen DataMapper once.
  • sentient_user There’s still no substitute, and I’m surprised this doesn’t get more love than it does.
  • show_for Not used enough in my opinion.
  • annotate_models: The community “solution” to “what columns does this table have in it?” appears to be, “open up schema.rb in another window”. So that’s what I do.
  • inherited_resources: Responders in Rails 3 cut down some of the cruft that I would use inherited resources for. I always assume Responders will be a good 80% solution and keep my eyes out for when to refactor the action into old style respond\_to blocks.
  • Data Migrations: There’s still no substitute – db/seeds.rb sucks for serious data migration tasks. There is a gem that provides similar abilities to the Rails plugin I linked to a few years ago: active_data_migrations. Sadly another place where worse (seeds.rb) seems to be better.
  • shoulda: RSpec won the war.
  • Timecop Still the best.
  • Macinist Factory Girl won. Thankfully FactoryGirl has gotten better syntax over the years (or I’ve gotten accustomed to it).
  • Mailtrap still maybe the best. An interesting up and comer is Mailcatcher, but I’m not 100% sold.
  • rr: Rspec’s mocking framework seems to have won the war here too.
  • jQuery’s timeago plugin Still the best.
  • BlueprintCSS: Now-a-days the winner is Twitter Bootstrap. I like the bootstrap-sass gem with the twitter-bootstrap-markup-rails gem abstracting away common constructs (alerts, tabs, etc).
  • utility_belt: All Hail Pry

So, we have better tools for a lot of those. The new picks are certainly on my “base rails tools” list.

New Base Gems/tools

Here are the new gems/tools I’ve either seen in common use, or always apply when I come into projects:

  • foreman every Rails app pulls in at least 1, if not 3 or 4, additional services. Redis, memcached, or solr are the usual suspects here, but it varies. Foreman lets me launch all those services with a simple command, stopping them all when I need to.
  • pry is the top debugger for Ruby. Especially when coupled with the pry-stack_explorer to explore the callstack and plymouth to open up pry when a test fails.
  • A Rails VM setup with Puppet. Every project I’m on I use this to create a VM and set up packages required for the system to operate.
  • cancan. While it needs some love, cancan is (sadly) still the authorization solution solution most used on Rails projects I see. I wrote a blog entry on organizing complex abilities. Cancan goes firmly in the “solutions in common use in apps I see”, not in the “tools I like using”.
  • rack-pjax. Useful if you’re using PJAX. If you need to optimize for speed you could write a method that sets or unsets the layout depending on if the request is PJAX or not… but rack-pjax is “drop in and you’re done”. I’m pretty sold on using PJAX for returning partial page content
  • rabl define your JSON/BSON/XML structure in a view file. jbuilder is the default in Rails 4, doing the same kind of thing, but I haven’t used it.

Right time, right place tools

Sometimes tools aren’t everyday tools. Sometimes the perfect tool, used in just the right place, is a godsend. Here is a list of tools like that, tools I’ll apply when the opportunity presents itself, although it only occasionally does:

  • pessimize writes gem versions to your Gemfile, so you can be very liberal with gem versions in development but then very conservative when time comes for production.
  • versioncake If I’m intentionally writing an API I want to version that API. Version Cake loads up the proper RABL or Jbuilder file for the specified version. (Yes, yes, see also Designing Hypermedia APIs).
  • SASS variables and the asset pipeline an approach when we need SASS global variables, working around limitations of the asset pipeline.
  • my ‘s’ helper. If I know I’m going to be on a project for a long while, and I join HTML sets together twice, I’ll bring this snippet in.

Conclusion

I’m excited to see what the next 3 years bring in Rails tools changes, and to see how this list stacks up then!

December 28, 2013

Rails Project Setup Best Practices

Filed under: ResearchAndDevelopment — Ryan Wilcox @ 1:26 am

As a long time Rails consultant, every new Rails project I come on to I go through the same dance:

  1. Does this project have a useful README.markdown? Maybe with setup instructions?
  2. No? Just Rail’s default README.markdown? Shucks.
  3. Does this project have a database.yml file?
  4. No? Great. Does this project have a sample database.yml file I can copy and get sane defaults for this project?
  5. Does this file have a .ruby-gemset file?
  6. Yes? Great. Does this .ruby-gemset have sane defaults, or is the gemset named project or something non-obvious?
  7. Is there a redis or solr config file I need to edit?
  8. Do I need to set up machine specific variables or settings anywhere? (For example, in .env, config/settings.yml, or config/secrets.yml, or even just in environments/development.rb?).
  9. No? Ok, great, does the app assume I’m running it on OS X with packages installed via Homebrew? (Because I’m usually not. And if I am running your project on bare metal, I prefer Macports.)
  10. Is there a Procfile?
  11. Yes? Great. Does that Procfile work for development, or is it a production-only Procfile?
  12. No Procfile? What services do I need to run to use the app? Redis? Solr? Some kind of worker queue mechanism?
  13. How do I run all the tests?
  14. rake db:setup
  15. rake spec
  16. Did something fail because I missed a config setting or some service isn’t running? If true, fix and GOTO 15.
  17. Awesome, it worked.
  18. Are there Cucumber or Selenium tests?
  19. Run those, somehow.
  20. Fire up Rails server
  21. When I visit the development version of the site, is there a special account I log in as? Or do I register a user account then browse through the code figuring out how to make myself an admin or registered user or what, then do that via rails console?

You could split these questions up into configuration questions and runtime questions. This blog entry will show best practices I try to install on (almost) every Rails project I touch.

Runtime

Runtime is the easiest, so I’ll tackle it first.

In my mind this is mostly solved by Foreman and a good Procfile, or set of Procfiles.

Setup with Procfiles and Foreman

A Procfile will launch all the services your app needs to run. Maybe you need Solr, Redis, the Rails server, and MongoDB up: you can set up a Procfile to launch and quit those services all together when you’re done.

Heroku uses Procfiles to make sure everything’s up for your app. Heroku’s usually my default, “pre-launch to mid-traction point” hosting choice because of its easy scaling and 2 minute setup process.

Heroku also provides tons of addons, adding features to your app. Sometimes these features are bug reporting or analytics, and sometimes the Heroku addons provide additional services. Two addons that do this are Redis 2 Go, and ElasticSearch from Bonsai.io.

If an app uses Redis, is deployed to Heroku, and uses the Redis 2 Go addon, then the app doesn’t need to have Redis in its Procfile.

However, when I’m developing the app I need these services running locally.

Foreman takes care of this, reading a Procfile (a default Procfile or one I specify) and firing up all of the services just like Heroku does. Don’t Repeat Yourself in action.

When I’m setting up a project that’s getting deployed to Heroku I create two Procfiles: one Procfile and one Procfile.development.sample. (I add Procfile.development to the .gitignore file in Git).

The Procfile.development.sample is important for two reasons:

  1. It lists all the services I’ll need to be running as a developer
  2. It can be used as is, or if a developer has say Mongo already running via a startup script, but the Procfile.development.sample tries to launch it again, they can copy the file, rename it to Procfile.development, and remove the line about starting up Mongo.

When I’m not deploying to Heroku I’ll still create a Procfile.development.sample for ease of starting up servers.

Running all the tests

Testing is big in the Rails world, and there’s a lot of ways to test Rails apps. RSpec with maybe Cucumber is usually what I see, but sometimes there’s a Selenium suite or Steak or something.

When I’m setting up a Rails project I write a quick Rake task to run all the test suites. For RSpec + Cucumber it looks something like this:

namespace :test do

  desc "Run both RSpec test and Cucumber tests"
  task "all" => ["spec", "cucumber"]
end

As a developer on the project – especially a new developer – I just want to type in one command and know I’ve tested all the app there is to test.

Configuration

When I’m setting up a project I create sample files for each configuration file that might be modified by a developer. So, files with names like:

  • config/database.sample.yml
  • ruby-gemset.sample
  • config/redis.sample.yml
  • .env.sample
  • config/secrets.sample.yml

But this still doesn’t solve our song and dance from the beginning of the blog entry: there’s still a lot to configure, even if I have sample files to copy and rename!

Like any good geek, I’ve replaced this frustration with a small shell script (template). Each project is different, and so each bin/install.sh will look a little different, but here’s a recent one I made for a non-trivial project:

#!/bin/bash

# If you want to go fancier, see some prompts in
# <http://stackoverflow.com/questions/226703/how-do-i-prompt-for-input-in-a-linux-shell-script>

if [ ! -e Procfile.development  ]
then
    cp Procfile.development.sample Procfile.development

    echo "Do you wish to edit Procfile.development?"
    select yn in "Yes" "No"; do
    case $yn in
        Yes ) $EDITOR Procfile.development; break;;
        No ) break;;
    esac
    done
fi

if [ ! -e config/database.yml  ]
then
    cp config/database.yml.example config/database.yml
    echo "See the default database.yml?"
    select yn in "Yes" "No"; do
    case $yn in
        Yes ) cat config/database.yml.example; break;;
        No ) break;;
    esac
    done

    echo "Do you wish to edit this database.yml?"
    select yn in "Yes" "No"; do
    case $yn in
        Yes ) $EDITOR config/database.yml; break;;
        No ) break;;
    esac
    done
fi

if [ ! -e config/redis.yml  ]
then
    cp config/redis.yml.example config/redis.yml
    echo "Do you wish to edit redis.yml?"
    select yn in "Yes" "No"; do
    case $yn in
        Yes ) $EDITOR config/redis.yml; break;;
        No ) break;;
    esac
    done
fi

if [ ! -e .ruby-gemset ]
then
    echo "Do you wish to create a .ruby-gemset file and edit it?"
    select yn in "Yes" "No"; do
    case $yn in
        Yes ) cp .ruby-gemset.copy .ruby-gemset; $EDITOR .ruby-gemset; break;;
        No ) break;;
    esac
    done
fi

if [ ! -e .env ]
then
    cp .env.sample .env
    echo "Do you wish to edit .env?"
    select yn in "Yes" "No"; do
    case $yn in
        Yes ) $EDITOR .env; break;;
        No ) break;;
    esac
    done
fi

It’s not the prettiest example of a shell script ever, but it’s easy and fast to modify and should run in all shells (I avoided fancy zsh tricks, even though zsh is my primary shell).

Run this and it will guide you through all the files you need to copy, asking you if you want to edit the config file when it’s in place. For opinionated files, like .ruby-gemset, the script will ask what you want to do.

Each of my sample files contain sane default values, which should work for the developer, but they don’t have to.

Thoughtbot has some initial thoughts on project setup too (they call it bin/setup), but they take a slightly different approach (and automatically set up different things). You could use there shell script along with mine if you wished.

My Ultimate New-To-This-Project Developer Experience

Since we’re talking about developer automation and project setup, I’d like to share my own dream experience:

  1. checkout code from Git repo
  2. “Oh, look, a Vagrantfile”
  3. $ vagrant up
  4. (15 minute coffee break while Vagrant boots up box and provisions it all for me, including Ruby setup)
  5. (During 15 minute coffee break, glance through the project’s README.markdown, see mention of running bin/install.sh)
  6. $ vagrant ssh
  7. $ cd $PROJECT
  8. $ bin/install.sh
  9. (Answers questions in install.sh and gets settings tweaked for this VM)
  10. $ rake db:setup
  11. $ foreman start -f Procfile.development
  12. $ rake test:all in a new tab. All tests pass.

Low barriers to entry, very automated project setup – help me get it set up right the first time. Help me be more productive faster.

You notice I called rake db:setup which creates a new database, loads the schema, and loads db/seeds.rb. Replace this step with “run migrations from 0” and “load in initial bootstrap data” if you wish. I’m usually in the “migrate from 0” camp, but usually find myself in a minority.

Anyway, If you compare the top list with this list you’ll see that the steps followed are very different. The first set of steps is hesitant: does this thing exist? Do I need to do X? The second set of setups is confident: The machine set this up for me and so hopefully everything is right.

In Summary

Here’s the best practices to take away from this blog entry:

  1. Consider creating a Vagrant setup for your project, including provisioning.
  2. Documentation in the README.markdown with basic “how to setup this project” instructions.
  3. Sample config files with values that are opinionated, but since they’re copied into place, easily changable.
  4. A bin/install.sh script like mine, or bin/setup script, like Thoughtbot’s.
  5. A Procfile just for developers
  6. A way to run all the tests, easily
  7. Load just enough sample data on a developer’s machine to allow them to get to major sections of your app without having to learn how to use it on day 1.

The easier it is for a developer to get up to speed on your project, the faster they can start getting real work done!

October 28, 2012

Modern Cocoa Concurrency / Asynchronous Processing Patterns

Filed under: ResearchAndDevelopment — Ryan Wilcox @ 9:35 pm

Intro

In the last several years Apple has been pushing us Cocoa developers away from thread based concurrency. Thread based concurrency tends to be bug-ridden, ceremony filled, and just unpleasant code.

There have also been some changes in the wider world of development that have changed how we as developers thing about concurrency.

Today there are six different patterns to choose from: NSOperationQueue, callbacks, futures, subscribers, GCD, and making your async APIs synchronous anyway.

This is meant to me a world-wind tour/survey of the various techniques Cocoa developers use to solve the thorny problems of asynchronicity and/or concurrency.

NSOperationQueue

A really nice Cocoa class for doing The Right Thing when given N tasks to perform. I like it with blocks.

It’s a simple API which means it’s great for quick things, but it also lacks power. But sometimes you don’t need power!

You can also limit the number of operations running at a time. For example to limit the number of concurrent downloads.

Callbacks

Imagine some code that looks like this:

[myObject makeANetworkRequestWithCompletionHandler: ^(id result, NSError* err) {

  NSLog(@"I am done");
}];

A lot of network APIs/Frameworks for Cocoa look this this (cough, coughFacebookcough, cough).

Callbacks are fine when you can decouple time from your program. “Whenever I get this data that’s fine, it’s just background data anyway”. Except imagine if it’s a list of documents stored on some server somewhere – parts of your UI might want that information to display data in a table or a count.

Yes, in some cases you can use interface feedback to tell the user to wait… Certainly a more responsive UI is better. But if you’re waiting for data from the network in order to parse and make another request on the network, you end up with callback/block soup.

Futures

Getting an object now that I can check and get my result “later” is appealing to me.

The most promising is DeferredKit – it returns you a defered object which you can easily add or chain callbacks too.

RESTKit uses the future pattern and allows you to assign callback blocks to properties on the object you are creating. These blocks are then called as part of the RESTKit object callback chain-of-events. Example:

record.onDidLoadResponse = ^(RKResponse* response) {
  // ...
};

Mike Ash also has a Friday Q&A about Futures

Reactive (Subscribers) model

Github has a ReactiveCocoa framework that does a bunch of crazy allowing you to subscribe to event completions and event changes.

I don’t yet completely understand the framework or mindset behind this. However, when my projects require a more comprehensive, and unified approach to async or concurrency – and I have a week to spare to properly grok this framework – I’ll be looking into ReactiveCocoa.

Grand Central Dispatch

Grand Central Dispatch is Apple’s approach at C level concurrency. There’s a lot in GCD, and I usually prefer using NSOperationQueue or something higher in the stack, but sometimes you really want a low level tool.

Like today. I needed to make 25 HTTP requests to a server, parse them, and let some other part of the app know when these requests had been parsed.

Grand Central Dispatch has two tools I tried:

  1. dispatch_async(dispatch_get_main_queue(), ^{...});
  2. dispatch_group_async
  3. dispatch_semaphore

dispatch_async

When given a task (the code in a block) it will run it at some point in the future. Coupled with dispatch_get_main_queue you’ll be able to run things on the main loop at some point in the future.

Wait, why would you want to do that? Answer? cough, coughFacebookcough, cough.

dispatch_group_async

You can create a group of dispatched tasks using dispatch_group_create(), dispatch_group_async(). You can even wait for all the tasks in the queue to complete with dispatch_group_wait().

dispatch_semaphore

Apple also provides a cheap, reference counting semaphore. dispatch_semaphore_create, dispatch_semaphore_signal and dispatch_semafore_wait are the important methods here.

You can even Construct your own dispatch_sync out of these primitives

dispatch_group_wait(), dispatch_semaphore and the current thread

the *_wait methods have one disadvantage: they are blocking. So, use case here is to do something like:

dispatch_group_t group = dispatch_group_create();

dispatch_group_async(group, 
    dispatch_queue_create("MINE", DISPATCH_QUEUE_SERIAL),
    ^ {
      NSLog(@"hello world");
    }
);

dispatch_group_wait(group, DISPATCH_TIME_FOREVER);
NSLog(@"done");    

Notice that dispatch_group_async above creates a new dispatch queue. This means that “hello world” will be printed out in some other thread. The current thread will halt, waiting for your dispatch to be complete, then it will print out “done”.

Now, this behavior is bad if you’re already on the context you are dispatching to. For example, imagine a function that is called on the main thread, like a button outlet.

- (IBOutlet) handleButton: (id) sender {
  dispatch_group_t group = dispatch_group_create();

  dispatch_group_async(group, dispatch_get_main_queue(), ^{

    NSLog(@"Hello world");
  });

  dispatch_group_wait(group, DISPATCH_TIME_FOREVER);
  NSLog(@"done");
}

This will actually cause a deadlock – you’ve told the main thread to sit there, waiting, until something finishes, but you’ve also scheduled something to happen on the main thread. Nothing will happen.

Apple’s documentation isn’t exactly explicit about this, and I had to prove this to myself with some experimentation and some stack overflow reading. However, all is not lost.

Synchronous APIs in an Async World

My main reason for doing this is interacting with Facebook. Facebook’s network APIs are cranky when they’re not on the main thread.

I also really want to avoid the ceremony that comes with asynchronous callback handling. Setting up something to process the data whenever it shows up, some kind of wait indicator for the users, and potentially confused users when they look at an unexpectedly blank table (because we haven’t gotten the data yet).

So, for very limited but very “save myself a lot of typing and re-architecting of this whole chunk of code” situations I actually want to stop the flow of code at the callsite. I don’t want to block a thread waiting for results (especially since I may need to execute GCD dispatches on this thread)… but I do want to defer execution of the next lines of code until the dispatch is complete.

I ended up implementing ASYNC_WAIT_ON_SAME_THREAD for exactly this purpose.

Implementing ASYNC_WAIT_ON_SAME_THREAD

In order to wait for an operation to complete we’ll want to avoid using GDC semaphores or waiting mechanisms. I eventually came up with this implementation of ASYNC_WAIT_ON_SAME_THREAD

void ASYNC_WAIT_SAME_THREAD(dispatch_queue_t queue, dispatch_block_t users_block) {

    __block BOOL allDone = NO;

    dispatch_async(queue, ^{
        users_block();

        allDone = YES;
    });

    while( !allDone ){
        // Allow the run loop to do some processing of the stream
        [[NSRunLoop currentRunLoop] runUntilDate:[NSDate dateWithTimeIntervalSinceNow:0.5]];
    }
}

Yes, it polls for a result. It’s inelegant, but the best solution I could come up with given the constraints.

ASYNC_WAIT_ON_SAME_THREAD disadvantages

Although things are happening in the main thread (your dispatches are running) the UI might feel very unresponsive to the user if you have many small dispatches lined up.

In fact, if you have an operation that takes 1 second, the user will see that lag. They press a button, your code uses ASYNC_WAIT and the app just sits there for 1 second. So ASYNC_WAIT is not a panacea, but in certain cases the code advantages outway the UI disadvantages.

Consider use of ASYNC_WAIT a code smell. It’s not that use of it is bad, but it represents a potentially dangerous area of code.

Then again, code smells are sometimes OK. IN the C++ world we have reinterpret_cast which tells the compiler, “Trust me, I know that this bit of data is actually this thing”. Use of a few reinterpret_casts to get yourself out of hairy situations is fine, but not every nail should be screwed with that particular socket wrench.

Conclusion

I hope this survey of the concurrency / asynchronous design patterns in Cocoa has been useful. I look forward to more and more work being done in this area by the community.

July 14, 2012

Rails 3.2 attr_accessible, RailsAdmin, and “accessible by admins”

Filed under: ResearchAndDevelopment — Ryan Wilcox @ 10:59 pm

First of all, my blog is now back up! Instead of self hosting my WordPress blog, now the fine people at ZippyKid host it. These guys are awesome: taking a mess of an import and making it Just Work. A+++ WOULD DO AGAIN

Now, back to real news…

The Problem: Security in Rails: Say hello to the secure boss (different from the old boss)

Because of some high profile Rails sites being hacked, Rails 3.2 changed the default Rails model behavior to “only let users (and developers) update attributes in this list”. This list varies by model.

A refresher into the hack

Rails has this clever feature where you can say, “update this record with the form data passed in”. A semi-clever hacker could use this ability to change fields that the Rails developer didn’t intend to be changed (“shove this value into the POST params, even though there’s no field named that on the HTML for this form”)

What Rails 3.2 did about it

Before Rails 3.2 You used to have models that look like:

class User < ActiveRecord::Base

end

In Rails 3.2, now you have models that look like:


class User < ActiveRecord::Base
# Setup accessible (or protected) attributes for your model

attr_accessible :name, :email, :password, :password_confirmation
end

The attr_accessible block says, “hey, that ‘update this record with form data passed in’ feature? That’s only allowed to touch these fields”.

In this example the name, email, password, and password_confirmation, but is NOT allowed to edit anything else. Perhaps you store access keys, or middle name in the User model. You have to explicitly change these values, and not use the ‘update this record with form data’ shortcut.

Enough background: You said something about RailsAdmin?

RailsAdmin is a clever piece of software that automates creating an admin interface for your Rails site. You have simple access to create, read update or delete records in your site

… but how does it play with that attr_accessible thing?

Glad you asked

Normally it works very well. You can see all the fields in your model, and if the attribute is not attr_accessible, then RailsAdmin will display the value as read-only.

Read-only you say? But I have values I want admins to be able to edit, but I only want admins (not everyone) to edit them. How do I?

Returning to the access_keys example, you want admins to be able to edit this value in RailsAdmin. You don’t want to make that attr_accessible because then anyone can edit that setting (introducing a security hole).

The solution: attr_accessible + as (a user)

attr_accessible has an oft-forgot as parameter. This allows you say, “this is allowed, only if I I’m doing this as a ______________ user

Using this feature you can declare models like


class User < ActiveRecord::Base
# Setup accessible (or protected) attributes for your model

attr_accessible :name, :email, :password,
:password_confirmation, :as => [:default, :admin]

attr_accessible :access_key, as: :admin
end

access_key will only be changable when you’re doing something as the admin role, and the other attributes will be enable both for the default role and the admin role

Configuring RailsAdmin to use the admin role

In your config/initializers/rails_admin.rb file, add the following line in the RailsAdmin.config do |config| block


config.attr_accessible_role { :admin }

Conclusion

And that’s all there is to it: use as: :admin, and configure RailsAdmin to post things “as an admin”. Good to go!

December 21, 2011

Using Fabric to import your Django models

Filed under: General Information,ResearchAndDevelopment — Ryan Wilcox @ 9:49 pm

A client wants me to write an import task for their Django app. They already use Fabric to deploy their site, so I figured that writing this script as a Fabfile would work out well.

The script requires me to import classes from their Django app. Specifically, I’m doing queries against their domain models, and adding things to the database, and I’d like to reuse the Django ORM classes already defined.

I finally got it working, but it was non-obvious. Here’s how I did it.

Setting up the module import path

In this project all the fabfiles go in a folder named “fab” in the Django project’s directory. So, I need to tell Python to look outside the fab folder for what it is trying to import.

But that’s not enough – I also need to import the Django project by name, so I need to go one more folder out (to the parent folder of the Django project).

Actually importing the Django models

the django.project nonsense is to set up an environmental variable (pointing to the settings.py file) – Django requires this variable to be set, and will error without it.

Next we import the Django project, and start using entities from an app inside it.

Easy, Huh?

October 19, 2011

Announcing: delegate_presenter: the simplest Presenter Gem that could possibly work

Filed under: General Information,ResearchAndDevelopment — Ryan Wilcox @ 8:19 pm

Lately there’s been a lot of buzz in the Rails community about using the Presenter Pattern to organize common view related code outside of the model.

Think of Presenters like Helpers: The Next Generation

I used Presenters on one project with great success. Today I was about to add presenters to a second project, when I said:

Self, you could copy and paste all this code from Project X to Project Y, or you could extract it into a gem

So, that’s what I did: introducing the delegate_presenter gem

Read more documentation on the Github page

August 5, 2011

Capistrano, system wide RVM and creating gemsets as part of deploy:setup

Filed under: General Information,ResearchAndDevelopment — Ryan Wilcox @ 11:29 am

Introduction to the Problem

Capistrano is the standard way to deploy Rails apps. Yesterday I was using Capistrano to deploy to a machine where I had installed RVM (Ruby Version Manager) at the system level.

I manually set up Ruby 1.8.7 and Ruby 1.9.2, because I need to run two applications on that machine (one a Ruby 1.8.7 app and one a Ruby 1.9.2 app). Using RVM for production deploys is great for this.

My cap deploy:setup task, however, complained that Ruby 1.9.2 wasn’t installed on the machine.

That’s funny, because I did install it, I thought. After banging my head up against the problem for a few hours, I finally posted the question to Stackoverflow.com: (Capistrano deploying to system wide RVM not seeing installed Rubies)

I got my answer: the message about the Ruby not being installed was misleading, it actually meant that the gemset wasn’t installed. Which it wasn’t (I was planning on doing that as part of the cap deploy:setup task.

Creating gemsets in your deploy:setup step

Ideally I want cap delpoy:setup to take care of eveything for me: installing some rubies, creating the appropriate gemsets, you name it. Because automated deployments mean everything should be automated (amirite?).

But then I get errors like this when I’m trying to create the gemset I want to use!

It’s non-obvious how to do this – and in fact the obvious way will not work!

Background

You see, require 'rvm/capistrano' hooks into the low levels of Capistrano’s run function, meaning everything happens in the context of the ruby+gemset that you declared in your Capfile. (Technically rvm/capistrano uses a user shell called rvm-shell, instead of bash or sh. This shell knows enough to properly set your paths to Ruby etc etc.

Normally this is awesome – that means that Capistrano knows about your Gemset, and installs gems there etc etc. Capistrano’s run command just does the right thing.

However, there are two cases where you want things to happen outside of rvm-shell:

  1. Installing the Ruby
  2. Creating the Gemset

If you try to do these things using run, Capistrano will give an error about Ruby not being installed, like it gave me. Even if RVM is trying to say, “I don’t see that gemset”, the error message will be about a missing Ruby.

The obvious solution, and why it doesn’t work (as a conversation)

The obvious thing you might try in your Capfile is this command:

run "rvm install 1.9.2"

Except, as I explained above, that won’t work. Here’s what’s going on, as a conversation.

You, to Capistrano: Run this command for me

Capistrano, to remote machine: Hey, I want to log into this machine, using the rvm-shell command, using Ruby 1.9.2 and gemset MY_APP. When I’m logged in please execute rvm install 1.9.2

Remote machine, to Capistrano: Could not log you in, an error happened when firing up rvm-shell. I could not find the ruby/gemset you wanted, so I can’t set the Ruby paths appropriately. I’m giving up and stopping because I can’t possibly do whatever that command was that you wanted me to execute

Capistrano, to you: I couldn’t install that Ruby you wanted me to install because I can’t activate that Ruby you want me to use for the gemset you want me to use – I don’t think that Ruby is installed!

You: Le sigh.

The solution: avoid rvm-shell for Ruby installation AND gemset creation

You might think that you need to avoid rvm-shell just for the installation of your Ruby. In fact, you need to avoid rvm-shell for both the installation of Ruby and the creation of your gemset!

How to avoid rvm-shell

Define this method in your Capfile.


def disable_rvm_shell(&block)
old_shell = self[:default_shell]
self[:default_shell] = nil
yield
self[:default_shell] = old_shell
end

Now, in the context of that block, run will execute commands by using sh/bash as a shell, instead of rvm-shell.

In your Capfile, install Ruby and your gemsets by:


disable_rvm_shell do
run "rvm install 1.9.2"
run "rvm use 1.9.2@MY_APP --create"
end

This installation process must come before any other command in your deploy:setup chain.

Conclusion

And that’s that – I really hope this helps someone out there!

May 22, 2011

Rails, REST, and .js (TL;DR: We’re doing it wrong)

Filed under: ResearchAndDevelopment — Ryan Wilcox @ 10:32 pm

In my last blog article I talked about Returning HTML content from AJAX requests. However, I’ve been struggling with a question since then:

I asserted (implicitly) that the following (and canonical!) pattern for doing AJAX replacement is wrong. I felt this strongly, but why is it wrong?

First, the current pattern

Normal Rails pattern for HTML replacement is to make an AJAX call to the .js format, in which we return something like the following back to the browser:


<%# show.js.erb %>
var str = '<div id="<%= dom_id my_object -%>" >
Name : <%= my_object.first_name -%>
</div>';
$("#destination_div").replace(str);

To this I say Ugh.

Why Ugh? Why is it wrong to return this in the .js format?

Why? The DOM transform operation above isn’t the representation of the object

Let’s review what Rails actions do in the respond_to block

  • format.html returns a HTML representation of the object (for display on screen) — aka: it renders the view
  • format.json returns a JSON representation of the object
  • format.xml returns an XML representation of the object

These are all pretty normal in the Rails world. The last two take the object and serialize it (using .to_json or render :xml => object. Straight from rails generate scaffold

You might imagine (from DHH’s RailsConf keynote introducing Rails 2) that other formats existed, with mostly the same result:

  • format.icl might return the iCal representation of the object
  • format.csv might return a CSV representation of the object

But in the canonical Rails show.js.erb file at the beginning of this blog post, the .js is not returning the Javascript representation of the object.

Can I do this with your object?


$.ajax(
...
success: function (data, textStatus, xhr) {
var obj = eval(data);
alert(obj.first_name); /* attribute, like JSON */
alert(obj.api_function() ); /* and here's a function call! */

Answer: No, because your .js “representation” is some DOM injection of an unparsable HTML string.

A pattern for returning an actual Javascript Object

If you return an actual Javascript representation of the object, you have the same data attributes available to you on the JS side as you do on the Ruby side.

For example, if your .js format rendered a .js.erb template like this:

Yes, this smacks on the If you’re using to_json, you’re doing it wrong article by the Gomiso guys. I absolutely agree with their core idea of maybe you should be using views to construct your json objects!

Notice here, we are returning a Javascript object, with data and methods that encapsulate functionality … which is exactly what objects are.

I also know that you use a Ruby object in different contexts than you do a Javascript object. In Ruby I want to perform my business logic – computing priority of due dates, in the case of my Todo object here. In Javascript I care about different operations – perhaps mostly related to AJAX and DOM manipulation.

For example, I’ve always struggled with getting my Rails routes into my unobtrusive Javascript. In Ruby (or .erb code) I use routes and path helpers like I should… and on the Javascript side I hardcode the routes. Heaven help me if I ever have to change the routes – I’ve solved only half the problem!

Since I implemented a routes method in my JS object above, the route will always be correct – because I’m using the normal Rails routes helpers to generate it!

In Rails 3.0.x, it’s non-obvious to keep the Javascript representation of an object in show.js.erb. This location is certainly it’s not where I would look for the source code to the object! Rails 3.1, I believe, will save this day. Imagine a Rails 3.1 app that uses the assert pipeline to store the Javascript representation in the following hierarchy:


app/
assets/
javascripts/
models/
todo.js.erb

Much better!

To Review

  1. The normal Rails pattern of returning DOM manipulation functions as the Javascript representation of the object is not RESTful
  2. … and inconsistent, because every other format (.json, .xml) returns the representation of the object
  3. We should return actual Javascript objects – data and methods to act on that data

So, how do we do AJAX replacement then?

If we can no longer use the .js format to return a DOM manipulation function, how do we return chunks of HTML to the browser (for a standard “AJAX request and replace” type functionality)?

Answer: Request a partial HTML representation of your object

My Returning HTML content from AJAX requests previous article

covers the how very well, but there are some other advantages:

  • We’re returning proper representations for proper formats
  • The Javascript that originally makes this AJAX call knows more about the structure of the DOM – and where it wants the result – than a show.js.erb that lives on the other conceptional side of the app

May 12, 2011

Returning HTML content from AJAX requests – a pattern for Rails 3

Filed under: General Information,ResearchAndDevelopment — Ryan Wilcox @ 5:25 pm

The problem: semantic formats for returned HTML for AJAX

In a previous Rails 2 project, we decided that Rails apps return 3+ kinds of content:

  • A complete HTML page, for user viewing
  • JSON (for JS/web API viewing)
  • A partial HTML page, for jQuery DOM swapping/

However, it was hard to know when to return a full HTML page, and when to :layout => false

So we invented a semantic format

Huh?

Rails actions typically go something like this

# from todo.rb

def show
@object = Todo.find(params[:id])
respond_to do |format|
format.html { render "show" } # being explicit here, render not really required
format.json {@object.to_json}
end
end

So, if the format.html gets called, how do you know if you should show the whole page, or just some partial page update (for example, to update the Todo item on the screen for some AJAX effect or another?

The solution

The solution was – for the Rails 2 project – to use a custom MIME type to identify when we wanted a snippet of HTML. Our AJAX requests looked like:


$.ajax({
url: "/todos/" + id,
beforeSend: function(xhr){ xhr.setRequestHeader("Accept", "text/html-partial") },
success: function(){....}

That Accept parameter set up the MIME type, and we added it to our config/initializers/mime_types.rb file and we were ready to rock

It’s not so simple in Rails 3

Setting up the MIME type

Rails, until about October 2010, didn’t respect MIME types that well. I believe that it would take this MIME type and return text/html

Rails 3 takes the MIME type, and returns it. So even if you got the rest of the example up, your browser would complain because it doesn’t know what to do with a text/html-partial MIME type.

So, instead of a MIME type, it’s best to use a format parameter, for Rails 3

Set up your config/initializers/mime_types.rb file to contain:


Mime::Type.register_alias "text/html", :partial

This is essentially just an alias for another MIME type (text/html).

So, we’re going to use a format

Because we don’t want the user to see “Unknown MIME type, save or open?” in their browsers, we are going to cheat a little bit and specify our format via an extension.

Our Javascript code should now look like


$.ajax({
url: "/todos/" + id + ".partial",
success: function(){....}

Your actions, knowing about full page HTML, and AJAX HTML

Now you need to set up your actions

# from todo.rb

def show
@object = Todo.find(params[:id])
respond_to do |format|
format.html { render "show" } # being explicit here, render not really required
format.partial { render "show", :layout => false }
format.json {@object.to_json}
end
end

Except we have a problem. If you actually try this out, you will get a Rails Missing Template error

Rails 3 takes the MIME type into consideration when constructing the template path/name to render.

format.html { render "show" }

— Rails looks for todo/show.html

format.partial { render "show", :layout => false }

— Rails looks for todo/show.partial

Now, ideally we want to share as much HTML as possible, so we have a problem

In my use case today, I was actually rendering a Rails partial – a partial that was also used in non Ajax situations. So renaming my file to be _something.partial.erb just wasn’t going to cut it

Actually, this behavior does introduce an interesting side effect: being able to further isolate snippets returned for AJAX vs full page reload requests, if the situation demands it.

But my situation demanded sharing. I’m sure separating things out will work for some requests, and it’s a handy thing to have… but most of the time I want Rails to read from the blah.html.erb file.

Setting up a Rails 3 ActionView::FileSystemResolver

So I decided to go monkey patching Rails, to provide the support I wanted.

Before I banged my head on my desk too hard, I found the following article on using FileSystemResolver for some other things:

Implementing A Rails 3 View Resolver

My resolver is very similar to that one:

Conclusion

  • We have a way to separate full page HTML requests from AJAX requests that only want a specific section of a page
  • We use the less magical extensions to provide formatting, instead of MIME types
  • We can provide AJAX specific templates if we want (blah.partial.erb
  • We fall back to .html if nothing specific exists, because we really want to share HTML code between a full page redraw and an AJAX redraw

April 28, 2011

CI Server for Rails Projects Poll Results

Filed under: ResearchAndDevelopment — Ryan Wilcox @ 6:56 pm

Today I did a poll, using PollEverywhere to get a feeling of the Ruby on Rails communities favorite CI server

PollEverywhere limits responses to 30 for the free version, and 50 for one pay level up. I spend the little bit of money to get the additional responses

And here they are:

Jenkins wins by a wide margin (50% of the participants).

Looks like I will be spending time with Continuous Cooking: a Vagrant project for Hudson, although Bamboo looks appealing.

« Previous PageNext Page »