You have unicode string in your code, you deploy it on heroku, and the app crashed with an error:

invalid multibyte char (US-ASCII)

Fear not, just do this:

heroku config:add RUBYOPT=’-Ku’

And that will magically fix the problems


So you decide you want a way for a number of people to post short articles to a Web site, and maybe allow for other people to leave comments. What do you do? That’s easy: jump to the white board and start sketching out a vast array of boxes, blobs, and arrows that define a sophisticated content management system with multi-level admin roles, content versioning, threaded discussion boards, syntax highlighting, the works.


Don’t be too quick to judge. It’s easy to fall into the trap of defining the wrong requirements for a project.

Part of the reason is that building software is (or should be) fun, and often bigger projects are more fun. There is also the tendency to think about “what if”, imagining all the things that maybe one day who knows you never can tell might be needed.

People also tend to think in terms of what’s familiar, of how things have been done in the past or by others.

There are many ways to satisfy the needs described in the first paragraph. Some don’t require writing any software at all.

For the Ruby Best Practices blog, the general goals were modest. Allow a core set of people to easily add new content. Allow other people to contribute content as well, but only with the OK from someone in that core group. (BTW, there are eight of us in the RBP blog team. See here for my take on the RBP team logo)

We wanted to allow the use of Textile, and not make people use a browser for editing. Basically, turn simple flat files into a Web site with minimal fuss.

Korma takes an interesting approach to building Web sites. The whole app is ~230 lines of Ruby. Its key function is to take content from a Git repository, run it through some templating, and write out the site files.

Relying on Git for that back end is stunningly perfect. Git provides versioning, access control, and distributed contributions.

It becomes the database layer common to most blogging tools. For free. Right off the bat, no need to write any admin tools or content versioning code .

At the heart of Korma is the grit gem. As the project blurb says, “Grit gives you object oriented read/write access to Git repositories via Ruby.” Very sweet.

The korma.rb file takes two arguments. The first is required, and is the path to a git repository. The second argument is optional; it tells Korma what directory to use for the generated content, and defaults to ‘www’, relative to where you invoke the program.

The app uses grit to reach into the current contents of the repo and parse the needed files. Files have to be committed to be accessible to Korma.

There is a configuration file that describes some basic site metadata, such as the base URL, the site title, and the author list. When called, korma.rb grabs this config file, sets some Korma::Blog properties, and then writes out the static files.

An early version of Korma used Sinatra; basically, Korma was a lightweight Web app to serve the blog posts. But as simple as it was, it was overkill, since there was no real dynamic content. It made no sense to have the Web app regenerate the HTML on each request, since it changed so infrequently.

A next version replaced the Web app part with static files, making it a straightforward command-line program. This solved another problem: how to automate the regeneration of files. The answer: use Git’s post-commit hook to invoke the program.

For example:

   # File .git/hooks/post-commit   
   /usr/local/bin/ruby /home/james/data/vendor/korma/korma.rb /home/james/data/vendor/rbp-blog /home/james/data/vendor/korma/www

Early versions also used Haml for site-wide templates. Not being a fan of Haml, I added in a configurable option to use Erb. It was nice and all, but it was a feature without a requirement. No one was asking for configurable templating, so that option was dropped and Erb replaced Haml as the default.

If you are wondering why working code was removed, consider that any code you have is something you have to maintain. As bug-free and robust as you may like to think it, no code is easier to maintain than no code. Configurable templating was simply not a problem we were needed to solve, and a smaller, simpler code base is more valuable than a maybe “nice to have.”

There was some discussion about the need or value of allowing comments. In the end they were deemed good, but there was no good argument for hosting them as part of the blog site itself. That meant a 3rd-party solution (in this case, Disqus) was perfectly acceptable. Again, a goal was to have comments, not to write a commenting system (as entertaining as that may be).

Using Git allows for yet another feature for free: easy one-off contributions. In many systems, allowing anyone to contribute means creating an account, granting some sort of access, managing all the new user annoyances. Or, an existing user has to play proxy, accepting content and entering it into the system on behalf of the real author. That’s work! With git, one can clone the repo, add new content, and issue a pull request back to the master branch. Anyone with commit rights to the master can then merge it in (or politely decline). No-code features FTW.

None of the blog design requirements are written in stone, and they may change tomorrow, but by identifying the real needs, addressing what was deemed essential, offloading what we could, and skipping the feature bling, we have a system that is easy to understand, easy to maintain, and easy to change.


“this article shows how I used test-driven development tools and processes on a Greasemonkey script.” Though it also includes free ninjas.

1. Long drop downs hate humans

When I do online banking I need to select from a large list of other people’s bank accounts to which I might like to transfer money too. It is the massive drop down list that I must scroll through that I wish to raise issue with today. The problem of having to give other people money is probably a different discussion.

And take those time-zone selector drop down lists, for example, the massively long list rendered by Rails’ time_zone_select helper. Granted, I am thankful for you letting me choose my timezone in your web app. Though for those of us not living in the USA we must hunt for our closest city in the list. Dozens of locations, ordered by time zone and not the name of the city (see adjacent image). Unfortunately you can’t easily type a few letters of your current city to find it. Rather, you have to scroll. And if you live in the GMT+1000 time zone group (Eastern Australia), you have to scroll all the way to the bottom.

5. Choose from a small list

So I got to thinking I’d like a Greasemonkey (for Firefox) or GreaseKit (for Safari) script that automatically converted all ridiculously long HTML drop down lists into a sexy, autocompletion text field. You could then type in “bris” and be presented with “(GMT+1000) Brisbane”, or given the less amusing banking scenario then I could type “ATO” and get the bank account details for the Australian Tax Office.

I mean, how hard could it be?

This article is two things: an introduction to Ninja Search JS which gives a friendly ninja for every drop down field to solve the above problem. Mostly, the rest of this article shows how I used test-driven development tools and processes on a Greasemonkey script.

Introducing Ninja Search JS

Ninja Search JS banner

Click the banner to learn about and install the awesome Ninja Search JS. It includes free ninjas.

Currently it is a script for Greasemonkey (FireFox) or GreaseKit (Safari). It could be dynamically installed as necessary via a bookmarklet. I just haven’t done that yet. It could also be a FireFox extension so it didn’t have to fetch remote CSS and JS assets each time.

Ninja Search JS uses liquidmetal and jquery-flexselect projects created by Ryan McGeary.

Most importantly of all, I think, is that I wrote it all using TDD. That is, tests first. I don’t think this is an erroneous statement given the relatively ridiculous, and unimportant nature of Ninja Search JS itself.

TDD for Greasemonkey scripts

I love the simple idea of Greasemonkey scripts: run a script on a subset of all websites you visit. You can’t easily do this on desktop apps, which is why web apps are so awesome – its just HTML inside your browser, and with Greasemoney or browser extensions you can hook into that HTML, add your own DOM, remove DOM, add events etc.

But what stops me writing more of them is that once you cobble together a script, you push it out into the wild and then bug reports start coming back. Or feature requests, preferably. I’d now have a code base without any test coverage, so each new change is likely to break something else. Its also difficult to isolate bugs across different browsers, or in different environments (running Ninja Search JS in a page that used prototypejs originally failed), without a test suite.

And the best way to get yourself a test suite is to write it before you write the code itself. I believe this to be true because I know it sucks writing tests after I’ve writing the code.

I mostly focused on unit testing this script rather than integration testing. With integration testing I’d need to install the script into Greasemonkey, then display some HTML, then run the tests. I’ve no idea how’d I’d do that.

testing running

But I do know how to unit test JavaScript, and if I can get good coverage of the core libraries, then I should be able to slap the Greasemonkey specific code on top and do manual QA testing after that. The Greasemonkey specific code shouldn’t ever change much (it just loads up CSS and more JS code dynamically) so I feel ok about this approach.

For this project I used Screw.Unit for the first time (via a modified version of the blue-ridge rails plugin) and it was pretty sweet. Especially being able to run single tests or groups of tests in isolation.

Project structure

summary of project structure

All the JavaScript source – including dependent libraries such as jquery and jquery-flexselect – was put into the public folder. This is because I needed to be able to load the files into the browser without using file:// protocol (which was failing for me). So, I moved the entire project into my Sites folder, and added the project as a Passenger web app. I’m ahead of myself, but there is a reason I went with public for the JavaScript + assets folder.

In vendor/plugins, The blue-ridge rails plugin is a composite of several JavaScript libraries, including the test framework Screw.Unit, and a headless rake task to run all the tests without browser windows popping up everywhere. In my code base blue-ridge is slightly modified since my project doesn’t look like a rails app.

Our tests go in spec. In a Rails app using blue-ridge, they’d go in spec/javascripts, but since JavaScript is all we have in this project I’ve flattened the spec folder structure.

The website folder houses the github pages website (a git submodule to the gh-pages branch) and also the greasemonkey script and its runtime JavaScript, CSS, and ninja image assets.

A simple first test

For the Ninja Search JS I wanted to add the little ninja icon next to every <select> element on every page I ever visited. When the icon is clicked, it would convert the corresponding <select> element into a text field with fantastical autocompletion support.

For Screw.Unit, the first thing we need is a spec/ninja_search_spec.js file for the tests, and an HTML fixture file that will be loaded into the browser. The HTML file’s name must match to the corresponding test name, so it must be spec/fixtures/ninja_search.html.

For our first test we want the cute ninja icon to appear next to <select> drop downs.

require("../public/ninja_search.js"); // relative to spec folder

  describe("inline activation button", function(){
    it("should display NinjaSearch image button", function(){
      var button = $('a.ninja_search_activation');
      expect(button.size()).to(be_gte, 1);

The Blue Ridge textmate bundle makes it really easy to create the describe (des) and it (it) blocks, and ex expands into a useful expects(...).to(matcher, ...) snippet.

The two ellipses are values that are compared by a matcher. Matchers are available via global names such as equals, be_gte (greater than or equal) etc. See the matchers.js file for the default available matchers.

The HTML fixture file is important in that it includes the sample HTML upon which the tests are executed.

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "">
<html xmlns="">

  <title>Ninja Search | JavaScript Testing Results</title>
  <link rel="stylesheet" href="screw.css" type="text/css" charset="utf-8" />
  <script src="../../vendor/plugins/blue-ridge/lib/blue-ridge.js"></script>

    <label for="person_user_time_zone_id">Main drop down for tests</label>
    <select name="person[user][time_zone_id]" id="person_user_time_zone_id" style="display: inline;">
      <option value="Hawaii">(GMT-10:00) Hawaii</option>
      <option value="Alaska">(GMT-09:00) Alaska</option>
      <option value="Pacific Time (US & Canada)">(GMT-08:00) Pacific Time (US & Canada)</option>
      <option value="Arizona">(GMT-07:00) Arizona</option>
      <option value="Mountain Time (US & Canada)">(GMT-07:00) Mountain Time (US & Canada)</option>
      <option value="Central Time (US & Canada)">(GMT-06:00) Central Time (US & Canada)</option>
      <option value="Eastern Time (US & Canada)">(GMT-05:00) Eastern Time (US & Canada)</option>

In its header it loads the blue-ridge JavaScript library, which in turn loads Screw.Unit and ultimately our spec.js test file (based on corresponding file name), so ninja_search.html will cause a file spec/ninja_search_spec.js to be loaded.

To run our first test just load up the spec/fixtures/ninja_search.html file into your browser.

Your first test will fail. But that’s ok, that’s the point of TDD. Red, green, refactor.

Simple passing code

So now we need some code to make the test pass.

Create a file public/ninja_search.js and something like the following should work:

  $(function() {
    $('select').each(function(index) {
      var id = $(this).attr('id');

      // create the Ninja Search button, with rel attribute referencing corresponding >select id="...">
      $('> rel="' + id + '">ninja search>/a>')

Reload your test fixtures HTML file and the test should pass.

Now rinse and repeat. The final suite of tests and fixture files for Ninja Search JS are on github.

Building a Greasemonkey script

Typically Greasemonkey scripts are all-inclusive affairs. One JavaScript file, named my_script.user.js, typically does the trick.

I decided I wanted a thin Greasemonkey script that would dynamically load my ninja-search.js, and any stylesheets and dependent libraries. This would allow people to install the thin Greasemonkey script once, and I can deploy new versions of the actual code base over time without them having to re-install anything.

Ultimately in production, the stylesheets, images, and JavaScript code would be hosted on the intertubes somewhere. Though during development that would be long-winded and painful to push the code to a remote host just to run tests.

So I have three Greasemonkey scripts:

  • public/ – loads each dependent library and asset from the local file system
  • public/ninja_search.local.user.js – loads compressed library and asset from the local file system
  • public/ninja_search.user.js – loads compressed library and assets from remote server

Let’s ignore the optimisation of compressing dependent JavaScript libraries for the moment and just look at the dev.user.js and user.js files.

The two scripts differ in the target host from which they load assets and libraries. loads them from the local machine and ninja_search.user.js loads them from a remote server.

For example loads local dependencies like this:


And ninja_search.user.js loads remote dependencies like this:


In the final version of ninja_search.user.js we load a simple, conpressed library containing jquery, our code, and other dependencies, called ninja_search_complete.js.

Using Passenger to server local libraries

The problem with loading local JavaScript libraries using the file:// protocol, inferred earlier, is that it doesn’t work. So if I can’t load libraries using file:// then I must use the http:// protocol. That means I must route the requests through Apache/Ningx.

Fortunately there is a very simple solution: use Phusion Passenger which serves a “web app’s” public folder automatically. That’s why all the javascript, CSS and image assets have been placed in a folder public instead of src or lib or javascript.

On my OS X machine, I moved the repository folder into my Sites folder and wired up the folder as a Passenger web app using PassengerPane. It took 2 minutes and now I had http://ninja-search.local as a valid base URL to serve my JavaScript libraries to my Greasemonkey script.

Testing the Greasemonkey scripts

I can only have one of the three Greasemonkey scripts installed at a time, so I install the file to check that everything is basically working inside a browser on interesting, foreign sites (outside of the unit test HTML pages).

Once I’ve deployed the JavaScript files and assets to the remote server I can then install the ninja-search.user.js file (so can you) and double check that I haven’t screwed anything up.

Deploying via GitHub Pages

The normal, community place to upload and share Greasemonkey scripts is This is great for one file scripts, though if your script includes CSS and image assets, let alone additional JavaScript libraries, then I don’t think its as helpful, which is a pity.

So I decided to deploy the ninja-search-js files into the project’s own GitHub Pages site.

After creating the GitHub Pages site using Pages Generator, I then pulled down the gh-pages branch, and then linked (via submodules) that branch into my master branch as website folder.

Something like:

git checkout origin/gh-pages -b gh-pages
git checkout master
git submodule add -b gh-pages website

Now I can access the gh-pages branch from my master branch (where the code is).

Then to deploy our Greasemonkey script we just copy over all the public files into website/dist, and then commit and push the changes to the gh-pages branch.

mkdir -p website/dist
cp -R public/* website/dist/
cd website
git commit -a "latest script release"
git push origin gh-pages
cd ..

Then you wait very patiently for GitHub to deploy your latest website, which now contains your Greasemonkey script (dist/ninja-search.user.js) and all the libraries (our actual code), stylesheets and images.


Greasemonkey scripts might seem like small little chunks of code. But all code starts small and grows. At some stage you’ll wish you had some test coverage. And later you’ll hate yourself for ever having release the bloody thing in the first place.

I wrote all this up to summarise how I’d done TDD for the Ninja Search JS project, which is slightly different from how I added test cases to _why’s the octocat’s pajamas greasemonkey script when I first started hacking with unit testing Greasemonkey scripts. The next one will probably be slightly different again.

I feel good about the current project structure, I liked Screw.Unit and blue-ridge, and I’m amused by my use of GitHub Pages to deploy the application itself.

If anyone has any ideas on how this could be improved, or done radically differently, I’d love to hear them!


Last week, in a few hours, I whipped together for Flight Control, a super fun iPhone game. The site allows users to upload screenshots of their high scores. I thought I would provide a few details here as some may find it interesting.

It is a pretty straightforward and simple site, but it did need a few permissions. I wanted users to be able to update their own profile, scores and photos, but not anyone else’s. On top of that, I, as an admin, should be able to update anything on the site. I’m sure there is a better way, but this is what I did and it is working just fine.

Add admin to users

I added an admin boolean to the users table. You may or may not know this, but Active Record adds handy boolean methods for all your columns. For example, if the user model has an email column and an admin column, you can do the following.

user = # => false = '' # => true

user.admin? # => false
user.admin = true
user.admin? # => true

Simple permissions module

Next up, I created a module called permissions, that looks something like this:

module Permissions
  def changeable_by?(other_user)
    return false if other_user.nil?
    user == other_user || other_user.admin?

I put this in app/concerns/ and added that directory to the load path, but it will work just fine in lib/.

Mixin the permission module

Then in the user, score and photo models, I just include that permission module.

class Score < ActiveRecord::Base
  include Permissions

class Photo < ActiveRecord::Base
  include Permissions

class User < ActiveRecord::Base
  include Permissions

Add checks in controllers/views

Now, in the view I can check if a user has permission before showing the edit and delete links.

<%- if score.changeable_by?(current_user) -%>
  <li class="actions">
    <%= link_to 'Edit', edit_score_url(score) %>
    <%= link_to 'Delete', score, :method => :delete %>
<%- end -%>

And in the controller, I can do the same.

class ScoresController < ApplicationController
  before_filter :authorize, :only => [:edit, :update, :destroy]

    def authorize
      unless @score.changeable_by?(current_user)
        render :text => 'Unauthorized', :status => :unauthorized

Macro for model tests

I didn’t forget about testing either. I created a quick macro for shoulda like this (also uses factory girl and matchy):

class ActiveSupport::TestCase
  def self.should_have_permissions(factory)
    should "know who has permission to change it" do
      object     = Factory(factory)
      admin      = Factory(:admin)
      other_user = Factory(:user)
      object.changeable_by?(other_user).should be(false)
      object.changeable_by?(object.user).should be(true)
      object.changeable_by?(admin).should be(true)
      object.changeable_by?(nil).should be(false)

Which I can then call from my various model tests:

class ScoreTest < ActiveSupport::TestCase
  should_have_permissions :score

Looking at it now, I probably could just infer the score factory as I’m in the ScoreTest, but for whatever reason, I didn’t go that far.

A sprinkle of controller tests

I also did something like the following to test the controllers:

class ScoresControllerTest < ActionController::TestCase  
  context "A regular user" do
    setup do
      @user = Factory(:email_confirmed_user)
      sign_in_as @user

    context "on GET to :edit" do
      context "for own score" do
        setup do
          @score = Factory(:score, :user => @user)
          get :edit, :id =>

        should_respond_with :success

      context "for another user's score" do
        setup do
          @score = Factory(:score)
          get :edit, :id =>

        should_respond_with :unauthorized

  context "An admin user" do
    setup do
      @admin = Factory(:admin)
      sign_in_as @admin

    context "on GET to :edit" do
      context "for own score" do
        setup do
          @score = Factory(:score, :user => @admin)
          get :edit, :id =>

        should_respond_with :success

      context "for another user's score" do
        setup do
          @score = Factory(:score)
          get :edit, :id =>

        should_respond_with :success

Summary of Tools

I should call flightcontrolled, the thoughtbot project as I used several of their awesome tools. I used clearance for authentication, shoulda and factory girl for testing, and paperclip for file uploads. This was the first project that I used factory girl on and I really like it. Again, I didn’t get the fuss until I used it, and then I was like “Oooooh! Sweet!”.

One of the cool things about paperclip is you can pass straight up convert options to imagemagick. Flight Control is a game that is played horizontally, so I knew all screenshots would need to be rotated 270 degress. I just added the following convert options (along with strip) to the paperclip call:

has_attached_file :image, 
  :styles => {:thumb => '100>', :full => '480>'},
  :default_style => :full,
  :convert_options => {:all => '-rotate 270 -strip'}


You don’t need some fancy plugin or a lot of code to add some basic permissions into your application. A simple module can go a long way. Also, start using Thoughtbot’s projects. I’m really impressed with the developer tools they have created thus far.

Include vs Extend in Ruby

Tháng Năm 20, 2009


Now that we know the difference between an instance method and a class method, let’s cover the difference between include and extend in regards to modules. Include is for adding methods to an instance of a class and extend is for adding class methods. Let’s take a look at a small example.

module Foo
  def foo
    puts 'heyyyyoooo!'

class Bar
  include Foo
end # heyyyyoooo! # NoMethodError: undefined method ‘foo’ for Bar:Class

class Baz
  extend Foo
end # heyyyyoooo! # NoMethodError: undefined method ‘foo’ for #<Baz:0x1e708>

As you can see, include makes the foo method available to an instance of a class and extend makes the foo method available to the class itself.

Include Example

If you want to see more examples of using include to share methods among models, you can read my article on how I added simple permissions to an app. The permissions module in that article is then included in a few models thus sharing the methods in it. That is all I’ll say here, so if you want to see more check out that article.

Extend Example

I’ve also got a simple example of using extend that I’ve plucked from the Twitter gem. Basically, Twitter supports two methods for authentication—httpauth and oauth. In order to share the maximum amount of code when using these two different authentication methods, I use a lot of delegation. Basically, the Twitter::Base class takes an instance of a “client”. A client is an instance of either Twitter::HTTPAuth or Twitter::OAuth.

Anytime a request is made from the Twitter::Base object, either get or post is called on the client. The Twitter::HTTPAuth client defines the get and post methods, but the Twitter::OAuth client does not. Twitter::OAuth is just a thin wrapper around the OAuth gem and the OAuth gem actually provides get and post methods on the access token, which automatically handles passing the OAuth information around with each request.

The implementation looks something like this (full file on github):

module Twitter
  class OAuth
    extend Forwardable
    def_delegators :access_token, :get, :post

    # a bunch of code removed for clarity

Rather than define get and post, I simply delegate the get and post instance methods to the access token, which already has them defined. I do that by extending the Forwardable module onto the Twitter::OAuth class and then using the def_delegators class method that it provides. This may not be the most clear example, but it was the first that came to mind so I hope it is understandable.

A Common Idiom

Even though include is for adding instance methods, a common idiom you’ll see in Ruby is to use include to append both class and instance methods. The reason for this is that include has a self.included hook you can use to modify the class that is including a module and, to my knowledge, extend does not have a hook. It’s highly debatable, but often used so I figured I would mention it. Let’s look at an example.

module Foo
  def self.included(base)

  module ClassMethods
    def bar
      puts 'class method'

  def foo
    puts 'instance method'

class Baz
  include Foo
end # class method # instance method # NoMethodError: undefined method ‘foo’ for Baz:Class # NoMethodError: undefined method ‘bar’ for #<Baz:0x1e3d4>

There are a ton of projects that use this idiom, including Rails, DataMapper, HTTParty, and HappyMapper. For example, when you use HTTParty, you do something like this.

class FetchyMcfetcherson
  include HTTParty


When you add the include to your class, HTTParty appends class methods, such as get, post, put, delete, base_uri, default_options and format. I think this idiom is what causes a lot of confusion in the include verse extend understanding. Because you are using include it seems like the HTTParty methods would be added to an instance of the FetchyMcfetcherson class, but they are actually added to the class itself.


Hope this helps those struggling with include verse extend. Use include for instance methods and extend for class methods. Also, it is sometimes ok to use include to add both instance and class methods. Both are really handy and allow for a great amount of code reuse. They also allow you to avoid deep inheritance, and instead just modularize code and include it where needed, which is much more the ruby way.

Pacecar & automatic named scopes

Tháng Năm 18, 2009



Pacecar adds named_scope methods and other common functionality to ActiveRecord classes via database column introspection.

Pacecar automatically includes the Pacecar::Helpers module into all ActiveRecord::Base classes.

To get all Pacecar functionality, you need to “include Pacecar” in your class.

To get some subset (for example, only the state functionality), you can do something like “include Pacecar::State” to get only the module(s) you want.


Assuming a database schema…

  class CreateSchema < ActiveRecord::Migration
    def self.up
      create_table :users, :force => true do |t|
        t.boolean :admin, :default => false, :null => false
        t.datetime :approved_at
        t.datetime :rejected_at
        t.string :first_name
        t.string :last_name
        t.text :description
      create_table :posts, :force => true do |t|
        t.string :owner_type
        t.integer :owner_id
        t.string :publication_state
        t.string :post_type
      create_table :comments, :force => true do |t|
        t.integer :user_id
        t.text :description

And some basic model declarations…

  class User < ActiveRecord::Base
    include Pacecar
    has_many :posts, :as => :owner
    has_many :comments
    has_many :articles
    has_ranking :comments
    has_recent_records :comments
    has_recent_records :articles, :comments

  class Post < ActiveRecord::Base
    include Pacecar
    PUBLICATION_STATES = %w(Draft Submitted Rejected Accepted)
    TYPES = %w(Free Open Private Anonymous PostModern)
    belongs_to :owner, :polymorphic => true
    has_state :publication_state
    has_state :post_type, :with => TYPES
    has_polymorph :owner

  class Comment < ActiveRecord::Base
    include Pacecar
    belongs_to :user

  class Article < ActiveRecord::Base
    belongs_to :user

All columns

Records where approved_at is not null, or where it is null…


Records where first_name is not null, or where it is null…


Records ordered by first_name (default to ‘asc’, can specify to override)…


Records where an attribute matches a search term (column LIKE “%term%”)…


Records where an attribute starts or ends with a search term…


Records where any non-state text or string column matches term…


Records where any of a list of columns match the term…

  User.search_for 'test', :on => [:first_name, :last_name]

Records where all of a list of columns match the term…

  User.search_for 'test', :on => [:first_name, :last_name], :require => :all

Boolean columns

Records that are all admins or non-admins…


The “balance” (count of true minus false for column in question)…


Datetime columns

Records approved before or after certain times…


Records with approved_at in the past or future…


Records with approved_at inside or outside of two times…


Records with certain year, month or day…


Records with a duration (time delta between two columns) of, over or under a certain number of days…

  User.with_duration_of(14, :approved_at, :rejected_at)
  User.with_duration_over(14, :approved_at, :rejected_at)
  User.with_duration_under(14, :approved_at, :rejected_at)

Polymorphic relationships

Records which have an owner_type of User…



Records with the most and least associated records…


Records with associated records since a certain time…


State columns

Records which are in a particular state, or not in a state…


Query methods on instances to check state…



First x records…

Named scopes

Because these are all named_scope, you can combine them.

To get all users that have a first_name set, who are admins and approved more than 2 weeks ago, ordered by their first name…


To get the top 10 commenters…

Rails & named_scope

Tháng Năm 18, 2009


Lately everyone has been going mad over a relatively new addition to rails: named_scope. Up until now, I hadn’t had the chance to try it out. I always just assumed that it was more syntactic fluff that seems to accumulate. Wow, was I wrong! It’s an unbelievably cool and useful idea. Here’s a really simple example. Say I have 2 models: Country and Region. They are as you would expect. A Region belongs to a Country and a Country has many Regions, etc. I use them pretty much for dropdown lists and things like that. Here’s some code:

  1. class Region < ActiveRecord::Base
  2. belongs_to :country
  3. end
  4. class Country < ActiveRecord::Base
  5. has_many :regions
  6. end

Prior to named_scope, if you wanted to get all Regions in alphabetical order (as an example), you’d have to do something like:

  1. class Region < ActiveRecord::Base
  2. #other code
  3. def self.ordered
  4. find(:all, :order => “name ASC”)
  5. end
  6. end

Which was ok, but this is nicer:

  1. class Region < ActiveRecord::Base
  2. named_scope :ordered, :order => “name ASC”
  3. #other code
  4. end

This creates a method on the Region class called ordered that can simply be called.

OK, so that’s cool and all, but really not that big a deal, right? Well, no – until you consider that you can chain calls to named_scope, thus adding more and more constraints. For example:

  1. class Region < ActiveRecord::Base
  2. belongs_to :country
  3. named_scope :ordered, :order => “name ASC”
  4. named_scope :by_country, lambda { |c| { :conditions => [“country_id = ?”,] } }
  5. named_scope :containing, lambda { |s| { :conditions => [“name like ?”, “%#{s}%”] } }
  6. end
  7. class Runner
  8. c = Country.find_by_name(“Canada”)
  9. regions = Region.by_country(c).containing(“A”).ordered
  10. puts “Provinces in Canada containing the letter A in ascending order: #{regions.inspect}”
  11. end

This generates SQL that looks like this:

SELECT * FROM `regions` WHERE ((name like ‘%A%’) AND (country_id = 1)) ORDER BY name ASC

Another reason that I like name_scope is that because models such as these are frequently eused in dropdowns, etc, you can use named scope to return a very lightweight version of the class, while excluding attributes that are unnecessary for a simple dropdown. For example:

  1. class Region < ActiveRecord::Base
  2. belongs_to :country
  3. named_scope :ordered, :order => “name ASC”
  4. named_scope :by_country, lambda { |c| { :conditions => [“country_id = ?”,] } }
  5. named_scope :containing, lambda { |s| { :conditions => [“name like ?”, “%#{s}%”] } }
  6. named_scope :simple, :select => “id, name”
  7. end

This allows me to only bring back the attributes that I need. And again, it could be chained with other named scopes

I think that this is a very powerful concept that allows developers to write more DRY and readable code.


When looking at my own Rails code and that of the community as a whole, I often see places where certain Rails techniques could have been used, but weren’t. As much for my own memory as yours, I thought I’d list down some Rails tricks and tips that can make your application or code more efficient:

Benchmark logic in your controller actions – It’s really easy. Use the benchmark class method available on all your models like this:

User.benchmark("adding and deleting 1000 users") do
	1000.times do
		User.create(:name => 'something')
		x = User.find_by_name('something')

Of course, your code would be a lot better ;-) The regular SQL logs are not shown when within the benchmark sections. Only the benchmark results are shown.


– Almost everyone is familiar with acts_as_tree, but acts_as_nested_set snuck into Rails quietly. It’s much like acts_as_tree, but with the added benefit that you can select all of the children (and their own descendants) of a node with a single query. A list of the instance methods is available.

Easier collections with to_proc – Sick of writing things like post.collect { |p| p.title } or { |p| p.activated? }.collect{ |p| p.title} ? A little Ruby hackery that allows you to convert symbols into proc references makes it easy. You can write post.collect(&:title) or instead! Learn a lot more about this.

Convert arrays to sentences in views – If you were collecting a bunch of names to be shown in a view, you might end up with an array like [‘Peter’, ‘Fred’, ‘Chris’], and joining these with commas and inserting ‘and’ before the final one is a common pain. Not so, if you use the array method to_sentence as provided in Rails. names.to_sentence would return Peter, Fred, and Chris.

Send files back to the user – Usually, static files can be retrieved by using the direct URL and circumventing your Rails application. In some situations, however, it can be useful to hide the true location of files, particularly if you’re sending something of value (e-books, for example). It may be essential to only send files to logged in users too. send_file makes it possible. It sends files in 4096 byte chunks, so even large files can be sent without slowing the system down.

Iterating through page elements with RJS – Changing page elements with RJS is easy, but what if you don’t know exactly which elements you want to change, and would instead prefer to address them with CSS queries? You can with RJS’s select method. For example:‘#items li’).each { |item| item.hide } . Powerful stuff!

Check for existence – When doing a Model.find(id), an exception can be returned if the item with an id of ‘id’ doesn’t exist. If you want to avoid this, use Model.exists?(id) first to get a true or false for whether that item exists or not.

Number helpers for common number tasks – All of these number helpers aren’t commonly used but provide great shortcuts: number_to_currency(1234567.948) # => $1,234,567.95 or human_size(1234567890) # => 1.1GB or number_with_delimiter(999999999) # => 999,999,999. There are others.

Testing different route configurations easilywith_routing is a test helper that allows you to temporarily override the default ‘routes’ in routes.rb for test purposes. Demonstration:

with_routing do |set|
  set.draw { set.connect ':controller/:id/:action' }
     ['/content/10/show', {}],
     set.generate(:controller => 'content', :id => 10, :action => 'show')

You can learn a little more here.

Get lots of info about requests – Checking and request.xhr? are popular ways to look for POST and AJAX requests, but some of the other request methods are lesser used. For example: request.subdomains can return an array of subdomains that you could use as part of your authentication scheme, request.request_uri returns the full local request URL, returns the full hostname, request.method returns the HTTP method as a lowercase symbol, and request.ssl? returns true if it’s an HTTPS / SSL request.

Improving session performance even more than with ActiveRecord – By default, Rails stores sessions on the local file system. Many users change this to using ActiveRecordStore to store sessions in the database. An even faster alternative is to use Memcached to store sessions, but that takes a lot to set up (and isn’t available unless you run your own servers, etc). But you can get faster than ActiveRecordStore by using Stefan Kaes’ SQLSessionStore. It circumvents the inefficiencies of ActiveRecordStore using his own direct SQL technique to store sessions.

Caching unchanging data at application startup – If you have data that doesn’t change between application restarts, cache it in a constant somewhere. For example, you might have a YAML or XML file in /config that stores application configuration data, and you could load it into a constant in environment.rb, making lookups quick and easy application-wide.

Check your views are rendering valid HTML / XHTML – It’s not for everyone, but if your output validates as correct HTML / XHTML, it’s a sign your views are going to render properly. Scott Raymond has developed a assert_valid_markup test helper that you can use from your functional tests.

Cleaner HTML output testing – Combine why’s Hpricot HTML parser and a special test extension, and you can have powerful tests like so: assert_equal “My title”, tag(‘title’) or assert element(‘body’).should_contain(‘something’). This might be ideal for developing tests to test user built templates. In any case, it’s nicer than assert_tag!

Run long-running tasks separately in the backgroundBackgrounDRb is a small framework, by Ezra Zygmuntowicz, that runs as a daemon in the background that can accept tasks your Rails application sends to it, and whose execution is totally separate to your Rails app. It’s extremely powerful, and useful for many tasks such as sending hundreds of e-mails, fetching URLs, and other things you don’t want to slow down the request times for your main app. One great demo is to develop a task that increments a variable by 1 and sleeps for 1 second. You can then make a Rails method that queries the variable, and see the distinct separation. Learn more.

Make ids in URLs more user friendly – Override the to_param method on your model and return something like “#{id}-#{title.gsub(/[^a-z0-9]+/i, ‘-‘)}” to get URLs like so: .. Much nicer for users, and you don’t need to change anything with Post.find(params[:id]) as the non numeric characters will be stripped automagically! Get a full explanation here.

Separate out slices of functionality into Engines – Everyone’s heard of Rails’ plugins, but pitifully few are using Rails Engines! Rails Engines are like plugins on steroids. They can contain their own models, controllers, and views, and integrate with any applications you run them under. This allows you to split out common fragments of functionality (login, user management, content management, etc.) into separate ‘engines’ to use in your different projects within minutes. No more writing dull login code! Rails Engines is a big deal, but it should be a far bigger deal.

Calculations – Do you want to get maximums, minimums, averages, or sums for data in your tables? ActiveRecord’s Calculations make these all possible. Person.average(‘age’), Person.maximum(:age, :group => ‘last_name’), and Order.sum(‘total’) all become a reality. Most can be customized pretty deeply with extra options, so go read about them if they’re not already part of your code.

XML or YAML output of your data – It’s not necessarily to create a Builder .rxml template for all XML output. ActiveRecord has a to_xml method that will output the object or result set in XML format. It works with simple objects, to complete tables (like User.find(:all).to_xml). Using includes works too, as with Post.find(:all, :include => [:comments]).to_xml. YAML is also supported, by using to_yaml instead.


Typhoeus is a mythical greek god with 100 fire breathing serpent heads. He’s also the father of the more well known Hydra. Like the fearsome beast, Typhoeus is a fearsome Ruby library that enables parallel HTTP requests while cleanly encapsulating handling logic. Specifically, it uses libcurl and libcurl-multi to run HTTP really fast. Further, it’s designed with the focus of creating client libraries that work with web services. These could be external services like Twitter or systems like CouchDB and SimpleDB or custom web services that you write yourself.

The libcurl interface is contained within the library. Rather than trying to get Curb to do what I wanted, I decided to start with a clean slate and write the c bindings myself. Other than the libcurl interface, it has a nice DSL for creating classes and client libraries for web services.

The inspiration for the library came from an interview with Amazon CTO Werner Vogels. In the interview he states that when a user visits the home page it calls out to up to 100 different services to construct the single page before returning it to the user. I like Amazon’s approach of a services architecture, specially AWS, and wondered if I could do the same thing in Ruby. Typhoeus is the result of that effort.

I set up a benchmark to test how the parallel performance works vs Ruby’s built in NET::HTTP. The setup was a local evented HTTP server that would take a request, sleep for 500 milliseconds and then issued a blank response. I set up the client to call this 20 times. Here are the results:

 net::http 0.030000 0.010000 0.040000 ( 10.054327)
 typhoeus 0.020000 0.070000 0.090000 ( 0.508817)

We can see from this that NET::HTTP performs as expected, taking 10 seconds to run 20 500ms requests. Typhoeus only takes 500ms (the time of the response that took the longest.)

Hopefully I’ve whetted your appetite. Before I get to the code examples and the API, I’d like to put in a plug for my employer kgb. They were nice enough to let me release this library as open source. We’re also hiring for good front end rails developers, designers, and anyone with experience in search, information retrieval, and machine learning. Please drop me a line if you dominate code and if you’re interested in joining an awesome team.

Finally, on the the codez. Here are some usage examples and notes from the readme (gist for easier reading).

# here's an example for twitter search
# Including Typhoeus adds http methods like get, put, post, and delete.
# What's more interesting though is the stuff to build up what I call
# remote_methods.
class Twitter
 include Typhoeus
 remote_defaults :on_success => lambda {|response| JSON.parse(response.body)},
 :on_failure => lambda {|response| puts "error code: #{response.code}"},
 :base_uri => ""

 define_remote_method :search, :path => '/search.json'
 define_remote_method :trends, :path => '/trends/:time_frame.json'

tweets = => {:q => "railsconf"})

# if you look at the path argument for the :trends method, it has :time_frame.
# this tells it to add in a parameter called :time_frame that gets interpolated
# and inserted.
trends = Twitter.trends(:time_frame => :current)

# and then the calls don't actually happen until the first time you
# call a method on one of the objects returned from the remote_method
puts tweets.keys # it's a hash from parsed JSON

# you can also do things like override any of the default parameters => {:q => "hi"}, :on_success => lambda {|response| puts response.body})

# on_success and on_failure lambdas take a response object. 
# It has four accesssors: code, body, headers, and time

# here's and example of memoization
twitter_searches = []
10.times do
 twitter_searches << => {:q => "railsconf"})

# this next part will actually make the call. However, it only makes one
# http request and parses the response once. The rest are memoized.
twitter_searches.each {|s| puts s.keys}

# you can also have it cache responses and do gets automatically
# here we define a remote method that caches the responses for 60 seconds
klass = do
 include Typhoeus

 define_remote_method :foo, :base_uri => "http://localhost:3001", :cache_responses => 60

klass.cache = some_memcached_instance_or_whatever
response = 
puts response.body # makes the request

second_response =
puts response.body # pulls from the cache without making a request

# you can also pass timeouts on the define_remote_method or as a parameter
# Note that timeouts are in milliseconds.
Twitter.trends(:time_frame => :current, :timeout => 2000)

# you also get the normal get, put, post, and delete methods
class Remote
 include Typhoeus

Remote.put("http://", :body => "this is a request body")"http://localhost:3001/posts.xml", 
 {:params => {:post => {:author => "paul", :title => "a title", :body => "a body"}}})

Important Update:I should have mentioned that some bits of C were pulled from Todd Fisher’s update to curb for the multi interface. The easy code is completely different and some of the C stuff in Multi has changed. However, a good chunk of the multi code comes straight from there. Thanks Todd. Sorry for not mentioning it earlier.


Making Tests Easy on the Fingers and Eyes

The Shoulda gem makes it easy to write elegant, understandable, and maintainable Ruby tests. Shoulda consists of test macros, assertions, and helpers added on to the Test::Unit framework. It’s fully compatible with your existing tests, and requires no retooling to use.

  • Context & Should blocks – context and should provide RSpec-like test blocks for Test::Unit suites. In addition, you get nested contexts and a much more readable syntax.
  • Macros – Generate many ActionController and ActiveRecord tests with helpful error messages. They get you started quickly, and can help you ensure that your application is conforming to best practice.
  • Assertions – Many common Rails testing idioms have been distilled into a set of useful assertions.

The Shoulda gem can be used for non-Rails projects


sudo gem install thoughtbot-shoulda --source=


Context Helpers

Stop killing your fingers with all of those underscores… Name your tests with plain sentences!

class UserTest < Test::Unit 
  context "A User instance" do
    setup do
      @user = User.find(:first)

    should "return its full name" do
      assert_equal 'John Doe', @user.full_name

    context "with a profile" do
      setup do
        @user.profile = Profile.find(:first)

      should "return true when sent #has_profile?" do
        assert @user.has_profile?

Produces the following test methods:

"test: A User instance should return its full name." 
"test: A User instance with a profile should return true when sent #has_profile?."

So readable!

ActiveRecord Tests

Quick macro tests for your ActiveRecord associations and validations:

class PostTest < Test::Unit::TestCase
  should_belong_to :user
  should_have_many :tags, :through => :taggings

  should_require_unique_attributes :title
  should_require_attributes :body, :message => /wtf/
  should_require_attributes :title
  should_only_allow_numeric_values_for :user_id

class UserTest < Test::Unit::TestCase
  should_have_many :posts

  should_not_allow_values_for :email, "blah", "b lah" 
  should_allow_values_for :email, "", "" 
  should_ensure_length_in_range :email, 1..100
  should_ensure_value_in_range :age, 1..100
  should_protect_attributes :password

Makes TDD so much easier.

Controller Tests

Macros to test the most common controller patterns…

context "on GET to :show for first record" do
  setup do
    get :show, :id => 1

  should_assign_to :user
  should_respond_with :success
  should_render_template :show

  should "do something else really cool" do
    assert_equal 1, assigns(:user).id

Helpful Assertions

More to come here, but have fun with what’s there.

assert_same_elements([:a, :b, :c], [:c, :a, :b])
assert_contains(['a', '1'], /\d/)
assert_contains(['a', '1'], 'a')