If you’ve got a project with multiple git remotes, and you’d like to update both of them with a single operation, there’s an easy way to do it. Just edit your .git/config file to include a new target named “all”:

[remote "origin"]
  url = git@github.com:ffmike/BigOldRailsTemplate.git
  fetch = +refs/heads/*:refs/remotes/origin/*
[branch "master"]
  remote = origin
  merge = refs/heads/master
[remote "codaset"]
  url = git@codaset.com:ffmike/bigoldrailstemplate.git
  fetch = +refs/heads/*:refs/remotes/codaset/*
[remote "all"]
  url = git@github.com:ffmike/BigOldRailsTemplate.git
  url = git@codaset.com:ffmike/bigoldrailstemplate.git

With this setup, pushes and pulls on my local master branch go to GitHub by default. But if I execute ‘git push all’ it updates both the GitHub repository and the Codaset repository. The one little annoyance is that master then thinks it’s ahead of the remote, but that’s easily remedied by executing a ‘git pull’ on it.

I’ve been using this in cases where I’m backing up a repo to a private Gitosis server as well as spreading public repos between multiple cloud services for easy backup.

I spent a good chunk of the last couple of days upgrading one of my Macs to Snow Leopard (OS X 10.6), and keeping notes as I went. I had several goals here:

• Develop a repeatable process I could use on my main development box
• Set up multiple versions of Ruby, as I expect 1.9 compatibility to be increasingly important
• Avoid MacPorts, as it’s been problematic for me and I don’t like two copies of everything
• Prevent disasters
• Migrate applications and documents from the previous setup on the same box

I’m pretty happy with where it all ended up, but please realize that this guide is only definitive for me: just because this is the way it worked for me does not mean it’s the only way to set things up for Rails development. I hope some folks find this helpful, but the main reason I wanted to work all this out was to handle our own in-house Macs.

1. Install external hard drive. I used a LaCie d2 Quadra. At $150 with free shipping (we happily pay for Amazon Prime) it’s cheap insurance. Didn’t bother with their Setup Assistant (who the heck still writes software that requires Rosetta?), just erased it with Disk Utility and formatted it as one big journaled OS X volume.

2. Install SuperDuper (http://www.shirt-pocket.com/SuperDuper/SuperDuperDescription.html). I’m sure Carbon Copy Cloner is just as good, I just happen to be a SuperDuper user. Even if you’re not a paying customer, SuperDuper is free to use in interactive mode, which is all this procedure requires.

3. Log out and log back in with the shift key held down to prevent startup items from running.

4. Launch SuperDuper and back up the internal hard drive to the external hard drive with the “Backup – All Files” script.

5. Restart, holding down the Option key when the box restarts. This will let you choose to restart from the backup you just made. Booting from the external drive will take a lot longer than from the internal one, but don’t skip this step. You’ll feel pretty darned stupid if you bootable backup isn’t. Hold down shift after selecting the drive to bypass startup items so you don’t clutter the backup.

6. While you’re in there, launch terminal and :

gem list > gemlist.txt
port list installed > portlist.txt

I didn’t use the port list for anything myself, but it was a good reference for things I needed to install otherwise.

7. Now it’s time to do some cleaning up of your backup to prepare for the later migration. Mac’s Migration Assistant doesn’t offer a lot of granularity, so you need to get rid of anything from your Applications, Documents, and other descendants of your home directory that you don’t want to move. The more you delete now, the faster the migration will go, so delete any applications and documents that you don’t want on the rebuilt Mac. Good time to clean up the Dock as well. I blew away my local gems as well at this point.

I also recommend completely blowing away /usr/local and /opt/local to make sure you’re doing a “clean everything” install even after migrating. Edit your .bash_login to remove them from the path as well (optionally, you may want to rename these directories to have them available for reference).

8. If you’re a PostgreSQL user, make a backup of your databases. Tyler Hunt has clear instructions for this.

9. Put in the Snow Leopard DVD. Ignore the Finder window when it pops up.

10. Power down the computer.

11. Unplug the backup drive. No sense taking chances.

12. Restart with the option key held down again. Choose the Mac OS X Install DVD to boot from.

13. The “Erase and Install” option is gone from the Snow Leopard installer, so when you get to the “Install Mac OS X” screen select Disk Utility from the Utilities menu instead.

14. Select the “Macintosh HD” partition on your hard drive. Select the Erase tab. Select the “Mac OS Extended (Journaled)” format. Click Erase. Grit your teeth and confirm.

15. When OS X is done reformatting the partition, close Disk Utility, click Continue on the OS X installer, and work your way through the rest of the prompts.

16. When you get to the “Do You Already Own a Mac?” screen (this is after the installation restart, so don’t panic when the machine restarts), plug the external drive back in. Select “From another volume on this Mac” and click Continue.

17. The top of the “Transfer Your Information” screen should show that you’re transferring from the external drive. Leave all the options checked and click Transfer.

18. Log into happy new OS X. Bask in it for a little bit. Wait a minute for Software Update to kick in, or run it manually and install whatever it comes up with. If it doesn’t find anything, that’s great.

19. While you’ve got the Snow Leopard DVD in the drive, pop option the Optional Installs folder and install XCode.

20. I personally prefer wget to curl (just habit, I suppose) which means that I have to install it:

GNU wget home page

Download wget-latest.tar.gz from http://ftp.gnu.org/gnu/wget/
tar -xzvf wget-latest.tar.gz
cd wget-1.11.4/
./configure
make
sudo make install

21. I’m trying a no-MacPorts approach this time around. This means installing a current readline by hand, since multiple pieces of software still reportedly have issues with Apple’s default version:

GNU Readline home page

wget ftp://ftp.cwru.edu/pub/bash/readline-6.0.tar.gz
tar -xvzf readline-6.0.tar.gz
cd readline-6.0
./configure
make
sudo make install

22. This is a good point to make sure your .bash_login and .bash_profile are the way you want them. For me, that’s an empty .bash_login and a pretty minimal .bash_profile. Here’s my .bash_profile, though you probably want to tune your own:

export PATH=/usr/local/git/bin:/usr/local/bin/src:/usr/local/bin:
  /usr/local/sbin:/usr/local/mysql/bin:~/bin:$PATH

export EDITOR='mate -w'
export PAGER=most

function parse_git_dirty {
  [[ $(git status 2> /dev/null | tail -n1) !=
    "nothing to commit (working directory clean)" ]] && echo "*"
}
function parse_git_branch {
  git branch --no-color 2> /dev/null |
    sed -e '/^[^*]/d' -e "s/* \(.*\)/[\1$(parse_git_dirty)]/"
}

export PS1='\[33[1;36m\]\h\[33[0;33m\] \w \[33[31;40m\]
  $(parse_git_branch)\[33[00m\] $ '

export HISTCONTROL=erasedups
export HISTSIZE=10000
shopt -s histappend

EVENT_NOKQUEUE=yes

23. Snow Leopard includes a recent version of SQLite, but you know, as long as we’re building things we may as well make it current.

SQLite Download Page

SQLite Build Instructions

wget http://www.sqlite.org/sqlite-amalgamation-3.6.17.tar.gz
tar -xvzf sqlite-amalgamation-3.6.17.tar.gz
cd sqlite-3.6.17/
./configure
make
sudo make install

24. Now for MySQL. If you have a previous version of MySQL installed, you probably have the old MySQL preference pane, which doesn’t work under Snow Leopard. Fortunately swoon.net has recompiled a 64-bit version. Grab it and install it. Then pop into System Preferences and stop your MySQL server if it’s running.

25. Given that there are prebuilt x64 binaries for OS X, I don’t feel any particular need to build MySQL from scratch. I’ve found anything past MySQL 5.1 to be problematic for Rails.

MySQL Download Page

Pick a mirror and download the “Mac OS X 10.5 (x86_64)” version. Mount the dmg and run the MySQL package and then the MySQL startup item package. Then start MySQL:

sudo /Library/StartupItems/MySQLCOM/MySQLCOM start

26. If you have existing MySQL databases to bring over from an old version, see the instructions from edward – remember to change the old directory name to match the version you were running before.

27. SQLite and MySQL are nice, but these days I’m using PostgreSQL for new Rails work. If you want a shortcut, there’s an installer that runs on OS X: grab it and run it. But to get best performance, you probably should build by hand. There are good instructions from Tyler Hunt for that, though you need to update the download version. Don’t install the postgres gem though; it’s slow and outdated, and anyhow there’s still work to do to get gems set up.

PostgreSQL source code

wget ftp://ftp.postgresql.org/pub/source/v8.4.0/postgresql-8.4.0.tar.gz
tar -xvzf postgresql-8.4.0.tar.gz
cd postgresql-8.4.0
./configure --enable-thread-safety --with-bonjour

And follow Tyler’s instructions from there for installing, creating a group and user, and hooking up launchd.

28. Reload your old PostgreSQL databases.

29. Databases in hand, let’s turn to ruby. Given that I expect a transition to ruby 1.9 over the next year, and I find myself wanting to test on odd rubies from time to time, I’m going to completely ignore the system ruby except to bootstrap a multiruby setup. Fortunately, the recently-released Ruby Version Manager is working great already.

Ruby Version Manager homepage

sudo gem install rvm
rvm-install
rvm install 1.8.6 -C --enable-shared,--with-readline-dir=/usr/local
rvm install 1.8.7 -C --enable-shared,--with-readline-dir=/usr/local
rvm install 1.9.1 -C --enable-shared,--with-readline-dir=/usr/local
rvm install 1.9.2 -C --enable-shared,--with-readline-dir=/usr/local
rvm 1.8.7 --default

I’m skipping REE, JRuby, and Rubinius for the moment, but will likely add them to the system later.

30. Time to install the two problematic gems. I’m just going to do this under 1.8.7 for now, but will update the other rubies later.

MySQL: (from Greg Benedict)

sudo env ARCHFLAGS="-arch x86_64" gem install mysql -- --with-mysql-config=/usr/local/mysql/bin/mysql_config

PostgreSQL: (from iCoreTech)

sudo -s
export ARCHFLAGS='-arch x86_64'
export PATH=/usr/local/pgsql/bin:${PATH}
gem install pg
exit

31. The rest of the gems should be easier. Grab the gemlist.txt file you created way back at the start and edit it to one gem name per line. Ditch anything you know you don’t need. Then fire up IRB and run:

File.open('/Users/mike/gemlist.txt').each_line {|line| `sudo gem install #{line.split.first}`}

Obviously change the file name to match where you put it on your own drive.

This could use some tweaking to deliver feedback.

[stalled at RedCloth?]

32. This is a good time to get git on the system. Fortunately, there’s an easy and functional OS X installer for that. Download, mount the disk image, and run the installer and shell script.

33. imagemagick used to be a bear, but at the moment there’s a good install script for it on GitHub. So, now that you have your shiny new copy of git:

git clone git://github.com/maddox/magick-installer.git
cd magick-installer/
./magick-installer.sh

34. I also need memcached locally. Wincent has good install instructions:

wget http://www.monkey.org/~provos/libevent-1.4.12-stable.tar.gz
tar xvzf libevent-1.4.12-stable.tar.gz
cd libevent-1.4.12-stable
./configure
make
sudo make install
wget http://memcached.googlecode.com/files/memcached-1.4.1.tar.gz
tar xvzf memcached-1.4.1.tar.gz
cd memcached-1.4.1
./configure
make
sudo make install

35. I’d like to run gem server locally on a startup basis to handle my gem RDoc needs, but I need to figure out how (or whether) that can be made to play nicely with rvm. Meanwhile, bdoc is an acceptable alternative:

sudo gem install manalang-bdoc

36. Finally (for now!) I prefer most as a pager to less or more. It supports paging forwards and backwards, split windows, and can even drop the file you’re paging right into your editor of choice.

Most home page

You’ll need some prerequisites first:

pcre:

wget ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-7.9.tar.gz
tar xvzf pcre-7.9.tar.gz
cd pcre-7.9
./configure
make
sudo make install

Oniguruma:

wget http://www.geocities.jp/kosako3/oniguruma/archive/onig-5.9.1.tar.gz
tar xvzf onig-5.9.1.tar.gz
cd onig-5.9.1
./configure
make
sudo make install

S-Lang:

wget ftp://space.mit.edu/pub/davis/slang/v2.2/slang-2.2.0.tar.gz
tar xvzf slang-2.2.0.tar.gz
cd slang-2.2.0
./configure
make
sudo make install

And now most at last:

wget ftp://space.mit.edu/pub/davis/most/most-5.0.0.tar.gz
tar xvzf most-5.0.0.tar.gz
cd most-5.0.0
./configure
make
sudo make install

And there you have it! Start to finish, this process took me about 3 hours on a MacBook Pro with around 90GB of data that I wanted to save. I expect it to take much longer on the desktop with the correspondingly larger hard drive, but much of that will be overnight backup and restore times. If you have any tweaks and suggestions, feel free to add them to the comments.

I recently had to implement some recurring items in a Rails application – we’ll call them “entries” here though you can use this technique for appointments, bills, or whatever else. After thinking about it for a while, I settled on:

• At any point, we needed to know what the next recurrence date was.
• The potential schedules needed to be data-driven (because the client was unsure what would be needed).

Here’s what I came up with. If your needs are different, you’ll no doubt need to alter it. First, the schedule model has a schedule_type, an interval, a specific value, and a description. The combination of these things lets me implement Schedule#next_date to return the fate for the next date given a schedule object and the current date. Here’s the model:

class Schedule

  has_many :entries

  attr_accessible :schedule_type, :description, :interval, :specific

  SCHEDULE_TYPE = {:first => 0, :last => 1, :weekly => 2, :monthly => 3}

def next_date(current_date)
  case schedule_type
  when SCHEDULE_TYPE[:first]
    first_day = (current_date + 1.month).beginning_of_month
    wday = first_day.wday
    if specific >= wday
      first_day + specific - wday
    else
      first_day + 7 + specific - wday
    end
  when SCHEDULE_TYPE[:last]
    if specific == -1
      (current_date + 1.month).end_of_month
    else
      last_day = (current_date + 1.month).end_of_month
      wday = last_day.wday
      if wday >= specific
        last_day - wday + specific
      else
        last_day - wday - 7 + specific
      end
    end
  when SCHEDULE_TYPE[:weekly]
    current_date + interval.weeks
  when SCHEDULE_TYPE[:monthly]
    current_date + interval.months
  end
end

So we can have a schedule that is “first Monday of the month” or “every 3 weeks” among other things. I stock these up using db-populate; here’s some of the population file:

Schedule.create_or_update(:id => 1,
  :schedule_type => Schedule::SCHEDULE_TYPE[:monthly],
  :interval => 1, :description => "Every month")
Schedule.create_or_update(:id => 2,
  :schedule_type => Schedule::SCHEDULE_TYPE[:monthly],
  :interval => 2, :description => "Every other month")

Schedule.create_or_update(:id => 4,
  :schedule_type => Schedule::SCHEDULE_TYPE[:weekly],
  :interval => 1, :description => "Every week")

Schedule.create_or_update(:id => 8,
  :schedule_type => Schedule::SCHEDULE_TYPE[:first],
  :specific => 0, :description => "First Sunday of every month")

Schedule.create_or_update(:id => 15,
  :schedule_type => Schedule::SCHEDULE_TYPE[:last],
  :specific => 0, :description => "Last Sunday of every month")

Schedule.create_or_update(:id => 22,
  :schedule_type => Schedule::SCHEDULE_TYPE[:last],
  :specific => -1, :description => "Last day of every month")

Then the entry model ties into the schedule model. Here are the important (for this purpose) bits of the entry class:

class Entry < ActiveRecord::Base
  belongs_to :schedule
  named_scope :ready_to_recur, lambda { &#124;date&#124;
    {:conditions => ["recurring = 1 AND next_date <= ? AND schedule_id IS NOT NULL AND next_created = 0", date ]} }
  before_save :set_up_recurrence

  def make_next_recurrence
    if recurring? && !schedule_id.nil? && !next_created?
      entry = Entry.create(
        :entry_date => next_date,
        :reference => reference,
        <more attributes here>
        :recurring => true,
        :schedule_id => schedule_id
        )
      update_attribute(:next_created, true)
    end
    entry
  end

  def set_up_recurrence
    if recurring? && !schedule_id.nil?
      self.next_date = Schedule.find(schedule_id).next_date(entry_date)
      self.next_created = false if next_created.nil?
    end
    true
  end
end

Finally, the whole thing is driven by a rake task that we run every night. This task finds all the entries that are ready to recur and creates the next entry:

desc 'Create the recurring entries for today'
task :daily_recurring_entries => :environment do
  Entry.ready_to_recur(Date.today).each do |entry|
    entry.make_next_recurrence
  end
end

A few people have asked me about the Rails application template that I’ve put together recently. For those who are interested, it’s now available for review and download as a gist. To use it, just specify the -m switch when creating a Rails application:

rails new_app_name -m lark_template.rb

I should warn you of two things. First, it’s a pretty heavyweight template, sticking a lot of stuff into the new application. This suits me, because I have a lot of things I use in just about every application, but it may not suit you. Second, Rails templates are not one-size-fits-all; you’d be better off treating this as a starting point to steal from than a finished template to use (though if you want to use it, feel free).

Here’s a list of what this template sets up:

Coding Tools

• Authlogic for user authentication, including password resets, anonymous_only, authenticated_only, admin_only application helpers
• World’s simplest authorization system: manage multiple string roles on users with User#add_role, User#remove_role, User#clear_roles, and User#has_role?
• Date formats: :us, :us_with_time, :short_day, :long_day
• live-validations for client-side JavaScript data entry validation. Add :live_validations => true to form_for declarations to hook this up.
• Paperclip for attachment management
• /pages/css_test will show most CSS styles in action
• Searchlogic for magic named scopes and search forms – http://rdoc.info/projects/binarylogic/searchlogic. Includes attribute_equals, attribute_does_not_equal, attribute_begins_with, attribute_like, attribute_ends_with, attribute_greater_than, attribute_null, attribute_blank etc. etc.
• Stringex for extra string functionality – acts_as_url, String#to_ascii, String#to_html, String#to_url, String#remove_formatting, String.random
• US State application helpers
• will-paginate for pagination

Database Tools

• Hooked up for PostgreSQL
• admin-data plugin for administrative UI. http://localhost:3000/admin_data will get you to the application’s data. On production, only admin can view data, no one can edit (modify config/initializers/admin_data.rb to adjust this)
• db-populate for seed data

Deployment Tools

• fast_remote_cache strategy for deployment
• rubiadhstrano for deployment recipes; automatically uses multiple targets, so: cap production deploy for deployment to production
• superdeploy for additional Capistrano tasks. cap -T for full list.

External Services

• Exceptional for error tracking. Go to /pages/kaboom to test after finishing Exceptional setup.
• New Relic for performance tracking

Testing Tools

• Shoulda and Test::Unit for testing
• Mocha for mocking
• Object Daddy for factories
• Generated code is already covered by tests
• parallel-test for faster testing. rake test:parallel:prepare[2] to set up two test databases. rake test:parallel[2] to distribute tests across two cores
• rack-bug for request/response/perf analysis. http://localhost:3000/__rack_bug__/bookmarklet.html to add bookmarklet to browser.
• shmacros for additional Shoulda macros: should_accept_nested_attributes_for, should_act_as_taggable_on, should_callback, should_delegate, more
• More extra shoulda macros: should_have_before_filter, should_have_after_filter
• metric-fu for static code analysis. rake metrics:all, configure in Rakefile
• inaction-mailer is installed for development environment, so mails sent during dev will end up as files in /tmp/sent_mails

I’ve got an application that I work on where the client wants to track which countries they’re seeing click-throughs from. They want this in real time and in the application’s UI, not in an external package such as Google Analytics. There are various ways to guess at country, but for the purposes of this application, basing it on the IP address and where its range is assigned proves to be good enough.

There are various services that will let you geolocate based on IP just by making an API call. But there’s no particular reason to go outside your own application for this; it’s pretty trivial to set up in Rails:

1) Download the “Complete (Country) One Table” SQL file from IPInfoDB. This is the raw data necessary to do the lookups. Decompress it and then run the resulting SQL file in your application’s database. This will give you an ip_group_country table. There’s no need to set up a matching model; we’re just going to hit it with raw SQL.

2) In this particular application, we’re maintaining an Event model that holds, among other things, the IP address of the request. So the easy answer is to geolocate instances of that model when they’re created or updated:

class Event < ActiveRecord::Base
  before_save :get_country_info

  def get_country_info
    segments = remote_ip.split('.')
    ip_atom = ((segments[0].to_i * 256 + segments[1].to_i) * 256 + segments[2].to_i)*256
    result = connection.execute("SELECT * FROM `ip_group_country` where `ip_start` <= #{ip_atom} order by ip_start desc limit 1;")
    row = result.fetch_row
    self.country_code = row[2]
    self.country_name = row[3]
  end

end

3) There is no #3. That’s it!

I’ve been poking at various continuous integration software lately. Most recently I gave Cerberus a spin, and I must say, even though the project is a bit immature, I like its approach. My test for CI is generally “can I use it on the Rails source” as I think that provides a level of complexity beyond what I’m likely to need for my own projects. Well, I had to contribute a bit of code to get Cerberus to that point, but it works. I also set up the box so that I can build with different ruby versions, thanks to multiruby. Here’s my notes on getting all the ducks in a row, starting with a bare-metal Debian Lenny installation.

Root Tasks

• log in as root
• adduser webadmin
• visudo – set webadmin up with sudo rights

webadmin Tasks

• log in as webadmin

Prerequisites for Ruby Compiles

• sudo apt-get install autoconf
• sudo apt-get install bison
• sudo apt-get install zlib1g-dev
• sudo apt-get install libzlib-ruby
• sudo apt-get install ruby-dev

rubygems install

• mkdir sources
• cd sources
• wget http://rubyforge.org/frs/download.php/56227/rubygems-1.3.3.tgz
• tar -xvzf rubygems-1.3.3.tgz
• cd rubygems-1.3.3/
• sudo ruby setup.rb
• sudo ln /usr/bin/gem1.8 /usr/bin/gem

fastcgi (we’ll need fcgi gem later)

• cd ~/sources
• wget http://www.fastcgi.com/dist/fcgi.tar.gz
• tar -xvzf fcgi-2.4.0.tar.gz
• cd fcgi-2.4.0/
• ./configure
• make
• sudo make install
• sudo apt-get install libfcgi-dev

Databases

• sudo apt-get install mysql-server-5.0 libmysqlclient15-dev
• sudo apt-get install sqlite sqlite3 libsqlite-dev libsqlite3-dev
• sudo apt-get install postgresql postgresql-server-dev-8.3
• sudo su – postgres -c ‘createuser -s webadmin’
• sudo su – postgres -c ‘createuser -s rails’

memcached

• sudo apt-get install memcached
• sudo /etc/init.d/memcached start

multiruby Install

• sudo gem install ZenTest
• sudo multiruby_setup mri:svn:tag:v1_8_6_368
• sudo multiruby_setup mri:svn:tag:v1_8_7_160
• sudo multiruby_setup mri:svn:tag:v1_9_1_129
• sudo multiruby_setup update:rubygems

Gem Installs

• sudo multiruby -S gem install –no-ri –no-rdoc –development test-unit
• sudo multiruby -S gem install –no-ri –no-rdoc –development fcgi
• sudo multiruby -S gem install –no-ri –no-rdoc –development memcache-client
• sudo multiruby -S gem install –no-ri –no-rdoc –development mocha
• sudo multiruby -S gem install –no-ri –no-rdoc –development mysql
• sudo multiruby -S gem install –no-ri –no-rdoc –development pg
• sudo multiruby -S gem install –no-ri –no-rdoc rack
• sudo multiruby -S gem install –no-ri –no-rdoc thin
• sudo multiruby -S gem install –no-ri –no-rdoc –development rake
• sudo multiruby -S gem install –no-ri –no-rdoc –development sqlite-ruby
• sudo multiruby -S gem install –no-ri –no-rdoc –development sqlite3-ruby
• sudo multiruby -S gem install geminstaller
• sudo multiruby -S gem install SystemTimer
• sudo multiruby -S gem install json

Note: the italicized installs currently fail on ruby 1.9.1.

git

• sudo apt-get install git-core

Cerberus

• sudo gem install cerberus

Note: Currently, you’ll need to either pull and build cerberus from my fork or from the ruby_builder branch in the official repo. With the next release after 0.5.0, you should be able to use the gem install method.

Create the Cerberus project and build it

• cerberus add http://github.com/rails/rails/tree/master APPLICATION_NAME=rails_master_187 SCM=git
• Edit the file ~/.cerberus/config.yml to include your own notifier information
• Edit the file ~/.cerberus/config/rails_master_187.yml to put in the custom build info:

---
publisher:
  mail:
    recipients: you@yourdomain.com
scm:
  url: git://github.com/rails/rails.git
  type: git
  # branch: 2-3-stable
builder:
  ruby:
    task: ci/ci_build.rb
    ruby_path: /home/webadmin/.multiruby/install/v1_8_7_160/bin/
    success: Rails build finished sucessfully
    failure: Rails build FAILED
    brokeness: (\d+) failures, (\d+) errors

• cerberus build rails_master_187

Build databases for ActiveRecord

Note: Some of this depends on having pulled a copy of Rails to the box; the easiest way is to let Cerberus do a failing build and then create the databases.

• cd ~/.cerberus/work/rails_master_187/sources/activerecord/
• mysql -uroot -e ‘grant all on *.* to rails@localhost;’
• mysql -urails -e ‘create database activerecord_unittest;’
• mysql -urails -e ‘create database activerecord_unittest2;’
• rake postgresql:build_databases

Ready to Go

Now you can run cerberus build rails_master_187 to run a single build, or cerberus build rails_master_187 force=TRUE to build even if nothing has changed in the repo, or cerberus buildall to build everything you’ve defined. Hook it up to cron if you want periodic checks of the repo.

One of the perennial code smells in many Rails applications is the use of migrations as a way to carry seed data (data that needs to be added to the database on deployment). This doesn’t work well because it’s tough to maintain and doesn’t play well with loading up the database from the schema file.

Rails 3 now has a new answer to this: a file convention and a rake task. The file is db/seeds.rb and it can contain arbitrary Ruby code, but generally it will look something like this:

cities = City.create([{ :name => 'Chicago' }, { :name => 'Copenhagen' }])
Mayor.create(:name => 'Daley', :city => cities.first)

This file then gets run by using the new db:seed rake task, or via rake db:setup which runs create, schema:load, and seed in succession.

As with some other parts of Rails, this is the simplest thing that can possibly work. For a somewhat different take (that also works in Rails 2) take a look at Michael Bleigh’s seed_fu or my own db-populate.

This morning, I’m pleased to be able to announce a new initiative that I’ve played a small part in launching: RailsBridge. The easiest way to explain the reasoning behind RailsBridge is to quote from the web site:

The RailsBridge Mission:

To create an inclusive and friendly Ruby on Rails community.

The RailsBridge Guidelines:

• First, do no harm. Then, help where you can.
• Bridge the gap from aspiring developer to contributing community member, through mentoring, teaching, and writing.
• Reach out to individuals and groups who are underrepresented in the community.
• Collaborate with other groups with similar goals.

RailsBridge is the product of intense discussions among 40 or so Rails and Ruby developers. We have quite a number of projects in the launching stage, from public Rails workshops to mentoring initiatives to some fun learning experiences for Ruby newcomers.

But the key message of RailsBridge is simple, and goes beyond any one project: the Rails community is the product of everyone who participates, and it can be a positive force for good in the world. We think there are enough Rails developers who feel this way that the somewhat negative image of Rails that is current in some parts of the web can be revised – not by claiming that we’re welcoming, but by actually acting that way.

If you’re already living in the Rails community that you want, great. But if recent events have convinced you that things could be improved, come join us! If we all work together to move things forward, then the awesome Rails codebase can be joined with an equally awesome community, and everyone wins. There’s more information on the RailsBridge home page, or you can come join us on the RailsBridge Google Group to help build a better future.

There has been some discussion in recent days in the Rails community about appropriate conference presentations, whether women feel welcome in the Rails community, and related issues. I don’t intend to review the entire mess here – you can find it if you want it. For what it’s worth, I think the original presentation was an inappropriate and regrettable mistake. However, far more disturbing to me are the reactions to the discussion on the part of some of the Rails community.

Folks, the idea that women are disproportionately underrepresented in engineering and software in general, and open source development in particular, should not be new and controversial in 2009 – anyone who cares to look can find such things as the FLOSSpols findings, or any amount of academic literature on the subject. Anyone who cares to take the time to actually talk to the women who are a part of the open source community will have no trouble getting an earful about how challenging it can be to participate.

I don’t want to make the mistake of speaking of “the open source community” as some monolithic bloc. People like Audrey Eschright, Aaron Quint, Peter Szinek, and Selena Deckelmann have written about ways to address some of the fundamental problems.

But unfortunately for me, in parallel to the public discussion there have been private ones. I can’t reveal details without breaking confidences, but suffice it to say that a significant number of Rails core contributors – with leadership (if that’s the right word) from DHH – apparently feel that being unwelcoming and “edgy” is not just acceptable, but laudable. The difference between their opinions and mine is so severe that I cannot in good conscience remain a public spokesman for Rails.

So, effective immediately, I’m resigning my position with the Rails Activists.

I realize that some people will see this as an act of prudery on my part, or a lack of a sense of humor, or some other personal failing. That’s OK, I don’t mind. Other people (who I have a good deal of respect for) have attempted to convince me that I could do more good by staying involved with the Rails power structure and trying to work from within to change things. At this point, unfortunately, I feel sufficiently outnumbered and unwelcome that that option is no longer open.

This does not mean that I will stop using Rails; on a technical level, it’s still a good fit for the projects that I work on. Nor does it mean that I will stop contributing to open source projects. But I’m no longer willing to donate a substantial portion of my free time to this particular project.

Now that Rails 2.3 is out and the core team has caught their collective breath, new features are starting to make their way into Rails. The latest: an implementation of touch for Active Record. This is in edge for both 2-3-stable and master (to be 3.0).

At its simplest, touch is just a bit of syntactic sugar that writes the current time to the updated_at or updated_on attribute of an ActiveRecord instance. This does not short-circuit the validation code, so if there are errors in the instance you’ll get ActiveRecord::RecordInvalid raised:

@order.touch

But wait, there’s more! You can specify a different attribute to touch if you’d like:

@order.touch(:shipped_at)

You can also specify touch options when defining a belongs_to relationship:

class Order
  belongs_to :customer, :touch => true
end

In this case, saving or destroying an order instance will touch the corresponding customer instance. This is useful in cases where, for example, you’re caching customers but want them invalidated from the cache when a counter cache changes. You can also specify a particular attribute on the parent model to touch:

class Order
  belongs_to :customer, :touch =>  rders_updated_at
end

I’ve spent a lot of time over the past few years working with Rails projects that were written by other people. Sometimes I’ve come on as a subcontractor to an existing codebase, sometimes I’ve taken over when another developer got bored or fired, sometimes I’ve been asked to do a code review.

Over the course of these engagements I’ve come up with a strategy for getting up and running on a new-to-me Rails codebase quickly. There aren’t any hard and fast rules; there are still a lot of variable factors. But overall, I find these guidelines useful:

Start in environment.rb and figure out which version of Rails the project needs. In my case, I’m working with everything from 1.1.6 to 2.3.2.1 at the moment, so using gem Rails is pretty much a non-starter. If a project doesn’t have vendored Rails when I get it, it will as soon as I figure out what version it wants to see. If you do vendor Rails into a shared project where others are working with gems, don’t forget to .gitignore vendor/rails.

Next I look at bringing the database up from scratch. If at all possible, I ask for a copy of a current production database to avoid doing this. If I can’t get one, I’ll start by running migrations, just to see whether the migrations have been maintained. If migrating from scratch blows up, there’s always schema.rb.

Next comes searching the code for require to see if I can figure out what gems the project needs (in rare cases, there are gems specified via the gem.config route, but so far I’m not seeing much of that). I install and upgrade any gems I can see as required, and then try to actually run the project via script/server. Usually this fails a few times as I discover missing dependencies, but I like to give it a few tries so I can have the code up and running while I explore it.

I spend a few minutes exploring vendor/plugins to see what non-gem plugins the project depends on as well. If it’s using any that I’m not familiar with, I check out the readme – assuming there is one.

With the app running, I turn to the MVC heart of things. Here, the models are my first stop. If there are a reasonable number of models (say, anything under about 30) I just start at the top and look at each one, making particular note of association declarations. I haven’t found an automatic ERD tool for Rails that I like, so I use this information to sketch out an ERD by hand, sorting out how the major entities connect to each other.

Next for me is routes.rb. Looking at this file is usually a good way to judge the sophistication of the previous developers. It also gives me some URLs to try out on the running code to see what happens. After I understand the basic routing, I’ll spend some time in controller code, looking to see if it seems unduly fat or otherwise confusing.

Finally I’ll spot check some views and helpers to see how clean the code looks. Usually I don’t try to read all the views, though, unless the first couple I touch on show me systemic problems.

By the time this process is done, I usually have a rough handle on how the code is structured. Combine that with some exploratory use of the running application on my local box, and Rails’ conventions, and I can go in and find the code that needs to be improved, evaluated, or fixed.

Rails 2.3.2.1 is out. There’s some confusion about this floating around already, so here’s the scoop as I understand it:

• The original Rails 2.3.2 gems were built including this commit from the Rails master tree, which is critical for proper functionality.
• The 2.3.2 tag in GitHub unfortunately points to this commit, which is before the critical fix.
• The net result is that rake rails:freeze:edge RELEASE=2.3.2 would freeze a bad version of Rails into your application.

To fix this, the Rails team has re-tagged the master tree at a safer spot, after the critical fix. This new tag is for release 2.3.2.1. So if you’re freezing Rails 2.3 into your applications (as opposed to running it from gems) be sure to use rake rails:freeze:edge RELEASE=2.3.2.1. That .1 makes all the difference.

If you’ve ever worked with a huge number of Active Record objects and watched your server memory at the same time, you may have noticed considerably bloat in your Rails processes. That’s because Active Record doesn’t support database cursors (alas!) so all of those records come into memory at once.

In Rails 2.3, ActiveRecord::Base is adding two methods to help with this problem: find_in_batches and each. Both of these methods return records in groups of 1000, allowing you to process one group before proceeding to the next, and keeping the memory pressure down:

find_in_batches is the basic method here:

Account.find_in_batches(:conditions => {:credit => true}) do |accts|
  accts.each { |account| account.create_daily_charges! }
end

find_in_batches takes most of the options that find does, with the exception of rder and :limit. Records will always be returned in order of ascending primary key (and the primary key must be an integer). To change the number of records in a batch from the default 1000, use the :batch_size option.

The each method provides a wrapper around find_in_batches that returns individual records:

Account.each do |account|
  account.create_daily_charges!
end

A couple of caveats: first, if you’re trying to loop through less than 1000 or so records, you should avoid the overhead of batches and use something like Account.all.each or a regular finder to get the records. Second, if the table is very active (i.e., has a constant stream of inserts and deletes), using the batch methods may miss records due to changes in the table between batches (whereas finding all records will at least give you a complete point-in-time snapshot).

I recently inherited a Rails application that used an old, old version of file_column to handle attachments. As part of bringing things up to date, I updated file_column – and then it turned out that none of the site’s thousands of images could be seen any longer. So here’s a note for anyone who ends up in the same situation.

It turns out that over the years, the file_column plugin changed the naming that it uses for the actual stored files. They used to live in folders with names like /42, and now they’re in folders with names like /0000/0042. So, if you go from old to new, and don’t do something, your attached files are not where the plugin is expecting to find them.

Now, one way to deal with this is to move all the files around. Here’s a snippet of code you can run in IRB to migrate your hard drive to the new locations, and I’m sure that’s a fine solution in many circumstances. But I had some additional constraints that made this impractical.

So instead, I set file_column back to the old naming. You’ll find the relevant code in PermanentUploadedFile#relative_path_prefix:

def relative_path_prefix
  @instance.id.to_s
  # raise RuntimeError.new("Trying to access file_column, but primary key got lost.") if @instance.id.to_s.empty?
  # File.join(*("%08d" % @instance.id).scan(/..../))
end

The commented-out code is the current version, the live code is the old version. With this change, the latest file_column will happily find files laid down by an older version.

If you’ve been following along at home, you know that memoization was added to Rails 2.2 as a shortcut for a common idiom:

def site_term_ids
  @site_term_ids || = cached_terms.split.map
     { |term| Term.find_by_query(term).id }
end

Memoization makes it possible to perform an expensive operation once, and then cache the results for future calls to the same property. What you may have missed, though, is that this isn’t baked into Active Record; it’s a separate part of Active Support. So if you’re getting mystery errors about memoization not working, be sure you’re including the right extension:

class Site < ActiveRecord::Base
  extend ActiveSupport::Memoizable

  def site_term_ids
    cached_terms.split.map
      { &#124;term&#124; Term.find_by_query(term).id }
  end
  memoize :site_term_ids

end

I got an IM this afternoon from a friend, pointing to part of the Active Record Query Interface guide:

Then you could call Client.males.all to get all the clients who are male. Please note that if you do not specify the all on the end you will get a Scope object back, not a set of records which you do get back if you put the all on the end.

Setting up a simple named scope and testing it in irb certainly seems to indicate that this passage is wrong:

Client.males.is_a?(Array) => true
Client.males.is_a?(ActiveRecord::NamedScope::Scope)
 => false

And yet, a moment’s reflection says that named scopes can’t simply be returning arrays of ActiveRecord objects: if they were, there would be no way to compose them.

As with other Rails questions, the easiest way to answer this is to fire up a text editor and dig into the Rails source – in this case, activerecord/lib/active_record/named_scope.rb, which contains the definition of the Scope object and the other named scope apparatus. There you’ll find this chunk of code:

NON_DELEGATE_METHODS = %w(nil? send object_id class extend find size
 count sum average maximum minimum paginate first last empty? any?
 respond_to?).to_set
[].methods.each do |m|
  unless m =~ /^__/ || NON_DELEGATE_METHODS.include?(m.to_s)
    delegate m, :to => :proxy_found
  end
end

So: except for a small set of explicitly-listed methods, the Scope object delegates everything to :proxy_found. And what is that? Read a bit further in the source and you’ll find out:

def proxy_found
  @found || load_found
end

def load_found
  @found = find(:all)
end

So: for any method not in its small list – say, the inspect that IRB uses to display the value of an expression – the Scope looks at its internal results array, running the find to do so if necessary. Thus we have an object that at first glance appears to be something other than what it really is. Problem solved.

One of the late-breaking changes to sneak into Rails 2.3 is localized views. If you’re staying at all up to date, you know that Rails sports a nifty new internationalization framework as of version 2.2. But in 2.3, if you want to show different views depending on the visitor’s locale, it’s quite simple.

For example, suppose you have a Pages controller with an about action. Ordinarily, this will render app/views/pages/about.html.erb. But if you set I18n.locale = :es, it will render app/views/pages/about.es.html.erb. If the localized template isn’t present, the undecorated version will be used.

But why stop there? Rails doesn’t impose any particular limits on the symbols that you can set for I18n.locale, so if you want, you can hijack this system to display different content depending on anything you want. For example, suppose you have some users who are “expert” who should see different pages. Just do something like this:

before_filter :set_expert_mode

def set_expert_mode
  I18n.locale = :expert if current_user.expert?
end

And then if you provide, say, index.expert.html.erb, that version will be displayed to expert users. If there’s no special view, they’ll just get the normal content.

Presumably you’ve seen those Web 2.0 sites that highlight random words in the marketing text to draw your attention. A little-known view helper in Rails makes this sort of thing trivial. For example, you can combine this markup in your view:

<%= highlight("Rails is a fabulous framework.
  When you start 'Riding the Rails' you'll never go back.",
  "Rails") %>

With this bit of CSS:

.highlight  { background:#ff0; }

And you get this result:

Easy enough. But wait, there’s more! You can specify an entire array of words and phrases to highlight:

<%= highlight(@spam.body, ['free', 'money', 'claim']) %>

Even better, highlight supports a :highlight option, which lets you specify a custom string to use for highlighting. The token \1 will be replaced with the text to be highlighted. This lets you change the HTML markup:

<%= highlight(@marketing_text, 'today',
  :highlighter => '<em>\1</em>') %>

This ability to insert arbitrary surrounding markup makes highlight more flexible, if you let yourself think out of the box:

<%= highlight(@memberlist, @users.map(&:name),
  :highlighter => '<a href="/users/\1">\1</a>) %>

It’s worth taking a dip into ActionView::Helpers occasionally to see what other bits of functionality are lurking that you’ve forgotten about.

You probably already know, at least vaguely, that Rails is working on JRuby compatibility (it’s most of the way there, but there are a few niggling bugs left). But did you know that Active Record also supports using a variety of databases via JDBC, which is a better fit for a JRuby-backed Rails installation than the usual sqlite or MySQL choices? As of a couple of days ago, you can easily see this in action by running the Active Record test suite. Here’s how:

Grab a copy of edge Rails. You need to have a copy recent enough that it includes this commit.

You need to have JRuby installed. For the best experience with Rails, you should grab and build JRuby from its own edge. The JRubyWiki has instructions on downloading source and building yourself. Even if you don’t know anything about java it’s pretty simple. Here’s how I did it on a Ubuntu box:

sudo aptitude install sun-java6-jdk
mkdir jruby_trunk
svn co http://svn.codehaus.org/jruby/trunk/jruby jruby
cd jruby
svn up
ant
ant jar-complete

I’ve got some aliases set up that make it easy to use JRuby as my interpreter when I want to. These go in .bash_login:

alias jr='export PATH=/home/myuser/jruby_trunk/jruby/bin:/home/myuser/.gem/jruby/1.8/bin:$ORIGPATH'
alias mr187p72='export PATH=~/.multiruby/install/1.8.7-p72/bin:$ORIGPATH'
alias mr191rc1='export PATH=~/.multiruby/install/v1_9_1_rc1/bin:$ORIGPATH'
alias mroff='export PATH=$ORIGPATH'

(Note that I have a couple of other Ruby versions installed on that box via multiruby as well).

You’ll need to install some gems to run the Rails tests. Note the special switch to install them where JRuby will see them:

jruby -S gem install mocha mysql postgres sqlite-ruby sqlite3-ruby fcgi rack jruby-openssl

To use any of the JDBC adapters, install the glue between Active Record and JDBC:

jruby -s gem install activerecord-jdbc-adapter

You have your choice of lots of JDBC adapters. Let’s install them all!

jruby -s gem install activerecord-jdbc-adapter
jruby -s gem install activerecord-jdbcderby-adapter
jruby -s gem install activerecord-jdbch2-adapter
jruby -s gem install activerecord-jdbchsqldb-adapter
jruby -s gem install activerecord-jdbcmysql-adapter
jruby -s gem install activerecord-jdbcpostgresql-adapter
jruby -s gem install activerecord-jdbcsqlite3-adapter

Time to get testing. For something easy, start with MySQL:

cd ~/rails/activerecord
jruby -S rake test_jdbcmysql

With edge Rails this morning, you’ll get 19 test failures out of 1812 tests and 6032 assertions – not too bad. How about something more java-like, the hsqldb database engine?

cd ~/rails/activerecord
jruby -S rake test_jdbchsqldb

I’m getting around 600 test failures with this one at the moment.

Similar testing tasks exist for all of the other adapters I listed above. With these bits of glue coming into play, and the Rails and JRuby teams working together, we’re making our way towards a point where Rails on JRuby is a robustly-supported solution.

As many of you know, I’ve been increasingly active in the Rails community in the last year. Working on projects like the Rails Guides, This Week in Edge Rails, the Rails 2.2 Release Notes, and the official continuous integration server for Rails have thrown me into contact with the Rails core team and other prominent folks in the Rails community. A few of us have gotten to talking, and discovered that we have similar concerns about Rails and the way it gets used in the real world.

As a result, I’m happy to announce the launch of ActionRails, a new consulting firm jointly owned by Pratik Naik, Michael “Koz” Koziarski, and myself. We’re not trying to be another group of developers who bang out Rails applications from start to finish (though any of us will be happy to talk to you individually about such projects). Rather, we’re interested in helping customers complete successful Rails deployments and helping existing teams improve their Rails skills.

ActionRails is already working for a few select clients, helping them stay on top of the Rails universe, reviewing code, offering course corrections, and evaluating application architecture (among other things). You can hop over to our site for more information, or just drop me a line and I’ll be glad to chat about what we can do for your team.

I’ve recently been working to integrate Amazon Simple Pay into an application. In general, this has been pretty straightforward, thanks to the simplepay gem, which was recently bumped to version 0.2.0. (Amazon also has their own Ruby sample for ASP integration, which came in handy for a first smoke test – though unfortunately at the moment the Amazon site is having trouble delivering that file).

For the most part, using the simplepay gem is quite straightforward. There’s a helper to build a payment form:

Click here to send us $120 every year
<%= simplepay_form_for(:subscription, {
  :amount                 => 120.00,
  :description            => "Annual Subscription",
  :recurring_frequency    => "1 year",
  :abandon_url            =>  amaz_abandon_customers_url,
  :collect_shipping_address => false,
  :immediate_return       =>  false,
  :ipn_url                =>  amaz_ipn_customers_url,
  :process_immediately    =>  true,
  :reference_id           =>  @customer.id,
  :return_url             =>  amaz_return_customers_url
}) %>

When Amazon executes callbacks, it sends a signature parameter along so that you can verify that the request is legitimate. This is the only spot where I ran into a little gotcha. The simplepay gem includes a helper to validate the parameters of a request to see if they match the signature, but you have to be sure to take out the parameters that Rails inserts before you call it:

def amaz_ipn
    params.delete('controller')
    params.delete('action')
     if AWS::SimplePay::IpnValidator.valid_ipn_post?
(Simplepay.aws_secret_access_key, params)
      # save the results somewhere
      render :action => 'show'
    else
      logger.info "bad request from Amazon Simple Pay"
      render :nothing => true, :status => 200
    end
  end

Also note that you should send back a 200 response even if you’re unable to verify the signature. This will prevent Amazon from sending the same request to you over and over again.

Over the years, ActionController::Base#render has grown quite a few options:

render :action => "edit"
render :template => "products/show"
render :file => "/u/apps/other/warning.html"
render :nothing => true
render :inline => "<% @product.name %>"
render :update {&#124;p&#124; p.replace_html "A", "Error!"}
render :text => "OK"
render :json => @product
render  ml => @product
render :js => "alert('error!');"

Those options will still be in Rails 2.3, but the first three have a simpler alternative now:

render "edit"
render "products/show"
render "/u/apps/other/warning.html"

Rails will figure out whether you’re rendering an action, template, or file, and do the right thing. In addition, if you like symbols, you can render an action via a symbol:

render :edit

Sometimes the fact that I don’t know all the ins and outs of Ruby, coupled with habits carried over from other languages, turns around to bite me. Here’s a console session demonstration of the latest problem I had:

>> a = b = 2
=> 2
>> a = 3
=> 3
>> b
=> 2
>> a = b = []
=> []
>> a << 2
=> [2]
>> b
=> [2]
>> a = []; b = []
=> []
>> a << 2
=> [2]
>> b
=> []

So, trying to initialize multiple arrays and multiple integers doesn’t work the same. Live and learn.

By now, you all know that you need to use attr_accessible to protect your application from having any old Active Record attribute changed by a malicious user. I find myself, in the course of consulting, looking at a great number of existing applications, and I’ve grown tired of having to open up every model and look. So, let’s be a bit smart: here’s a rake task that you can drop in any existing application to do the audit for you:

namespace :utility do

  desc 'Find models that are not using attr_accessible'
  task :audit_attr_accessible => :environment do

    all_models = Dir.glob(
      File.join(Rails.root, 'app', 'models', '*.rb')
      ).map{|path| path[/.+\/(.+).rb/,1] }
    ar_models = all_models.select{|m|
       m.classify.constantize < ActiveRecord::Base}

    ar_models.each do &#124;model&#124;
      model_class = model.classify.constantize
      if model_class.send("attr_accessible").empty? &&
         model_class.send("attr_protected").empty?
        puts model_class.class_name +
          " allows unprotected mass assignment"
      end
    end

  end

end

(Hat tip Matt for the snippet to find all models). ]]>