My ugly mug

Hi, I'm Thomas Cannon!

I work on Freckle with Amy Hoy and Thomas Fuchs, and I teach people how to add an extra layer of polish to their work. Previously I worked for BMW and Clemson University, and have a bunch of smaller apps I tinker with on the side. Follow me on Twitter!

Porting your test suite to MiniTest, part 2: how to port your tests

This is the last part of a series I wrote about porting Freckle’s test suite to MiniTest. Missed the first part?

About a month ago we ported Freckle’s test suite from test-spec to MiniTest. Doing so not only made our test suite faster, but it also paved the road for upgrading Rails in the future. Plus, we now have Nyancat running our tests

While porting over a test suite is not a technically challenging problem, it is certainly daunting. You’re changing thousands of lines of code, and sometimes you have to rethink tests entirely. And to make matters worse, your tests don’t just go from “green” to “red”. Instead, they go from “green” to “OH GOD, WHY?! NOTHING WORKS, EVERYTHING IS ON FIRE!”, which can be pretty demoralizing. But thankfully with some patience, a few regular expressions, and by divying up the work; the process becomes relatively painless.

Previously, I talked about some of the benefits of porting your tests to MiniTest and using the “spec-assert” style. Now I’m going to talk about the nitty-gritty of actually getting the test suite ported. First up are some general workflow tips to keep in mind when porting. After that I’ll talk about some of the technical details and solutions to “gotchas” you’re bound to run into. Finally, I’ve included some regular expressions that make porting a breeze. Just plug them into your text editor, and find-and-replace away!

How to port your test suite

Use the “Ruby on Rails Testing Cheat Sheet”

Seriously, this cheat sheet (written by my friend Eric Steele) saved me hours of time: You should buy his book, What do I test?, because it’s fantastic and has the best DHH testimonial ever:

Only work on one type of test at a time

Functional tests are different than unit tests, which are different than integration tests, and so on. By only focusing on one type of test at a time it’s easier to get in a groove and crank them out. And splitting the team up to work on different types of tests makes the porting faster and makes sure you don’t step on each other’s toes.

Only work on one file at a time

The quickest way to be overwhelmed and not make any progress is to run:

rake test

Instead, work on files in isolation:

ruby test/unit/user_test.rb

When all the tests in a file pass, move onto the next one. You’ll probably run into errors caused by running all the tests from rake test, but wait until you’ve fixed them individually before you tackle that.

Pick some good music

Porting these tests can take a while and it can be pretty monotonous, so you’ll want to find something that will stop you from chanting “Redrum”. I ended up listening to the music from Cowboy Bebop when I was porting tests, and it made the process enjoyable.

Regular expressions all the way down

Seriously, the best way port a test suite is to heavily use regular expressions and find-and-replace. Your tests should have a (relatively) standard format, which makes them prime targets for some well-crafted regexes. Here are some regular expressions to get your started.

Technical details

We ran into a few technical hiccups when porting tests from test-spec and MiniTest. Thankfully they were all pretty easy to solve.

Setting up MiniTest

To set up your test suite to use MiniTest, you’ll have to update your Gemfile and test_helper.rb.

Here are the gems you’ll probably need to install:

You’ll also have to update test_helper.rb to include the following:

require 'action_view/test_case'
require 'minitest-spec-rails'
require 'test_help'
require 'webmock/minitest' #if you're using webmock
require 'test_helper/add_allow_switch' #if you used test-specs `add_allow_switch`
require 'test_helper/deprecated_helpers' #if you use the deprecated helpers gist below
require 'test_helper/fix_nested_describes_in_controller_tests' #for Rails 2.3 LTS only
require 'test_helper/test_assertions' #adds a useful assertions

If you were using test-spec’s add_allow_switch helper, you’ll want to add the following gist to your test helpers:

You can add the following test helper, which will provide helpful deprecation warnings for some test-spec helpers:

The following test helper fixes a bug with nested describes blocks in functional tests for Rails 2.3:

The following test helper adds a lot of helpful asserts to your test suite, like ordered array and set comparison (assert_equal_list and assert_equal_set respectively), ActiveRecord attribute validation, and layout assertions:

Wrap your tests in a class, even if you’re using a describe block

minitest-spec-rails doesn’t properly recognize non-model classes, which will cause random errors in your test suite. Thankfully, you can nest describe blocks inside of class declarations, so just wrapping each test file with the appropriate class declaration ensures nothing breaks:

# Unit tests for models
class UserTests < ActiveSupport::TestCase
  describe User, "permissions" do
    it "should have permissions to view" do
      assert user.has_permission?(:view)

# Functional tests (app/controllers)
class UsersControllerTest < ActionController::TestCase
  describe UsersController, "#show" do
    it "should a user" do
      get :show, :id => users(:bob).id
      assert_equal_set users(:bob), assigns(:user)

# Integration Test
class UserIntegrationTest < ActionController::IntegrationTest
  describe "Getting Users" do
    it "should get a user" do
      get "/users/#{users(:bob).id}"

      assert_select '#name', :text => "Bob Kerbin"

# Unit test for non-model class (e.g: `lib/`)
class NameParsingTest < ActiveSupport::TestCase
  describe NameParsing do
    it "should be able to intelligently parse a first name" do
      assert_equal "Bob", NameParsing.parse_name("Bob Kerbin").first_name

before and after blocks execute for every test

There are no longer different before/after :each or before/after :all blocks. You only have before/after do ... end, which runs for every test in the scope.

Performance testing in Rails 2.3 LTS is broken without a monkeypatch

Rails 2.3’s performance testing framework doesn’t like MiniTest when is used instead of Test::Unit. This problem was fixed in Rails 3+, and I’ve written a monkey-patch that will add MiniTest support to ActiveSupport::Testing::Performance:

Brush up on your Regex-fu

The biggest part of porting from “spec-shoulda” to “spec-assert” was rewriting the assertions in a test. Thankfully these should be fairly standard, which means you can use regular expressions to quickly knock out tests. And since the rest of your team is also porting tests, sharing the regular expresisons you’ve created will make everyone’s job easier

Tip: go from most → least specific regular expression

Sometimes the general-case regular expressions will gobble up a test that would be better expressed with a different assertion. For example, consider the following test:

user.valid?.should == true

This test should be rewritten as:

assert user.valid?

But when you use the x.should == y replacement regexes, you’ll end up overwriting the test to:

assert_equal true, user.valid?

If you modify the regex to check for x.should == true and replace it with assert x, you’ll keep your tests nice and clean.

Here are some few regular expressions to get your started

That’s it!

Porting your test suite to MiniTest and rewriting from “spec-shoulda” to “spec-assert” styles can be a long process, but it has some significant benefits. It makes your test suite run faster by reducing overhead. It also makes your tests easier to scan and understand, which is critical during code reviews and introducing new members to the team. Finally, it gets you closer to Rails’ defaults, which makes it easier to upgrade when the time comes.