Easy way to speed up your automated tests

Just a note before we start, this will be a Ruby on Rails centric post with code samples in written in Ruby. However, the concepts in this post can be applied to any tech stack, whether that would be Laravel, Django, Phoenix, and even mobile platforms like iOS and Android.

Most software developers know that interacting with a database can be expensive. For example, one expensive query on a large dataset can seriously hamper your application's performance. In the case of Ruby on Rails, this may include eager loading associations and avoiding ActiveRecord altogether and writing raw SQL queries. However, in many of the codebases I work on, I see that many developers do not adhere to this principle when writing automated tests.

For example, let's say that we're working on a Rails project for managing a list of books, the authors, and publishers. We have three models: Book, Author, and Publisher. Book belongs to Author and Publisher, and both Publisher and Author has many books. Below are the three hypothetical models.

Book

class Book < ApplicationRecord
  belongs_to :author
  belongs_to :publisher

  validates :name, presence: true
  validates :author, presence: true
  validates :book, presence: true
end

Author

class Author < ApplicationRecord
  has_many :books
  has_many :publishers, through: :books

  validates :name, presence: true
end

Publisher

class Publisher < ApplicationRecord
  has_many :books
  has_many :authors, through: :books

  validates :name, presence: true
end

And now, for the purposes of writing our tests, let's say that we're using the FactoryGirl gem have three factories set up for each of the models defined above.

Book Factory

FactoryGirl.define do
  factory :book do
    author
    publisher
    name { Faker::Lorem.word }
  end
end

Author Factory

FactoryGirl.define do
  factory :author do
    name { Faker::Name.name }
  end
end

Publisher Factory

FactoryGirl.define do
  factory :publisher do
    name { Faker::Company.name }
  end
end

Okay, all of our models and factories are set up now. We are finally ready to write our tests!

First common mistake: Creating unnecessary records

I'll give an example of a BooksController with the index method and then provide a "typical" test that I see often. This is the "bad" test.

BookController:

class BooksController < ApplicationController
  def index
    @books = Book.all
  end
end

BookControllerTest

require "test_helper"

class BooksControllerTest < ActionDispatch::IntegrationTest
  setup do
    10.times { create(:book) }
  end

  test "GET index" do
    get books_url
    assert_response :success
  end
end

Here, we have a pretty simple BooksController where we load up all of the books in the index method. And in our BooksControllerTest, we are creating 10 books and then testing the index route of the BooksController. This test will pass.

However... this test is not optimally written. Our Book factory has two belongs_to associations: Author and Publisher. In the setup block of our BooksControllerTest, where we loop 10 times to create 10 books:

setup do
  10.times { create(:book) }
end

This line of code will create not only 10 books, but a new set of authors and publisher each time the loop is executed. This line of code is devious as it looks like it'll only insert 10 books into your test database. But in reality, it's inserting 30 records (1 author and 1 publisher for each book), which is 3 times more records than the test needs.

These are unnecessary database writes that may seem innocent when your test suite is small, but it can seriously slow down your overall test suite as you write more tests.

The optimal way to write the BooksControllerTest should be:

require "test_helper"

class BooksControllerTest < ActionDispatch::IntegrationTest
  setup do
    @author = create(:author)
    @publisher = create(:publisher)
    10.times { create(:book, author: @author, publisher: @publisher) }
  end

  test "GET index" do
    get books_url
    assert_response :success
  end
end

Modifying your tests this way, where you specify the author and the publisher that each individual book will belong to will ensure that you do not create unnecessary records (in this case, new set of authors and publishers) in our database, significantly improving performance.

In one of the projects I've worked on, just implementing this reduced the time needed to run the test suite from something like 13 minutes down to 7 minutes. You can see how this simple change can speed up the test suite by a lot.

Second common mistake: Writing to the database to check validations

In our example, all of our models validates the presence of the name field. Let's write a test to make sure our models are adhering to this validation. I'll show an example with the author model.

require "test_helper"

class AuthorTest < ActiveSupport::TestCase
  test "invalid when name is not present" do
    author = create(:author, name: nil)
    assert author.new_record?
    assert !author.valid?
  end

  test "valid when name is present" do
    author = create(:author, name: "Chris Jeon")
    assert !author.new_record?
    assert author.valid?
  end
end

Above are two simple validation tests that check for the presence of the name field in the Author model. The above tests will pass, however they are not efficient. The reason for this is the same as the first common mistake which is unnecessary writes to the database.

You don't have to actually write to the database to check the model validations. You can simply build a new instance of the model and then check the validity from there. Let's rewrite this test to run faster.

require "test_helper"

class AuthorTest < ActiveSupport::TestCase
  test "name must be present" do
    author = build(:author, name: nil)
    assert !author.valid?

    author.name = "Chris Jeon"
    assert author.valid?
  end
end

Calling the build method on FactoryGirl will instantiate a random Author model rather than creating a new author and writing that author to the database. Also, rather than writing to the database to check the validity of the instance of the model, you can just call valid? on the model to see if the model is valid or not. This approach saves unnecessary writes to the database which in turn speeds up your test suite.

Closing Note...

You can see that neither of the two examples above are difficult concepts to grasp. It's just avoiding unnecessary database operations in your tests. These simple adjustments can make drastic performance improvements to the speed of your test suite.

Also as I mentioned at the beginning of this post, while this post was Rails centric, the same concepts can and should be applied to whatever language/framework you're using.