Easy way to speed up your automated tests
Just a note before we start, this will be a Ruby on Rails centric post with code samples in written in Ruby. However, the concepts in this post can be applied to any tech stack, whether that would be Laravel, Django, Phoenix, and even mobile platforms like iOS and Android.
Most software developers know that interacting with a database can be expensive. For example, one expensive query on a large dataset can seriously hamper your application's performance. In the case of Ruby on Rails, this may include eager loading associations and avoiding ActiveRecord
altogether and writing raw SQL queries. However, in many of the codebases I work on, I see that many developers do not adhere to this principle when writing automated tests.
For example, let's say that we're working on a Rails project for managing a list of books, the authors, and publishers. We have three models: Book
, Author
, and Publisher
. Book
belongs to Author
and Publisher
, and both Publisher
and Author
has many books. Below are the three hypothetical models.
Book
class Book < ApplicationRecord
belongs_to :author
belongs_to :publisher
validates :name, presence: true
validates :author, presence: true
validates :book, presence: true
end
Author
class Author < ApplicationRecord
has_many :books
has_many :publishers, through: :books
validates :name, presence: true
end
Publisher
class Publisher < ApplicationRecord
has_many :books
has_many :authors, through: :books
validates :name, presence: true
end
And now, for the purposes of writing our tests, let's say that we're using the FactoryGirl gem have three factories set up for each of the models defined above.
Book Factory
FactoryGirl.define do
factory :book do
author
publisher
name { Faker::Lorem.word }
end
end
Author Factory
FactoryGirl.define do
factory :author do
name { Faker::Name.name }
end
end
Publisher Factory
FactoryGirl.define do
factory :publisher do
name { Faker::Company.name }
end
end
Okay, all of our models and factories are set up now. We are finally ready to write our tests!
First common mistake: Creating unnecessary records
I'll give an example of a BooksController
with the index
method and then provide a "typical" test that I see often. This is the "bad" test.
BookController:
class BooksController < ApplicationController
def index
@books = Book.all
end
end
BookControllerTest
require "test_helper"
class BooksControllerTest < ActionDispatch::IntegrationTest
setup do
10.times { create(:book) }
end
test "GET index" do
get books_url
assert_response :success
end
end
Here, we have a pretty simple BooksController
where we load up all of the books in the index
method. And in our BooksControllerTest
, we are creating 10 books and then testing the index
route of the BooksController
. This test will pass.
However... this test is not optimally written. Our Book factory has two belongs_to
associations: Author and Publisher. In the setup block of our BooksControllerTest
, where we loop 10 times to create 10 books:
setup do
10.times { create(:book) }
end
This line of code will create not only 10 books, but a new set of authors and publisher each time the loop is executed. This line of code is devious as it looks like it'll only insert 10 books into your test database. But in reality, it's inserting 30 records (1 author and 1 publisher for each book), which is 3 times more records than the test needs.
These are unnecessary database writes that may seem innocent when your test suite is small, but it can seriously slow down your overall test suite as you write more tests.
The optimal way to write the BooksControllerTest
should be:
require "test_helper"
class BooksControllerTest < ActionDispatch::IntegrationTest
setup do
@author = create(:author)
@publisher = create(:publisher)
10.times { create(:book, author: @author, publisher: @publisher) }
end
test "GET index" do
get books_url
assert_response :success
end
end
Modifying your tests this way, where you specify the author and the publisher that each individual book will belong to will ensure that you do not create unnecessary records (in this case, new set of authors and publishers) in our database, significantly improving performance.
In one of the projects I've worked on, just implementing this reduced the time needed to run the test suite from something like 13 minutes down to 7 minutes. You can see how this simple change can speed up the test suite by a lot.
Second common mistake: Writing to the database to check validations
In our example, all of our models validates the presence of the name
field. Let's write a test to make sure our models are adhering to this validation. I'll show an example with the author model.
require "test_helper"
class AuthorTest < ActiveSupport::TestCase
test "invalid when name is not present" do
author = create(:author, name: nil)
assert author.new_record?
assert !author.valid?
end
test "valid when name is present" do
author = create(:author, name: "Chris Jeon")
assert !author.new_record?
assert author.valid?
end
end
Above are two simple validation tests that check for the presence of the name
field in the Author
model. The above tests will pass, however they are not efficient. The reason for this is the same as the first common mistake which is unnecessary writes to the database.
You don't have to actually write to the database to check the model validations. You can simply build a new instance of the model and then check the validity from there. Let's rewrite this test to run faster.
require "test_helper"
class AuthorTest < ActiveSupport::TestCase
test "name must be present" do
author = build(:author, name: nil)
assert !author.valid?
author.name = "Chris Jeon"
assert author.valid?
end
end
Calling the build
method on FactoryGirl will instantiate a random Author
model rather than creating a new author and writing that author to the database. Also, rather than writing to the database to check the validity of the instance of the model, you can just call valid?
on the model to see if the model is valid or not. This approach saves unnecessary writes to the database which in turn speeds up your test suite.
Closing Note...
You can see that neither of the two examples above are difficult concepts to grasp. It's just avoiding unnecessary database operations in your tests. These simple adjustments can make drastic performance improvements to the speed of your test suite.
Also as I mentioned at the beginning of this post, while this post was Rails centric, the same concepts can and should be applied to whatever language/framework you're using.