by Martin Führlinger, Backend Engineer
Within the backend workforce we normally attempt to automate issues. Subsequently we’ve got tons of exams to confirm correctness of our code in our gems and companies. Automated exams are executed a lot sooner and with a lot increased protection than any tester can do manually in an analogous time. Through the years numerous performance has been added, and because of this, numerous exams have been added. This led to our take a look at suites changing into slower over time. For instance, we’ve got a service the place round 5000 exams take about Eight minutes. One other service takes about 15 minutes for round 3000 exams. So why is service A so quick and repair B so sluggish?
On this weblog publish I’ll present some unhealthy examples of find out how to write a take a look at, and find out how to enhance automated testing instruments to make exams sooner. The Runtastic backend workforce normally makes use of `rspec` together with `factory_bot` and jruby.
The Take a look at File Instance
The next code exhibits a small a part of an actual instance of a take a look at file we had in our take a look at suite. It creates some customers and tries to seek out them with the UserSearch use case.
describe Customers::UseCase::UserSearch::ByAnyEmail do
describe "#run!" do
topic { Customers::UseCase::UserSearch::ByAnyEmail.new(search_criteria).run! }
let(:current_user_id) { nil }
let(:search_criteria) { double(question: question, dimension: dimension, quantity: quantity, current_user_id: current_user_id) }
let(:default_photo_url) { "#{Rails.configuration.companies.runtastic_web.public_route}/belongings/consumer/default_avatar_male.jpg" }
def expected_search_result_for(consumer)
UserSearchResultWrapper.new(consumer.attributes.merge("avatar_url" => default_photo_url))
finish
shared_examples "discover customers by electronic mail" do
it { count on(topic.class).to eq UserSearchResult }
it { count on(topic.customers).to return_searched_users expected_users }
it { count on(topic.more_data_available).to eq more_data? }
finish
let!(:s_m_user) { FactoryBot.create :consumer, electronic mail: "s.m@mail.com" }
let(:runner_gmail_user) { FactoryBot.create :consumer, google_email: "runner@gmail.at" }
let!(:su_12_user) { FactoryBot.create :consumer, electronic mail: "su+12@gmx.at" }
let(:su_12_google_user) { FactoryBot.create :consumer, google_email: "su+12@gmx.at" }
let!(:user_same_mail) do
FactoryBot.create :consumer, electronic mail: "consumer@rt.com", google_email: "consumer@rt.com”
finish
let!(:combined_user) do
FactoryBot.create :consumer, electronic mail: "user1@rt.at", google_email: "user1@google.at"
finish
let!(:johnny_gmail_user) { FactoryBot.create :consumer, google_email: "johnny@gmail.com" }
let!(:jane_user) { FactoryBot.create :consumer, electronic mail: "jane@electronic mail.at" }
let!(:zorro) { FactoryBot.create :consumer, electronic mail: "zorro@instance.com" }
earlier than do
FactoryBot.create(:consumer, google_email: "jane@electronic mail.at").faucet do |u|
u.update_attribute(:deleted_at, 1.day.in the past)
finish
runner_gmail_user
su_12_google_user
finish
context "the question is '123'" do
it_behaves_like "discover customers by electronic mail" do
let(:dimension) { 4 }
let(:quantity) { 1 }
let(:question) { [123] }
let(:expected_users) { [] }
let(:more_data?) { false }
finish
finish
context "the question accommodates invalid emails" do
it_behaves_like "discover customers by electronic mail" do
let(:question) do
["s.m@mail.com", "su+12gmx.at", "", "'", "johnny@gmail.com"]
finish
let(:dimension) { 50 }
let(:quantity) { 1 }
let(:expected_users) do
[
expected_search_result_for(s_m_user),
expected_search_result_for(johnny_gmail_user)
]
finish
let(:more_data?) { false }
finish
finish
finish
finish
So let’s analyze the take a look at: It has a topic, which signifies what to check. On this case: Run the use case and return the end result. It defines a shared example which accommodates the precise exams. These shared examples assist as a result of the exams are grouped collectively and they are often reused. This manner it’s potential to simply arrange the exams with totally different parameters and name the instance through the use of it_behaves_like. The take a look at above accommodates some consumer objects created with let and a earlier than block, which known as earlier than every take a look at. The it-block accommodates two contexts to explain the setup and calls the shared instance as soon as per context. So mainly this take a look at runs 6 exams (Three exams within the shared_example, referred to as twice). Operating them regionally on my laptop computer outcomes on this:
Customers::UseCase::UserSearch::ByAnyEmail #run! the question is '123' behaves like discover customers by electronic mail ought to return searched customers ought to eq false ought to eq UserSearchResult the question accommodates invalid emails behaves like discover customers by electronic mail ought to eq UserSearchResult ought to eq false ought to return searched customers #<UserSearchResultWrapper:0x7e9d3832 @avatar_url="http://localhost.runtastic.com:3002/belongings/consumer/def....jpg", @country_id=nil, @gender="M", @id=51, @last_name="Doe-51", @first_name="John", @guid="ab c51"> and #<UserSearchResultWrapper:0x33c8a528 @avatar_url="http://localhost.runtastic.com:3002/belongings/consumer/def....jpg", @country_id=nil, @gender="M", @id=55, @last_name="Doe-55", @first_name="John", @guid="abc55"> Completed in 34.78 seconds (information took 20.66 seconds to load) 6 examples, zero failures
So about 35 seconds for six exams.
Let vs Let!
As you may see, we’re utilizing let! and let. The distinction between these two strategies is, that let! at all times executes, and let solely executes if the reference is used. Within the above instance:
let!(:s_m_user) let(:runner_gmail_user)
“s_m_user” is created at all times, “runner_gmail_user” is created provided that used. So the above let! usages are creating 7 customers for the exams.
Earlier than Block
The before block can also be executed each time earlier than the take a look at. If nothing is handed to the earlier than methodology, it defaults to :every. The above earlier than block creates a consumer, and references 2 different customers, which then instantly are created, too.
So we’re creating 10 customers for every take a look at.
rspec-it-chains
As each it is a single take a look at, the shared instance accommodates Three single exams. Each take a look at will get a clear state, so the customers are created once more for every take a look at. Having a number of it blocks one after one other, referring to the identical topic, someway appears to be like like a sequence.
Take a look at setup
What do the exams really do? The primary one passes a web page dimension of Four with a question “123” to the search use case and expects, as no consumer has 123 within the electronic mail attribute, no customers to be discovered.
context "the question is '123'" do
it_behaves_like "discover customers by electronic mail" do
let(:dimension) { 4 }
let(:quantity) { 1 }
let(:question) { [123] }
let(:expected_users) { [] }
let(:more_data?) { false }
finish
Finish
So we’re creating Three instances (3 it blocks) 10 customers however count on no consumer to be discovered.
The second context passes a number of the emails, and a few invalid ones into the search, and count on 2 customers to be discovered.
context "the question accommodates invalid emails" do
it_behaves_like "discover customers by electronic mail" do
let(:question) do
["s.m@mail.com", "su+12gmx.at", "", "'", "johnny@gmail.com"]
finish
let(:dimension) { 50 }
let(:quantity) { 1 }
let(:expected_users) do
[
wrap(s_m_user),
wrap(johnny_gmail_user)
]
finish
let(:more_data?) { false }
finish
finish
So we’re creating Three instances 10 customers to have the ability to discover 2 of them in a single take a look at and get the fitting flag in one other take a look at.
Having a better have a look at the shared_example:
it { count on(topic.class).to eq UserSearchResult }
it { count on(topic.customers).to return_searched_users expected_users }
it { count on(topic.more_data_available).to eq more_data? }
you may see that the primary one just isn’t even anticipating something user-related to be returned. It simply expects the use-case to return a selected class. The second really exams if the end result accommodates the customers we need to discover. The third it block checks if the more_data_available flag is about correctly.
General, we’ve got 6 exams, needing 35 seconds to run, creating 10 customers for every take a look at (60 customers fully) and calling the topic 6 instances, and we mainly solely look forward to finding 2 customers as soon as.
Clearly, this may be improved.
Enchancment
To start with, let’s eliminate the it chain, mix it inside one it block.
shared_examples "discover customers by electronic mail" do it "returns consumer information" do count on(topic.class).to eq UserSearchResult count on(topic.customers).to return_searched_users expected_users count on(topic.more_data_available).to eq more_data? finish finish
Combining it blocks is sensible in the event that they normally take a look at an analogous factor (as above). For instance, doing a request and anticipating some response physique and standing 200 doesn’t must be two separate exams. Combining two it blocks which take a look at one thing totally different, nonetheless, doesn’t make sense, reminiscent of exams for the response code of a request and if that request saved the information accurately within the database.
This ends in the exams ending inside ~ 15 seconds, solely 2 examples.
The following step is to not create the customers if they don’t seem to be wanted. Subsequently let’s swap to let as an alternative of let!. Additionally take away the earlier than block as it’s, and solely create some correct quantity of customers mandatory for the take a look at. The exams appear to be this in finish:
describe Customers::UseCase::UserSearch::ByAnyEmail do
describe "#run!" do
topic { Customers::UseCase::UserSearch::ByAnyEmail.new(search_criteria).run! }
let(:current_user_id) { nil }
let(:search_criteria) { double(question: question, dimension: dimension, quantity: quantity, current_user_id: current_user_id) }
let(:default_photo_url) { "#{Rails.configuration.companies.runtastic_web.public_route}/belongings/consumer/default_avatar_male.jpg" }
def expected_search_result_for(consumer)
UserSearchResultWrapper.new(consumer.attributes.merge("avatar_url" => default_photo_url))
finish
shared_examples "discover customers by electronic mail" do
it "return consumer information" do
count on(topic.class).to eq UserSearchResult
count on(topic.customers).to return_searched_users expected_users
count on(topic.more_data_available).to eq more_data?
finish
finish
let(:s_m_user) { FactoryBot.create :consumer, electronic mail: "s.m@mail.com" }
let(:runner_gmail_user) { FactoryBot.create :consumer, google_email: "runner@gmail.at" }
let(:su_12_user) { FactoryBot.create :consumer, electronic mail: "su+12@gmx.at" }
let(:su_12_google_user) { FactoryBot.create :consumer, google_email: "su+12@gmx.at" }
let(:user_same_mail) do
FactoryBot.create :consumer, electronic mail: "consumer@rt.com", google_email: "consumer@rt.com"
finish
let(:combined_user) do
FactoryBot.create :consumer, electronic mail: "user1@rt.at", google_email: "user1@google.at"
finish
let(:johnny_gmail_user) { FactoryBot.create :consumer, google_email: "johnny@gmail.com" }
let(:jane_user) { FactoryBot.create :consumer, electronic mail: "jane@electronic mail.at", fb_proxied_email: "jane@fb.at" }
let(:zorro) { FactoryBot.create :consumer, electronic mail: "zorro@instance.com" }
let(:deleted_user) do
FactoryBot.create(:consumer, google_email: "jane@electronic mail.at").faucet do |u|
u.update_attribute(:deleted_at, 1.day.in the past)
finish
finish
context "the question is '123'" do
earlier than do
s_m_user
finish
it_behaves_like "discover customers by electronic mail" do
let(:dimension) { 4 }
let(:quantity) { 1 }
let(:question) { [123] }
let(:expected_users) { [] }
let(:more_data?) { false }
finish
finish
context "the question accommodates invalid emails" do
earlier than do
s_m_user
su_12_user
johnny_gmail_user
finish
it_behaves_like "discover customers by electronic mail" do
let(:question) do
["s.m@mail.com", "su+12gmx.at", "", "'", "johnny@gmail.com"]
finish
let(:dimension) { 50 }
let(:quantity) { 1 }
let(:expected_users) do
[
expected_search_result_for(s_m_user),
expected_search_result_for(johnny_gmail_user)
]
finish
let(:more_data?) { false }
finish
finish
finish
finish
And lead to
Customers::UseCase::UserSearch::ByAnyEmail #run! the question is '123' behaves like discover customers by electronic mail return consumer information the question accommodates invalid emails behaves like discover customers by electronic mail return consumer information Completed in 8.16 seconds (information took 22.34 seconds to load) 2 examples, zero failures
As you may see, I do create customers, even when I don’t count on them to be within the end result, to show the correctness of the use case. However I don’t create 10 per take a look at, only one and three. A number of the above customers usually are not created (or used) in any respect now, however as the unique take a look at file accommodates extra exams, which in the long run want them once more for different contexts, I stored them within the instance too.
So now we solely create Four customers, as an alternative of 60. By simply adapting the code a bit, we’ve got the identical take a look at protection with solely 2 exams as an alternative of 6, and solely needing Eight as an alternative of 35 seconds, which is 77% much less time.
FactoryBot: create vs construct vs attribute_for
As you may see above, we’re utilizing FactoryBot closely to create objects through the exams.
let(:consumer) { FactoryBot.create(:consumer) }
This creates a brand new consumer object as quickly as `consumer` is referenced within the exams. The downside of this line is that it actually creates the consumer within the database, which is fairly usually not mandatory. The higher method, if relevant, can be to solely construct the item with out storing it:
let(:consumer) { FactoryBot.construct(:consumer) }
Clearly this doesn’t work should you want the item within the database, as for the take a look at instance above, however that extremely is dependent upon the take a look at. One other much less recognized function of FactoryBot is to create solely the attributes for an object, represented as hash.
let(:user_attrs) { FactoryBot.attributes_for(:consumer) }
This is able to create a hash containing the attributes for a consumer. It doesn’t even create a Consumer object, which is even sooner than construct.
A potential easy take a look at can be:
describe Consumer do
50.instances do
topic { FactoryBot.create(:consumer) }
it { count on(topic.has_first_login_dialog_completed).to eq(false) }
finish
finish
Because the has_first_login_dialog_completed methodology solely wants some attributes set on a consumer, regardless of whether it is saved in a database, a construct can be a lot sooner than a create, operating the take a look at 100 instances to additionally use the impact of the just-in-time compiler of the used jruby interpreter. This manner the actual distinction between create and construct is extra seen. So switching from .create to .construct saves about 45% of the execution time.
Completed in 1 minute 1.61 seconds (information took 23.Four seconds to load) 100 examples, zero failures
Completed in 34.87 seconds (information took 21.69 seconds to load) 100 examples, zero failures
Abstract
So easy enhancements within the exams can result in a pleasant efficiency enhance operating them.
- Keep away from it-chains if the exams correlate to one another
- Keep away from let! in favor of let, and create the objects inside earlier than blocks when mandatory
- Keep away from earlier than blocks creating numerous stuff which will not be mandatory for all exams
- Use FactoryBot.construct as an alternative of .create if relevant.
Keep watch over your test-suite and don’t hesitate to take away duplicate exams, perhaps already out of date exams. As (in our case) the exams are operating earlier than each merge and on each commit, attempt to maintain your take a look at suite quick.
***
