Automated tests replace tedious manual testing whenever you change your code with nearly instant feedback¹. Automated tests can also serve as a living documentation of your system and its behavior.

To earn these benefits, you need to write the tests with their reader in mind. The reader – quite often you – must understand the test’s prerequisites, purpose, and expected outcomes. In other words, we have to write brief, explicit, and predictable tests, so maintaining them is sustainable in the long run.

RSpec is a testing framework for Ruby, providing a DSL to write expressive, human-readable automated tests. Tests written using RSpec read like common English and use a rich set of syntax to describe the system’s behavior.

The pattern

In the last decade, I found one pattern that made my tests notably better for their readers. In a 2011 article, Bill Wake called this pattern arrange-act-assert (AAA or 3A). You might recognize a striking similarity between arrange-act-assert and the given-when-then pattern from behavior-driven development.

We are great at recognizing structural patterns. If the structure of our tests consistently follows a pattern, the readers will understand them quicker and, when needed, change and maintain them confidently.

1. The “arrange” step sets up the test case, instantiates any necessary collaborating objects, configures settings, and prepares the database, …
2. The “act” step covers the main action. Usually, this means a method call in a unit test, an HTTP request in integration tests, and browser interactions within system tests.
3. The “assert” step compares the results with the expectations. Is the return value equal to the expected? Did the expected side-effect happen?

You won’t find any of these terms (arrange, act, assert) in RSpec’s DSL, but that only means that RSpec has a different terminology.

RSpec’s grammar

Let me show you how I think about and structure my tests using RSpec. I will do that by writing and refactoring some tests for a simple object. The simple object will be a socially adept robot who is aware of its robot nature.

When I write tests for an application, I formalize the tests as factual statements about the system I’m describing on a high level:

A social robot greets and introduces itself to the user.
A social robot confirms that it is not a human.

When I’m writing unit tests, I switch my subjects and verbs to class names and method names:

SocialRobot#introduce prints a greeting and introduction to the STDOUT.
SocialRobot#human? returns false.

These statements translate well into RSpec’s grammar. To scaffold a test, you need to dissect the sentences into their subjects, verbs, and the rest.

The subject of the sentence becomes the subject of the test, and RSpec indicates it using #describe. The verb, in high-level tests (system, integration, …) becomes part of the behavior with the rest of the sentence, in unit tests becomes part of the subject usually as a nested #describe block.

# High-level tests like system and integration tests
RSpec.describe "A social robot" do
  it "greets and introduces itself to the user"
end

# Lower-level unit tests
RSpec.describe SocialRobot do
  describe "#introduce" do
    it "prints a greeting and introduction to the STDOUT"
  end
end

RSpec has sophisticated grammar, and we just started with the scaffold of a test. I’m going to continue with the unit test variant to cover the majority of the testing pyramid.

Let’s Arrange

RSpec has several helpers to express and define the necessary steps to prepare for a test. The #before method is RSpec’s trivial way of saying arrange. Within its block, you can set up everything that is required by the test.

RSpec.describe SocialRobot do
  describe "#introduce" do
    before do
      I18n.locale = :en
      @name = "Bob"
      @test_subject = SocialRobot.new(@name)
    end
    # ...
  end
end

class SocialRobot
  def initialize(name = "Bob")
    @name = name
  end
end

What about tear down? You can implement #after and #around hooks for your tests. Like #before, these methods accept arguments to specify when to execute them. They allow us to clean up the application state, roll back database changes, and delete files created by the tests…

Breaking up “arrange”

RSpec gets more sophisticated than that. It has words to define the object you’re testing, your test subject, and its collaborators. So when you arrange your test, you can give them names.

Subject - what are we testing?

I’ll start with the #subject(name = nil) {} method. I use #subject to tell future me, my colleagues and RSpec about the object under testing. Specifying the subject this way will also allow us to reference it implicitly. This is why you can write, for example, it { is_expected.to eq("foo") }.

I prefer to name these subjects, so when I’m writing more complex expectations, I can use descriptive names. Naming things is especially handy when the subject differs between test cases.

RSpec.describe SocialRobot do
  describe "#introduce" do
    subject(:robot) { SocialRobot.new(@name) } # 👈 Extracted from #before

    before do
      I18n.locale = :en
      @name = "Bob"
    end

    # ...
  end
end

Let - define the collaborators

Let’s follow it up with #let(name, &block). Using #let, you can extract, name (and memoize) the collaborators of your test subject. The test collaborators are the objects that your test subject needs to exist and function. Often these objects are substituted with mocks, doubles, and spies.

RSpec.describe SocialRobot do
  describe "#introduce" do
    subject(:robot) { SocialRobot.new(name) } # 👈 Uses the memoized method instead of the ivar
    let(:name) { "Bob" }                      # 👈 Extracted from #before

    before { I18n.locale = :en }

    # ...
  end
end

A word of warning about let

Use #let sparingly! A couple can improve readability, but more will quickly deteriorate your test. If I find a test surrounded by a bunch of let statements, usually there is a whiff of the Long Parameter List code smell too.

Another issue I often see is using let to store expectations. This will force the reader to jump back and forth in the test code. Jumping around makes us dizzy and confused. Keep the expectations together with the assertions where they belong.

Joe Ferries wrote an excellent article about potential overuse issues. I think “the dose makes the poison,” so be mindful of using let.

Act and Assert

After arranging everything for the test, it’s time to implement the rest of those factual sentences. This is the meat of every test, where you implement the verb by calling the tested method and setting expectations. In RSpec, both act and arrange steps are enclosed within #it² blocks. There are distinct ways to implement assertions and expectations using RSpec’s DSL.

I find it idiomatic for RSpec to use the principle of command-query separation to choose how I set assertions. Using a consistent assertion style makes my tests predictable and keeps me writing cleaner code.

Command-Query Separation (or CQS for short) states that every method should either be a command that performs an action and creates side effects, or a query that returns data to the caller, but not both. In other words, asking a question should not change the answer.

The general form of setting expectations is expect(<value>).to <matcher>, or you can also negate the expectation using .not_to, turning the expression into expect(<value>).not_to <matcher>.

RSpec calls its assertion helpers matchers and mocks. RSpec encourages us to write common English-sounding expectations, aided by its rich set of methods to set expectations and verify message passing between objects. Knowing these methods and using the most adequate to express the more complex expectations is so rewarding³. You can even rely on them to document your test cases. Furthermore, you can write your custom matchers to extend RSpec’s DSL.

Assert return values

Based on CQS, return values come from query methods. If you use Ruby on Rails, these query methods are not exclusive to ActiveRecord queries but mean any method that returns a value, most preferably without side-effects.

We can turn the “SocialRobot#human? returns false.” sentence into the following test:

RSpec.describe SocialRobot do
  describe "#human?" do
    it "returns false" do
      expect(robot.human?).to eq(false)
    end
  end
end

class SocialRobot
  def human? = false
end

Ruby is a brief but comprehensive language, and RSpec inherits these characteristics. In my humble opinion, the above test is succinct and English-like. It is explicit with the method call and the expected value next to each other. I like to keep most of my tests in this form. It’s easy to understand and maintain and requires minimal effort from developers coming from a different languages or framework.

For teams with seasoned Ruby developers RSpec provides briefer ways to express the same expectation using some meta-programming and nifty helpers. The previous example can be boiled down to a single-line test carrying the same meaning and providing a similar documentation value.

RSpec.describe SocialRobot do
  it { is_expected.not_to be_human }
end

So how does this work? Because the #human? method (in this example) will always return false we can put the test directly within the main describe block. You can also omit the description of the it block because it will be inferred from the expectation’s matcher. You can use the #is_expected method, a shorthand for the expect(subject) expression. Finally, we can use the be_* matcher to invoke the matching predicate method on the test subject and assert it returns true. In the case of be_human the matcher will call the #human? method on the subject, which is an instance of the SocialRobot class.

Assert side effects and behavior

Writing clear and concise expectations for side effects is fiddly because setting the expectation must be split into two parts. Executing the command (calling the function/method) and setting expectations on the side effects. Setting expectations on the side effects can happen in multiple formats.

Using a query method

The simplest case is when a query method is used to check the side effects of a command method. This looks exactly like the above, with an extra line to call the command method.

RSpec.describe SocialRobot do
  describe "#name=" do
    it "sets the name" do
      robot.name = "Bob"
      expect(robot.name).to eq("Bob")
    end
  end
end

class SocialRobot
  attr_accessor :name
end

In this situation, from time to time, I find myself setting two expectations. One before I call the command-method and another afterwards. This might sound paranoid, but so many times had I seen cases, where the query method returned the expected value even before the call to the command method. It is really important that you have a high level of trust in your tests.

Expecting a change

RSpec’s creators already considered the previous issue and created the expect { ... }.to change { ... } form. The change { } matcher uses a query method to test side effects and ensure a change in the return value.

RSpec.describe SocialRobot do
  describe "#name=" do
    it "sets the name" do
      expect { robot.name = "Chris" }
        .to change { robot.name }.from("Bob").to("Chris")
    end
  end
end

This is usually the form I use to test methods with side effects. It gives me the confidence that if the test arrangement/setup interferes with the expectation, the test will fail. Keep in mind that mixing Ruby’s block syntax and method chaining can be hard to read and confusing even for seasoned Ruby programmers.

Using mocks

Mocks are useful for setting expectations about the collaboration between objects. Mocks verify if an object has received the messages. Mocks are handy when the side effect should be avoided due to resource requirements (calculating large primes or calling a long chain of methods) or the irreversible consequences (payment).

RSpec has a feature-rich mocking library with mocks, stubs, and spies. Introducing the library could fill a book, so I’m keeping it to a single example.

Note: Using mocks to set expectations, in other words, to assert, swaps up the order of arrange-act-assert steps. RSpec has spies expect(...).to have_received() to keep the order, but I personally favor setting message-passing expectations upfront.

The last feature of the social robot is introducing itself through the standard out stream.

RSpec.describe SocialRobot do
  describe "#introduce" do
    it "prints a greeting and introduction to the STDOUT." do
      expect(STDOUT).to receive(:puts).with("Hi! I'm Bob")

      robot.introduce
    end
  end
end

class SocialRobot
  def introduce
    puts "Hi! I'm #{@name}"
  end
end

RSpec offers a rich set of expressive matchers and has something for this test as well. Because expecting output on streams is a common test assertion, RSpec has a specific matcher for this:

RSpec.describe SocialRobot do
  describe "#introduce" do
    it "prints a greeting and introduction to the STDOUT." do
      expect { robot.introduce }.to output("Hi! I'm Bob\n").to_stdout
    end
  end
end

Conclusion

My intention behind this article was to capture why arrange-act-assert is a great structural pattern for organizing automated tests. It provides a recognizable and predictable rhythm that aids the readers.

Another useful pattern for further refining tests is command-query-separation. CQS provides a single litmus test for deciding how to test specific methods, and it guides me in writing cleaner code.

Using factual statements is highly effective when considering the system or object under testing because they naturally match RSpec’s grammar.

Finally, RSpec, just like Ruby, has several sophisticated ways to describe your intentions. Many of its nifty tools are prone to abuse or misuse. Keep your reader in mind when you use them.

Final solution

require 'rspec/autorun'

class SocialRobot
  attr_accessor :name

  def initialize(name = "Bob")
    @name = name
  end

  def human? = false

  def introduce
    puts "Hi! I'm #{@name}"
  end
end

RSpec.describe SocialRobot do
  subject(:robot) { described_class.new }

  it { is_expected.not_to be_human }

  describe "#name=" do
    it "sets the name" do
      expect { robot.name = "Chris" }
        .to(change { robot.name }.from("Bob").to("Chris"))
    end
  end

  describe "#introduce" do
    it "prints a greeting and introduction to the STDOUT." do
      expect { robot.introduce }
        .to output("Hi! I'm Bob\n").to_stdout
    end
  end
end

Here’s a snippet of how the file looked before and after the pull request changes.

# spec/models/product_spec.rb - Before
RSpec.describe do
  subject(:product) { FactoryBot.create(:product) }

  it "is valid" do
    expect(product).to be_valid
  end
end

# spec/models/product_spec.rb - After
RSpec.describe do
  subject(:product) do
    picture = Rack::Test::UploadedFile.new("image.jpg", "image/jpg")
    manual = Rack::Test::UploadedFile.new("manual.pdf", "application/pdf")

    product = FactoryBot.build(:product)
    product.translations.build(locale: "en", picture:, manual:)
    product.translations.build(locale: "fr", picture:, manual:)
    product.translations.build(locale: "hu", picture:, manual:)
    product.save!
    product
  end

  it "is valid" do
    expect(product).to be_valid
  end
end

The test setup became cumbersome because additional validations were needed to reflect the business expectations.

The model represents a product with localized files containing the product image and the manual. Previously, the product was valid without any translations, but now, at least one translation must be present with at least the picture. This created a somewhat complex challenge for the factory definition.

 # app/models/product.rb
 class Product < ApplicationRecord
   translates :picture, :manual # Using the Globalize gem
   accepts_nested_attributes_for :translations

+  validates :translations, length: { minimum: 1 }
+  validates_associated :translations
 end

Solution

The goal was to return to a simple but expressive one-liner from the overly verbose test setup. After a couple of minutes of pairing, the following expression took shape to create the test objects. It is explicit about the translations enough for the test requirements and allows further refinement when needed.

subject(:product) { FactoryBot.create(:product, with_translations: [:en, :nl, :hu])}

Because at least one translation is required, the base factory has to create a translation. However, the localized content is not essential and is rarely checked.

I had the following considerations,

• For the majority of cases, the translation details are irrelevant
• It should be easy to create none or multiple translations
• It should be easy to specify the details of the translations when needed

With these in mind, I turned to the transient attributes of FactoryBot. This feature allows for hiding the complexity of object creation for testing behind transient attributes.

Transient attributes are attributes only available within the factory definition, and not set on the object being built.

I added an attribute containing a list of locale codes for which I wanted to generate translations. This attribute allowed generating the required translations using this transient attribute in conjunction with the translations_attributes method defined by the accepts_nested_attributes_for method of ActiveRecord.

I set only a single locale code as a default, so the factory does the minimum to generate a valid object. The list can be overridden using the with_translations attribute to generate a different set of translations, allowing the generation of none.

# spec/support/factories/products.rb
FactoryBot.define do
  factory :product do
    brand { "ACME corp" }
    price { 10.99 }

    transient do
      with_translations { %i[en] }
    end
    # Defined by accepts_nested_attributes_for :translations
    translations_attributes do
      picture = Rack::Test::UploadedFile.new("sample.jpg", "image/jpg", true)
      manual = Rack::Test::UploadedFile.new("sample.pdf", "application/pdf", true)

      with_translations.map do |locale|
        { locale:, picture:, manual: }
      end
    end
end

With the new factory definition, I could return to a single-liner to create the necessary objects without losing much of the explicitness of the test setup.

# spec/models/product_spec.rb - Before
RSpec.describe do
  subject(:product) { FactoryBot.create(:product, with_translations: [:en, :fr, :hu]) }

  it "is valid" do
    expect(product).to be_valid
  end
end

Conclusion

In conclusion, FactoryBot showed its power again, helping me keep the test concise and flexible. It is worth the time to glance through the examples of each FactoryBot feature to get an idea of when and where they can support writing excellent code.

Here is an example that pulls a container image. If the image does not exist, it’s okay to continue. Your steps may look different, more involved or use another command.

steps:
  - name: 'gcr.io/cloud-builders/docker'
    entrypoint: 'bash'
    args: ['-c', 'docker pull eu.gcr.io/project/myimage:$TAG || exit 0']

If your step hides if it fails because it is OK to let it fail, I have a better option for you.

What is the problem with the code above?

There are two problems with ignoring the exit code of the step.

1. The step will lie about the exit code, and you will not be able to see when the command fails. Seeing an always passing step provides a false assurance that everything is fine.
2. The step is convoluted by the error handling code. It is no longer possible to glance at the args and know what the step will do.

Let it fail

The Google Cloud Build configuration schema has a ton of underutilised features.

There are two handy features for this situation. Please note that both of them are experimental.

• allowFailure: <boolean>
• allowExitCodes: <array|string>

These options allow you to tell the runner to ignore if the step fails. With the latter, you can even specify which error codes are allowed.

steps:
  - name: 'gcr.io/cloud-builders/docker'
    args: ['pull', 'eu.gcr.io/project/myimage:$TAG']
    allowFailure: true
  - name: 'gcr.io/cloud-builders/docker'
    args: ['pull', 'eu.gcr.io/project/myimage:$TAG']
    allowExitCodes: [1]

Alternatives

If this solution is not for you, it may be worth looking at the script property and how you can use it to run bash scripts .

steps:
  - name: 'gcr.io/cloud-builders/docker'
    script: |
        docker pull eu.gcr.io/project/myimage:$TAG || exit 0