So I've been using rspec capybara pretty much exclusively for my acceptance tests for about a year now.

The main reason for this is that I, like some others, find its syntax to be good enough for my purposes, and don't necesarily feel any benefit from a tool like cucumber.

Now I understand that there are probably great test suites out there that use cucumber. It's just that so far, I've not managed to encounter a test suite that didn't become unwieldy and take a long time to understand and/or debug.

In general I've found that I've been lucky enough to work with some very talented and smart developers, who write clean easy-to-understand code that satisfies the business needs incredibly well. But in these same teams we've had a test suite exhibits a bunch of anti-patterns and are at times hard to understand. I think part of this is that if you ever follow github's pull request model for building applications, then you end up with people focusing on reviewing code first. Maybe purely because of the order of the alphabet:

app -> features -> spec

Cucumber anti-patterns

Anyway, some of the anti-patterns I've encountered.

  • Shared global steps:

    • Big bag of global methods:
    • No convention for how/where defined
    • polluting Object when not using World helper method
    • Unrelated test breakages and brittle suites
    • Hard to find steps
  • Complex regexes that force the developer to become a ruby interpreter

  • Layers of abstraction hiding intent:

    • paths.rb
    • web_steps.rb

So the move away from web_steps.rb was a good one, which made a lot of sense to me at the time, and made me think more clearly about what the tests should be describing.

rather_than_doing_this.feature

Given I have a user with:
|email             | password |
|email@example.com | password |

When I fill in 'email' with 'email@example.com'
And I fill in 'password' with 'password'
And I click "submit"
Then I should see "Login successful"

this_is_preferred.feature

Given a user
When I succesfully authenticate
Then I should see "Login successful"

Why I don't like cucumber

Let me preface this rant by saying:

I'm sure I and many others have ended up going down the wrong route as far as people who do cucumber properly are concerned. And I'm sure the authors of cucumber and people that have become incredibly proficient at it and worked in the perfect environment are fine. But I have to guess that my experience has been different to the ideal.

After getting rid of web_steps the thing I found myself doing was writing the most inane of feature descriptions and moving a lot of the boilerplate stuff into step definitions. It's great for describing unquestionably true things about what the domain should do. But the other promise of cucumber; Gherkin, a shared language to communicate business requirements in, has IMHO, fallen flat.

I've never seen scrum masters/devs/project managers/testers/product managers/clients actually read through, work with, or give meaningful feedback on cucumber features.

Gherkin is pretty much a language that only developers use. Yes it's English and/or whatever human language of your choice. But try and get anyone to write it and it tends to be an overly verbose way of avoiding having a conversation about and understanding requirements. Then the time delay between having that conversation and writing it down in gherkin, then managing to decipher and navigate your way between all the different step_definitions and helper methods of various sorts means that you lose so much information and precious time during valuable feedback loops that it becomes meaningless to work in this way.

Then a bunch of stuff moved into helper methods and/or shared steps and this is where the spaghetti features started to happen. At times you'd have generic sounding steps shared across fifty features and because they're all global in scope, you find yourself just having to grep to find the definition. Although you can't even do that because you can't second guess what the regex is. You find yourself writing a regex to find a regex.

So I had enough of this type of pain. Which may well not be valid concerns if you're one of the developers who wrote cucumber or you happened to have

I'd love to get rebuttals on the above rant, from people that have done the client-engagement thing, or found a productive happy medium. Particular so, from people that have moved from/at least used capybara with rspec to a reasonable extent.

Capybara works for me

Where I have had great success is in writing out capybara steps. The DSL is much easier for me to reason about, and it's not that difficult for semi-technical clients to follow what you are doing.

I've had success with sitting with a product person/client, having a discussion about a feature, coming to a shared understanding, then actually typing out the steps and getting a tight-feedback loop and agreement at that point. I can be saying "I click here", "Fill this in", "Then see X" And a client can interject and say, no not quite like that, more like this. I've actually successfully written features whilst pretty much pairing with someone non-technical.

The same cannot be said for cucumber. People glaze over if you start acking for steps or doing anything more than writing gherkin. And even if it's just gherkin, it can be difficult at times to get client engagement in a process that to them looks like you making tedious notes of the conversation they've just had. A lot of people love to have some feeling of control over the product, "No, click there, and Y happens, not X". "X is when you fill in and select this" So being engaged in the rspec-capybara process is feasible. But if you're cuking properly, you shouldn't be having that conversation. Or at least, not whilst writing gherkin, which pragmatically is probably the most you're going to get co-operation on from a client.

Why SitePrism rocks (and also why I probably haven't grokked cucumber yet)

SitePrism rocks.

No more:

  • duplicated css/xpath selectors in steps
  • second guessing which damn file to look in

I love it because you're back speaking in the language of your domain at the highest level of your specs.

It's not

visit "/potentially_changing_url/1"

#or in Rails
visit specifically_rails_coupled_helper_path(@my_model)

#or cucumber and paths.rb
visit some_damn_method_matched_in_a_big_file_by_arcane_regexes

but rather

page_definitions.rb

class SigninPage < SitePrism::Page
  set_url_matcher /change_in_one_place/
  element :button_in_rapidly_evolving_dom, ".remain .nicely #decoupled"
end

acceptance_spec.rb

feature "Sign in" do
  let(:user) do
    create(:user, email: email, password: password)
  end
  let(:password) { 'password' }
  let(:email) { "email@example.com" }

  scenario "success" do
    sign_in_page = SigninPage.new
    sign_in_page.load
    sign_in_form = sign_in_page.sign_in_form
    sign_in_form.email.set email
    sign_in_form.password.set password
    sign_in_page.button_in_rapidly_evolving_dom.click
    expect(page).to have_content "Logged in successfully"
  end
end

I love this stuff and find my specs far more readable. And all that shared code and spaghetti goes away

Whilst I don't necessarily agree with having particularly complex tests there is another advantage to this in that once you have objects too, it brings up a whole load of refactoring possibilities, that I've only started to investigate:

let(:user) do
  create(:user, email: email, password: password)
end
let(:password) { 'password' }
let(:email) { "email@example.com" }

scenario "success" do
  sign_in_page.load

  sign_in_form.tap do |f|
    f.email.set email
    f.password.set password
    f.button_in_rapidly_evolving_dom.click
  end

  expect(page).to have_content "Logged in successfully"
end

def sign_in_form
  @sign_in_form ||= sign_in_page.sign_in_form
end

def sign_in_page
  @sign_in_page ||= SigninPage.new
end