A few weeks ago I was lucky enough to attend CukenFest in London. It was a series of 3 events, BDD Kickstart, CukeUp, and CukenSpace. BDD Kickstart was a two day workshop meant to teach the participants how to build software via Behavior Driven Development and how Cucumber can fit into that. CukeUp was a day of talks centered around BDD with speakers from the community. CukenSpace was an Open Space which was a combination of Cucumber contributors and community members sharing experiences, hacking, and planning the future of Cucumber.
Even as a maintainer of cucumber-js and an experienced user of Cucumber, the BDD kickstart class gave me a number of great takeaways (listed below). To me, cucumber had largely been a testing tool and a form of living documentation though not always a very pretty form of documentation. BDD however is all about discussion and I learned quite a few things I want to apply on my future projects.
We had a version of CukenSpace last year with just Cucumber contributors, and opening it up to the community gave us some great new viewpoints. We were able to gain a few new contributors, got to hear peoples' experiences with BDD and cucumber, and gained some valuable outside perspectives.
As with last year, getting some face to face time with other contributors also helps spur contributions and improvements to cucumber. I paired with Julien Biezemans (the original author of cucumber-js) on speeding up the cucumber-js test suite which resulted in the feature tests dropping from 9 minutes to 1 minute on TravisCI by avoiding using child processes whenever possible. I also got some great feedback from Matt Wynne (a core cucumber-ruby contributor) that will hopefully lead to a great improvement in the error output of cucumber-js. Essentially, scenarios that error will be printed in "pretty" format with the error inline.
- Tests don't tell you if your code works, they only tell you when its broken
I really like this way of thinking as tests help us ensure we don't break existing features when we add new ones. Drawing a parallel to code coverage, I think that is best used to find sections of your codebase that aren't tested. Those sections can then be deleted or updated with tests.
- Use the word "should" in Then steps
I really like this simple rule as on a few projects we have had some back and forth on how to make the "Then" steps different from a related "Given" step. For example: "Given I have a file" and "Then I have a file". I have accidentally used a "Given" step definition in a "Then" step many times. The use of the word "should" instead of "must" is also meant to help facilitate discussion as requirements can change overtime. The word "should" also parallels expectation libraries (like chai and rspec) which sometimes expose an interface that involves the word "should" (although I have always prefered the expect syntax).
- Write your scenarios upwards (write Then, When, Given)
To do this write your "Then" step first. Next, write your "When" step above the "Then" step. Finally write your "Given" step above the "When" step. I like following this order because it focuses on the outcome (Then) and action (When) first and prevents you from getting sidetracked on what at times can be a complex context (Given).
- What, not how. (Declarative, not imperative. No user interface in steps.)
I really like this as it focuses you on the business logic of the application. As an example the cucumber-js tests are very verbose and focus mostly on the how. For some apps the "how" may be important to demonstrate, but won't be needed in every scenario. For example, having all the necessary steps for signing up and being granted admin privileges versus a step that says "Given I am logged in with admin privileges".
- Max 5 steps per scenario
I have not been able to achieve this on previous projects. I want to aim for this in the future though as I think it would be a good exercise to keep your scenarios shorter and easier to understand.
- When testing user interfaces, the action (When) and outcome (Then) steps interact with the interface but the context (Given) does not.
This is not something I've followed in the past but something I feel would be really powerful. For example, this will hopefully prevent retesting the login flow on every scenario if you can login a user with an API and inject the credentials into the browser.
- Follow the test pyramid
The test pyramid suggests having a large amount of unit tests, and a smaller amount of end to end tests. I think Cucumber is best used for the end to end tests as it allows you describe the core business logic in a manner non-technical people can understand. This keeps the test suite fast, as too many feature tests can easily slow down the suite. I've also found that having only feature tests can sometimes make the internals harder to refactor as without unit tests it can be hard to see what each part of the system is used for.
Future of cucumber
The successor to relish (used by rspec) which is being built as the place where all team members can discuss features. It has a simple UI that nicely displays your features and gives you the ability to comment on steps to discuss changes. Test result integration is also planned where results could be uploaded via CI.
- Cucumber GUI
Built using electron, it will be a new way to display test rules. Developers can pipe the event protocol formatter to this app to have realtime updates of test results. No more scrolling through output in the terminal. A spike of this was demoed using cucumber-ruby which implements an initial version of the event protocol. The idea is that this tool can be used by all cucumber implementations (once they implement the event protocol).