Write Software That Behaves!

Behavior Driven Development

Behavior Driven Development is the process of writing high-level scenarios verifying a User Story meets the Acceptance Criteria and the expectations of the stakeholders. The automation behind the scenarios is written to initially fail and then pass once the User Story development is complete.

BDD is largely a collaboration tool which forces people working on a project to come together.  It bridges the gap between stakeholders and the development team.  Practicing BDD is fairly painless, but does require a meeting to discuss the intended behavior the scenarios will verify.  The meeting to write the BDD tests is usually an informal one which is led by the Product Owner or stakeholder and involves members of the development team.  The goal is to collaborate so everyone is on the same page as to what this User Story is trying to achieve: Start with the story. Talk about the “so that”. Discuss how the software currently works and how this story will change or impact current functionality.

Scenarios are written from a user’s perspective. Because they are run on a fully functioning system with real data, they can be expensive (meaning the time it takes to write, execute, and maintain the tests).  However, the scenarios will serve as executable documentation for how the software behaves. This is useful for developers to understand each other’s code and gives them tools to refactor with confidence.  Over time, the tests will evolve as they are maintained and also serve as easy-to-read descriptions of features. Documenting behavior in this manner is useful when onboarding new team members by communicating the software’s functionality.

Things to ask yourselves when writing scenarios:

• What functionality is not possible or feasible to cover with lower-level automated tests?  How do these tests differ from unit tests, integration tests, or service tests?
• What is the “happy path” of the User Story functionality? This is the type typical usage of the software.
• What is the atypical usage?  Are there variations of inputs that are possible, but used less frequently?
• How should the system handle bad input?
• How does the system prevent bad output?  How does it display or log errors?
• What is the impact on other parts of the system?
• What are the integration points – other components, other applications?  Should the tests include some verification of this integration, or is it covered elsewhere?

The process that has worked well for me is to, first, write the scenarios together with the Product Owner.  The PO should lead this discussion and write the steps in a way that makes sense to stakeholders.  At least one Developer and one Tester should also be present.  Some like to call this the “Power of Three”.  Read through the Acceptance Criteria asking the questions listed above. Try to use consistent language in the steps and use terms that make sense to people outside the development team.  It may be tempting to write steps involving user interaction with the software, like this:

Given I am on the Login screen

When I enter “user1” in the username field

And I enter a “mypassword” in the password field

And I click the Login button

Then I should see the error message “Username/password is invalid”

I have found, it is better to describe the behavior in broader terms not closely tied to the application layout itself.  For example:

Given I am on the Login screen

When I enter invalid credentials

Then I am not logged in and a meaningful error message is displayed

This way, the code behind each step is closely tied to the application or the user interface, not the test scenarios themselves.  As the application grows, the test steps themselves will be less likely to require changes, isolating the maintenance to the code behind the test scenarios.

Now, once the scenarios are defined using existing steps as well as some new ones, the Testers can partially implement the new steps to fail.  For example, adding assertions that a file exists on the file system.  Or, writing code returning a negative result for now.  Maybe the code will eventually query the database to return a positive result, or maybe it will ensure some value is displayed on the UI.  Sometimes this part is minimal; sometimes it can include almost all of the step implementation.  Lastly, during feature implementation, the Developer writes the final test code to make the scenario pass.  I encourage Developers and Testers pair at this stage. This type of teamwork keeps the Tester engaged in how the code is being implemented and ensures they understand how the software works.  An informed Tester is a good Tester.

As you probably know, automated tests provide more value the more frequently they are executed.  This is why you want to be smart about the tests covered at the user level.  Automated testing is an investment.  The team should view the tests as code as well.  Automated tests require maintenance to keep them passing and best if shared by all members of the team.  When practicing BDD, make sure all scenarios provide value. They should not be too difficult to automate.  Be careful not to include too many variations of input data.  If possible, cover the various inputs using tests at lower levels: unit, integration, service-level. Use the BDD scenarios to cover what is not, or better yet, cannot be covered by other types of automated tests. Don’t be afraid to get rid of a scenario altogether if it doesn’t provide value.  It is okay to run some tests manually as long as the team understands manual tests are executed much less frequently, so feedback is delayed.

Tips for BDD:

• Write them to be executed locally on a Developer’s machine
• Monitor execution time and keep it to a minimum
• Scenarios should not be dependent on each other
• Scenarios can be executed multiple times in a row and still pass (some cleanup may be necessary)
• To keep the number of steps from getting out of hand, pass in variables to the steps to maximize reuse
• Keep your steps organized in separate files by major functional area
• Scenarios are grouped to allow for a quick regression test of each major functional area, or a full regression of the entire system
• Use source control for your test code

When BDD is done properly (before implementation), the real value is gained by simply collaborating and discussing the expected behavior of the software!  Once implementation is done, the scenarios ensure the software meets the needs of the stakeholders.  From then on, the automated tests act as a safety net for developers to refactor code and implement new features with confidence.  Teams should strive to make execution of at least a portion of the BDD tests part of their Continuous Integration build/deployment process and make the test results visible.  Failing test scenarios around existing functionality should be a top priority to fix.

Have fun!  And write software that behaves!

Tool references: http://cukes.info/ (Ruby)

http://www.specflow.org/specflownew/ (.NET)

http://jbehave.org/ (Java)