Is ‘BDD’ testing? Model of BDD for TestingGo Back
- Posted by Rachel
Is ‘BDD’ testing?
Well, some of the activities involve testers, but it’s not real testing, right?
It’s complicated… We need to talk about this and I want to understand, so please bear with me!
Sarcasm aside, behaviour driven development is a big deal within software development and opinions on it are polarised. BDD started to gain traction just as I started contracting as an automation specialist, so I have a lot of personal experience of going from a misguided sycophant who loved misusing BDD, to an evangelised pitchfork-wielding zealot who was convinced BDD would lead to the death of tester – and I’ve been back and forth over a period of six years.
Whilst I’ve mellowed out over those six years, I must admit that until a year or so ago I still took a dim view of BDD, especially as my approach towards automation in testing has evolved. This resulted in a talk I gave at TestBash Manchester in 2016 called “The deadly sins of acceptance scenarios” during which I was hardly singing from the rooftops about how great BDD is, duly noted in this blog:
“…the negativity around BDD or acceptance scenarios feels like the negativity I’ve encountered around Microservices and I’d like to hear some well-thought out, positive experience reports. It feels like all of the balanced or thoughtful talks tend to be quite negative really and I don’t really see a great deal of value in using BDD over more straight-forward approaches such as TDD…” (Matthew Bretten)
Matt’s observation is correct in that my talk wasn’t really giving the full picture of BDD and his comments resonated with some personal thoughts that I had on the subject at the time, namely:
- What is BDD exactly?
- Is BDD testing?
- Or is it something else?
My inspiration for the talk was based on the frustration of seeing so many mistakes being made with automation in testing, which I blamed on developers and testers not truly understanding the proper use of scenarios (which I still believe is the case). My intention was to highlight those mistakes, based on personal experiences, to show that others could use them as heuristics to help them see the traps they were falling into (which I still feel is useful).
But it was hard talk to write. As I began putting it all together I was surprised by how much it challenged my ideas of BDD and scenarios. I found my understanding severely lacking in places. At first, it put me into a bit of a panic. But with some support from awesome people who answered my questions, I began to rebuild my understanding of BDD.
A Model of BDD for Testers
After my talk at TestBash Manchester, I felt I was too focused on scenarios and so I set myself the following task:
- Immerse myself in BDD literature to help me understand BDD better. Understand where testers’ misconceptions have come from and build a model that would make BDD clearer for testers.
And I have been immersed. Whilst I am by no means finished learning about BDD, I found the activity of further learning about BDD to be a very useful exercise.
For this article I simply wanted to share my model of BDD in the hope that others may find it useful, to set them on the right path towards implementing BDD, and start a discussion. I also hope it will help others to avoid the traps testers fall into with BDD.
So here is my model. As you can see straight away there is a lot more to BDD than just Gherkin syntax and automation.
I’ll explore the collaboration and outside-in development parts of this model over the following sections, before attempting to answer whether ‘BDD’ is testing, and finishing with what lies beyond BDD.
“If you’re not having conversations, you’re not doing BDD.” (Liz Keogh)
Through interviews and conversations, I have seen that a lot of testers tend to perceive BDD as the use of Gherkin syntax and automation tools; they tend to pay very little attention to the collaborative aspects of BDD. I suspect that part of the reason is due to testers’ first exposure to BDD coming from Dan North’s article “Introduction to BDD”, which is heavy on tooling and light on collaborative exercises. This isn’t a criticism of the article since it was one of the first to be written. However, BDD has changed over the years and even Dan himself has said that the work he did with JBehave was more of a ‘thought experiment’.
“There are things about your domain that you don’t know, or you’ve misunderstood, or that nobody’s thought of yet. By talking through examples in groups, the chances of uncovering these gaps and misunderstandings is increased.” (Liz Keogh)
BDD is about conversations
It’s about talking about your domain from the user’s perspective to ensure you are building the right thing, and this can be achieved by coming together as a team to have conversations. We involve members from testing, development, design and the business in an informal meeting that is more typically known as ‘three amigos’ to discuss and question what we plan to build. The goal is to dispel any incorrect assumptions we may have, and remove any ignorance we have around what we want to deliver.
There are tools we can use during our three amigos sessions to help us. A tester’s goal is to generate questions and ensure the conversation stays inside the scope of the feature. As a tester, questions are your best tool for collaboration sessions. I find the following techniques or phrases help in a session:
- The ‘five Ws’ – what, who, where, when, why
- “This may be a stupid question but…” – those stupid questions weed out a lot of assumptions
- “So just to confirm…” – again, verbalising what is required will weed out assumptions
Gherkin and examples
Another tool, synonymous with BDD, is Gherkin. Gherkin uses Given-When-Then syntax to allow us to create examples in the form of scenarios, to demonstrate how we might expect acceptance criteria to behave. For example:
- Given the user has not ordered yet
- When the user adds a book into the shopping cart
- Then a discount of 10% is applied to the total
Unfortunately, Gherkin syntax is another of the tripping points for testers. To be clear, Gherkin should not be used to create test cases; the goal of Gherkin is to help distil examples of how we expect acceptance criteria to behave. For example, a series of examples may focus on behaviour around a boundary of a specific value that results in different behaviour. It’s important to remember, as testers, that these boundaries are put into examples because they are core to what the business wants to deliver. They are not there as boundary value tests.
Gherkin examples are there to illustrate behaviour and not to exhaustively test, and it’s important that testers, when involved in three amigos sessions, use their skills and knowledge to raise questions and share information, not design hundreds of tests. As an aside, if you are still using test cases in your day-to-day testing, then keep them separate from Gherkin examples and consider adopting new approaches such as exploratory testing.
Writing examples in Gherkin isn’t easy. However, a new approach called example mapping has been created by Matt Wynne and is designed to help teams keep their discussion in scope, handle questions and ensure the right examples are being created. In short, example mapping uses different coloured post-it notes as visual aids to help keep track of rules (acceptance criteria), examples (Gherkin scenarios) and questions. You can read more about how testers have fared with example mapping here:
To me, the collaborative phase is the single most important part of BDD. The focus on discovery, raising questions and revealing information about what we plan to build is a cornerstone of testing. It’s vital that we ensure this part of the process is carried out and that testing is involved.
Testers’ input into the delivery of a feature is more profound in the collaborative phase than in outside-in development (see next section). I would even argue that you could execute the collaborative phase, not pursue the automation side of things and still see huge benefits. I now firmly believe that to carry out BDD successfully you must have the collaborative phase, and to miss this part out will leave you exposed to failure.
Part 2 will be published next week.
About The Author
Mark is a tester, teacher, mentor, coach and international speaker, presenting workshops and talks on technical testing techniques. He has worked on award-winning projects across a wide variety of technology sectors ranging from broadcast, digital, financial and public sector working with various web, mobile and desktop technologies.
Mark is part of the Hindsight Software team, sharing his expertise in technical testing and test automation and advocating for risk-based automation and automation in testing. He regularly blogs at mwtestconsultancy.co.uk and is also the co-founder of the Software Testing Clinic in London, a regular workshop for new and junior testers to receive free mentoring and lessons in software testing. Mark has a keen interest in various technologies, regularly developing new apps and devices for the Internet of things. You can get in touch with Mark on twitter: @2bittester