BDD Discussion
BDD Divergence
The BDD Acceptance Testing thread over at the BDD forum is showing up what I consider to be the big problem in BDD at the minute, mixed messages.
On one side you have an approach that uses a context and specification approach to apply BDD style tests to all levels of specification, this is the approach where you might have a class called When_my_account_is_declined containing a specification called A_notification_is_sent_to_interested_parties. In my view this style is useful, I've been using it for unit testing and I like it a lot. Having said that I'm not intending to take a dump of the context/specification names for use by non-programmers in the team (e.g. domain experts) because a lot of the specifications will be too low level or technology focused. So far I'm primarily using this style to get nice readable specifications for use by the developers, so as a different style of TDD.
On the other side is the story/scenario driven approach that Dan North is pushing for, he also describes why he likes to separate these tests from classical programmer tests and the part they play in your process.
I personally think that you can use the two approaches together if you want to, using the given/when/then approach for acceptance tests and context/specification (or just normal TDD) for all other tests (including unit tests). You can of course use the context/specification style for the acceptance tests and dump given/when/then. Whether this is a good idea depends on whether the benefits Dan's approach brings (such as improving communication with non-programmers in the team) justify the effort of using the given/when/then style and to be honest I haven't used his style enough yet to have a strong view on it.
Are we automating Stories or Scenarios
Two interesting points that Dan makes in the acceptance testing thread are:
"As you quite rightly said, you can express any scenario in just a regular
example, or xunit test case."
..and..
"The scenario runner (not story runner - you don't run stories)"
He is clarifying that we run scenarios not stories. What is a scenario in this context, Scott gives a good definition in the BDD terminology thread:
"A scenario is a way to express acceptance criteria using an example-
driven approach. A scenario can express more than one criteria if
needed. "
The fact that you automate scenarios rather than acceptance tests is a good point to make.
Terminology
In the discussions on BDD we all seem to use consistent terminology, however the BDD terminology thread at the BDD forum is a good starting point to getting some consistency (which can only improve the discussion).
So unfortunately the xBehave frameworks have pushed "story runner" when in actuality it is "scenario runner, organized into stories". I think this is just semantics to cover the issue that stories are meant to be flexible and disposable, and not put into code.
ReplyDeleteI want acceptance test in scenarios, but I want a way to organize them as well. I want something better than FIT. But I don't know what that is.
"I want acceptance test in scenarios, but I want a way to organize them as well. I want something better than FIT. But I don't know what that is."
ReplyDeleteYeah I totally agree, and I'm certainly still open to the idea that the xBehave frameworks aren't a good solution.
Personally I quite like the xBehave syntax and mindset (though so far I've got limited experience) but I do have issues regarding how best to do acceptance testing.l
So really I struggle with questions like how many acceptance tests should I write, what level should they be at and so on rather than struggling with whether to use given/when/then style for them.