From tests to specifications (and everything in between)

I came to a really neat realisation a little time ago that when using test-drive development (TDD), the testing of your software defines the specification of your software. This may be obvious but it really highlights the fundamental difference in approach between the agile approach to software and other more specification first approaches (like the waterfall model) – with agile methodologies, the spec of the written software is only defined by the tests. Done properly, there is no disconnect between the specification and the implementation, only between the backlog and the specification.

I’ve recently being looking at behaviour driver development (BDD) and how that fits with TDD and general good practice in development. There’s loads of bluster around BDD and no end of frameworks and new languages that define special new ways of generating your integration tests such that normal people can read the tests and yet it still magically actually performs the test. I was always uncomfortable though with the notion that BDD and TDD were actually all that different when it comes down to it; I couldn’t see what was offered by the various tools.

I came across this lovely blog post, which very much clarified my own thoughts (and confirmed a few too). The central point of the post, which I’ll restate here, is BDD is all about communication. Given that, BDD is just TDD with proper communication.

There are TDD purists who will argue that if you’re doing more than unit testing (e.g. you’re doing integration testing too), you’re not doing TDD. I would argue tosh, the whole of your code should benefit from a genuine test driven approach. At the highest level, the test should be derived from the user stories, and as subsequent requirements are introduced, they are manifested as tests that should be satisfied, which then may well depend on further capabilities which are described by more tests. At some point (depending on the size of the project), you reach the unit test stage. As already described, this hierarchy is then your specification.

So how does one fit BDD into all this? Well, it simply then becomes about the process by which tests are defined. A test is always the satisfying of a need, from the highest level which the end user probably cares about (which maps to the user stories), down to the lowest level which the developer cares about (e.g. there should be a frobinator method on this class which frobinates the two inputs). But here’s the point, all the language used to describe those tests are in terms of what behaviour is required, which is exactly what BDD is about!

In python, my implementation strategy for this is simply to use unittest and then describe every test set and individual test in terms of what the software should do. So something like “There should be a widget which calculates how many tins of custard I need to buy” or “This class should have a dot product method that returns the dot product of itself with an argument.”. This makes it crystal clear what the intended behaviour is and just thinking along these documentation lines clarify the purpose of the test in a way that I never had with simply writing tests to satisfy some hypothetical functionality.

In effect, the docstring becomes a first class component of the tests. Without the descriptive behaviour driven docstring, the test is invalid.

Now, I think this is great for various reasons, but two are worth point out: firstly it totally removes the disconnect between BDD and TDD and secondly, it acts as a really neat guide for architecting the software. I found myself neatly guided through the design process by describing my software using tests. I think it’s because the process of defining tests forces one to break the expected functionality into chunks. If you find later that your architecture can be improved, then great, isn’t refactoring one of the central tenets of TDD? (which neatly maps to rearchitecting in the general sense).

This all leads to an interesting result, which is particularly relevant in the context of medical software, which is in the world of IEC 62304, which fits neatly into the waterfall model, but not necessarily very well with agile techniques. When one has the testing system described as above, one already has a plain English representation of the specification – and better than that, all the tests that define it and by extension an implementation of the software that satisfies the spec perfectly! It’s a small step to write a sphinx plugin to auto-generates the spec on demand.

There’s an interesting series of blog posts (beginning here) I was reading in which agile methodologies for medical software is discussed. One solution (described as “Agile methods, solution 2”) is supported very nicely by what I describe above.

Anyway, back to writing some tests…


About Henry Gomersall

I'm a engineer wanting to make things actually useful. I'm someone that wants to drive technology and ideas to be helpful for everyone. I'm someone that realises the disconnect between technology and utility and I hope to correct that...
This entry was posted in Engineering, Programming. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s