Monday, May 17, 2010

Infected but not driven

The least I can say is that I'm test infected: when a coverage report shows lines of code that are not exercised by any test, I can't help but freak out a little (unless it appears that this code is truly useless and can be mercilessly pruned). This quasi obsession for testing is not vain at all: time and again I have experienced the quality, stability and freedom of move a high test coverage gives me. Things work, regressions are rare and refactoring is a bliss thanks to the safety net tests provide.

So what's with TDD... and me?

There are some interesting discussions going on around TDD and its applicability, which I think are mostly fueled by the heavy insistence of TDD advocates on their particular way of approaching software development in general and testing in particular. The more time I spend thinking about these discussions, the more it becomes clear to me that as far as testing is concerned, the usual rule of precaution of our industry applies: ie. it depends.

To be frank, I'm having a hard time with the middle D in TDD: as I said, I'm test infected, low test coverage gives me the creeps, but my process of building software is not driven by tests. From an external viewpoint, it is driven by features so that would make it FDD. From my personal viewpoint, it is driven by gratification, which makes it GDD.

Being gratified when writing software is what has driven me since I'm a kid: I didn't spend countless hours hurting my fingers on a flat and painful ZX-81 keyboard for the sake of it. I did it to see my programs turned into tangible actions on the computer. It was gratifying. And this is what I'm still looking for when writing software.

But let's go back to the main point of this discussion: TDD. With all the industry notables heavy-weighting on writing code while being driven by test, should I conclude there's something wrong with my practice? Or is the insistence on test first just a way to have developers write tests at all?

Adding features to a system, at least for the kind of systems I'm working on, mainly consist in implementing a behavior and exposing it through some sort of a public interface. Let's consider these two activities and how testing relates to them.


When I write simple utility functions, like chewing on some binary or data structure and spitting out a result, I will certainly write tests first because I will be able to express the complete intended behavior of the function with these tests.

Unfortunately, most of the functions I write are not that trivial: they interact with functions in other modules in non-obvious manners (asynchronously) and support different failure scenarios. Following a common Erlang idiom, these functions often end up replying a simple ok: such a result is not enough to drive the development of the function (else fun() -> ok end would be the only function to write to be done). In fact, testing first this kind of functions implies expressing with mock expectations all the interactions that will happen when calling the top function. That's MDD (Mock Driven Development) and it's only a letter away from making me MAD. Sorry but writing mocks first makes me nauseous.

My approach to developing and testing complex functions is, to me, more palatable as it leads to a faster gratification: I start by creating an empty function. Then I fill it with a blueprint of the main interactions I am envisionning expressed as comments. Afterwards, I reify this blueprint by turning the few comments in the original function into a cascade of smaller functions. At this point, I fire-up the application and manually exercise the new function: this is when the fun begins as I see this new code coming to life, finding implementation bugs and fixing potential oversights. After being gratified with running code, I then proceed to unit test it thoroughly, exploring each failure scenarios with mocks and using a code coverage tool to ensure I haven't forgotten any execution branch in my tests.

This said, there is another behavior-related circumstance under which I will write tests first: when the implemented behavior is proven wrong. In that case, writing tests that make the problem visible before fixing it is the best approach to debugging as it deals with the problem of bad days and lurking regressions.

Public Interface

Writing usable modules imply designing interfaces that are convenient to use. Discussing good API design is way beyond the point here. The point is: could writing tests first be a good guide for creating good interfaces? The immediate answer is yes, as by eating your own dog food first makes you more inclined into cooking it into the best palatable form possible (anyone who has had to eat dog food, say while enduring hazing, knows this is a parabola).

In my practice, I have found things to be a little different, again for less than trivial functions, which unfortunately compose most of a complex production system. For this class of functions, I have found that the context of a unit test is seldom enough to fairly represent the actual context where the functions will be used. And consequently, the capacity to infer a well-designed interface based on these tests first and alone is not enough. Indeed, a unit test context is not reality: look at all the mocks in it, don't they make the whole set look like a movie stage? Do you think it's air you're breathing?

When creating non-trivial public functions, I've found a great help into going through a serious amount of code reading in the different places where it is envisioned these functions will be used. Reading a lot of code before writing a little of it is commonplace in our industry: while going through the reading phase, you're actually loading all sort of contextual information in your short term memory. Armed with such a mental model, it becomes possible to design new moving parts that will naturally fit in this edifice. So that I guess that practice would be RDD (reading driven development).

Daring to conclude?

I find it hard to conclude anything from the dichotomy between my practices as opposed to what TDD proponents advocate. I consider myself a well-rounded software professional producing code of reasonably good quality: unless I'm completely misguided about myself, I think the conclusion is that it's possible to write solid production code without doing it in a test-driven fashion. If you have the discipline to write tests, you can afford to not being driven by them.


Dom Farr said...

I haven't really minded the middle D, but some people, including me, don't like the T in TDD, and prefer to replace it with E as in Example-Driven Development.

Taking the E to a B, as in Behavior-Driven Development is my new flavor.

As for mocks, I find them much more useful now than in the past; Mockito for Java is very powerful, mocking even concrete classes, and has a readable DSL.

However, mocks can easily overwhelm a test class hiding the actual purpose. When this happens I ask myself if my classes have the right responsibilities and start refactoring.

That said, I agree with you, always driving design using any of the ?DDs isn't realistic and good quality production code can be made with static analysis tools, pair programming and high test coverage.

David Dossot said...

Good point on refactoring when too many mocks are involved: I agree it's almost always the symptom of a god function that does way too much.

Nick Zhu said...

Nice post David, thanks for sharing.

2 cents from me: my approach to the Behavior example you gave is to have the blue print created first either in mind or in code, then test the each step of the blueprint separately first then try to wire them together. Usually if the steps are broken down and isolated enough the final overall test cases will not require too much mocking.

And I totally agree the TDD doctrine that is currently being pushed by some evangelists are sometimes not applicable or practical or even useful in this matter. When you are following an advice, even a perfectly good advice, blindly bad things happen.