Friday, May 23, 2008

Unit Tests: No Future?

Industry expert Andrew Binstock has just posted an entry in his blog titled "Is the popularity of unit tests waning?", where he discusses the staggering state of the practice of unit testing.

Andrew asks the crucial question of why did we end in this situation. I do not pretend to have the answer, but here are some patterns that I have observed, which, I think, could decrease the appeal of unit testing.

Bad unit testing practices

It is very easy to write fragile unit tests, for example by using stubs when mocks would be enough or by creating time sensitive tests that work intermittently or stop working after a while. Similarly, it is common to see integration-like tests sleep into the realm of unit testing, making these tests fragile, slow and dependent of external resources.

These bad practices tend to lead to a decrease of confidence in unit tests, which then can lead to a progressive reduction of their usage. For example, one programmer can decide he will not write any tests anymore for DAOs after struggling with poorly written tests.


Bad software design

There is a tight relationship between good code design and testability. I have already blogged about how increasing test coverage can lead to a better design: unfortunately, not all developers are ready to review their design to make their code more testable.

To be fair, our languages and frameworks often force us to write code that is hard or uninteresting to test. Who wants to write unit tests for infamous JavaBeans getters and setters? Not Allen Holub, for sure!

Other programmatic idioms like equals/hashcode often exhibit a high cyclomatic complexity: writing complete tests for those would be tedious, unless one use test generating products like Agitator, from late Agitar, or Jtest, from Parasoft.


Test benefits blindness

Management tend to be blind to the benefits of unit testing and very aware of its costs. A project I know have been deemed to have gone overboard with unit testing. At the same time, this project has the code base that is the most maintainable, flexible and fun to work with. It seems something prevents businesses to see the value added of solid test practices, as if it was all about some obscure geeky self satisfaction activities (as writing "perfect code" would be).


No Future?

Is there any hope, then? I think that, like for most of our problems on this little planet, education is the key. Whether we look at software engineers or business managers, there is still a great lack of education about unit testing, how it works and why it pays back.

2 comments:

Nick Zhu said...

I recently had the experience working with some developers who were new to the idea of unit test. The first feedback I got from the retrospective was "unit test does not work!" After some digging, I found out they are recreating the entire framework stack with hundreds of lines of code in each test case. With some paired programming and mentoring, now they all learned the right way of writing the test case and even use the testability as a metric to drive their design.

But too many of these incidents in our industry developers taking on a new concept without proper training and mentoring end up reaching the wrong conclusion some times even with misleading feedback to management. The more I worked with people new to XP practices, the more I believe how important for a team a good mentor or coach can be.

I still remember the shock when the first time I learned the right way of doing TDD from my mentor. I thought I was doing TDD for a few years already back then, but I was just simply ... hacking.

David Dossot said...

Nick, thank you for your feedback.

I realize that mentoring is certainly key to unit test adoption and success, especially knowing that most developers do not read, hence will not learn from others how to improve their practices on their own.