Test Driven Devlopement Reality Check

Over the last few months I've been reading blogs and articles evaluating the scope and scale of TDD -

Scaling Test-Driven Development
Unit testing; how far do you push the envelope?
TDD Proven Effective! Or is it?
What's more important: TDD or acceptance testing?

But what really got my attention was a blog by James Coplien, "Religion's Newfound Restraint on Progress", and a debate about the value of TDD between Coplien and Bob Martin recently posted at InfoQ. James Coplien has been on the forefront of software development trends for a couple decades. He is an object oriented and design patterns guy from way back and an agile coach today. His opinion is worth is paying attention. That's why it was quite a surprise to read how opposed he is to TDD. Whether you agree or not it's certainly worth some consideration. All the quotes below are by James Coplien from his blog or his comments on several other blogs (you'll need to read the blog comments for the full text of some of the quotes below).

'The answer, I think, is to focus on Quality and to find the techniques that work best and are the most cost-effective for your project. Just answering "both" to the "TDD-or-acceptance" question is not only at the wrong scope, it is missing the whole point.'

I think Coplein's objections come down two main points, cost effectiveness and the effects that TDD may have on a system's architecture. Further he is concerned that the procedural nature of TDD just does not fit well with the object prardigm.

'Most of TDD is based on xUnit, and the unit of any given test is a procedure.'

'Testing the functionality of object member functions causes one to focus on the functions and lose sight of the objects ...'

I've been programming with objects since the late 1980's so this hits home. The overall quality of an object oriented application is much more than the quality of the individual objects. It's about how the objects interact and fit together. We need to look at a higher level than a method or even a class to understand that interaction and create a good design.

'... this implies that TDD doesn’t make sense for a system where objects are the primary organizing principle. I don’t know how to test an object. Objects are about structure; what needs to be tested is the degree to which they map the user’s cognitive model of the domain.'

I'm still trying to absorb this completely, but my take again is that it's not about a single method or class, it's about the whole system and the objects it is constructed from. On the other hand I know from experience that tests inform class design. When a class is hard to test, perhaps I feel the need to make a private method public to test it properly, I take that as a sign that the class probably needs work.

'In the past five years I have yet to run into a single person who is doing it for other than one of two reasons: that it makes them feel good about themself, or that they are a lemming.'

Ouch, seems a bit strong and even hurts a little because I have to admit that good JUnit tests do make me happier. (The rest of these quotes are from "Religion's Newfound Restraint on Progress").

'Discussions in this area inspire passion. Why? ... People have tied their Agile success to their introduction of TDD practices into their work. It's the one thing they can do as individuals rather than as an enterprise.'

It's true, but I think part of it is also that "good" developers have pride in their work. TDD and unit tests in general are a way we get to directly impact the quality of our code without interference.

'On one hand we should be encouraged to use every weapon in our arsenal; on the other hand, we'll never get done if we try to do everything. It then becomes a matter of cost effectiveness. Integration and system testing have long been demonstrated to be the least efficient way of finding bugs. Recent studies (Siniaalto and Abrahamsson) of TDD show that it may have no benefits over traditional test-last development and that in some cases has deteriorated the code and that it has other alarming (their word) effects. The one that worries me the most is that it deteriorates the architecture. And acceptance testing is orders of magnitude less efficient than good old-fashioned code inspections, extremely expensive, and comes too late to keep the design clean.'

'Sorting these things out requires a system view. It used to be called "systems engineering," but the New Age Agile movements consistently move away from such practices. It shows in the quality of our interfaces, in the incredible code bloat we have today relative to two decades ago..'

It seems reasonable to ask if complete unit test coverage is really the most cost effective way to reduce bugs? I know unit testing has improved the quality of the applications I've worked on the last eight years. But could we have written fewer tests and gotten the same level of quality?

I've always been skeptical that TFD or TDD alone can produce a good architecture for the long haul. As much as I love refactoring, I'm not convinced that I can refactor my way to a great design. The idea that there is no need for any overall system design is crap. At the same time the idea that an architect can completely design a system upfront is just as crappy. Some level of system design and vision needed, but we have to learn from the system and adjust the design as we iterate. I'm a big proponent of "domain driven design" and the bottom line is I just don't think a real domain design will just emerge from coding, testing and refactoring. Domain design takes hard work and thinking, design effort not coding. Writing code is too low-level, too focused on details rather than useful domain abstractions.

'Agile is about engaging that user; TDD, about engaging the code. TDD is a methodology. TDD is about tools and processes over people and interactions. Woops.'

That's a pretty stinging indictment. If true if TDD isn't following agile principles. But I wonder if some the power of TDD is the simplicity and explicitness of its method. The best developers can succeed no matter what the process, but we all no that most software is not written by star developers. TDD sems simple enough fpr anyone to understand and follow to at least improve the quality of class level design and that's be worth something.

'It's about thinking, not about checklists.'

I can't disagree here because I've know too many developers who want easy answers or rules to follow. Software isn't like that, it's to much of a craft and takes constant attention and thinking to do well.

Overall I don't agree with James Coplien, but I think the discussion is worth having and hope everyone will keep an open mind. I know that unit testing (and lot's of it) has improved the quality of the my code and code written by my teams over the last eight years. I'm not a believer in one true way for software development or anything else. I think it is necessary to create a software process that works for you, taking the best practices that work for your in you environment.