Showing posts with label software process. Show all posts
Showing posts with label software process. Show all posts

Friday, February 22, 2008

Test Driven Devlopement Reality Check

Over the last few months I've been reading blogs and articles evaluating the scope and scale of TDD -

Scaling Test-Driven Development
Unit testing; how far do you push the envelope?
TDD Proven Effective! Or is it?
What's more important: TDD or acceptance testing?

But what really got my attention was a blog by James Coplien, "Religion's Newfound Restraint on Progress", and a debate about the value of TDD between Coplien and Bob Martin recently posted at InfoQ. James Coplien has been on the forefront of software development trends for a couple decades. He is an object oriented and design patterns guy from way back and an agile coach today. His opinion is worth is paying attention. That's why it was quite a surprise to read how opposed he is to TDD. Whether you agree or not it's certainly worth some consideration. All the quotes below are by James Coplien from his blog or his comments on several other blogs (you'll need to read the blog comments for the full text of some of the quotes below).
'The answer, I think, is to focus on Quality and to find the techniques that work best and are the most cost-effective for your project. Just answering "both" to the "TDD-or-acceptance" question is not only at the wrong scope, it is missing the whole point.'
I think Coplein's objections come down two main points, cost effectiveness and the effects that TDD may have on a system's architecture. Further he is concerned that the procedural nature of TDD just does not fit well with the object prardigm.
'Most of TDD is based on xUnit, and the unit of any given test is a procedure.'

'Testing the functionality of object member functions causes one to focus on the functions and lose sight of the objects ...'
I've been programming with objects since the late 1980's so this hits home. The overall quality of an object oriented application is much more than the quality of the individual objects. It's about how the objects interact and fit together. We need to look at a higher level than a method or even a class to understand that interaction and create a good design.
'... this implies that TDD doesn’t make sense for a system where objects are the primary organizing principle. I don’t know how to test an object. Objects are about structure; what needs to be tested is the degree to which they map the user’s cognitive model of the domain.'
I'm still trying to absorb this completely, but my take again is that it's not about a single method or class, it's about the whole system and the objects it is constructed from. On the other hand I know from experience that tests inform class design. When a class is hard to test, perhaps I feel the need to make a private method public to test it properly, I take that as a sign that the class probably needs work.
'In the past five years I have yet to run into a single person who is doing it for other than one of two reasons: that it makes them feel good about themself, or that they are a lemming.'
Ouch, seems a bit strong and even hurts a little because I have to admit that good JUnit tests do make me happier. (The rest of these quotes are from "Religion's Newfound Restraint on Progress").
'Discussions in this area inspire passion. Why? ... People have tied their Agile success to their introduction of TDD practices into their work. It's the one thing they can do as individuals rather than as an enterprise.'
It's true, but I think part of it is also that "good" developers have pride in their work. TDD and unit tests in general are a way we get to directly impact the quality of our code without interference.
'On one hand we should be encouraged to use every weapon in our arsenal; on the other hand, we'll never get done if we try to do everything. It then becomes a matter of cost effectiveness. Integration and system testing have long been demonstrated to be the least efficient way of finding bugs. Recent studies (Siniaalto and Abrahamsson) of TDD show that it may have no benefits over traditional test-last development and that in some cases has deteriorated the code and that it has other alarming (their word) effects. The one that worries me the most is that it deteriorates the architecture. And acceptance testing is orders of magnitude less efficient than good old-fashioned code inspections, extremely expensive, and comes too late to keep the design clean.'
'Sorting these things out requires a system view. It used to be called "systems engineering," but the New Age Agile movements consistently move away from such practices. It shows in the quality of our interfaces, in the incredible code bloat we have today relative to two decades ago..'
It seems reasonable to ask if complete unit test coverage is really the most cost effective way to reduce bugs? I know unit testing has improved the quality of the applications I've worked on the last eight years. But could we have written fewer tests and gotten the same level of quality?

I've always been skeptical that TFD or TDD alone can produce a good architecture for the long haul. As much as I love refactoring, I'm not convinced that I can refactor my way to a great design. The idea that there is no need for any overall system design is crap. At the same time the idea that an architect can completely design a system upfront is just as crappy. Some level of system design and vision needed, but we have to learn from the system and adjust the design as we iterate. I'm a big proponent of "domain driven design" and the bottom line is I just don't think a real domain design will just emerge from coding, testing and refactoring. Domain design takes hard work and thinking, design effort not coding. Writing code is too low-level, too focused on details rather than useful domain abstractions.
'Agile is about engaging that user; TDD, about engaging the code. TDD is a methodology. TDD is about tools and processes over people and interactions. Woops.'
That's a pretty stinging indictment. If true if TDD isn't following agile principles. But I wonder if some the power of TDD is the simplicity and explicitness of its method. The best developers can succeed no matter what the process, but we all no that most software is not written by star developers. TDD sems simple enough fpr anyone to understand and follow to at least improve the quality of class level design and that's be worth something.
'It's about thinking, not about checklists.'
I can't disagree here because I've know too many developers who want easy answers or rules to follow. Software isn't like that, it's to much of a craft and takes constant attention and thinking to do well.

Overall I don't agree with James Coplien, but I think the discussion is worth having and hope everyone will keep an open mind. I know that unit testing (and lot's of it) has improved the quality of the my code and code written by my teams over the last eight years. I'm not a believer in one true way for software development or anything else. I think it is necessary to create a software process that works for you, taking the best practices that work for your in you environment.

Wednesday, October 3, 2007

The Discipline of Agile

An article by Scott Ambler at Dr. Dobb's does a nice job refuting the idea that agile development is not disciplined. I think some of this might come from the lack of ceremony on agile development projects. Things get done and no one makes a big deal of it. Software is normally usable (and very likely releasable) at the end of every iteration, so the actual release can be anti-climatic. Contrast that with traditional methods where every phase or milestone is a big deal and release is often an earth shaking event. Maybe the traditionalists are just mistaking all their ceremony for real discipline.

Friday, September 28, 2007

Never Start With the Data Model

My bias is for object oriented design and domain modeling. I am convinced that you don't build great software by starting design with a data model. I think starting with the data model is just about the worst idea and pretty much guarantees mediocre software. The best systems start with a domain model and work through the various layers of the system down to the database, not the other way around.

The database should be left until as late in development as possible. Experience shows this is not the case because creating the database is usually one of the first things on a project task list. Instead we should be asking how soon do we really need a database? The unit tests will be better and faster when they don't need a database. You know that data access should be encapsulated behind an interface. Why not leave the database until we know what the data requirements of an at least partially working system are?

Honestly, I can't think of a harder way to try to conceptualize a system than through a data model. Do your users understand a database schema? Is a database schema the model of your system you want to carry around in your head? Great software requires a model that can be used to communicate with users, developers, product owners, and domain experts. I don't think a data model is how any business person conceptualizes their business processes. What you need is a consistent coherent domain model that can can be used to make sense of a system at any level. You must be able to "peel the onion" to discover details as needed. Design by data model immediately forces all the nasty gnarly implementation details front and center. It is a well understood user interface design principle that bad designs expose details of the data model via data entry screens. The worst designs percolate the data model from top to bottom.

The best software solutions model a problem in the problem domain. A data model is a solution in the database domain not the problem domain. So why is the database schema where design so often starts? Maybe it's the mainframe mentality in big corporations, or naive developers who don't really understand object orientation, or just a general ignorance of object oriented design and domain modeling. Eric Evans wrote great book on domain driven design that is not as widely read as it deserves to be. In that book he talks about the "ubiquitous language" of a software project -
"A domain model can be the core of a common language for a software project. The model is a set of concepts built up in the heads of people on the project, with terms and relationships that reflect domain insight. These terms and interrelationships provide the semantics of a language that is tailored to the domain ..."
I don't see such a language ever coming from a data model.

What prompted me to write this was a post touting the new release of an "active record" object relational mapping framework for Java and some controversy it generated in this post. As you can probably guess, I'm not really a fan of the active record design pattern. But reading the posts and the comments, and knowing how popular Ruby on Rails is, it's clear that lot's of developers are very big fans.

I don't get it. Maybe it's because active record is so easy to understand. I suppose active record might work for small systems. But I would never want to use it on any even moderately sized enterprise software project. I see enough ugly code, highly coupled, tangled classes, and mangled hierarchies without database access code mixed into my business classes and immediately coupling my design to the database. I shudder to imagine the maintenance headaches it would cause. At least I'm in good company -
"Another argument against Active Record is the fact that it couples the object design to the database design. This makes it more difficult to refactor either design as a project goes forward."

Martin Fowler, Patterns of Enterprise Application Integration
"In an object oriented program, UI, database, and other support code often gets written directly into the business objects. Additional business logic is embedded in the behavior of the UI widgets and database scripts. This happens because it is the easiest way to make things work, in the short run.

When the domain-related code is diffused through such a large amount of code, it becomes extremely difficult to see and to reason about. Superficial changes to the UI can actually change business logic. To change a business rule may require require meticulous tracing of UI code, database code, or other program elements."


Eric Evans, Domain Driven Design

I hope the fans of active record know what they could be getting themselves into, but as usual I doubt that is the case.

Tuesday, September 4, 2007

What makes software so hard?

Great article in the August issue of The Rational Edge, "What makes software so hard?". I have been convinced for some time that software is a design and not a manufacturing process. The article clearly describes the difference in explaining where in each type of process lie -
"On the other hand, for software projects, the relationship between
architecture and design versus production is not only inverted, but in
fact approaches infinity: since the production or manufacturing costs
for software are almost nil, almost all cost for a software
project comes from the workflows dealing with the creative parts --
that is, the invention and design of software."
Another great quote is the author's comparison of software to writing a novel. The metaphor fits so well -
"In a sense, creating software is far more closely related to writing a novel than to any traditional engineering effort -- an author is free to construct whatever type of story, limited only by his imagination and whatever facts may be necessary to the narrative. Furthermore, the author must construct the architecture of the story in his head, and to make sure the design and implementation of the story is consistent with the overall architecture. The quality of the author's product is to a high degree based on whether he succeeds in keeping the details of the story consistent with the architecture."
It is the notion that software is like manufacuring that has really turned me against big process improvement efforts like CMMi that are pushed from the top down. Anyone in a corporate environment has likely lived through at least one such effort. They usually seem so to be attempts to create predictability that does not exist in software and turn developers into interchangeable pieces that can be shuffled around a Gantt chart.

Friday, August 24, 2007

Discpline and Software Development

Jeff Atwood writes over at Coding Horror that "Discipline Makes Strong Developers". Discipline is important and the best developers are certainly disciplined. But what sort of discipline are we actually talking about? Let's start with what sort of discipline we are not interested in -
  • Not imposed discipline.
  • Not about drill sergeants or enforcers.
  • Not about being able to code in low level languages like C or assembler.
  • Not about being anal-retentive.
  • While I have great respect for Watt's Humphrey, it's not about recording and reporting every minute detail of your work day.
Disciplined software development is about -
  • self-discipline
  • focus
  • attitude
  • approach
  • organization
  • responsibility and accountability
This includes the having the discipline to -
  • write the unit tests
  • add the Java Doc comments
  • find the best name for a variable or method
  • leave the code better than you found it
  • fix problems not of your making
  • improve your skills
  • learn new things
  • keep the coding standards
  • use source control
  • check-in changes frequently
  • run the units often
  • run the unit tests before every check-in
  • write a test for a bug before you fix it
  • get your code reviewed
  • fix the problem not just treat the symptom
  • keep code consistent
As a technical lead I struggle with this. I have no desire to be the cop, but there can be too much bad code to ignore. One helpful option is automation. Use analysis tools to identify coding problems, violations of coding standards, and generate test coverage metrics. Automate your builds and do a nightly integration build or preferably a continuous integration build after every change. Run the analysis as part of every integration build. Fail the build if there are to many violations and notify the development team of every failure.

I think that ultimately it is up the individual developer to have pride in his or her work. I think the Pragmatic Programmers said it best. A developer must "Care About Your Craft" and "Think! About Your Work".

Tuesday, August 21, 2007

What is software design?

Michael Feathers has a recent blog "It's All Design" where he says:

When I think about what we do in software development, I find it hard to imagine similar things happening in other fields.

He goes on to speculate that an auto designer would never be given a requirement such as there must be fifteen drink holders. I think he's wrong and that we operate in the same way as in many other fields. Come on, there really are vehicles with fifteen cup holders and I can't imagine any auto designer doing that without such a requirement.

In the comments Michael says "I try to imagine what happens in conversations with architects.". Well my wife has a degree in Architecture, but worked in the field only briefly before moving into software development. She has found great parallels between the work in both fields. We are currently working with an architect to design an addition to our house and one of the first things our architect did was ask about any features and requirements we had in mind.

Don't get me wrong, I agree that software development is just one big design process. I really hate how the software development process is so often compared to a manufacturing process. In software the manufacturing is not the development of an application it is copying that application to a disk. What we do, even the lowest level coding, is design not manufacturing.

But I don't agree with Michael Feathers that requirements and design are the same thing:

Maybe requirements is just a word that we use because we're dividing our design work between two groups.. a group that determines the higher level design (what the product will do); and the lower level design (how the product will do it).

Requirements frame design but are the same as design. Something like a performance requirement does not say anything about the design of a product. Software development is a design process and requirements are part of the process, but a stating a requirement is not the same as making a design decision.

Thursday, August 16, 2007

"Mocks Aren't Stubs" Revisited

Reading the book "xUnit Test Patterns: Refactoring Test Code" made me take another look at Martin Fowler's 2004 article "Mocks Aren't Stubs". I'm glad I did because the article has been significantly updated (it's practically a rewrite and nearly twice as long) to reflect new thinking about mock objects. The first thing I noticed was the new terminology that is consistent with the "xUnit Test Patterns" book. The second thing I noticed is the new ideas about the different styles of using mock objects. The updated article is definitely worth a read or a re-read.

It looks like we are beginning to develop a shared language around unit testing, like what happened with refactoring and design patterns. It's interesting to watch this kind of thing as it matures and we learn more. It reminds we of how much object oriented development has changed over the last twenty years.

Thursday, July 5, 2007

What's Your Build Process?

So what is your build process like? Is simple or complicated? Is it manual or automated? How many of the Five R's of Agile SCM Baselines does it satisfy? I've been doing a lot of work on software build process lately and here is a simple four step process that mostly satisfies those "Five R's" -

  1. Get the latest sources from the SCM repository (branch or mainline).
  2. Execute the build against the latest sources.
  3. Commit the results of the build back to the SCM repository.
  4. Label the results of the build in the SCM repository (branch or version as needed).
Four steps, looks pretty simple. I can't think anything simpler that would be very robust. So how about automating the process? What would be simplest way to get this fully automated and as mistake proof as possible. There are lot's of build server out there, both open source and commercial, it seems like on of them ought to be able to handle the job.

I've evaluated quite a few build servers recently (I'll post reviews later) and can't find one suitable product. They all easily handle continuous integration builds, but none can automate all four builds steps. Obviously, all the servers can mange step 2, and they all partially handle step 1, i.e. getting the sources from the main line. All the servers can check out source code, but if you want to do it from more than one branch you need a configuration for each branch. None of the servers can commit anything back to the repository without external scripting of some sort let alone branch your project. If I need to write scripts or maintain multiple configurations what have I gained over just doing the four step process manually?

This is for Java application development and Ant is used for the building. I've looked at enhancing the Ant builds and as much as I like Ant, it is just not well suited for this task by itself. It might be possible to write custom Ant tasks, but I really don't want to create a custom build system that needs to be maintained. It's enough work just maintaining the Ant builds as they are. Something seems wrong when it is so hard to automate such a simple process.