Make it Maintainable and Readable First

You're probably aware of the Donald Knuth quote "Premature optimization is the root of all evil (or at least most of it) in programming.". I reviewed some code recently that reminded me of that quote and why making code readable and maintainable is the first order of business. The code I reviewed reversed the usual order for checking for a null reference to -

if (null != instance) { ...}

I don't like this code because it uses a non-standard idiom and forces me to stop and read the code from right to left to understand it. I've run across this idiom once before and the best reason I have gotten for using it is that it is supposed to be faster. Well, I can't imagine why it would be faster and even it if was there are certainly better performance optimizations that don't affect code readability. Being curious I decided to test the this optimization and wrote a little java test code.

public class NullTest {
public static long testNullFirstAgainstNonNull(int count, boolean display) {
  long start = System.currentTimeMillis();
  Object obj = getObject();
  int x = 0;

  for (int i = 0; i < count; i++) {
    if (null == obj) {
      x++;
    }
  }

  long end = System.currentTimeMillis();
  if (display) System.out.println("if (null == [non-null])   --  " + (end - start));
  return x;
}

public static int testNullFirstAgainstNull(int count, boolean display) {
  long start = System.currentTimeMillis();
  Object obj = getNullObject();
  int x = 0;

  for (int i = 0; i < count; i++) {
    if (null != obj) {
      x++;
    }
  }

  long end = System.currentTimeMillis();
  if (display) System.out.println("if (null != [null])       --  " + (end - start));
  return x;
}

public static int testNullLastAgainstNonNull(int count, boolean display) {
  long start = System.currentTimeMillis();
  Object obj = getObject();
  int x = 0;

  for (int i = 0; i < count; i++) {
    if (obj  == null) {
      x++;
    }
  }

  long end = System.currentTimeMillis();
  if (display) System.out.println("if ([non-null]  == null)  --  " + (end - start));
  return x;
}

public static int testNullLastAgainstNull(int count, boolean display) {
  long start = System.currentTimeMillis();
  Object obj = getNullObject();
  int x = 0;

  for (int i = 0; i < count; i++) {
    if (obj != null) {
      x++;
    }
  }

  long end = System.currentTimeMillis();
  if (display) System.out.println("if ([null] != null)       --  " + (end - start));
  return x;
}


private static Object getObject() {
  return new Object();
}

private static Object getNullObject() {
  return null;
}

public static void main(String[] args) {
  int count = 100000;
  testNullFirstAgainstNonNull(count, false);
  testNullFirstAgainstNull(count, false);
  testNullLastAgainstNonNull(count, false);
  testNullLastAgainstNull(count, false);
  testNullFirstAgainstNonNull(count, false);
  testNullFirstAgainstNull(count, false);
  testNullLastAgainstNonNull(count, false);
  testNullLastAgainstNull(count, false);

  count = 100000000;
  System.out.println(" Results from " + count + " executions.");
  System.out.println("============================================================");
  testNullFirstAgainstNonNull(count, true);
  testNullFirstAgainstNull(count, true);
  testNullLastAgainstNonNull(count, true);
  testNullLastAgainstNull(count, true);
  System.out.println();
  testNullFirstAgainstNonNull(count, true);
  testNullFirstAgainstNull(count, true);
  testNullLastAgainstNonNull(count, true);
  testNullLastAgainstNull(count, true);
  System.out.println();
  testNullFirstAgainstNonNull(count, true);
  testNullFirstAgainstNull(count, true);
  testNullLastAgainstNonNull(count, true);
  testNullLastAgainstNull(count, true);
  System.out.println();
  testNullFirstAgainstNonNull(count, true);
  testNullFirstAgainstNull(count, true);
  testNullLastAgainstNonNull(count, true);
  testNullLastAgainstNull(count, true);
  System.out.println();
  testNullFirstAgainstNonNull(count, true);
  testNullFirstAgainstNull(count, true);
  testNullLastAgainstNonNull(count, true);
  testNullLastAgainstNull(count, true);
  System.out.println();
}
}

The test tries to prime execution and warmup the JVM by running each test before displaying any results. The usual caveats about micro-benchmarks, etc. apply to the results below. I ran this under to Sun JVMs, 1.4 and 6, with similar results that show no performance benefits and just reinforce for me the importance of writing code that is readable, maintainable and understandable.

$ /c/j2sdk1.4.2_17/bin/java -version
java version "1.4.2_17"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2_17-b06)
Java HotSpot(TM) Client VM (build 1.4.2_17-b06, mixed mode)

$ /c/j2sdk1.4.2_17/bin/java NullTest
Results from 100000000 executions.
============================================================
if (null == [non-null])   --  141
if (null != [null])       --  156
if ([non-null]  == null)  --  141
if ([null] != null)       --  172

if (null == [non-null])   --  141
if (null != [null])       --  156
if ([non-null]  == null)  --  140
if ([null] != null)       --  172

if (null == [non-null])   --  141
if (null != [null])       --  172
if ([non-null]  == null)  --  140
if ([null] != null)       --  157

if (null == [non-null])   --  140
if (null != [null])       --  172
if ([non-null]  == null)  --  141
if ([null] != null)       --  156

if (null == [non-null])   --  141
if (null != [null])       --  172
if ([non-null]  == null)  --  140
if ([null] != null)       --  172


$ java -version
java version "1.6.0_07"
Java(TM) SE Runtime Environment (build 1.6.0_07-b06)
Java HotSpot(TM) Client VM (build 10.0-b23, mixed mode, sharing)

$ java NullTest
Results from 100000000 executions.
============================================================
if (null == [non-null])   --  62
if (null != [null])       --  63
if ([non-null]  == null)  --  62
if ([null] != null)       --  63

if (null == [non-null])   --  62
if (null != [null])       --  78
if ([non-null]  == null)  --  63
if ([null] != null)       --  62

if (null == [non-null])   --  63
if (null != [null])       --  62
if ([non-null]  == null)  --  63
if ([null] != null)       --  62

if (null == [non-null])   --  63
if (null != [null])       --  62
if ([non-null]  == null)  --  63
if ([null] != null)       --  62

if (null == [non-null])   --  63
if (null != [null])       --  62
if ([non-null]  == null)  --  63
if ([null] != null)       --  62

Just Say No to Deflaut Configuration

What I'm talking about is when an application sets its configurable properties using default values because the configuration for the environment the application is intended to run in cannot be set. Default configuration can be the result of specific application logic or a feature of the configuration framework or mechanism used. How this usually plays out is that some generic set of values, often the ones used in the "dev" environment, are used instead of the values for a specific environment like "qa", "stage", or "prod". Sometimes a debug or info message or, if you are lucky, a warning message will be logged. But just as often there is no direct evidence that the desired configuration has not been used.

At first blush having a default configuration to fall back on seems like a good idea. It can simplify development, especially unit testing, because things just work. There is no need to understand or be bothered with the details about configuration before you've even started coding. Where this will come back to bite you, and I guarantee that sooner or later it will, is when someone forgets to configure your application before a new production deployment. Everything looks great, the smoke tests pass, and everyone is happy. At least they are happy until, for some unknown reason, the pricing for widget XYZ is wrong or the customer ABC just seems to have disappeared. Since you turned off info and debug logging in production, there is nothing in your logs to indicate a problem. Now you get to spend some time frantically looking through code for the change, that doesn't exist, that is causing the problems.

Using default configuration violates the principle of failing fast. When there is a problem in your application, the application code should fail as soon as possible and it should be crystal clear why it failed. You don't want to detect a problem through side effects. When there is a serious error, and what could be more serious than incorrect configuration, I want to see a big red flag waving or at least a big loud error message in the logs! I want my application to fail before it can startup and cause even greater problems by reading or writing the wrong database or processing messages from the wrong message queue.

Bonehead Exception Handling

I found this bit of Java exception handling code in a major open source project -

 
try {
    ....  
} catch (ClassNotFoundException e) {
  throw new IntegrationException("Rule Set jar is not correctly formed.");
} catch (IOException e) { 
  throw new IntegrationException("Rule Set jar is not correctly formed."); 
};

The specifics of the code in the try block are unimportant, what does matter is how the exceptions thrown by that code are handled. Besides being completely amateurish, the exception handling is wrong in at least three ways.

First it completely loses the original exception. Whatever context the original exception had is just gone. Since Java 1.4 it has been possible to include the root cause of the error. Before Java 1.4 the programmer could at least have appended the original error message.

Second it turns two completely different types of errors in a one type. Great so now I get an error that could have two very different causes and no way to figure out the actual cause.

Third the error message doesn't say anything at all useful. There is absolutely nothing in the error message that helps me understand or fix the problem. What does "not correctly formed" mean and what in the world does being or not being well formed have to do with a ClassNotFoundException or an IOException?

I suppose I should take some solace in the fact that at least it reports an error. But frankly this code is only slightly better than just swallowing the errors.

10 Principles for Developer Testing

1) Code a Little, Test a Little

Test driven development is all the rage and is a great way to do "test a little, code a little". But it is important to remember that it is the results not how you achieve them that matter most. As long as test code is written concurrently with implementation code, it doesn't matter if you follow strict test driven development practices because when the tests are done you won't be able to tell what method was followed. The only wrong way to write test code is after the implementation code is done.

2) Use Tests to Improve the Design

Testability is an excellent indicator of design quality. If your code is hard to test then the design needs to be fixed. Look for warning signs and use the tests to guide the design. Hard to test code includes singletons, static calls, and too many parameters. Warnings signs of a design that needs help include making private methods public for testing, complicated test setup, excessive test assertions and mocking concrete classes.

3) Write as Much Test Code as Implementation Code

On average you should expect to write at least as many lines of test code as there are implementation code.

4) Test the Right Thing

It's a unit test, so it should be testing one unit. Since most of us are doing object oriented development the unit is a class. Make sure you are only testing the class under test and not some infrastructure such as database or web server. Make sure you are only testing the class under test and not other classes it may depend on, the dependent classes should already have been well tested by their own unit tests. Make sure you test the things that are likely to break and that usually does not include things like simple getters and setters.

5) Test the Error Handling

You need to test more than the expected paths through the code. Make sure to test what the code will do under error conditions. Is it catching the right exceptions? Does it handle errors gracefully and cleanup things like database connections?

6) Always Write a Test Before Fixing a Bug

Whenever you find a bug write a test that fails because of the bug before you try and fix the bug. If you don't have a test before you attempt fix the bug how exactly will you know the bug is gone? Adding a test for every bug will also help ensure that the same bug does not creep back into the system later.

7) Mock Objects are Your Friends

If you are not using mock objects you need to start. There are numerous excellent and mature open source mock object frameworks for most any language. The benefits of mock objects are to great to ignore. Mock objects let you write truly isolated unit tests, speed up you unit tests, and test things such as error handling that would probably be impossible to automate without mocks.

8) Use a Code Coverage Analyzer

Test coverage analyzers are wonderful tools that can help find inadequately tested code. But always keep in mind that test coverage is about the journey not the destination. Your goal is to have better tested code not necessary 100% code coverage. There are many excellent open source and commercial coverage tools available so there is no reason not to use one.

9) Don't be Fooled by High Code Coverage Numbers

Depending on the metric used it is possible to have 100% coverage with garbage unit tests. Statement coverage is the poorest way to measure coverage because it only says which lines were executed and says nothing about what branches or paths in the code were tested. Branch coverage tells you which branches in a control structure such as an if or switch statement were executed. So branch coverage combined with statement coverage is pretty good. A better metric is path coverage that will tell you which execution paths through the code were covered.

10) Know Your Cyclometric Complexity

Cyclometric complexity is a measure of how many execution paths there are through a piece of code. A method or function with no control flow, such as a if statement, will have a complexity of one. The complexity goes up for each execution branch such as and if or else statement. Cyclometric complexity is a good way to get a feel for how many tests a method needs to cover all execution paths through it.

Behavior Driven Development is not Domain Driven Design?

I had been aware of behavior driven development for a while, but researching my previous post on Test Driven Development I came across something that really had me scratching my head. The general view of BDD is an evolution or refinement of TDD with the goal of making TDD more natural and easier to understand. A big part of BDD is a change in mindset from thinking about testing to thinking about behavior and class design. To this end doing BDD has behavior classes, not test cases. BDD also has methods that specify behavior with names that clearly describe the expected behavior and begin with "should" instead of "test". All this is well and good and I expect will make things easier to understand, especially for developers new to agile development.

What has me more than a little bothered is the linking of BDD with "domain driven design" and the "ubiquitous language" as described by Eric Evans in his excellent book. The Behaviour-Driven Development web site and Wikipedia entry read as if BDD is somehow a logical extension of domain driven design. I read everything I could find and saw nothing to backup the claim. Maybe the site is incomplete, or perhaps it just shows the weakness of implementing the Behaviour-Driven Development site as a wiki. So far I am not convinced that BDD has the depth to accomplish domain driven design. From Domain-Driven Design -

"Domain-driven design is not a technology or a methodology. It is a way of thinking and a set of priorities, aimed at accelerating software projects that have to deal with complicated domains."

Stories will not produce the rich domain model that is needed. Here is the template for a story -

Title (one line describing the story)

Narrative:
As a [role]
I want [feature]
So that [benefit]

Acceptance Criteria: (presented as Scenarios)

This is just an variation on the agile user story and is still too simple, low level, to narrow in focus to result in deep domain knowledge. BDD may give you a well crafted set of classes that implement a feature or story. But how will those classes look in the context of the domain? Will the result superficial or insightful? Will the object connections and interactions be deep and rich, full of meaning, or shallow and empty?

Will the developers following BDD understand the vision of the domain model? Will they understand that changing the code is changes the model and the changing the model requires changing the code? It's an issue of the old saying "not seeing the forest for the trees". How do you see the big picture when the focus is so tight on a story and feature and the classes that implement them.

"Getting the Words Right" and method names that start with "should" and read like sentences won't guide anyone toward a ubiquitous language for the domain. Getting the language right requires a thorough understanding the the problem domain and that does not come from coding user stories.

Don't get me wrong I'm a big believer in agile development and unit testing. I understand that TDD and BDD are as much about design as they are about testing. I been using JUnit since something like 1999. I know how much better unit tests make my code and how much they can improve the design of the classes being tested. But I don't use tests to for domain modeling.

References

DanNorth.net » Introducing BDD
Beyond Test Driven Development: Behaviour Driven Development (Google Video)
In pursuit of code quality: Adventures in behavior-driven development
Test Early » Using BDD to drive development

Test Driven Devlopement Reality Check

Over the last few months I've been reading blogs and articles evaluating the scope and scale of TDD -

Scaling Test-Driven Development
Unit testing; how far do you push the envelope?
TDD Proven Effective! Or is it?
What's more important: TDD or acceptance testing?

But what really got my attention was a blog by James Coplien, "Religion's Newfound Restraint on Progress", and a debate about the value of TDD between Coplien and Bob Martin recently posted at InfoQ. James Coplien has been on the forefront of software development trends for a couple decades. He is an object oriented and design patterns guy from way back and an agile coach today. His opinion is worth is paying attention. That's why it was quite a surprise to read how opposed he is to TDD. Whether you agree or not it's certainly worth some consideration. All the quotes below are by James Coplien from his blog or his comments on several other blogs (you'll need to read the blog comments for the full text of some of the quotes below).

'The answer, I think, is to focus on Quality and to find the techniques that work best and are the most cost-effective for your project. Just answering "both" to the "TDD-or-acceptance" question is not only at the wrong scope, it is missing the whole point.'

I think Coplein's objections come down two main points, cost effectiveness and the effects that TDD may have on a system's architecture. Further he is concerned that the procedural nature of TDD just does not fit well with the object prardigm.

'Most of TDD is based on xUnit, and the unit of any given test is a procedure.'

'Testing the functionality of object member functions causes one to focus on the functions and lose sight of the objects ...'

I've been programming with objects since the late 1980's so this hits home. The overall quality of an object oriented application is much more than the quality of the individual objects. It's about how the objects interact and fit together. We need to look at a higher level than a method or even a class to understand that interaction and create a good design.

'... this implies that TDD doesn’t make sense for a system where objects are the primary organizing principle. I don’t know how to test an object. Objects are about structure; what needs to be tested is the degree to which they map the user’s cognitive model of the domain.'

I'm still trying to absorb this completely, but my take again is that it's not about a single method or class, it's about the whole system and the objects it is constructed from. On the other hand I know from experience that tests inform class design. When a class is hard to test, perhaps I feel the need to make a private method public to test it properly, I take that as a sign that the class probably needs work.

'In the past five years I have yet to run into a single person who is doing it for other than one of two reasons: that it makes them feel good about themself, or that they are a lemming.'

Ouch, seems a bit strong and even hurts a little because I have to admit that good JUnit tests do make me happier. (The rest of these quotes are from "Religion's Newfound Restraint on Progress").

'Discussions in this area inspire passion. Why? ... People have tied their Agile success to their introduction of TDD practices into their work. It's the one thing they can do as individuals rather than as an enterprise.'

It's true, but I think part of it is also that "good" developers have pride in their work. TDD and unit tests in general are a way we get to directly impact the quality of our code without interference.

'On one hand we should be encouraged to use every weapon in our arsenal; on the other hand, we'll never get done if we try to do everything. It then becomes a matter of cost effectiveness. Integration and system testing have long been demonstrated to be the least efficient way of finding bugs. Recent studies (Siniaalto and Abrahamsson) of TDD show that it may have no benefits over traditional test-last development and that in some cases has deteriorated the code and that it has other alarming (their word) effects. The one that worries me the most is that it deteriorates the architecture. And acceptance testing is orders of magnitude less efficient than good old-fashioned code inspections, extremely expensive, and comes too late to keep the design clean.'

'Sorting these things out requires a system view. It used to be called "systems engineering," but the New Age Agile movements consistently move away from such practices. It shows in the quality of our interfaces, in the incredible code bloat we have today relative to two decades ago..'

It seems reasonable to ask if complete unit test coverage is really the most cost effective way to reduce bugs? I know unit testing has improved the quality of the applications I've worked on the last eight years. But could we have written fewer tests and gotten the same level of quality?

I've always been skeptical that TFD or TDD alone can produce a good architecture for the long haul. As much as I love refactoring, I'm not convinced that I can refactor my way to a great design. The idea that there is no need for any overall system design is crap. At the same time the idea that an architect can completely design a system upfront is just as crappy. Some level of system design and vision needed, but we have to learn from the system and adjust the design as we iterate. I'm a big proponent of "domain driven design" and the bottom line is I just don't think a real domain design will just emerge from coding, testing and refactoring. Domain design takes hard work and thinking, design effort not coding. Writing code is too low-level, too focused on details rather than useful domain abstractions.

'Agile is about engaging that user; TDD, about engaging the code. TDD is a methodology. TDD is about tools and processes over people and interactions. Woops.'

That's a pretty stinging indictment. If true if TDD isn't following agile principles. But I wonder if some the power of TDD is the simplicity and explicitness of its method. The best developers can succeed no matter what the process, but we all no that most software is not written by star developers. TDD sems simple enough fpr anyone to understand and follow to at least improve the quality of class level design and that's be worth something.

'It's about thinking, not about checklists.'

I can't disagree here because I've know too many developers who want easy answers or rules to follow. Software isn't like that, it's to much of a craft and takes constant attention and thinking to do well.

Overall I don't agree with James Coplien, but I think the discussion is worth having and hope everyone will keep an open mind. I know that unit testing (and lot's of it) has improved the quality of the my code and code written by my teams over the last eight years. I'm not a believer in one true way for software development or anything else. I think it is necessary to create a software process that works for you, taking the best practices that work for your in you environment.

Big Ball of Mud

This post at Object Mentor reminded me that I'd never actually read the "Big Ball of Mud" paper by Brian Foote's and Joseph Yoder. Quoting from the paper -

"A BIG BALL OF MUD is haphazardly structured, sprawling, sloppy, duct-tape and bailing wire, spaghetti code jungle. We’ve all seen them. These systems show unmistakable signs of unregulated growth, and repeated, expedient repair."

I'm sure everyone has seen one, worked on one, and even created one. I'm also certain that in general things aren't much different since that paper was first published in 1999. Here is a link to a "Big Ball Of Mud" Google video presentation by Brian Foote from August 2007. I guess that since the authors are still talking about the pattern we should expect it to be around for a while longer.

The next time you need to wade into the swamp, you might want to take a copy of the book "Working Effectively with Legacy Code" with you.

Testability and Design

As Michael Feathers notes, 'The Deep Synergy Between Testability and Good Design', there is a lot of discussion about how to test private methods. I think he's absolutely correct that the need to test a private method is a hint about the design. I've been reviewing a class that I was never completely happy with because it exposes private methods for testing. Looking at it today I'm sure it would be possible to extract a class that would improve the design and fix the problem. It would have been much easier to clean this up when I was writing the code than to find the time to fix it now. I wish I had listened closer to the hint that my test was giving me.

It's pretty amazing how much we learn about the goodness of a design when try to test it. In the past I used to write fewer and larger classes than I do today. Most of the reason for the change is for better unit tests.

The Discipline of Agile

An article by Scott Ambler at Dr. Dobb's does a nice job refuting the idea that agile development is not disciplined. I think some of this might come from the lack of ceremony on agile development projects. Things get done and no one makes a big deal of it. Software is normally usable (and very likely releasable) at the end of every iteration, so the actual release can be anti-climatic. Contrast that with traditional methods where every phase or milestone is a big deal and release is often an earth shaking event. Maybe the traditionalists are just mistaking all their ceremony for real discipline.

Never Start With the Data Model

My bias is for object oriented design and domain modeling. I am convinced that you don't build great software by starting design with a data model. I think starting with the data model is just about the worst idea and pretty much guarantees mediocre software. The best systems start with a domain model and work through the various layers of the system down to the database, not the other way around.

The database should be left until as late in development as possible. Experience shows this is not the case because creating the database is usually one of the first things on a project task list. Instead we should be asking how soon do we really need a database? The unit tests will be better and faster when they don't need a database. You know that data access should be encapsulated behind an interface. Why not leave the database until we know what the data requirements of an at least partially working system are?

Honestly, I can't think of a harder way to try to conceptualize a system than through a data model. Do your users understand a database schema? Is a database schema the model of your system you want to carry around in your head? Great software requires a model that can be used to communicate with users, developers, product owners, and domain experts. I don't think a data model is how any business person conceptualizes their business processes. What you need is a consistent coherent domain model that can can be used to make sense of a system at any level. You must be able to "peel the onion" to discover details as needed. Design by data model immediately forces all the nasty gnarly implementation details front and center. It is a well understood user interface design principle that bad designs expose details of the data model via data entry screens. The worst designs percolate the data model from top to bottom.

The best software solutions model a problem in the problem domain. A data model is a solution in the database domain not the problem domain. So why is the database schema where design so often starts? Maybe it's the mainframe mentality in big corporations, or naive developers who don't really understand object orientation, or just a general ignorance of object oriented design and domain modeling. Eric Evans wrote great book on domain driven design that is not as widely read as it deserves to be. In that book he talks about the "ubiquitous language" of a software project -

"A domain model can be the core of a common language for a software project. The model is a set of concepts built up in the heads of people on the project, with terms and relationships that reflect domain insight. These terms and interrelationships provide the semantics of a language that is tailored to the domain ..."

I don't see such a language ever coming from a data model.

What prompted me to write this was a post touting the new release of an "active record" object relational mapping framework for Java and some controversy it generated in this post. As you can probably guess, I'm not really a fan of the active record design pattern. But reading the posts and the comments, and knowing how popular Ruby on Rails is, it's clear that lot's of developers are very big fans.

I don't get it. Maybe it's because active record is so easy to understand. I suppose active record might work for small systems. But I would never want to use it on any even moderately sized enterprise software project. I see enough ugly code, highly coupled, tangled classes, and mangled hierarchies without database access code mixed into my business classes and immediately coupling my design to the database. I shudder to imagine the maintenance headaches it would cause. At least I'm in good company -

"Another argument against Active Record is the fact that it couples the object design to the database design. This makes it more difficult to refactor either design as a project goes forward."

Martin Fowler, Patterns of Enterprise Application Integration

"In an object oriented program, UI, database, and other support code often gets written directly into the business objects. Additional business logic is embedded in the behavior of the UI widgets and database scripts. This happens because it is the easiest way to make things work, in the short run.

When the domain-related code is diffused through such a large amount of code, it becomes extremely difficult to see and to reason about. Superficial changes to the UI can actually change business logic. To change a business rule may require require meticulous tracing of UI code, database code, or other program elements."

Eric Evans, Domain Driven Design

I hope the fans of active record know what they could be getting themselves into, but as usual I doubt that is the case.

There Aren't Any Rules

I know a lot of programmers won't agree but there are few, if any, "rules" for writing good code. It doesn't matter what platform or language or type of software you are writing. If you think someone can hand you a set of rules that you can follow to write good code you are mistaken. I try not to call anything a rule anymore. Now I call them guidelines or principles. The difference is that a guideline needs to be interpreted and judged to know when it should be applied. A guideline might also apply to more than one case.

Sorry, but I reserve the right to change my mind tomorrow about what I told you today was the right thing. This profession is too young, changes too fast, and is too much of a craft for rules written in stone. I've been doing this long enough to know that many things we consider best practices today are likely to change tomorrow. Remember hungarian notation? At one point you would have flunked a code review if you didn't use it, now you'd be more likely to flunk if you did. How about object oriented design? I remember when inheritance and polymorphism were going to save the world. Now too much inheritance is a sure sign of bad design. How about unit testing? We're still discovering the best techniques for things like mock objects.

Much of what makes good code depends on situation and context. Michaels Feathers says it very clearly -

"Good programming is contextual. Practice is contextual. We can articulate rules about how to use language constructs correctly, but they're just guidelines. Context is king."

When Good Code Looks Bad, Michael Feathers

I'd like to add that it is also about having the knowledge and judgment to know which guideline to apply when. There aren't any shortcuts, it takes judgment, practice, and study to write good code. Becoming a good programmer is hard work and you won't be one by following a set of rules. But you might become one if you practice and study the art of programming.

Metrics To Improve Unit Testing

At Test Often Joe Ponczak lists seven metrics that you can use to make your methods more testable, "Seven Metrics to Improve Your Unit Testing". Experienced unit testers will probably be familiar with the metrics but it never hurts to review. I mainly mention this because it is the blog for CODIGN Software which makes some nice reasonably priced Eclipse plug-ins for writing JUnit tests and analyzing code coverage.

p.s. I sure don't like the white on black color scheme on the blog though.

What makes software so hard?

Great article in the August issue of The Rational Edge, "What makes software so hard?". I have been convinced for some time that software is a design and not a manufacturing process. The article clearly describes the difference in explaining where in each type of process lie -

"On the other hand, for software projects, the relationship between
architecture and design versus production is not only inverted, but in
fact approaches infinity: since the production or manufacturing costs
for software are almost nil, almost all cost for a software
project comes from the workflows dealing with the creative parts --
that is, the invention and design of software."

Another great quote is the author's comparison of software to writing a novel. The metaphor fits so well -

"In a sense, creating software is far more closely related to writing a novel than to any traditional engineering effort -- an author is free to construct whatever type of story, limited only by his imagination and whatever facts may be necessary to the narrative. Furthermore, the author must construct the architecture of the story in his head, and to make sure the design and implementation of the story is consistent with the overall architecture. The quality of the author's product is to a high degree based on whether he succeeds in keeping the details of the story consistent with the architecture."

It is the notion that software is like manufacuring that has really turned me against big process improvement efforts like CMMi that are pushed from the top down. Anyone in a corporate environment has likely lived through at least one such effort. They usually seem so to be attempts to create predictability that does not exist in software and turn developers into interchangeable pieces that can be shuffled around a Gantt chart.

The Mysteries of Debugging?

Why do so many programmers find debugging so hard? Sure there are exceptionally wicked bugs, but most of the time we make debugging harder than it needs to be. The only secret I know of is having the right attitude and using the right approach.

How Not to Debug

Trial and error

Just guess at what the problem is. Add lots of print statements to the code and hope one of them shows you what the problem is. Make changes until the problem goes away. You don't need to know the cause as long as the bug is fixed.

Blame it on the ...

That's impossible, certainly it can't be a mistake in my code. It must be the compiler, database, network, and so on and so on.

Don't Understand the Problem

Don't dig deep enough to understand the cause. Fix the first and most obvious thing you find. Then fix the next most obvious thing when the same bug shows up again later.

Gentlemen Start Your Debuggers

That's what the debugger is for so fire that sucker up and start stepping. We'll just step through every line of code until we find the bug. You've got nothing but time right?


The Right Approach to Debugging

First don't panic! Programming is about solving problems and a bug is just another problem to solve. Of course you must approach debugging in a logical and organized way. The first thing you need are some some clues. Using the clues you can develop a theory and tests to validate the theory. Once your theory is validated you can implement a fix. This is very similar to applying the scientific method :

1. Gather data
2. Form a hypothesis
3. Perform experiments that test the hypothesis
4. Prove or disprove the hypothesis
5. Rinse and repeat as needed

Techniques for Successful Debugging

Repeatability

You need to reliably reproduce the bug. If you can't reproduce it when needed you can't test it or know when it is fixed. Reproducing a bug can be the hardest part of debugging.

Find the simplest test case that demonstrates the bug. You want to make it quick and easy because you will need to recreate the bug many times. The harder the bug is to recreate the less sure you will be of the cause and your solution. It is often worth the effort to create the smallest simplest program with the least code and clutter that shows the bug.

Analyze All the Available Data

Before rushing into a theory about the cause of a bug you need to make sure you have completely analyzed all the data that you have about it. Don't jump to conclusions because your first instinct will often be wrong. Look at the problem from as many directions as possible first.

Make sure you understand what the data is saying about the problem. We've got a great new technology called an exception that holds enough information all by itself to tell you the exact problem and the line of code where it occurred. Take the time to read and understand the full exception output. Time and gain I find myself pointing out to a programmer that the exception is telling them exactly what the problem is and all they need to do is read the exception output.

Turn on as much application logging as possible and take the time to thoroughly examine the log or trace files. There are so many freely available open source, high quality, easy to use, logging and tracing frameworks available for every platform that there is no excuse for any application not to generate high quality error and debug logs.

Narrow Things Down

Use a binary search or divide and conquer technique to zero in on the problem code. You need an organized hunting expedition not a haphazard ramble through the code to find bugs.

Look at what has changed recently. If things worked fine last week then figure out what has changed in the code or its runtime environment and look there first.

Explain the Bug to Somebody Else

When you aren't making any progress, stop, take a breath and find someone else to talk the problem over with. So often the simple act of explaining a problem generates an insight before you are even finished with the explanation. If just explaining things doesn't work then the other person may have a great idea of their own.

Fix the Real Problem

The symptom you see may not be the actual bug. You need to find and fix the root cause not the symptom. Be sure you are fixing the problem and not just treating a symptom. When you find the problem look around for any similar problems. We all tend to make the same mistake more than once.

Write a Test Before You Fix

First, The test will be a good demonstration of the bug and when the test succeeds it can be proof that the bug is fixed. Second, you just spent valuable time finding and fixing the bug and a test will help ensure that it does not come back to waste your time again.

The Compiler is Not Broken

The compiler, database, wahtever is not broken. There could be bug there, but don't start from that assumption it will just waste time. Believe me it is not a bug in the compiler, compiler writers are way smarter than you or me.

One Change at a Time

Never make more than one change before testing the bug again. If you make two changes how will you know which one fixed the bug and if which change is actually needed.

Check the Simplest Thing First

Bugs are often caused by some silly mistake or oversight and the simple things are easy to check and fix. The unlikely things are hard to check, so save them for later.

Use the Debugger

I saved this one almost for last because it should be a last resort. Debuggers are wonderful and powerful tools! But debuggers can also be tedious, time consuming, and confusing. Sometimes they are the only way to figure a problem out, but to use a debugger effectively you first need to narrow down the code that needs to be checked. Don't let the debugger be the first tool you reach for.

Use Tools to Find Bugs Before You Deploy

The best way to fix a bug is to never let it get deployed. It should not be necessary to remind any programmer to turn on as many compiler warnings as possible, but unfortunately, I know it is. When you have all the compiler warnings removed from your code, run a static analysis tool aver it too. There are many open source and commercial code analysis tools available so get at least one and use it to analysis ALL of your code for bugs.

References

"The Pragmatic Programmer: From Journeyman to Master" by Andrew Hunt, David Thomas
"Code Complete" by Steve McConnell
"The Practice of Programming" by Brian W. Kernighan, Rob Pike
Debugging strategy: easy stuff first
Fix The Bug, Not The Symptom

New Application of the Builder Pattern

At "Mistaeks I Hav Made" Nat Pryce writes about an alternative to the Object Mother pattern that is based on the Builder pattern I blogged about previously. It looks like a useful application of the Builder pattern. Object Mother is a technique for creating test data for unit tests. I've used Object Mothers and the article is right that overtime they get messy and bloated.

In he last line of the article Nat says -

"In some cases, Builders have so improved the code that they ended up being used in the production code as well."

Which is a nice validation of underlying the Builder pattern. I wonder if they used the same builder, complete with the default values, in the production code or wrote new ones.

Developer Testing is Habit Forming (and Habit Changing)

A while back Tim Ottinger wrote a great article about how "Testing Will Challenge Your Conventions". Among the points that really hit home for me were -

1. "Interfaces suddenly seem like a really good idea...". Mock objects can be so useful when used correctly that creating an interface is often the first thing I do.
   
2. "Singletons and static methods no longer seem like a great way to do work...". As I hope everyone knows by now the Singleton pattern is way overrated and makes writing truly isolated unit tests very hard.
3. "Private makes less sense...". I still struggle with making a method public just for testing but sometimes there is no other way. Whenever I find the need to do so, I give my design a hard look to make sure it is as good as it should be.
4. "You need to be able to pass a class everything it might need at construction time...". To write isolated unit tests your classes cannot configure themselves. Of course long constructor parameter lists can be a problem but I can use the builder pattern as one way to counter that.
5. "Smaller methods are the norm." When I think about the size of methods I write now compared to a few years ago I am amazed at how small the methods are now. It is getting to where I can't read or don't have the patience to read methods over 10 to 15 lines long.

That's a few of my thoughts go read the whole things for yourself.

Discpline and Software Development

Jeff Atwood writes over at Coding Horror that "Discipline Makes Strong Developers". Discipline is important and the best developers are certainly disciplined. But what sort of discipline are we actually talking about? Let's start with what sort of discipline we are not interested in -

• Not imposed discipline.
• Not about drill sergeants or enforcers.
• Not about being able to code in low level languages like C or assembler.
• Not about being anal-retentive.
• While I have great respect for Watt's Humphrey, it's not about recording and reporting every minute detail of your work day.
  

Disciplined software development is about -

• self-discipline
• focus
• attitude
• approach
• organization
• responsibility and accountability

This includes the having the discipline to -

• write the unit tests
• add the Java Doc comments
• find the best name for a variable or method
• leave the code better than you found it
• fix problems not of your making
• improve your skills
• learn new things
• keep the coding standards
• use source control
  
• check-in changes frequently
• run the units often
• run the unit tests before every check-in
• write a test for a bug before you fix it
  
• get your code reviewed
  
• fix the problem not just treat the symptom
• keep code consistent

As a technical lead I struggle with this. I have no desire to be the cop, but there can be too much bad code to ignore. One helpful option is automation. Use analysis tools to identify coding problems, violations of coding standards, and generate test coverage metrics. Automate your builds and do a nightly integration build or preferably a continuous integration build after every change. Run the analysis as part of every integration build. Fail the build if there are to many violations and notify the development team of every failure.

I think that ultimately it is up the individual developer to have pride in his or her work. I think the Pragmatic Programmers said it best. A developer must "Care About Your Craft" and "Think! About Your Work".

What is software design?

Michael Feathers has a recent blog "It's All Design" where he says:

When I think about what we do in software development, I find it hard to imagine similar things happening in other fields.

He goes on to speculate that an auto designer would never be given a requirement such as there must be fifteen drink holders. I think he's wrong and that we operate in the same way as in many other fields. Come on, there really are vehicles with fifteen cup holders and I can't imagine any auto designer doing that without such a requirement.

In the comments Michael says "I try to imagine what happens in conversations with architects.". Well my wife has a degree in Architecture, but worked in the field only briefly before moving into software development. She has found great parallels between the work in both fields. We are currently working with an architect to design an addition to our house and one of the first things our architect did was ask about any features and requirements we had in mind.

Don't get me wrong, I agree that software development is just one big design process. I really hate how the software development process is so often compared to a manufacturing process. In software the manufacturing is not the development of an application it is copying that application to a disk. What we do, even the lowest level coding, is design not manufacturing.

But I don't agree with Michael Feathers that requirements and design are the same thing:

Maybe requirements is just a word that we use because we're dividing our design work between two groups.. a group that determines the higher level design (what the product will do); and the lower level design (how the product will do it).

Requirements frame design but are the same as design. Something like a performance requirement does not say anything about the design of a product. Software development is a design process and requirements are part of the process, but a stating a requirement is not the same as making a design decision.

"Mocks Aren't Stubs" Revisited

Reading the book "xUnit Test Patterns: Refactoring Test Code" made me take another look at Martin Fowler's 2004 article "Mocks Aren't Stubs". I'm glad I did because the article has been significantly updated (it's practically a rewrite and nearly twice as long) to reflect new thinking about mock objects. The first thing I noticed was the new terminology that is consistent with the "xUnit Test Patterns" book. The second thing I noticed is the new ideas about the different styles of using mock objects. The updated article is definitely worth a read or a re-read.

It looks like we are beginning to develop a shared language around unit testing, like what happened with refactoring and design patterns. It's interesting to watch this kind of thing as it matures and we learn more. It reminds we of how much object oriented development has changed over the last twenty years.

Java Mock Object Frameworks Reviewed

I've been reading the book "xUnit Test Patterns: Refactoring Test Code" so you are going to get a few posts on unit testing. The book is huge, 27 chapters and 944 pages, and packed with useful information. This clearly was no hastily compiled book, the author has invested a lot time and effort. The book has a website here. Right now I'm reading the chapter on "Test Doubles", what you and I would probably call mock objects, but the author classifies into five types: Dummy Object, Test Stub, Test Spy, Mock Object, and Fake Object. The classification is sensible and really makes you think about how you use mocks and stubs. I've been using mock objects for years and never really thought that much about it.

Despite using mock objects for years, I haven't kept up with the the mock objects frameworks. Until recently I've been using the original static mockobjects.com libraries and had not tried any of the dynamic mock frameworks like DynaMock, EasyMock, or jMock. I've finally gotten tried of writing custom mock objects and decided ti was time to try something new. Over the last couple weeks I've been testing EasyMock, jMock, and rMock. So here are my thoughts on the those frameworks.

EasyMock 1 (Java 1.3 or 1.4)

EasyMock uses recording to set expectations. A mock instance is created and the expected method calls are specified by method calls with the expected parameters.

• Documentation is decent and better than other v1 kits.
  
• Includes a tutorial with source code.
• Generally seems simpler than the others to understand.
• Some extra code, instances of the control and mock instance for each mock.
• Need calls to the replay and verify methods for each mock used in a test.
• Recording expectations using actual method calls is an easy to understand metaphor.
• Specifying return values is not so easy to understand. I thought it clashed with the recording model. The inconsistencies with the model of recording of expectations makes things harder.
  
• Had to record method calls, using null values, even when I did not care about what parameters were used to invoke a method. It seems like this would make it harder for someone else to understand the intent of the test.
  
• Less proxy casting than jMock, but two variables are needed for each mock. Need the mock and a a reference to the interface that the mock implements.
• Tests extend standard JUnit TestCase.
  

jMock 1 (Java 1.3 or 1.4)

jMock uses expectation specification and essentially implements it's own little language for setting expectations on a mock.

• Documentation is ok, but could be better. There are lots of classes so it can be hard to figure out where to look for something in the Java Doc. The tutorial is not included in the download. There is no full example with source code.
• jMock Usage is very consistent across all aspects of setting expectations and return values.
• Expectation definition at first seems verbose, but needs fewer lines of code than EasyMock. Expectations and return values are specified together which can make them easier to read and understand.
  
• Method names are specified as strings and may hamper refactoring.
  
• Casting proxies is annoying.
• Tests must extend MockObjectTestCase.
  

rMock 2 (Java 1.3 and up)

rMock follows the same model of recording expectations as EasyMock.

• Quite a bit of documentation, but it was not as useful. Had to generate my own Java Doc. No source code examples are included.
  
• This framework seems to want to totally redefine how unit tests are written. It implements a whole new assert framework which I found confusing, but it does work with JUnit.
• No advantage over EasyMock.
• No special support for JUnit 4 features or Java 5+.
• Tests must extend RMockTestCase.
  

EasyMock 2 (Java 5 and up)

This is a nice upgrade from version 1.

• Documentation the same quality as version 1. Includes a tutorial with source code.
• Much improved over version 1.
• Less code than version 1, no control objects, no proxy casting.
• Does require static imports for the cleanest looking code. More static imports than jMock.
  
• Still requires calls to replay and verify methods for every mock in a test.
  
• Still relatively the simplest to understand.
• Same inconsistencies in the model between recording expectations and setting return values as the previous version.
• Tests extend standard JUnit TestCase.
  

jMock 2 (Java 5 and up)

This is a significant upgrade that I found much easier to use.

• Documentation is better. The tutorial is not included in the download. Still no full source code examples.
  
• Less code, no proxy casting.
• Does require static imports for the cleanest looking code.
• Model is more consistent and simpler.
• The syntax of expectation setup needs a little getting used to.
• All mock variables must be declared final.
• Seems to have the best features of record and play back without the inconsistencies.
• Tests no longer have to extend MockObjectTestCase. Extending MockObjectTestCase is probably still the easiest thing for JUnit 3.
  

While I prefered JMock, EasyMock is also an excellent framework. Either jMock or EasyMocl would be a good choice. I do not recommend rMock. The choice between EasyMock or jMock will come down to personal preference and perhaps the skills of your developers. I think the slightly steeper learning curve of jMock is worth it for the consistency of its model. I was pleasantly surprised at how much easier jMock 2 was to use over jMock 1.