What makes software so hard?

Great article in the August issue of The Rational Edge, "What makes software so hard?". I have been convinced for some time that software is a design and not a manufacturing process. The article clearly describes the difference in explaining where in each type of process lie -

"On the other hand, for software projects, the relationship between
architecture and design versus production is not only inverted, but in
fact approaches infinity: since the production or manufacturing costs
for software are almost nil, almost all cost for a software
project comes from the workflows dealing with the creative parts --
that is, the invention and design of software."

Another great quote is the author's comparison of software to writing a novel. The metaphor fits so well -

"In a sense, creating software is far more closely related to writing a novel than to any traditional engineering effort -- an author is free to construct whatever type of story, limited only by his imagination and whatever facts may be necessary to the narrative. Furthermore, the author must construct the architecture of the story in his head, and to make sure the design and implementation of the story is consistent with the overall architecture. The quality of the author's product is to a high degree based on whether he succeeds in keeping the details of the story consistent with the architecture."

It is the notion that software is like manufacuring that has really turned me against big process improvement efforts like CMMi that are pushed from the top down. Anyone in a corporate environment has likely lived through at least one such effort. They usually seem so to be attempts to create predictability that does not exist in software and turn developers into interchangeable pieces that can be shuffled around a Gantt chart.

The Mysteries of Debugging?

Why do so many programmers find debugging so hard? Sure there are exceptionally wicked bugs, but most of the time we make debugging harder than it needs to be. The only secret I know of is having the right attitude and using the right approach.

How Not to Debug

Trial and error

Just guess at what the problem is. Add lots of print statements to the code and hope one of them shows you what the problem is. Make changes until the problem goes away. You don't need to know the cause as long as the bug is fixed.

Blame it on the ...

That's impossible, certainly it can't be a mistake in my code. It must be the compiler, database, network, and so on and so on.

Don't Understand the Problem

Don't dig deep enough to understand the cause. Fix the first and most obvious thing you find. Then fix the next most obvious thing when the same bug shows up again later.

Gentlemen Start Your Debuggers

That's what the debugger is for so fire that sucker up and start stepping. We'll just step through every line of code until we find the bug. You've got nothing but time right?


The Right Approach to Debugging

First don't panic! Programming is about solving problems and a bug is just another problem to solve. Of course you must approach debugging in a logical and organized way. The first thing you need are some some clues. Using the clues you can develop a theory and tests to validate the theory. Once your theory is validated you can implement a fix. This is very similar to applying the scientific method :

1. Gather data
2. Form a hypothesis
3. Perform experiments that test the hypothesis
4. Prove or disprove the hypothesis
5. Rinse and repeat as needed

Techniques for Successful Debugging

Repeatability

You need to reliably reproduce the bug. If you can't reproduce it when needed you can't test it or know when it is fixed. Reproducing a bug can be the hardest part of debugging.

Find the simplest test case that demonstrates the bug. You want to make it quick and easy because you will need to recreate the bug many times. The harder the bug is to recreate the less sure you will be of the cause and your solution. It is often worth the effort to create the smallest simplest program with the least code and clutter that shows the bug.

Analyze All the Available Data

Before rushing into a theory about the cause of a bug you need to make sure you have completely analyzed all the data that you have about it. Don't jump to conclusions because your first instinct will often be wrong. Look at the problem from as many directions as possible first.

Make sure you understand what the data is saying about the problem. We've got a great new technology called an exception that holds enough information all by itself to tell you the exact problem and the line of code where it occurred. Take the time to read and understand the full exception output. Time and gain I find myself pointing out to a programmer that the exception is telling them exactly what the problem is and all they need to do is read the exception output.

Turn on as much application logging as possible and take the time to thoroughly examine the log or trace files. There are so many freely available open source, high quality, easy to use, logging and tracing frameworks available for every platform that there is no excuse for any application not to generate high quality error and debug logs.

Narrow Things Down

Use a binary search or divide and conquer technique to zero in on the problem code. You need an organized hunting expedition not a haphazard ramble through the code to find bugs.

Look at what has changed recently. If things worked fine last week then figure out what has changed in the code or its runtime environment and look there first.

Explain the Bug to Somebody Else

When you aren't making any progress, stop, take a breath and find someone else to talk the problem over with. So often the simple act of explaining a problem generates an insight before you are even finished with the explanation. If just explaining things doesn't work then the other person may have a great idea of their own.

Fix the Real Problem

The symptom you see may not be the actual bug. You need to find and fix the root cause not the symptom. Be sure you are fixing the problem and not just treating a symptom. When you find the problem look around for any similar problems. We all tend to make the same mistake more than once.

Write a Test Before You Fix

First, The test will be a good demonstration of the bug and when the test succeeds it can be proof that the bug is fixed. Second, you just spent valuable time finding and fixing the bug and a test will help ensure that it does not come back to waste your time again.

The Compiler is Not Broken

The compiler, database, wahtever is not broken. There could be bug there, but don't start from that assumption it will just waste time. Believe me it is not a bug in the compiler, compiler writers are way smarter than you or me.

One Change at a Time

Never make more than one change before testing the bug again. If you make two changes how will you know which one fixed the bug and if which change is actually needed.

Check the Simplest Thing First

Bugs are often caused by some silly mistake or oversight and the simple things are easy to check and fix. The unlikely things are hard to check, so save them for later.

Use the Debugger

I saved this one almost for last because it should be a last resort. Debuggers are wonderful and powerful tools! But debuggers can also be tedious, time consuming, and confusing. Sometimes they are the only way to figure a problem out, but to use a debugger effectively you first need to narrow down the code that needs to be checked. Don't let the debugger be the first tool you reach for.

Use Tools to Find Bugs Before You Deploy

The best way to fix a bug is to never let it get deployed. It should not be necessary to remind any programmer to turn on as many compiler warnings as possible, but unfortunately, I know it is. When you have all the compiler warnings removed from your code, run a static analysis tool aver it too. There are many open source and commercial code analysis tools available so get at least one and use it to analysis ALL of your code for bugs.

References

"The Pragmatic Programmer: From Journeyman to Master" by Andrew Hunt, David Thomas
"Code Complete" by Steve McConnell
"The Practice of Programming" by Brian W. Kernighan, Rob Pike
Debugging strategy: easy stuff first
Fix The Bug, Not The Symptom

New Application of the Builder Pattern

At "Mistaeks I Hav Made" Nat Pryce writes about an alternative to the Object Mother pattern that is based on the Builder pattern I blogged about previously. It looks like a useful application of the Builder pattern. Object Mother is a technique for creating test data for unit tests. I've used Object Mothers and the article is right that overtime they get messy and bloated.

In he last line of the article Nat says -

"In some cases, Builders have so improved the code that they ended up being used in the production code as well."

Which is a nice validation of underlying the Builder pattern. I wonder if they used the same builder, complete with the default values, in the production code or wrote new ones.

Developer Testing is Habit Forming (and Habit Changing)

A while back Tim Ottinger wrote a great article about how "Testing Will Challenge Your Conventions". Among the points that really hit home for me were -

1. "Interfaces suddenly seem like a really good idea...". Mock objects can be so useful when used correctly that creating an interface is often the first thing I do.
   
2. "Singletons and static methods no longer seem like a great way to do work...". As I hope everyone knows by now the Singleton pattern is way overrated and makes writing truly isolated unit tests very hard.
3. "Private makes less sense...". I still struggle with making a method public just for testing but sometimes there is no other way. Whenever I find the need to do so, I give my design a hard look to make sure it is as good as it should be.
4. "You need to be able to pass a class everything it might need at construction time...". To write isolated unit tests your classes cannot configure themselves. Of course long constructor parameter lists can be a problem but I can use the builder pattern as one way to counter that.
5. "Smaller methods are the norm." When I think about the size of methods I write now compared to a few years ago I am amazed at how small the methods are now. It is getting to where I can't read or don't have the patience to read methods over 10 to 15 lines long.

That's a few of my thoughts go read the whole things for yourself.

Discpline and Software Development

Jeff Atwood writes over at Coding Horror that "Discipline Makes Strong Developers". Discipline is important and the best developers are certainly disciplined. But what sort of discipline are we actually talking about? Let's start with what sort of discipline we are not interested in -

• Not imposed discipline.
• Not about drill sergeants or enforcers.
• Not about being able to code in low level languages like C or assembler.
• Not about being anal-retentive.
• While I have great respect for Watt's Humphrey, it's not about recording and reporting every minute detail of your work day.
  

Disciplined software development is about -

• self-discipline
• focus
• attitude
• approach
• organization
• responsibility and accountability

This includes the having the discipline to -

• write the unit tests
• add the Java Doc comments
• find the best name for a variable or method
• leave the code better than you found it
• fix problems not of your making
• improve your skills
• learn new things
• keep the coding standards
• use source control
  
• check-in changes frequently
• run the units often
• run the unit tests before every check-in
• write a test for a bug before you fix it
  
• get your code reviewed
  
• fix the problem not just treat the symptom
• keep code consistent

As a technical lead I struggle with this. I have no desire to be the cop, but there can be too much bad code to ignore. One helpful option is automation. Use analysis tools to identify coding problems, violations of coding standards, and generate test coverage metrics. Automate your builds and do a nightly integration build or preferably a continuous integration build after every change. Run the analysis as part of every integration build. Fail the build if there are to many violations and notify the development team of every failure.

I think that ultimately it is up the individual developer to have pride in his or her work. I think the Pragmatic Programmers said it best. A developer must "Care About Your Craft" and "Think! About Your Work".