Technical Debt is a Process Smell

John Boal — Wed, 14 May 2008 14:13:49 GMT

Definition: Technical Debt is a term that we in the Agile world use to describe things in the code that perhaps aren't engineered as well as they could (should) be. It can refer to many things, like areas in the code that we should probably refactor to make them more readable, poorly performing code, obsolete or clunky architecture, encapsulation abstraction, or other class problems, or the like. It's debt because we know about it, we "owe" it to be done, but are focusing on other things instead.

Technical debt is the avoidance of doing work that we would otherwise do in the course of doing our best practices in engineering. It's a fact of life that most every Agile team has to deal with, in my experience. I find that there are various means of "dealing with" technical debt, some of which I will discuss below as a matter of practicality. However, I see it really as a symptom of a deeper problem. The team isn't finding good engineering as valuable as completing stories could be the underlying problem.

Teams can be pressured to deliver more than they should (this is common), and as such, could be focused on delivering the current sprint's stories rather than fixing things they find that are not as they should be. The team also could be creating more technical debt for itself by not focusing on good engineering in the first place. When junior people design things, sometimes they don't address all the points of performance and code maintainability. Trade-offs are made that can end up causing lots of rework later.

The concept of Agile software development, is heavily based on refactoring. Sometimes I find that teams use refactoring with a little "r" when they should be Refactoring. They make small internal changes, but are reluctant to change overall design, even when it is truly warranted. When we add a new feature, sometimes the design needs to change to accommodate it. THIS IS EXPECTED behavior in an Agile development process. Yet, time after time I hear developers say "well, I don't want to change all that NOW..." when that's really exactly what they should be doing. The concept of iterative development should be allowing us to ship software at the end of every cycle, and we really can't consider the software shippable if it has these poor designs still left over from earlier days.

Refactoring the design shouldn't be catastrophic though. But, there is always some impact that is almost always fundamentally left out when planning. Imagine we have an apple, and we need to add some cherries to it in this sprint. We might just smash the cherries on the outside of it, and call it good. But that would leave us a lumpy ugly mess... Rather, we might take another approach, and put the apple and the cherries all in a blender, and pour the mix into a new shape mold and freeze it. The new shape will be smooth and round, yet it will encompass the new features of having both apple and cherry flavors. It's a dumb example, but you get the idea. Refactoring can take many forms.

Handling Technical Debt
Now, I have been on a few Agile teams and I am a realist. Technical debt is most certainly here to stay. We just aren't in an ideal world, and we can't always take the time we really need to make sure that the iteration's deliverable is as pristine as it should be. So, now how can we handle this debt in a practical way?

Here are some ways I have seen teams deal with Technical Debt:

Ignore it.
1. This don't work so good, in my opinion, but yes I have actually seen it done. It works, so we'll use it as-is.
TODO it.
1. //TODO in the code is a flag that there is something more that needs attention. Also, //FIXFIX and //BUGBUG are others I have seen. Some of the tools give us mechanisms to flag these as work items, but in my practical experience this is just about the same as #1. Nobody ever gets back to these things to fix them. They just lose visibility and get buried. Not very effective in my book.
Backlog it.
1. Create a backlog item called Technical Debt. Each instance where something needs attention is recorded as a task for this story. They can be estimated, and prioritized. Backlog items have visibility, so they don't get lost. Here is the problem - Technical debt now CAN be prioritized. And, it usually is - right to the bottom of the list. This really doesn't solve the issue either, unless you have a product owner who is savvy enough to know that good engineering means happier customers, developer, and management in the long run.
Bug it.
1. Log a bug for each Technical Debt item discovered. This is probably the most effective means I have yet seen for the actual reduction or elimination of technical debt. In reality, I think this is the most appropriate. The new story requires changes that just haven't been made. It's not really "done" until this is taken care of. Ongoing maintenance of code should have an appropriate stakeholder with a voice in the Product Owner's scope of attention.
Job-Jar it.
1. Put it into the bucket of things that need to be done but aren't on either the sprint or product backlogs. I think this is marginally effective, because nobody should technically ever have time to get to this stuff, they really should be pulling things off the product backlog onto the sprint backlog. But, I thought I'd mention this concept since someone brought it up.

Are there other ways to deal with Technical Debt? What ways have you had success? Please leave me a comment and let me know.

Bugs on an Agile Team

John Boal — Wed, 07 May 2008 02:02:00 GMT

How do you handle bugs? There are many ways organizations handle bugs in software. I have what I think is a very workable approach, see if you agree.

The environment: The team is a dozen software developers and testers (total) and the project is run as a Scrum project. Iterations are 3 weeks.

Scenario 1:
QA is testing the build, and there is a bug discovered. The date/time field data that's entered in local time is stored in UTC, but the data comes back as UTC (not local) when retrieved.

QA logs a bug. Priority 2 Severity 2
Bug is assigned to the Product Owner for the team.
PO looks at the bug, and makes the call that the currently signed-off stories are more important than this bug.
The bug is estimated in story points by the team as 2 points.
The bug is put on the product backlog, prioritized along with the other bugs and stories.
The team can't "ship" the sprint's release with this bug outstanding, but the PO knows that they aren't planning a release for another two sprints anyway.

Scenario 2:
QA is testing the build and discovers that if a name is entered that is longer than 50 characters, the site crashes with an unhandled exception.

QA logs a bug, Priority 1 Severity 1
Bug is assigned to the Product Owner for the team.
PO knows that this bug is serious enough to warrant being fixed right away.
PO meets with the team, and the bug is estimated at 5 story points.
The team's velocity last sprint was 80, and they have committed to 78 points for this sprint. The team left only 2 points "spare" and didn't allocate time for bug fixes.
PO suggests that the bug be fixed this sprint, but the team does not feel they will have time enough to do that along with the other stories.
PO removes the lowest priority story from the sprint backlog, and replaces it with the bug instead.
The story goes back on the product backlog as the top story in priority.

There are also other permutations that may work in different organizations:

Abort the sprint, and replan a new sprint including the bug fix.
Allocate some time and time-box it just like a spike. The team may not be interested in working overtime, but try to negotiate it anyway. Most teams will want to fix bugs like this even if it means they work a few extra hours. Buy pizza.

The current team I am on doesn't follow these guidelines, and they insist that every single bug be fixed for every sprint, or the story can't be accepted as done. While I think this is a lofty goal, in my experience it just isn't practical. Maybe it's my old school days at Microsoft where "done" meant it was "code complete" but not "bug free." I think that Scrum gives us the ability to continue the current sprint and continue to develop the features as committed, and delay bug fixes until the next sprint. The only time when this doesn't work is the case where this sprint's story delivery is scheduled to be the release. And even in that case, most organizations allow a small window for last-minute fixes which could be used to take care of the bug.

If the developers are doing TDD, they won't be writing bugs in the first place... I see more than a handful of bugs per sprint at a clear process smell. In that case, the process should be evaluated to see why the bugs are being created in the first place. Is it unclear story descriptions? or more likely, is it a lack of testing in the functional, integration, and acceptance areas? Any way it goes, bugs seem to be a symptom, not the actual problem. Every bug I have written myself was due to a lack of a test for that area. Find a bug, write a test! Please make sure that your developers realize that when they estimate fixing bugs, that it should always include writing tests that first fail when the bug is present, then pass when the bug is corrected. Bugs should not be able to be resolved without at least a unit test to illustrate that they are really fixed.

Handling bugs can complicate a project manager's life, so if they are cropping up, be sure to address the root cause, and all will be happier.

Bits 'n Widgets - bugs

Technical Debt is a Process Smell

Bugs on an Agile Team