Apparently TFS doesn't allows us to track the effort spent fixing bugs. I mean, we know how many bugs we have, their state, their criticality, when they were reported, when they were resolved and by who (among a lot of other things) but we don't have the answer to the most basic and important questions.
Simple questions like :
- how much effort took to fix this bug?
- How much effort have we spent fixing bugs?
- what is the ratio of our development time vs our bug fixing time?
From a development point of view, maybe this is isn't important (at least for argument sake :-)). But from a management point of view, this is very important (I would dare to say crucial). How can we determine how much bugs are costing us if we don't know the effort we spent at it? or more importantly if the ratio between bug fixing time and development time is high. Perhaps we have to launch an initiative to improve the quality of code to reduce the bugs.
At first sight TFS doesn't allow us to answer this question.
So how can we do it?
I've tried to get some guidance from the patterns & practices Team Development with TFS Guide
Even though the guide doesn't explicitily talks about this topic, there are some hints we can use to get a possible solution.
At a point it says:
«You access bugs and task work items from TFS to determine what work you need to do. For example, work items might have been assigned by your project manager, by another developer, or by the test team.»
So at first glance both bugs and task can represent units of work. This is further enforced by «Team members use work items to track their personal work queue and assign work to each other for example, in the form of bugs or tasks»
But then bugs and tasks are clearly separated.
«Each work item type can be defined as follows:
•It has a purpose and intended use. For instance, bugs are used to track quality defects, tasks are used to track scheduled work,»
So we can pretty much conclude the bugs are used to communicate potential problems while tasks represent units of work that need to be performed. This clearly separates the waters between the purpose of these two work item types.
From the quoted text, (I think) I can safely say, they are partly ambiguous and somewhat contradictory in terms. From this I've defined two possible solutions.
The one I've called the simple & pragmatic approach and the purist approach. :-)
The simple approach is quite light, it doesn't impose any additional workload on the development process, it is quite clean, and easier to track the effort spent on bug fixing. Alas it isn't perfect and it fails to handle more complex scenarios.
Track how much effort is still necessary to fix the bug and how much effort has already been spent on the bug. If this sounds familiar, it's because it's the exact same mechanism that MSF (both Agile and CMMI) process templates use.
This is a simple solution, that has many advantages.
- Quite easy to track the remaining effort and the effort already spent. Its a simple matter of summing the values on the datawarehouse.
- A mechanism that is familiar to developers. Just use the same procedures they use for tasks
- A clean separation between tasks and bugs.
However it has some disadvantages. Notably on "big" scenarios. For example, suppose that you have a bug that spans multiple person/modules/teams/.... Something that requires the intervention of multiple persons to fix the bug, or to complicate matters a little suppose the act of fixing the bug, isn't just correcting it. The fix must pass through a quality assurance team, that documentation has to be updated,etc,etc.
In simple terms, I guess the correct thing to say would be a bug can trigger several tasks,
- The developer fixes it
- The developer/tester writes a regression test to make sure that the bug doesn't reappear
- The Testers test the fix
- The documentation team updates the documentation
- ....
The purist approach would be:
The bug is just a container for information, all tasks that are related to the bug in question should be created independently. A relation to the bug should be established through a link.
This is nice not only when you need to track effort by bug, but also by person.
This solution is much more complete. However completeness brings the complexity cost. In terms of development costs, they are a (little) higher. More tasks to perform, a more complex workflow.
It also brings some problems:
- When code is checked in, which work item should be associated with the checkin? The task or the bug (or both?). But this is a minor issue that can be easily solved by conventioning a rule.
- In terms of reporting and data extraction from the datawarehouse the things are much more difficult then the other solution.
So we have two options. A more complete approach that comes with a cost, or a much simpler approach that has some drawbacks.
The simpler solution only requires some minor changes to the process template. The second one, requires some reporting to get the data.
Internally we have decided on the simple & pragmatic approach. So far it is working quite nicely, we have opted for this solution. That it is optimal with the cost that brings. The only drawback is the fact then a bug must be address by several people. It is impossible to track the effort spent by each person.
This post was going to be titled Tracking bug fixing costs in Team Foundation server but Martin suggested this more enticing and sexy title.