In over 10 years of developer experience, I worked for different companies, in different roles, but I always found the same problem over and over again: bug reporting sucks. I spent some time thinking about how to avoid some usual problems in this area and I realize that we could apply the same technique we apply to improve the code: reviews.
The Problem
Bugs descriptions are created by different kind of people: developers, QAs, support, sales, etc. Depending on the role, the bug description will be readable and clear or vague and cryptic. Vague and cryptic descriptions are not useful as is impossible to determine what is the problem being descriptive.
The problem with this kind of description is the same we have when we write ugly and poor documented code: the times goes on and nobody understand what was the meaning of the original written intention. When nobody understand what the issue is about, the issue is left open for long periods of time, wasting the time of the team reviewing the issue again and again, until someone decides to fix the problem closing the issue as “not reproducible” or “incomplete”. That removes the bug from the list of pending issues but the error is still there waiting to be discovered again.
If writing bug descriptions has the same problem we found in the code, why not to use the same technique we use to improve it? I’m talking here about code reviews, a process intended to improve the quality of the code. The thing here is to do bug reviews. That means: review all bug descriptions right after they are created or updated.
The idea behind the bug review is to try to catch problems in the bug descriptions in order to resolved them while the bug is still fresh in the mind of the people who filled it. Doing this let us clarify the problem and improves the chances to fix the bug. When a problem is detected in the bug description a comment should be added to the issue as soon as possible.
The benefits of the bug review are:
- Improves the bug descriptions through a feedback loop with the bug filler.
- Reduce the time the team waste looking at an issue trying to understand what is the described problem.
- Detect “no issue/won’t fix” or duplicate issues more quickly providing quick feedback to the bug filler
- Detect quick fixes more quickly. Fixing this type of issues is a good way to add value to the user community
Bug reviews in practice:
As in a code review, there are different aspect of the bug review that are important, some are easier to review than others, some require more skills than others
Readability:
Issues should be understandable by any person related to the project. Use concepts that are related to the domain of the project. Don’t assume that people understand all your expressions. Express clearly how to reproduce the issue step by step. That means: step by step. Depending on the context a simple phrase like “deploy the application to JBoss” cannot be understand by all the people working in the project. Maybe could be hard to list all the sequences of steps required to do that task, but why not to add that information in a separate place to issues could just link to that document? Clarifying concepts in bugs descriptions is as useful as clarifying concepts in user documentation.
Formatting standards:
Follow some rules to fill the different bug fields. Is there any schema for bug’s names? How to assign user impact? Should include test cases to reproduce the issue? How to measure user impact? In which module should be the bug assigned to?
Low coupling:
All the information needed to reproduce the bug should be available in the bug description. References to related bugs could be useful. Avoid references to external systems (DB, JMS servers, etc) that could change from the time the issue is created until the time the issue is fixed. There are exceptions when the setup process to reproduce an issue from scratch is overwhelming. The idea is that an issue should be reproduced any time without worrying about external resources.
High cohesion:
All the information reported in the bug should be related to only one issue. Is a common practice to describe more than one problem in the same bug description. This usually happens when the sequence of steps to reproduce them is almost the same and affect the same part of the system. Is easier to fill one bug describing multiple problems, but there are problems. First it makes the people working on it to think about more of one thing, people need to focus on a given problem. Also it can make the progress more slower in situations when one of the partial issues can not be easily resolved, so the complete issue cannot be closed.
Performance:
Bug description should be filled in such a way that lets the people working on it to understand and reproduce the problem as fast as possible. Usually the time used to fill a bug is much less than the time needed to discover and fix it. So, why not to use more time during the bug filling step? The people how discovered the issue has already spent some time trying to discover the sequence of steps that procedure the issue. Why not to spent more time clarifyng the causes of the problem? This could require to write automated test to reproduce the issue wrapped inside a maven project or investigate an issue using WebSphere to check if it is also reproducible using ActiveMQ (easier to set up in a developer box)
Conclusion:
The human mind works in funny ways sometimes, so is not guaranteed that we get correct bug descriptions when we use reviews, but we expect to have a less number of problems when we follow this practice.
The hard part of the bug reviews is how to introduce the concept/habit in the development process. Some reviews need the expertise of old team members, while some others could be done by any team member . The important thing here is that review process is a TEAM activity.
How much time to use on the bug reviews? How deep should be the review? Well, that part of the process is not so clear to me at the moment. What is clear to me is that this process helps to reduce the time the team waste wondering what a bug is about.
