Mark McSweeny and I were talking about some of the challenges facing conventional testers on agile projects. One such challenge is what to do with bugs found during development, such as during Test-Driven Development when a tester and developer are pairing. It doesn’t seem fair to the developer to formally log bugs on a story before they have completed development on it. Many of them will be moot once the story is done, but some of them might squeak through. How do we keep track of them in a constructive way?
When they are pairing and developing tests, new test cases are added as they are generated, and the code is added to make them pass. However, sometimes when bugs are found during story development, the tester can overwhelm the developer and impede development progress. At this point, the developer and tester pair can split up, and the develoepr can pair with another developer and work towards completing the story. As a tester, what do I do with the bugs we discovered but couldn’t get the unit tests finished for?
On small teams, I will keep a running tab in my notes of these bugs. When the story is complete, I check my notes and test these scenarios first, and then log the ones that weren’t fixed during the story as “bug stories”. This is fine if there are a small number of developers, and I’m the only tester. This doesn’t scale well though. On slightly larger teams, I have also used a wiki to record these bugs which the other testers also reviewed and used. When they tested a story when it was complete, they would check the wiki first for any of these bugs. Any of these that weren’t addressed in development were then logged as bug stories or in a fault-tracking system. This can create classes of bugs which can create problems. It was hard to maintain the two systems, the wiki and the bug tracker.
As I was describing some of the problems I’ve come across with bugs found during story development, Mark cut through my usual verbosity with clarity, and said I was describing: “virtual bugs”. This is a lot more concise than my five minute hand waving explanation of this different class of bugs.
I have started calling the bugs found during story development “virtual bugs”. My question to other conventional testers on agile projects is: “How do you deal with virtual bugs”? Please share your experiences.