Bug Bashes - how to make them less sucky

Bug Bashes are when a bunch of people (cross disciplines) test the product. I have found that they can be useful if done correctly. (note: i personaly don't like them, but when they are done correctly, i hate them less:) )



1. Product needs to be stable.

There should have been a couple iterations between the dev and tester that own the area.

Unit testing should have been completed.

The tester that owns the area should have a goal that no one will find any bugs because they already found it. If your tester feels this way about the product, then schedule the bug bash.

What happens if you have a bug bash too soon. It is similar if you ship a product too soon... a lot of noise. Alot of people getting frustrated because they are hitting basic bugs, duplicate bugs in the database. The area owner dev and test waste a ton of time trying to repro and filter the bugs. Majority of the bugs are not repro or dup's. You are lucky to get one good bug if your product was not really ready for the bash... when 50% of the bug bashers hit 1 basic bug, 90% of the bugs entered are all around that bug... sort of like an open gate way to hell.


2. Guided Bug Bashes
If you really want to get something out of the bug bash, you need to provide guidence. Otherwise, 90% of the people are going to spend 80% of the time all going through the exact same path (which is probably covered by automation anyways). At the end of an unguided bug bash, you end up with nothing! You don't know where people were testing, you don't know what they tried, you don't know anything... basically you wasted peoples time and got no results (or superficial results that the product is good to go).

Details you should provide to the testers who are going to participate in a bug bash.
a) Provide a goal - set the tone - example... "let stop ship" - 'be malicious' - 'break the build'
b) Provide Areas of Risk -
provide details of why an area might be risky
provide how to get to the risky area
provide various entry points to your feature set
list past bugs in the area - *** super important
recent fixes
c) Provide a list of Integration Points - how the feature integrates with the rest of the product
d) Bug Template - provide a template for entering bugs. Along with a bug template, you should provide a definition list example Toolpane, ToolPart - so the testers can describe the bug with the correct definition.

3) Instead of Bug Bashes
I typically don't like bug bashes - when i am participating, half of the time it is installing and configuing, and the second half is me just figuring out the feature set. I feel very unproductive and felt like the 4 hour crunch was more just a surface ramp up on the feature set.

Instead of Bug bashes - i prefer SWAT teams - highly specialized, individually pick persons to do focused testing in specific areas. There is no reason why everyone has to bug bash at the same time. Before I start talking more about alternatives to bug bashes, I will write a future post...

4) Prizes
Yup prizes. You are taking people away from their work to help you do your work. Prizes are great motivators! It can be as simple as winning a huge plastic penguin! I think people love to win, they will do anything... cheat, lie. This is the kind of mentallity you want the testers to have when bashing on your feature set. They are less likely to verify, and more likely to try and break the product... push the limits.