Redefining why we run Test Passes and Regress Bugs

Redefining why we run Test Passes and Regress Bugs

I feel like running a test pass and regressing bugs has sort of lost their real intended purpose or potential. It has become something for management, and less about making the product better.

1) Test Passes

Right now, people will mind numbingly run through test cases - following test scripts exactly. They don't question what they ran before, what they are currently running or what they are going to run... basically the tester is half asleep. (funny thing is they were probably half asleep when writing the test cases in the first place!!) :)

Here are ways to get the most out of running a test pass!

NEW GOAL - if the testers are to run a full test pass (does not matter if its area owner or a contractor) - make it a requirement that they must find X number of bugs per day when going through the test pass.

What will happen:

1. The test cases they are running, now become a baseline (guideline), the tester now must push the tests to a new level to find bugs.
2. Test Cases are a simple entry point/code paths to consider.
3. The tester will step back at different points and look at a group of test cases - holes will be found!
4. Running a test pass will take a bit longer, but if the tester found a problem area - let them flush out the issues then and there.
5. You will sleep better at night knowing your testers were testing vs verifying!
6. Testers will now challenge the test cases themselves.

2) Regressing Bugs

Regressing bugs - typically testers try to get through them as fast as possible. They want to get them out of the way.

NEW GOAL -for every 3 bugs the tester regresses they need to find 1 bug. (or some ratio).

What will happen:

1. Pushes the tester to not only try the specific bug that was fixed, but to branch out and fully exercises other code paths that could have been affected by the fix.
2. Fixing bugs became just a little less risky - since the tester is trying to Break vs Verify.
3. Regressing a bug is no longer something to get out of the way or 'off your plate'. Fixed bugs tells the tester 'look at me... new code that has never been tested' which typically translates to potential bug farms!
4. Testers hopefully will not wait to regress their bugs, but see it as an opportunity to find more bugs. This means faster turn around and quicker stabilization.

With the new goals above - I guess you can think of test passes and regressing bugs as a way to perform focused ad-hoc testing!! (wow... that's sooo deep!!)