Look at the Tester - Non-Obvious ways to find bugs

As a tester, I was always trying to figure out what area i should hit next. Where are the holes, the problem areas, the places that have bugs. One of the most efficient way is to watch the testers, figure them out, understand how they test, what their background is, etc.

Here is a list of testers I have seen over the years that you should look for. They all have a high potential for shipping low quality products if no one is watching them.

Note: the labels i gave them are my own and just for fun.

1) The Complainers

Listen to the hall way conversations. What areas or developers are the testers complaining about? If testers are complaining about a specific dev or area. Any kind of complaining demonstrates a problem with the dev test relationship. Sometimes you won't hear verbal complaints, instead you will notice that you never see the developer and tester talk.

Another reason testers might be complaining is because they are overloaded. They can't get to everything at once and no one is stepping into help them out. This causes the tester to do only surface testing, and deep in depth testing is not done just because of time constraints.

Types Of Bugs:

If the problem is dev/test relationship then there is always huge holes in areas that these two own and they will probably be very very very basic bugs.

If the problem is the tester is overloaded. Work with the tester, and say any testing you do is unofficial and under the table - but ask them what areas they are worried about etc. Any type of testing is bonus in their eye and it will alleviate some of the stress.

Reduce Risk:

For dev/test relationship: From a management point of view, you should definitely separate these two and assign this area to a new tester.

For the overloaded: move areas around. Bring other testers in. NOTE: i have randomly heard 'mythical man month' as an excuse not to do anything... what a bull. Get the area owner what they need or give them the option of forming a SWAT team - where they drive and pick out individuals to help and test specific areas that need to be covered.

2)The Underachievers

Look for the testers, where skills/experience do not match what they are producing. There can be a thousand reasons for this... but in the end, results are usually the same. If you see a tester that is not producing at the level they are capable of... that means they have checked out. (Time to move in on their area. :) ) Motivation and interest is gone, and they are probably just doing the bare minimum to stay afloat. You will know if they checked out by a simple test... Enter bugs in their area, if they don't react, they've checked out. (same test you would do if you are trying to see if an animal is dead, poke it with a stick and see if it moves :) ).

Types of Bugs:

All over the place. You never know when they lost focus. Look for recently fixed bugs, since bug regression is one of the first places that they will skimp on.

Management:

Got to move these guys to something new. To a new team, new product, fresh start. You might want to hold them tight and close to you because at one point they might have been awesome. But for some reason, the passion is dead, and you will never see it again (for you specifically)... cut the cord and let/help them move on.

4)Repeat offenders

If a tester dropped the ball on a feature they shipped before, they are going to do it again. Since they did not get fired the first time... why try harder? You will always find bugs in their area. Why? Because they are not getting it and don't care enough to try to get it. Typically these are senior testers. Funny thing is with these guys, you would never know by meeting them that they suck, because they don't think they did anything wrong. Sometimes these guys are big talkers with low bug counts or they are the guys that are just floating by.

Type of Bugs:

Everything under the sun. I have found that just following the specification design document found tons of problems. Other times I have found problems where the feature integrated with another product. You can always ask this tester what they hate the most, and you will probably get your answer of where to start looking for holes.

5)New Hires
I love new hires... motivation is high, work ethics high, they are young and energetic... only problem is they are new. When you are testing a new hires area, its best to actually teach them different techniques and let them run with it. I feel that its very important to give new hires the right feature set - set them up to learn what it takes to be a top tester. Features you should give a new hire, 1) is where alot of other features overlap it and 2) give them a high bug count feature. Reason to give overlapping/high traffic features, means other testers will uncover bugs that the new hire missed (reduce risk). And the reason to give the new hire a feature that typically generates a high bug count (like UI) is... it is easier to go deeper into the product if they have a UI entry point, so they can go top-down (I rarely ever see people go from bottom up (database level to UI)).

Types of Bugs:

Leave their areas alone. Better to take the opportunity to mentor them, show them what they missed etc. Don't take away the opportunity for them to practise finding bugs in their area.

5)Hurts to Think

When creating a test plan, it requires alot of analysis and thinking about their features and how it will integrate with the rest of the product. This requires thinking. So look at the test plan (if they are required). Definition of 'test plan' - is a document outlining how the tester is going to approach testing the area. Typically the testers that put no effort or thought into the test plan... will do the same when testing their area. It's the first sign there is going to be trouble.

Types of Bugs:

The type of problems you will see out of their area, are major redesigns later in the cycle. Huge integration misses.

6) The Verifiers

Verifying sucks. Its the worst thing ever. All they tried was very basic unit tests, which the developer already did. Basically its open season. I bet any test case that you try that even remotely pushed the limits will break. Instead of testing their area, its best to convert them to becoming testers. They can be converted over, it just might be that no one has mentored them, or showed them how to be successful. (This is all assuming that they want to be in testing, and have motivation to become a great tester)!

Type of Bugs:

Only thing that was tested was basic unit tests. Since there is so much testing left its best to mentor these testers and teach them how to be super stars if the motivation is there. You will see by bug count if they truly want to learn. And if they are all talk and no action then its time to start testing their area.

7)The Push Overs

The push overs are the testers that can't stand up for a bug. They find a serious issue, but get over ruled by their Program Management or Developer. Instead of pushing for the bug, they let it go. All three are at fault in making the wrong decision about a bug, but sometimes the tester claims innocent because in their world their responsibility ends at 'logging the bug'.

Types of Bugs:
The types of bugs you will find that make it to the final product are usability issues. Usability is a common type of bug where testers opinion is void and null because of the common saying 'your a tester, you are not a real world customer'.

Management:

It is best to pair up a push over with a senior program manager or senior developer. It usually the younger pm's that make the bad calls or the young developer that does not want to do the work.

8) The Keeper/Old Timer

The keeper is someone that has owned an area for several releases. People always think, this is good because they are the area expert, but its not!!! Totally opposite, its the worst thing you can do and you are in for trouble. They have become desensitized, no longer question anything. They no longer go through an area with a fine tooth comb. They skim over things, they test for 'it generally works'. Its the same old. Passions starts to ween, attention/focus starts to wander to other things.

It is better for the team, if this person takes a new area on and someone else takes over their old area. Why? 1) The keeper will have a hard time letting go, which means you will still see bugs coming in by the old area owner even though he has a new area (so you get two sets of eyes) 2)Expertise is distributed to more then one person on the team 3) You have a fresh set of eye looking at the feature 4) The keeper is less likely to get bored and leave your team.

Types of Bugs:

Regressions will not be found. Basic scenarios could be broken since the tester assumes they are working.