Quick Testing Tips Your daily feed of short software testing tips…

When you can, group bugs for investigation

While I wouldn't normally consider this a "quick testing tip," on some teams testers also help with initial defect investigation to help identify root cause. If that's the case, and you're one of the testers who works with a production support team to isolate and more clearly define (and fix) production bugs, then this tip is for you.

Before you pull a bug off the top of the stack for investigation, do a quick search to make sure there aren't similar issues in the backlog that should be investigated at the same time. I've found that if you're looking for three similar issues when trying to recreate a similar set of problems, that you can make headway (on some of them) faster than others. It's also possible that simply by looking at how the bugs are clustering, you might gain better insight into what you'll need to do to recreate or isolate them.

Tips for reviewing defects

We spend a lot of time talking about how to write good bug reports and how to advocate for issues you feel are important. But what if you're on the other side? What if you're reviewing defects and/or running the meeting where you assign priority and severity? What if you make the decision for what gets fixed?

Here are some tips that can help you make sense of what you're looking at. Note, these tips are focused on the way the bug report is written. It's difficult to talk about hard and fast rules for prioritization without knowing the specifics of the the project, project team, and context. It's also not intended to be in any way a complete list. Just some of the things you might think about...

• Try to identify any viewpoints that might be embedded in the defect report. If there is a bias to the report, don't let that necessarily influence your decision on how sever the issue is. Ask for clarifications or more data if needed.
• Ask yourself how your stakeholders would view the defect report. Switch viewpoints. What would they care about if they were in the room?
• Look for inconsistencies between defect reports and ask for clarifications. Why might something work in one instance but not in another? What information is missing that can help you (and the team reviewing) make sense of those inconsistencies?
• Try to understand the implications of the ticket. Look past what's reported and see if anything occurs to you about the nature of the bug. What does that mean for the project team?
• Is there any key information omitted or glossed over? Is there summary data without a link to the detailed data? Try to find areas where more research might need to be done before conclusions are drawn.

Developing headlines for bugs that get fixed

One component of defect reporting is bug advocacy. That is, working to get the bugs you feel are important fixed. To be an effective bug advocate, you need a variety of skills. One of them is the ability to write effective ticket headlines. To construct an effective headline, you need to both know your audience (who are the people who review and prioritize these bugs) and you need to know what they want fixed (what do they consider relevant, what do they care about, how do they picture themselves in the corporate world). It's this information that allows you to craft the appropriate headline or lead for a bug report.

When you're drafting your headline:

• be specific
• create an initial definition of the problem (or the possible problem) for the reader
• don't draw conclusions
• check your spelling and grammar

Logging a politically difficult bug

On some projects, the word defect is another word for politics. If you've never been on one of those projects, you're lucky. But they exist. If you find yourself on one of those teams, or even if you're not on a politically charged team but still have to log a difficult issue, you might consider the following tips:

• Restrict your claims about the defect to those that can be supported by the data you have. Don't do too much editorializing in the ticket. Just list out what you know, what you don't know, and let the team figure out what to do from there.
• Be sure that before you log the ticket you spend some time searching for data that refutes the idea that the issues is in fact a defect. Try to prove yourself wrong. Really do some digging.
• Make sure that all the data you list is relevant to the issue you've logged. Don't make it easy for someone to ignore your issue because it's either not clear why some data is there or because it includes a subset of information that can be trivialized. People will choose to fight the weakest set of data you include, so make sure it's all strong and speaks for itself.
• Make sure you have enough data for the issue you've logged. For example, if you're reporting a performance issue, don't include just one test run. That's not enough. You'll need to establish a pattern of behavior for issues like that.

Attach the HTML

When you're testing a web application and you find an error, save the HTML. When you writeup the defect report, a lot of times it can be helfpul if you paste in the HTML for the page. If it's a lot of code, just save it off and attach it as a file. There's often a lot of informaiton found in the HTML that isn't rendered on the screen.

Reporting issues where you don’t know the root cause

When reporting issues where you don't know the root cause, be sure you include the following information (if applicable):

• screen shots
• log files (local and server if you can get them)
• machine stats (CPU usage, memory usage, disk, etc...)
• what was happening:
  • What were you doing (steps, activities, etc...)?
  • What else might have been going on at the same time (volume, batch jobs, etc...)?
  • What else was doing on right before the error occurred?
• copies of the configuration files at the time, or a summary of key configuration settings

Intermittent bugs

At CAST 2008, during a conversation on intermittent bugs, Jeff Fry provided the following tips to help isolate what's going on:

• Try varying your inputs
• Switch browsers
• Very concurrency and load
• Change your speed of execution
• Change your perspective (six thinking hats)
• Change the platform you're running on
• Try using a recording tool or additional monitoring tools