Quick Testing Tips Your daily feed of short software testing tips…

500 word minimum

To help them stay focused and productive, some writers give themselves daily minimums. In a recent issue of Writer's Digest, author Hank Schwaeble said he uses 500 words a day as his minimum. When most authors do this, they have little to no expectation that those are 500 "publishable" words. That is, they don't think they will be polished or won't need editing. I'd go so far to say, they might not even have an expectation to publish them at all. It's more about just making sure you log time doing the most basic unit of work - writing - so not all your time gets taken up by editing, revising, contacting editors, or other administrative work. Different writers do it for different reasons.

There are some other examples of how people in our industry use quotas like this. Jerry Weinberg’s Rule of Three is one: "If I can’t think of at least three different interpretations of what I received, I haven’t thought enough about what it might mean." And I have some similar rules for chartering that tell me that if I don't come up with at least X test/charter ideas, then I haven't really explored the problem enough. I would never expect I'd execute all the charters I come up with, but because I'm a believer in overproduction and abandonment, I feel it leads to better work on my part.

Think about different aspects of what you do on a day to day basis and see if there are any areas where you might benefit from overproduction and abandonment. What "minimums" could you put in place to help you get better at developing more test ideas, or help you better execute certain tasks, or just help you break through a wall you might have while learning something new? Sometimes setting aside judgments about the quality of your work, and just getting time logged doing it, can be an important step at becoming more effective at it.

Pulling apart sequence and activity diagrams

I like using sequence and activity diagrams to generate test ideas. When looking at a sequence and activity diagrams, ask yourself what would happen in the system if a specific line didn't happen or didn't happen correctly. For example, if you look at the following example from Donald Bell's article on UML Sequence Diagrams:

If I were to use this to create test cases, I'd ask questions like:

• What happens if the amount doesn't come back?
• What happens if the amount that comes back is in an invalid or unexpected format?
• What happens if the amount that comes back is corrupted, very large, or if the connection is interrupted?
• (similar questions for balance)
• If the balance is less than the amount, what happens if I note the returned check without adding the insufficient fund fee? Can I do that?
• What if, while noting the returned check, I use an invalid check number, use a check that's already been noted previously, or don't pass a check number?
• (etc...)

You get the idea. When generating the test ideas, just list them out as you think of them. If you have a sequence or activity diagram of any complexity, you'll get a long list. Afterward, you can work with other project team members to shrink the list down into the scenarios you feel will provide the most value to the project.

Understanding your point of view (persona, etc…)

All testing is done from a point of view. Some teams call these personas. For each round of testing you're doing, it can be helpful to explicitly state your point of view up front. Are you taking the point of view of the customer? The point of view of a sys admin? The point of view of a developer who's going to need to triage an issue in production? The point of view of a hacker? Whatever your point of view is, try to keep it clear in your mind as you're testing. If you're keeping test notes, write it down in your notes.

Each point of view will have different strengths and weaknesses based on what you're testing. After you're done testing, ask yourself what you would have done differently (if anything) based on a different point of view. If you feel you might have missed something important, capture that test idea somewhere.

Testing is questioning

I think it was Jon Bach's work on open book testing that really drove the analogy of every test being a question home for me. Each test you execute is really just a question you're asking the software. If you can think of a question to ask, you've though of a test to execute. It's elegant in it's simplicity.

So what does this mean for your testing? Because for each test you're thinking of running, you can use this analogy as a tool to develop a better test:

• Before you run each test, in your mind, clearly articulate the question that you're asking. I've found that this not only can help clarify for me what I'm really trying to accomplish with any given test, but it also helps me think of things that I need to verify after my test has been executed. Once I think of a test as a question, I recognize that there might be multiple ways to answer it. That gives me multiple things I need to verify (like log files, database entries, etc...).
• Before you run each test (or more practically, any given set of tests), try to express the tests in several different ways to help clarify them. This is an attempt to clarify the scope of the test. For example, what do I really mean when I say "Ensure that when you stack an utterance you can't bypass required script logic?" Does it matter what I stack? Does it matter what area of script logic? How would I know if I bypassed it successfully? Is there a way for it to be partially bypassed? How would I know? How else could I phrase the question my test is asking to help make these things more clear?
• It can also be helpful to ask if any given test should be broken out into smaller tests. Are there smaller and simpler questions that could be asked? What are they? Do they tell you more or less?
• Finally, the last thing you can do is ask yourself if you test might have multiple right answers. Does pass really mean pass? And does fail really mean fail? What conditions could take place that might make you change your answer? While this might not change this specific round of testing, it might give you ideas for other tests.