Unlearning for testers
Today I want to present a non-technical advice: unlearning.
The traditional definition* of unlearning stands for giving up present habits and/or knowledge:
–verb (used with object)
1. to forget or lose knowledge of.
2. to discard or put aside certain knowledge as being false or binding: to unlearn preconceptions.
–verb (used without object)
3. to lose or discard knowledge.
_____________________________________________________
* -Taken from Dictionary.com
As we can see from this definition, there are negative and positive sides of unlearning.
How to benefit from unlearning?
- Use it as a brainstorming tool
- Use it as a problem-solving tool
- Use it for retrospective analysis and learning
As an example, below I put a few core assumptions about software development and testing that have already been unlearnt or currently being unlearnt in the industry.
- One needs requirements and test cases to do testing
- Testing can only be conducted after a build is deployed on test environment
- Verification tests passed -> no bugs
- A role of written documentation is critically important in software quality assurance
- Software quality can be unambiguously defined by requirements
- No bugs found -> no bugs in software
- No bugs in software -> customer satisfaction
- Programmers do programming, testers do testing
- Years of experience with tools, or techniques, or technologies is the most important hiring criteria
Crash test your application
Image courtesy: theinspirationroom.com
Sometime ago I wrote about a quick way to shutdown a hanged application.
But, in fact, shutting down a working application could help you exposing certain types of resource management defects. This is how ends become means.
Sample tests, suggested below, were tried on different Windows Desktop applications, sometimes helping to reveal serious issues.
Local Tests
- Start an application. Begin working process then shutdown the application in the middle of it. Start the application. See if it reports than the previous session wasn't properly finished (and does it suggest something about it, like "start in safe mode").
- Start an application. Begin working process then shutdown the application in the middle of it. Start the application. See if it can access any previously opened files, and how is it handling and reporting the issues.
- Change settings in an application, apply changes. Shutdown the application. If the application saves settings only at exit point, all changes will be lost.
- Change settings in an application, apply changes and shutdown the application simultaneously. Start the application. See if it can access configuration files, does it report that they might be corrupted, does it suggest repairing them? (you can also try corrupting the files manually).
- If an application allows running 2 copies: open 2, then shutdown one of them. Observe behavior of the remaining one.
Server Tests
- Start a client. Shutdown a client. Try to login as the same user: a) immediately b) after a timeout given.
- Start a client. Shutdown a client. Repeat this process (you may want to automate that) while monitoring how Server side of an application allocates and releases (does it?) RAM and hard-drive resources.
- Start a client. Shutdown a client. Repeat this process (you may want to automate that) for a while - you may see errors like "max number of connections exceeded", etc.
Let us know results of your exploration.
Ten 20 Years Old Usability Heuristics
Jakob Nielsen, one of the world's top experts in user interface design and application usability, was once already referred in Quick Testing Tips.
In today's tip I want to bring readers' attention to Usability Heuristics as ready-to-use test oracles. Please read entire list via the link provided, I put here just one as an example (per author's copyright requirements).
User control and freedom
Users often choose system functions by mistake and will need a clearly marked "emergency exit" to leave the unwanted state without having to go through an extended dialogue. Support undo and redo.
SEED NATALI: GUI Step Automation Heuristic
Whenever you implement automation of GUI steps, suggested heuristic might help you to keep track of operations you need to code.
(click on the picture to see mind map in online version with expanded nodes)
SEED NATALI acronym stands for the following.
- Synchronize till object
- Exists
- Enabled
- Displayed
- verify Number of Arguments
- verify Type of Arguments
- Log test flow
- Investigate any issues occurred
Don’t get trapped by reuse
I'm a big fan of reusing previous testing artifacts. If you have a test bed ready and available, I'm interested in taking a look at it if it can help me shave time off my project. However, don't let past testing artifacts trap you or bias you. Not only can they have the wrong focus, but they run the potential of slowing you down instead of speeding you up. If you have a lot of rework to do, or if you need to spend a lot of time disambiguating coverage, you might spend more time reviewing than you would have spent just starting from scratch.
Here are some things I look at when trying to determine if an existing test bed is a time sink:
- There's no clear summary of what's included in the test bed.
- No one on the team was around when the test bed was originally used. It's coverage is hearsay.
- A preliminary review of the test bed shows that several different types of updates are required to make them useful (data elements, screen shots, endpoints, etc...).
- 100% of the original test bed development and execution was done by an external provider and the artifacts were "turned over" at the end of the project.
Quick tests for landing pages
Mary Flaherty wrote an article recently titled Are Your Landing Pages Driving Clients Away? In the article, she provides some great tips, that I think also make great test ideas. Here's a quick summary:
- Does the landing page have a clear call to action? In her words, "Where's the doorbell?" Test ideas around this might include asking users if they know what they're suppose to do next once they hit the landing page. Or watching where they go and seeing if there's convergence or not.
- Does the landing page present all the options in a manageable way? That is, can you quickly see what the options are, and find what you might need? In her words, "Which door is which?" Test ideas around this might include asking users to find specific information and seeing how long it takes for them to find it from landing - again looking for convergence. I might also ask users to write down what they think the site contains without navigating past the landing, and time how long it takes them to do it.
- If you collect visitor information, only ask for what's absolutely necessary. Test ideas include looking at the business process being served by collecting the information and evaluating how it would change if something wasn't collected. Is that ok? Less might be more at this early stage...
- How is information laid out and what's the quality of the copy editing. She identifies this as the risk of "TMI." Test ideas include making sure the site has copy guidelines (and they are followed), ensuring the writing is appropriate for the audience (jargon, concepts, structure, etc...), and making sure the layout is visually appealing. Mary also talks about dumping unneeded animations, choosing relevant images, and keeping information up to date and relavent. All of these are possible tests and/or heuristics for possible problems with the site.
Check out the full article for more ideas.
MARTA for managing risk
In a recent blog post, Antony Marcano posted his mnemonic for managing risk: MARTA
Mitigate
Avoid
Reduce
Transfer
Accept
Read the full post for details behind each point.
A heuristic for regression testing
Recently, I've devised a mnemonic for regression testing: RCRCRC
Manual calibration
Many times when performance testing, I'll manually calibrate my performance test script by loading the page manually and comparing those load times with those generated from my test tool. Doing this with a load of one user allows me to get an idea of how much my tool is skewing the numbers. Often, I find the tool is just a tad bit worse in terms of response times. If it's too far off, that normally tells me something is wrong.
