Leveraging integrated tools
For many teams (especially large teams), the team often has a suite of tools in place to help facilitate the development process. It could be Microsoft Visual Studio, Rational Team Concert, or even the Atlassian suite of products. Regardless of the tools in place, if there's an integration across the tools and your team isn't using that integration effectively then you might want to take a look at how you can better leverage all the features you've already paid for.
This tip, stemming from a frustration on the part of an IWST tester who constantly has to manually re-enter data from one tool to another, points to a problem in cross-team workflows. If these tools were not in place, perhaps it wouldn't be that big of a deal. However, since the company has already paid for these tools, it's really frustrating when the testing team can't take advantage to the features simply because people haven't been educated on the implications to other teams of how they use the tools.
If you have these tools in place, pull together a small cross-functional team to look at how people are using the tools. And don't just talk about what you think is happening - really look at what's happening on projects. Based on that, see if there are opportunities to better leverage the integrated features of the platform, and roll out changes supported by training and helping people understand the value gained by the changes.
This tip was part of a brainstorm developed at the September 2011 Indianapolis Workshop on Software Testing on the topic of "Documenting for Testing." The attendees of that workshop (and participants in the brainstorm) included: Charlie Audritsh, Scott Barber, Rick Grey, Michael Kelly, Natalie Mego, Charles Penn, Margie Weaver, and Tina Zaza.
Reduce documentation – charters over test cases
If you're looking to shave some time off of your test documentation tasks, take a look at working some charters into your test plan so you have fewer scripted test cases to write. Charters are simple to document - you start with defining a mission, an anticipated length, and if you'd like a bit more structure you can include a bullet-list of coverage items so you're sure the person who executes the charter hits all the critical parts. Contrast that with a typical Microsoft Word test case template which includes columns for step, expected result, screenshots, data, setup tasks, comments, etc... It's just a lot of work, for (many times) very little return. If you have not tried using charters before, check em' out.
This tip was part of a brainstorm developed at the September 2011 Indianapolis Workshop on Software Testing on the topic of "Documenting for Testing." The attendees of that workshop (and participants in the brainstorm) included: Charlie Audritsh, Scott Barber, Rick Grey, Michael Kelly, Natalie Mego, Charles Penn, Margie Weaver, and Tina Zaza.
Sketching with a pencil
Okay, it's actually The Pencil. The Pencil Project powered by Mozilla.
What is that?
The Pencil Project's unique mission is to build a free and opensource tool for making diagrams and GUI prototyping that everyone can use.
Top features:
•Built-in stencils for diagraming and prototyping
•Multi-page document with background page
•Inter-page linkings!
•On-screen text editing with rich-text supports
•Exporting to HTML, PNG, Openoffice.org document, Word document and PDF.
•Undo/redo supports
•Installing user-defined stencils and templates
•Standard drawing operations: aligning, z-ordering, scaling, rotating...
•Cross-platforms
•Adding external objects
•Personal Collection
•Clipart Browser
•Object snapping
•Sketchy Stencil
•And much more...Licensing and Versions:
Pencil will always be free as it is released under the GPL version 2 and is available for virtually all platforms that Firefox 3 can run. The first version of Pencil is tested against GNU/Linux 2.6 (Fedora, Ubuntu and Arch) with GTK+, Windows XP and Windows Vista/7.
And here's what I created as a trial in a few minutes.
Documented == Unambiguous ?
A quick reminder: writing can be as much ambiguous as saying. Or even more.
FireShot
I found really cool web page capture program over the holiday call FireShot. FireShot is a simple tool that lets you capture and annotate web pages. I needed something that would capture more than a simple screenshot since I couldn't size the page small enough to fit it into one image. This was perfect. Snap the shot, mark it up, send it off. Simple - and it's just a browser plugin. Check it out.
Folder Structure Template Document (XMind Mindmap)
Many times while documenting folder structure I used to take a screenshot of a tree in MS Explorer and put it to the document along with some written descriptions.
There were drawbacks, but it was acceptable as "quick and dirty". Now, with a mind mapping tool, like XMind, we can finally make it "quick, nice, maintainable, and interactive".
And here's why.
- You can quickly and easily customize appearance (size, font, shape, color) of each topic
- You can insert images into topics to further improve visual presentation (2 clicks job)
- For every topic, representing a folder, you can provide that many additional details:
- short description as a Label
- long description as a Note
- attached documents
- Hyperlink to a physical folder
- sub-structure as a Floating Topic
- Relationship to another Topic
- With a free version, you can export as an image or html document; with a Pro version you can export to PDF or Word
- And all of that with a quick and convenient visual drag-n-drop interface
The image below is linked to the downloadable *.xmt template on XMind.net web-site. A generic Automated Testing Suite folder structure was taken as a sample.
XMind mindmap - folder structure template document
Test Design with Mind Maps
Today's tip is two-fold.
As a first part, it's a great example of a rapid test design practice with XMind mind mapping tool, provided as experience report by Darren McMillan.
- Mind mapping
- Increases creativity
- Reduces test case creation time
- Increases visibility of the bigger picture
- Very flexible to changing requirements
- Can highlight areas of concern (or be marked for a follow up to any questions).
- Grouping conditions into types of testing
- Generate much better test conditions
- Provides more coverage
- Using templates of testing types makes you at least consider that type of testing, when writing conditions.
- When re-run these often result in new conditions being added & defects found due to the increased awareness
- Lean test cases
- Easy to dump from the map into a test management tool
- If available the folder hierarchy can become your steps
- Blend in easily with exploratory testing. Prevents a script monkey mentality.
- Much lower cost to generate and maintain, whilst yielding better results.
As a second part, I link you back to 2006, to the article "X Marks the Test Case: Using Mind Maps for Software Design" by Rob Sabourin.
- Mind Maps to Help Define Equivalence Classes
- Identify the variables
- Identify classes based on application logic, input, and memory (AIM)
- Identify invalid classes
- Mind Maps to Identify Usage Scenarios
- Mind Maps to Identify Quality Factors
Do we really need a script or can we use a checklist?
On a current project, we have a lot of testing that requires deep domain knowledge. Given the level of domain knowledge, we're planning on getting domain experts to do a good portion of the testing. This has uncovered an interesting opportunity for us. To date, most testing done in this organization has been scripted (giant Word documents full of steps and screenshots). In the past I believe this was done because the testers didn't have the requisite domain knowledge to execute the tests without a lot of help. But now, by making this shift, we can reassess if we really need test scripts.
We've decided that for much of the testing, we'll switch to checklists instead of test scripts. While we might still have a couple of test procedure documents which provide high-level outlines for how to do something, the details of the testing will be in some (relatively) simple Excel checklists. This saves us a lot of work, since we don't need to update a bunch of artifacts. I suspect it will also increase our test execution velocity by reducing the overhead of running a series of tests.
Kaner has been talking a lot recently about the value of checklists. If you've not reviewed the work, check it out. Look for opportunities where you might leverage checklists instead of some of the more traditional heavyweight scripts associated with testing in IT corporate environments.
Template for session notes
When I teach people how to do exploratory testing, a common point of confusion is around what to put in your notes. While I often tell people it depends on you, what you're testing, and the company you're working for - they still want some concrete advice. So I often show some examples from past projects and provide the following template:
- Mission: list out what you're testing with this charter
- Environment: list out meta information related to your test environment (versions, configuration, location, etc...)
- Risk: as you test, list out what risks you're looking for while testing
- Coverage: as you test, list out what areas, features, users, data, or other meaningful dimension you're covering while testing -- it's worth noting, I also instruct them to list out what they didn't have time to cover...
- Techniques: as you test, list out what you're doing... what techniques you're using, how you develop tests, etc... (in math class, this would be the "show your work" section of the document)
- Status: as you test, list out questions that occur to you that you need to get answered later, possible issues/problems/bugs you find while testing, notes about automation or future test sessions, etc....
- Obstacles: as you test, list out things that get it your way or ideas you have for things that would make your testing more effective -- this can be tools, hardware, information, training, etc...
For those readers who do session based testing on a regular basis, you'll notice I don't capture some of the classic items like setup time and time spent investigating issues. If you need to capture those metrics (or other metrics your team uses), simply add them in. Over time your session notes morph to become your own and you'll develop a format that works for you.
I'd be interested to see what other people capture.
Reading effectiveness tool
Many times, when testing websites, we forget about the static content of the site. Here is a simple tool that applies some simple heuristics to help gauge the effectiveness of the content on your site. Even if you don't end up using it, as you read through it you can get some ideas for how to make content more accessible.
