How does Citrix Improve Response Times
The other day 1202Performance had an interesting post on how Citrix improves response times. While I don't think the post addresses the subtleties of how Citrix works, I think the simplification might be useful to those new to Citrix. You can check it out here.
Stella
Stella is a handy web application testing tool developed by Solutious Inc, a small software company based in Montreal. Stella is a very lightweight Ruby test tool for functional and performance testing. Like JMeter, Stella doesn't simulate a browser; it generates HTTP requests and parses the responses. Currently, Stella provides support for automatic parsing of HTML, XML, XHTML, YAML and JSON.
A few months ago, I wrote an introductory article on Stella for SearchSoftwareQuality.com - you can find it here.
BrowserMob
Today's tip comes via Tim Koopmans. Tim recently posted on landing page load time and how tools like BrowserMob can help. Based on his post, I went over and took a look at BrowserMob and ran a couple of tests on my personal website. There were a couple of interesting things I found in the free tool they provide:
- They provide test results from four locations: Washington DC, Dublin, San Fransisco, and Dallas.
- They provide historical results across test runs:
- They provide a detailed breakdown of load times by object, by download site:
Based on these detailed results, I was even able to find out that I have a couple of 404's showing up in my current WordPress theme. I rather like the simple interface, and I find tools like this can be quite helpful when taking an initial look at a site's load time and where that time is going.
Tie performance to business goals
Following up on the performance pitch post, here are some tips for helping get your technology team talking in the language of your business team:
- Take the time to define both top-line and bottom-line application business metrics
- Work to prioritize that list of metrics to better understand what’s most important (creating tiers can help)
- Identify what processes and transactions will affect those key metrics
When the team finds a possible performance issue later on, they can then translate what might otherwise be a generic metric (we're X seconds slower on transaction Y) into something that has meaning to the business (given that we're X seconds slower on transaction Y, we expect abandonment to go up Z%).
Defining those metrics, prioritizing them, and tying those to transactions doesn't necessarily need to be complicated. For some applications it will be. But for most applications I've tested, I suspect we could have done this over a couple one-hour workshops using Excel. Don't make it harder than it needs to be.
soapUI
I'm surprised I haven't posted about soapUI yet given how often I use it. It's one of the first web service testing tools I ever used, and I haven't had much need to look elsewhere. soapUI is great for both functional testing and performance testing. They also have a great service mocking feature.
I've never used the pro version, so I can't speak to those features or the support that comes with it. But I will tell you this, the first (and only) time I had an issue with the tool I contacted the developers and they had a build for me the next day with a fix. I was amazed at the turnaround. I'm sure not all fixes can happen that fast, but it still says something about their passion for the tool and the people that use it.
You can checkout the full feature set here: http://www.soapui.org/features.html
The performance pitch
Stoyan Stefanov recently posted a fantastic set of slides on pitching better website performance. The slides look at different websites and how performance affected the core business. It's a cool idea. I think material like this is useful for a variety of reasons, but most of all because it provides insights into how I can better transform my technical work into something that talks about business value. Each slide is a mini case study on looking at a company and translating performance into a business driver important to them. As testers, we need to be good at that. The more exposure we get to materials like this, the easier it becomes to recognize similar opportunities in our own organizations.
PageTest
Following up on yesterday's post, another tool from Stoyan Stefanov's recent post on performance testing tools is PageTest. From the Stoyan's post:
"AOL's PageTest is an IE plugin but also a hosted service which is a great way to show your boss/client performance details without inconveniencing them with challenging download and installation activities. PageTest gives you a waterfall view of the page load and a checklist of things to improve, plus some screenshots of interesting moments during load and even a video - an excellent view of how the page looks like in slow speeds. The hosted service can show you the dial-up experience in 4 different places in the world.
For some other cool tools, checkout Stoyan's full post.
YSlow
Stoyan Stefanov recently did a post on performance testing tools. In that post, he mentioned Yahoo! YSlow. From the YSlow website:
"YSlow analyzes web pages and suggests ways to improve their performance based on a set of rules for high performance web pages. YSlow is a Firefox add-on integrated with the Firebug web development tool. YSlow grades web page based on one of three predefined ruleset or a user-defined ruleset. It offers suggestions for improving the page's performance, summarizes the page's components, displays statistics about the page, and provides tools for performance analysis, including Smush.it™ and JSLint."
Statistical Aggregate Report listener
If you're a regular JMeter user, you might be interested in taking a look at the Statistical Aggregate Report listener. The Statistical Aggregate Report listener is a JMeter plugin that kinda combines the Graph Results listener and the Aggregate Report listener into one handy little chart.
Perfmon for SQL Server Analysis
At the moment I'm using Perfmon to gather information on an SQL server and application I'm testing.
To know what are the best counters to use, I headed over to this great post by Brent Ozar where he explains in detail what perfmon counters you need and how you setup them up.
Here's what he recommends:
Performance Monitor Counters for SQL Server Analysis
These are listed OBJECT first, then COUNTER
- Memory - Available MBytes
- Paging File - % Usage
- Physical Disk - % Disk Time
- Physical Disk - Avg. Disk Queue Length
- Physical Disk - Avg. Disk sec/Read
- Physical Disk - Avg. Disk sec/Write
- Physical Disk - Disk Reads/sec
- Physical Disk - Disk Writes/sec
- Processor - % Processor Time
- SQLServer:Buffer Manager - Buffer cache hit ratio
- SQLServer:Buffer Manager - Page life expectancy
- SQLServer:General Statistics - User Connections
- SQLServer:Memory Manager - Memory Grants Pending
- System - Processor Queue Length
Of course, that's the easy part. The real difficulty is being able to parse the log file and know what to do with the data.
Thankfully, Brent describes how to format the results in excel. He also describes how to analyze the data, starting with the the CPU, then Memory and finally Disk Metrics.
Personally, I'm going to go to my client's DBA and ask them what he thinks of the data, as he knows what I'm testing. It's not in my clients best interest to spend excessive amounts of time analysing this type of data when some-one internally is able to do so far quicker.
