Load Testing Home

Load Testing Articles

Load Testing Links

Load Testing Books

Load Testing Tools

Load Testing Keywords

Load Testing

performance testing

What
"Testing conducted to evaluate the compliance of a system or component with specified performance requirements." [IEEE]
BS 7925-1.British Computer Society Specialist Interest Group in Software Testing (BCS SIGIST)

Performance testing is a sub-set of non-functional requirements testing. Other techniques that appear in this area include: conformance testing, conversion testing, documentation testing, penetration testing, recovery testing, serviceability testing, storage testing.

However as with most things, one mans performance test is another mans stress test.

Test techniques that I would include in performance testing are load testing and stress testing. The reason I choose to include these as part of performance testing, is that they evaluate how the behaviour is modified at under various levels of usage.

Why?
To establish that the system or component under test will behave as we expect, under the conditions we expect. Note that we are not setting out to blindly "hammer" to system into submission or destruction. The end result should that management can have confidence in the software. Additionally they will have enough information to base business related decisions on.

Who?
Usually done within the developing organisation. Although, it may be contracted out to a specialist test lab or consultancy. Contracting out is popular due to the heavy investment required in tools and training.

In terms testing roles, performance testing can be split. Typically this is split between a test analysis and test automation. Thus the test analyst will decide which tests are to be run. Test automators or engineers will actually create the scripts and configure the tools. In addition, more junior staff may act as technicians to execute them. Technicians are used because, once the test suite is written it can become a mundane task to automate them.

Where?
At the developing site or a specialist facility.

When?
The timing of performance testing depends on the model or methodologies being used. In the case of the Waterfall, perfomance testing will take place at the very end of the development lifecycle. In the worst case scenario, imagine a system that has tested without fault in functional testing with a single user. When a load of 3 concurrent users is simulated, it crashes with due to continous data locks etc. Huge amounts of time are required to fix high profile defects, at or near release time.

In incremental testing, the system is built up by adding components or sub-systems one at a time. As each component is added, the system can be tested to see if it meets requirements. Of course the individual components or sub-systems can be tested for performance by themselves.

In the case of the Rational Unified Process (RUP), performance may take place during the very earliest phase of Inception. It is Inception that the business risk is mitigated. Also one possible core architecture is put forward. Simple performance testing might be done to confirm that the architecture is suitable (in a very coarse grained manner.) This would be the case for such systems generating large numbers of transactions.

How?
Performance testing requires huge numbers: of users, data and transactions. Such requirements mean that manual testing is unfeasible. Automated testing is really the only way to go. Of course there are many tools available on the market, for instance Loadrunner.

Types of tools include, data preparation, execution. They are generally known as Computer Aided Software Testing Tools (CAST)

The difficulty with performance testing is replicating the real world environment, especially the randomness of individual events, whilst maintaining the predictability of the big picture. Various statistical methods are available to help in this process. My particular favourite is Monte Carlo scenario modelling. In a Monte Carlo simulation, we follow the rules of the casino. Thus if we shake a dice once, we do not know if it will be a 1 through to a 6. However if we roll it 6 million times, we will get a million ones, a million twos, etc.

Google
Web www.loadtesting.force9.co.uk

Load Testing Bestsellers
The bestselling books on Amazon.

Articles

Load Testing

test case

performance testing

Test Tools

Web Performance Tuning

Other Related Websites
Test Techniques
Testing Performance
Nonfunctional Testing
Automated Testing

Visit our site of the month Load Testing at loadtesting.force9.co.uk