In load testing the software under test in subjected to various levels of "load", to test its behaviour.

Load testing is not defined in BS7925-1. It can mean different things depending on whom you are speaking. It is most definitely a non-functional test. The aspect of behaviour that concerned is that of performance.

The testing has to be conducted within a structured framework otherwise its results are open to question.

Various elements need to be considered before engaging in Load Testing.

Plannning Load Testing is like any other activity. Good quality is the result of planning and foresight, enabling the test to be ran over and over again.

Objectives What do we hope to achieve? This question has to be answered in relation to the specified requirements of the Software Under Test (SUT). The tester needs to know how the SUT a) should be behave under, b) a given load.

For example for a large site it may be accepted as a business risk that the SUT will run slower at high loads. However, this needs confirming, otherwise the business runs the risk of the SUT due to underestimating the effect of higher loads.

On what? Which aspect of the system is to bear the specified load? The answer ranges from a single component to a Web Services system made up of many third party applications. As the SUT becomes larger, it becomes more complex. Consequently the load testing suite becomes more complex.

For example it is decided a website checkout facililty needs testing. This can be tested in isolation as a component or as part of the system. If it can only be tested once the integration has taken place and a fault occurs, was the problem with the cart or somewhere else in the system.

Iterative development styles suit load testing, as they successively build up a full system. The RUP with its inception etc phases, gives many opportunities. During the Inception and Elaboration phases the core architecture can be assessed. In construction smaller components can be tested as they added incrementally and then as an integrated whole system.

The Sequential or Waterfal method is less suited, due to its single big bang approach to integration. Thus after months or even years of development, can the system as a whole be load tested. The problem is that it is difficult to establish what you are testing and the causes of failure.

Which Load? How is the load on the SUT to be generated? Multiple users or a small number of users generating large numbers of transactions? The important factor is to replicate the situation the SUT will meet in the real world. For example a bank system, processing payments will have one user processing a lot of data. Whereas a consumer website may have millions of users conducting one or two transactions a day.

Should the data be increased as the test progresses, or at a steady sustained level for a long period?

Should the load be of a single type or multiple? For example in our checkout example, a certain percentage of the transactions should be of the user dropping out of the transaction. Again this mirrors the real world.

Testers looking at load testing may want to consider using Monte Carlo simulations. Here data is randomly generated according to a set of rules. I.e. we do not know which transaction is coming next, but it has a 1 in 5 chance of being a drop out from the checkout.

An important point, often overlooked is that the load may spike. Advertising may make the SUT suddenly very attractive to users. The classic example of failure to realise this, was the Victoria's Secrets affair. Here a fashion show of lingerie, during the half time break in the Super Bowl was heavily promoted. Consequently millions of Americans logged on, and lo and behold the servers crashed under the strain.

Expected Outcomes The tester has to be aware of what to expect. Should this not be the case, then raise it with the analyst. Just because load testing is non-functional, we still need to know what to expect.

Logging As the test progresses we need to know what exactly a) the load being applied and b) the behaviour exibhited. This has two benefits, firstly we can keep track of any deterioration of performance as the load increases.

Tools Due to the large amount of transactions, many Load Testing can only realistically be undertaken with Computer Aided Software Tools (CAST). LoadRunner and Rational Robot are the front runners for this type of testing.