Limitations of Performance Testing

Load testing is a powerful and cost-effective technique for analyzing web application performance. Even the most carefully constructed load-testing environment cannot reproduce the real world perfectly.

Some of the limitations include:

Network Imitation: Load tests are executed in a LAN. Real world traffic, on the other hand is sent over a WAN, and can encounter many kinds of problems, from network latency to traffic surges.

Users Simulation: Virtual users are only a simulation of real users. There is no way to make their behaviors and actions perfectly mimic the human idiosyncrasies. Real users at actual browsers exhibit the infinite variety of human behaviors no software tool can reproduce.

Browser Emulation: Web browsers are very complex programs that use even more complex technologies like multi-threading and document caching. If browsers are not emulated accurately, the amount and types of requests virtual users send to the web application being tested will not correspond to real-life traffic.

Measurements: A truism in the sciences says that the act of observation alters the thing observed. When monitoring a web application, care must be taken not to slow down the application and so influence its behavior.

Random number generator: To simulate real users as accurately as possible, a load test requires the generation and assignation of random quantities. This can be an issue, since even well-known random number generators have been known to have problems.

Source : “Software Testing – Effective Methods, Tools and Techniques“ book authored by our director, Mr Pradeep Oak and published by McGraw Hill.