Stability Testing in Software Testing

What is Stability Testing?

Stability Testing is a type of non functional software testing performed to measure efficiency and ability of a software application to continuously function over a long period of time. The purpose of Stability testing is checking if the software application crashes or fails over normal use at any point of time by exercising its full range of use.

Stability Testing is done to check the efficiency of a developed product beyond normal operational capacity, often to a breakpoint. There is greater significance is on error handling, software reliability, robustness and scalability of a product under heavy load rather than checking the system behavior under normal circumstances.

Stability testing assesses stability problems. This testing is primarily intended to stress the software component to the maximum. It is a Non-Functional Technique.

Stability testing is also referred to as a Load or endurance testing.

Risks if System under test has not undergone Stability Test

For an application under test where a large number of users are introduced and applications that have to run for months without restarting, a number of problems are likely to occur:

The possible error can be faced,

• the system slows down
• the system encounters functionality problems
• the system shows wired behavior
• the system crashes altogether

In Software Engineering, Stability Testing typically involves exercising the system with heavy users (virtual) and measuring the performance parameters to verify whether the system can support the anticipated load.

Why do Stability Testing

This kind of testing helps users to understand the ways the system will work in real-life situations.

Hence, Stability Testing allows you to check,

• Provide confidence in the stability of your system under test.
• Ensure that your system can handle large programs.
• Monitor the effectiveness of your system.
• Test system stability under stress.

It plays an important role in product development as it is used to determine the limitations of a software product under test before it is released or the areas of more improvement before the product goes live or at Production.

A very common example of Stability testing technique is

Online Shopping Portals: Stability testing will check how the website will behave when –

• High Amount of data entered at peak time
• Number of hits at a certain specific time
• Page load issue at the same time
• Behavior of system
• Responsiveness of the system and many more come under Stability Testing

Another example

A CPU test is a popular form of stability test under Performance Testing technique. This test checks for processor stability and also monitors its performance as the processor’s workload is increased.

How to do Stability Testing

• To determine the scope and objective of the testing, we must ensure that the Application Server(s) do not crash during the Load Test executions.
• To determine the Business issues, verify the system performance and load as per end user perspective.
• To assign the different Responsibilities and Roles like -Creating Test plan, Test Case design, Test case review, Test execution, etc.
• To ensure the Test deliverables within the specified time
• To ensure proper Load Testing tools and experience team is present for the same.
• To measure the risk and cost involves in the testing. This will determine the cost of each execution in terms of CPU utilization and memory.
• Determine the Defect tracking and reporting and their proper mapping with the requirements.

Test Case for Stability Testing for CPU Performance

• To verify the Upper limit of the system.
• How system crashes or recovers.
• A total number of transactions completed per request.
• Whether or not transaction response stays steady or increases over time.
• How the system behaves under heavy load.
• Its response and behavior under heavy load.

Test Reports for Stability Testing

Several statistics are gathered and measured during test executions; these numbers are analyzed in order to generate a report and to identify possible performance problems.

Examples of statistics collected under test are:

• Transaction Response Times: The average time is taken to perform transactions during the test. This statistic will evaluate whether the performance of the server is within the acceptable minimum and maximum transaction performance time periods defined for the system.This information will evaluate the time taken in processing the request by the web server and sent to the application server, which in most of the cases will make a request to a database server.
• Hits Per Second: The number of hits made on the server by users. These statistics benefits to determine the number of load users generate, with respect to a number of hits.
• Throughput: The amount of throughput on the Web server during the test which is measured in bytes. Throughput means the amount of data that the users received from the server at any given time. This statistic helps to evaluate the amount of load that users generate.
• Transaction per second: These are the total number of completed transactions (both successful and failed) performed during a test. This statistic helps to check the actual transaction load on the system.
• CPU: CPU percentage utilization spent during a test.
• Memory: Memory usage during a test.
• Disk: utilization of disk spaces spent during a test.

Stability Testing Basics

Stability Testing comes under Performance Testing –a technique that is performed to check some of the quality attributes of Software like stability, reliability, and availability.

This testing is used to determine how fast a system or sub-system performs under a particular workload.

Performance Testing has many types and stability testing is one of them.

• Stress testing: It is a testing type which checks the robustness of the system beyond the system capacity.
• Spike testing: It is used to check the behavior of a system by increasing a load of a system instantly. The objective is to check at which point the system will be having performance issues, or it will pass.
• Scalability testing: It is used to check the capabilities of a system. How effective the system will going to behave in growing needs, change in size and change in volume.
• Volume testing: It is a non-functional testing technique where software under test is subjected to a huge volume of data, and the behavior of a system is checked and verified accordingly.
• Load or Stability testing: (already discussed above)

Stability Testing Tools

Some of the tools for Performance Testing are as –

• WebLOAD
• LoadRunner
• Apache JMeter
• NeoLoad
• CloudTest
• Loadstorm
• LoadUI
• WAPT
• LoadImpact
• Loadster
• Httperf
• OpenSTA

How This Test Fits the Performance Testing Family

Performance testing is an umbrella term. The variants below differ only in the shape of the load applied and the duration it is held for, which is why they are so often confused with one another.

Test type	Load pattern	Question it answers
Load testing	Expected peak load, short duration	Does the system meet its targets under normal peak traffic?
Stress testing	Increased beyond capacity until failure	Where does it break, and does it fail gracefully?
Spike testing	Sudden extreme surge, then withdrawal	Does it survive and recover from a traffic shock?
Endurance testing	Normal load held for many hours	Does performance degrade over time?
Soak testing	Sustained load over an extended period	Are there memory leaks or resource exhaustion?
Stability testing	Varying load across conditions	Does the system stay reliable as conditions change?
Volume testing	Normal users, very large data volume	Does it cope as the database grows?

Endurance and soak testing are frequently treated as synonyms. In common usage they are: both hold a sustained load for a long period. Where teams do distinguish them, endurance testing focuses on whether response times drift upward, while soak testing focuses on resource consumption such as memory, file handles, and connection pools. Running one usually gives you the evidence for both.

Key Metrics to Capture During the Test

A performance run is only as good as what you record while it executes. Capture these six on the server and the client side, then compare them against the baseline rather than against a gut feeling.

Metric	What it tells you	Warning sign
Average response time	Typical user experience	Any upward drift across the run
95th percentile response time	The experience of the slowest users	Far above the average, meaning inconsistency
Throughput	Requests handled per second	Falling while load stays constant
Error rate	Share of failed or timed-out requests	Any rise above the agreed threshold
CPU and memory use	Server resource headroom	Memory that climbs and never returns
Database connections and threads	Pool exhaustion	Counts that grow steadily without release

Read the average and the percentile together. An average of 800 ms with a 95th percentile of 900 ms describes a consistent system. The same average with a 95th percentile of 9 seconds means one user in twenty is having a bad time, and the average is hiding it.

Watch the shape, not just the value. In any long-running test, a flat resource line is a pass and a rising one is a leak, even when the absolute number is still comfortably inside the limit at the moment the run ends.

FAQs