Soak Test in Software: Meaning & Examples
Thomas Hamilton
⚡ Smart Summary
Soak Testing applies a sustained, realistic load to an application over an extended period to expose problems that only appear with time. Memory leaks, connection exhaustion, and slow performance drift are the defects it is designed to catch.
What is Soak Testing?
Soak Testing is a type of non functional testing that is used to measure performance of a software application under a huge volume of load for an extended period of time. The goal of Soak testing is to ensure whether the software application sustains high volume of usage and to check what would happen outside its design expectations.
The image below depicts a testing cycle that shows at which stage the Soak Testing (Type of Performance Test) is performed on an application.
In this type of testing, what basically monitored is the memory utilization by an application in a system. It is testing at a system level, to find whether the system will stand up to a very high volume of usage and to see what would happen outside its design expectations.
Why do Soak Testing?
A system may behave normally when used for 2 hours, but when the same system is used continuously for 10 hours or more than that then it may fail or behave abnormally/randomly/it may crash. To predict such failure Soak Testing is performed.
When to do Soak Testing?
Soak Testing should be done in the following scenarios: –
- Before the built is deployed to the client i.e. prior to the release of any application on a specific platform, it needs to go through a successful series of load tests at high or equivalent traffic levels. After that soak testing is performed. It helps us to determine how to run any particular application for an extended period. If issues like memory leaks/memory corruption are found during the period i.e. when it is on Soak, then it should be immediately reported.
- The best time to do a soak testing is over the weekends as an application need to be in a running state for as long as over a day or night. It totally depends on the limitations of the testing situation. Soak tests are one of the most important compliance requirements which need to be very strictly followed by every company.
Soak Testing Strategy
Long Session Soak Testing is a strategy where a system is under load for a longer time.
A simple example is where the user stays logged into a system for many hours executing a number of business transactions. In this way, a lot of data gets created. There can be lots of load on the system/database server which can result in stalling/crashing of the system/database server.
Under Long Session Soak Testing, multiple day’s (say 30 days) activities are performed in a restrained time frame (say 2 days). The number of transaction in this restrained timeframe should match or surpass multiple days’ worth of transactions. The focus should be on the number of transactions processed. The most important part of Soak Testing is to check the available memory in the CPU and the amount of the memory that will be in usage. We need to record the memory usage at the start and the end of a soak test. If necessary, then the memory usage of facilities such as Java Virtual Machines are also important and needs to be monitored.
Below are few more checks that need to be done by any user/tester before they begin with Soak Testing:
a) Monitor the database resource consumption.
b) Monitor the server resource consumption (ex- CPU usage).
c) Soak test should run with realistic user concurrency.
Characteristics of Soak Testing
A standard Soak Testing Method should have the following characteristics: –
- The duration of most Soak Test is often determined by the time available.
- Any application must run without any interruption if it requires an extended period of time.
- It should cover all the scenarios that are agreed upon by the stakeholders.
- Mostly every system has a regular maintenance window time period and the time between such window periods is a key driver for determining the scope of a Soak Test.
Examples of Soak Testing
- In the case of banking domain when there is a large amount of data from merchants, the tester will put the system under load continuously for 70hrs to 150 hrs to check how the application behaves during this loading period.
- Suppose there are 33,000 logins, which needs to be put through the system, it represents seven and a half days of activity. In this case, a 60- 70 hours of Soak Test can be started by Friday evening around 6 pm which can be completed by Monday morning at 6 am. Only with such a test, it will be possible to observe any degradation of performance under the controlled conditions.
- In the case of Video games, Mobile applications, etc. involve leaving the game or application in a running state for a prolonged time period, in various modes of operation- such as idling, paused at the title screen and so on to find out whether an application can handle the continuously expected load.
Common issues observed during Soak Testing
- Memory allocation (memory leaks that would eventually result in a memory crisis or rounding errors that manifest only over time).
- Database resource utilization (Failure to close database cursors under some conditions which would eventually result in the entire system stalling).
- It can also lead to performance degradation, i.e. to ensure that the response time after a long period of sustained activity is as good as it was at the beginning of the test.
- Failure to close connections between tiers of a multi-tiered system under some circumstances which could stall some or all modules of the system.
- The gradual degradation of a response time of some functions as internal data-structures become less efficient during a long test.
How This Test Fits the Performance Testing Family
Performance testing is an umbrella term. The variants below differ only in the shape of the load applied and the duration it is held for, which is why they are so often confused with one another.
| Test type | Load pattern | Question it answers |
|---|---|---|
| Load testing | Expected peak load, short duration | Does the system meet its targets under normal peak traffic? |
| Stress testing | Increased beyond capacity until failure | Where does it break, and does it fail gracefully? |
| Spike testing | Sudden extreme surge, then withdrawal | Does it survive and recover from a traffic shock? |
| Endurance testing | Normal load held for many hours | Does performance degrade over time? |
| Soak testing | Sustained load over an extended period | Are there memory leaks or resource exhaustion? |
| Stability testing | Varying load across conditions | Does the system stay reliable as conditions change? |
| Volume testing | Normal users, very large data volume | Does it cope as the database grows? |
Endurance and soak testing are frequently treated as synonyms. In common usage they are: both hold a sustained load for a long period. Where teams do distinguish them, endurance testing focuses on whether response times drift upward, while soak testing focuses on resource consumption such as memory, file handles, and connection pools. Running one usually gives you the evidence for both.
Key Metrics to Capture During the Test
A performance run is only as good as what you record while it executes. Capture these six on the server and the client side, then compare them against the baseline rather than against a gut feeling.
| Metric | What it tells you | Warning sign |
|---|---|---|
| Average response time | Typical user experience | Any upward drift across the run |
| 95th percentile response time | The experience of the slowest users | Far above the average, meaning inconsistency |
| Throughput | Requests handled per second | Falling while load stays constant |
| Error rate | Share of failed or timed-out requests | Any rise above the agreed threshold |
| CPU and memory use | Server resource headroom | Memory that climbs and never returns |
| Database connections and threads | Pool exhaustion | Counts that grow steadily without release |
Read the average and the percentile together. An average of 800 ms with a 95th percentile of 900 ms describes a consistent system. The same average with a 95th percentile of 9 seconds means one user in twenty is having a bad time, and the average is hiding it.
Watch the shape, not just the value. In any long-running test, a flat resource line is a pass and a rising one is a leak, even when the absolute number is still comfortably inside the limit at the moment the run ends.
