Response Time Testing: How to Measure API Performance
Thomas Hamilton
⚡ Smart Summary
Response Time Testing measures how long a system takes to answer a request, from the moment a user sends it to the moment the application reports completion. This guide covers the metrics, the three thresholds that matter, and the tools that capture them.
What is Response Time Testing?
Response Time Testing measures the time taken for one system node to respond to the request of another. It is the interval between a request arriving and the corresponding process completing. For example, you have API, and you want to know exactly how much time it takes to execute it and return data in JSON. Response Time measures the server response of every single transaction or query.
Response time starts when a user sends a request and ends at the time that the application states that the request has completed.
How to Measure Response Time
To Measure Response Time, we can use test tools by surrounding an important business process with Start and End transactions. A business process may be an action or a set of actions which users perform in application to complete a business task like login to the application or to purchase a book on Amazon.com
For the same process to Measure response time for an API, the response will vary slightly from tool to tool. Here is why
- Step 1) Method of calculating metrics gathered by each API response time test tool
- Step 2) Tools Simulate the load and capture speed which can make a difference in the response time
- Step 3) Extra items recorded when monitoring user loads
- Step 4) Computing metrics gathered by each tool increases the response time due to high resource consumption.
- Step 5) The architecture of the two tools may be different
Types of Response Metrics
| Response Time Metrics | Explanation |
|---|---|
| Average Response Time | The Average Response Time is the average time taken for every round trip request. The Average Response Time includes the loading time of HTML, CSS, XML, images, JavaScript files, etc. Therefore, the average is affected when slow components are present in the system. |
| Peak Response Time | The Peak Response Time helps us to find potentially problematic components. It helps us to find all the irregularity in the website or system where a certain request is not handled correctly. For example, there may be a large database query executed which can affect the response time. This query does not permit the page to load at the desired time. |
| Error Rate | The Error Rate is a mathematical calculation which displays the percentage of problem requests against all requests. This percentage counts all HTTP status codes displaying an error on the server. It also counts requests that time out. |
Three Response Time Thresholds That Matter
Two figures matter most in any response time report:
- Average response time
- Maximum, or peak, response time
Together they show how long a user typically waits, and how long the unluckiest user waits.
Following are key Response time testing example values
| Response Time | Significance |
|---|---|
| 0.1 Second | It is most preferred response time. If the response time is 0.1, users always feel that the application or system is responding instantly, and do not feel any interruption. |
| 1.0 Second | This is the accepted upper limit for an uninterrupted experience. Users are unlikely to feel any interruption, though they may experience some delay. The response time of more than 1-second may interrupt user experience. |
| 10 Seconds | It is a maximum limit after which response time goes beyond the acceptable limit. However, in today’s time, if response time exceeds 6 seconds, the user will leave that site or quit the application. |
Aim for the 0.1 to 1 second band wherever possible. Users adapt to slower systems, but satisfaction falls sharply beyond about 2 seconds, and every additional second increases abandonment.
Response Time Testing Tools
Many response time testing tools are available. Two of the most widely used are:
1) JMeter
Apache JMeter is an open source load testing tool. It is designed to measure the performance of websites. This load testing tool can be used in DevOps methodology.
Features:
- JMeter allows performing load and performance test for various server types.
- This load testing tool store its test plans in XML format which allows users to generate the test plan using a text editor.
- The tool can be used to perform automated and functional testing of the applications.
Download link: Apache JMeter downloads
2) Load runner
Loadrunner from Micro Focus is a widely used Load Testing tool. It is one of the best alternatives to JMeter that offers a user-friendly monitoring and analysis interface with colored charts and graphics.
Features:
- Supports continuous testing
- LoadRunner can effectively work on several enterprise environments
- It is possible to run multiple protocol script with Loadrunner
- Integration with various development tools like Visual Studio, Jenkins, Selenium, etc.
Download link: LoadRunner Professional
How This Test Fits the Performance Testing Family
Performance testing is an umbrella term. The variants below differ only in the shape of the load applied and the duration it is held for, which is why they are so often confused with one another.
| Test type | Load pattern | Question it answers |
|---|---|---|
| Load testing | Expected peak load, short duration | Does the system meet its targets under normal peak traffic? |
| Stress testing | Increased beyond capacity until failure | Where does it break, and does it fail gracefully? |
| Spike testing | Sudden extreme surge, then withdrawal | Does it survive and recover from a traffic shock? |
| Endurance testing | Normal load held for many hours | Does performance degrade over time? |
| Soak testing | Sustained load over an extended period | Are there memory leaks or resource exhaustion? |
| Stability testing | Varying load across conditions | Does the system stay reliable as conditions change? |
| Volume testing | Normal users, very large data volume | Does it cope as the database grows? |
Endurance and soak testing are frequently treated as synonyms. In common usage they are: both hold a sustained load for a long period. Where teams do distinguish them, endurance testing focuses on whether response times drift upward, while soak testing focuses on resource consumption such as memory, file handles, and connection pools. Running one usually gives you the evidence for both.
Response Time Testing: Key Takeaways
- Response time is the time taken for one system node to respond to the request of another.
- The Average Response time in performance testing is the average time taken for every round trip request.
- Peak response time reveals which components are potentially problematic.
- The Error Rate is a mathematical calculation which displays the percentage of problem requests.
- Three crucial response time values are: 0.1 seconds, 1.0 second and 10 seconds.
- Two of the most widely used response time testing tools are JMeter and LoadRunner.
