What is Software Testing Metric?
In software testing, Metric is a quantitative measure of the degree to which a system, system component, or process possesses a given attribute.
In other words, metrics helps estimating the progress, quality and health of a software testing effort. The ideal example to understand metrics would be a weekly mileage of a car compared to its ideal mileage recommended by the manufacturer.
"Software testing metrics - Improves the efficiency and effectiveness of a software testing process."
Software testing metrics or software test measurement is the quantitative indication of extent, capacity, dimension, amount or size of some attribute of a process or product.
Example for software test measurement: Total number of defects
In this tutorial we will learn-
- Steps to test metrics
- Why do Test Metrics?
- Types of Metrics
- Manual Test Metrics
- Test Metrics Life Cycle
- How to calculate test metric
- Test Metrics Glossary
|Sr#||Steps to test metrics||Example|
|1||Identify the key software testing processes to be measured|
|2||Use the data as baseline to define the metrics|
|3||Determination of the information to be tracked, frequency of tracking and the person responsible|
|4||Effective calculation, management and interpretation of the defined metrics|
|5||Identify the areas of improvement depending on the interpretation of defined metrics|
"We cannot improve what we cannot measure" and Test Metrics helps us to do exactly the same.
- Take decision for next phase of activities
- Evidence of the claim or prediction
- Understand the type of improvement required
- Take decision or process or technology change
Read more about its Importance of Test Metrics
- Process Metrics: It can be used to improve the process efficiency of the SDLC ( Software Development Life Cycle)
- Product Metrics: It deals with the quality of the software product
- Project Metrics: It can be used to measure the efficiency of a project team or any tools being used by the team members
Identification of correct testing metrics is very important. Few things need to be considered before identifying the test metrics
- Fix the target audience for the metric preparation
- Define the goal for metrics
- Introduce all the relevant metrics based on project needs
- Analyze the cost benefits aspect of each metrics and the project lifestyle phase in which it results into the maximum output
Manual test metrics is classified into two classes
- Base Metrics
- Calculated Metrics
Base metrics is the raw data collected by Test Analyst during the test case development and execution (# of test cases executed, # of test cases). While, calculated metrics is derived from the data gathered in base metrics. Calculated metrics is usually tracked by the test manager for test reporting purpose (% Complete, % Test Coverage).
Depending on the project or business model some of the important metrics are
- Test case execution productivity metrics
- Test case preparation productivity metrics
- Defect metrics
- Defects by priority
- Defects by severity
- Defect slippage ratio
Different stages of Metrics life cycle
Steps during each stage
To understand how to calculate the test metrics, we will see an example of percentage test case executed.
To obtain the execution status of the test cases in percentage, we use the formula.
- Percentage test cases executed= (No of test cases executed/ Total no of test cases written) X 100
Likewise, you can calculate for other parameters like test cases not executed, test cases passed, test cases failed, test cases blocked, etc.
- Rework Effort Ratio = (Actual rework efforts spent in that phase/ total actual efforts spent in that phase) X 100
- Requirement Creep= ( Total number of requirements added/No of initial requirements)X100
- Schedule Variance= ( Actual efforts – estimated efforts ) / Estimated Efforts) X 100
- Cost of finding defect in testing= ( Total effort spent on testing/ defects found in testing)
- Schedule slippage: (Actual end date – Estimated end date) / (Planned End Date – Planned Start Date) X 100