What is Software Test Estimation?
Test Estimation is a management activity which approximates how long a Task would take to complete. Estimating effort for the test is one of the major and important tasks in Test Management.
Why Test Estimation?
Two questions you can expect from your clients when discussing potential test engagements are
For small projects, these questions are relatively easy to answer. But for the big project like Testing Guru99 Bank website, you must think hard to answer those questions.
What to Estimate?
- Resources: Resources are required to carry out any project tasks. They can be people, equipment, facilities, funding, or anything else capable of definition required for the completion of a project activity.
- Times : Time is the most valuable resource in a project. Every project has a deadline to delivery.
- Human Skills : Human skills mean the knowledge and the experience of the Team members. They affect to your estimation. For example, a team, whose members have low testing skills, will take more time to finish the project than the one which has high testing skills.
- Cost: Cost is the project budget. Generally speaking, it means how much money it takes to finish the project.
How to estimate?
List of Software Test Estimation Techniques
- Work Breakdown Structure
- 3-Point Software Testing Estimation Technique
- Wideband Delphi technique
- Function Point/Testing Point Analysis
- Use – Case Point Method
- Percentage distribution
- Ad-hoc method
Following is the 4 Step process to arrive at an estimate
You will learn how to combine these techniques to find the estimate for Guru99 Bank case study.
Step1) Divide the whole project task into subtasks
Task is a piece of work that has been given to someone. To do this, you can use the Work Breakdown Structure technique.
In this technique, a complex project is divided into modules. The modules are divided into sub-modules. Each sub-module is further divided into functionality. It means divide the whole project task into the smallest tasks.
Use the Work Break Down structure to break out the Guru99 Bank project into 5 smaller tasks-
After that, you can break out each task to the subtask. The purpose of this activity is create task as detailed as possible.
| Task | Sub task |
|---|---|
| Analyze software requirement specification | Investigate the soft requirement specs |
| Interview with the developer & other stakeholders to know more about the website | |
| Create the Test Specification | Design test scenarios |
| Create test cases | |
| Review and revise test cases | |
| Execute the test cases | Build up the test environment |
| Execute the test cases | |
| Review test execution results | |
| Report the defects | |
| Create the Defect reports | |
| Report the defects |
Step 2) Allocate each task to team member
In this step, each task is assigned to the appropriate member in the project team. You can assigned task as follows
| Task | Members |
|---|---|
| Analyze software requirement specification | All the members |
| Create the test specification | Tester/Test Analyst |
| Build up the test environment | Test Administrator |
| Execute the test cases | Tester, Test Administrator |
| Report defects | Tester |
Step 3) Effort Estimation For Tasks
There are 2 techniques which you can apply to estimate the effort for tasks
- Functional Point Method
- Three Point Estimation
Method 1) Function Point Method
In this method, the Test Manager estimates Size, Duration, and Cost for the tasks
Step A) Estimate size for the task
In Step 1, you already have broken the whole project task into small task by using WBS method. Now you estimate the size of those tasks. Let’s practice with a particular task “Create the test specification”
The size of this task depends on the functional size of the system under test. The functional size reflects the amount of functionality that is relevant to the user. The more number of functionality, the more complex system is.
Prior to start actual estimating tasks effort, functional points are divided into three groups like Complex, Medium Simple as following:
Based on the complex of software functions, the Test Manger has to give enough weightage to each functional point. For example
| Group | Weightage |
|---|---|
| Complex | 5 |
| Medium | 3 |
| Simple | 1 |
Let’s take a simple example exercise to get clearer:
Take a look the software specification of website Guru99 Bank over here, the software engineer have already described the software modules in detail, can you determine the complexity of website’s features by giving the weightage for each modules?
More complex the function point, more is the effort to test it is. The website is divided into 12 function points, you can determine the complexity of each function points as follows-
| No. | Module Name | Applicable Roles | Description | Weightage |
|---|---|---|---|---|
| 1. | Balance Enquiry | Manager Customer | Customer:A customer can have multiple bank accounts. He can view balance of his accounts only Manager: A manager can view balance of all the customers who come under his supervision | 3 |
| 2. | Fund Transfer | Manager Customer | Customer:A customer can have transfer funds from his “own” account to any destination account. Manager: A manager can transfer funds from any source bank account to destination account | 5 |
| 3. | Mini Statement | Manager Customer | A Mini statement will show last 5 transactions of an account Customer:A customer can see mini-statement of only his “own” accounts Manager: A manager can see mini-statement of any account | 3 |
| 4. | Customized Statement | Manager Customer | A customized statement allows you to filter and display transactions in an account based on date, transaction value Customer:A customer can see Customized- statement of only his “own” accounts Manager: A manager can see Customized -statement of any account | 5 |
| 5. | Change Password | Manager Customer | Customer:A customer can change password of only his account. Manager: A manager can change password of only his account. He cannot change passwords of his customers | 1 |
| 6. | New Customer | Manager | Manager: A manager can add a new customer. Manager: A manager can edit details like address, email, telephone of a customer. | 3 |
| 7. | New Account | Manager | Currently system provides 2 types of accounts
He can have multiple current accounts for different companies he owns. Or he can have a multiple current and saving accounts. Manager: A manager can add a new account for an existing customer. | 5 |
| 8. | Edit Account | Manager | Manager: A manager can add an edit account details for an existing account | 1 |
| 9. | Delete Account | Manager | Manager: A manager can add a delete an account for a customer. | 1 |
| 10. | Delete Customer | Manager | A customer can be deleted only if he/she has no active current or saving accounts Manager: A manager can delete a customer. | 1 |
| 11. | Deposit | Manager | Manager: A manager can deposit money into any account. Usually done when cash is deposited at a bank branch. | 3 |
| 12. | Withdrawal | Manager | Manager: A manager can withdraw money from any account. Usually done when cash is withdrawn at a bank branch. | 3 |
STEP B) Estimate duration for the task
After classifying the complexity of the function points, you have to estimate the duration to test them. Duration means how much time needs to finish the task.
- Total Effort: The effort to completely test all the functions of the website
- Total Function Points: Total modules of the website
- Estimate defined per Function Points: The average effort to complete one function points. This value depends on the productivity of the member who will take in charge this task.
Suppose your project team has estimated defined per Function Points of 5 hours/points. You can estimate the total effort to test all the features of website Guru99 Bank as follows:
| Weightage | # of Function Points | Total | |
|---|---|---|---|
| Complex | 5 | 3 | 15 |
| Medium | 3 | 5 | 15 |
| Simple | 1 | 4 | 4 |
| Function Total Points | 34 | ||
| Estimate define per point | 5 | ||
| Total Estimated Effort (Person Hours) | 170 | ||
So the total effort to complete the task “Create the test specification” of Guru99 Bank is around 170 man-hours
Once you understand the effort that is required, you can assign resources to determine how long the task will take (duration), and then you can estimate labor and non-labor costs.
Above example also shows the importance of the member in your team. If you have talented and experienced members, you can finish the assigned task in the small time, and your project will finish at the deadline or sooner.
STEP C) Estimate the cost for the tasks
This step helps you to answer the last question of customer “How much does it cost?”
Suppose, on average your team salary is $5 per hour. The time required for “Create Test Specs” task is 170 hours. Accordingly, the cost for the task is 5*170= $850. Now you can calculate budget for other activities in WBS and arrive at overall budget for the project.
As a project manager, you have to decide how to get the most return for your company’s investment. The more accurate your estimate of project cost is, the better able you will be to manage your project’s budget.
METHOD 2) Three Point Estimation
Three-Point estimation is one of the techniques that could be used to estimate a task. The simplicity of the Three-point estimation makes it a very useful tool for a Project Manager that who wants to estimate.
In three-point estimation, three values are produced initially for every task based on prior experience or best-guesses as follows
When estimating a task, the Test Manager needs to provide three values, as specified above. The three values identified, estimate what happens in an optimal state, what is the most likely, or what we think it would be the worst case scenario.
Let’s see how to use the above three values in the following example
For the task “Create the test specification”, can you estimate the test effort? Remember that you have to cover all the modules of the Guru99 Bank website as done in Function Point Method
You can estimate as following
- The best case to complete this task is 120 man-hours (around 15 days). In this case, you have a talented team, they can finish the task in smallest time.
- The most likely case to complete this task is 170 man-hours (around 21 days). This is a normal case, you have enough resource and ability to complete the task
- The worst case to complete this task is 200 man-hours (around 25 days). You need to perform much more work because your team members are not experienced.
Now, assign the value to each parameter as below
The effort to complete the task can be calculated using double-triangular distribution formula as follows-
In the above formula, parameter E is known as Weighted Average. It is the estimation of the task “Create the test specification”.
But your boss may ask you
In the above estimation, you just determine a possible and not a certain value, we must know about the probability that the estimation is correct. You can use the other formula:
In above formula, the SD mean Standard Deviation, this value could give you the information about the probability that the estimation is correct.
Now you can conclude the estimation for the task “Create the test specification”
To complete the task “Create the test specification” of Guru99 Bank website, you need 166.6 ± 13.33 Man-hour (153.33 to 179.99 man-hour)
Step 4) Validate the estimation
Once you create an aggregate estimate for all the tasks mentioned in the WBS, you need to forward it to the management board, who will review and approve it.
The member of management board could comprise of the CEO, Project Manager & other stakeholders.
The management board will review and discuss your estimation plan with you. You may explain them your estimation logically and reasonably so that they can approve your estimation plan.
Test estimation best practices
This topic introduces general tips on how to estimate Testing accuracy.
- Add some buffer time: Many unpredictable things may happen to your project, such as a talented team member quits his job suddenly, the testing takes more time than estimated to complete… etc. That why you need include some buffer in your estimation. Having a buffer in the estimation enables to cope for any delays that may occur.
- Account Resource planning in estimation: What should you do if some members in your team take long leaves? It may delay the project.Resource planning in estimation plays a key role. The availability of resources will help to make sure that the estimations are realistic. Here you have to consider the leaves for your team member, generally long leaves.
- Use the past experience as reference: Experiences from past projects play a vital role while preparing the time estimates. Because some project may be some similarity, you can reuse the past estimation. For example, if you use to do a project like testing a website, you can learn from that experience, try to avoid all the difficulties or issues that were faced in past projects.
- Stick to your estimation: Estimation is just estimate because it may go wrong.In early stages of the project, you should frequently re-check the test estimations and make modification if needed. We should not extend the estimation after we fix it, unless there are major changes in requirement, or you have to negotiate with customer about the re-estimation
Other Techniques
Wideband Delphi Technique, Use – Case Point Method, Percentage distribution, Ad-hoc method are other estimation techniques
- Let's do an exercise -for the Flight Reservation Application prepare a Work Breakdown Structure of the
- various testing tasks like - Check Login Functionality, Check New Order Functionality,Check Fax Functionality, and other similar functionality and Estimate the effort required to test these functionalities
- For example login functionality can be tested in 2 hours. Likewise prepare a list of all the tasks and corresponding effort. Pause the training tutorial and complete the exercise. I hope you made an educated guess of the effort required
- This is Bottom-Up Strategy for Test Estimation. The technique is called bottom- up since based on the tasks which is at the lowest level of the work breakdown hierarchy you estimate the duration , dependencies and resources.
- In bottom-Up strategy , estimates are not taken by a single person but all stakeholders, individual contributors , experts and experienced staff members collectively. The idea is to draw on the collaborative wisdom of the team members to arrive at accurate test estimates
- Now since you have considerable experience on the flight reservation system. Use this experience to estimate the effort required for full Functional Testing of the website. - http://newtours.demoaut.com/
- This site's functionally is identical to the Flight Reservation Application , just that it is web based. Pause the tutorial and do the exercise now
- I hope based on your experience you made a good estimate on the effort required to test the website
- This is the Top - Down Approach to estimation which is based on experience.
- Another technique is to classify project based on their size and complexity and then seeing how long a project of a particular size and complexity have taken in past.
- Another approach is determining Average Effort Per Test Case in past for similar projects and then using estimated test cases of the current project and arriving at total effort
- More sophisticated estimation models involve complex mathematical models. In practice , majority of the projects use top-down approach for estimation.
- Test estimates can be affected by many factors like timing pressures , people factors , geographic distribution of the test team and so on
