Setting up a right test environment ensures software testing success. Any flaws in this process may lead to extra cost and time to the client.
In this tutorial, you will learn-
What is a Test bed?
In general, a test bed is a software development environment. It allows the developers to test their modules without affecting the live production servers. The test bed is not confined to developers only but also used by testers. It is referred as test environment as well.
What is a Test Enviornment?
A testing environment is a setup of software and hardware for the testing teams to execute test cases. In other words, it supports test execution with hardware, software and network configured.
Test bed or test environment is configured as per the need of the Application Under Test. At few occasion, test bed could be the combination of the test environment and the test data it operates.
Key areas to set up in Test Environment
For test environment, key area to set up includes
- System and applications
- Test data
- Database server
- Front end running environment
- Client operating system
- Browser
- Hardware includes Server Operating system
- Network
- Documentation required like reference documents/configuration guides/installation guides/ user manuals
Process of Software Test environment setup
Tests are limited to what can be tested and what not should be tested.
Following people are involved in test environment setup
- System Admins,
- Developers
- Testers
- Sometimes users or techies with an affinity for testing.
The test environment requires setting up of various number of distinct areas like,
Setup of Test Server
Every test may can not be executed on local machine. It may need establishing a test server, which can support applications.
For example: Fedora set up for PHP, Java based applications with or without mail servers, cron set up, Java based applications, etc.
Network
Network set up as per the test requirement. It includes,
- Internet setup
- LAN Wifi setup
- Private network setup
It ensures that the congestion that occurs during testing doesn't affect other members. (Developers, designers, content writers, etc.)
Test PC setup
For web testing, you may need to setup different browsers for different testers. For desktop applications, you need various types of OS for different testers PCs.
For example, windows phone app testing may require
- Visual Studio installation
- Windows phone emulator
- Alternatively, assigning a windows phone to the tester.
Bug Reporting
Bug reporting tools should be provided to testers.
Creating Test Data for the Test Environment
Many companies use a separate test environment to test the software product. The common approach used is to copy production data to test. This helps the tester, to detect same issues as live production server, without corrupting the production data.
The approach for copying production data to test data includes,
- Set up production jobs to copy the data to a common test environment
- All PII (Personally Identifiable Information) is modified along with other sensitive data. The PII is replaced with logically correct, but non-personal data.
- Remove data that is irrelevant to your test.
Testers or developers can copy this to their individual test environment. They can modify it as per their requirement.
Privacy is the main issue in copy production data. To overcome privacy issues you should look into obfuscated and anonymized test data.
For Anonymization of data two approaches can be used,
- BlackList: In this approach, all the data fields are left unchanged. Except those fields specified by the users.
- WhiteList: By default, this approach, anonymizes all data fields. Except a list of fields which are allowed to be copied. A whitelisted field implies that it is okay to copy the data as it is and anonymization is not required.
Also, if you are using production data, you need to be smart about how to source data. Querying the database using SQL script is an effective approach.
Test Environment Management
Test Environment Management deals with maintainence and upkeep of the test bed.
List of activities by the Test environment management function include,
- Maintenance of a central repository with all the updated version of test environments.
- Test environment management as per the test team demands.
- As per the new requirements creating new environments
- Monitoring of the environments
- Updating/deleting outdated test-environments
- Investigation of issues on the environment
- Co-ordination till an issue resolution.
Test Environment Checklist
| 1 | Check whether required equipment for testing is available? | If this is not the case, analyze the supply time! |
| Check whether peripheral equipment is available? | Such as scanners, special printers, handhelds, etc. | |
| 2 | Are the needed applications specified? | An application such as excel, word, drawings, etc. |
| For the new software does the test environment exist for the organization? | Has the organization experience with use and maintenance of the software? | |
| 3 | Check whether the standard test data sets are available? | With regression test set, consider the defect administration to collect test data. |
| Do agreements with the test data owners about the test data exist? | Consider functional maintenance. | |
| 4 | Check whether a single point of contact exist for test environment maintenance? | If no, prepare a list of all possible members involved in keeping the test environment running. It should include their contact information as well. |
| Does agreement reached about the readiness and quality of the test environment? | For instance, acceptance criteria, maintenance requirements, etc. Also, check whether other / extra quality attributes for environments are there in agreement. | |
| Do all members involved in the maintenance process are known? | ||
Beside these, there are few more questions to answer before setting up the test environment.
- Whether to develop an internal Test Environment or to outsource?
- Whether to follow an internal company standard or follow any External (IEE, ISO, etc.)?
- How long test environment be required?
- Differences between the test and production systems and their impact on test validity must be determined.
- Can you re-use any existing setup for other projects in the company?
Challenges in setting up Test Environment Managment
- Proper planning on resource usage
Ineffective planning for resource usage can affect the actual output. Also, it may lead to conflict between teams.
- Remote environment
It is possible that Test environment is located geographically apart. In such case the testing team has to rely on the support team for various test assets. (Software, hardware, and other issues).
- Elaborate setup time
Sometimes test set up gets too elaborated in cases of integration testing.
- Shared usage by teams
If the testing environment is used by development & testing team simultaneously, test results will be corrupted.
- Complex test configuration
Certain test requires complex test environment configuration. It may pose a challenge to the test team.
Best practices for setting up a Test Environment Management
- Understand the test requirements thoroughly and educate the test team members.
- Connectivity should be checked before the initiation of the testing
- Check for the required hardware and software, licenses
- Browsers and versions
- Planning out the Scheduled use of the test environment.
- Automation tools and their configurations.
Summary:
- A testing environment is a setup of software and hardware on which the test team will conduct the testing
- For test environment, key area to set up includes
- System and applications
- Test data
- Database server
- Front end running environment, etc.
- Few challenges while setting up test environment include,
- Remote environment
- Combined usage between teams
- Elaborate setup time
- Ineffective planning for resource usage for integration
- Complex test configuration
