Test Environment in Software Testing

What is a Test Environment?

A testing environment is a setup of software and hardware for the testing teams to execute test cases. In other words, it supports test execution with hardware, software and network configured.

Test bed or test environment is configured as per the need of the Application Under Test. On a few occasion, test bed could be the combination of the test environment and the test data it operates.

Setting up a right test environment ensures software testing success. Any flaws in this process may lead to extra cost and time to the client.

Test Environment Setup: Key Areas

For the test environment, a key area to set up includes

• System and applications
• Test data
• Database server
• Front-end running environment
• Client operating system
• Browser
• Hardware includes Server Operating system
• Network
• Documentation required like reference documents/configuration guides/installation guides/ user manuals

Process of Software Test environment setup

Tests are limited to what can be tested and what not should be tested.

Following people are involved in test environment setup

• System Admins,
• Developers
• Testers
• Sometimes users or techies with an affinity for testing.

The test environment requires setting up of various number of distinct areas like,

Step 1) Setup of Test Server

Every test may not be executed on a local machine. It may need establishing a test server, which can support applications.

For example, Fedora set up for PHP, Java-based applications with or without mail servers, cron set up, Java-based applications, etc.

Step 2) Network

Network set up as per the test requirement. It includes,

• Internet setup
• LAN Wifi setup
• Private network setup

It ensures that the congestion that occurs during testing doesn’t affect other members. (Developers, designers, content writers, etc.)

Step 3) Test PC setup

For web testing, you may need to set up different browsers for different testers. For desktop applications, you need various types of OS for different testers PCs.

For example, windows phone app testing may require

• Visual Studio installation
• Windows phone emulator
• Alternatively, assigning a windows phone to the tester.

Step 4) Bug Reporting

Bug reporting tools should be provided to testers.

Step 5) Creating Test Data for the Test Environment

Many companies use a separate test environment to test the software product. The common approach used is to copy production data to test. This helps the tester, to detect the same issues as a live production server, without corrupting the production data.

The approach for copying production data to test data includes,

• Set up production jobs to copy the data to a common test environment
• All PII (Personally Identifiable Information) is modified along with other sensitive data. The PII is replaced with logically correct, but non-personal data.
• Remove data that is irrelevant to your test.

Testers or developers can copy this to their individual test environment. They can modify it as per their requirement.

Privacy is the main issue in copy production data. To overcome privacy issues you should look into obfuscated and anonymized test data.

For Anonymization of data two approaches can be used,

• BlackList: In this approach, all the data fields are left unchanged. Except those fields specified by the users.
• WhiteList: By default, this approach, anonymizes all data fields. Except for a list of fields which are allowed to be copied. A whitelisted field implies that it is okay to copy the data as it is and anonymization is not required.

Also, if you are using production data, you need to be smart about how to source data. Querying the database using SQL script is an effective approach.

Test Environment Management

Test Environment Management deals with the maintenance and upkeep of the test bed.

List of activities by the Test environment management function include,

1. Maintenance of a central repository with all the updated version of test environments.
2. Test environment management as per the test team demands.
3. As per the new requirements creating new environments
4. Monitoring of the environments
5. Updating/deleting outdated test-environments
6. Investigation of issues on the environment
7. Co-ordination till an issue resolution.

Test Environment Checklist

Hardware

• Check whether required equipment for testing is available? If this is not the case, analyze the supply time!
• Check whether peripheral equipment is available? Such as scanners, special printers, handhelds, etc.

Software / connections

• Are the needed applications specified? An application such as excel, word, drawings, etc.
• For the new software does the test environment exist for the organization? Has the organization experience with use and maintenance of the software?

Environmental data

• Check whether the standard test data sets are available? With the regression test set, consider the Defect administration to collect test data.
• Do agreements with the test data owners about the test data exist? Consider functional maintenance.

Maintenance tools/processes

• Check whether a single point of contact exists for test environment maintenance? If no, prepare a list of all possible members involved in keeping the test environment running. It should include their contact information as well.
• Does the agreement reached about the readiness and quality of the test environment? For instance, acceptance criteria, maintenance requirements, etc. Also, check whether other/extra quality attributes for environments are there in agreement.
• Do all members involved in the maintenance process are known?

Besides these, there are a few more questions to answer before setting up the test environment.

• Whether to develop an internal Test Environment or to outsource?
• Whether to follow an internal company standard or follow any External (IEE, ISO, etc.)?
• How long the test environment is required?
• Differences between the test and production systems and their impact on test validity must be determined.
• Can you re-use an existing setup for other projects in the company?

Challenges in setting up Test Environment Management

1. Proper planning on resource usageIneffective planning for resource usage can affect the actual output. Also, it may lead to conflict between teams.
2. Remote environmentIt is possible that a Test environment is located geographically apart. In such a case, the testing team has to rely on the support team for various test assets. (Software, hardware, and other issues).
3. Elaborate setup timeSometimes test set up gets too elaborated in cases of Integration Testing.
4. Shared usage by teamsIf the testing environment is used by development & testing team simultaneously, test results will be corrupted.
5. Complex test configurationCertain test requires complex test environment configuration. It may pose a challenge to the test team.

Best practices for setting up a Test Environment Management

1. Understand the test requirements thoroughly and educate the test team members.
2. Connectivity should be checked before the initiation of the testing
3. Check for the required hardware and software, licenses
4. Browsers and versions
5. Planning out the Scheduled use of the test environment.
6. Automation tools and their configurations.

Summary

• A testing environment is a setup of software and hardware on which the test team will conduct the testing
• For the test environment, a key area to set up includes
• System and applications
• Test data
• Database server
• Front-end running environment, etc.
• Few challenges while setting up a test environment include,
• Remote environment
• Combined usage between teams
• Elaborate setup time
• Ineffective planning for resource usage for integration
• Complex test configuration