Test Environment in Software Testing

What is a Test Environment?

A testing environment is a setup of software and hardware for the testing teams to execute test cases. In other words, it supports test execution with hardware, software and network configured.

Test bed or test environment is configured as per the need of the Application Under Test. On a few occasion, test bed could be the combination of the test environment and the test data it operates.

Setting up a right test environment ensures software testing success. Any flaws in this process may lead to extra cost and time to the client.

Test Environment Setup: Key Areas

For the test environment, a key area to set up includes

System and applications
Test data
Database server
Front-end running environment
Client operating system
Browser
Hardware includes Server Operating system
Network
Documentation required like reference documents/configuration guides/installation guides/ user manuals

Process of Software Test environment setup

Tests are limited to what can be tested and what not should be tested.

Following people are involved in test environment setup

System Admins,
Developers
Testers
Sometimes users or techies with an affinity for testing.

The test environment requires setting up of various number of distinct areas like,

Step 1) Setup of Test Server

Every test may not be executed on a local machine. It may need establishing a test server, which can support applications.

For example, Fedora set up for PHP, Java-based applications with or without mail servers, cron set up, Java-based applications, etc.

Step 2) Network

Network set up as per the test requirement. It includes,

Internet setup
LAN Wifi setup
Private network setup

It ensures that the congestion that occurs during testing doesn’t affect other members. (Developers, designers, content writers, etc.)

Step 3) Test PC setup

For web testing, you may need to set up different browsers for different testers. For desktop applications, you need various types of OS for different testers PCs.

For example, windows phone app testing may require

Visual Studio installation
Windows phone emulator
Alternatively, assigning a windows phone to the tester.

Step 4) Bug Reporting

Bug reporting tools should be provided to testers.

Step 5) Creating Test Data for the Test Environment

Many companies use a separate test environment to test the software product. The common approach used is to copy production data to test. This helps the tester, to detect the same issues as a live production server, without corrupting the production data.

The approach for copying production data to test data includes,

Set up production jobs to copy the data to a common test environment
All PII (Personally Identifiable Information) is modified along with other sensitive data. The PII is replaced with logically correct, but non-personal data.
Remove data that is irrelevant to your test.

Testers or developers can copy this to their individual test environment. They can modify it as per their requirement.

Privacy is the main issue in copy production data. To overcome privacy issues you should look into obfuscated and anonymized test data.

For Anonymization of data two approaches can be used,

BlackList: In this approach, all the data fields are left unchanged. Except those fields specified by the users.
WhiteList: By default, this approach, anonymizes all data fields. Except for a list of fields which are allowed to be copied. A whitelisted field implies that it is okay to copy the data as it is and anonymization is not required.

Also, if you are using production data, you need to be smart about how to source data. Querying the database using SQL script is an effective approach.

Test Environment Management

Test Environment Management deals with the maintenance and upkeep of the test bed.

List of activities by the Test environment management function include,

Maintenance of a central repository with all the updated version of test environments.
Test environment management as per the test team demands.
As per the new requirements creating new environments
Monitoring of the environments
Updating/deleting outdated test-environments
Investigation of issues on the environment
Co-ordination till an issue resolution.

Test Environment Checklist

Hardware

Check whether required equipment for testing is available? If this is not the case, analyze the supply time!
Check whether peripheral equipment is available? Such as scanners, special printers, handhelds, etc.

Software / connections

Are the needed applications specified? An application such as excel, word, drawings, etc.
For the new software does the test environment exist for the organization? Has the organization experience with use and maintenance of the software?

Environmental data

Check whether the standard test data sets are available? With the regression test set, consider the Defect administration to collect test data.
Do agreements with the test data owners about the test data exist? Consider functional maintenance.

Maintenance tools/processes

Check whether a single point of contact exists for test environment maintenance? If no, prepare a list of all possible members involved in keeping the test environment running. It should include their contact information as well.
Does the agreement reached about the readiness and quality of the test environment? For instance, acceptance criteria, maintenance requirements, etc. Also, check whether other/extra quality attributes for environments are there in agreement.
Do all members involved in the maintenance process are known?

Besides these, there are a few more questions to answer before setting up the test environment.

Whether to develop an internal Test Environment or to outsource?
Whether to follow an internal company standard or follow any External (IEE, ISO, etc.)?
How long the test environment is required?
Differences between the test and production systems and their impact on test validity must be determined.
Can you re-use an existing setup for other projects in the company?

Challenges in setting up Test Environment Management

Proper planning on resource usageIneffective planning for resource usage can affect the actual output. Also, it may lead to conflict between teams.
Remote environmentIt is possible that a Test environment is located geographically apart. In such a case, the testing team has to rely on the support team for various test assets. (Software, hardware, and other issues).
Elaborate setup timeSometimes test set up gets too elaborated in cases of Integration Testing.
Shared usage by teamsIf the testing environment is used by development & testing team simultaneously, test results will be corrupted.
Complex test configurationCertain test requires complex test environment configuration. It may pose a challenge to the test team.

Best practices for setting up a Test Environment Management

Understand the test requirements thoroughly and educate the test team members.
Connectivity should be checked before the initiation of the testing
Check for the required hardware and software, licenses
Browsers and versions
Planning out the Scheduled use of the test environment.
Automation tools and their configurations.

What is Test Bed in Software Testing?

A Test Bed in Software Testing is a software development environment. It allows developers to test their modules without affecting the live production servers. Test bed is not only confined to developers but also used by testers. It is referred to as a test environment for rigorous and transparent testing of new technologies.

Summary

A testing environment is a setup of software and hardware on which the test team will conduct the testing
For the test environment, a key area to set up includes
System and applications
Test data
Database server
Front-end running environment, etc.
Few challenges while setting up a test environment include,
Remote environment
Combined usage between teams
Elaborate setup time
Ineffective planning for resource usage for integration
Complex test configuration