Game Testing Tutorial: How to Test Mobile/Desktop Apps
Thomas Hamilton
⚡ Smart Summary
Game Testing is the quality control process applied to video games, aimed at finding defects across gameplay, graphics, audio and networking so that the build shipped to players remains stable, compliant and enjoyable.
What is Game Testing?
Game Testing is a software testing process for testing video games for quality control. The main goal of game testing is to identify and discover defects and bugs in a video game and to improve its stability and performance. Game testing is a component of game development that helps ensure the video game being deployed is bug-free.
Before the individual techniques make sense, it helps to see where testing sits inside the wider development cycle.
Game Development Lifecycle
Pre-Production: During this stage the game idea, storyboard, features, requirement analysis and documentation are produced. This phase includes the technical design document and feature specifications, game architecture, frame overlay and animation. The following items are considered:
- Music, camera (zoom in and out, replay, cinematic view), player and action attributes
- Game flow logic, rules, and conditions to attain the next level
- Object and event triggers, scores, player movement and positioning, player statistics
- Non-interactive sequences, special effects, title screens, multi-button actions
- Gamepad, movie clip, shock/vibration effects, legal texts, use of button functions, use of analog and digital mode
The three stages and the activities inside each of them are shown below.
Production: During this stage the actual coding is done. This phase includes coding and integration of the various modules.
Testing and Deployment: During this stage functional tests, regression testing, and the Alpha, Beta and Gold milestones are performed. Testing of coverage and flows, data integrity, algorithm-specific testing, path testing and incremental testing is also done using mobile game testing tools.
How Game Testing is different from other Software Testing
Testing games is a repetitive process — every new build may have bugs and must be tested thoroughly.
All game testing follows a basic structure irrespective of the size of the game and the time required for producing it.
The quality assurance professional needs to study the game rules and requirements, and to understand the overall game component architecture, the file architecture, the flow, the file structures and the dependencies associated with the game. With every new prototype of the game, the test documents need to be revisited to reflect any change in the specifications, new game testing test cases and new configuration support. A video game tester should also ensure that no new issues were introduced.
Game tester jobs involve:
- Classify the requirements based on the intended purpose and target audience.
- Identify the user and system requirements and classify them into functional, non-functional and domain requirements.
- Identify testable items, non-testable items, targets and measures for functional and non-functional requirements.
- Check whether the functional requirements are complete, consistent and comprehensible.
- Identify customizable requirements and conflicting requirements.
- Identify interdependent requirements, which is one of the core game testing jobs.
- Prioritize the requirements based on uniqueness, complexity and criticality.
- Identify the theme of the game, characters, animation, AI, cinematics, camera view and gameplay.
If you want to become a video game tester, here is a free tutorial worth checking out: How to Become a Video Game Tester.
Types of Game Testing
Below are the popular game testing techniques. Each one targets a different class of defect, and a release-ready title normally passes through all of them rather than choosing between them.
1) Functional Testing
Functionality QA testers look for generic problems within the game or its user interface and graphics, such as game mechanic issues, stability issues and game asset integrity. User interface testing ensures the user-friendliness of the game. This is the same discipline as functional testing in any other application, applied to gameplay rules instead of business rules.
Example: Checking colors and backgrounds, menu structure, screen orientation and screen resolution, font size, alignment errors, usability, system navigation such as loading time, timeout and display, sorting, confirmation messages, sequences, animation and audio elements of the game, instructions and dialogue messages. It also covers user interactions, user interfaces, transaction testing, calibration and accuracy testing of mobile phone cameras, screen resolutions, mobile responsive design testing and audio quality testing.
2) Compatibility Testing
Checking whether the game is compatible across different devices and on different configurations of hardware and software. Broad device coverage matters more for games than for most applications, because GPU drivers and screen aspect ratios change how the title renders. See compatibility testing for the general technique.
Example: Install and uninstall the game on all supported consoles, desktops and mobiles.
3) Performance Testing
The overall performance of the game is checked. Performance tuning is performed to optimize game speed. The wider practice is covered in performance testing, and the mobile-specific angle in mobile app performance testing.
Important parameters checked during performance testing:
- Response time on client and servers, transaction completion times, peak load performance, longevity, network coverage, memory leakage, low memory, low battery, time taken to download the application, simultaneous (multiple user) access to the application server, speed, throughput, reliability and scalability.
- Battery consumption and graphics performance: Measure the battery consumption of the mobile game. Battery consumption must be optimum over long hours, and game responses should be satisfactory under varying heavy loads across different devices.
- Processor and memory constraints: Performance counters are used to measure the CPU and memory consumption of the application.
- Network connectivity: Measures the response time of the mobile game on different network types (Wi-Fi, 3G, 4G, 5G). It gives an overall insight into how well the game will perform on unreliable networks, and also checks connectivity between mobile devices, data centers and the cloud. Peak times, jittery connections, duplication of data, packet loss and fragmentation of data are all monitored.
- Testing mobile game performance, especially for MMO titles.
4) Conformance / Compliance Testing
This covers marketplace guideline compliance (for example, Apple App Store policies) and enterprise policy compliance (for example, prohibited content). Compliance may also refer to regulatory bodies such as PEGI and ESRB. The game targets a particular content rating, and if there is objectionable content inappropriate for the desired rating, it is identified and reported. Even a single violation in a submission for license approval may have the game rejected, incurring additional costs in further testing and resubmission.
Example: If the game is to be published in European countries, test for PAL conversion; if the game is produced for North America, test for NTSC conversions.
5) Localization Testing
Localization testing becomes essential when a game is targeted at global markets. Game titles, content and texts need to be translated and tested with devices in multiple languages. These tests can be performed quickly with the help of cloud-based device access and test automation.
Example: Localization needs specific to the MENA region (Middle East / North Africa), Arabic localization (right-to-left text support, bi-directional displays), pseudo-localization testing, double-byte characters for East Asian languages, local time and date, currency, address formats and other local requirements.
6) Soak Testing
This game automation testing technique involves leaving the game running for a prolonged period in various modes of operation — for example, idling, paused, or sitting at the title screen. Soaking can identify memory leaks or rounding errors.
Example: The game has begun and the character is made to stand idle for 24 hours. This technique is used to detect crashes brought on by memory leaks and other faults in the game engine.
7) Recovery Testing
In software, recovery testing checks how well the application recovers from crashes, hardware failures and other similar failures. The application is forced to fail, and it is then observed how it recovers from the failure conditions and the environment.
Example: While a gaming application is running, suddenly restart the gaming console and validate the data integrity.
8) Security Testing
Security testing is done to check how safely the software works when exposed to external threats. It covers data protection from external threats, uncontrolled system access restrictions, data breaches, operating system flaws, communication system flaws and weak encryption algorithms.
Example: Changing a URL from /login to /play on a gaming site should not allow direct access to the games.
9) Other Games Testing
Testing of real or virtual characters. In multiplayer video games, connectivity to the server and synchronization of game status are two critical areas to be tested.
Example: Multiplayer 3D racing games.
Testing of new features such as game status updates, friend invitations and premium gift sharing. This ensures a rich gaming experience for the user.
Example: Facebook, blogs.
Sound Testing
Testing whether there is an error in loading the files, listening to sound files for errors or distortions, and using a CC profiler to analyze the color commentary.
Database and Game Statistics
Database verification using debugging to investigate whether the game is using the data correctly. Ensure that the data is loaded in the right place and shows the correct information.
White-box Testing
White-box testing for games focuses on the architectural, integration and system aspects of the mobile game.
- Code Inspection: Source code is reviewed, and program logic, common programming errors and compliance with coding standards are analyzed.
- Focus Testing: Chunks of code are fed to the isolated modules, and the output is analyzed.
- Data Analysis: Data usage, interpretation and manipulation are analyzed and validated for the different modules.
- Path and Flow Testing: The correct sequence of objects is executed.
- Algorithm-specific Testing: Testing a particular game scenario or feature by setting data variables and data values in the code and executing it in the runtime environment.
- Artificial Intelligence Analysis: The run statistics of the programmable moves and plays of the AI component are generated. The result is validated to check whether all the programmable moves are used. Example: a side grip on the snowboard, and plays such as a combination punch or kick in multidirectional action.
Assistive gaming using adaptive technology
Assistive gaming is also known as accessibility gaming. Features are designed using adaptive technology for individuals with various disabilities such as low vision, blurred vision, blindness, inability to distinguish colors, and speech, hearing, cognitive, motor and mobility impairments. The verification approach follows the same principles as accessibility testing in mainstream software.
Cardinal Direction (CD) and Tower of London (TOL) are two popular games that have been modified for visually challenged users. In these games, visual stimuli are replaced with audio inputs.
A video game tester should note the following while testing such a game:
- The colors should flash in a pattern, and tones should play for each color.
- Each color should be accompanied by an audible tone.
- Visual data needs to be described in words so that visually impaired players do not face issues receiving it through screen readers.
- The player should hear sounds in the game in three dimensions and must be able to navigate the world using the touchscreen, 3D audio and spatialized sound.
Game Metrics That a Tester should Know
Test results alone do not tell a studio whether a release succeeded. The following metrics are the numbers a tester is expected to read alongside the defect report.
DAU/MAU (daily active users / monthly active users): The ratio of active users who play each day over the number of monthly active users. It is also commonly referred to as the stickiness factor.
Session: Every time a user opens the app it counts as a session. Here the focus is on the average number of sessions per DAU.
Download Rank: The rank of a game in a particular app store (iOS, Android Play) by monthly game downloads.
Retention: A very important metric for an Android game tester on a free-to-play game. To calculate retention, separate the users into cohorts based on the day the application was downloaded.
Performance metrics: These track the performance of online or persistent games — the frame rate at which a game executes on a client hardware platform, or in the case of a game server, its stability. Performance metrics can be used to monitor changing features and updates.
Key Risks in Game Testing
The risks below most often turn a technically working build into a commercially disappointing one.
- The game does not create a compelling experience for the targeted audience.
- The game does not have a player-centric design.
- The fun factor and addictive gameplay are missing.
- The game is not unique, competitive or fast paced.
- The game fails because of technical issues, broken features, critical bugs, bad music and sound, or poor video.
- Game development cost goes over budget.
- The aesthetic design and the gameplay are not kept simple.
