Why Android Testing?
Android is the largest operating system in the world. At the same time, Android is fragmented. there are tons of devices and Android versions that your app must be compatible with.
It doesn't matter how much time you invest in design and implementation , mistakes are inevitable, and bugs will appear.
Myths of Android Testing
Many enterprises develop android Testing strategies that are based on common misconceptions. This section examines a few popular myths and realities of Android testing.
Myth #1:All Android devices are the same... test on emulators is enough
Let's start with a simple example. An application works perfectly on emulator s but on some real devices, it crashes during execution
Application crashes during execution on real device
Emulators are not sufficient for your mobile testing. You must test your app on real devices.
Myth #2:Testing on some common devices is enough
- On different devices, your application looks different because different devices have different hardware, screen sizes, memory etc. You must test your application on different devices, OS versions, carrier networks and locations.
Myth#3:Exploratory testing just before launch is enough
- Generally in all testing, we design the test cases then execute them. But in Exploratory testing, test design and execution all will be done together.
- In exploratory testing, there's no plan and no preparation, then tester would do tests that he wants to do. Some functions will be tested repeatedly, while some functions will not be tested altogether.
Myth#4:If there some bugs in application, users will understand
- If application doesn't work and has bugs, users uninstall your app
- Quality issues are the first reason for bad review in Google Play. It affects to your reputation and you lose customer's trust.
Therefore its essential to have a proper android testing strategy in place
Android Testing Strategy
A correct android testing strategy should include the following
- Unit Test
- Integration Test
- Operational Test
- System Test
Unit Tests include sets of one or more programs which are designed to verify an atomic unit of source code, such as a method or a class.
Android platform comes pre-integrated Junit 3.0 framework. It's open source framework for automating Unit Testing. Android Testing Framework is powerful tool for developer to write the effective unit test program.
The integration of Android and JUnit framework
An addition to Unit testing is User Interface (UI) tests. These tests relate to UI components of your target application. UI tests ensure that your application return the correct UI output in response to sequence of user actions on device.
Common user UI actions on application
In Integration Testing, all unit tested modules, are combined and verified. In Android, integration tests often involve checking integration withAndroid components such as Service testing, Activity testing, Content Provider testing, etc
Types of integration test on Android
There's many testing frameworks are used to conduct integration test for Android such as Troyd, Roboelectric, Robotium.
- Operational are also called Functional Tests or Acceptation Tests. They are high level tests designed to check the completeness and correctness of application.
- In Android, FitNesse is open-source framework that makes it easy to conduct operational tests for target application.
In System Testing the system is tested as a whole and the interaction between the components, software and hardware is checked.
In Android, System Testing normally includes
- GUI tests
- Usability tests
- Performance tests
- Stress tests
In the above list, Performance Testing is given more focus. You can use tools like Traceview to conduct performance test on Android .This tool can help you debug your application and profile its performance.
Automated ANDROID TESTING
As android is fragmented, testing on multitude of devices is necessary. But this will also cost you money. Automated Android Testing can help reduce costs
Benefits of automated android testing
- Reduce time for executing test cases
- Increase productivity of your development process
- Early bug detection, save cost on software maintenance
- Quickly found and fix the bugs on implementation
- Ensure the quality of software
We will study the following 3 frameworks
- Android Testing framework
- Robotium Testing framework
- Roboelectric Testing framework
Android testing framework
One of the standard testing frameworks for Android application is Android testing framework. It is a powerful and easy-to-use testing framework that is well integrated with the Android SDK tools.
Android testing framework Architecture
- Application package is your target application which needs to be tested
- InstrumentationTestRunner is the Test Case runner that executes test case on target application. It includes:
2a) Test tools: A SDK tools for building test. They are integrated in Eclipse IDE or run as command line.
2b) MonkeyRunner: A tool that provides APIs for writing program which control an Android device or emulator outside of Android code.
- Test package are organized into test projects. This package follows naming convention. If the application under test has a package name of "com.mydomain.myapp" than Test package should be "com.mydomain.myapp.test" .Test package includes 2 objects as below:
3a) Test case classes:include test methods to executed on target application.
3b) Mock objects : includes mock data that will be used as sample input for test cases.
Android Test Case Classes
AndroidTestCase class diagram
- TestCase includes JUnit methods to run JUnit test
- TestSuite is used to run set of test cases
- InstrumentationTestSuite is a TestSuite that injects Instrumentation into InstrumentationTestCase before running them.
- InstrumentationTestRunner is the test case runner that execute test case on target application.
- AndroidTestCase extends JUnit TestCase. It contains methods for accessing resources like Activity Context.
- ApplicationTestCase verifies the Application classes in a controlled environment.
- InstrumentationTestCase verifies a particular feature or behavior of target application , for e.g., verify UI output of application.
- ActivityTestCase is base class that supports testing the Application Activities.
- ProviderTestCase is class for testing single ContentProvider.
- ServiceTestCase is used to test Service classes in testing environment. It also supports Service's life cycle.
- SingeLauchActivityTestCase is used to test single Activity with an InstrumentationTestCase.
- ActivityUnitTestCase <Activity> is used to test single isolated activity.
- ActivityInstrumentationTestCase2<Activity> extends the JUnit TestCase class. It connects you to target application with instrumentation. With this class, you can access application's GUI component and send UI event (keystroke or touch event) to the UI.
Below is an example of ActivityInstrumentationTestCase. It verifies the UI operation of Calculator application, check the correctness of the UI outputs.
ActivityInstrumentationTestCase2 testing example
Robotium testing framework
Standard Android testing framework has some limitation as below
- Unable to handle multiple activities
- Test execution performance is slow
- Test cases are complex & hard to implement
Robotiumframework is the better choice to conduct testing on Android application
Robotium is open source framework and is considered an extension of Android test framework. Using Robotium, developer can create robust automatic GUI test cases for Android applications. Moreover, developer can write functional, system and acceptance test scenarios, spanning multiple Android activities.
Advance features of Robotium
Robotium Test Case Classes
Robotium uses set of classes (com.jayway.android.robotium.solo) for testing. This class supports test cases that span over multiple activities. Solo is integrated with the ActivityInstrumentationTestCase2.
Integration Robotium and ActivityInstrumentationTestCase2
Tester can write test cases without knowledge of application design (black box testing) by using Robotium test case classes. It is an outstanding feature compare to Android test case classes.
To use Robotium in your Android test project, you need follow the steps below
Using Robotium to conduct testing on Android application
Roboelectric testing framework
Testing using Android Testing framework with device or emulator is difficult. Building and running test is slow and take much development effort. To fix this problem, there's another choice - Roboelectric testing framework.
Roboelectric framework allows you to run Android tests directly on JVM without the need for a device or an emulator.
Advance features of Roboelectric
Roboelectric Test Case Classes
Operation of Roboelectric
- As shown above, Roboelectric can perform following actions:
- Register and create a Shadow class
- Intercept the loading of Android class
- Uses javaassist to override the method bodies of Android class
- Bind Shadow object to Android class
- This allows the code under test to execute without Android environment.
Others testing framework
- Besides testing frameworks which were mentioned above, there are many other testing frameworks such as:
- Calculon, a Java DSL for Android Activity Testing which runs on Dalvik Virtual Machine
- Android Mock, a framework for mocking interfaces and classed on Dalvik VM
- Roboguide, a dependency injection in Android application
- Android Junit Report, a custom instrumentation test runner for Android that generates XML reports for integration with other tools.
Best practices in Android Testing
- Application developers should create the test cases at the same time when they are writing the code
- All test cases should be stored in version control-together with source code
- Use continuous integration and run tests every time the code is changed
- Avoid using emulators and rooted devices