Advanced UI testing
with Espresso…
1. Use Test Orchestrator
Test Orchestrator allows running every test within its own invocation of Instrumentation, which means that every single test will start with the completely clear state and this is a key to create valuable and stable tests.
Tests will not affect each other, which might be really helpful if one of the tests cases changes app state significantly.
android {
defaultConfig {
...
testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner" testInstrumentationRunnerArguments clearPackageData: 'true'
}
testOptions {
execution 'ANDROIDX_TEST_ORCHESTRATOR'
}
}
dependencies {
androidTestImplementation 'androidx.test:runner:1.1.0'
androidTestUtil 'androidx.test:orchestrator:1.1.0'
}
Test orchestrator is also available from support libraries if you are not using AndoridX yet.
2. There is a single way to deal with permissions
Searching through the StackOverflow led me to threads where people struggling with test failures because of permissions dialog. Some of them even wrap them into test cases, really. There is no official way to test them, so the only thing that we can do is to grant all the permissions that are required on the view that we want to test.
@Rule @JvmField val grantPermissions = GrantPermissionRule.grant(
android.Manifest.permission.ACCESS_FINE_LOCATION,
android.Manifest.permission.ACCESS_COARSE_LOCATION,
android.Manifest.permission.WRITE_EXTERNAL_STORAGE)3. Test in isolation
Concept officially introduced in AndroidX gives the possibility to test your fragment in isolation, which means that other app functionalities will not affect your testing view. Other advantages of this approach:
- Saving time. You don’t need to go through the whole app to reach the view that you want to test.
- Less mocking/faking. You are not forced to mock all the data which might block the access to testing view when you are going through the whole app to reach it.
- No unexpected side effect. Somebody’s else functionalities (for e.g rating app dialog) which may be opened on your view, controlled by root activity will not affect your test, because you are using side-effect free container to run the tests
but…
“There can be no light without the dark.”
Horace Slughorn
We have to meet a certain condition:
- Fragments cannot refer to explicitly named activities or other contexts, because of swapping containers will end with a runtime exception.
If you are not using AndoridX you simply write your own TestActivity and the Test Rule to open your fragment isolated.
class FragmentTestRule<T>(
initialTouchMode: Boolean,
launchActivity: Boolean
) : IntentsTestRule<TestActivity>(TestActivity::class.java, initialTouchMode, launchActivity) where T : Fragment {
override fun afterActivityLaunched() {
super.afterActivityLaunched()
activity.runOnUiThread {
activity.supportFragmentManager
.beginTransaction()
.replace(R.id.activity_container, T::class.java.newInstance())
.commit()
}
}
}4. Use Idling Resources instead of Thread.sleep()
One of the biggest problems with Espresso is that it does not wait for asynchronous operations to finish, it led to the test failures because the framework wasn’t aware of it.
We can simply use Thread.sleep() to freeze test progression until some time, but there are 3 majority issues with this approach:
- We are not 100% sure if our assumptions to wait for e.g 2 seconds for the request execution is enough.
- In most cases, we are assuming more time than needed. We want to be sure that the test will not fail because of our bad assumption. It means more time spending running the tests locally/CI/Firebase Test Lab.
- Every code change in this area might extend the time that we need for our sleep’s.
The solution is to use Idling Resources. In simple words, it has the ability to notify Espresso about the asynchronous operations that need some time to execute, and again notify it when those operations are finished. A good starting point will be wrapping OkHttpClient into an idler, so our servers calls are done. But what about other asynchronous operations which are not server calls? This is the story for another article... in few words, you can wrap your Network and Computation scheduler with an idler and provide them through the dagger into your code, so every operation which uses those two injected schedulers will be handled properly in your UI tests.
5. Run your tests on Firebase Test Lab
The reasons behind why some developers or companies are not convinced to test the UI of their apps, it is because they are pretty hard to maintain and it is not hard to write a few flaky tests which will fail randomly, especially if the environment is not correctly prepared (look at the points up). Another reason might be the problem how and where those tests should be running. Running tests locally might not give the same results for all developers in the same project. Emulators on CI are working really, really bad so I do not recommend to use them. The answer is Firebase Test Lab. It is a tool provided by Google which allows running UI tests on virtual and real devices, gives us access to screenshots, video recording, and stacktrace which in case of failure are priceless. The advantages of using Firebase Test Lab:
- Running tests on multiple devices at the same time
- No interruptions while testing. The devices at Google are prepared pretty well, so animations are disabled, notification will not appear without an intention. (For e.g text message notification)
- You can run tests with the Orchestrator
- End-to-End tests with real API calls are possible. If Google has no internet connection then who does?
- It works like a charm unlike to CI emulators.
To simplify the process of running the tests on Firebase Test Lab I’ve prepared a plugin for Android projects. All you need to create a service account related to your firebase project and add a few lines of code in your build.gradle
apply plugin: 'firebase.test.lab'
firebaseTestLab {
keyFile = file("test-lab-key.json")
googleProjectId = "your-project-app-id"
devices {
nexusEmulator {
deviceIds = ["hammerhead"]
androidApiLevels = [23]
}
}
}This a part of my Android Developments series. Please check it, if you are interested in more content from me.
If you like my work hit 👏 button and let me know what you think in comments!
Stay in touch on twitter!
