How to get started with automated testing of an Android application
Getting started with automation for an Android application does not mean covering every screen and device. It is much more useful to choose a few important user journeys, prepare reliable data for them and verify that the tests give the team fast, clear feedback. This foundation can then be expanded according to risk and actual use of the application.
Why a large plan often fails to deliver quick results
Android exists in many versions, display sizes and manufacturer customisations. Trying to test every combination leads to a large matrix, long runs and results that the team does not have time to evaluate. On the other hand, a single emulator on a developer’s computer does not cover differences that emerge on specific hardware.
Another problem is choosing simple but unimportant scenarios. A test opens a list or changes an insignificant setting, but does not verify sign-in, a purchase or first launch. The number of tests grows, while the risk to the user remains almost unchanged.
On mobile, a bug also has a longer path to a fix: a new release needs to be distributed, and not all users update at the same time. The article Why a bug in a mobile application is harder to fix than on the web explains this in more detail.
Five steps towards the first useful coverage
1. Determine what must work
Choose three to five journeys with the greatest impact on the customer or revenue. These are usually installation and first launch, sign-in, the product’s main function and payment. The overview of mobile scenarios suitable for automation can help with the selection.
2. Prepare a known initial state
A test needs dedicated accounts, predictable data and a way to return the application to a known state before a run. External services and payment gateways use test modes. Without this preparation, even a correctly written scenario will fail randomly.
3. Verify the tool in a pilot
Appium and Maestro use different approaches to writing and running tests. We therefore try the same critical scenario on the specific application first. We examine element availability, system dialogs, diagnostics and who will maintain the tests.
4. Divide runs by purpose
Short smoke tests can run on an emulator after a relevant change and quickly detect a fault in a basic journey. We run broader coverage regularly or before a release and supplement it with selected physical devices. Their list is based on user analytics and product risks, not on the current selection of phones. We describe the differences in Emulator or physical device.
5. Assign responsibility for results
Every run must have an owner, a clear report and a rule for what happens after a failure. Video, screenshots, logs and other diagnostic data need to be configured according to the tool and environment used. A test whose result nobody evaluates does not help, even in a perfect pipeline.
What to watch out for
End-to-end tests do not replace unit and integration tests. They are slower and more demanding to maintain, so we use them primarily to cover important journeys through the entire interface. A CAPTCHA, one-time codes or third-party services may require a test mode or a controlled replacement.
What you gain
The team gains its first value without waiting for a large project. A short suite detects important faults early, while targeted physical devices add information about behaviour in a real environment.
Gradual expansion also keeps run time and maintenance under control. Every new scenario has a clear reason, owner and place in the testing strategy.
Next step
List three journeys without which a customer cannot use the application, and prepare a test account and expected outcome for each one. Verify one journey in a short pilot on both an emulator and a physical device. Only then decide on the tool, frequency and scope of the next suite.