Head of Client Strategy
July 16, 2024
We’ve been building teams overseas for over a decade. Download our definitive guide to hiring international software developers.
Ever feel like you’re swimming against the tide with your mobile test automation efforts? You’re not alone. Many teams struggle to push their automation coverage past the 10% mark, often due to the complexity and sheer volume of work required. We understand the challenges - tight deadlines, resource constraints, and constantly evolving mobile app features.
But what if there’s a proven approach to driving more meaningful mobile test automation coverage in a short period?
Here’s a condensed version of the methods we use when helping our clients break through the barriers of automation.
Let’s dive in.
Before any team can chart a course to improved coverage, it’s important to examine where automation currently stands. This first involves taking a hard look at your current metrics: how many tests you have, application coverage, requirements coverage and the percentage of automated vs. manual tests.
It’s almost never practical to set a goal of 100% automation coverage for many reasons. Instead, we encourage teams starting from ground zero to simply get to 10%. Once you’re past square 1, then it’s more realistic to scale to 30%. With momentum, teams can often quickly get to 50%. Some teams stay around 50% and that works well for their business - and others continue pushing to 75% and beyond.
No matter where you’re at now, simply start by identifying the test cases that will have the most significant impact on the business itself. Every line of test script is a liability you have to manage, so it’s important to place any investment of time and energy towards automated tests that reduce risk and operational overhead. This is why it never hurts to get the buy-in of key stakeholders from multiple teams.
At Perform, we use a specialized client intake assessment for mobile app teams that maps metrics, personnel, tools, and more into an easy-to-consume format. This also helps teams identify gaps and develop a more actionable step-by-step roadmap to follow.
You’re welcome to book a session here if you’d like to learn more [Mobile Strategy Booking Page]. Or, if you’re flying solo, here’s a few points you can examine:
Key Actions:
It almost goes without saying that not all test cases are created equal. Prioritize those that trace directly back to your most critical business requirements. End-to-end traceability is essential here - especially as you showcase the value of your organization’s investment in mobile test automation.
The Pareto Principle (80-20 rule) almost always applies for quality engineering teams. More often than not, you’re better off focusing on the 20% of high-impact test scenarios that cover an outsized 80% of application requirements.
In the book, Enterprise Continuous Testing (1), Wolfgang Platz (Founder, Tricentis) details how organizations can consider the factors of frequency and impact to determine which tests make most sense to automate first. It’s a quick read but a great resource for teams looking for tried-and-true approaches to test automation.
Key Actions:
Naturally, a robust test automation framework is key to scaling your efforts. If your team gravitates towards open source and have the resources to build and maintain it, you can continue using Appium. For teams that are operating with resource contraints, be prepared for challenges.
Regardless of whether you use Appium or a commercial framework like Katalon, Tosca, or Testim, it’s always a best practice to emphasize modularity in your test design. A modular approach not only facilitates reusability but also significantly cuts down on maintenance time, freeing up more bandwidth for creating new test scripts.
Once teams escape what we like to call ‘the maintenance trap’, it’s like watching a car take off after the driver had their foot on the gas and the brakes at the same time.
Key Actions:
For teams that consider maintenance to be their biggest barrier, environment stability is often described as a close second. This is challenging in any project - but can be especially difficult as it relates to mobile. You’ll likely need a plan in place to ensure your app not only passes tests on a single device type and OS, but also passes on dozens (if not hundreds) of device / OS configurations.
While simulators and emulators are helpful for developer-driven tests earlier in the cycle, they often fall short in functional and end-to-end tests. Virtual devices can also be notoriously difficult for developers to debug against since they don’t mimic hardware and real world network conditions.
For quality engineering teams focused on mobile, this is why it’s valuable to have a mobile device cloud in place. Solutions like Kobiton, HeadSpin and TestGrid give teams the ability to run automated tests against real mobile phones and tablets that are hosted in a datacenter - or even from their own test lab.
These solutions also give automation engineers the ability to run the same tests against a variety of device configurations. Massive efficiency gains can be unlocked when you transition from sequentially running tests on a few local devices to running parallel tests against 25+ devices at once.
We examine this topic in further detail in the article, “Beyond Simulators and Emulators: 5 Signs it’s Time to Adopt a Real Device Cloud”
Key Actions:
Integration with your CI/CD pipeline is non-negotiable for continuous mobile test automation. Automated workflows ensure that your tests run consistently with every code change, providing immediate feedback and catching issues early.
When CI/CD pipelines play the role of an ‘orchestrator’, they can also help automation engineers and SDET’s reclaim valuable time that’s otherwise spent on manual tasks. For instance, when a new build is deployed, there’s no reason to have to manually kick off automated test suites every time.
A full examination of CI/CD workflows is outside the scope of this article, but you can find more detail in “10 Must-Have Automated CI/CD Workflows for Mobile App Teams”
Key Actions:
Aside from automation frameworks and device clouds, there are solutions on the market that can help accelerate time-consuming tasks like test case scripting. While traditional record & playback tools tend to be brittle, there’s tools that can turn a set of manual test steps into a ‘rough draft’ of an Appium script.
One example is Kobiton’s Script Generator capability. Of course, this still requires an engineer to add more loops, parameters, and asserts in their IDE to make the script more robust and reusable. However, capabilities like this can help an engineer build a script in less than half the time.
It’s also worth considering tools that offer ‘self-healing’ functionality. Solutions like Katalon Studio help engineers flag scripts that are prone to breaking after recent application changes. The Katalon framework can also detect when locators break during execution - and then find alternative locators and continue running the test without failing.
As a last recommendation in this section, teams often find benefit from having a robust test management system in place. These solutions usually serve as a ‘system of record’ for test results and help simplify reporting. Whether you use a standalone solution like qTest or Testrail, or a Jira plugin like Zephyr or X-Ray, it’s important to establish a practice for maintaining traceability amongst requirements, test cases, test executions, and defects.
Key Actions:
As we conclude our 7-step roadmap to improving mobile test automation, let’s address the importance of continuous improvement and regular monitoring. As time goes on, don’t fall back into old patterns. Continue measuring 'north star metrics' that help quality engineering drive towards business objectives rather than placating vanity metrics.
This is often done with the help of a testing solutions partner, but it’s still a discipline that a team can adopt in-house.
Key Actions:
Ready to take your mobile test automation to the next level? Reach out to Perform for a free consultation or to learn more about our services. Let us help you optimize your testing processes and achieve higher coverage with ease. Contact us today!
Shane Shelton
Sr. Director
Application Performance and Development Operations, McGraw Hill Education
Founded by engineers - for engineers.
Expert consulting and staffing for software engineering at scale.
