Why do so many teams struggle with quality assurance (QA) testing for mobile apps? We explore challenges associated with mobile testing and provide specific strategies and tools to help teams improve. If you want a more streamlined and cost effective testing process, read on!

Mobile App QA Testing: Strategies and Tools to Improve Your Process

Over the last decade, the mobile industry has exploded with more than 5 billion mobile device users worldwide. Companies today need to ensure not only that they can deliver beautiful, intuitive experiences on mobile, but also that they can deliver them on every operating system (OS) and device.

For cross-platform development, many have turned to 

 to build and scale mobile apps on multiple OSs from a single codebase. However, this still presents the challenge of testing the app on the wide variety of devices available in the market to ensure consistency and usability.

In this article, we examine why quality assurance (QA) testing is so challenging for mobile apps in particular and provide guidance for organizations looking to improve their testing process. We discuss both automated and manual testing to help companies find the testing strategy that is right for them. We also call out the specific tools that we recommend for testing teams.

What is mobile testing and why is it so difficult?

To start, let's define mobile testing to set a baseline for understanding. Mobile app testing is the process of assessing functionalities of mobile apps. The purpose of mobile app testing is to ensure the quality of the app, meet specified requirements, reduce the risk or errors, and improve user satisfaction.

To understand why mobile app testing is so challenging, it's easiest to think of it in comparison with web app testing. Web apps run in the browser, which means that tests are limited by the types of browsers and different versions of those browsers. Most engineers will test on the top 7-10 browsers in use today, and for cross-platform browsers like Chrome and Firefox, will test additionally on Windows and Mac OS.

Mobile app testing is more complex because of the sheer number of different OSs and different models of devices that exist. Over time, more and more OSs are being built up independently, beyond the traditional OSs like Apple iOS, Android, and Windows. In addition, each OS supports a wide variety of different devices that range in size and capability.

Of the traditional OSs, Android poses the most significant challenge because there are many different original equipment manufacturers (OEM) that develop Android phones and often modify aspects of the Android OS based on their specific needs. As a result, engineers have struggled to determine how to set best practices for ensuring mobile apps work properly on all devices.

Having a strong testing process is instrumental to overcoming these challenges and ensuring your mobile app is of high quality. We have outlined our process below in hopes that it will help organizations find clarity in how to meet best practices for mobile app testing.

List all of the relevant tests for your mobile app

In order to create a comprehensive testing plan, we start by documenting all of the tests that need to be evaluated to ensure the quality of the app. Documenting tests allows the team to track, understand, and resolve any quality issues that may arise. We use 

 for test case management, as it helps to organize the QA process and ensure every case is properly tracked and resolved.

When listing all of our test cases, we always consider the overall goal and experience of the app itself. Be sure to review all requirements and key benchmarks to determine what tests need to be covered.

Determine which test cases should be automated

Once a comprehensive list of tests has been determined, each test should be evaluated to determine which can be automated and which should be performed manually.

In most cases, automated testing is the best way to improve test efficiency and receive instant feedback. However, there are some cases where it is not technically feasible to automate the tests or it may cost a lot of resources to automate a single step. In these cases, we review whether automating the scenario would return a greater ROI compared to manual testing. If there is concern that time and resources would be wasted when automating a complex test, we always choose to perform it manually.

 makes it easy for us to automate a variety common tests. Some examples include:

We highly recommend Appium as an automation testing tool because its open-source, simple to execute, and able to test cross-platform. You can find some examples of Appium tests 

Research what devices you will need to test

As mentioned earlier, the sheer number of different devices available on the market presents a significant challenge for mobile testing. While it would be extremely time-consuming and expensive to test every device on the market, it is possible to research and understand which devices need to be tested.

For Apple iOS, testing is a bit more straightforward because it is closed ecosystem with the same OS for phone and tablet. With some research, it is easy to identify the most popular and most recent models that will need to be tested. We typically test on the latest 6 versions, checking both phone and tablet.

Android is much tricker because there are so many manufacturers that modify the Android OS code. To cover this broad range, we pick the most popular devices from each manufacturer to test. 

 is a great source of data we use to determine which devices are most popular. We also consider devices with different screen resolutions, memory size, and connectivity to ensure we are able to check a wide variety of devices.

For all other OSs, we also recommend performing research to determine the most popular devices to test, as well as testing on latest 6 versions. Again, it is important to check a variety of screen resolutions as well to make sure the mobile app will display correctly for users on all different screens.

Create a testing plan for testing different models

Once we have determined our list of tests and devices to test, we are able to structure our testing plan.

Testing on real devices is always preferred when possible for the highest accuracy of results. However, procuring mobile devices for testing can be expensive and often impractical depending on the distribution of the team. Instead, we often utilize cloud-based tools and emulators for testing like 

To test all models efficiently, we group devices based on their screen resolution and cycle through the needed tests on each group. We typically create testing plans based on the number of groups we will test each week. Depending on the app, we plan for about 3-8 weeks of QA testing prior to launch or new feature release. Most testing is performed concurrently with development to maximize efficiency and ensure features are complete upon release.

With more and more technology innovations appearing on mobile, mobile app testing is more critical than ever. Having the right process in place is instrumental to ensuring your app will delight users and run smoothly.

To learn more about how you can develop and test your mobile app, feel free to 

 and we would be happy to answer any of your questions.

Just as the spread of COVID-19 was gaining momentum, Anthem launched a virtual care app, Sydney Care. Find out how Foxbox Digital collaborated with the Sydney Care team as the lead mobile app development partner.

How We Helped Anthem Launch a Mobile Telehealth Product During a Pandemic

On January 1st, 2020, just as the spread of COVID-19 was gaining momentum, Anthem, Inc., a leading health benefits company that serves over 110 million people through their affiliated companies, launched a virtual care app, 

Foxbox Digital collaborated with the Sydney Care team as the lead mobile app development partner, because of Foxbox’s experience and deep knowledge of 

Sydney Care, available on iOS and Android, utilizes AI-driven guidance to empower users to assess their symptoms and then choose the virtual care option that’s best for them. Additionally, through its COVID-19 Resource Center, Sydney Care provides an end-to-end solution that helps employees more safely return to the workplace.

“We built a solution that some of our customers have reviewed as being ‘life-changing.’ The ability to address health issues without physically going to a doctor, and in the comfort of their own home, is a rewarding experience to be part of, and we continue to see the growing impact of our telehealth platform.”
- Plamen Petrov, Ph.D., Vice President, Exponential Technologies, Anthem, Inc.

In 2020, Sydney Care was downloaded over 260,000 times, while thousands of users continue to utilize the application daily to diagnose symptoms and seek guidance for their health conditions from a device that lives in their pockets. Additionally, over 41,000 people have assessed their COVID-19 symptoms using Sydney Care.

The pandemic has forced many professionals to work from home and prevented critical care for many non-COVID patients. Sydney Care became a desired application for many people who needed preventative care, and also consumers with chronic illnesses.

“I was able to quickly chat with a doctor this afternoon about allergies and pick up my prescription within an hour. This beat taking 3-hours out of my workday, fighting traffic, and sitting in a waiting room (especially right now during a pandemic) to see a doctor. I found a new way to do health care,”
- jscrmil1, 04/14/2020 iOS App Store

A Delightful Mobile Experience (Built in React Native)

As the React Native mobile development experts, we interacted with Anthem’s Product, UX, Business, Integration, Authentication, Backend Development, and Security teams to build both iOS and Android native applications, using a single React Native codebase.

React Native allowed us to develop one app that works natively on iOS and Android instead of building two separate apps with disconnected codebases. Using React Native as the programming language enabled Anthem to streamline development and maintenance costs while still creating a stellar mobile consumer experience.

Through our collaboration and contributions, we delivered these core features and benefits:

Integrated Video and Chat for Virtual Doctor Visits

: Chat in real-time over video and text with healthcare providers.

:This triage functionality in the app allows a seamless experience for consumers to check their symptoms and get personalized guidance.

:A login-less experience for existing customers, as well as integration with other insurance providers.

: All of the technical enterprise security protocols needed without sacrificing the experience for the consumer.

: The ability to pay via credit, debit, or via insurance provider, all within the app.

A COVID-19 Return to Workplace Solution Built in 10 days

As Covid-19 started growing, we quickly pivoted along with Anthem’s product and design teams to develop an 

AI-powered Covid-19 daily-check in solution

 that helps protect workers by keeping potentially infected employees away from the workplace.

We developed the front-end and back-end of this new functionality and published on both app stores (iOS and Android) within 10 days—in addition to a previously-created COVID-19 assessment solution. This service is currently being used by many large organizations, some with 30,000+ employees, and more partnerships to come.

Better Healthcare Experiences Start with a Seamless Digital Experience

Together, with the partnership of Anthem’s robust global design, product, and integrated teams, we assisted in the development of an easy-to-use, direct-to-consumer mobile telehealth solution. With the added complexity of working within an enterprise environment, we continued to be flexible and agile in our approach of acting like a startup within an enterprise.

We’re honored to work alongside a leader in healthcare and innovation.

“They are professionals, talented, smart. They are committed, with a very strong core team. They owned the problem."
- Elliott Torres, Staff VP, AI and Architecture

Agile empowers businesses to rapidly identify and respond to new ideas. Living through a pandemic has proved these innovation techniques to be more important than ever. Learn how you can support this vital process.

How You Can Empower Innovation by Becoming an Advocate for Your Agile Development Team

Living through a pandemic has shown us that businesses need to be flexible and resilient to thrive in this new era of business. In a time when so much is uncertain, businesses must be willing and able to constantly adapt to changing circumstances.

Agile provides an execution framework for accomplishing exactly that. By taking people out of their functional silos and placing them in customer-focused multidisciplinary teams, the agile approach allows businesses to rapidly identify and respond to new ideas.

Even those outside of the agile development team have an important role to play in supporting agile at their organization. We will examine how leadership can act as advocates to the agile development team, and therefore empower growth and innovation.

To start, let's review the difference between agile and traditional (waterfall) methodologies for project management.

Waterfall works well for linear projects where the team knows exactly what they need to do to get the job done. The traditional command and control, sequential “do this then that” approach.

On the other hand, agile works well for dynamic projects focused on new ideas and innovation, where multidisciplinary teams uncover unknowns and need to learn and adapt to rapidly changing circumstances to create the best possible outcome.

 It is the practice that enables organizations to act on their new ideas.
- Tim Brown, IDEO CEO

What Makes Agile Adoption So Challenging?

Moving from a more traditional waterfall, command and control approach, to agile isn’t an easy task. The most common reason why agile adoption fails is because company leadership is not fully bought into the process.

Successful agile software development requires an openness to address unknowns via an interactive process of 

 Leadership must understand the importance of innovation as it pertains to company growth, and learn to overcome their fear of change so the business can progress and evolve.

If the organization does not support the team in experimentation, the team will not be empowered to practice true agile.

Empowering Growth and Innovation Through Agile Development

For agile to be truly successful, it is essential that business leaders become agile advocates. This means they are not only bought into the idea that agile will provide better outcomes, but they actively support their team to by paving the way for them to succeed.

Here are ways that business leaders can support their agile development team, and therefore empower growth and innovation.

Recognizing problems help us improve and grow. The best agile teams take failures and turn them into learning experiences and successes.

While “fail fast” is a philosophy commonly associated with agile, what agile really allows teams to do is “learn fast.”

 Without making missteps, the team will not have the opportunity to learn where they can improve.

This can be scary for leaders that are not used to an agile environment. However, when agile teams are able to overcome small failures, they learn how to prevent larger ones in the long run.

Trust the Team to Make Their Own Decisions

Agile teams need the ability to work quickly and address unknowns when they arise. Long bureaucratic approval processes can become significant roadblocks in allowing teams to access the tools that they need to innovate.

In order for teams to capture opportunities, they need to be able to act autonomously.

 This requires full trust from the organization that teams are experts in their own domain. With this trust, teams will be empowered to optimize processes as well as ask for, modify, and use the tools they need to create the best possible work.

Understand the Difference Between Capacity and Throughput

Focusing on throughput means that the emphasis is put on how much a team is producing, and not how much each individual member is working.

By not forcing each team member to constantly perform at their maximum capacity, they have more time available to support each other and solve problems when they arise.

Efficiency is often lost when trying to do too much.

 The most important thing should always be the outcomes the team is delivering as a whole.

Commit to Stability to Achieve Innovation

In order for the team to focus on impact and delivery, the organization should prevent them from experiencing instability or shock. 

Distractions are a threat to the productiveness of the team and can prompt counterproductive coping behaviors and poor decisions.

The organization as a whole needs to be educated on agile so that they also understand when their own actions can be detrimental to company innovation.

Successful agile development is contingent on 

 from all levels of an organization. When an agile team is empowered to explore unknowns, the product becomes more innovative and 

better serves the user and the organization as a whole.

If you want to learn more, here are some helpful resources:

Leading agile transformation: The new capabilities leaders need to build 21st-century organizations

New Requirements For Agile Leadership: How To Lead Differently For Agile Success

To Build an Agile Team, Commit to Organizational Stability

Foxbox Digital is an award-winning digital product agency. Based in Chicago, we've helped Fortune 50s and high-growth startups deliver digital experiences.