Building An Effective Device Matrix For Mobile App Testing

Mobile apps are at the forefront of digitization. Today, there’s an app available for everything from online banking, bill payments, and online education to online shopping, food delivery, and much more. With such significant usage of apps in each domain, the usage of apps is bound to evolve shortly.

With the evolving software and hardware specifications from leading mobile vendors, businesses are bound to face one challenge – delivering cross-platform compatible apps faster. The trends and preferences of mobile users keep varying due to fragmentation. For example, an Android user might switch to using an iOS device after a year or vice versa.

Naturally, businesses willing to target a wider audience need to test apps across devices and platforms that users across the globe widely prefer. Hence, building a device matrix for effective mobile app testing becomes imperative for mobile teams.

This article aims at explaining how teams can build a comprehensive yet effective device compatibility matrix for better coverage keeping in mind some critical mobile app testing parameters. Let’s dive in by answering a few fundamental questions.

What is a Device Matrix?

While developing a mobile app, developers are likely to have a list of mobile OS versions and specifications they expect their apps to be compatible with. When this list is finalized and documented, it is often referred to as the mobile device matrix.

The image below represents a sample mobile device matrix.

Image source

A device compatibility matrix is a document that clearly defines the scope (and cost) of development and testing efforts related to specific platforms and handsets.

Why do Teams Need To Create A Device Compatibility Matrix?

The mobile device landscape is highly fragmented as numerous handsets from unique vendors such as Apple, Samsung, etc., are available and updated yearly. Additionally, every handset has a unique Android or iOS version installed on it. These handsets keep updating their operating systems at regular intervals, exacerbating the fragmentation scenario.

Naturally, it is not feasible for businesses to test their apps on every single device-OS combination available digitally.

Here’s where the device compatibility matrix comes into the picture. The device compatibility matrix limits the development and test coverage of mobile apps to a specific set of operating system versions and specification sets.

Restricting the number of compatible mobile OS versions helps teams focus on the most popular ones relevant to their target audience. It also helps reduce the app’s exposure to cross-platform bugs. Additionally, testing and optimizing apps on the latest device-OS combinations will enable developers to align the app with the latest mobile releases in the market globally.
Moreover, a well-defined scope helps deliver the best possible user experience for the selected platforms.

This drives us to an important question.

How to Analyze Ideal Test Environments for Building A Device Compatibility Matrix?

Evaluating the ideal device-OS combinations for comprehensive app testing is a critical task.
This phase demands thorough research on the geographical usage of mobile apps.

Listed below are a few tools or sites that one can leverage to figure out the most prominent mobile OS versions and handsets based on specific areas:

1. Google Analytics

Teams must leverage this analytics tool to evaluate the widely used mobile OS and handsets from particular vendors (like Apple, Samsung, etc.). It also indicates the locations users are coming from. This will help identify the most preferred mobile OS and handsets in the device compatibility matrix.

2. StatCounter

StatCounter is one of the best websites to get statistical insights concerning technologies. You can extract data for popular screen resolutions, and OS platforms used in specific countries like the USA, Europe, etc. The image below represents the current scenario of popular screen resolutions globally.

One can also further refine the data based on geographic location. Users can view the data in multiple formats such as Graphical charts, Maps,  Pie charts, etc.).

This helps devs gain deeper insights. One can also refer to these credible public sources for procuring relevant information:

1. NetMarketShare
2. Apple’s Official Support Pages
3. Android Dashboards

Data Points to Consider Before Building Device Compatibility Matrix

Mobile teams need to consider certain data points to evaluate accurate app usage across specific mobile OS or device types. These include:

1. Platform-based research: It helps understand the medium via which a user is accessing the mobile app. For example, it helps determine whether a user is coming from an Android or iOS handset. Additionally, one can also identify the version of Android or iOS the user is coming from.
2. Brand Usage: It helps analyze whether people from specific geographies are using handsets from a particular brand (like Apple, Samsung, Motorola, etc.)
3. Screen Sizes: There are numerous handsets released from unique vendors. Each device has unique screen sizes. Analyze the most popular screen resolution among the target audience and ensure it is included in the device matrix.

Gathering data from the sources above will help teams better understand their audience (local or global). Moreover, research on these data points will help teams better understand the Device-OS combinations most relevant for targeting the app audience.

BrowserStack’s mobile testing lab fulfills this need by offering real mobile devices for manual and automated app testing. This also means that teams do not need not worry about setting up any on-premise device labs.

One simply needs to sign up for a free trial and start testing on the latest Android and iOS devices on the cloud!

Test Your Mobile Apps For Free

Ending Notes

Rolling out mobile apps that deliver a seamless user experience across platforms and devices is a big challenge. However, creating a device matrix after thorough research makes it relatively easier.

Creating a device matrix in the initial stages of development is the best practice as it helps developers and testers understand their goals better.  The data-driven approach helps teams make informed decisions during development and testing. Last but not least, remember that an updated device matrix helps teams release an app with the essential element in mind – the actual users.