Android app testing specifics

A decade-long rivalry between mobile platforms is over, at least for a while. Overpowering its key competitors, in 2015 Google’s Android became the most widely used mobile platform on the global scale, and the expansion continues. Statista reports that in September 2017, Google Play Store counted 3.3 million apps, a dramatic increase by 900 hundred thousand in comparison with the data of just one year ago. Given the scale, the providers of mobile app testing services have to single out key reference points for Android testing.

Android app testing: key points

Most of Android app peculiarities stem from the open-source nature of this platform. Google generously allows mobile device manufacturers to use and customize Android as they please. As a result, various versions of Android power various mobile devices, which presents considerable challenges for testing engineers. Now, let’s discuss these challenges in detail.

Fragmentation

Android gladly welcomes original equipment manufacturers (OEMs), so the number of vendors and devices relying on this platform is immense. For instance, in 2015, Android began to power 600 new mobile devices alone, and the number is on a steady rise. It’s impossible to test an app on all the devices available, so testing engineers study statistics and choose the most popular models and manufacturers for a target region. Popular models and manufacturers usually predefine the OS versions popular in the region. In the U.S., for example, Samsung Galaxy S5 – S8 and Pixel Android 7 on Android 6 – 7 make the top of the list of the most popular manufacturers and models bound with OS versions.

Varying screen sizes and densities

In 2015, Android powered the devices with 24,000 screen sizes. This includes mobile phones, smartphones, tablets, TVs and wearables. Testing an app over all these screens is impossible. Therefore, testing engineers have a strategy allowing them to choose relevant screen sizes. Android developers support four types of screen sizes: small, normal, large, and extra-large. Testing engineers pick the smallest and the largest screen from the list of targeted devices and test the app on them. The same goes for the density. For example, in the U. S., Samsung Galaxy S5 with 432 density per inch (dpi) is the smallest smartphone screen, and Samsung Galaxy S8 with the 567 dpi makes the largest screen.

Having discussed overall aspects of Android-based mobile app testing, we turn to purely technical side of testing, such as android app testing strategy, types and methods of testing and their specifics.

Android app testing strategies

Choosing a suitable testing strategy depends on the type of an app. A native app relies on the capabilities of the mobile platform, so testing native mobile apps on Android requires a special attention to UI testing and integration with Android operating system. In this case, Android community UI guidelines may come in handy. Testing hybrid apps and mobile web apps is similar to testing a web application.

Most of Android testing specifics concern the app non-functional features. Let’s consider them.

Non-functional testing for Android apps

Non-functional testing efforts for Android app testing go beyond the well-known performance testing. They include:

1. App installation and deletion

Proper testing of the app’s installation and deletion is necessary for positive user experience. When being installed or deleted, the app shouldn’t involve other apps on the device or personal information stored there.

Another important point is handling updates. In case the app requires an update, the user should receive a relevant notification. Unfortunately, Android apps often update quietly, and the user loses track of the memory space available on the device, which hinders the UX.

2. App behavior when internet connection is unstable

Nowadays, users take the mobile device’s ability to connect to the internet for granted. However, an internet connection isn’t always smooth and seamless. Unfortunately, Android apps don’t always handle faulty connection well. They may shut down without the user’s permission or even crash. Thus, testing engineers should specifically check how the app handles such conditions as intermittent connection, transition (Wi-Fi – 3G/4G) on the go and loss of connection. In this case, manual testing efforts are most important as they simulate real-life conditions better.

3. Battery and device efficiency

As an operating system, Android is rather heavy. Many apps tend to work in the background, even when a user tries to close them. This feature exhausts the device battery, so its service life inevitably lessens. Besides, the overall device efficiency drops down when hordes of apps run in the background. Testing engineers should assure that when the app isn’t in active use it doesn’t abuse the battery. For this matter, they should specifically check if the app sends or receives data when backgrounded.

Android offers various options, namely battery saving mode (Android 5.0 and up) and Standby mode (Android 6.0 and up) providing for economical use of the device power capacity when accessing network. Unfortunately, these advanced features are available for newer Android versions starting with Android 5.0. For older versions (Android 2.3  – Android 4.4), it is vital to assure that the app shuts down on the user’s demand.

4. Security issues

Unfortunately, Android’s openness presents a major security pitfall. Android has a relatively lax publishing policy that enables practically anyone to publish an app in Google Play Store. As a result, Google Play may contain many infected apps. Downloaded by a user, these apps may transmit the infection to the rest of the apps on the device. And if these infected apps contain sensitive information, this may wreak havoc. To patch possible pitfalls and ensure safety, penetration testing specialists should run a comprehensive security testing.

Android app testing: Manual or automated?

Android development teams strive to deliver quality apps swiftly and efficiently. At a first glance, this makes automated testing a reasonable choice to reduce time and efforts spent on testing. However, it’s not that simple. As mobile apps are usually compact, manual testing is often more efficient.

In case automation testing is necessary (tight deadlines, extensive compatibility tasks, vast market targets and device coverage), experts advise to use automated testing to cover:

• Repetitive test suites requiring extensive manual efforts (e.g., regression testing).
• Test suites that cannot be run manually (e.g., performance testing).

Though this list is not that long, optimal test automation may take up 70 – 80% of testing efforts in a project. Selenium-based Android app testing tools (Appium and Selendroid) make an efficient solution here. Android developer community recommends some testing tools (Android Studio, Espresso, Roboelectric, etc.). What’s more, they provide tutorials, which facilitates automated testing.

Summing up

Challenges in testing Android apps stem from the extensive popularity of this platform all across the globe and the specifics of the platform. These peculiarities include:

• High fragmentation (device makes and OS versions).
• Variations in screen sizes and density.

Testing teams address these challenges on the basis of regional statistics. They choose most popular device models and OS versions in the target region and then select devices with largest and smallest density per inch (dpi) from the compiled list.

As for the specifics of the platform, testing teams should focus on non-functional features of Android apps, such as app installation and deletion, battery and device efficiency, varying connectivity conditions and security testing.

When choosing a relevant strategy for Android app testing, a testing team should rely on the peculiarities above as well as the project specifics. This ensures efficient testing and, hence, the project’s success.