Mobile automated testing with Selenium: An overview

In September 2017, the number of mobile apps in Google Play Store alone reached 3.3 million, Statista says, with the rate of about 1,300 new apps per day. With release cycles so fast, vendors of mobile app testing can’t rely on manual testing only. They will have to recur to mobile test automation to ensure the quality of apps while meeting their tight deadlines.

However, it’s not so easy to choose the right tool for mobile automated testing. We suggest testing engineers look at familiar frameworks with a well-established reputation. For example, Selenium, although initially intended for web applications, has something up its sleeve for mobile testing as well. Let’s check out what it is.

Possible options

Being a well-known software testing framework for web apps, Selenium also powers several automated mobile app testing tools. The most popular are Appium and Selendroid. Let’s take a look at these tools to decide which one is better for you.

Appium

Appium is an open source automated mobile testing framework for testing native and hybrid mobile apps, as well as mobile web apps. At its core, Appium is a Node.js HTTP server that establishes and runs multiple Selenium WebDriver sessions. Appium philosophy promotes the freedom of choice in test practices, frameworks, and tools, which results in a range of benefits:

• Appium supports testing on Android and iOS. In case you have to test a mobile app on iOS and Android, you will need only one script for both.
• The framework supports a variety of programming languages: Java, JavaScript, Python, Ruby, C# and more. Thus, test automation engineers can choose their preferred language to write scripts.
• Unlike many other automation tools, Appium doesn’t require recompiling the app to interact with it. This ensures that the tested app is actually the app that will be submitted to the app store afterwards.
• Scripting with Appium is easy to learn for testing engineers familiar with Selenium.
• Appium can be connected to Selenium Grid or a cloud (e.g., TestDroid) for scaling up.
• Appium supports parallel testing on several Android devices. With the launch of Appium 1.7 and XCode 9, this feature became available for iOS real devices and simulators.
• Appium supports testing on real devices, simulators and emulators.
• Appium has a robust community sharing their knowledge and ideas about this tool.

To ensure quality automation, Appium fully relies on Selenium frameworks and libraries.

Selendroid

Selendroid is a mobile automation framework tailored for native and hybrid Android apps. It also supports testing of mobile web apps. Selendroid is often called “Selenium for mobile apps.” This means that the framework provides the whole range of Selenium testing capabilities with only one limitation: specific targeting of Android platform. Selendroid uses WebDriver client API that provides for full integration with existing Selenium frameworks and numerous programming languages: Java, Python, C#, PHP, JavaScript and more. Selendroid doesn’t require recompiling the app under test. The framework supports parallel testing and integration with Selenium Grid for scaling up. Now let’s dig deeper to find specific Selendroid features.

Selendroid: Specific features

• Selendroid offers a handy feature called hot plugging. This means you can plug another device without restarting or interrupting the test. Selendroid detects new devices automatically and runs tests on them.
• Selendroid offers Dynamic Extendibility. This feature allows users to add their own code to the framework at runtime. You can edit your test script, while the test is still on.
• Selendroid can be used for testing older Android devices with APIs ≤ 17. In fact, this backward compatibility is a major competitive advantage of Selendroid. Selendroid also supports newer Android APIs (up to 19).

As we can see, both frameworks fully use Selenium capabilities. However, Appium has a broader focus as it caters not only for Android, but also for iOS native, hybrid and web mobile apps. But what about the testing process?

Appium and Selendroid: Testing process

Test Development

Writing scripts with Appium and Selendroid requires programming skills. However, scripting in these tools is very easy for testing engineers familiar with Selenium. Scripting may take up much time when the app under test has to interact with the device in a specific way. For instance, a messenger that uses the camera and microphone should have a seamless connection with those. Thus, testing engineers should write relevant test cases to test this connection, which increases time and effort. However, Appium solves this difficulty allowing an app to be opened within an app (e.g., a camera in a messenger), while Selendroid doesn’t have this capability.

Test management

Like the majority of open source tools, Appium and Selendroid lack a convenient test management system. Selendroid Inspector has a simple test recorder that identifies the element clicked and saves the locator and the command to Java tab.  As for Appium, this simple recorder is only available for iOS apps. In both cases, the recorder saves the actions (clicks, swipes, etc.) and locators but not test scripts.

As the developers expand the mobile app functionality, the team needs a convenient reporting tool, and neither Appium nor Selendroid provides any. The testing engineers have to connect to a third-party reporting system, which increases time and effort.

Mobile-specific gestures

Both frameworks successfully address this challenge. Appium provides a convenient API for testing mobile gestures. Using TouchAction, a testing engineer can create a chain of events simulating user behavior and set the timing for a particular gesture. Later on, a series of TouchActions can be added to MultiTouch, which ensures performing the selected chains of events as if they happened at once. Still, some experts argue that the number of gestures in Appium is limited. Besides, testing some gestures only works on Android, not iOS, and vice versa.

In its turn, Selendroid tests mobile-specific gestures employing the Advanced User Interaction API. This API provides for testing simple and complex gestures, such as clicking multiple elements while holding down the control key.

Testing hardware

When it comes to testing hardware, Appium does have some limitations. Appium for Android uses UI Automator API ≥ 17, which means it will work only for devices on Android 4.0 and above. Though latest versions of Appium comprise the Selendroid mode for Android versions with lower API, it is less efficient than the Selendroid framework itself. For example, the Appium Selendroid mode can’t test mobile web apps. What’s more, Appium and the Appium Selendroid mode can’t use one script to test the same app, and the activities of this app in Appium may differ from those in Appium Selendroid mode. So, it’s better to use Selendroid for testing apps on older Android versions (API 10 – 17).

Appium and Selendroid: What to test?

Appium is a good solution for automating functional, regression, integration and compatibility testing of native Android and iOS apps. However, you should bear in mind that the testing process may slow down when testing iOS and hybrid apps.

Selendroid is suitable for automation of functional, regression, integration and compatibility testing of all sorts of Android apps for Android versions with API 10 – 19, as well as hybrid and mobile web apps.

Appium vs. Selendroid: Who wins?

Though Selendroid offers a range of handy features (hot plugging, testing complex mobile gestures, backward compatibility), the tool loses to Appium in terms of target platforms and community support. These two factors also contribute to robust enhancements in Appium: in nine months of 2017, Appium developer community released updates 11 times. On the other hand, Selendroid was updated last in 2015.

Based on the above, we conclude that Appium is a better choice for automated mobile app testing than Selendroid, unless Android is your only target platform. Still, it has its limitations, as testing of iOS and hybrid apps may run slow.