Automated Testing of a Secure Mobile App with Appium

Customer

The Customer is an international digital security company. The Customer offers secure B2B and B2C solutions and cybersecurity consulting services.

Challenge

The Customer was developing a secure messaging app for Android 5.0 and above and iOS 8.0 and above. Their in-house testing team introduced automated functional testing in addition to manual testing to ensure fast releases. The testers wrote a set of simple positive Appium scripts to detect critical functional issues. However, the in-house testing team couldn’t combine manual and automated testing and didn’t perform Appium automated testing efficiently due to a heavy workload. As a result, the quality of releases worsened. Also, the Customer needed to introduce performance testing, which became impossible due to the testing team overload.

Seeking to make Appium automated testing more efficient and reduce the load on their testing team, the Customer turned to ScienceSoft, their long-term partner. One of the key project challenges was a tight timeframe. The Customer planned to release a fully functional secure app for beta-testing in only five months.

Solution

ScienceSoft’s automated testing engineers studied the available project requirements and drafted a suitable process for Appium automated testing in the project. The team analyzed the Customer’s test automation scripts and introduced several improvements:

• To run tests with different sets of values for each test, automated testing engineers used a data-driven approach.
• To reflect the app’s business logic in a number of scripted user actions, automated testing engineers used a keyword-driven approach.
• To speed up the testing process, ScienceSoft’s testing team used Jenkins continuous integration server. This way, the testing team set up a schedule for a script start without a tester’s participation, as Appium doesn’t allow for an automatic test start.

In the course of the project, the testing team performed automated functional testing with Appium for the Android app version. They also wrote separate Appium scripts for critical path testing of the iOS app version, as it was still under development and scripts for the Android version couldn’t be used.

All Appium scripts could be updated and reused.

ScienceSoft’s team also managed to save time by using Appium scripts for the app performance evaluation. The team paid specific attention to the following parameters:

• Test execution duration
• Number of user operations per execution
• Volume of data per operation and data type

To ensure vast device coverage and detect device-specific issues, ScienceSoft’s testing team ran testing on real devices (chosen according to global use statistics) and emulators.

Results

ScienceSoft’s automated testing team enhanced the Customer’s testing process for automated functional testing and performed testing within the set time limit. The Customer obtained automated Appium scripts for both functional testing of the Android app version and critical path testing for the iOS app version. The Appium scripts were also used to identify possible performance issues. The scripts could be updated and used further.

Technologies and Tools

Appium, Java, JetBrains IntelliJ Idea, Android SDK, Xcode Instruments, Maven, Gradle, Jenkins.