Automated web application testing: PM’s guide

Creating a quality and efficient web application requires considerable test effort, which in some cases is tedious and time-consuming. This makes project managers recur to automated testing services, as test automation saves time and effort on monotonous tasks and allows a testing team to concentrate on tests requiring human perspective.

Yet, automated testing isn’t a universal remedy. Developing an effective automation requires a clear understanding of the process by all team members from testing engineers to PMs. Unfortunately, in many cases, team members concentrate exclusively on their work at hand, and intra-team communication lags behind.

In our article, we are trying to bridge this communication gap with the help of an overview of web application test automation. We will look at automated web application testing from the PM’s point of view and highlight the most important issues.

Test automation step by step

Time to automate

In the long run, test automation saves time and effort. However, it has its own cost, so it will be a viable option only if the project meets certain conditions:

1. Project scale. For small short-term projects (less than six-months-long), automated testing doesn’t pay off.
2. Project maturity. For medium and large projects, test automation only makes sense when the project has already reached a certain level of stability and maturity and no longer experiences drastic changes in UI or business logic.

Automation method: Data-driven testing

Web applications can be data intensive. Thus, data-driven testing (DDT) may make a reasonable strategy. According to ISTQB, DDT is a special “scripting technique that stores test inputs and expected results in a table or spreadsheet.” The data are separated from scripts. This enables test engineers to run the same script for different input values. For data-intensive applications, DDT significantly reduces time and efforts spent on scripting.

What to automate?

Some types of testing can be automated, which spares a testing team monotonous tasks and improves testing quality. These types of testing include:

• Regression testing. Costly in terms of time and effort, regression testing makes number one candidate for automation. However, automation doesn’t eliminate all regression testing issues. To make your regression testing work effectively and reduce testing costs, you should ensure its proper organization.

• Cross-browser testing. This type of testing assures that your web applications perform equally well across the specified browsers. The task involves two challenges that consume the time of a testing team:

1. Ever-changing browser versions. Firefox and Chrome alone are updated every 6-8 weeks. Thus, by the time you are ready to deploy the web application, your target browsers may have seen several updates.
2. Different operating systems. While many users may run the latest browser versions (they often update automatically), it’s very unlikely that users also have the latest operating systems installed.

Automated cross-browser testing addresses the challenges above and ensures a web app’s compatibility with specified versions of browsers and operational systems 6 times faster than manual testing. Usually, it lasts less than 5 hours versus 30+ hours for manual testing.

What’s more, you can fill two needs with one deed. The majority of cross-platform automated testing tools contain quality UI testing automation, as an adequate UI rendering is the cornerstone of cross-browser testing.

• UI testing. This type of testing is critical. It predefines the popularity of your web application. To make automation most valuable, experts advise to start off early, with the simplest features, like switching from one web page to another.

However, UI test automation cannot replace human perspective. Thus, expert exploratory testing is a must. It is also advisable to team up exploratory testers and customers or their proxies to make sure the web application provides a positive user experience.

• Performance testing and its variations (stress and load testing). This type of testing assures that a web application can successfully endure an extended period of activity or the peak user load. This type of testing is mostly automated, as reaching the necessary load level or stress conditions manually is infeasible. However, you shouldn’t treat performance testing as a purely mechanic effort. Here are some points to rely on:

1. Though this type of testing is non-functional, positive UX should still make the central reference point. Thus, you shouldn’t run after excellent performance indicators at the expense of UX.
2. Test data should be realistic. For instance, you may simulate billions of users accessing the web application, but this will hardly ever be the case in real life.

Automation of all the testing types above doesn’t always make sense. Before you decide to automate, you should carefully consider the efforts and costs involved in the automation process and the value it brings. It could also be useful to start small, that is, to automate simple test cases to see how it works for your team and how much time and effort they invest. This way the project team will avoid unexpected delays.

Choosing the right test automation tool

No tool is a magic wand addressing every challenge automated web application testing may bring to a QA team. Therefore, it’s vital to examine the tools that meet the needs of your project and choose the most relevant one. In some cases, it might be better to use a combination of automated testing tools for web applications that work best for certain types of testing to ensure time- and effort-wise testing process.

Test maintenance

Automated test suites are also a specific type of software. To stay valuable, they need maintenance efforts:

1. Upgrading test suites. Changing the scripts along with the changes in software goes without saying. However, in some cases, it’s unclear whether any change is needed or not. In this case, developers may help. To assure inter-team collaboration, some project teams establish continuous integration. Using this approach, the developers integrate the code into a project depository daily or several times a day. The testing team also has access to the repository. This helps modify test scripts and address bugs in the code faster.
2. Cleaning up. A testing team regularly reviews the scripts and terminates the obsolete and ineffective ones.

In the long run, these efforts reduce testing time and cost, as they contribute to creating efficient and up-to-date test scripts.

Additional challenges

AJAX

AJAX ensures responsive and engaging UI, but complicates automated testing efforts. AJAX allows a web application to call to the server and refresh a part of a webpage without reloading it. Refreshing can happen any time AJAX makes a call to the server, and an automated testing engineer has to wait for the response to proceed with testing.

For this matter, automated testing tools provide several waiting patterns. For instance, Selenium WebDriver observes six of them. Still, testing specialists need additional time to figure out which pattern to use for a particular project. These peculiarities increase testing time and effort, which in its turn increases testing cost.

Mobile web

As the use of mobile devices already prevails over that of desktop and portable computers, developing a mobile web app along with a web application seems a reasonable decision. Of course, this will increase testing efforts adding even more challenges:

• Device fragmentation.
• Different screen sizes and operating systems.
• Different device makes relying on diverse software and hardware.

Though professional testing teams overcome these challenges employing adequate testing strategies and tools, they may significantly increase testing effort and cost.

Wrapping it up

For a PM, introducing automated web application testing requires planning and clear understanding of the process. Typically, web app automation comprises four steps:

1. Choosing the time to automate.
2. Deciding what to automate.
3. Choosing the right automation tool addressing the project goals and peculiarities.
4. Keeping in mind test maintenance efforts.

Web app automation works towards reducing testing time and effort only when you have quality test scripts, effective testing process and clear communication within the team.