What is regression testing? Short overview

Regression testing often darkens the landscape for providers of software testing and their project managers. To dispel the clouds, a PM should first get a deep understanding of what regression testing is and what it means for the project.

Regression testing assures that recent changes in software haven’t altered or broken the old functionality.

For example, you developed an alarm clock mobile app. This app has just two core functions: setting the time and activating a sound signal to wake you up. You run functional testing to make sure the app works, get a positive result and rest assured. Later on, you decide that the app may well notify you about important events in your calendar. You add the feature and test it. It works well, but this time you retest the wake-up functionality of your app to make sure a new function didn’t hamper the existing one. This example fully illustrates the regression testing definition.

Regression testing stages

Regression testing comprises several stages:

1. Compiling regression test suite. A testing team chooses efficient functional and non-functional test cases that repeatedly found bugs and include them in the suite.
2. Performing regression testing.
3. Debugging.
4. Retesting. Testing engineers verify that the bug is fixed. In case it’s not fixed, they return another bug report to developers. Then the team discusses the bug to decide if it’s worth fixing. While critical and major bugs must be fixed, trivial bugs that users won’t even notice may stay in the application. The team makes this decision when fixing the bug requires too much time and  extensive efforts.
5. Regression testing. When all the bugs are addressed, a testing team runs regression test suite again to check that bug-fixing hasn’t destroyed or the hindered application functionality.

Regression testing and project methodology

The way regression testing works also depends on the project methodology. To have a clearer view, let’s consider extreme cases: regression testing in waterfall and Agile.

Regression testing in Waterfall

In waterfall projects, testing starts at the final stage of a project, when the project team isn’t going to modify the product unless it is critical. This stonewall project stability predetermines regression testing peculiarities in Waterfall. Considering the scope of test suite, regression testing makes a lengthy process taking up 1-2 months. However, the major stumbling block here is debugging. If the code was written a while ago, the developers may have difficulties remembering it. What’s worse, a developer who wrote the code may no longer work in the project, which turns debugging in a prolonged guessing game. Besides, change requests in a waterfall project require rework at all project levels and subsequent retesting, which extends the time frame further.

Regression testing in Agile

In Agile, software development process falls into short cycles (iterations or sprints) that last 4 – 6 weeks and result in a complete development of a part of the project. Ideally, test engineers should run regression testing at the end of each sprint. However, this is rarely the case, because in Agile projects teams have to meet tight deadlines and regression testing simply doesn’t fit in. To assure the quality, Agile teams follow two regression testing approaches: full and partial regression testing. Full regression testing takes place before major releases. Partial regression testing is performed at the end of every iteration. It covers only features changed and adjacent areas. Agile teams also use other test prioritization techniques: risk-based approach, collaborative approach and test automation permitting to reduce testing time and effort while preserving the product quality.

Regression testing: how to perform?

Regression testing can be either manual or automated.

Manual regression testing

Manual regression testing is performed by manual testing engineers. In this case, regression testing doesn’t require any testing tools except those for bug reporting (Atlassian Jira, etc.). In some cases, test engineers even use Microsoft Excel to report bugs. The testing process here boils down to identifying the software parts and areas that might be affected by recent changes. The next step is to employ relevant functional test cases to check whether old features work correctly. This fully corresponds to regression testing definition.

Automated regression testing

Automated testing is performed by testing engineers with a knowledge of certain programming languages and a good command of at least one automated testing tool. The choice of a tool depends on the type of software to be tested (desktop, web, mobile application or their combination). Automation engineers don’t invent test cases (scripts). Developing test scripts, they rely on effective manual test cases that repeatedly find bugs. Thus, in regression testing, manual efforts usually precede test automation.

Regression testing cost

According to Statista, testing and QA cost will reach 39% of overall project budget by 2018, and  40% - 70% in this cost belongs to regression testing. As regression testing increases the project cost, eliminating it altogether may seem a tempting step. Tempting, but not really efficient, as regression testing is a gatekeeper preventing bugs from getting into production. Therefore, a wise strategy here is to use specific approaches allowing the team to run regression testing swiftly and efficiently and safeguard the project quality.