QA process maturity: Models and capabilities

HP online survey of IT professionals representing small, medium and large enterprises (2017) reports that in computer software development Agile practices significantly outweigh waterfall approach: 91% vs. 9%. But what does it bring to Agile project teams? A need to work faster with no damage to the quality of the deliverables, which requires evaluating and improving the maturity of QA processes through the internal or external audit. So why QA process maturity is so important, and how to build it? We explore the matter in detail.

Why QA maturity matters?

A mature QA process should fully cover the stakeholder’s needs and ensure the stable delivery of a quality product. But how to make sure the QA process is mature? The telltale sign signaling the process immaturity are customers’ complaints, a common result of delays and various quality issues in Agile projects. So, in case the customers don’t complain about the product quality, is the QA process undoubtedly mature? Not really. What if, at some point, a customer is unhappy with the deliverables? Hectic on-the-spot measures may not work well, as figuring out what to do will require additional time and only enrage the customer. This is exactly the point where a mature QA process can save the day. A mature QA process provides behavior and action patterns for handling possible project challenges. These clear-cut sets of actions help project teams to resolve issues efficiently reducing the time and negative effects.

It looks like a mature QA process is a must-have for any project team, but how to build it? Luckily, there are several QA process maturity models to choose from. We consider them below.

QA maturity models: Overview

QA maturity models can be classified into two broad types according to the degree of customization. These model types are custom models and globally recognized models. Software development companies and testing vendors usually go for one of the two types. Their choice depends on the company specifics. For example, a software development company producing software for the automotive industry may well go for a custom maturity model. For a testing vendor, a globally recognized model makes a better choice, as it can be applied to a wide range of projects. Some adjustments to the customer’s processes or the product specifics may still be required, but their scope is usually bearable.

Now let’s consider custom and globally recognized models in more detail.

Custom models

These models are developed for a particular organization, its business specifics, goals, customers, etc. These models allow for flexibility in assessing the process maturity and mapping the journey to overall quality. Such models make a good option for software development companies that have development and testing in-house.

The key drawback of custom models results from their benefits. No doubt, full reliance on the company’s specifics fits them like a glove. However, when a company needs to cooperate with other vendors or employ specialists for a particular project, their get-in may take some time and efforts, thereby increasing the time-to-market and project costs.

Globally recognized models: TMMi and TPI

Globally recognized models cater for a wide range of companies and largely facilitate cooperation with customers and other vendors as well as employee onboarding. These models concentrate on software testing as the means to deliver a quality product.

TMMi model: QA process perspective

The most common and widely used model is the Test Maturity Model Integration (TMMi). This model is the Capability Maturity Model (CMM) adapted to software testing and QA. TMMi clearly defines the criteria to conform to at every maturity level, and a vendor should be compliant to all process areas of lower stages before moving to the next level. So, if the vendor doesn’t have a managed test environment, but does have a defined training program (see the picture below), they can’t be certified as a level-3 team.

Compliance to the TMMi model requires internal and external process audits. Moreover, to be certified, a vendor should show their processes correspond to a planned TMMi level for at least two releases. So what are the TMMi levels, and what are their benchmarks for QA?

Level 1 (Initial). At this level, testing is purely ad hoc, that is, unstructured and hectic. Level-1 teams don’t care about careful planning or observing standards. Their key focus is to deliver the product on time and preferably without major quality issues. However, the quality of products delivered this way is often far from expected.

Benchmarks for QA:

• Slow and unstable product
• Testing is only an auxiliary to debugging
• No documented testing process

Level 2 (Repeatable). At this maturity level, testing teams already have established and documented procedures, so the processes can be repeated. This facilitates onboarding and resolving the upcoming issues. The main feature here is the existence of the basics of project management.

Benchmarks for QA:

• Testing levels (smoke tests, module testing, integration testing, regression testing and user acceptance testing) are present
• The process is repeatable and well-documented
• Testing is still performed late in the SDLC

Level 3 (Defined)

Level-3 testing teams not only have a documented process but also share knowledge and best practices with their colleagues. They may also have on-the-job training options.

Benchmarks for QA:

• Testing starts at the requirement gathering stage
• Training sessions are in place
• Project-specific testing approaches

Level 4 (Measured)

At level 4, testing teams require some quantitative measures of processes developed and observed at levels 1 to 3. This helps the teams to identify problematic areas in their processes and map the road to further process improvement. As a result, such properties of product quality as reliability, stability and maintainability acquire quantitative equivalents. It’s also advisable to have a closer look at the way the team members follow the adopted practices.

Benchmarks for QA:

• Testing is a measured process
• Regular reviews and inspections in place
• The team aims at making testing more effective

Level 5 (Optimization)

Level-5 testing teams have all the testing approaches and practices well-tuned. However, they don’t stop at this point. They constantly strive for improving their testing process to achieve faster testing, faster time-to-market and reduced costs with no damage to product quality.

Benchmarks for QA:

• Relying on process measurements, the team is able to improve the testing process
• The goals of finding bugs and completing the project on time are now stably reached. Level-5 teams aim at defect prevention and process optimization

Summing up, the TMMi process model pictures the evolution of testing process from an element of debugging to constantly improving QA. So why don’t project teams unanimously go for it? In fact, implementing TMMi requires considerable time and efforts, which may scare off medium-sized and small organizations. Luckily, there are other options available, and one of them is Test Process Improvement (TPI) model.

TPI model: A generous offer

TPI model provides a clear picture of the QA process maturity in an organization and helps devise the ways to improve it and make testing more efficient and less pricey. For this matter, the model singles out twenty key areas that describe the four testing process cornerstones. These cornerstones are a Life cycle (L) of test activities in connection with the development cycle, good quality Organization (O), suitable Infrastructure and tools (I), and Techniques (T) that can be easily applied to testing activities. These key areas cover various aspects of the testing process, from the familiar Test strategy and Test design to the less common Moment of involvement and Office environment. The team assesses each key area and assigns A – D levels of maturity to them, where A corresponds to the lowest maturity level and D to the highest. It’s important that not every key area has a four-level implementation. For example, Office environment only has A-level implementation, while Metrics provides for four levels.

Key areas are also classified into fourteen scales of maturity:

• Scales 1 – 5 – Controlled
• Scales 6 – 10 – Efficient
• Scales 11 – 13 – Advanced

Just like in TMMi, a higher maturity level requires meeting all requirements of lower levels. Besides, for each key area, the TPI model provides a description of what should be in place to reach a level along with a set of checkpoints for each scale. Checkpoints are obligatory requirements for a certain level (A to D). This helps the team to make a grounded decision about the quality of testing in their organization. It’s vital to remember that all key areas cannot have equally high levels of maturity, as the teams prioritize various areas depending on their specifics and/or the project under test.

Having evaluated the testing process, the team proceeds with drafting the TPI matrix that shows the current and the required state of testing.

The matrix is used to visualize the present project state and draft a plan for improvements. Basically, improvements here mean striving for a higher maturity level. The TPI model also provides improvement suggestions on the basis of the evaluation. Unlike the checkpoints, improvements suggestions have a purely advisory character, and it’s up to the project team whether to follow them or not.

TPI models offer an all-round support facilitating the evaluation process. This helps even small businesses to make a legit evaluation of testing process and draft an improvement plan. The evaluation can be run whenever the team feels it is needed. Thus, TPI fosters continuous improvement of the testing process.

QA process maturity: Making a choice

A mature QA process helps project teams mitigate all sorts of unexpected situations that may pop up at any time. But how to set up a mature process? This calls for an adequate QA maturity model that agrees with the business processes and specifics. The key models are:

• Custom models. These models are drafted for a particular company with regard to the business specifics and/or the customer’s needs. At the same time, these models create a certain dependency on a particular customer. So, to move on to another customer, the model may require complete reshaping, which calls for additional input of money, time and efforts from the vendor.
• Globally recognized models that we chose to picture with the help of the popular TMMi and TPI models:
  • The TMMi model. This globally recognized model is a powerful tool for assessing and improving process maturity. However, TMMi is most suitable for large enterprises, as its implementation requires considerable investments in terms of money, knowledge and manpower.
  • The TPI model. TPI model is recognized globally. Unlike TMMi, this model suits any type of enterprise. It provides easy-to-follow guidelines (maturity level descriptions, checkpoints) that help the vendor to assess their process maturity independently and outline the path for further improvement.

The models described are widespread, but not the only ones, so any enterprise engaged in software testing can choose a model that suits their business needs best. This will help reduce the number of unmanageable situations.