How to achieve high test maturity using TMM

Let’s answer the obvious question first.

What is Testing Maturity?

The software testing process is an ever-shifting entity. It has to keep changing to accommodate changing requirements, device-browser-OS releases, and numerous other factors that show up with increasing frequency. Inability to change will cause the pipeline to stagnate and roll out software that doesn’t match current performance and design standards.

Testing Maturity designates the extent to which the process is optimized and formalized to accept and incorporate changes. This ranges from ad-hoc practices, formally established stages, result metrics, and optimization efforts on each level. Testing Maturity reveals how well the process is shaped, measured, managed, monitored and the results it yields.

A mature test process generally tends to comprise:

• Established policies: Well-defined and documented testing policies adhered to across an organization.
• Test Planning: A documented test plan and process which specifies objectives, resource allocation, schedules, and a breakdown of tasks.

All you need to know about Test Planning right here.

• Test lifecycle: The step-by-step process consisting of phases and activities. Involves planning, review, design, execution, maintenance, and result analysis.
• Team: The testing team that crafts test cases, runs them, and examines the results.
• Relevant metrics: Test-related metrics required to ascertain test performance and product quality.

Good to know: Essential Metrics for the QA Process

• Testing Tools: The right tools to enable test execution, result reporting, debugging, and documentation. Tools must also allow tracking of results, error recording, and performance evaluation.

Try Testing Your Website for Free

A mature testing process requires support from management and has to be built into the organizational culture. It is capable of continuous growth, improvement, and expansion.

What is Testing Maturity Model (TMM)?

The Testing Maturity Model (TMM) in Software Testing was developed in 1996 at the Illinois Institute of Technology. It lays out a set of levels through which a team or organization can move towards higher test maturity.

TMM in software testing can also be used to evaluate how mature the current test pipelines are. Once the current maturity has been measured, the TMM can also help decide targets to achieve when moving to higher maturity levels.

The Testing Maturity Model is based on the Capability Maturity Model (CMM), created earlier by the same Institute.

Levels of TMM in Software Testing

Every TMM level is a stage of test maturity. It defines testing capability and goals. Each TMM level contains:

• Goals to be met to achieve that particular benchmark of maturity.
• Scope, limitations, and boundaries of the level.
• Tasks and responsibilities required to achieve said goals.

Source

TMM Level 1 – Initial

At this level, testing is inconsistent, disordered, chaotic, and undefined. There is no documentation in place, which makes reusing test cases close to impossible. There are also no standards or templates for QA deliverables. Testing is mostly confined to debugging, without too much focus on overall product quality. The intention at this stage is to run tests without encountering any significant failures.

This stage verifies that the software fulfills the users’ basic requirements. It cannot do more since the testing process at Level 1 lacks more than minimal resources, tools, and skilled personnel. The success of testing depends on individual skills without much team or organizational support.

TMM Level 2 – Managed

At this level, the test process is defined, structured, and established into a cohesive, executable strategy. Test plans, cases, and procedures are put in place with a focus on requirements and specifications. Formal test design methods are used, and process areas are set up:

• Test Policy and Goals
• Test Planning
• Test Techniques and Methods
• Test Environment

The primary intent, at this stage, is to monitor and verify that software doesn’t just function but meets particular requirements. This moves beyond debugging, and actively explores techniques that evaluate software quality.

TMM Level 3 – Defined

At this level, the testing pipeline is integrated into the software development lifecycle. It becomes a legitimate and permanent part of the V-Model (Validation Model). Test phases are conducted in parallel with each developmental phase – as the V model requires.

In each sprint, the sprint development phase is followed by testing. An independent QA team examines the software with predetermined requirements in mind. However, at this stage, testing objectives also take risk management into account.

Start implementing risk-management with risk-based testing in Agile.

Test planning is done at the beginning of project ideation, right after requirements have been outlined. Test strategy is laid out with risk management in mind. Tests are run according to schedule, results are examined to verify that software satisfies requirements and is designed to mitigate risk as far as possible.

The process areas of relevance here are:

• Test Organization
• Test Life Cycle and Integration
• Control and Monitor

TMM Level 4 – Management & Measurement

At this stage, requirements and goals are measured for better management. Reviews and inspection activities are introduced into the lifecycle as a part of testing.

QA activities now include measuring quality markers such as usability, reliability, and maintainability. Bugs are recorded and ascribed levels of severity.

Are you correctly identifying Bug Severity vs Bug Priority in Testing?

Test cases are also documented and chronicled in a singular database to be reused and utilized in regression testing. The intent here is to provide greater visibility and data regarding test process and software quality.

Relevant process areas are:

• Peer Reviews
• Test Measurement & Quantification
• Software Quality Determination

TMM Level 5 – Optimized

At this level, after the steps undertaken in the previous levels, testing has become a comprehensively defined process – result-oriented and highly productive in terms of cost benefits and effectiveness. Techniques, methods, and metrics are validated and consistently applied while making space for continuous improvement in every iteration. Steps are also taken to prevent defects by eliminating obvious inadequacies in processes.

Tools are widely and extensively used to support tests, record, analyze and report results. The entire process operates at the highest possible strata of operation to achieve the following:

• Defect Prevention
• Quality Control
• Test Process Optimization

How to achieve high test maturity using TMM?

Studying the Test Maturity Model in software testing will help QAs, QA managers, and relevant stakeholders identify what their test cycles need to graduate to the next stage of the TMM. To start with, match QA operations with each TMM stage’s elements to figure out what level it is currently at. Then, institute the steps needed to increase test maturity to the next optimal point.

Let’s look at the basics of this evolution:

From Level 1 to Level 2

At Level 1, testing is unsystematic and inconsistent. The entire process is unpredictable, mostly reactive and just lacking in control.

Moving to Level 2 requires basic project management to be put into place. This includes defining and implementing rudimentary processes, standards, and methods – which have to be finalized, documented, and made apt for reusing. This level is called “Defined,” which is why it is all about creating rules and applying them.

Getting the basics right: How to set up QA processes from scratch

From Level 2 to Level 3

Once basic procedures have been instituted, convey them to relevant personnel. Chances are that people will have to be trained to apply the processes and standards effectively and efficiently.

Additionally, QAs have to be motivated, and given all the resources they need to comprehend the new direction testing will take. Conducting webinars and training sessions can be helpful with acquainting them with the newly conceptualized methods. Ensure that they are encouraged to apply these concepts in their day-to-day activities.

Getting to Level 3 requires prioritizing documentation, process standardization, and increased integration of people and pipelines.

From Level 3 to Level 4

At Level 4, take all the processes concretized at Level 3 and measure them in terms of understandable quantification. The key here is to achieve control, not just of the whole pipeline but of individual components and tasks. This allows for accurate and optimal assigning of effort and resources. It also lets managers adjust processes if necessary without negatively impacting product quality.

An easy way to quantify is to divide larger methods into smaller segments and then use quantitative metrics to evaluate each component. Adjust the details as necessary for maximum productivity.

Level 4 is called Management & Measurement for apparent reasons. But it is also sometimes called Predictable because the intent here is to generate enough data to predict what a process needs to work and how to use the prediction for optimal performance.

Start with regular audits of existing processes. Check if teams are following defined procedures as closely as possible, and if not, help them get there.

Level 4 to Level 5

Level 5 is the pinnacle of test maturity. To get to that stage, innovation must take center stage. New modifications must be devised to improve predetermined methods continuously. In other others, it is the application of the Agile mindset to QA operations.

Use the quantifiers established in Level 4, and re-engineer methods to see if they will yield better output. Incorporate new tools, frameworks & technologies. Invest in research, keep up with studies, and remain aware of advancements in the field.

Study other organizations, especially competing ones. Benchmark their methods, learn from them, and use those learnings to innovate and improve. Keep in mind that continuous process improvement is a characteristic of the best QA operations in existence.

Conclusion

TMM in software testing is a handy framework to judge QA processes and develop better ones. However, no matter the level of test maturity, something that is non-negotiable for consistently accurate results is real device testing.

No matter the test, it needs to be run on a real device cloud. This applies to manual testing and automation testing. Without real devices, there is no way to be sure that software is performing as it would in real user conditions such as a low battery, incoming calls, weak network strength, etc.

BrowserStack’s device cloud provides 2000+ real browsers and devices for on-demand testing anytime, from anywhere. Its cloud Selenium grid offers the same number of real devices and browsers for automated Selenium testing. Users can also opt for Cypress testing on 30+ versions of the latest browsers across Windows and macO. Use instant, hassle-free parallelization to get faster results without compromising on accuracy.

Simply sign up for free, select the required device-browser-OS combination, and start testing.