Fix test suite

Who owns automated tests when nobody runs them?

A company can have hundreds of automated tests and still lack effective regression protection. Nobody needs to know when to run them, who reads the result or who decides what to do with a failure. The suite then exists in a repository but has no influence on a release.

The problem is rarely the absence of one “owner of everything”. Test automation has several kinds of ownership: somebody owns the meaning of scenarios, somebody owns operation and somebody owns the code. If those responsibilities are not named, each person naturally expects somebody else to deal with them.

A test that is not used is not a safety net

A test creates value only within a decision flow. A change triggers the check, the result arrives in time, a person understands it and uses it to stop a release, fix the product or update an invalid test. If one link is missing, there is no feedback.

Decline often looks harmless. At first, the suite runs manually before a major release. A run grows to two hours, some scenarios fail intermittently and the report lives in a location known only to the author. The team runs it later, then runs only part of it and eventually stops. When it returns six months later, nobody can distinguish product defects from obsolete locators, data and environments.

The number of files and the last green screenshot do not reveal whether the suite protects today’s product. Better questions are when the entire suite last ran, how many results somebody assessed and which decision it influenced.

The word “owner” hides five roles

A small team can assign all roles to two people. A larger organisation will distribute them. What matters is that every role has a named person or team rather than a general label such as “QA”.

The test author does not have to operate it forever. A supplier can build and maintain a suite, but the company still needs somebody who understands the result and owns the release decision. A DevOps team does not automatically own scenario correctness merely because it operates the pipeline.

Start with an operating agreement

This does not need to be a legal contract. It is a short agreement that answers practical questions:

• which tests run for a pull request, daily and before a release;
• the environment, accounts and data they use;
• where the report lives and how long evidence is retained;
• who receives an alert and when initial classification is due;
• which failures block the next step;
• who may disable a test temporarily and under which conditions;
• how repairs, product defects and accepted risks are recorded;
• who approves adding or removing a scenario.

Keep the agreement close to the suite, for example in its README and pipeline definition, and make it reflect reality. A document promising a daily run does not solve anything if access works only from a former colleague’s laptop.

Match execution frequency to purpose

Not every test must run for every commit. A common mistake is making one entire slow suite a mandatory gate. The pipeline becomes longer, developers begin finding ways around it and ownership dissolves again.

A layered model is more practical:

1. Fast change checks take minutes and protect the most important functions. They run for pull requests or before merging.
2. Broader regression runs after deployment to a test environment, overnight or according to the release rhythm.
3. Specialised scenarios involving external integrations, extensive data or long durations have a separate schedule and owner.
4. Production synthetic checks use safe accounts and operations and belong to operational monitoring, not an unattended general test suite.

Each layer needs an appropriate response time. A failed pull request should provide feedback while the author is still working. An overnight run needs morning triage. A monthly run that nobody opens is merely scheduled infrastructure consumption.

Who responds to a failed test

The person performing the first assessment need not be the person who fixes the test. Their job is to distinguish at least four possibilities: a likely product defect, a bad or obsolete test, an environment problem and an unclear result requiring further diagnosis.

A useful report includes the application version, scenario, failed step, expected and actual result, and proportionate evidence. For a web test, this might include a screenshot, console and network trace; for an API, the response and correlation identifier. Reports must not expose passwords, tokens or personal data.

The team needs a simple escalation rule. When failure relates to the function just changed, the change author or owning product team takes the first look. If it concerns the framework or shared test data, the suite’s technical owner takes over. An unclassified red result should not persist for weeks, because every later result disappears into the same noise.

Temporary disabling needs a return date

Quarantine can be sensible. An unstable test need not block every release while it is being diagnosed. The risk begins when disabling becomes a permanent state with no owner.

For every disabled test, record the reason, linked task, owner and date of the next decision. Only three sustainable outcomes remain: repair and return the test, rewrite it for new behaviour, or remove it by agreement. An endless skipped-test list creates a false picture of coverage.

The criteria for which automated tests to delete help with this cleanup. Deleting a test with no value is better than pretending it exists; removing an important check without replacement is a product risk decision.

Measure use, not only test count

A dashboard saying “850 tests” does not describe suite health. More useful operational indicators include:

• the share of scheduled runs that actually completed;
• time from failure to initial classification and resolution;
• the number and age of quarantined tests;
• rerun rate and flaky-test rate;
• feedback duration for each layer;
• failures resulting in a product fix, test fix or environment fix;
• key risks with no current scenario.

Do not use these numbers to rank individuals. They reveal a process that lacks capacity, information or explicit responsibility.

When nobody uses the suite anymore

The first step is not enabling all 850 tests as a blocking gate. Start with an inventory: when did each test last pass, which risk does it protect, does the journey still exist, and what is needed to run it? Then choose a small group of the highest-value tests and restore the complete flow from execution to response.

If the suite has been failing for a long time, follow a recovery process for an inherited failing test suite. The first stage is not about achieving green at any cost. It establishes a trustworthy baseline: a reproducible environment, understandable report, result owner and known blocking rules.

Add further scenarios by risk after stabilisation. At the same time, remove or archive those whose purpose nobody can explain. Version control retains the history; the pipeline need not carry every past idea.

What clear ownership changes

The team receives feedback while there is still time to act. Failures do not wait in an inbox, exceptions have a name and date, and suite debt becomes visible. Developers do not need to guess whether to ignore a red test, while the product owner knows which risk an exception accepts.

Clear roles also simplify handover between an internal team and supplier. Operation is no longer hidden in the author’s personal habits but described through a process, access model and measurable response.

Next step

Choose one test layer and assign a named person or team to each of the five areas: value, execution, initial assessment, code and the release decision. Run it once and follow every failure until closure. The point where the flow stops without a clear next person is the first ownership problem to repair.