Why tests fail intermittently and how to stabilise them
A test fails once and passes when run again without any code change. This is called an unstable or flaky test. If it happens frequently, the team cannot tell whether to investigate the application, the test, or the environment, and feedback from automation loses its value.
The problem: the most common sources of instability
The test is not tied to application state. A fixed wait, such as two seconds, may be enough on a fast machine but not on a busy one. A problem can also arise when the test waits only for an element to appear, not for the operation it needs to verify to finish.
The selector is tied to appearance. An exact path through the HTML or a generated CSS class can change during a redesign without any change in behaviour. The test can no longer find the element even though the user can still see it.
Tests share state. Two concurrent scenarios use the same account, order, or setting. One changes the data that the other currently expects, so the result depends on the order of the runs.
The environment or an external service fluctuates. The test environment may be overloaded, the database may not contain the expected data, or the payment gateway may respond late. If the external service is the subject of the test, this is a relevant result. If it is not, its outage unnecessarily hides the state of your application.
The scenario depends on time and order. A time zone, the end of the month, token expiry, or a test that can pass only after another test all produce results that are difficult to reproduce on a different machine.
How to approach stabilisation
First collect a history: the test name, environment, browser, duration, error message, and rerun result. Only then separate application defects from test and infrastructure problems. You can find a more detailed process in how to measure flaky tests.
Replace fixed pauses with waiting for a specific state, such as a completed response or a visible result of the operation. Before common actions, Playwright automatically checks whether an element is ready, and its retrying assertions can wait for the expected state. This helps, but it cannot fix incorrect data or an unstable service.
Find elements by their meaning to the user, such as a role and name, or by an agreed test identifier. Each test should prepare its own data and be able to run independently of order. Simulate an external dependency only when it is not the purpose of the test; keep integration verification in a separate layer.
Repeated parts of the interface can be separated with the Page Object Model. This pattern reduces duplication, but it is not a substitute for correct waiting and data isolation.
What to watch out for
Automatically retrying a failed test is useful for collecting evidence or temporarily keeping the pipeline usable, but it must not hide the problem. Track how many tests pass only on the second attempt and create a plan to repair them. Depending on the configuration, a trace, screenshot, or video can help with diagnosis; no recording, however, replaces a clear error message and verification of the expected result.
What you gain
- The team distinguishes a product defect from a test or environment problem more quickly.
- Less time is lost to reruns and manual investigation.
- Tests can run more safely in parallel and around releases.
- Trust in the automation results is gradually restored.
Next step
Select the tests that have passed only after a retry in recent weeks and rank them by frequency and impact. If you would like to measure and remove the causes systematically, contact us. We will propose what to fix at the source and what to separate temporarily from the main pipeline.