Order-dependent tests: how to find and fix them
A test passes alone but fails in the full suite, after a particular scenario, or once parallel execution is enabled. The likely cause is a hidden dependency rather than randomness. Diagnosis must separate order dependency from a concurrency collision.
When an earlier test changes the next result
An order-dependent test inherits state from an earlier test. It may need another scenario to create a user, or receive an account, basket, or global setting its predecessor changed. This can fail with one worker because changing the order is enough.
It may pass alone in a clean environment but fail after one particular test. Conversely, it may pass only after a “preparation” test because it does not create the required data.
The failure appears in the second test even though the first created its cause. Running that scenario alone hides the problem and sends the team towards CI or timing.
Order dependency is different from a concurrency collision
A concurrency collision occurs when two tests change the same resource simultaneously. They may pass serially in either order, yet overlap in parallel. One worker deletes a file, occupies a fixed port, or changes an account while another uses it.
Both defects stem from shared mutable state. Changing the order exposes state from a completed test, while changing the worker count exposes overlapping tests. Serial execution may therefore conceal the collision rather than repair it.
Playwright runs test files in parallel by default and tests within a file in declaration order. It recommends isolated tests over interdependent serial groups and does not guarantee file order during parallel execution. See the Playwright parallelism guide.
An example with a shared account and basket
Consider two end-to-end (E2E) scenarios using e2e-shopper@example.test. Test A empties the basket, adds a desk lamp, and checks the price, but does not remove the item. Test B signs in to the same account and expects an empty basket with a catalogue link.
Test B passes alone but finds the lamp after A and fails. The reverse order passes because A clears the basket first. This is order dependency: completed test A leaked state into test B.
Now make B empty the basket before adding an office chair. Both tests pass serially in either order. Concurrently, B may clear the basket after A adds the lamp. A then misses its item, or B finds both products. This is a concurrency collision that reversing the serial order may not reproduce.
The same pattern occurs with a shared user, order, voucher, database record, export directory, or fixed port used by a local service.
Use a diagnostic matrix instead of random reruns
Keep the commit, application build, configuration, data, and tool versions fixed. Change only the execution mode. This matrix turns an ambiguous failure into a testable hypothesis.
| Execution | What to observe | Likely conclusion |
|---|---|---|
| Suspect test alone with known state | Whether it needs another test’s setup or already fails by itself | Failure points to its own setup, the application, or the environment |
| Suspect test immediately after a possible predecessor | Whether state from the first test changes the second result | A repeatable difference supports an order dependency |
| The same pair in reverse order | Whether the problem follows one sequence | One failing order identifies who leaves or expects the state |
| The complete target group in serial | Whether failure disappears without temporal overlap | Improvement is evidence of a collision, not a finished repair |
| The same group with intended concurrency | Whether tests compete for an account, file, port, or record | Failure only under concurrency supports the collision hypothesis |
For every run, retain the first error and data identifiers. In the basket example, record the account, run ID, worker, and items before decisive steps. A screenshot alone shows the symptom, not who made the change.
Change one condition at a time
Changing the order, worker count, account, and cleanup together makes a green result inconclusive. State the hypothesis: “Test B fails when test A leaves an item in the shared basket.” Keep both tests and the application unchanged, then reverse only their order.
For a concurrency hypothesis, keep the data and sequence fixed and change only the worker count. Next, preserve the intended concurrency but give one test a different account. The result should follow that condition. This also applies when tests pass locally but fail in CI.
Give every test an explicit initial state
A durable repair starts with explicit setup. Test B must create a customer with an empty basket or establish that state through an API or fixture, not rely on A. Use the interface only when basket creation is under test. Otherwise, a test interface is usually shorter and more precise.
Assign mutable data to one test or worker. Include the run ID, scenario name, and worker index in a traceable identifier. Tests may share a stable read-only catalogue, but not an account, basket, or voucher they change. See test data for E2E tests for the wider design.
Selenium recommends not sharing test data, cleaning up stale data another test could acquire, and creating a WebDriver per test. This does not require a database copy per scenario. Isolate only mutable state.
Isolate browsers, files, and ports
A new account does not remove cookies from an earlier sign-in. Playwright creates an isolated browser context per test, with separate cookies, local storage, and session storage. Reusing one page or context across tests bypasses this incognito-like boundary. A clean context still does not isolate backend data.
With Selenium, create a new driver per test and quit it even after failure. Its fresh browser guidance says a new instance of a common browser driver starts with a clean user profile by default.
Resources outside the browser need boundaries too. Save exports in a test-owned directory, not shared downloads/report.csv, and put the run ID in temporary names. Assign ports to local servers through an allocator or a range reserved for that run and worker. Processes cannot share one fixed port.
Cleanup is a safety net, not the next test’s prerequisite
Run cleanup after passes and failures, restricted to records owned by that test or run. Make it repeatable without letting its error replace the original failure. Retain diagnostic IDs before deletion.
The next test still must not rely on its predecessor’s cleanup. A worker may stop, a service can fail during teardown, and a sent email cannot be reversed. Explicit setup provides the initial state. Targeted cleanup prevents data accumulating.
When a sequential workflow belongs in one test
Some sequences are genuine workflows. A user may create an order, pay, then request a refund. If the complete workflow is under test and each step needs the previous result, use one test with one state and outcome.
Splitting that journey into three tests creates a chain whose second part cannot run alone. A payment failure also invalidates the refund result. Helper functions can keep steps readable, while one test owns the lifecycle. Separate tests should prepare a paid order without another scenario.
How to verify the repair
Run the repaired test alone with known state, after its original predecessor, and in reverse order. Repeat the group serially and with the CI worker count. Beyond a green result, confirm that each test used its own ID, directory, port, and browser session. Only deliberately retained data should remain.
Keep the original failing condition as a reproduction experiment. If the shared-account pair still fails while separate accounts pass under the same concurrency, the cause is better supported than by one successful retry. In larger suites, track first attempts and group matching failures. See how to measure flaky tests.
What you gain
Independent tests can run alone, in another order, and in parallel. Results describe application behaviour from a known state, not a basket or directory’s history. Safe concurrency can shorten execution, while each failure has a smaller search area.
Next step
Choose a pair where one test passes alone and fails in the suite. Run all five matrix modes and record “condition, mechanism, symptom”. Isolate mutable data, then repeat the original failing condition. If similar dependencies affect more of the suite, contact us. We can measure and repair them before regular CI execution resumes.