Test data for E2E tests: how to make runs independent and repeatable
An end-to-end (E2E) test checks a complete journey through the interface and connected systems. Even a good scenario will fail if it expects an order another test changed or an account left locked by an earlier run. A reliable suite needs a design for data as much as for test steps.
The goal is not necessarily to erase the whole database after every test. It is to ensure that every scenario knows its initial state, does not collide with a concurrent test, and leaves an understandable trail when it fails.
Why a clean browser is not enough
A modern testing tool can open a fresh profile without cookies or local storage for every test. Playwright, for example, gives individual tests isolated browser contexts. That isolation ends at the browser boundary. Two sessions may still change the same user, basket, stock item, or setting in a shared backend.
The problem often stays hidden in serial runs. With parallel workers, two tests may create the same email address, consume the same voucher, or delete each other’s record. Test-file order must not serve as a data contract: a scenario that passes only after a “preparation” scenario is not independent.
A realistic example: two checkouts and one voucher
The suite contains a successful-order test and a test for rejecting an invalid payment. Both use buyer@example.test, product SKU-100 with the last two units in stock, and one-use voucher E2E10. In serial execution, setup restores the state before each test and everything appears reliable. With two workers, both scenarios read the same stock and try to redeem the same voucher.
The first test reduces stock and consumes the voucher. The second can then fail somewhere other than its intended assertion: instead of a rejected payment, it sees an invalid voucher or unavailable product. A retry after the first worker finishes passes, so the report labels the scenario flaky. The cause is not a slow UI but a data contract that failed to identify the voucher and stock as mutable resources.
The repair separates three kinds of data. The product catalogue can remain shared and read-only, but each worker gets its own stock item or reservation. A factory creates a voucher containing the run ID, and the test account belongs to a specific worker. Cleanup, meaning removal or reset after a test, selects only records with that ID. If the test fails, the identifier on its order connects database state to one CI report.
Define a data contract for each scenario
Record four things for every critical E2E test:
- Initial state: the accounts, roles, products, and settings it expects to find.
- Created changes: the records it adds or modifies and anything it sends to an external system.
- Ownership scope: the data owned only by this test, one worker, or the complete run.
- End of lifecycle: what is deleted, reset, allowed to expire, or deliberately retained for diagnosis.
This contract reveals hidden assumptions early and determines whether a test needs an account per scenario, per worker, or a stable read-only catalogue.
Seed, factory, or API setup
A seed is a predefined baseline created with an environment or database. It suits stable references such as countries, roles, and read-only products. Version it with the schema and make it repeatable. If tests modify it, it is no longer dependable.
A factory creates data for a particular test: a customer with a role, a stocked product, or an order in a requested state. Keep defaults realistic, but make verified properties explicit. A factory with dozens of options can obscure what it created.
API setup is usually faster and more precise than clicking through the UI. Playwright documents using APIRequestContext to prepare server state before opening a page. Use UI setup when creating the data is under test; otherwise a setup-form defect can obscure the intended check.
Direct database writes may fit a controlled environment, but can bypass validation, events, and caches. Put them behind a helper interface, version it with the schema, and confirm they create a state reachable through the application.
Identifiers that are unique and traceable
Build email addresses, user names, or order numbers from the scenario, run ID, and worker index. Randomness can prevent collisions but hampers investigation. checkout-refund_run842_w3, for example, identifies the owning test report.
Playwright exposes worker indices and its parallel execution guide uses them to separate users. The principle applies elsewhere: each concurrent process needs its own mutable state. Share only data that tests treat as immutable.
Store the run ID as a label or record metadata when the domain permits it. Cleanup can then target an exact group instead of applying a dangerous rule such as “delete everything that looks like test data”.
Cleanup is part of the design, not a final safety net
Run cleanup in the test’s final phase (teardown) or through a fixture that manages setup and cleanup, even if the main assertion fails. Report its outcome separately so a cleanup error does not replace the original reason for failure. Before deleting anything, retain the identifiers and diagnostics required to reproduce the problem.
Complete cleanup is not always possible. Audit records may be immutable, an asynchronous job may still use the entity, and a sent email, invoice, or webhook cannot be recalled. Use an isolated test tenant, meaning a separate customer space, a controlled external-service substitute (stub), expiring data, or a scheduled environment reset, and state which side effect remains.
Broadly privileged cleanup can damage other tests. Restrict it to a unique prefix or ID for the current run, and do not connect it to production. For personal data, address minimisation, access, and retention; test data and GDPR explains the strategic boundaries.
A practical minimum model
Most suites can combine a versioned seed for immutable references, a factory or API for mutable records, a unique prefix per run, and targeted teardown. One test should pass alone, in a different order, and concurrently. If it cannot, temporarily using one worker can confirm a collision, but it does not replace isolation.
Give test factories and cleanup code at least small integration checks of their own. Faulty setup can create an impossible state, while faulty cleanup can gradually fill an environment even when the E2E scenarios themselves appear green.
How to verify that the data model works
First run one scenario by itself in a clean environment. It must not require another test or a manually created record. Then repeat it without resetting the complete database; unique identifiers and targeted cleanup must prevent collisions. The third step runs it concurrently with the worker count intended for CI.
Verification should not end with a green report. Query records by run ID afterwards: only data the contract deliberately retains should remain. Confirm that cleanup did not touch reference data or another worker’s records. If cleanup fails, the report should list undeleted IDs so they can be removed safely later.
Run a negative exercise periodically. Deliberately interrupt the test after creating an order, simulate an unavailable API during teardown, or terminate a worker. Check whether cleanup runs, can continue idempotently, and whether scheduled maintenance removes orphaned records. An idempotent step can be repeated safely without creating another unwanted effect.
Track uniqueness conflicts, setup failures, and records older than the agreed lifetime. A zero count in one run is not a guarantee, but the trend reveals whether the environment is accumulating debris. If setup time alone grows, consider data shared only within one worker for expensive resources while preserving separation between workers and resetting the state each test changes.
What you gain
Independent data reduces intermittent failures, enables safe parallel execution, and shortens diagnosis. The identifier shows which test created a record, while the data contract shows what should remain after the run. The suite produces comparable evidence instead of a result shaped by the environment’s history.
Next step
Choose one frequently failing E2E scenario and write down its initial state, changes, ownership, and cleanup. Replace its shared account with a run-specific identifier and prepare state through a targeted API or factory. Then run the test alone, ten times in sequence, and concurrently with a copy; each mode reveals a different kind of hidden dependency.