Where does your data go during AI-assisted testing?
When a language model is involved in testing, you need to know what data you make available to it, where that data is sent and how long it is retained. The answer depends on the model you choose, your contract with the provider and the deployment method. This article explains how to configure AI-assisted testing so that the model receives only the data required for a specific task.
What is the problem?
The concern has a sound basis. With a cloud-based model, requests are processed in the provider’s infrastructure. Data flows may be different with a local or private deployment, but logging, telemetry and access still need to be reviewed.
When working with a browser, the model may see text, element names, page addresses and data entered into forms. Without access to the relevant part of the application, it would merely guess the selectors and steps. The database is therefore not the only source of risk. Sensitive information may also appear on screen, in a log, in an error message or in a test file.
How we address it
A secure setup begins before the first test is generated:
- We define the permitted data and environments. An isolated test environment with generated or properly anonymised data is preferable. Production data and secrets such as API keys and passwords are not copied into the prompt.
- We restrict access. The model and its tools receive access only to the pages, accounts and permissions required for the scenario. Playwright MCP is a working tool, not a security boundary, so we configure its access as carefully as we would a person’s access.
- We review the provider’s terms. We check the processing location, data retention, the use of inputs to improve the model, contractual terms and available data-protection settings. These conditions vary by service and plan.
- We retain control of the output. A person reviews the generated code before it is stored in your repository. If you need an audit trail, you must define in advance which uses of the tools and which types of data will be recorded.
A non-disclosure agreement (NDA) is useful, but it does not replace technical safeguards. Protection needs to be based on data minimisation, an isolated environment, access controls and verified provider terms.
What to watch out for
Anonymisation is not the same as replacing a name with a random value. If a person can still be identified from an order, an address or a combination of other details, the data may remain personal data. For regulated or sensitive systems, the setup should therefore be assessed together with your security and legal teams.
What you gain
- A clear view of what data the model processes and for what purpose.
- A lower risk of unintentionally disclosing customer data, passwords or internal information.
- Tests in your repository that you can continue to run and modify without the AI tool.
Next step
Before introducing AI, list the environments, accounts and types of data the tool may use. If you would like to review the setup for a specific scenario, the next step can be a no-obligation consultation.