Exploratory testing: what automation alone will not find
An automated test reliably repeats the check it was given. It cannot independently judge everything that surprises a user, feels ambiguous or emerges from a combination of states its author did not anticipate. This is where exploratory testing belongs: the tester learns about the product, designs the next experiment and performs it within the same working session.
This is not random clicking without preparation. A well-run exploratory session has an objective, a time box, notes and a debrief. It complements automated regression testing; it does not replace it.
A passing regression suite does not prove a good product
Imagine a checkout where automation verifies the product, address, payment and order confirmation. Every assertion passes. A person may still notice that the mobile keyboard covers the action button, the error message does not identify the field to fix, or returning from the payment provider leaves the basket in an uncertain state.
The automation did not make a mistake. It checked exactly what the scenario specified. The limitation lies in the boundary of a predefined check: if nobody added an assertion for the clarity of the message or a journey involving the browser’s Back button, a green result says nothing about either one.
The more dynamic an application becomes, the more its behaviour depends on the order of actions, data, permissions and previous state. A formally correct response can remain confusing to a user. A requirement can also be implemented precisely even though the requirement itself overlooked an important situation.
What exploratory testing means in practice
In a scripted test, the steps and expected outcome are known before execution. An exploratory test starts with a mission, not a complete click-by-click script. A mission might say: “Explore changing the delivery address after returning from payment and look for states in which the user cannot tell whether an order was placed.”
The tester then works in a short, focused session. An observation from one action influences the next. If double submission produces an unusual state, the tester investigates it, tries navigating back, refreshes the page, switches role or interrupts the network. A fixed script would not take these branches until somebody explicitly designed them.
The ISTQB Advanced Level Test Analyst syllabus describes sessions guided by a test charter and followed by a review of the results. The charter keeps attention on a risk while leaving the tester free to respond to new information.
Where predefined automation often falls short
Exploration provides the most value where correctness cannot be reduced to one value that is easy to compare.
- An ambiguous user experience. The function works technically, but wording, the sequence of steps or feedback leads a person towards the wrong decision.
- Unexpected event sequences. A user opens two tabs, navigates back, refreshes, changes data halfway through a process or resumes after the session expires.
- Combinations of roles, data and states. Individual rules work, but their interaction creates a conflict—for example, a discount, a partially refunded order and a restricted account.
- Weak or missing requirements. Automation can confirm conformity with a specification. It is less suited to discovering that the specification ignored an interruption or gave the user no safe route back.
- Visual and content inconsistencies. Visual regression can compare images, while a person is better placed to judge the meaning of unfortunate text wrapping, misleading copy or an inappropriate information hierarchy.
- New risks after change. A redesign, a new authentication method or an integration change can introduce behaviour that historical tests do not model at all.
Some of these checks can be automated later. Exploration first helps discover what is worth checking. That is a different job from repeating a known expectation quickly.
How to prepare a useful session
A practical session can take 45 to 90 minutes. Longer blocks increase fatigue and make accurate notes harder. Six inputs are enough to begin:
- Risk and mission. Replace “test the profile” with what you want to learn, such as how unsaved changes behave when connectivity is lost.
- Scope. Name the roles, devices, environment and product areas included in the session. It is equally useful to state what is outside it.
- Suitable data. Prepare normal, boundary and incomplete data. A session loses focus if the tester spends half of it waiting for an account or order in the right state.
- Oracle. Decide how behaviour will be evaluated: a requirement, business rule, previous release, comparable feature or a reasonable user expectation.
- Evidence. Concise notes should capture time, data, environment and evidence. For a failure, add steps, a log, screenshot or network request as appropriate.
- Debrief. Separate confirmed defects, requirement questions, ideas for further testing and areas that the session did not reach.
The output is not a click count. It is a better risk map and a decision about what to fix, investigate further or add to repeatable regression coverage.
Combining exploration and automation
The two approaches reinforce each other. Automation can create an account, populate a basket or bring the application into a complex state before the session. The tester avoids repeating known setup and can spend the time exploring new behaviour.
Stable findings flow in the other direction. When exploration reveals an important defect and the team can state the expected outcome unambiguously, it becomes a candidate for an automated regression test. Not every finding needs automation: a one-off cosmetic issue or a check that needs human judgement may remain manual. The same decision criteria apply when considering tests that are not worth automating.
A sensible rhythm might combine a short automated check for every change, targeted exploration of a new or risky feature before a release, and an occasional broader session covering areas the routine regression suite has not touched for some time. Frequency should follow risk and the pace of change, not a universal calendar.
Who should lead an exploratory session
Exploration does not belong only to somebody with “tester” in their job title. A developer knows technical boundaries and can deliberately investigate concurrent events. A product specialist understands business exceptions. A designer is likely to notice ambiguous feedback, while support staff know the situations that cause real customers difficulty.
An experienced tester contributes test techniques, distance from the implementation and a disciplined way of changing one condition at a time. Pair exploration can therefore be especially effective: a tester works with a developer, domain expert or designer. One person provides product context while the other guides the experiment and records findings.
Rotate these roles. A feature author may unconsciously repeat the intended path and miss an alternative, whereas somebody without context may not recognise an important business rule. A short risk briefing and subsequent debrief combine both perspectives without turning the session into an unstructured group review.
For recurring sessions, share the heuristics used and the questions left unanswered. The next person will not start from zero, but still has room to explore by another route and challenge earlier assumptions.
What exploratory testing does not replace
Exploration is not proof of complete coverage. Two sessions run by different people will not be identical, and a finding without notes is harder to reproduce. Results depend on domain knowledge, curiosity and the ability to ask productive questions.
It also does not replace specialised performance measurement, a security assessment, accessibility testing or legal review. It may reveal a symptom, but confirmation often requires suitable tools and expertise. For routine regression, automation provides speed and consistency that a manual session cannot reproduce economically.
Exploration therefore needs an owner and a deliverable. If a team merely says that “someone will click through it”, the activity disappears under schedule pressure and its quality cannot be assessed.
Practical value for the team
A focused session gives the team information it did not have when predefined tests were written. It can uncover a gap in a requirement before that gap produces a series of flawed implementations, or reveal a user problem that technical metrics do not show.
It also makes the automation backlog more precise. Instead of expanding the suite according to the number of available test cases, the team adds checks where it has seen a concrete risk. This combination reduces the false confidence created by a high count of green tests and feeds real usage knowledge back into the test strategy.
Next step
Choose one recently changed or user-sensitive feature. Write a one-line mission, reserve 60 minutes and classify the output as defects, questions, new risks and regression candidates. The first well-documented session will already show what your current automation verifies—and what it currently says nothing about.