What is not worth automating?
Automation is sold as a goal: cover everything and leave nothing to people. It sounds good, but in practice it can be one of the most expensive routes to a test suite. Every test used over the long term needs ongoing maintenance, and for some of them the investment may never pay off.
Common problems that need to be solved
An automated test has two costs. The first is a one-time cost: writing it. The second is ongoing: repairing it when application changes affect the scenario, evaluating failed runs, and running it on relevant changes according to the pipeline configuration.
The second cost exists even during periods when the test finds no product defect. This is why an overly broad suite can eventually consume more maintenance effort than the team can reasonably provide—not because the tests were written badly, but because there are more than the team can sustain.
One simple question decides the issue: how many times will you run the test, and what will it save each time?
How to address it: what to leave to people
Scenarios you run twice a year. Annual closing, a data migration, or an unusual configuration for one customer. With this frequency, the cost of creating and maintaining a test may exceed the saving from a few manual checks.
Exploratory testing. A test verifies whether the application does what you wrote into it. By itself, however, it will not find every type of defect that you did not consider when designing the scenario. An experienced tester who spends half a day exploring a new feature can find problems that have no scenario in a prepared script, because the tester looks for what seems strange, not only what violates the specification.
Visual assessment. Whether a button overlaps text, the design is readable, or a form appears trustworthy. A machine can compare screenshots and report that something changed. A person decides whether that change is acceptable. Visual comparison can also be sensitive to small rendering differences, so it needs sensible tolerances and human evaluation.
One-off checks. Verification after a migration, a check after deploying a one-time fix, or testing an experiment that you will either disable or rewrite in a month. In such cases, creating a maintainable test may take longer than the check itself.
Features that will be rewritten in a month. If you know that a screen will change substantially, consider waiting before automating it or verifying the scenario at a lower, more stable level. Otherwise, you may invest in a test that soon loses its value.
User experience. Whether people can navigate the application, whether the process is understandable, or whether an extra step is frustrating. This is a question for a user, not an assertion.
What a sensible scope gives you
A smaller, clearer scope can shorten both runs and maintenance. If the suite also provides reliable results, the team can evaluate failures with greater confidence.
When we design a strategy, the output also includes a list of what not to automate, with the reason. This is not a concession. Deliberately leaving a rarely used scenario for manual verification can save the cost of both creating and managing its test.
Whether automation pays off depends mainly on frequency. For a group of scenarios run only a few times a year, the answer is usually quite clear: automation generally does not pay for itself. You can find a balanced view of the arguments for and against automation in the advantages and disadvantages of automation; if you would like to review your own case, contact us.
What to watch out for
This is not a call to automate nothing. Automated regression testing of critical journeys checked at every release can provide high value. The difference is that this is a deliberate decision based on frequency and risk, not an attempt to reach one hundred per cent coverage because it looks good in a report.
Next step
If you are considering automation and do not know where to stop, contact us. We will review your regression scenarios and propose what to automate, what to leave to people, and why—even if the resulting scope is smaller than you expected.