Test strategy and audit

Types of software testing: an overview and when to use each one

Unit, regression, API, acceptance and performance tests are not five mutually exclusive choices. Each name answers a different question: how much of the system is under test, what we want to learn, how the test is performed, or why we are running it at this particular time. This overview helps product specialists, developers and testers describe coverage precisely and choose tests according to risk rather than the most familiar term.

Why one list of test types is not enough

Imagine an automated scenario that calls the order API after deployment, creates an order and checks the final price. It can simultaneously be a dynamic, functional, integration, API, regression and smoke test. There is no contradiction. “Integration” describes the scope of participating parts, “functional” the purpose of the check, “API” the interface, “automated” the method of execution, and “smoke” its role at a particular point in the process.

Without this distinction, discussions become unproductive. A team says it needs more integration tests when the real gap is performance testing. Or it calls every browser test end-to-end (E2E), even though the test uses simulated back-end services and verifies only one front-end component. The result is duplication at one level and blind spots elsewhere.

A good test strategy therefore does not select one “correct type”. It combines several perspectives and uses risk-based testing to decide where each piece of evidence has the greatest value:

Names also differ between organisations. Before using metrics or planning a suite, record the test boundary, the real dependencies and the expected outcome alongside the term. Precise content matters more than the label.

Types by level: from one unit to the complete journey

A unit test verifies a small unit of behaviour under controlled conditions. It might cover a function that calculates VAT, a validation rule, or a class that decides whether a discount applies. It often replaces the database, network or clock with a controlled substitute. Results are usually fast and failures easy to localise, but a unit test alone cannot confirm that the parts communicate correctly.

A component test checks a larger unit through its public boundary. A component might be a front-end form, a library or one deployable service. Its internal parts may be real while remote services are replaced. The label is particularly useful when “unit” means different things to different people.

An integration test checks the connection between two or more real parts: an application and a database, a service and a message queue, a client and an API, or several modules. It reveals serialisation, schema, configuration, transaction and protocol defects that an isolated test cannot see. An integration may be narrow—one boundary only—or broad and span several services. State the scope directly in the test name or documentation.

A system test observes the assembled product as a whole against system requirements.

An end-to-end test follows a complete flow from one end to the other, such as from submitting an order to stock reservation and confirmation. It often uses the UI, but that is not required: a complete flow can also start through an API. Equally, not every UI test is E2E—an isolated form test in a browser may still be a component test. Unit, integration and end-to-end tests compares these boundaries in detail.

The test pyramid without dogma recommends composing a suite from tests of different granularities. It does not prescribe universal percentages. The practical question is: what is the lowest level at which we can verify this risk credibly, and at which boundaries do we still need evidence that the whole works together?

Types by objective: functionality and quality attributes

Functional testing checks what the system should do. It covers, for example, the correct price calculation, rejection of an invalid voucher, user authorisation, or an order moving to an allowed state. Identity coverage must also account for the different flows involved in 2FA, OAuth and social sign-in. Functional testing includes positive cases, error paths, boundary values and business rules. A functional test can run at any level, from a unit test of a calculation to a complete purchase.

Non-functional testing examines how the system performs its task and which properties it has under defined conditions. It is not one collection of “other” tests. Each property needs its own questions, data, environment and criteria:

• Performance testing measures response time, throughput, resource use and stability under a defined load. Performance testing for APIs and microservices explains what to measure beneath the UI, while Core Web Vitals and load tests separate browser experience from back-end capacity. The distinction between load, stress, spike and soak testing lies in the shape and purpose of the load, not merely the number of virtual users. Capacity cannot be inferred without a traffic model; see the practical guide to how many concurrent users a website can handle. Before a run, also decide what can be measured safely in staging and what in production. Evaluate the distribution of timings too, because p95 and the average describe different behaviour.
• Security testing looks for exploitable weaknesses within a defined scope. It may cover permissions, dependencies, configuration, inputs, authentication or resistance to known classes of attack. Automated scanning, code review and penetration testing provide different kinds of evidence. A security test is not automatically a complete security audit or confirmation of compliance.
• Accessibility testing checks whether people with different abilities can perceive and operate the content. An automated check can find only some problems, so an accessibility scanner alone is not enough. A useful combination includes, for example, a manual keyboard journey, checks of accessible names and states, and realistic scenarios such as an accessible checkout and forms. An audit against selected criteria has a broader scope than one automated test.
• Compatibility testing checks supported combinations of browsers, operating systems, devices, resolutions, languages, versions or data formats. Selection should follow the support policy and actual use, not an attempt to cover everything. For the web, a cross-browser testing strategy works alongside checks of locales, currencies, dates, URLs and fallback content. Visual regression testing can add coverage of appearance across combinations. On mobile, combine emulators and physical devices sensibly and check operating-system entry points such as Android notifications and deep links separately. Desktop applications have their own boundaries, covered in WPF and WinForms testing and running WPF and WinForms tests in CI.

Functional and non-functional objectives can meet in one scenario, but their criteria must not be merged. A purchase can succeed functionally yet take an unacceptable amount of time; a fast response may contain the wrong price. Each claim needs separate verification.

Static and dynamic, manual and automated

Static testing evaluates a work product without executing the code under test. It includes reviews of requirements, designs, source code and test cases, as well as automated analysis. It can expose an ambiguous acceptance condition or a risky flow before an executable feature exists. Static does not mean manual: an analyser can work automatically.

Dynamic testing executes the code under test and compares observed behaviour with an expectation. It can be a manual exploratory scenario, a unit test or a load run. A dynamic test demonstrates behaviour under selected conditions; it does not prove that the system behaves correctly for every possible input.

In manual testing, a person performs the steps and evaluates the result. It suits situations where observation, learning and judgement matter, such as exploratory testing of a new feature or assessing whether an interface is understandable. In automated testing, a tool performs a repeatable check. It suits frequent, stable scenarios with an unambiguous result, but requires design, data, an environment and maintenance.

This is not a contest between two teams. Manual and automated testing complement one another, and the decision should be made scenario by scenario. That also applies to a small team without a dedicated tester: responsibilities can be shared, but the need for human judgement does not disappear. Ask when automation is worthwhile and what is not worth automating, rather than targeting an arbitrary automation percentage.

Black-box, white-box and exploratory testing

Other labels describe the information on which a test is based. In black-box testing, the tester examines inputs and observable outputs without deriving the test from the internal implementation. This perspective can be used for a public API, a form or an entire system. Knowledge of the business context is not prohibited; “black box” describes a view of behaviour, not an uninformed person.

In white-box testing, the design of the check uses knowledge of the code or structure. A test may target a particular decision branch, data flow or exception path. Structural coverage can identify unexecuted areas, but executing a line does not show that the test checked the correct result. Grey-box testing is a practical label for a combination: the test uses a public boundary, but scenarios and diagnostics are selected with knowledge of the architecture, database or protocol.

Exploratory testing is a way of working in which learning about the product, designing the test and performing it continually inform one another. A tester may define a time-boxed mission, such as exploring cart behaviour while the price changes concurrently, and adapt the next steps in response to findings. It is neither aimless clicking nor merely repeating a manual script. Automated tools can prepare data, vary inputs and collect evidence while a person directs the investigation.

These terms do not define the level either. A black-box check can be a component or E2E test; a white-box approach can be used in an integration test. When planning, record whether you need to confirm a requirement from the outside, inspect internal structure or discover risks that do not yet have a fixed scenario.

API and UI tests describe the entry point

An API test communicates through a programming interface without clicking through screens. It can prepare input precisely and verify the status code, response body, schema, authorisation and side effects. The API testing checklist and practical guide to REST API testing help with selection. Asynchronous boundaries also need scenarios for signatures, duplicates and delayed webhooks and for transactional email triggers and duplicates. For a third-party dependency, deliberately choose a mock, sandbox or real service. Between services, contract testing provides a different kind of assurance by protecting the recorded expectations of consumers and providers.

A UI test operates or renders the user interface. It confirms that a person can find, complete and use an element, and that the layers below it are connected correctly within the test scope. It is usually more sensitive to data state, timing and interface changes. Sending every combination through a browser is therefore inefficient; why hundreds of UI tests cannot replace API tests explains the difference in coverage.

Both interfaces can be used at different levels. An API test can isolate one service or traverse the entire system. A UI test can check a component or a complete purchase from cart to payment. The parts actually involved, not the driver used, determine the scope.

Smoke, sanity, regression and acceptance tests

These names primarily describe the purpose of a selection and the point at which it runs:

• A smoke test is a short, broad check that a build or deployment is fit for further testing. It might check that the application starts, sign-in works, the catalogue loads and a key API is available. It is not a substitute for in-depth regression testing.
• A sanity test is a narrow, targeted selection after a small change or fix. It confirms that the changed area is basically coherent and worth further testing. Organisations sometimes use smoke and sanity interchangeably; the agreed scope and the gate controlled by the result matter more.
• Regression testing asks whether a change has damaged previously working behaviour. The selection may be small for a pull request and wider before a release or overnight. A good regression pack follows change impact and risk rather than mechanically running everything; it should also be reviewed regularly to decide which tests can be deleted without losing important coverage.
• Acceptance testing checks whether the solution meets agreed needs and conditions for acceptance. It can be automated or manual and operate through an API or UI. An acceptance test is therefore not synonymous with E2E: a simple business rule can be accepted at a lower level if that level produces credible evidence.

These selections can overlap. One key acceptance scenario can also belong to regression, with a shortened version in the smoke suite. In CI/CD, it helps to divide tests by feedback point and speed instead of running one long pack for every change. If the same tests pass locally but fail in CI, examine differences in the environment, data, parallelism and time separately.

How to choose an appropriate combination

Start with a decision, not a catalogue of names. For one feature or change, follow this process:

1. Name the consequence of failure. What happens to the user and the business if the result is wrong, slow, unavailable, inaccessible or exploitable? Consider both severity and likelihood.
2. Divide the flow into rules and boundaries. For an order, price calculations and allowed states are rules; the database, stock system, payment service, email and UI are boundaries. Every critical boundary needs proportionate evidence.
3. Choose the lowest credible level. Discount combinations mainly belong in unit or component tests. Order serialisation and database writes belong in integration tests. A few representative journeys confirm the system or E2E flow.
4. Add quality objectives. A functional scenario does not reveal how many concurrent orders the system can handle, whether it is operable by keyboard, or whether an ordinary user can access someone else’s data. Define separate non-functional checks and criteria for relevant risks.
5. Choose the interface and method. An API is often efficient for many data combinations, while a UI suits key interactions. Consider automating a repeated, stable check; let a person explore a new or ambiguous area. Design the required data deliberately—a production database copy is not a test data strategy, and E2E runs need independent, repeatable test data.
6. Set the run time and response. A fast suite can run for every change, a smoke test after deployment and wider regression according to risk. For every failure, make clear whether it blocks a merge or deployment or requires investigation. Track instability too; intermittently failing tests reduce confidence in the result.
7. Review evidence and gaps. A test needs a meaningful expectation, an owner and a readable result. QA metrics should expose risk and feedback time, not just a growing test count.

One release can therefore include hundreds of fast unit checks of rules, dozens of component and integration tests, contracts between services, a small number of E2E journeys, targeted regression, smoke checks after deployment, and a separate performance or accessibility test. The exact proportions depend on the product’s architecture and risks.

Testing, monitoring and auditing are not the same

Testing creates evidence about behaviour under selected conditions at a particular time. Monitoring continuously collects signals from a running production system to support incident detection and response. A synthetic check may technically execute a scenario similar to an automated test, but its operational purpose, frequency, safe data and alerting are different. See synthetic monitoring after release and the comparison of synthetic and real-user monitoring. A production scenario must also be designed to avoid fake orders and damage to data. An availability signal alone may miss a broken purchase, which is why uptime is not enough.

An audit is a systematic assessment of a defined product, process or body of evidence against agreed criteria. It may use test results, reviews and interviews, but it is not synonymous with a test run. A QA audit, for example, assesses risk coverage, process, environments and suite health; it does not automatically add missing tests or take over continuous production observation.

The three activities complement one another. Tests reduce uncertainty before a change, monitoring shows real production behaviour and an audit helps uncover systemic gaps. One layer should not create the false impression that it has replaced the others.

What precise naming gives you

When a team classifies tests along several dimensions, it can see more easily that it has, for example, many automated UI regressions but no integration checks at the payment boundary or no performance criterion. More precise names also improve diagnosis: a failed narrow integration test points to a different area from an entire E2E flow.

The goal is not a perfectly completed taxonomy. It is a shorter route to a trustworthy result, fewer duplicate checks and deliberate coverage of risks that can affect users or the business.

Next step

Select the ten most important scenarios in the product and create one row for each with six fields: risk, property under test, system boundary, level, interface and run timing. Then mark duplicates, uncovered boundaries and tests without a clear expectation. This small audit will give you a concrete list of moves and additions before you buy a tool or expand the suite.