p95 and the average: why one response-time number is not enough
A performance-test report says that the average response time is half a second. Customers may nevertheless report that some operations take substantially longer. Both statements can be correct: an average alone does not show how the individual response times are distributed.
The value of an average depends on the distribution of the values from which it was calculated. Website response times can have a long tail of slower requests.
This is why response times are measured in percentiles. p95 is the threshold not exceeded by 95% of measured responses; p99 is the equivalent threshold for 99%. Compared with the average, they describe the slower tail of the distribution, but they do not show the individual slowest requests.
What the average hides
Imagine one hundred requests. Ninety-five of them complete quickly because they use a cache. The remaining five go to the database and wait for several seconds. The average for the hundred can still look acceptable even though five measured requests were substantially slower.
The fast cases pull the average down, and the slow cases disappear within one number. The report may then fail to reveal a portion of requests with disproportionately long response times.
An individual customer does not experience the average but the response time of their requests. Percentiles therefore help complete the performance picture, although they cannot by themselves determine the number or behaviour of customers without additional data.
How we measure
We evaluate response times with percentiles and other metrics. p95 means the value below which 95% of the response times in a particular measurement fall. If p95 is two seconds, at least 95% of the measured requests completed within two seconds. This does not automatically mean that the same share of customers had the same experience, because one person may send several requests.
- p95 shows the upper response-time threshold for 95% of measured requests. It is often used as one of the limits for an acceptable response time.
- p99 is the threshold not exceeded by 99% of measured responses. A deterioration can indicate a change in the slower tail of the distribution, which then needs to be investigated with other metrics, logs, or profiling.
We monitor percentiles together with the error rate and throughput. Response time alone is not enough: a system can reject some requests quickly and still report short response times. If system metrics are also available, we compare them on a shared timeline.
This comparison helps form a hypothesis about why p99 deteriorated. The timing relationship still needs to be confirmed with logs, profiling, a configuration review, or a repeated test after a targeted change.
Percentiles can be more sensitive to deterioration before the average changes significantly. We therefore monitor them together with the average, error rate, and throughput, not as the only number.
What you gain
You get a more complete description of the measured response times than a single average provides. When system capacity is evaluated, percentiles help assess the threshold not exceeded by a selected share of requests under a particular load. By themselves, however, they do not show the individual slowest requests or the group of customers to whom those requests belonged.
You also get better evidence for agreeing response-time limits, for example in an SLA. The average can look acceptable even when the threshold of the slower tail deteriorates; p95 or p99 can reveal such a change.
If system metrics are available during the test, we can compare worse response times with, for example, SQL-query duration, waits for database connections, or server utilisation and propose the order of further investigation.
Next step
If you have only an average response time, add at least p95, p99, the error rate, throughput, and the sample count to the report. Always interpret the results in the context of the specific scenario and load. If you are also interested in backend behaviour under load, see API testing.