What a percentile actually tells you

A report comes back and says the student is at the 78th percentile on numerical reasoning. Almost everyone who reads that sentence takes it to mean one of two things, and both are wrong. It does not mean the student got 78 per cent of the questions right. It does not mean they are 78 per cent good at numerical reasoning. It is not a measure of the student at all, in isolation.

What it does mean

A percentile is a statement about position in a group. The 78th percentile means that the student's score was equal to or higher than that of about 78 per cent of the people in a particular comparison group. That is the whole of it. Change the comparison group and the number changes, though the student has not changed at all.

This is the first practical consequence and it is a large one. A percentile is meaningless until you know what it is a percentile against. A student at the 78th percentile compared with all school students of their age is describing a different situation from a student at the 78th percentile compared with candidates who have already qualified for a competitive programme. Any report that gives a percentile without naming the comparison group has withheld the information needed to read it.

Why the middle is crowded and the ends are not

The second consequence is less obvious and causes more misreading. Percentiles are not evenly spaced in terms of actual ability. Most people cluster near the middle of the distribution, so a small change in raw score in the middle produces a large jump in percentile, while the same change near the top or bottom barely moves it.

In practice this means that the difference between the 45th and the 60th percentile is often a couple of extra correct answers, whereas the difference between the 95th and the 98th represents a much larger real gap. Two students separated by fifteen percentile points in the middle of the range may be, for every practical purpose, the same. Families read the gap as meaningful because the numbers look far apart.

A percentile tells you where someone stands in a queue. It does not tell you how far apart the people in the queue are.

What it is useful for

None of this makes the percentile a bad statistic. It is useful precisely because it is a position, and position is often what the decision needs. If the question is whether a student's verbal reasoning is unusual for their age, a percentile answers it directly and a raw score does not. If the question is whether verbal sits well above numerical for this particular student, comparing the two percentiles answers it, provided both were normed on the same group.

The comparison within a single student's profile is usually where the useful information sits. A person's shape — which of their scores stand higher than the others — is more informative for a career conversation than their overall level, and the shape is what percentiles make visible.

What to ask when you are handed one

• Against whom? Which group was the norm sample, and does the student resemble it? (AERA, APA, & NCME, 2014)
• How old are the norms? A comparison group assembled long ago may no longer describe the population the student belongs to.
• What is the margin? A percentile is derived from a score, and the score has measurement error around it, so the percentile does too.
• Is this the same norm group as the other scores in the report? If not, the profile shape cannot be read.

A percentile is a good tool used within its limits and a misleading one outside them. The limit is simple to remember: it is a fact about a group, reported about a person.