Every score has a margin around it
A number reported to the decimal place invites a precision that no psychological measurement possesses.
Psychological measurement is measurement, and all measurement carries error. This is not a flaw peculiar to psychology. A weighing scale has error; a thermometer has error. The difference is that people intuitively allow for a little imprecision in a bathroom scale and allow for none at all in a printed test report.
Where the error comes from
Some of it is the instrument: no set of questions samples a mental capacity perfectly. Some of it is the occasion: the student slept badly, or the room was noisy, or they guessed well on two items. Some of it is the person on the day. None of these can be eliminated, and a serious instrument does not pretend otherwise. It quantifies them.
The standard error of measurement is the technical expression of this. It gives the width of the band within which the person's true standing most probably lies, given the score they obtained. A well-constructed test states it in the manual. (AERA, APA, & NCME, 2014) A report that omits it is presenting an estimate as a fact.
What this means when you read a profile
It means that small differences between scores should not be interpreted. If two of a student's scores are close together, and the margin around each is wider than the distance between them, then those two scores are — for practical purposes — the same score. Ranking a student's own abilities in order, from strongest to weakest, is therefore usually an over-reading. Only the differences that exceed the margin are safe to talk about.
If the gap between two scores is smaller than the error around them, the gap is not information. Treating it as information is how a report becomes fiction.
This is the reason a careful interpreter speaks in bands rather than points, and says “clearly above average” rather than “at the 71st percentile”. The band is not vagueness. It is the accurate statement; the point estimate was the vague one, dressed up.
Why the second sitting looks different
Parents frequently ask why a retest gave a different number. Some of the change is real. Much of it is the same error operating twice. This is expected and does not indicate that either result was wrong, nor that the instrument is unsound. It indicates that a single administration is an estimate.
It also explains why serious decisions are not built on one score from one sitting. Where the decision matters, more than one source is used — an instrument, a record of performance over time, and the observations of people who know the student. When these agree, confidence is warranted. When they disagree, the disagreement itself is the finding, and it is worth more attention than any of the three alone.
The practical rule
Read a score as a region, not a point. Ask what the margin is. Do not interpret differences smaller than the margin. And treat any report that gives no margin at all as a report that has not told you how much to trust it.
References
Every work below was checked against a primary or catalogue record. Where a volume or page range could not be confirmed it is left out rather than guessed.
- American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. American Educational Research Association.
