Correlation, causation and the career report

Almost everything known in psychology about careers is correlational. People who score in a certain way tend to end up in certain fields; people in certain fields tend to report certain satisfactions. This is genuinely useful knowledge. It becomes misleading at the exact moment it is restated as a causal claim, and the restatement usually happens between the manual and the report.

The three explanations that survive any correlation

When two things are found together, at least three accounts remain available, and no amount of additional correlational data separates them.

The first is that one caused the other. The second is that the causation runs the other way. The third is that something else produced both. In career research the third is nearly always live, because the same background — schooling, family occupation, language, city, income — shapes both what a person scores and what field they can enter.

A finding that engineers score high on spatial reasoning does not establish that spatial reasoning produces engineers. Engineering entrance examinations select for it, which produces the same correlation with the arrow reversed.

Selection is the most common trap

That reversal is worth dwelling on, because it accounts for a great deal of what appears in occupational profiles. Any group that was selected on a characteristic will show that characteristic. Reporting it as though it were a discovery about the occupation is circular, and describing it to a student as a requirement is worse: it converts an artefact of an admissions process into a statement about their prospects.

Restriction of range

A second technical point with large practical consequences. When a group has already been filtered — everyone in the sample got into the course — the variation in the filtered characteristic is compressed, and correlations computed within that group come out smaller than they are in the population (Sackett & Yang, 2000).

This produces a familiar and false conclusion: that the entrance test "does not predict" performance in the course, because among admitted students the relationship looks weak. Among admitted students it must look weak; the ones for whom it predicted poorly were not admitted. The comparison that would answer the question is unavailable, because it was never run.

What a report should therefore say

Careful language on this point is a good indicator of a careful instrument. Compare two sentences that could sit under identical data.

• "Your profile indicates you would succeed in research roles."
• "People whose profiles resemble yours are found more often in research roles than in the population generally."

The second is what the evidence supports. (Messick, 1995) It is longer, less satisfying, and true. A report that consistently chooses the first formulation is making claims its data cannot carry, and the student — who has no way of auditing this — will act on them.

For the person reading a report

Ask of each sentence: is this a description of an association, or a prediction about me? Associations are information to think with. Predictions about an individual, derived from group associations, carry far more uncertainty than their phrasing suggests, and the phrasing is doing the persuading.