← The notebook
Volume № 01 · § 03 · filed 07 May 2026

Reading the transcript: what reasoning shows that scores cannot

Two candidates, one score, different work entirely. The number says they tied; the reasoning says which one you want.

7 minmethodreasoningreview

Flatten an interview to a number and two very different candidates become identical. One scored a four by racing to a solution and defending it stubbornly; the other scored a four by moving slowly and catching their own mistake. Same number. You want the second one.

§ 01

What the aggregate hides

Candidate 1, reached it fast and defended it hard
4 / 5
Candidate 1, recovered from being wrong
weak
Candidate 2, reached it slowly
4 / 5
Candidate 2, caught their own mistake
strong
weakerstronger
Fig. 1Schematic, not a measurement. Two candidates who tie on the headline score and separate completely on the criterion that would decide it.

This is why every score in Evals ships with the reasoning that produced it, not a summary written after the fact but the criterion-by-criterion account of what the examiner saw and how it weighed it.

The score is the index. The reasoning is the record.
§ 02

Reading reasoning is a learnable skill

01

Look for where the examiner hesitated

Criteria scored with caveats are where the interesting judgment lives. A clean five tells you less than a hedged three.

02

Look at what the candidate did when wrong

No aggregate captures the recovery, and the recovery is most of what you are hiring.

03

Follow surprise into the transcript

When the reasoning surprises you, that is either the examiner catching something you would have missed, or a rubric flaw you now get to fix.

Either way you learn something. A bare number teaches nothing, twice: once about the candidate, once about the bar.

watchglass (2026). Reading the transcript: what reasoning shows that scores cannot. The notebook, volume № 01, § 03. watchglass.app.