All research
note
Attribution is not accuracy
Grounded generation systems are routinely evaluated as if faithfulness to retrieved text were the same thing as being right. It is not. We separate the two and show where the confusion costs you.
- evaluation
- retrieval
- grounding
A retrieval-grounded system can be perfectly faithful to its sources and still be wrong, because the sources were wrong, stale, or the wrong sources for the question. Reporting a single attribution score hides all three failure modes.
Three failures wearing one number
- Corpus error — the passage says it, and the passage is wrong.
- Staleness — the passage was correct on a date that is not today.
- Retrieval miss — a better, contradicting passage existed and was never retrieved.
Only the third is visible to an attribution metric, and only indirectly. We argue for reporting attribution alongside a retrieval-recall estimate and an explicit as-of date, and for treating a high attribution score on an unaudited corpus as uninformative rather than reassuring.
More research
- Spacing intervals at topic level, not card levelAn in-progress evaluation of scheduling review by topic mastery rather than per-item, using StudyFlow's early cohort data.
- Latency budgets for interfaces that feel instantA practical set of per-interaction time budgets, and what we give up to hold them. Measured on the products in this lab rather than borrowed from a 1968 paper.