Key answer
Identify the failed acceptance condition before diagnosing the cause. “Not good” does not distinguish expression, sources, understanding and capability. Multiple causes may coexist.
Full explanation
*Sufficient Context* separates sufficient information from correct open-ended answers: models can fail with sufficient material and sound certain with insufficient material. Avoid blaming every error on prompt wording. Sections 3–4
Start from a concrete error:
· Goal/evaluation mismatch: describe the expected output, such as beginner instructions rather than an expert review; resolve reviewers' brevity/completeness tradeoffs.
· Missing material: determine whether necessary information was supplied and accessible. Rephrasing cannot invent a real budget.
· Misreading: locate the source sentence, field or unit; check negation, confused people and table alignment. Partial recall is not full understanding.
· Execution/capability: once goals and sources are clear, inspect calculation, reasoning, tools and output, using calculators, tests or small tasks.
· Faulty evaluation: verify reference answers and grading methods. Anthropic recommends inspecting traces to distinguish agent mistakes from graders rejecting valid solutions. Engineering guide
This result changes “refund requested” into “refund received.” Locate the supporting source and determine whether it describes request, approval or completion. Correct it and check other statuses for the same mistake.
This supplies location and evidence. If the cause remains uncertain, retain the case and change one condition, such as supplying a missing field. Simultaneous source/model/criteria changes obscure causation. AI's account of why it erred is a diagnostic hypothesis; confirm through inputs, records and retries.
References
2. Anthropic (2026), Demystifying evals for AI agents, Step 6: Check the transcripts