Key answer
Express when to check what and what passing means, rather than appending “never do that again.” Checks should address observable behavior, not attitudes.
Full explanation
CheckList combines capabilities with basic cases, changes that should preserve answers and contrasts that should change them, validating behavioral testing in three NLP tasks. Sections 2–3 Adapt failures into a few representative cases.
If AI turns “refund requested” into “refund received,” replace “do not guess/confuse/be careless” with one rule:
For process states, distinguish request, approval and completion. Output states need source support; without a completion record, do not report completion.
Keep three examples: application only, approval, and receipt recorded. Changing names/orders should preserve state judgments; adding trustworthy receipt evidence should change them. This checks understanding and prevents blanket “unknown” responses.
Record four things: trigger, previous error, correct criterion and test cases. Store original input/output separately rather than including everything in every prompt. Missing-information errors need evidence checks; calculation/format errors may suit formulas, scripts or validators.
Convert this error into one check for similar tasks, with the original failing case and a contrast requiring a different conclusion. Merge duplicates in existing rules and do not broaden it to unrelated tasks.
Verify the rule itself, then test old failures and new cases after revision. Low-impact rare details need not enter universal prompts forever. Select relevant checks by task and remove obsolete ones when tasks, models or sources change. A checklist's value is detecting errors, not growing longer.