Key answer
Score resemblance and correctness separately. Tone, phrasing and structure concern imitation; facts, calculations, conditions and conclusions concern correctness. Both can improve, or one can improve while the other deteriorates.
Full explanation
Min et al. found that demonstration format, input distribution and label space could help without complete dependence on correct input-label mappings. Learning an output's appearance does not establish learning the right rule. This comes from classification and multiple-choice experiments, not evidence that all examples teach only format. Paper
Anthropic's evaluation guide also separates task quality from tone/style and recommends realistic tests and explicit criteria for comparing versions. This is evaluation guidance, not proof of a perfect prompt. Official guide
For customer service, check unsupported refund promises, review durations and omitted eligibility conditions separately from friendliness and structure. A friendlier reply containing an unconfirmed arrival date should be recorded as “style improved, factual error,” not concealed by an overall impression.
Evaluate this reply: first check each fact and promise against the material, then assess tone and structure. Report these separately; resemblance to the example cannot offset factual errors.
Keep new cases out of the prompt for testing, including ordinary and easily confused exceptions. Hold other settings fixed and compare repeated outputs with and without examples. AI-assisted scoring saves time, but key facts still require original-material checks; “entirely correct” is not acceptance evidence by itself.
References
2. Anthropic, Define success criteria and build evaluations, official guide