When human reviewers disagree about the right AI answer

An AI evaluation may count every mismatch as a model error while ignoring that two qualified reviewers would label the same example differently. The team then optimises against a target whose meaning is unclear.

Inspect the disputed examples

Ask reviewers to label an appropriate sample independently before discussing it. Record the reason for disagreement rather than only the final majority choice.

Datasheets for Datasets proposes documenting dataset composition and collection. Annotation procedures and their limits belong in that record.

An illustrative customer-support dataset may use “urgent” to mean either commercial importance or immediate service risk. The label cannot support a reliable evaluation until the team resolves that distinction.

Improve the instruction without erasing uncertainty

Write clearer definitions and examples where possible. Preserve an ambiguous category or escalation route when the evidence does not support one correct answer.

For consequential domains, involve the appropriate specialists. Agreement among general annotators is not a substitute for qualified judgment.

Keep changes versioned. Relabelling difficult cases after seeing a model’s predictions can distort comparisons if the revised dataset is not applied consistently.

Report what the evaluation establishes

Distinguish performance on clear cases from performance on disputed cases. Explain whether the deployed workflow requires automatic action or supports a human decision.

Use disagreement to improve the product’s questions and inputs. Sometimes the right response is to request missing context rather than demand that the model infer an answer.

Keep a small set of disagreement examples with the annotation guide. New reviewers can use them to understand the boundary and the accepted escalation route. The examples should explain the reasoning rather than teach staff to reproduce a label without understanding the ambiguity that made the case difficult.

An evaluation becomes stronger when its ground truth can be explained. The goal is not perfect agreement at any cost, but a defensible account of what counts as correct and where legitimate uncertainty remains.