PROJECT PREPARATION · JUNE 2026
Describe the task in one clear sentence
Before listing criteria, explain what the reviewer is trying to decide. Comparing two responses for instruction-following is a different task from editing a help article for clarity, even when both involve the same text.
Define the unit of work
State what counts as one item, which context is included and how reviewers should handle missing material. If a response refers to a previous message, say whether that message will be available.
Make the criteria observable
Broad terms such as “good”, “useful” and “natural” need project definitions. Include examples that show the boundary between ratings and explain which criterion takes priority when they conflict.
Specify the source rules
Say whether reviewers may browse externally, use only supplied documents or flag claims for later checking. Define how to record source references and what to do when evidence is unavailable.
Show the required output
Provide one completed example in the format you want returned. It should include a rating or decision, a brief rationale and any uncertainty field the project needs.
Give questions a destination
Name the project role responsible for unresolved decisions and say where questions should be recorded. If the guide changes, use a dated revision and decide whether earlier items need another review.