QA Calibration: Find Where Your Reviewers Disagree

10 Views
0 Uses

Prompt

ChatGPT
Claude
Grok
OpenRouter
You are a QA programme lead. Several reviewers scored the same tickets independently. I need to find where they disagree and why, then fix the rubric.

Our rubric: {{QA_RUBRIC}}
Each reviewer's scores for each ticket, as a table or list (reviewer initials are fine): {{REVIEWER_SCORES}}
The tickets that were scored (customer details removed): {{CALIBRATION_TICKETS}}

Do this:
1. For each ticket and criterion, show the spread of scores, and list the criterion-and-ticket pairs where reviewers differ by more than one point.
2. For each large gap, explain the likely cause: ambiguous rubric wording, a missing "not applicable" rule, a reviewer applying a personal standard, or a genuinely borderline ticket. Quote the rubric line involved.
3. Propose an agreed score for each disputed item, with reasoning tied to the rubric text.
4. Rewrite any rubric wording that caused disagreement, showing the old and new versions.
5. Write a short agenda for a 45-minute calibration meeting covering the three biggest gaps.

Rules: calculate only from the scores given, and show your working. Don't add statistics I haven't supplied. Don't name a "worst reviewer"; focus on the rubric and the decisions. If one reviewer is consistently harsher or more lenient across all tickets, describe it neutrally as a pattern to discuss.

Customize this prompt

Fill in the variables below to personalize this prompt:

ChatGPT
Claude
Grok
OpenRouter

Model Settings

Recommended Models

ChatGPT Claude Gemini

Temperature

0.7

Max Tokens

2000

Additional Notes

Best for: QA leads keeping scores fair across reviewers

About this prompt

“QA Calibration: Find Where Your Reviewers Disagree” is a free prompt for ChatGPT in the Business category, 207 words long, added to The Prompt Index on October 28, 2026 by The Prompt Index. Copy it with the button above, paste it into ChatGPT or any other AI chat assistant, and send it.

It has 3 placeholders to fill in first: {{QA_RUBRIC}}, {{REVIEWER_SCORES}}, {{CALIBRATION_TICKETS}}. Use the “Customize this prompt” fields to personalise it before you copy.

Recommended models: ChatGPT, Claude, Gemini.

0 upvotes, sent to an AI tool 0 times, viewed 10 times.

Browse all Business prompts, the AI prompt database or use the AI prompt finder to search every prompt.