AI Errors: Measuring Hallucination Rates

Measuring Hallucination Rates

Hallucination rate is a measurement of how often an AI system produces content that is not supported by the provided context or known facts. In practice, you need a clear rule for what counts as “unsupported,” because the same output can be judged differently depending on the task and evidence source. For example, a chatbot that answers medical questions from general knowledge may be scored differently than a model that must quote from a supplied document. A measurement plan also needs a way to handle partial support, where a sentence mixes correct and incorrect claims.

To make this measurable, teams usually define a unit of evaluation such as a claim, a sentence, or a QA pair. Then they label each unit as supported, unsupported, or unverifiable given the test materials. Unverifiable items matter because they prevent unfair penalties when the model cannot access the needed evidence. I’ve seen evaluation spreadsheets where “unverifiable” gets merged into “wrong,” which inflates hallucination rates and makes comparisons meaningless.

Versioning affects results too. If you run the same prompt on a model that changed from a hypothetical “v1.3” to “v1.4” on 2026-03-12, the hallucination rate can shift even when your scoring rubric stays constant. That’s why measurement reports should include model version, decoding settings (like temperature), and the exact test set.

Common Pain Points And Errors

Many evaluations treat hallucination as a single number without specifying the evidence boundary. When the boundary is unclear, “hallucination” becomes a synonym for “I disagree,” which breaks scientific comparison. A second common issue is using a test set that mirrors the model’s training distribution too closely, which can make hallucination rates look better than they are for edge cases. A third issue is scoring only the final answer text while ignoring intermediate steps like retrieved passages, citations, or tool outputs.

Hallucination measurement depends on supporting technologies. Retrieval-augmented generation (RAG) systems rely on a retriever that can miss relevant documents; the generator then fills gaps with plausible text. If you score only the final response, you may attribute retriever failures to “hallucination,” even though the evidence never reached the model. Tool-using systems add another dependency: a calculator or database query can fail, time out, or return empty results, and the model may still produce a narrative explanation.

Another pain point is mixing tasks. Summarization of a provided document and open-ended question answering use different evidence assumptions. In summarization, unsupported claims are often easier to detect because the source text is known. In open-ended QA, the model may be correct based on general knowledge, but your evaluator may mark it unsupported because it was not present in the test materials. This mismatch is why rubric design must match the product’s evidence policy.

Finally, human labeling introduces variability. Two annotators can disagree on whether a claim is “unsupported” versus “unverifiable,” especially for nuanced statements like risk estimates or guideline interpretations. If you do not measure inter-annotator agreement, you cannot tell whether the hallucination rate reflects model behavior or labeling noise.

How To Measure Hallucination

Define The Evidence Boundary

Start by writing a one-page rubric that states what evidence the model is allowed to use. For RAG, the boundary is the retrieved passages; for document-grounded tasks, the boundary is the provided document; for general QA, the boundary is “known facts,” which requires a reference set such as clinical guidelines or curated knowledge bases. Then define labels: supported (explicitly supported), contradicted (explicitly conflicts), unsupported (no support), and unverifiable (insufficient information in the reference set). A practical aside: I’ve found that adding examples to the rubric reduces drift more than adding extra categories.

Choose the evaluation unit. Claim-level scoring is usually more informative than sentence-level scoring because a single sentence can contain multiple claims. If you must score sentences, split long sentences during annotation so that “partly supported” does not get forced into one label. Keep the rubric consistent across runs so that changes in hallucination rate reflect model or prompt changes, not scoring changes.

Build A Test Set With Coverage

Create a test set that includes both typical and adversarial cases. Typical cases validate baseline performance; adversarial cases probe failure modes like missing context, ambiguous prompts, and conflicting instructions. For health-adjacent topics, include items where the correct answer depends on specific qualifiers such as age group, contraindications, or measurement units. A useful number target is 200–500 prompts for a first measurement pass, then 50–100 additional prompts for targeted stress tests. Smaller sets can work for internal debugging, but confidence intervals become wide.

Record the exact inputs: prompt text, system instructions, retrieval settings (top-k, reranker on/off), and any tool calls. If you use a retriever, log the retrieved passages and their IDs so you can audit whether the model had the chance to be correct. Without retrieval logs, you cannot separate “no evidence retrieved” from “evidence retrieved but ignored.”

Score With Human Labels And Checks

Use human annotation for the primary hallucination metric, then add automated checks as a secondary signal. Human labels should follow the rubric and include a short justification for each unsupported or contradicted claim. Automated checks can help with surface-level issues like numeric inconsistency, date mismatches, or citation format errors, but they rarely capture semantic hallucinations reliably. If you use an LLM-based judge, treat it as a noisy proxy and validate it against human labels on a held-out subset.

Track inter-annotator agreement such as Cohen’s kappa or Krippendorff’s alpha, depending on your label scheme. When agreement is low, refine the rubric and retrain annotators. A mild frustration many teams hit: they fix the model before fixing the rubric, then wonder why the “improvement” disappears when they re-score with better labeling.

Report Uncertainty And Breakdown

Report hallucination rate with uncertainty, not as a single point estimate. For a binary supported/unsupported decision per claim, you can compute a confidence interval for the unsupported fraction; for multi-class labels, you can report per-class rates. Also break results down by category: missing evidence, conflicting evidence, numeric claims, and citation errors. This breakdown helps you decide whether to improve retrieval, adjust prompting, or change decoding settings.

Decoding settings affect output diversity. Higher temperature can increase lexical variation and sometimes increases unsupported claims, though the direction depends on the task. If you run experiments, keep temperature and top-p fixed across comparisons, and document them. I’ve seen teams change temperature and retrieval top-k at the same time, which makes it impossible to attribute the change in hallucination rate.

Educational Case Examples

Example 1: Document Summarization With Missing Sections. A team tests an AI summarizer on a set of policy documents. The rubric labels claims as supported only if the exact idea appears in the document. In 60% of failures, the model invents a detail that is not present because the source document omits that section. The team then adds a “no evidence, say you don’t know” instruction and changes the prompt to require quoting key phrases. The hallucination rate drops, but the “unverifiable” rate rises because the model refuses to fill gaps.

Example 2: RAG Q&A With Retrieval Gaps. Another team evaluates a Q&A assistant that retrieves up to top-5 passages. The rubric treats claims as supported only if they match retrieved passages. Many unsupported answers occur when the retriever returns irrelevant passages due to a query mismatch. After they adjust retrieval (for example, increasing top-k from 5 to 10 and enabling a reranker), the unsupported rate decreases, while the contradicted rate stays similar. This pattern suggests the main issue was evidence retrieval rather than the generator’s tendency to fabricate.

Checklist And Comparison Table

Use the following checklist to decide whether a hallucination-rate measurement is interpretable.

1. Write a rubric that defines supported, contradicted, unsupported, and unverifiable.
2. Log the evidence available to the model (retrieved passages, document IDs, tool outputs).
3. Score at claim level when possible, or split sentences into atomic claims.
4. Use at least 200 prompts for a first estimate, then add targeted stress tests.
5. Measure inter-annotator agreement and revise the rubric when agreement drops.
6. Report hallucination rate with uncertainty and a breakdown by error type.

Common Mistakes That Skew Rates

One mistake is scoring only the presence of “I don’t know” refusals rather than measuring factuality. A model can refuse often and still produce unsupported claims when it does answer. Another mistake is using a single judge model to label hallucinations without calibration. LLM judges can be consistent yet wrong in systematic ways, especially when the rubric requires strict evidence matching.

Teams also overfit the test set. If you iterate prompts based on the same labeled examples, the hallucination rate can improve on that set while degrading elsewhere. A practical fix is to keep a held-out evaluation set that you never touch during prompt tuning. I’ve seen evaluation dashboards that quietly reuse the same 100 prompts across months, which makes the trend look stable while the real-world error rate drifts.

Another error is ignoring numeric and unit claims. Hallucinations often appear as incorrect dosages, dates, thresholds, or units. If your rubric treats all claims equally, numeric errors can be diluted. Add a category for numeric claims and score them with stricter matching to reference values.

Finally, do not treat “unverifiable” as “supported.” If the evidence boundary is retrieval-based, unverifiable claims often indicate retrieval failure or missing context. Mixing these categories changes the meaning of the hallucination rate and can lead to the wrong engineering decision.

FAQ

What Counts As A Hallucination?

A hallucination is a claim that the model outputs without support from the defined evidence boundary, or that contradicts the reference. The rubric must specify whether “unverifiable” claims are separate from unsupported claims.

How Do I Choose A Test Set Size?

For a first estimate, 200–500 prompts often give a usable starting point, then you add 50–100 targeted cases for stress tests. Smaller sets can still help debugging but produce wide uncertainty intervals.

Should I Score Claims Or Sentences?

Claim-level scoring usually yields clearer error analysis because one sentence can contain both supported and unsupported parts. If you must score sentences, split long sentences into atomic claims during annotation.

Can Automated Judges Replace Human Labels?

Automated judges can screen outputs, but they rarely replace human labels for the primary metric. Validate the judge against human labels on a held-out subset and report the judge’s error rate.

How Do Retrieval Failures Affect Hallucination Rates?

In RAG systems, retrieval gaps can cause unsupported answers even when the generator follows the evidence boundary. Log retrieved passages and score supported versus unverifiable to separate retrieval issues from generation issues.

Author's Insight

Hallucination-rate measurement works only when the evidence boundary is explicit and consistent across runs. Most “mysterious” differences between reported rates come from rubric drift, test-set mismatch, or unlogged retrieval/tool behavior. A careful evaluation separates unsupported claims from unverifiable claims so you can diagnose whether the system failed to retrieve evidence or failed to reason over evidence. When you report results, include uncertainty and a breakdown by error type, because a single percentage hides the failure mode that matters for safety and trust.

Key Takeaways

• Define hallucination using a written evidence boundary and a label set that includes “unverifiable.”
• Score at the claim level when possible, and log evidence inputs like retrieved passages and tool outputs.
• Use a test set with both typical and adversarial coverage, then report uncertainty and error breakdowns.
• Calibrate any automated judge against human labels, and keep a held-out evaluation set to avoid overfitting.