How to Verify LLM Claims Against Trusted Data with Python

Valid JSON can still contain a false claim. Imagine a large language model (LLM) returning {"workshop_code":"WS103","attribute":"format","value":"online"} for a workshop assistant. The fields have the expected names, but the trusted catalog says that WS103 is a hybrid workshop.

To verify structured LLM claims against trusted data, parse each output as untrusted JSON, require the expected fields, find the named entity in an authoritative dataset, and check that the citation points to that entity’s record. Normalize the claimed and trusted values with the same type-specific function, then approve only exact matches and route every other case with a clear reason.

This tutorial implements that narrow check without an API key or another model. It does not decide whether arbitrary prose is true. Instead, it checks one structured attribute at a time against a trusted source that your application controls.

Setup and Teaching Data

You need Python 3.10 or later and basic knowledge of functions and dictionaries. The verification code uses Python’s standard library. Matplotlib is needed only to reproduce the result chart.

Create a virtual environment, activate it, and install Matplotlib. On macOS or Linux, run:

python -m venv .venv
source .venv/bin/activate
python -m pip install matplotlib

On Windows PowerShell, use .venv\Scripts\Activate.ps1 for the activation command. The executed build used Python 3.13.2 and Matplotlib 3.11.0.

Download these files and save them beside your Python program:

• trusted workshop catalog
• saved structured LLM claims
• executed verification results

The catalog and claim cases are original synthetic teaching data released under CC0 1.0. Synthetic means they were created for this lesson. The workshop provider, record IDs, schedules, fees, languages, access details, model outputs, and mistakes are fictional.

The second file uses JSON Lines, often called JSONL. Each line is one complete JSON value. In this dataset, an outer test record stores a case ID, the expected route, and model_output, which is the raw string that an application received from a model. This design lets one record contain a deliberately malformed model response without breaking the whole JSONL file.

Understanding the Verification Gates

A schema answers a structural question: does this output have the fields and types that the program expects? It does not answer the factual question: do those fields agree with trusted data? The structured JSON tutorial explains how to enforce the first boundary with an API and Pydantic. Here, we start after generation and enforce the second boundary with ordinary Python.

One claim moves through these gates in order:

raw model text
  -> valid JSON object with four required keys
  -> known workshop and approved attribute
  -> citation belongs to that workshop
  -> claimed value has the required type
  -> claimed value equals the catalog value

Each gate has a separate failure status. This is more useful than one failed result. Invalid JSON points to generation or parsing. An unknown workshop may be an invented identifier. A citation mismatch shows that the evidence pointer is wrong. A value mismatch shows that the record exists but does not support the claim.

Order matters. If the workshop code is unknown, there is no safe catalog row to read. If the attribute is not approved, the program should not accept an arbitrary column name. If the citation names a different row, matching a value somewhere else in the catalog must not rescue it.

A verified result is also limited. It means one allowed attribute matches one catalog row after normalization. It does not prove that a longer answer is complete, helpful, or free of unsupported prose.

Load the Catalog and Inspect a Saved Output

Load the CSV into a dictionary keyed by workshop_code. A dictionary makes the entity lookup explicit: catalog["WS103"] returns the trusted row for WS103. Python’s official csv documentation recommends opening CSV files with newline="" and provides DictReader for rows with column names.

The following code also reads each JSONL envelope. At this stage, it parses only the trusted test file, not the nested model output:

import csv
import json


with open(
    "trusted_workshop_catalog.csv",
    encoding="utf-8",
    newline="",
) as handle:
    catalog = {
        row["workshop_code"]: row
        for row in csv.DictReader(handle)
    }

with open("saved_llm_claims.jsonl", encoding="utf-8") as handle:
    cases = [json.loads(line) for line in handle]

print("catalog rows:", len(catalog))
print("claim cases:", len(cases))
print(cases[0]["model_output"])

The executed code printed:

catalog rows: 8
claim cases: 15
{"workshop_code":"WS101","attribute":"format","value":"In person","citation":"CAT-001"}

The first model-output string contains four fields. workshop_code identifies the entity. attribute identifies the fact being asserted. value contains the proposed fact. citation points to a stable catalog record.

Do not trust this string because it appears inside a valid outer record. It still came from the model boundary. The next step parses and checks it separately.

Normalize Values Without Hiding Differences

Text can differ without changing meaning. In person, in_person, and in person should compare as the same format. Likewise, the JSON number 27.5 and the CSV text 27.50 represent the same fee.

Normalization converts equivalent representations into one comparable form. It must be narrow. Converting case and whitespace is reasonable for a weekday. Removing words from a workshop title could hide a real difference.

Start with text, money, and boolean normalizers. Decimal(str(value)) avoids converting a binary floating-point approximation directly into a decimal value:

from decimal import Decimal, InvalidOperation


def normalize_text(value):
    if not isinstance(value, str) or not value.strip():
        raise ValueError("expected non-empty text")
    return " ".join(
        value.casefold().replace("_", " ").split()
    )


def normalize_money(value):
    if isinstance(value, bool) or not isinstance(
        value, (str, int, float)
    ):
        raise ValueError("expected a decimal number")
    try:
        return Decimal(str(value)).quantize(Decimal("0.01"))
    except InvalidOperation as exc:
        raise ValueError("expected a decimal number") from exc


def normalize_bool(value):
    if isinstance(value, bool):
        return value
    if isinstance(value, str):
        choices = {
            "true": True,
            "yes": True,
            "false": False,
            "no": False,
        }
        normalized = value.strip().casefold()
        if normalized in choices:
            return choices[normalized]
    raise ValueError("expected true/false or yes/no")

Time needs a different rule. Parse a 24-hour time and normalize it to the zero-padded HH:MM form:

from datetime import datetime


def normalize_time(value):
    if not isinstance(value, str):
        raise ValueError("expected time text in HH:MM form")
    try:
        parsed = datetime.strptime(value.strip(), "%H:%M")
        return parsed.strftime("%H:%M")
    except ValueError as exc:
        raise ValueError("expected time text in HH:MM form") from exc

Now connect each approved attribute to its parser. This allowlist is both a data contract and a safety boundary:

PARSERS = {
    "format": normalize_text,
    "weekday": normalize_text,
    "start_time": normalize_time,
    "fee_eur": normalize_money,
    "language": normalize_text,
    "step_free_access": normalize_bool,
}

REQUIRED_KEYS = {
    "workshop_code",
    "attribute",
    "value",
    "citation",
}

The catalog contains workshop_name, but it is not in PARSERS. This checker therefore refuses claims about that field. Add an attribute only after deciding how to normalize it and adding cases for matching, mismatching, missing, and badly typed values.

Verify One Claim and Preserve the Reason

The verifier returns a pair: a machine-readable status and a short reason for a person. First parse the raw model string. The current Python json.loads() documentation states that invalid JSON raises JSONDecodeError, so catch that specific exception.

Next, enforce the exact keys before reading them. Requiring exact equality rejects missing and unexpected fields instead of silently ignoring them:

def verify_output(raw_output, catalog):
    try:
        parsed = json.loads(raw_output)
    except json.JSONDecodeError:
        return "invalid_json", "output is not valid JSON"

    if not isinstance(parsed, dict) or set(parsed) != REQUIRED_KEYS:
        return (
            "invalid_shape",
            "output must contain exactly the four required keys",
        )

    workshop_code = parsed["workshop_code"]
    if not isinstance(workshop_code, str) or workshop_code not in catalog:
        return (
            "unknown_workshop",
            f"{workshop_code!r} is absent from the catalog",
        )

    attribute = parsed["attribute"]
    if not isinstance(attribute, str) or attribute not in PARSERS:
        return (
            "unsupported_attribute",
            f"{attribute!r} is not an approved attribute",
        )

Only after those gates pass does the function retrieve the trusted row. It checks row identity before comparing values, then uses the same parser on both sides:

    record = catalog[workshop_code]
    if parsed["citation"] != record["catalog_record_id"]:
        return (
            "citation_mismatch",
            "citation does not identify the workshop's catalog row",
        )

    parser = PARSERS[attribute]
    try:
        claimed_value = parser(parsed["value"])
        trusted_value = parser(record[attribute])
    except ValueError as exc:
        return "invalid_value", str(exc)

    if claimed_value != trusted_value:
        return (
            "value_mismatch",
            f"claim={claimed_value!s}; catalog={trusted_value!s}",
        )

    return (
        "verified",
        f"matched {attribute} in {record['catalog_record_id']}",
    )

Using the same parser prevents one-sided cleanup. For example, K04 compares the JSON number 27.5 with the CSV text 27.50. Both become Decimal("27.50"), so the harmless representation difference passes. K09 compares 25.00 with 18.00, so it remains a real mismatch.

Run Every Route and Read the Results

Apply the function to every saved case. An assertion checks the implementation against the expected teaching label. In a production run, new claims will not have expected_status; keep labels in a separate reviewed regression set.

results = []

for case in cases:
    status, reason = verify_output(
        case["model_output"],
        catalog,
    )
    assert status == case["expected_status"]
    results.append(
        {
            "case_id": case["case_id"],
            "status": status,
            "reason": reason,
        }
    )

The complete artifact build produced this real summary:

catalog_rows=8 cases=15 matched_expected=15
verified=6
value_mismatch=3
unknown_workshop=1
unsupported_attribute=1
citation_mismatch=1
invalid_value=1
invalid_shape=1
invalid_json=1

matched_expected=15 means every deterministic case reached its intended branch. It does not measure a model’s factual accuracy. Six claims passed this checker, while nine were routed away from automatic approval.

Four selected cases show how the reasons help diagnosis:

K01: verified | matched format in CAT-001
K07: value_mismatch | claim=online; catalog=hybrid
K12: citation_mismatch | citation does not identify the workshop's catalog row
K15: invalid_json | output is not valid JSON

K07 is structurally valid and cites the correct row, but its value conflicts with that row. K12’s weekday matches the WS101 row, yet its citation points to the record for WS102. The checker rejects it because a matching value with the wrong evidence pointer is not a verified result.

Read the green bar as claims that passed this exact contract. Read each red bar as a separate work queue. A value mismatch may need corrected generation or refreshed data. Unknown identifiers and citation mismatches may point to grounding problems. Invalid JSON and shape failures belong to the model-output parsing boundary.

Put the Check After Generation, Not Inside the Prompt

Prompt instructions can request citations and trusted values, but Python should enforce the final rule. Save the raw response first, then parse and verify it before it reaches a user or database:

model response -> structured claim verifier -> approved claim
                                      \-----> review or rejection

For a real application, store the knowledge-base version with every decision. A catalog can change after a model response is generated. Without a version, a later rerun may compare the old claim with new data and produce a different result.

Also store the status, reason, entity ID, attribute, citation, model name, prompt version, and time. Do not log secret values or personal data merely because logging is convenient. Set retention and access rules for the data your application actually handles.

Use reviewed test cases before changing a normalizer or adding an attribute. If the task also uses retrieval, measure whether the right evidence was found separately. The offline RAG evaluation tutorial shows why retrieval, citations, claim support, completeness, and safe abstention should remain separate checks.

High-impact decisions need more than this program. Route ambiguous claims, changing policies, and consequential outcomes to a qualified reviewer. Deterministic checks are valuable because their limits are visible, not because they replace judgment.

Common Mistakes and Troubleshooting

Treating valid JSON as factual support. JSON validity proves only that text can be parsed. Check entity identity, citation ownership, allowed attributes, value type, and value equality separately.

Comparing every value as text. The strings 27.5 and 27.50 differ even though they describe the same amount. Apply an attribute-specific parser to both the claim and trusted value. Do not apply one broad cleanup rule to every field.

Accepting any requested attribute. A model could name a private, deprecated, or meaningless column. Keep a small allowlist such as PARSERS, and review each addition.

Checking the value before the citation. A matching value can occur in several records. Confirm that the citation belongs to the named entity before comparing the field.

Converting invalid values to defaults. Turning "free" into 0.00 would hide a malformed fee. Return invalid_value and preserve the reason instead.

Reading the live catalog without a version. A later update can make old results impossible to reproduce. Save a catalog version or immutable snapshot identifier with the run.

Using assert as the production guard. Python can skip assertions when run with optimization. Assertions are useful here for development cases. Production routing must use the explicit return status, while automated tests should use a test framework or checks that cannot be disabled.

Reporting verified as “the answer is true.” This verifier checks one approved attribute. It cannot assess unsupported sentences, missing details, reversed meaning outside the structured field, or whether the trusted catalog itself is correct.

The result chart fails on a server. Select Matplotlib’s Agg backend before importing matplotlib.pyplot, then save the figure instead of opening a window. The current Matplotlib backend documentation lists Agg as a non-interactive backend.

Recap and a Safe Next Step

Structured output makes a claim inspectable, but it does not make the claim correct. A dependable verification path parses the raw JSON, enforces an exact contract, locates the entity, validates the citation, normalizes the selected attribute on both sides, and approves only a supported match.

The 15 executed cases exercised eight outcomes. Six matched the fictional catalog, three contained different values, and six more reached distinct identity, citation, type, shape, or parsing failures. Those separate statuses turn a vague “model error” into evidence that a developer or reviewer can act on.

As a next step, add one new catalog attribute and four labeled cases: a direct match, a meaningful mismatch, an invalid type, and a citation mismatch. Choose its parser deliberately. That small exercise preserves the central rule: narrow claims, explicit evidence, and visible failure reasons.