Lesson 4 - Grounding and Citations

Welcome to Grounding and Citations#

You can retrieve now. The knowledge base searches by similarity, and in the last lesson the agent could call retrieval as a tool mid-loop. But retrieval alone isn’t trustworthy yet. Hand a model some passages and ask it a question, and it will happily blend the passages with whatever it thinks it knows — and when the passages don’t actually cover the question, it falls back to its old habit: a fluent, confident guess. The missing piece is discipline. A grounded agent answers only from what it retrieved, cites which passage each claim came from, and refuses — honestly — when nothing relevant comes back. This lesson builds that discipline into one function, and it’s what turns “the model read some text” into “the agent gave a checkable, trustworthy answer.”

By the end of this lesson, you will be able to:

• Build a grounding gate that refuses out-of-knowledge questions before the model is ever called
• Prompt the model to answer only from numbered sources and cite them inline like [1]
• Set a similarity floor that cleanly separates real matches from keyword noise
• Explain why an honest refusal is a feature, not a failure

Let’s start with the gate that prevents hallucination.

The Grounding Gate#

The single most important line in a grounded agent is the one that decides whether to answer at all. Before you build the prompt, before you call the model, you check what retrieval came back with. If the knowledge base has nothing relevant, you don’t hand the model an empty context and hope — you refuse, immediately, without calling the model. A model that is never called cannot hallucinate.

That’s the grounding gate. Search first, inspect the top score, and only proceed to the model if the match is real:

MIN_SCORE = 0.15  # similarity floor: real matches score 0.28+, keyword noise <0.14

def answer_with_citations(client, kb, question, *, system, k=3, model="claude-haiku-4-5"):
    hits = kb.search(question, k=k)
    # Grounding gate: if nothing relevant, refuse instead of hallucinating.
    if not hits or hits[0][2] < MIN_SCORE:
        return {"answer": "I don't have that in my knowledge base.",
                "sources": [], "grounded": False}

    context = "\n".join(f"[{i}] ({src}) {txt}" for i, (src, txt, _) in enumerate(hits, 1))
    prompt = (f"Answer using ONLY the sources below. Cite them inline like [1]. "
              f"If they don't cover it, say you don't know.\n\nSources:\n{context}\n\n"
              f"Question: {question}")
    r = client.messages.create(model=model, max_tokens=512, system=system,
                               messages=[{"role": "user", "content": prompt}])
    answer = "".join(b.text for b in r.content if b.type == "text")
    return {"answer": answer, "sources": [(src, sc) for src, _, sc in hits], "grounded": True}

Read the flow top to bottom. kb.search returns hits sorted best-first, each a (source, text, score) tuple. The gate checks two things: are there any hits, and is the top score at least MIN_SCORE? If either fails, the function returns a refusal with grounded=False and never touches the client. Only when a real match exists do we build a numbered context block and prompt the model — instructing it explicitly to answer from those sources and to cite them inline. The return value carries the answer, the sources (with their scores, for display or audit), and a grounded flag the caller can trust.

The Similarity Floor#

Why 0.15? The floor exists because retrieval always returns something — even a query with no real match scores against the closest passage by accident. You need a threshold below which a “match” is just keyword noise, not a real answer. With this module’s keyword embedding, the numbers separate cleanly:

• Genuine matches — a Kyoto-foliage question against the Kyoto passage — score roughly 0.28 to 0.34.
• An on-topic-but-weaker passage, sharing a word or two, lands around 0.12.
• A pure hash-collision of unrelated text — “scuba diving coral reefs” against a travel guide — tops out near 0.135.

So a floor of 0.15 sits comfortably above the noise and below the real matches: it lets the genuine answers through and rejects everything else. The boundary isn’t magic — it’s measured. With real semantic embeddings the gap between signal and noise is even wider, so the floor is easy to set; you tune it per corpus by looking at the scores your real questions produce versus the scores your nonsense questions produce, and drawing the line in between.

Two Outcomes, Both Verified#

The same function handles both the question it can answer and the one it can’t — and the difference is visible in the return value and in how many times the model was called. Both outcomes below were verified for real against an SDK-shaped mock client (no API key needed): the retrieval, the grounding gate, the message assembly, and the refusal path are all exercised exactly as written.

Outcome A — grounded. Ask "When does Kyoto's autumn foliage peak?". Retrieval finds the Kyoto passage well above the floor, so the gate passes, the model is called exactly once, and the result comes back grounded=True with kyoto-guide in its sources and an inline [1] in the text:

out = answer_with_citations(client, kb, "When does Kyoto's autumn foliage peak?",
                            system="You are Atlas. Ground every claim in the sources.")
# out["grounded"]  -> True
# out["sources"]   -> [("kyoto-guide", 0.31), ...]
# out["answer"]    -> "Autumn foliage in Kyoto peaks in mid to late November [1],
#                      with mild temperatures of 10-18°C ideal for temple walks [1]."
#                      (example — exact model wording varies; the [1] citation is real)

Outcome B — refusal. Ask "What is the capital of Brazil?". Nothing in this travel knowledge base matches, so kb.search returns no relevant hit, the gate trips, and the function returns the honest refusal with grounded=False. Crucially, the model is called zero times — verified with client.calls == 0:

client2 = ScriptedClient([])  # empty script: the model must NOT be called
out2 = answer_with_citations(client2, kb, "What is the capital of Brazil?",
                             system="You are Atlas.")
# out2["grounded"] -> False
# out2["answer"]   -> "I don't have that in my knowledge base."
# client2.calls    -> 0   (gated before hitting the model)

That client.calls == 0 is the whole point. The refusal isn’t the model deciding to decline — it’s your code refusing on the model’s behalf, before the model can invent anything. The Claude API call in the grounded path is standard and correct; only the illustrative answer text varies run to run, which is why it’s labeled an example.

Practice Exercises#

Exercise 1: Trace the gate#

Walk through answer_with_citations for a question whose best retrieval score is 0.13. Does the model get called? What does the function return?

Exercise 2: Why inline citations?#

The prompt asks the model to cite sources inline like [1] rather than just listing sources at the end. Why does inline citation matter more than a bibliography?

Exercise 3: Tune the floor for a new corpus#

You point the agent at a different knowledge base and find that genuine matches now score around 0.45 while noise tops out near 0.30. Should you keep MIN_SCORE = 0.15? Where would you set it?

Summary#

Retrieval gives the agent sources; grounding discipline makes it trustworthy. answer_with_citations searches the knowledge base, then runs a grounding gate: if there are no hits or the top score is below MIN_SCORE (0.15), it returns an honest refusal without calling the model at all — a model that’s never called can’t hallucinate. When a real match exists, it builds a numbered context block and prompts the model to answer using only those sources and to cite them inline like [1], returning {answer, sources, grounded}. The floor of 0.15 is measured, not guessed: genuine matches score ~0.28-0.34, keyword noise tops out near 0.135, so the line sits cleanly between them. Two outcomes — a grounded, cited answer with exactly one model call, and a refusal with zero model calls — were both verified against an SDK-shaped mock.

Key Concepts#

• Grounding gate — check retrieval before the model; refuse out-of-knowledge questions so the model is never called and can’t hallucinate.
• Inline citations — prompt the model to answer only from numbered sources and tag each claim [1], making the answer auditable claim by claim.
• Similarity floor — a measured threshold (0.15 here) that separates real matches from keyword noise; tune it per corpus.
• Refusal is a feature — “I don’t have that in my knowledge base” earns more trust than a confident fabrication.

Why This Matters#

Grounding and citations are what move a retrieval agent from “demo” to “deployable.” Every production RAG system — support bots, legal assistants, internal copilots — lives or dies on this discipline: answer from sources, show your work, and refuse when you can’t. A retrieval pipeline without a grounding gate is a hallucination machine with extra steps, because the model will still guess on the questions you can’t answer. With the gate, citations, and a tuned floor, you have an agent whose every claim points back to a source you control — and that’s exactly what a real product needs.

Next Steps#

Continue Building Your Skills#

You can now make an agent answer only from what it retrieved, cite each claim back to a numbered source, and refuse honestly when the knowledge base comes up empty — gating the model behind a measured similarity floor so it never gets the chance to hallucinate. Next you’ll combine everything in this module — the knowledge base, agentic retrieval, and this grounding discipline — into Atlas, a complete retrieval-augmented agent.