Test an LLM Pipeline Without API Calls in Python

An application that uses a large language model (LLM) usually does more than send one prompt. It may find a document, build instructions, call a model, parse its reply, and reject an unsafe or unsupported answer. If all that work lives in one function, every test may require a network connection, an API key, money, and a model response that can change.

This tutorial builds a small help-center pipeline that can be tested without a model service. We will put the model behind one narrow Python interface, replace it with a fake model, and execute nine predictable tests. A fake is a small object that behaves like an external dependency but returns data that we control.

The lesson is about software boundaries, not answer quality. For model selection and prompt evaluation, see Compare LLM Prompts with a Paired Bootstrap in Python. Here, we ask a different question: can the surrounding Python code handle every expected path correctly?

Prerequisites and Setup

You need Python 3.10 or later and basic knowledge of functions, classes, and dictionaries. The pipeline uses only Python’s standard library. Matplotlib is needed only to reproduce the test-results chart.

Create and activate a virtual environment, then install Matplotlib. On macOS or Linux, run:

python -m venv .venv
source .venv/bin/activate
python -m pip install matplotlib

On Windows PowerShell, replace the activation command with .venv\Scripts\Activate.ps1.

The executed build used Python 3.13.2 and Matplotlib 3.11.0. You can download the two lesson datasets:

• fictional product guides; and
• pipeline test cases.

Both datasets are original synthetic teaching data released under CC0 1.0. They contain no customer records or measurements from a real product.

Mental Model: Test the Fixed Parts Around the Variable Part

Think of the pipeline as a row of five boxes:

question -> retrieve -> build prompt -> model -> validate -> result

Four boxes are ordinary Python. Given the same input, retrieval, prompt construction, parsing, and validation should produce the same result. The model box is different. It is an external service whose output can vary, even when the prompt stays the same.

Testing every box together with a live model makes failures difficult to locate. If a test fails, you do not immediately know whether retrieval chose the wrong guide, the prompt omitted a field, the network request failed, the model returned unexpected text, or validation accepted bad data.

Instead, replace only the model box during most tests:

question -> retrieve -> build prompt -> fake response -> validate -> result

The fake response is controlled input for the next stage. It lets us ask precise questions. What happens when the response is not JSON? What happens when the citation names a guide that was never retrieved? These are tests of our program, so they should not depend on a model deciding to make the required mistake at the right moment.

This boundary does not remove live-model testing. It divides the work. Deterministic tests check the pipeline’s rules on every code change. A smaller integration test later checks that a real client can cross the network boundary and return the expected response shape.

First Separate the Pipeline into Steps

Our fictional product has four short guides. Each row has an ID, a topic word, and some help text:

guide_id  topic     text
G01       password  Reset a portal password from Profile > Security. The reset link expires after 20 minutes.
G02       invoice   Download a monthly invoice from Billing > Documents. New invoices appear on the second day of each month.
G03       export    Export a project table from Data > Export. CSV and JSON formats are available.
G04       member    Invite a project member from Settings > Members. Only project administrators can send invitations.

The example is deliberately small. You can inspect the correct guide by eye before thinking about embeddings or a vector database. The pipeline has four responsibilities:

1. Find a guide for the question.
2. Build a prompt from the question and guide.
3. Ask a text model for a response.
4. Parse and validate that response.

This separation gives each failure a name. No matching document is no_context. Text that is not valid JSON is invalid_json. A valid JSON value with the wrong structure is invalid_shape. A citation that does not match the retrieved guide is invalid_citation. An empty answer is invalid_answer. A valid result is ok.

Named results also help the code that calls this pipeline. A user interface can ask the user to rephrase after no_context, report an invalid model response after invalid_json, and send a suspicious answer for review after invalid_citation. One vague failed result would hide these differences.

Start with a transparent retriever. It looks for the guide’s topic as a complete word in the question:

def retrieve(question: str, guides: list[dict[str, str]]):
    words = set(question.lower().replace("?", "").split())
    for guide in guides:
        if guide["topic"] in words:
            return guide
    return None

This is not a production search engine. Its value is that its behavior is obvious and deterministic: the same input always returns the same result. Later, you can replace this function with database search while keeping the rest of the pipeline unchanged.

Next, make prompt construction a pure function. A pure function calculates its result only from its arguments and causes no external change. That makes its exact output easy to test.

def build_prompt(question: str, guide: dict[str, str]) -> str:
    return (
        "Answer only from the guide. Return JSON with answer and citation.\n"
        f"GUIDE_ID: {guide['guide_id']}\n"
        f"GUIDE: {guide['text']}\n"
        f"QUESTION: {question}"
    )

For the first password question, the executed build created this prompt:

Answer only from the guide. Return JSON with answer and citation.
GUIDE_ID: G01
GUIDE: Reset a portal password from Profile > Security. The reset link expires after 20 minutes.
QUESTION: Where can I reset my password?

Because retrieval and prompt building are ordinary functions, neither needs a fake model. Test them directly with fixed input and expected output.

Define a Small Model Boundary

The pipeline should not depend directly on one vendor’s client class. It only needs something with one method that accepts a prompt and returns text. Python’s Protocol lets us describe that shape:

from typing import Protocol

class TextModel(Protocol):
    def complete(self, prompt: str) -> str: ...

A protocol is an interface based on available methods. A class does not need to inherit from TextModel; it only needs a compatible complete() method. This is called structural typing. The current Python typing documentation explains this behavior and notes that Protocol was added in Python 3.8.

Now create the fake. It stores one response and records the most recent prompt:

class FakeModel:
    def __init__(self, response: str):
        self.response = response
        self.last_prompt = ""

    def complete(self, prompt: str) -> str:
        self.last_prompt = prompt
        return self.response

This object has no API key, network request, or random output. A test can prepare a successful JSON response, malformed text, or a false citation. Recording last_prompt also lets a test confirm what the pipeline sent.

Passing the model into the function is dependency injection. The function receives its dependency instead of creating it internally. A real adapter and FakeModel can therefore use the same pipeline:

def run_pipeline(question: str, model: TextModel):
    ...

If run_pipeline() created a vendor client inside itself, replacing that client would require patching library internals. The small parameter keeps the change explicit.

Parse and Validate at the Boundary

The model response is untrusted text, even when the prompt requests JSON. Parse it inside try and convert known failures into clear statuses:

import json
from dataclasses import dataclass

@dataclass(frozen=True)
class RunResult:
    status: str
    answer: str = ""
    citation: str = ""

def run_pipeline(question: str, model: TextModel) -> RunResult:
    guide = retrieve(question, GUIDES)
    if guide is None:
        return RunResult(status="no_context")

    raw = model.complete(build_prompt(question, guide))
    try:
        parsed = json.loads(raw)
    except json.JSONDecodeError:
        return RunResult(status="invalid_json")

    if not isinstance(parsed, dict):
        return RunResult(status="invalid_shape")
    if parsed.get("citation") != guide["guide_id"]:
        return RunResult(status="invalid_citation")
    if not isinstance(parsed.get("answer"), str) or not parsed["answer"].strip():
        return RunResult(status="invalid_answer")

    return RunResult(
        status="ok",
        answer=parsed["answer"],
        citation=parsed["citation"],
    )

The Python json documentation confirms that json.loads() turns a string into Python data and raises JSONDecodeError for an invalid JSON document. Catching that specific exception is clearer than catching every possible error.

Notice the order. Retrieval stops before the model call if there is no guide. Parsing happens before the code checks whether the result is a dictionary. Only then does it read dictionary fields. The citation must equal the retrieved guide ID, and the answer must be a non-empty string. Prompt instructions alone cannot enforce these rules; Python code can.

The executed successful case returned:

status:   ok
answer:   Open Profile, then Security.
citation: G01

ok means the response passed these structural checks. It does not prove that the prose is ideal or that the underlying guide is correct. Those need separate evaluation and content review.

Execute Success and Failure Tests

One successful demonstration does not test a pipeline. Our CSV contains nine cases: four supported questions, one unsupported question, one malformed response, one response with the wrong guide ID, one JSON array where an object is required, and one empty answer.

Each test creates a fake with the required response, runs the same public function, and compares the actual status with the expected status:

runs = []

for case in CASES:
    model = FakeModel(response_for(case))
    result = run_pipeline(case["question"], model)

    assert result.status == case["expected"]
    runs.append({
        "case_id": case["case_id"],
        "expected": case["expected"],
        "actual": result.status,
    })

The complete artifact build printed this real output:

cases=9 passed=9
T01: expected=ok               actual=ok
T02: expected=ok               actual=ok
T03: expected=ok               actual=ok
T04: expected=ok               actual=ok
T05: expected=no_context       actual=no_context
T06: expected=invalid_json     actual=invalid_json
T07: expected=invalid_citation actual=invalid_citation
T08: expected=invalid_shape    actual=invalid_shape
T09: expected=invalid_answer   actual=invalid_answer

Read each line as a tested contract. T05 shows that an unsupported question stops before the model is called. T06 shows that plain prose cannot pass as structured output. T07 shows that valid JSON is still rejected when its citation is false. T08 rejects a valid JSON array because the pipeline requires an object with named fields. T09 rejects an answer that contains only spaces.

The chart is not a model-quality score. It shows test coverage by outcome. Four successful paths and five distinct stop paths were executed. When you add a new status, add at least one case that reaches it.

Connect a Real Model Later

Keep vendor-specific code in a small adapter. Its job is to translate complete(prompt) -> str into the chosen SDK’s current request and response format:

class RealModelAdapter:
    def __init__(self, client, model_name: str):
        self.client = client
        self.model_name = model_name

    def complete(self, prompt: str) -> str:
        response = self.client.generate(
            model=self.model_name,
            prompt=prompt,
        )
        return response.text

The method names above are intentionally generic because provider APIs change. Follow the official documentation for your chosen provider, then place only that changing code in the adapter. Do not put retries, retrieval, citation rules, or business decisions inside it.

Keep two test layers:

• Fast deterministic tests use FakeModel on every code change.
• A smaller integration suite uses a real model to verify credentials, request format, response format, and model behavior.

The fake proves your Python control flow. It cannot prove that a live model follows the prompt. The integration test proves that connection, but it may cost money and may change across model versions.

Common Mistakes and Troubleshooting

The fake repeats the pipeline logic. A fake should return controlled text, not decide which answer is correct. If it implements retrieval or validation, both production and test code can share the same bug.

Only successful JSON is tested. Include missing keys, empty answers, arrays instead of objects, malformed JSON, unsupported questions, and false citations. This tutorial demonstrates most of these cases; add a separate missing-key case if your caller handles it differently. Test each status your caller handles.

The model client is created inside run_pipeline(). Pass it as a parameter or constructor argument. This makes the dependency visible and replaceable.

A valid citation is treated as proof. This example checks that the citation ID matches the retrieved record. It does not check whether every claim is supported. Production systems may need sentence-level evidence checks and human review for high-impact answers.

The simple retriever misses different word forms. This lesson’s exact-word rule will not match invoices when its topic is invoice. Add explicit normalization, a search library, or embedding retrieval when your requirements need it. Keep retrieval behind its own function so the tests can change with it.

A fake test is reported as model performance. Label deterministic outputs clearly. Report live model name, version, prompt, sampling settings, dataset version, and run date when you evaluate real model behavior.

Recap

A testable LLM pipeline is a group of small steps with a narrow model boundary:

• retrieval and prompt building are independent functions;
• TextModel describes the one model operation the pipeline needs;
• dependency injection lets a fake and a real adapter use the same code;
• JSON, citations, and required fields are checked in Python; and
• failure paths are tested as carefully as successful answers.

Start with the boundary, not a framework. Once deterministic tests protect the surrounding code, you can change the model client or retrieval implementation with much less uncertainty.