Lesson 4 - Giving the Agent Eyes: Self-Checking

Welcome to Giving the Agent Eyes#

In the last lesson you approved every action, which is safe but doesn’t scale — you can’t sit and click “yes” forever. To let an agent run with fewer approvals, you need it to check its own work as it goes, catching mistakes before they reach you. This is Module 3 — the whole “a loop is only as good as its check” idea — applied to an autonomous loop. The good news is that everything you learned about building honest checks transfers directly; the twist is that now the agent runs the check, so you have to give it something real to check against.

By the end of this lesson, you will be able to:

• Explain why an autonomous loop needs the agent to check its own work
• Give the agent a concrete rule to verify against, not just its opinion
• Rank the kinds of self-check from strongest to weakest
• Apply Module 3’s cautions — ground truth and verifier collusion — to agents

Let’s start with why the self-check is what makes autonomy safe.

The Self-Check Is What Replaces Your Approvals#

Think back to the “verify” step in the agent’s loop — gather, act, verify, repeat. In ask mode, you were effectively the verify step: nothing proceeded without your approval. To move toward hands-off, something has to take over that job, and the only candidate that scales is the agent checking its own work. So the entire question of “can I trust this agent to run on its own?” comes down to one thing: how good is its self-check?

This should feel familiar, because it’s the module-3 lesson in a new setting. There, a weak check meant the loop confidently declared a false “done.” Here it’s the same, but faster and across more files: an agent with a weak self-check will happily report “all sorted!” while three files are misplaced and one was renamed wrong. A strong self-check catches those before you ever look. The agent’s autonomy is only as trustworthy as the check it runs on itself — so that check is what you invest in.

Give It a Rule, Not Just an Opinion#

Not all self-checks are equal, and the ranking is exactly the one from Module 3’s kinds-of-checks: the more your check rests on a concrete rule, the stronger it is.

• Run a rule or command (strongest). Give the agent a check that produces a clean pass or fail: “confirm zero files remain loose in the top folder,” “confirm the index lists every subfolder,” “confirm no file’s contents changed.” These are objective — the agent can’t talk itself past “are there loose files? yes/no.” This is Anthropic’s own guidance for agent loops: the best form of feedback is a clearly defined rule the work either passes or it doesn’t. Where a task has a checkable rule like this, giving it to the agent is the single biggest thing you can do for reliability.
• Re-read the result against the definition of done (solid). Have the agent compare what it actually did to your written criteria, item by item, quoting evidence — the honest-review pattern from Module 3, run by the agent on itself. Good for the parts a hard rule can’t capture.
• Just ask itself “looks good?” (weakest). Letting the agent judge its own work with no concrete anchor is the verifier-collusion trap from Module 3: it shares its own blind spots and tends to approve what it made. Avoid relying on this alone.

The practical move is to write the check into the brief. Alongside the goal, tell the agent how to verify: “When you’re done, confirm every file is in a subfolder and the index lists all of them; if not, fix it and check again.” Now the strong, rule-based check runs on every round, automatically.

The Module 3 Cautions Still Apply — More So#

Handing the check to the agent doesn’t escape the false-done problem from Module 3; it raises the stakes, because now the checker and the doer are literally the same agent. Two cautions carry straight over.

Anchor to ground truth. The agent’s check is strongest when it points at something outside the agent’s own judgment — the actual files, a real count, your written criteria, the source document. “Confirm each renamed file’s new name matches its actual contents” anchors to the files; “does this seem well-organized?” anchors to nothing. Give the agent external things to check against, exactly as you would build a ground-truth check for yourself.

Beware the agent grading its own homework. When the same agent does the work and judges it, it can share the blind spot that caused the error — the verifier-collusion trap. Rule-based checks dodge this (a count doesn’t care who did the work), which is another reason to prefer them. For higher-stakes tasks, you can also add a genuinely independent check — a second pass, a different tool, or your own spot-check of the important parts — the two-hat idea from Module 3, applied to an agent. The more it matters, the less you let a single agent be the only judge of its own work.

A Harborlight Example#

The owner wants the folder-tidy loop to run with fewer approvals, so they strengthen its self-check. Into the brief, next to the goal, they add the verification: “When done, confirm: (1) no files remain loose in the top folder, (2) the index lists every subfolder and its contents, (3) no file’s contents were modified — only moves and renames. If any check fails, fix it and re-verify.” These are rule-based, anchored to the actual folder.

Now the agent runs and, at the verify step, actually checks those rules rather than just feeling finished. On one run it catches that two files are still loose and the index is missing a folder — its own rule-based check flags it — so it fixes both and re-verifies before reporting done. Because the check is concrete and anchored to the real files, the owner trusts it enough to drop from approving every action to a single review at the end. The self-check earned the autonomy. And for the one genuinely sensitive part — the invoices — they still spot-check personally, because that’s exactly where they don’t want a single agent to be the only judge.

Practice Exercises#

Exercise 1: Rank the self-checks#

For “rename each photo to its date,” rank these agent self-checks strongest to weakest: (a) “does the folder look tidy?”, (b) “confirm every filename matches the photo’s date metadata,” (c) “re-read the list of new names against the rule ‘YYYY-MM-DD format.’”

Exercise 2: Write the self-check into the brief#

Add a verification instruction to this goal: “Summarize each document in the folder into a combined summary file.”

Exercise 3: Spot the collusion risk#

An agent writes a data-cleaning script and then “verifies” the data is clean using the same script. What’s the risk, and how would you reduce it?

Summary#

An autonomous loop is only as trustworthy as the agent’s own self-check, because that check is what takes over the “verify” job you did by hand in ask mode — and it’s the module-3 principle in a new setting: a weak self-check means a fast, confident false “done” across many files. The strength ranking is the same as Module 3’s: run a rule or command (strongest — a clean, objective pass/fail the agent can’t fudge, and Anthropic’s own advice for agent loops), re-read the result against the definition of done (solid, for the softer parts), and “looks good?” (weakest — the verifier-collusion trap). The practical move is to write the check into the brief so a strong, rule-based verification runs every round. And the Module 3 cautions apply more, since the doer and checker are the same agent: anchor the check to ground truth (the real files, a real count, your criteria) and, for what matters, add an independent check rather than letting a single agent be the only judge of its own work. A strong self-check is precisely what earns the agent the right to run without your approvals.

Key Concepts#

• Self-check — the agent verifying its own work; what replaces your per-action approvals as you go hands-off.
• Rule-based check — an objective pass/fail (a count, a match, a command) the agent can’t talk past; the strongest kind.
• Check in the brief — writing the verification steps alongside the goal so they run every round.
• Verifier collusion (for agents) — the same agent doing and judging the work, sharing its own blind spot.

Why This Matters#

The self-check is the hinge between “an agent I have to babysit” and “an agent I can trust to run” — and getting it right is entirely an application of the check-building skills you already have. It’s also where a lot of autonomous-AI failures actually come from: not a dumb agent, but a missing or vibe-based self-check that let confident errors through at speed. Give an agent a concrete rule to verify against and you get reliable hands-off work; skip it and you get fast mistakes. You now have goal, safety, a first run, and a self-check. The last piece is making all of this stick to a folder — so you don’t re-explain the brief and the checks every session. Next, you’ll write project memory: CLAUDE.md and AGENTS.md.

Continue Building Your Skills#

You can now give an agent eyes: a rule-based self-check, written into the brief and anchored to ground truth, that catches its own mistakes and earns it the right to run with fewer approvals — while you keep an independent check on what matters most. That self-check is the difference between babysitting an agent and trusting one. Next, you’ll make the whole setup permanent by writing project memory — CLAUDE.md and AGENTS.md — so your brief and checks travel with the folder, and then you’ll run the full module project.