
The moment I stopped believing my AI
My AI agent told me a task was done. It wasn’t. Not because it lied — because it had no way to know. It had written the code, run it, and the output looked right to it. But “looks right to the model that wrote it” is the weakest possible verification there is. There’s a phrase for this that stuck with me: a model grading its own homework is obviously going to pass itself.
I kept running into the same failure mode. The agent would report “done” with confidence, and I’d find the build broke, a test failed, or the output was subtly wrong. The agent wasn’t malicious. It was just grading its own work with the same eyes that produced it — and those eyes are generous. The more I watched this happen, the clearer it became: the problem wasn’t the model. It was that I had no verification layer between “the agent says done” and “it’s actually done.”
Reported done is not done
I started with a simple rule: there are two kinds of “done.” There’s reported done — the agent claims it finished. And there’s verified — an independent check confirmed it. A task is only done when a gate actually observed the result, not when an agent or script said so. That one distinction changed everything.
The key word is independent. The verification can’t come from the same model that did the work, because it shares the same blind spots. So I built gates that don’t care about the model’s opinion at all: exit codes from builds, type checks, linters, test suites. Deterministic checks that return a number, not a prose summary. The agent can’t argue with a failing test.
The cold reviewer
For the things you can’t check with an exit code — is this research sound? does this claim follow from the result? — I learned a trick from a research harness I studied. The reviewer reads cold: you hand it only the file paths and the raw records, never the agent’s summary of what it did. If you give the reviewer the agent’s own narrative, it just agrees with the narrative. Give it the raw material and it has to form its own opinion.
It sounds obvious, but it’s easy to get wrong. The natural instinct is to have the agent summarize its work and pass that summary to a second model for a verdict. That’s not verification — that’s the first model’s bias wearing a second model’s coat. The reviewer has to be able to disagree, and it can only do that if it sees the same evidence the worker saw, not a filtered version of it.
What I learned
Reliability doesn’t come from a better prompt. It comes from a verification layer that stops a model from grading its own homework. The moment I stopped asking “is it done?” and started asking “what independently confirms it’s done?”, my agents got more trustworthy — not because they changed, but because I stopped believing them on faith.
Do you trust your AI’s “done” — or do you verify it? I’d love to hear how you handle the gap between what an agent claims and what’s actually true.
Wat vond je van dit bericht?