Home / Vault & Second Brain / A Mirror Only Nods Back

A Mirror Only Nods Back

A brass turnstile gate in a dark void with three glowing bars — one missing — and a mirror reflecting the same gate

Three answers were not enough

For months my verification skill had three outcomes: passed, failed, and unknown. That felt complete. A check either saw something, saw something wrong, or could not see at all. Then I read a short thread from someone running agents in production, and he used three words where I used three: verified, contradicted — and acknowledged. That last one stopped me. Acknowledged: the API said yes, something was written, and nobody ever read it back to compare it against anything.

I recognised it immediately, because it is what most of my own cron jobs do. They write a file, print a line to stdout, and exit zero. Nothing in that sequence is a lie, and nothing in it is proof either. When I counted the jobs on this machine this week, 59 of 211 only ever write to stdout — no durable record, no read-back, nothing to hold next to the result.


Reading is not comparing

The fourth outcome is uncomfortable because it looks like success. The record exists. The file is there. The request came back 200. Every signal a dashboard likes is green — and the gate has still not compared anything, because nobody wrote down what the value should have been before the action ran.

So I added the rule I should have had from the start: if the source exists but the expected value was not fixed before the action, the outcome is not a pass. It is incomparable. And a read-back that fails — timeout, permissions, wrong object — is not a failure either. “I could not look” and “it is wrong” are different claims, and collapsing them into one word is how a broken monitor becomes a confident one.

A mirror only nods back

The second change came from a seven-minute video, of all things. Its opening example: a returns agent is asked whether a final-sale jacket may be refunded. Rule R3 says no. The model says yes. Ask it again — it says yes again. A mirror only nods back. The fix in that video was a verifier built like a turnstile with three bars: the rule lives in code, the reference comes from a source the model did not write, and every write is read back.

I had two of those three bars. The rule was in code, the read-back was in place. The bar I was missing is the middle one: on the jobs where a model grades its own prose, the yardstick sits in the same head as the mistake. And there was a second detail I had never written down — a gate needs to be tested in both directions. A check that has only ever been shown bad input proves that it fires, not that it can ever let good work through. A gate that is always red is as useless as one that is always green, and it takes longer to notice.


What I actually changed

Not a new component. Two fields in a log line I already wrote, plus one habit. The line now carries PRODUCENT= — which model, on which checkout — because a silent model fallback makes an artefact a different artefact, and without that field “this version is good” cannot be told apart from “some other version was good”. It also carries the fourth outcome, and a limit of exactly one revision when a gate sends work back, enforced by a counter in code rather than by an instruction that says “reconsider once”.

The habit is the smallest part and probably the most useful: before a write, write the expected value down — a count, an id, a hash, an amount. “It exists” is not a comparison. “It equals what I wrote down ten seconds ago” is.

None of this makes my agents more capable. It makes them harder to fool — including by themselves. That seems like the better trade.

So here is my question for you: in your own setup, when a job says “done”, what do you actually have to hold next to it? If the honest answer is “a 200 and a good feeling” — what would it cost you to write the expected value down first?

Wat vond je van dit bericht?

Leave a Reply

Your email address will not be published. Required fields are marked *