
The anomaly appears later
Last week my home server ran at a load of 11.8 for three and a half hours and nobody noticed. Not my monitoring, not my reflexes, not me. The culprit was a runaway headless Chromium process — a leftover from an AI coding tool — chewing 957% CPU. And here’s the part that kept me up: the input that started it was completely normal.
That’s the uncomfortable truth about securing AI agents that I keep running into. A security researcher put it better than I ever could: “Detection becomes difficult when the user’s request is normal and only the retrieved content is hostile. There may be no suspicious login, no exploit signature, and no obviously malicious prompt. The anomaly appears later.”
I’ve spent the last few weeks auditing my own AI setup — the files my agents read, the MCP servers they’re connected to, the tools they can call. Classic security thinking says: scan the input, look for the signature, catch the bad prompt. But my runaway Chromium proved that’s the wrong layer to watch.
The input was fine. The behavior was not.
In a retrieval-driven agent system, the attack itself is invisible to classic detection. The user request is normal. The retrieved content — a document, a web page, a tool description — is hostile. There’s no exploit signature to match, no suspicious login to flag. The anomaly doesn’t show up in the input; it shows up later, in what the agent does.
My own incident was the same shape. The Chromium process started from a legitimate command. Nothing about the input was suspicious. But three hours later a gpu-process was eating nearly a thousand percent of a core, and my load-based monitoring — which only reports load as a change, not as a value — never fired. The signature-based reflex I’d built simply couldn’t see it.
Shift from suspicious input to anomalous behavior
The lesson I’m taking away is a shift in where I point my detection. Instead of asking “was the input suspicious?”, I should be asking “did the agent do something unexpected after a normal input?” That means watching behavior, not just scanning content:
- An agent that suddenly calls a privileged tool after a routine retrieval — that’s a signal, even if the prompt looked clean.
- A process that spikes CPU hours after a normal command — that’s a signal, even if the command was legitimate.
- A tool description that reads like an instruction — that’s a signal, even if it came from a server I trust.
This is the same principle on every layer: the anomaly appears later, so your detection has to live where the behavior happens, not where the input arrives.
The question I’m sitting with
If “the anomaly appears later” is the core problem — and my own incident proves my system monitoring has the same blind spot as my input scanning — then how do you build one behavioral-anomaly detector that works on both levels? One that catches an unexpected privileged tool call after normal retrieval, and a CPU spike from a legitimate-looking process? Is there a single “baseline of normal behavior” that covers both, or are these fundamentally two different monitors you can’t pull onto one model?
I don’t have the answer yet. But I’m convinced the question is the right one — because the input was fine, and the anomaly still appeared. Have you caught an anomaly that your input-scanning missed? I’d love to hear how you’re watching behavior instead of just scanning content.
Wat vond je van dit bericht?