Why a model sounds more certain when it's wrong than when it's right

An agent that can act needs boundaries: validate inputs, filter outputs, stop dangerous actions, handle errors cleanly. Guardrails turn an experiment into a system you can trust with real work.

A production agent can move money, delete data, send emails to customers. What it can't do is just as important as what it can.

Guardrails operate on multiple layers: prompt constraints, tool whitelists, output filters, rate limits, sandboxed execution. And the most important guardrail of all: threat modeling before the system goes live.

The defence ladder

The most common attack on an agent is also the cheapest: you talk to it. Prompt injection means smuggling instructions into a place where the model expects data - a customer message, a document, a web page some tool just fetched. The model doesn't distinguish between "this is my assignment" and "this is the text I'm supposed to talk about". To it, both are the same thing: characters in a context.

Defending against it isn't a decision, it's a ladder. Every rung adds exactly one layer, and every layer has the attack it breaks on:

Level	Defence	What breaks it
0	none	just ask
1	hardened system prompt	role play, reframing
2	output filter, hard-wired	encoding, acrostics, letter by letter
3	output filter, model-based	paraphrase, spreading it across a story
4	input filter, hard-wired	synonyms, another language
5	input filter, model-based	harmless-looking tasks with a hidden payload
6	everything at once + domain restriction	little left but narrative

The ladder has an axis that's easy to miss on first reading: input versus output and hard-wired versus model-based. A hard-wired filter looks for characters - it's fast, free and blind to meaning, which is why Base64 slips past. A model-based guard understands meaning - it's slow, costs a second model call, and can itself be talked around, because it too is just a model with a prompt.

And: input filters act too early to know what will happen, output filters too late to stop what already has. That's why the top of the ladder isn't a better layer, it's all of them at once.

What 279,000 real attacks show

You don't have to speculate about how well these layers work. In Gandalf the Red, Lakera analysed the attacks on a public prompt-injection game - 279,000 real attempts by real people against systems defended in graded steps. Three of their findings are the difference between game knowledge and professional knowledge:

• Layering works, single layers don't. None of the layers holds on its own; each has its attack. Only the combination drives the success rate down noticeably - not because any layer gets better, but because an attack suddenly has to outwit all of them at the same time.
• Domain restriction is the single strongest measure. A bot that may only talk about one topic is far harder to hijack than a jack-of-all-trades. Not because it's better guarded - but because there's barely a way to address it outside its topic in the first place.
• Defence in the system prompt costs usability, even when it blocks nothing. Every "never do this" sentence makes the model more cautious, more terse, more suspicious of harmless questions. Security and usability are one joint objective, not two separate ones. Which is why a game where only security counts is no template for production guardrails.

The OWASP Top 10 for LLM applications has listed prompt injection as LLM01 since its first edition - the top spot, and unchanged there in the 2025 edition. Its recommendations read like the ladder above, with one explicit caveat: no single measure solves the class. OWASP talks about mitigation, not about a fix.

An example from our own house

The most effective measure appears on no filter list, because it isn't one: user text does not belong in the system prompt. That is exactly what this site's comment agent does in src/lib/comments/agent.ts - the system prompt is a fixed string, and the comment follows as a user message:

/**
 * System prompt of the comment agent. The comment itself goes into the model
 * as a user message, NEVER in here - that dampens prompt injection:
 * instructions inside the comment stay data.
 */
export function buildCommentAgentPrompt(candidates: SearchDoc[]): string {

The difference is structural, not cosmetic. If the comment text were interpolated into the system prompt, an "ignore all previous instructions" would sit on the same level as the operator's own instructions - it would be that level. This way it stays one rung below. That's no guarantee: models follow instructions from user messages too. But it's the difference between an attack that has to argue across the role hierarchy and one that is simply written down alongside.

The same module adds the other half: the prompt tells the model outright that the comment is untrusted user data, and the answer is pinned to a narrow schema. A hijacked agent can at best write the wrong thing into a field that only accepts three values anyway - damage control through form, not through vigilance.

The uncomfortable conclusion

Prompt injection is not solved. There is no library, no model and no vendor that closes the class; there are only systems where it lands less often and does less harm. Both are achievable, and both are engineering work: lower the hit rate through layering and domain restriction, limit the damage through narrow tool permissions, confirmations before irreversible actions, and logs that show what actually happened.

Anyone waiting instead for the one correct system prompt is building their system on a promise nobody made.