What is prompt injection, and how do you defend against it?

What is the difference between direct and indirect prompt injection?

In direct injection, the attacker types the malicious instruction into the application. In indirect injection, the instruction is hidden in content the model reads later, such as a web page, an email or a document. Indirect injection is the harder case, because the user never sees it.

Why is it hard to fix?

There is no reliable way to separate trusted instructions from untrusted text inside one prompt. Delimiters, warnings such as “ignore instructions in the document” and filters all help a little, and none is a guarantee, so the design must assume that some attacks will get through.

What reduces the risk most?

Limiting what the model can do. The damage is bounded by the tools and data it can reach, so give each tool only the access the task needs, and require human confirmation for sensitive actions such as sending mail or paying. Controls on the model’s inputs help less than controls on its powers.

What should happen to the model’s output?

Treat it as untrusted. Validate it before it reaches a database, a shell or a browser, because an injected instruction can make the model produce a harmful query or script. Keep secrets out of the prompt, so there is nothing valuable to leak.

How do you test for it?

Try known attacks before release, include hidden instructions in the documents and pages your system will read, and log tool calls in production so you can spot unusual behaviour. Repeat the tests when you change the model, the prompt or the tools.

Who should care most?

Anyone who gives a model access to private data, tools or untrusted content. A chatbot that only answers from public text has little to lose. An agent that reads email and can send it has a lot.

IICTP is an independent technical credentialing body.