Defending Against Misuse
Explore techniques to defend AI features against prompt injection and malicious misuse. Understand how to treat conversation content as data, apply schema caps, and scope data access narrowly. Learn common engineering pitfalls and how to maintain robust defenses in production AI systems.
We'll cover the following...
- What prompt injection actually looks like
- Why the defenses already in place blunt this
- Treating conversation content as data, never instructions
- Scoping data access as narrowly as actions
- Seeing this defense in a real product
- Where these risks are named beyond this course
- Three ways engineers get this wrong
- What’s next
Every prompt, tool, and workflow this course has built assumed the conversation it reads is what it appears to be. Real conversations aren’t guaranteed to be that innocent. This lesson covers defending against content that’s actively trying to manipulate the model, and shows that some of the strongest defenses aren’t new, they’re the same schema caps and confirmation gates from two chapters ago, doing security work nobody labeled as security at the time. In this lesson, we will cover:
What prompt injection actually looks like in a real conversation
Why the defenses from the capabilities chapter already blunt it
Treating conversation content as data, never as instructions
Scoping what data a request can access, at the same level of care as the actions it can take
How the same defense shows up in a real product, and the same risks in a real industry standard
The three ways engineers get this wrong
What prompt injection actually looks like
Prompt injection is content, hidden inside something the model reads as ordinary input, written to make the model ignore its actual instructions and follow different ones ...