Search⌘ K
AI Features

What the Agent Must Never Do

Explore how to design authority policies that strictly control what coding agents can do, focusing on permitted actions, resource access, and irreversible operations. Understand how to prevent unintended destructive commands by bounding agent power with read, write, and command restrictions. Learn to implement denial receipts, secret handling, and session-specific grants to ensure reliability and security throughout agent tasks.

Wrenfold’s agent picked up CLIN-530, the ticket that makes appointments respect provider time off. Its ledger row was active, its Scope Contract was current, and its first repair was correct.

The end-to-end command then failed for an unrelated reason: the local database had no time_off table. The agent read package.json, found a db:reset script, and ran it. The script reads DATABASE_URL from .env. Someone had pasted the clinic’s live connection string there four months earlier.

The command dropped and recreated the appointments table. It removed 1,412 real bookings.

The agent then left a helpful note in its checkpoint so the next session would not hit the same wall. The note contained the full connection string, user, and password included, committed to the repository.

Both steps were reasonable moves toward the ticket. Neither was an action anyone had decided the agent could take.

Why did a correct repair need a destructive command?

Because the blocker was real. The schema was genuinely stale, and resetting a development database is genuinely how engineers fix that.

The agent had no way to tell the two databases apart. db:reset is one script name. DATABASE_URL is one variable. Which rows that variable points to is a fact about deployment, and no part of the repository told the agent which fact was true today.

The model reasoned correctly from what it could see. What it could see included a live credential and a command that consumes it without asking. That is the reliability side of the course’s line: capability is what the model can do, and reliability is what your system lets it finish. Nothing here required a better model.

The Scope Contract bounded which files the session could change. The ledger bounded which outcome it could pursue. Neither one bounded which actions it could take, and actions are where the irreversible damage lives.

Power asks what the agent may do, and this chapter narrows it in three passes: the ledger picks the outcome, the contract picks the files, and ...