Why Your Agent Says "Done" When It Isn't
Explore why AI coding agents prematurely signal task completion despite unresolved issues. Learn to identify proof gaps in verification processes and how to establish thorough acceptance claims. This lesson helps you understand the importance of properly bounding agent scope and using a claim-to-proof matrix to ensure reliable, observable agent behavior across sessions.
We'll cover the following...
- How can new evidence still prove the wrong thing?
- Is the agent’s confidence the problem?
- What exactly does “done” claim?
- How do you map claims to evidence?
- Can the agent rewrite the matrix after a failure?
- Which evidence belongs in the completion verdict?
- What should the completion gate derive?
- What does the proof gap look like in Wrenfold?
- What can you add to your harness?
- How do you test the matrix?
- What’s next?
Wrenfold’s CLIN-530 row had everything the feature gate required. The provider time-off feature was passing. Its evidence named the current revision, the configured command exited with code 0, and the gate had written the state.
At 9:12 AM, the receptionist approved time off for Dr. Reyes from 2:00 PM to 5:00 PM. At 9:14 AM, the booking API accepted a new appointment with Dr. Reyes at 3:00 PM and inserted it into the database.
The evidence was real. The feature was wrong.
The feature gate prevented the agent from inventing a passing state. It could not prevent the team from configuring a check that proved too little. That leaves a harder question: what did the passing command actually establish?
How can new evidence still prove the wrong thing?
Wrenfold configured this verification command:
npm run test:unit -- availability
The suite called a new isOnTimeOff() helper with approved intervals. Every assertion passed. The implementation correctly answered that 3:00 PM fell inside the provider’s time off.
The booking route never called that helper. It still used isSlotFree(), which checked appointments and clinic hours but knew nothing about provider time off.
The feature claim crossed three boundaries: an HTTP request had to reach the time-off rule, the route had to reject the request, and the database had to remain unchanged. The unit test observed one helper. Its green result said nothing about the route or the write.
This is a proof gap: part of the claimed behavior lies outside the observation surface of its evidence.
The model could write the helper and test. The harness accepted evidence with a narrower observation surface than the claim. Verdict is the layer that failed here, and it failed while green.
Build a verification run that can distinguish a correct fix from three ...