Information Provenance
Explore how provenance annotation enhances reliability in AI systems by tracking source, confidence, timestamp, and agent role for each finding. Understand its role in multi-agent workflows and long sessions, and learn techniques to resolve conflicting outputs effectively, enabling auditable and trustworthy AI results.
We'll cover the following...
Two agents investigate the same contract clause and reach different conclusions. One classifies it as an indemnification clause; the other classifies it as a limitation of liability. Both are plausible. Both used the same source document. Without provenance, a record of which agent produced which finding, from which passage, with what confidence, and at what point in the session, there is no principled way to decide which output to trust. The session has two answers and no mechanism to choose between them. Provenance annotation is the mechanism that enables conflict resolution, and it is the difference between a system that produces outputs and one that produces auditable outputs. By the end of this lesson, we will be able to:
Explain why provenance annotation is necessary for long-running and multi-agent workflows
Define the four provenance fields: source, confidence, timestamp, and agent role
Implement a provenance-annotated finding structure and apply it to the extraction output
Apply a conflict resolution strategy that uses provenance to choose between competing findings
Why provenance matters
In a single-turn extraction, every output is produced by the same model call using the same input. Provenance is trivially traceable. In a multi-turn session or a multi-agent pipeline, the ...