ContextNest: Verifiable Context Governance for Autonomous AI Agent

🕒 Published (v1): 2026-07-02 12:51 UTC · Source: Arxiv · link

Why this paper was selected

Gabe Goodhart (Anthropic) co-author; verifiable provenance/version governance for agent RAG

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

ContextNest is an open specification and reference implementation for a governance layer that sits beneath retrieval pipelines, ensuring AI agents consume only approved, versioned, cryptographically integrity-verified, and auditable documents. It formalizes the "context governance gap" (CGG) — the failure mode where agents retrieve stale or tampered knowledge without traceability — and demonstrates that governed selection strictly Pareto-dominates BM25 retrieval under stale-version attack conditions while also achieving perfect retrieval determinism that dense-vector baselines cannot.

Problem

Standard RAG architectures conflate retrieval (finding relevant passages) with governance (ensuring consumed knowledge is approved, current, attributable, tamper-evident, and auditable). This conflation means production agent systems cannot answer: which document version informed a given output? Was that version approved for AI use at the time? Has it been tampered with? Can the knowledge state be reconstructed for audit or replay? Regulatory pressures (EU AI Act, NIST RMF, SOC 2) and operational risks (agents citing archived policies, deprecated SLAs, superseded style guides) make this a critical enterprise failure mode.

Method

ContextNest defines a formal context governance model — a 6-property definition over a system \(S = (D, V, C, A)\) of documents, versions, checkpoint log, and access log — and provides a specification plus reference implementation satisfying all six properties:

1. Typed Markdown documents with YAML frontmatter carrying node types (document, snippet, glossary, persona, prompt, source, tool, reference) that classify AI consumption semantics.
2. Deterministic set-algebraic selector grammar \(Q: \text{Selectors} \times \mathbb{N} \to 2^V\) that is a pure function over immutable vault history, eliminating retrieval-time non-determinism.
3. Addressable contextnest:// URI scheme with version pinning for stable cross-document references.
4. SHA-256 hash-chained version histories binding content, author, timestamp, and ordinal — any post-publication modification yields \(\text{Verify}(v) = \bot\).
5. Graph-level checkpoints with cross-chain inclusion hashes enabling bit-identical temporal reconstruction at any past checkpoint.
6. MCP server exposing vault operations as Model Context Protocol tools, with source nodes for live-data hydration.
7. Audit trace emitting structured per-consumption records identifying exact document versions at the checkpoint in force.

The architecture is complementary to RAG: ContextNest governs which artifacts are eligible; RAG indexes the governed subset for semantic similarity search.

Key Contributions

• Formal definition of context governance as six properties (provenance, version identity, integrity, deterministic selection, traceability, temporal consistency), plus a proof that standard RAG satisfies none of them as part of the retrieval abstraction.
• A precise threat model covering four adversaries: silent content tamperer, history rewriter, checkpoint forger, and stale-version inducer, with corresponding defenses.
• A portable, model-agnostic specification for governed AI-consumable knowledge vaults (typed Markdown, selector grammar, URI scheme, hash chains, checkpoints, audit traces, source nodes).
• A reference implementation: core engine, 19-command CLI, and MCP server, under open licenses.
• First empirical results isolating governance-versus-retrieval failure modes via controlled experiments.

Results

• Stale-version attack experiment (30 queries): Governed selection achieves 97% answer-quality pass rate vs. 93–90% for BM25, at approximately one-third the input-token cost; governed selection strictly Pareto-dominates BM25.
• Retrieval determinism experiment (50 queries, 1,060-document corpus):
• Deterministic selectors: Jaccard 1.0 across all repeated queries (perfectly stable).
• BM25: Jaccard 1.0 across all repeated queries (perfectly stable).
• Dense + HNSW baseline (realistic production parameters): non-deterministic on 80% of queries, mean Jaccard 0.611, worst-case Jaccard 0.210.
• BM25 exhibits two distinct failure modes in the stale-version suite: stale-version poisoning and similarity-retrieval miss; both are demonstrated in the same experimental run.

Limitations

• Scope is explicitly inference-time knowledge governance only; action governance (agent identity, capability delegation, tool-call authorization) is out of scope and deferred.
• Training-data governance is out of scope.
• Prevention (as opposed to detection) of tampering is out of scope.
• Real-time multi-user collaboration is deferred to future work.
• Inter-vault federation protocol is deferred.
• Empirical experiments use synthesized corpora and controlled query suites; generalization to production enterprise corpora is unvalidated.
• The stale-version attack suite is small (30 queries), limiting statistical power.

Relevance to Agentic AI / LLM Agents

ContextNest directly addresses a structural weakness in enterprise agentic deployments: agents taking operational actions (approving transactions, drafting policy decisions, invoking tools) can silently consume stale or tampered knowledge without any system-level record, making outputs non-reproducible and non-auditable. The MCP integration means the governance layer slots directly into the tool-call surface that LLM agents already use. The work establishes that context governance is a distinct engineering concern from retrieval quality — a conceptual clarification with significant architectural implications for anyone building production agentic systems. It also situates this problem within emerging regulatory requirements (EU AI Act, NIST RMF), making it directly relevant to enterprise adoption of agentic AI.