CAMO: An Agentic Framework for Automated Causal Discovery from Micro Behaviors to Macro Emergence in LLM Agent Simulations¶

🕒 Published (v1): 2026-04-16 06:49 UTC · Source: Arxiv · link

Why this paper was selected

Agentic causal discovery from micro-behaviors in LLM agent simulations

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

CAMO is a five-agent meta-framework that wraps an LLM-driven agent simulation to automatically recover the causal mechanisms linking micro-level agent behaviors to macro-level emergent outcomes. It outputs a computable Markov boundary \(\text{MB}^H(Y)\) and a minimal explanatory subgraph \(E_Y\) centered on a target emergent variable \(Y\). A fast–slow self-evolving loop integrates domain priors, observational constraint-based discovery, and simulator-internal counterfactual probing to iteratively refine and orient the causal graph.

Problem¶

LLM-empowered agent simulations readily exhibit emergent macro-level outcomes (coordination, polarization, norm formation), but provide no principled account of the causal pathways from micro-level agent decisions and meso-level interaction structure to those outcomes. Existing causal discovery methods assume fixed variable spaces and cannot handle the adaptive, non-linear factor spaces that arise in multi-agent simulations; existing simulation studies describe rather than causally explain emergence.

Method¶

CAMO orchestrates five workflow LLM agents that operate outside the simulation while treating the inner simulation as a black-box stochastic data-generating process:

A1 (Worldview Parser): Given a user query \(Q\) and target \(Y\), retrieves domain texts \(T\) and parses them into candidate variables tagged by scale (micro/meso/macro), preserving conflicting hypotheses as competing worldviews.
A2 (Worldview Integrator): Aligns worldviews into a shared decision worldview \(W^{(k)}\) with canonical factor construction rules (ratios, rolling statistics, graph metrics); maintains a retained mechanism graph \(G_W^{(k)}\) of upstream pathways.
A3 (Causal Cartographer): Runs an add–prune refinement over an induced factor space \(H\) using estimated information gain \(\hat{\Delta I}(Z) := \hat{H}(Y \mid V^{(t)}) - \hat{H}(Y \mid V^{(t)}, Z) > \tau_I\), then applies constraint-based causal discovery (PC/FCI) under structural constraints from \(W^{(k)}\) to produce a CPDAG/PAG. From the stabilized boundary \(\text{MB}^H(Y)\), it constructs \(E_Y = \{Y\} \cup \text{MB}^H(Y) \cup \text{Conn}_{\min}(R \Rightarrow \text{MB}^H(Y))\), the minimal upstream connecting subgraph from root variables \(R\) to the boundary.
A4 (Simulation Scriptwright): Converts high-priority ambiguous CPDAG edges into executable paired-rollout intervention scripts (e.g., prompt overrides, policy changes) scored by an importance–uncertainty metric.
A5 (Counterfactual Adjudicator): Executes \(do(X{=}x)\) vs. \(do(X{=}x')\) contrasts; edges with consistently nonzero oriented effects become hard constraints in the fast loop.

Control flow uses a fast loop (A3→A4→A5 cycle under a fixed worldview) nested inside a slow loop (A2 revision triggered when counterfactual evidence contradicts the current worldview \(W^{(k)}\)).

Two theoretical results support correctness: Theorem 4.1 bounds boundary size growth per step (\(|B_{t+1}| \leq |B_t| + m_t\)); Theorem 4.2 shows residual dependence \(F_t = I(Y; X_{\text{obs}} \mid V^{(t)})\) decays geometrically in expectation at rate \((1-pC)^t\).

Key Contributions¶

First multi-agent framework targeting micro-to-macro causal discovery specifically in LLM-based simulations, combining domain priors, observational data, and simulator-internal counterfactuals.
Separation of causal identification (Markov boundary \(\text{MB}^H(Y)\) for prediction/intervention) from mechanistic explanation (\(E_Y\) tracing upstream pathways), with formal sufficiency and minimality proofs (Proposition 2).
Fast–slow self-evolution loop that updates hypotheses when counterfactual evidence contradicts the active worldview, reducing hallucinated priors without assuming global identifiability.
Demonstrated on four emergent settings spanning structured benchmarks (O2O delivery) and open-ended simulations (Smallville, AgentSociety).

Results¶

Factor discovery (O2O benchmark, Table 1): - CAMO: MB-F1 = 1.00, Anc-F1 = 0.98; all recovered factors are true boundary or ancestor variables (OT = 0) - Best baseline (single-round data-grounded): MB-F1 = 0.29, Anc-F1 = 0.35

Causal structure recovery (Table 2, observed-variable projection): - CAMO (DeepSeek-V3.2): F1 = 0.96, SHD = 1, Anc-F1 = 1.00, FPR = 0.02, 0 reversed/unoriented edges - Best non-CAMO baseline (LLM-KBCI): F1 = 0.85, SHD = 3, Anc-F1 = 0.74, FPR = 0.03 - Pure SCD baselines (PC): F1 = 0.26, SHD = 23

Intervention ranking without ground truth (Table 3), averaged across Smallville + two AgentSociety tasks: - CAMO: MAP@5 = 0.64 / 0.84 / 0.71 across three emergent targets - Best baseline per task: MAP@5 up to 0.28 / 0.63 / 0.16

Ablation (Table 4, GPT-5 mini): - Removing A1&A2 collapses MB-F1 from 0.80 to 0.00, SHD from 5 to 23 - Removing counterfactual (A5) has marginal MB-F1 impact but increases FPR and OT factors

Limitations¶

Ground-truth evaluation relies on a single O2O delivery simulation calibrated to Meituan data; generalization to other emergent settings is demonstrated qualitatively without ground-truth graphs.
Convergence constants \(p\) and \(C\) (Theorem 4.2) are estimated post-hoc from the \(\{F_t\}\) trajectory, not guaranteed a priori; convergence speed is uncontrolled in practice.
Counterfactual edge resolution (Proposition 1) requires the simulator to admit realizable interventions (prompt/policy overrides); black-box simulators without controllable interfaces cannot use A4–A5.
The induced factor space \(H\) (ratios, rolling statistics, graph metrics) is constructed from fixed templates; mechanistic factors outside these templates are not discovered.
Computational cost scales with the number of ambiguous edges forwarded to A4–A5 and with the number of refinement rounds; no wall-clock or token-cost analysis is reported.

Relevance to Harnesses / Meta-Harnesses¶

CAMO is architecturally a meta-harness: an outer multi-agent workflow that wraps, instruments, and actively probes an inner LLM-agent simulation to extract causal structure. The fast–slow loop is a control harness pattern — an outer scheduler that triggers inner refinement cycles and escalates to hypothesis revision when evidence quality degrades, directly analogous to the self-repair loops seen in agentic harnesses. The A4–A5 counterfactual subsystem is equivalent to a test harness that injects controlled perturbations into the inner system and observes responses, treating the simulation as a testable black box. For researchers building harnesses that need to understand why their inner agents produce certain outcomes — not just what they produce — CAMO offers a principled template for causal instrumentation of multi-agent pipelines. Its separation of causal identification from mechanistic explanation also maps cleanly onto harness design principles: a minimal monitoring interface (\(\text{MB}^H(Y)\)) plus an upstream trace (\(E_Y\)) mirrors the instrumentation vs. interpretation split common in production harnesses.