AI Economist Agent: An Agentic Framework for Model-Grounded Economic Analysis with RAG, Knowledge Graphs, and Large Language Models

🕒 Published (v1): 2026-06-18 10:18 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

This paper introduces the AI Economist Agent, an agentic framework that grounds LLM-generated economic narratives in executed mathematical models via a knowledge graph. The LLM acts as a planner and orchestrator — selecting models, retrieving evidence, and writing reports — but all quantitative claims are produced by explicitly run computational functions, not by the model itself. Two applications (U.S. inflation/Fed policy and bank CRE stress testing) demonstrate that model-grounded GraphRAG substantially improves grounding scores over LLM-only and RAG-only baselines.

Problem

LLMs produce fluent economic prose but cannot reliably ground model-dependent claims (e.g., inflation persistence paths, stress-test capital ratios) in traceable mathematical models and real evidence. Standard RAG supplies text evidence but cannot distinguish retrieved model names from executed model outputs, conflating narrative grounding with actual computation. Existing economic AI tools do not separate the roles of evidence retrieval, model selection, model execution, and report narration.

Method

The framework builds a typed directed knowledge graph containing five categories of objects: text evidence spans, temporal facts, economic scenario/variable nodes, model catalog objects (specifications, equations, assumptions), and execution records (ModelRun, ForecastOutcome, ModelOutputPoint, BankMetric). Given scenario \(s\), the pipeline produces:

\[\text{Report} \leftarrow (s,\, E_s,\, P_s^{\text{pre}},\, C_s,\, r_s,\, m_s,\, O_s,\, P_s^{\text{post}})\]

where \(E_s\) is retrieved evidence, \(P_s^{\text{pre}}\) and \(P_s^{\text{post}}\) are pre- and post-run graph paths, \(C_s\) is graph-retrieved model context, \(r_s\) is the structured execution request from the model-selection agent, \(m_s\) is the ModelRun record, and \(O_s\) is the computed output table. The planner agent produces an analysis plan; the model-selection agent returns a structured request \((model spec, model family, implementation family, execution tool, expected outputs)\); a computational runner validates and executes one of 14 implementation families (VAR/BVAR, SVAR, DSGE-lite, regime-switching, stochastic volatility, ARIMA, state-space, factor-augmented, ML surrogates, etc.). The graph is cleared and reloaded per application to avoid cross-contamination. Three conditions are compared: LLM-only (scenario only), RAG-only (scenario + text evidence), and Model-grounded GraphRAG (full pipeline).

Key Contributions

• Architecture that separates text evidence, model specifications, and execution records into distinct typed graph nodes, preventing RAG-only conditions from accidentally inheriting model-grounding by reading model names from prose.
• Structured model-selection interface where the LLM chooses among graph-retrieved ModelSpecification objects and returns a validated (model spec, implementation family, execution tool) triple — never free-text model names.
• Post-run graph write-back: ModelRun, ForecastOutcome, and BankMetric nodes are created after execution, making the full scenario→model→output path traceable and queryable.
• Demonstration across two heterogeneous economic domains (macro inflation analysis via DSGE-lite; bank stress testing via regime-switching) using the same agent workflow.
• Judge-agent evaluation framework scoring Evidence, Graph, Model, Numeric, Coherence, and Audit dimensions independently.

Results

Application 1 (U.S. inflation/Fed policy, DSGE-lite):

Condition	Evidence	Graph	Model	Coherence	Audit
LLM-only	2.0	1.0	0.0	4.0	2.0
RAG-only	5.0	1.0	0.0	5.0	5.0
Model-grounded GraphRAG	5.0	4.0	5.0	4.0	5.0

Application 2 (CRE stress test, regime-switching): same qualitative pattern — RAG-only improves evidence scores but scores 0.0 on model grounding; Model-grounded GraphRAG achieves high graph and model scores.

• Knowledge graph scale: 2,500 evidence rows, 420 model specifications, 1,260 equations, 14 executable implementation families.
• Model outputs are reported in synthetic macro-financial units; results are explicitly not empirical forecasts.

Limitations

• All model outputs are in "synthetic macro-financial units" — the framework is demonstrated as a proof-of-concept pipeline, not validated against real forecasts or historical data.
• Evaluation is performed by a judge LLM using the same local model, not by human experts or holdout benchmarks, introducing potential circularity.
• The knowledge graph (evidence, model specs) is constructed from synthetic/curated rows, not a live economic database, limiting generalizability claims.
• Graph queries are task-specific hand-crafted queries rather than learned retrievers; scalability to open-ended scenarios is untested.
• No ablation of individual components (e.g., temporal separation, post-run write-back) — only the three coarse conditions are compared.
• The paper explicitly states the comparison "is not a test of whether a longer prompt produces a longer report" but does not control for token count differences across conditions.

Relevance to Harnesses / Meta-Harnesses

This paper is a direct instance of a domain-specific meta-harness: a multi-agent orchestration layer that wraps heterogeneous tools (graph retrieval, computational model runners, report generators, a judge agent) inside a single, reproducible workflow driven by a planner LLM. The separation of planner, retriever, model-selector, executor, and judge agents mirrors the harness pattern of decomposing complex tasks into typed sub-agents with controlled interfaces. The graph-state write-back after model execution — enabling a second retrieval pass over enriched graph context — is an example of stateful, multi-phase harness design where intermediate tool outputs are persisted and re-queried rather than passed directly through the LLM context. For researchers tracking harness architectures, the key design principle here is strict I/O typing between agents: the model-selection agent is constrained to emit a structured request against a controlled registry, not free-form text, which is a generalizable pattern for preventing agents from hallucinating tool calls.