Skip to content

Leveraging Dual Process Theory in Language Agent Framework for Real-time Simultaneous Human-AI Collaboration

🕒 Published (v1): 2025-02-17 15:09 UTC · Source: Arxiv · Venue: ACL 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

DPT-Agent is a language agent framework that integrates Dual Process Theory (DPT) into real-time simultaneous human-AI collaboration by coupling an FSM-based System 1 (fast, low-latency decisions) with an LLM-driven System 2 (deliberate reasoning via Theory of Mind and asynchronous reflection). The key insight is that neither pure System 1 (fast but incapable) nor pure System 2 (capable but too slow) alone suffices for real-time tasks; the framework bridges them via a code-as-policy generator. Evaluated on a hard Overcooked variant with rule-based agents and 71 human participants, DPT-Agent outperforms ReAct and Reflexion baselines on both score and subjective collaboration quality.

Problem

LLM-based agents excel at turn-by-turn collaboration but fail at simultaneous real-time tasks because: (1) large, capable models have unacceptable inference latency for sub-second decision cycles, while small, fast models lack capability; (2) existing frameworks lack autonomy—they require human input to produce actions rather than operating as true independent collaborators; and (3) LLMs fail at functional Theory of Mind, where reasoning about partners cannot be translated into timely decisions.

Method

DPT-Agent models the task as a two-agent decentralized MDP \(\langle S, \{A^i\}, \{A^h\}, \rho, P, r \rangle\) and decomposes the agent into two asynchronous components:

System 1 (fast path): A Finite-State Machine (FSM) runs continuously at the game tick rate (\(0.25\)s/step), producing macro actions (e.g., ingredient preparation, assembly) which an A-based executor converts to atomic actions. The FSM logic is dynamically overwritten by code-as-policy*: the LLM periodically generates executable Python-like code that reconfigures FSM transitions based on the latest beliefs and guidelines, computed as \(c_t = \text{LLM}(\tau_{t-\lambda:t}, b^n, g^m)\).

System 2 (slow path, asynchronous): Two LLM modules run in parallel without blocking System 1. The Theory of Mind (ToM) module periodically ingests history \(\tau_{0:t}\) and updates a natural-language belief about the human partner: \(b^n = \text{LLM}(\tau_{0:t_n}, b^{n-1})\). The Asynchronous Reflection module uses the belief and game history to iteratively update a "Behavior Guideline": \(g^m = \text{LLM}(\tau_{0:t_m}, b^n, g^{m-1})\). The full System 2 policy is \(\pi^{S2}: \mathcal{T} \times \mathcal{B} \times \mathcal{G} \mapsto \mathcal{B} \times \mathcal{G}\), and System 1 is \(\pi^{S1}: \mathcal{S} \times \mathcal{B} \times \mathcal{G} \mapsto \mathcal{A}\).

Key Contributions

  • Empirical analysis showing that independent System 1 LLMs (small/fast) and System 2 LLMs (large reasoning models with long CoT) both fail in real-time tasks—establishing a motivating baseline for the DPT decomposition.
  • DPT-Agent framework: FSM-based System 1 for zero-latency decision continuity, code-as-policy bridge for System 2→System 1 knowledge transfer, and asynchronous ToM + reflection so LLM reasoning never blocks action output.
  • New Overcooked environment layouts (New Counter Circuit, New Asymmetric Advantages) designed specifically for real-time simultaneous human-AI evaluation.
  • Human-subject evaluation (n=71) validating both objective scores and subjective collaboration quality over competing frameworks (ReAct+FSM, Reflexion+FSM, DPT-Agent w/o ToM).

Results

  • Single-agent (real-time task): DPT-Agent w/o ToM achieves higher score efficiency than ReAct and Reflexion across 11 LLMs evaluated; all LLM-based System 1/2 approaches score below the FSM baseline alone, validating the FSM-first System 1 design.
  • Collaboration with rule-based agents: DPT-Agent (with ToM) outperforms ReAct, Reflexion, and DPT-Agent w/o ToM on Map 1 (New Counter Circuit) across 11 LLMs.
  • Human collaboration (n=71, GPT-4o-mini backbone): DPT-Agent outperforms ReAct+FSM, Reflexion+FSM, and DPT-Agent w/o ToM on both objective score and subjective user ratings on two layouts; ablation confirms the ToM module's incremental contribution.
  • Reasoning models (QwQ-32b, o3-mini, DeepSeek-R1 series) using long CoT as pure System 2 score lower than many smaller models when used independently, underscoring the latency-capability tradeoff.
  • Total compute: 517.5 A800 GPU-hours + 228 H100 GPU-hours + $735 in API costs across 20-run repetitions.

Limitations

  • Evaluated only on Overcooked (a constrained, grid-world cooking game); generalization to more open-ended simultaneous collaboration tasks is unproven.
  • FSM design requires manual task-specific state and transition definitions; the framework is not fully self-contained for novel domains.
  • Code-as-policy generation assumes the LLM can produce syntactically valid, semantically correct FSM-modifying code—failures in code generation are not analyzed.
  • ToM module relies on observed trajectory history; it may fail when human behavior is highly stochastic or when history is short (game start).
  • All human experiments used GPT-4o-mini as the backbone; comparative results with stronger models in the human-study condition are absent.
  • The asynchronous reflection interval \(\lambda\) and ToM trigger frequency are hyperparameters whose sensitivity is not characterized.

Relevance to Harnesses / Meta-Harnesses

DPT-Agent is a concrete instance of a cognitive meta-harness: a framework that orchestrates multiple LLM and non-LLM subsystems (FSM, code-as-policy, ToM, asynchronous reflection) with explicit, structured communication contracts between them, rather than relying on a single monolithic LLM call. The code-as-policy generator acting as a runtime bridge between a slow deliberative module and a fast procedural executor is directly analogous to the controller-in-the-loop pattern in harness architectures. The asynchronous design—where System 2 reflection runs decoupled from the action loop—demonstrates a key harness design principle: separating reasoning latency from execution latency, which is relevant to any harness that must maintain real-time SLA while still invoking heavy LLM reasoning. The ToM belief state passed as structured natural-language context across modules exemplifies inter-agent state management that harness designers must reason about.