More Than Irrational: Modeling Belief-Biased Agents¶

🕒 Published (v1): 2025-11-15 21:14 UTC · Source: Arxiv · Venue: AAAI 2026 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

This paper formalizes a class of computational-rational (CR) user models where seemingly irrational behavior is explained as optimal decision-making under biased beliefs induced by bounded memory. The authors derive an efficient online inference method based on nested particle filtering (NPF) that simultaneously recovers a user's latent memory-decay parameter and tracks their biased belief state from passive action observations. The framework is validated in a T-maze navigation task and extended to an assistive-POMDP that adapts intervention type and timing to the inferred cognitive bound.

Problem¶

Existing user models in human-AI collaboration either assume perfect memory or attribute sub-optimal behavior to irrationality without a principled mechanistic account. Prior computational-rationality work on bounded agents (e.g., bounded inference budget) does not generalize to bounded memory; memory-aware models are application-specific. The gap is a general, tractable framework for modeling and inferring memory-bounded agents online from passive observations.

Method¶

The CR user model extends a standard POMDP by replacing the agent's Bayesian belief update with one conditioned on a corrupted internal memory history \(\tilde{h}_t\). A parameterized stochastic memory process \(f_\theta\) evolves the internal memory (e.g., with probability \(\theta\) the memory of a target observation is replaced by a default value at each step), introducing systematic bias into the agent's belief \(\tilde{b}_t\). The agent then acts optimally with respect to \(\tilde{b}_t\) via a softmax policy over the Q-function learned under biased beliefs. For inference, the unknown cognitive bound \(\theta\) and latent memory state \(\tilde{h}_{t-1}\) are jointly estimated online using nested particle filtering (NPF): \(N_\theta\) outer particles track \(p(\theta \mid h_t)\); for each outer particle, \(N_{\tilde{h}}\) inner particles track \(p(\tilde{h}_{t-1} \mid h_t, \theta^i)\). Outer particle weights are updated by the likelihood \(\pi^*( a_{t-1} \mid \tilde{b}_{t-1}; \theta^i)\), requiring precomputed policies for each candidate \(\theta\). An assistive-POMDP wraps this inference loop: the AI assistant's hidden state is the CR agent's \((θ, \tilde{h})\), maintained via NPF, and an assistance policy (no action / memory hint / action hint) is optimized with PPO on top of this belief.

Key Contributions¶

Formal CR user-model family where a parameterized memory process \(f_\theta\) makes biased-belief and sub-optimal behavior explicit and mathematically grounded.
Efficient online Bayesian inference algorithm (NPF) that reduces exact inference cost from \(O(|S|^t \cdot t!)\) to \(O(N_\theta N_{\tilde{h}} t |S|)\).
Demonstration that the latent cognitive bound is practically identifiable from passive observations (≤100 steps).
Assistive-POMDP instantiation that learns adaptive, timing-aware intervention strategies calibrated to inferred memory capacity.

Results¶

Behavioral expressiveness: CR agents with \(\theta \in \{0.0, 0.4, 0.7, 1.0\}\) produce qualitatively distinct, cognitively plausible trajectories in the T-maze (no exploration for \(\theta=1.0\); recheck loops for \(\theta=0.7\); robust memorization passes for \(\theta=0.4\); optimal direct path for \(\theta=0.0\)).
Inference accuracy: Final posterior-mean (PM) error = \(0.0087 \pm 0.0035\) (mean ± SE over 5 seeds × 11 ground-truth \(\theta\) values, \(\tau=3.0\)).
Data efficiency: PM error reduced 90% from \(t=1\) baseline by \(t=45\) steps; 95% reduction by \(t=78\) steps.
Robustness to temperature: Method converges under \(\tau \in \{1.0, 3.0, 5.0\}\) and an adaptive schedule; degrades at \(\tau=10.0\) (highly stochastic actions).
Adaptive assistance: Learned policy correctly withholds intervention for \(\theta \le 0.3\), provides memory hints for \(\theta \in [0.4, 0.8]\), and escalates to direct action hints for \(\theta \ge 0.9\).

Limitations¶

Both the inference method and the assistant assume full knowledge of the environment dynamics (\(T\), \(O\)), which is unrealistic in open-world settings.
Memory model \(f_\theta\) in experiments is a simple Bernoulli corruption of a single object observation; this is insufficient to capture realistic, multifaceted memory processes.
Validation is simulation-only (T-maze); no real human participants or complex real-world tasks.
Policy precomputation over the discrete \(\theta\) grid is a scalability bottleneck for continuous or high-dimensional cognitive-bound spaces.

Relevance to Agentic AI / LLM Agents¶

As AI systems increasingly act as autonomous collaborators or assistants, accurately modeling human users — including their cognitive limits — is critical for safe and effective delegation. This work provides a principled, tractable mechanism for an AI agent to infer why a human collaborator behaves sub-optimally and adapt its assistance accordingly, directly relevant to human-in-the-loop agentic pipelines. The assistive-POMDP framing maps cleanly onto the structure of LLM-agent orchestration where the agent must decide when to act autonomously versus prompt or correct the human. The nested particle filtering approach is a practical inference primitive that could be integrated into agent planning loops that maintain beliefs about user state.