The World Is Bigger! A Computationally-Embedded Perspective on the Big World Hypothesis

🕒 Published (v1): 2025-12-29 12:31 UTC · Source: Arxiv · Venue: NEURIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

This paper formalizes the "big world hypothesis" — that an agent's world is always larger than itself — by modeling agents as automata computationally embedded within a universal-local environment (a Markov process capable of simulating any algorithm). It introduces interactivity, an algorithmic-complexity-based measure of an agent's capacity for continual adaptation, and proves any embedded agent is implicitly capacity-constrained and suboptimal if it stops learning. Empirically, deep linear networks sustain and scale interactivity, while deep nonlinear (ReLU) networks fail.

Problem

Prior continual learning formalizations impose explicit capacity constraints (information-theoretic bounds, memory limits, compute caps) that are ad hoc, hard to enforce, and block beneficial scaling. The agent and environment are traditionally treated as separate entities, making it impossible to derive capacity constraints naturally. The paper addresses the gap: how to ground the big world hypothesis in a constraint that is implicit, scale-compatible, and computationally rigorous.

Method

1. Universal-local environment: Defines an algorithmic Markov process over a countably infinite state-space whose transition dynamics are (a) computationally universal (can simulate any Turing machine) and (b) uniformly local (each index's transition depends only on a finite boundary-space of neighbors, as in Conway's Game of Life).
2. Embedded automaton: An agent is represented as a boundaried Markov process (automaton) simulated within this environment. When the automaton's boundary-space coincides with its input-output space, the paper proves it is equivalent to a stateful policy acting on a POMDP over the infinite environment state-space.
3. Interactivity: Defined via conditional algorithmic (Kolmogorov) complexity as I_T(A) = K(future behavior | ∅) − K(future behavior | past behavior) — i.e., how much past experience compresses the algorithmic description of future behavior. High interactivity requires both complex and past-predictable future behavior.
4. RL algorithm for interactivity: Approximates Kolmogorov complexity via temporal difference (TD) prediction errors. Agent-relative interactivity = difference between static (unconditional) and dynamic (online-updated) TD errors. A model-based meta-gradient policy is trained to steer toward experiences that are learnable but currently unpredicted — a bi-level optimization resembling MAML/cross-prop.
5. Evaluation: Environment-free self-predicting agent that observes only its own actions, directly benchmarking continual learning algorithms without a fixed external environment.

Key Contributions

• Formal definition of universal-local environments (universal + uniformly local Markov process) and proof that any embedded automaton faces a POMDP over infinite state-space.
• Proof (Theorem 1) that maximum interactivity is asymptotically bounded by the agent's computational capacity.
• Proof (Theorem 2) that an interactivity-seeking agent is capacity-limited and suboptimal if it stops learning — recovering the big world desiderata without explicit constraints.
• Interactivity as a sequence-based (algorithmic, not Shannon) measure of continual adaptability, tied to the agent's own reference machine.
• An RL algorithm that operationalizes interactivity via TD errors and meta-gradient policy optimization.
• Empirical finding: deep linear networks scale interactivity with depth/width; deep nonlinear (ReLU) networks collapse — connecting to loss-of-plasticity literature.

Results

• Deep ReLU networks fail to sustain interactivity in the self-predicting evaluation task; their action sequences have no predictable structure.
• Deep linear networks sustain interactivity and produce structured, non-stationary wave-like action sequences that are locally predictable by a static linear function but globally require a dynamic one.
• Increasing depth in deep linear networks produces large interactivity gains (also more oscillatory behavior); increasing width yields marginal gains.
• No external environment baselines reported; comparisons are between network architectures (linear vs. ReLU) at varying depth/width.

Limitations

• Interactivity is grounded in Kolmogorov complexity, which is uncomputable; the TD-error approximation is heuristic and agent-relative, not a tight proxy.
• Evaluation is purely synthetic and environment-free; transfer to real RL environments with external reward is not demonstrated.
• The self-predicting agent setting has no external observation signal, limiting ecological validity.
• Only linear vs. ReLU architectures tested; no comparison with recurrent networks, transformers, or other architectures relevant to practical continual learning.
• The bi-level meta-gradient optimization is computationally expensive and not analyzed for convergence or stability guarantees.

Relevance to Agentic AI / LLM Agents

This paper provides a theoretical foundation for why any agent embedded in a sufficiently rich world must continually learn — a core desideratum for long-lived autonomous agents. The interactivity framework formalizes the intuition that capable agents should seek out novel-but-learnable experiences, directly relevant to open-ended exploration and self-improvement in agentic systems. The finding that deep linear networks outperform nonlinear ones on continual adaptation benchmarks touches on the plasticity-stability tradeoff that is central to deploying LLM-based agents in non-stationary real-world deployments. The environment-free, self-predicting evaluation paradigm offers a potential framework for benchmarking continual learning in agents independent of task-specific scaffolding.