WALL-E: World Alignment by NeuroSymbolic Learning improves World Model-based LLM Agents¶

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

WALL-E aligns a pretrained LLM's internal world model with actual environment dynamics by inductively learning three types of symbolic knowledge (action rules, knowledge graphs, scene graphs) from exploration trajectories and encoding them as executable Python code rules. A training-free, RL-free model-predictive control (MPC) loop then uses the aligned neurosymbolic world model to validate and refine actions before execution. On Mars (Minecraft-like) and ALFWorld, WALL-E achieves state-of-the-art performance among training-free LLM agents.

Problem¶

LLMs used as world models suffer from a "world alignment gap": their commonsense priors diverge from the specific dynamics of a target environment, causing incorrect state-transition predictions, hallucinations, and policy failures. Existing approaches either fine-tune LLMs on environment experience (expensive, inflexible) or rely on hand-crafted PDDL/prompt engineering (environment-specific, low scalability). Model-free LLM agents skip world-model verification entirely, risking unsafe or suboptimal trajectories.

Method¶

NeuroSymbolic Learning (4 stages): 1. Trajectory comparison — at each episode, compare the LLM world model's predicted trajectory against the real environment trajectory; classify transitions as correctly or incorrectly predicted. 2. Symbolic knowledge extraction — use LLM inductive reasoning over a sliding window of k real transitions to extract: (a) action rules (deterministic precondition→outcome constraints), (b) a knowledge graph of entity feasibility and prerequisite relations, and (c) a scene graph tracking global spatial context to compensate for partial observability. 3. Code translation — prompt the LLM to implement extracted rules as Python functions that return (feedback, success flag, suggestion) tuples. 4. Pruning via maximum set coverage — verify rules against ground-truth transitions (discard misfiring/non-executable rules), then greedily select the compact subset R* ⊆ R_code that maximally covers incorrectly predicted transitions D_inc under a cardinality limit l.

MPC loop: At each step, the LLM agent proposes an action; the neurosymbolic world model predicts the outcome via ALIGN(), which checks code rules against the LLM's raw prediction. If the rule's success flag conflicts with the model's, the world model is re-invoked with corrective feedback/suggestions. The agent replans until the action passes verification or a replan limit is hit. NSLearning and MPC alternate each episode.

Key Contributions¶

Training-free, RL-free "world alignment" via neurosymbolic learning that extracts environment-specific symbolic knowledge (action rules + KG + SG) directly from exploration without handcrafted prompts or fine-tuning.
An LLM-based MPC framework where the LLM agent acts as a look-ahead optimizer, replacing costly classical MPC search (CEM/DFO) with LLM reasoning.
A maximum-coverage-based code rule pruning algorithm that selects compact, non-redundant, executable rule sets.
SOTA results among training-free agents on both Mars and ALFWorld benchmarks.

Results¶

Mars (vs. best LLM baseline IfR): reward improves from 5.5 to 6.7 (+17.9%); score improves from 7.4% to 15.3% (+107%); across counter-commonsense settings, reward gain ≥16.1%, score gain ≥51.6% over all LLM baselines.
ALFWorld (134 tasks): WALL-E achieves 98% success rate after only 4 iterations, surpassing RAFA (95%), Reflexion (86%), AdaPlanner (91%), and human performance (91%).
WALL-E uses only 5,600 environment steps vs. DreamerV3's 1 million steps, while DreamerV3 achieves higher absolute Mars scores (avg reward 7.9 vs. 6.7).
Ablation — action rules alone: reward 5.1, score 8.3%; KG+SG alone: reward 4.4, score 5.2%; full combination: reward 6.7, score 15.3%.
Ablation — symbolic knowledge applied only to agent: +30.9% improvement; applied to world model: +46.3% improvement.
Pruning is critical: without pruning, reward drops to 1.5 and score to 1.6% vs. 6.7/15.3% with full pruning.

Limitations¶

Performance degrades when multiple environment modifications are combined simultaneously (e.g., Terrain+Survival+Task Dependency), because learned rules are tuned to specific dynamics and struggle with compound interactions.
High variance on "Survival" setting due to stochastic enemy spawns (cows), indicating sensitivity to environmental randomness not captured by deterministic rules.
Code rules depend heavily on observation information; the approach has not been demonstrated in raw visual environments (only text/symbolic observations).
Comparison with DreamerV3 shows WALL-E does not match RL-trained agents given sufficient training budget — the method targets sample efficiency, not asymptotic performance.
Greedy maximum-coverage pruning is a heuristic; no formal guarantees on rule-set optimality.

Relevance to Agentic AI / LLM Agents¶

WALL-E directly addresses the core bottleneck of deploying LLMs as autonomous agents in novel environments: the mismatch between LLM priors and environment-specific dynamics. By framing this as a "world alignment" problem solvable without gradient updates, it offers a practically deployable alternative to RL/fine-tuning approaches and advances the model-based agent paradigm. The integration of symbolic knowledge as executable code rules — rather than natural-language prompts — provides deterministic constraint enforcement, a key reliability requirement for agentic systems. The work situates itself in a growing line connecting world models, inductive program synthesis, and MPC-style planning for LLM agents (alongside RAP, RAFA, WorldCoder), pushing toward agents that safely self-correct before acting.