Skip to content

IPR-1: Interactive Physical Reasoner

🕒 Published (v1): 2025-11-19 13:04 UTC · Source: Arxiv · Venue: CVPR 2026 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

IPR (Interactive Physical Reasoner) couples a physics-centric discrete latent action space (PhysCode) with a latent world model that scores VLM-proposed actions via short-horizon imagined rollouts, then reinforces the VLM using GRPO on those imagined advantages. Evaluated on G2U—a new 1,000+ heterogeneous-game benchmark with Survival/Curiosity/Utility tiers—IPR with an 8B backbone surpasses GPT-5 overall and zero-shot transfers to 50 unseen games, improving steadily with more training games and interaction steps.

Problem

Existing paradigms each address only a slice of interactive physical reasoning: VLMs reason semantically but cannot predict physical consequences; world models explore broadly by imagining futures but imitate surface visual correlations rather than causal dynamics; RL optimizes under shaped rewards but overfits to task-specific shortcuts and is sample-inefficient. No unified agent and no large-scale benchmark tests whether a model can internalize shared physical and causal mechanisms across hundreds of visually heterogeneous environments and transfer zero-shot to new ones.

Method

Three staged components:

  1. PhysCode pre-training. A VQ-VAE encodes action primitives conditioned on DINOv3 visual features, optical flow (privileged training signal only), and T5 semantic embeddings. Modality dropout and gate-sparsity regularization ensure inference relies only on appearance + semantics while the codebook clusters actions by shared physics (gravity, contact, inertia) and separates them when dynamics differ. Output is a discrete latent vocabulary {c_1…K}.

  2. Latent world model + critic. Given the current DINOv3 feature z_t and a PhysCode sequence, a feature-level world model f_θ rolls out predicted features ẑ_{t+k} in latent space (L1 feature-prediction loss), avoiding pixel reconstruction noise. A Q-learning critic head V_θ(f_t, a_t) is trained via TD backups to assign predicted returns to PhysCode actions.

  3. Prediction-reinforced reasoning (GRPO). Qwen3-VL-8B's tokenizer is extended with PhysCode tokens. The VLM proposes B candidate PhysCode sequences; the world model scores each via short-horizon rollout and computes advantages A^(b). The policy is updated with GRPO: L = (1/B) Σ A^(b) log π(a^(b)|f_t, g) − β KL(π_φ ‖ π_0). At inference, the top-scoring sequence is routed to environment controls via a domain-specific mapper.

Key Contributions

  • G2U benchmark: 1,000+ games spanning 7 diversity axes (category, control interface, visual complexity, view perspective, causal mechanism, physical principle, operational difficulty) with a Maslow-inspired Survival/Curiosity/Utility evaluation hierarchy.
  • IPR paradigm: world-model rollout prediction as a reinforcement signal for VLM policy optimization in a shared action space.
  • PhysCode: physics-centric VQ latent action representation that bridges world-model prediction and VLM reasoning, resolving cross-game key aliasing and language grounding failures.
  • Empirical evidence that competence scales with number of training games and interaction steps, and zero-shot transfers to unseen games.

Results

  • Overall: IPR (8B) Avg. Rank 4.9/28; GPT-5 best variant 13.3/28; Qwen3-VL-8B baseline 18.2/28.
  • Survival: IPR Mean 0.252, Ratio@Top-3 72.0% vs. GPT-5 best 0.140/24.4% and Qwen3-VL-8B baseline 0.105/14.0%.
  • Curiosity: IPR Mean 1.173 vs. DreamerV3 best 1.402 (world models explore broadly but fail at utility).
  • Utility: IPR Mean 0.493 vs. GPT-5 best 0.371; PPO best 0.609 but collapses on curiosity.
  • Ablation (Tab. 3, normalized relative scores): VLM+PPO achieves best survival (1.00) but suppresses curiosity (1.79); IPR achieves highest curiosity (2.77) and highest utility (1.34) by combining world-model prediction with GRPO.
  • PhysCode transfer (leave-n-out): PhysCode FID 80.35 vs. keyboard 110.9 and language 82.51 in held-out evaluation; cosine similarity 0.9919 vs. 0.9894/0.9892.
  • G2U scaling (Fig. 7): zero-shot performance on all three tiers increases monotonically with training game count N.

Limitations

  • Evaluation confined to game environments; real-world and robotic transfer is stated future work, not demonstrated.
  • World-model imagination is short-horizon only; compounding errors accumulate over longer rollouts.
  • Optical flow (key to PhysCode shaping) is unavailable at inference, relying on appearance/semantics alone—a gap not fully characterized.
  • Physics taxonomy used for stratification is acknowledged as coarse, with noted anomalies (e.g., inertia already covered by projectile/impulse).
  • Human play data collected at 60 FPS for only 4 minutes per title; data quality and coverage variation across 1,000 games is not fully analyzed.

Relevance to Agentic AI / LLM Agents

IPR directly instantiates a key agenda for embodied LLM agents: replacing static, corpus-trained priors with experience-driven, interaction-grounded reasoning. The PhysCode + world-model-scoring loop is a concrete, working example of how a VLM can be given a look-ahead capability without sacrificing its semantic reasoning—a missing piece in current VLM/VLA agent designs that plan open-loop. The GRPO-based reinforcement from imagined rollouts is architecturally close to process-reward-model training in reasoning agents, but applied to physical consequence prediction rather than math verification. The scaling result—more diverse interaction experience → better zero-shot transfer—supports the thesis that agentic physical reasoning is a capability that can be accumulated progressively rather than baked in at pretraining time, which has implications for continual learning and curriculum design in agent systems.