Skip to content

MindJourney: Test-Time Scaling with World Models for Spatial Reasoning

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

MindJourney is a training-free test-time scaling framework that couples a frozen VLM with a controllable video-diffusion world model to enable active 3D spatial exploration. Given a spatial reasoning query, a Spatial Beam Search iteratively plans egocentric camera trajectories, renders novel views via the world model, and accumulates the most informative imagined observations for final answer generation. On the SAT benchmark, this yields an average 7.7% accuracy gain across four heterogeneous VLMs with no fine-tuning.

Problem

State-of-the-art VLMs perceive 2D images but lack an internal 3D world model, causing them to fail on perspective-shift and egocentric-motion reasoning tasks that require mentally simulating movement through a scene. Existing test-time scaling methods (best-of-n, guided decoding, RL chain-of-thought) are text-centric and supply no geometry-aware physical evidence to help with this gap.

Method

MindJourney consists of two tightly coupled components:

Video-Diffusion World Model: Given a reference RGB frame and a camera pose trajectory expressed in SE(3), a pose-conditioned video diffusion model (either the authors' Search World Model, SWM, built on Wan2.2-TI2V-5B, or the off-the-shelf Stable-Virtual-Camera) synthesizes a coherent egocentric rollout. The action space is discrete: {move-forward d, turn-left θ, turn-right θ}.

Spatial Beam Search: At each of n steps, every beam node is expanded by appending up to k consecutive repetitions of each primitive action (yielding ≤3k candidates per node). The world model renders the resulting frames. A Search VLM scores each candidate on two criteria—exploration utility s_exp (should we keep exploring this path?) and helpfulness s_help (does this view answer the question now?)—using natural-language trajectory descriptions plus the rendered frame as input. Candidates below thresholds γ_exp / γ_help are pruned; the top-B by s_exp form the next beam layer; the top-H by s_help are written to an evidence buffer. After search terminates, a QA VLM receives the original image plus all buffered observations and descriptions to produce the final answer.

SWM is trained on a mix of Habitat 2.0 synthetic navigation data (for geometric precision), RealEstate-10K, and DL3DV-10K (for visual diversity).

Key Contributions

  • First training-free test-time scaling framework for spatial reasoning that uses an interactive video-diffusion world model instead of text-only search.
  • Spatial Beam Search algorithm with dual VLM-based heuristics (s_exp, s_help) for question-aware pruning of a discrete egocentric trajectory space.
  • Model-agnostic: demonstrated gains on GPT-4o, GPT-4.1, InternVL3-14B, and OpenAI o1, with two distinct world models (SWM and SVC).
  • Shows that world-model-based TTS is orthogonal to RL-based TTS (o1 + MindJourney outperforms either alone).

Results

SAT-Real (150 real-image queries): - GPT-4o: 60.3% → 70.6% (+10.3pp) with SWM - GPT-4.1: 74.0% → 80.6% (+6.6pp) with SWM - InternVL3-14B: 59.3% → 66.6% (+7.3pp) with SWM - OpenAI o1: 74.6% → 84.7% (+10.1pp) with SWM (new SOTA on SAT-Real) - Average gain across four VLMs: +7.7pp

SAT-Synthesized (500-question subset): - Average gain: +8.0pp - GPT-4o: 61.0% → 72.3% (+11.3pp) with SVC - OpenAI o1: 72.4% → 78.6% (+6.2pp) with SVC - MindJourney-augmented GPT-4.1 surpasses vanilla o1 on both splits

Limitations

  • Single-reference-view assumption: cannot treat multiple query images as separate exploration portals; multi-source fusion is left for future work.
  • World models are query-agnostic: the generator may hallucinate views irrelevant or contradictory to the specific question's implicit assumptions.
  • SWM trained mostly on indoor synthetic data degrades on outdoor scenes at deeper search depths (>2 steps on SAT-Real), as shown in ablations.
  • Deeper search beyond 3 steps is currently impractical due to compounding world-model generation errors.
  • Computational cost scales with beam width and search depth; o1 experiments required offloading search to GPT-4o to remain tractable.

Relevance to Vision-Language Models

MindJourney directly addresses a core limitation of VLMs—their inability to reason about 3D structure from static 2D observations—without requiring any architectural modification or fine-tuning, making it a plug-and-play upgrade path for any capable VLM. It demonstrates that test-time compute can be spent not only on language-space search (chain-of-thought, RL) but also on perception-space expansion via generative world models, opening a new axis of test-time scaling for multimodal reasoning. The finding that world-model TTS is additive on top of RL-based TTS (o1) suggests these mechanisms capture largely orthogonal information sources. For VLM researchers, the work establishes spatial 3D reasoning as a concrete failure mode and benchmarks it rigorously, motivating future VLMs to internalize 3D world priors during training rather than deferring entirely to external simulators.