Skip to content

MentalThink: Shaping Thoughts in Mental SVG World

🕒 Published (v1): 2026-07-03 00:00 UTC · Source: HuggingFace · link

Why this paper was selected

Novel think-with-SVG paradigm enabling executable visual reasoning in MLLMs

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

MentalThink introduces a think-with-SVG paradigm that equips MLLMs with an executable mechanism for "mental" visualization: the model generates SVG code as an intermediate reasoning step, renders it deterministically, and conditions subsequent reasoning on the rendered image. A two-stage training framework (SFT for syntactic alignment followed by multi-turn GRPO reinforcement learning) teaches the model to iteratively construct, inspect, and refine visual hypotheses. The approach achieves 55.1% on VSIBench and 76.0% on MindCube, surpassing both proprietary models (GPT-5, Gemini-2.5-pro) and all spatial-specific baselines from a 7B backbone.

Problem

Existing MLLM reasoning paradigms face a fundamental trade-off: language-centric CoT methods verbalize spatial information without a structured spatial representation, causing the reasoning chain to drift from geometric reality and produce visually unfaithful hallucinations. Structured alternatives—bounding boxes, cognitive maps—reduce rich scenes to sparse primitives, losing fine-grained shape information. External-tool-based approaches break end-to-end differentiability and are constrained by rigid APIs. No prior work uses a rich, compositional, machine-readable visual format as the medium of thought rather than the output of thought.

Method

The core is the SVG-augmented reasoning state \(s^{svg} = (q, v, \xi^{svg})\) where each reasoning step \(\xi_i^{svg} = (\ell_i, \sigma_i)\) interleaves natural language \(\ell_i \in \mathcal{L}\) with optional SVG code \(\sigma_i \in \mathcal{S} \cup \{\emptyset\}\).

Stage 1 – SFT cold start. The model is trained on ~200k samples across three functional pillars: (1) 50k Visual-Syntactic Alignment (pixel-to-vector bridging), (2) 50k Visual Thought Externalization (explicit SVG-mediated reasoning chains), (3) 100k Fundamental Spatial Perception from diverse spatial datasets. Loss minimizes \(\mathcal{L}_{SFT}(\theta) = -\mathbb{E}_{\tau \sim D_{SFT}} \sum_t \log \pi_\theta(\xi_t^{svg} \mid s_t^{svg})\).

Stage 2 – Multi-turn GRPO RL. Each turn follows an interactive loop: 1. Policy rollout: sample action \(a_t = (\ell_t, \sigma_t) \sim \pi_\theta(\cdot \mid s_t)\) 2. Deterministic rendering: \(I_t = R(\sigma_t)\) via parse → layout → rasterize pipeline 3. State transition: \(s_{t+1} = s_t \oplus (\ell_t, \sigma_t, I_t)\)

The hybrid reward is: $\(R(\tau) = \lambda_{fmt} \cdot r_{format}(\sigma_t) + \lambda_{ans} \cdot r_{answer}(a_T) \cdot \mathbb{1}(t = T)\)$ with \(\lambda_{fmt} = 0.25\), \(\lambda_{ans} = 0.75\). \(r_{format}\) is a dense grammar-validity reward; \(r_{answer}\) is sparse, firing only at final step \(T\). Maximum horizon \(T_{max} = 5\). The backbone is Qwen2.5-VL-7B, trained on 32 H800 GPUs.

Key Contributions

  • First to use SVG as an intermediate reasoning modality rather than a generation target, enabling "mental rendering" within the thinking chain
  • Formal framework for SVG-augmented CoT with a defined structured visual language \(\mathcal{S} = \{\sigma : \sigma = \langle E, A, H \rangle\}\) enforcing syntactic validity and deterministic renderability
  • Two-stage progressive training (SFT syntactic alignment → multi-turn GRPO) enabling both structural grounding and iterative self-correction
  • Hybrid dense/sparse reward function that simultaneously enforces SVG structural integrity and answer correctness
  • Empirical demonstration of adaptive System 1/System 2 behavior: perception tasks invoke SVG at only 18.5% rate with 1.19 avg turns, while complex reasoning tasks invoke it more frequently

Results

  • VSIBench: 55.1% (MentalThink-RL) vs. 31.0% (Qwen2.5-VL-7B backbone), +24.1%; beats GPT-5 (55.0%) and Gemini-2.5-pro (53.5%)
  • MindCube: 76.0% vs. 36.0% (backbone), +40.0%; outperforms MindCube-3B-RawQA-SFT (51.7%) by 24.3 points
  • ViewSpatial: 62.5% vs. 36.8% (backbone), +25.7%
  • OmniSpatial: 43.4% (second-best open-source after InternVL3-8B at 49.3%)
  • Ablation (VSIBench avg):
  • Backbone only: 31.0%
  • +SVG-Reasoning data (SFT): 44.4% (+13.4%)
  • +General Spatial Perception data: 53.9% (+9.5%)
  • +RL: 55.1% (+1.2%)
  • Paradigm comparison (controlled):
  • Think-with-SVG vs. Think-with-Text: MindCube 57.8% vs. 42.5%; VSIBench 39.4% vs. 24.5%
  • Think-with-SVG vs. Think-with-BBox: MindCube 57.8% vs. 41.1%; VSIBench 39.4% vs. 17.4%

Limitations

  • All experiments use a single 7B backbone (Qwen2.5-VL-7B); generalization across backbone scales is not demonstrated
  • Maximum reasoning horizon is hard-capped at \(T_{max} = 5\) turns, limiting depth for problems requiring more iterative refinement
  • Hard syntax failure (when \(\sigma_t \notin \mathcal{S}\)) terminates the trajectory, making robustness to SVG generation errors critical
  • SVG is inherently 2D; the model must project 3D spatial understanding into a 2D vector representation, which may lose depth cues
  • SFT data is synthesized via a teacher-student pipeline, introducing potential quality ceiling from the teacher model
  • RL reward relies on final-answer correctness which may be ambiguous or proxy-misaligned for open-ended spatial descriptions

Relevance to Vision-Language Models

MentalThink directly advances the MLLM reasoning paradigm by replacing purely linguistic chain-of-thought with a structured, executable visual intermediate, addressing a core weakness of VLMs—poor geometric grounding. The use of SVG as a natively code-compatible visual language exploits VLMs' pretraining on markup languages (HTML/XML), making it architecturally well-suited without requiring external visual encoders. The multi-turn GRPO training loop, where rendered images are fed back as new visual tokens, establishes a closed-loop visual self-correction mechanism that generalizes beyond spatial tasks to any domain requiring structured geometric reasoning. For VLM researchers, this offers a concrete alternative to the dominant text-only CoT paradigm and demonstrates that executable intermediate representations—rather than descriptive ones—can dramatically improve visual faithfulness.