Skip to content

ThinkBLOX: 3D Indoor Scene Generation with Progressive Reasoning

🕒 Published (v1): 2026-07-15 07:42 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

ThinkBLOX is a VLM-based progressive reasoning framework for 3D indoor scene generation that replaces one-shot layout prediction with a state-conditioned, step-by-step reason-then-act process. It combines supervised fine-tuning on a large CoT-annotated dataset with a novel Tier-Decoupled GDPO reinforcement learning scheme to jointly optimize physical validity, semantic plausibility, and reasoning–action consistency. The result is a system that natively supports local interactive editing without requiring global re-optimization.

Problem

Most recent VLM-based scene generators use a one-shot paradigm — predicting the entire layout \(L\) in a single forward pass — which makes local edits (inserting, moving, or replacing individual objects) prohibitively expensive, as any modification triggers global re-optimization or full reconstruction. Classical autoregressive graphics methods place objects sequentially but are restricted to closed-set categories and cannot follow open-vocabulary instructions. Neither family of methods delivers both semantic flexibility and fine-grained interactive editability.

Method

ThinkBLOX frames layout generation as a Markovian progressive reasoning process: given text prompt \(p\) and target object set \(O = \{o_n\}_{n=1}^N\), the model generates layout \(L = \{(o_n, p_n, \theta_n, b_n)\}_{n=1}^N\) through sequential reasoning steps \(r_1, r_2, \ldots, r_N\), where each step is conditioned on the current rendered scene image \(I_{n-1}\) and partial layout \(L_{n-1}\):

\[r_n \sim p(r_n \mid p, L_{n-1}, I_{n-1}, o_n)\]

Placement parameters are recovered via \(( p_n, \theta_n, b_n ) = \text{Parse}(r_n)\).

Training proceeds in two stages:

  1. SFT on ThinkBLOX-Data-200K: Qwen2.5-VL-7B is fine-tuned on 224,757 procedural <think>…</think><answer>…</answer> placement pairs derived from IL3D scenes (≥5 objects each). CoT rationales — covering spatial occupancy analysis, semantic layout reasoning, and bounding-box inference — are generated by GPT-4o from ground-truth layouts.

  2. Tier-Decoupled GDPO (RL post-finetuning): Three reward tiers are defined:

  3. \(S_\text{hard}\): format reward \(R_\text{format} \in \{-1, 0\}\); collision penalty \(R_\text{coll} = -\tanh(\lambda C_\text{coll})\) with \(\lambda=5\); boundary adherence \(R_\text{layout} = -\tanh(\text{penetration depth})\).
  4. \(S_\text{soft}\): functional alignment \(R_\text{align} = \mathbf{v}_\text{func} \cdot \mathbf{v}_\text{target}\) (cosine similarity); semantic consistency \(R_\text{render} = e^{\log p_\text{yes}}\) queried from GPT-4o.
  5. \(S_\text{cons}\): reasoning consistency \(R_\text{cons}(r_n, a_n) = -\text{dist}(\text{Parse}(r_n), a_n)\).

Within each tier \(k\), rewards are aggregated and normalized independently to form tier-wise advantages \(A_k^{(i,j)}\). The final advantage is \(A_\text{sum}^{(i,j)} = \sum_k w_k A_k^{(i,j)}\), followed by batch-wise normalization. This intra-tier coupling / inter-tier decoupling prevents reward conflicts during multi-objective policy optimization via GDPO.

Key Contributions

  • ThinkBLOX framework: VLM progressive reasoning-action pipeline for 3D indoor scene generation, natively supporting global and local layout generation/rearrangement.
  • ThinkBLOX-Data-200K: 224,757 procedural placement pairs from IL3D, each with multi-view rendered context, GPT-4o-generated CoT rationales, and structured JSON layouts.
  • Tier-Decoupled GDPO: RL scheme that partitions heterogeneous rewards (\(S_\text{hard}\), \(S_\text{soft}\), \(S_\text{cons}\)) into decoupled normalization tiers, enabling stable simultaneous optimization of physical validity, semantic aesthetics, and reasoning–action consistency.
  • Interactive editability: Progressive formulation supports object-level operations (add, move, replace, re-placement) without global scene reconstruction.

Results

The paper text provided is truncated before the quantitative experiments section; specific benchmark numbers and per-method comparisons are not present in the supplied text. The abstract claims ThinkBLOX "significantly outperforms recent one-shot and iterative baselines in physical plausibility, semantic alignment, and interactive editability," but no figures or tables are available to report.

Limitations

  • CoT rationales are generated by GPT-4o rather than human annotators, introducing potential noise or hallucinated spatial reasoning in training data.
  • Semantic consistency reward (\(R_\text{render}\)) relies on GPT-4o inference at training time, adding cost and a dependency on a closed proprietary model.
  • The approach is evaluated on synthetic IL3D-derived data; generalization to real-world captured scenes or highly diverse floor plans is not demonstrated in the provided text.
  • The text provides no ablation quantifying the individual contribution of each reward tier or the benefit of tier decoupling over flat multi-reward GDPO.
  • Progressive inference is inherently sequential (one object at a time), which may be slower than one-shot methods for large scenes.

Relevance to Vision-Language Models

ThinkBLOX directly advances the application of VLMs (specifically Qwen2.5-VL-7B) to structured 3D spatial reasoning by demonstrating that chain-of-thought grounding can bridge the gap between high-level semantic planning and precise geometric action. The Tier-Decoupled GDPO formulation is a transferable technique for multi-objective RL fine-tuning of VLMs in domains where rewards are heterogeneous and partially competing — a broadly relevant problem for embodied, robotic, and generative spatial AI. It also contributes a large-scale CoT-annotated multimodal dataset where each sample ties visual scene state to explicit reasoning trajectories, a data paradigm increasingly central to training stronger VLMs for grounded tasks. For researchers tracking VLMs, this work represents a concrete example of how SFT + RL post-training on structured reasoning data can extend VLM capabilities from perception and QA into generative action planning under physical constraints.