Skip to content

MindOmni: Unleashing Reasoning Generation in Vision Language Models with RGPO

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

MindOmni is a unified multimodal LLM that integrates Chain-of-Thought reasoning into image generation via a novel reinforcement learning algorithm called RGPO (Reasoning Generation Policy Optimization). It enables "reasoning generation"—producing images that require world knowledge, mathematical logic, or spatiotemporal inference—rather than merely responding to literal text prompts. The system is built on Qwen2.5-VL and OmniGen and trained in three stages culminating in RL-based CoT optimization.

Problem

Existing text-to-image systems are limited to literal textual conditioning and cannot perform explicit logical inference (e.g., resolving "animal with 3+6 lives" or "New York at noon → Sydney Opera House scene"). Unified MLLMs that support multimodal inputs still lack genuine reasoning-driven generation; prior CoT-based generation attempts (e.g., GoT) are constrained to spatial layout planning with rigid templates and separate pipeline components, and do not leverage RL to automatically produce diverse, accurate reasoning chains.

Method

MindOmni combines Qwen2.5-VL (VLM backbone) with OmniGen (decoder-only diffusion module) via a two-layer LLM decoder connector. Training proceeds in three stages:

  1. Stage 1 – Connector pretraining: Only the connector is trained on image-caption and X2Image pairs using diffusion loss and KL distillation loss (rectified flow).
  2. Stage 2 – CoT SFT: Model is fine-tuned on coarse-to-fine CoT instruction data (coarse description as answer, fine-grained description as reasoning content) within <think>...</think><answer>...</answer> tags.
  3. Stage 3 – RGPO: A GRPO-derived RL algorithm that samples G=4 rollout groups per query, each producing a text reasoning chain and an output image. Two reward signals guide policy updates:
  4. Format reward: binary check that output conforms to <think>/<answer> structure.
  5. Consistency reward: CLIP cosine similarity between generated image and ground-truth reference prompt.

Advantages are normalized across the group. Two separate KL regularizers—one for text rollouts (between πθ and πref text distributions) and one for image rollouts (KL on latent features via feature mapping ϕ)—prevent catastrophic forgetting of base generation ability. The final objective clips the probability ratio (PPO-style) and subtracts both KL penalties weighted by β^T=0.01, β^I=0.06.

For vision editing, VAE features are concatenated alongside ViT features to preserve low-level texture and spatial layout.

Key Contributions

  • RGPO algorithm: extends GRPO to multimodal generation by computing rewards from both image (CLIP-based consistency) and text (format adherence), with separate KL divergence regularizers for text and visual modalities.
  • Unified three-stage training pipeline enabling a single 7B model to handle vision understanding, reasoning generation, and fine-grained visual editing end-to-end.
  • Reasoning generation capability: explicit CoT-driven image synthesis that handles mathematical equations, cultural/spatiotemporal knowledge, and physical reasoning.
  • High-fidelity editing: dual ViT + VAE encoding to preserve texture, posture, and spatial layout in reference-based edits.
  • Demonstration that CoT SFT (Stage 2) is necessary for RGPO to be effective; skipping it degrades both GenEval and WISE scores.

Results

  • WISE reasoning generation benchmark (overall score): MindOmni 0.71 (with thinking), vs. BAGEL 0.60, FLUX 0.50, PixArt 0.47, OmniGen baseline 0.44; MindOmni* (higher-quality data + thinking) reaches 0.71.
  • GenEval (text-to-image alignment): MindOmni 0.83 overall, best among unified MLLMs; comparable to BAGEL (0.82) and BLIP-3o (0.83).
  • DPG-Bench: MindOmni achieves 89.1–89.7 global score, on par with BAGEL and BLIP-3o.
  • MMMU (understanding): MindOmni 51.6% vs. Janus-Pro 41.0% (+10.6 pp) and MetaMorph 41.8% (+9.8 pp on MMBench context).
  • MMBench: 83.2%, vs. LLaVA-OV 80.8%, Janus-Pro 79.2%.
  • Ablation: removing Stage 2 drops WISE from 0.60 to 0.49; removing consistency reward drops WISE from 59.8 to 59.4; removing image KL drops GenEval from 0.81 to 0.78.

Limitations

  • Consistency reward uses CLIP similarity to a text reference prompt rather than ground-truth image comparison, which may not fully capture semantic correctness of complex reasoning outputs.
  • The training corpus for RGPO consists of plain-text reasoning sequences; the model has not yet demonstrated robust reasoning generation for arbitrary out-of-distribution queries.
  • Increasing CoT output length during RL does not monotonically improve WISE score, indicating that length-based scaling of test-time compute does not straightforwardly transfer to generation quality.
  • Fine-grained tasks (chess, tic-tac-toe) require domain-specific fine-tuning data, limiting zero-shot generalization.
  • No evaluation of hallucination rates or factual grounding in the generated images beyond CLIP-based proxy metrics.

Relevance to Vision-Language Models

MindOmni directly advances the VLM frontier by extending RL-based reasoning (DeepSeek-R1/GRPO paradigm) from text-only and visual-understanding tasks into image generation, demonstrating that the same CoT + RL recipe that improves VQA and math reasoning can drive generative image semantics. The dual-modality KL regularization in RGPO is a practically important technique for preserving VLM understanding capability while fine-tuning a generation head—relevant to any researcher building joint understanding-generation models. The work also highlights that explicit, verifiable CoT supervision (Stage 2 SFT) is a prerequisite for RL to be effective in generation, paralleling findings in text-domain RLHF research.