Skip to content

Think When It Matters: Conditional VLM Reasoning for Social Navigation with RL Policies

🕒 Published (v1): 2026-07-13 01:27 UTC · Source: Arxiv · link

Why this paper was selected

Conditional VLM reasoning with RL for social navigation; conditional compute in embodied agents

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

HUMA is a hybrid robot navigation framework that selectively invokes a fine-tuned VLM only when a human enters the robot's proximity zone, otherwise delegating to a lightweight RL policy. This conditional activation resolves the latency-vs-semantics trade-off inherent to prior pure-RL and pure-VLM social navigation systems. On Social-MP3D it achieves a 20% success rate gain and 20% collision reduction over the best baseline.

Problem

Pure RL navigation policies are fast but lack semantic social reasoning and generalize poorly to complex human interactions. VLM-based navigation systems provide rich contextual understanding but are too computationally expensive for real-time deployment. No prior system achieves both simultaneously; the literature lacks a principled arbitration mechanism between the two regimes.

Method

HUMA frames navigation as a Goal-Conditioned POMDP and routes control through a Personal Space Compliance (PSC) switch. The switch computes a rolling window compliance score:

\[\text{PSC}_W = \frac{1}{W} \sum_{i=t-W+1}^{t} \mathbf{1}[d_i > 1.0\,\text{m}]\]

When \(\text{PSC}_W \geq \theta\), the RL policy (Falcon backbone, DD-PPO trained) handles navigation. When \(\text{PSC}_W < \theta\) (a human is too close), control passes to a Qwen3-VL 2B model fine-tuned with LoRA (\(r=16\), \(\alpha=32\), ~17M trainable parameters, 0.81% of total). The VLM receives a fused visual token: a depth map colorized with a JET colormap, Canny-edge contours overlaid, and a linguistically encoded goal vector. It outputs discrete action tokens (e.g., [FORWARD], [LEFT]) for the next \(n\) timesteps. The RL reward combines standard PointNav rewards with a Social Cognition Penalty \(R^t_\text{scp} = r_\text{coll} + r_\text{prox} + r_\text{traj}\), where \(r_\text{prox}\) scales exponentially inside a 2 m safety envelope and \(r_\text{traj}\) penalizes future path intersection with human trajectories over a look-ahead horizon \(H\).

Key Contributions

  • Conditional VLM arbitration: PSC-score-gated switching mechanism that invokes the VLM only during proximity-sensitive episodes, avoiding unnecessary inference overhead.
  • Visual token pre-processing for depth-only VLM input: JET colormap normalization + Canny edge overlay produces denser spatial tokens than raw RGB or depth, with lower latency (2,425 ms/call vs. 5,427 ms/call for RGB+depth).
  • LoRA-adapted Qwen3-VL 2B trained on SNEI dataset, achieving action accuracy within 1% of larger models while using 33% fewer parameters.
  • Social Cognition Penalty with exponential proximity term and multi-step trajectory obstruction penalty integrated into DD-PPO training.
  • Real-world deployment on the Mirokaï mobile robot validating the hybrid policy in physical environments.

Results

  • Social-MP3D: SR 70.35% vs. NavThinker 47.33% (+23.0 pp), H-Coll 20.82% vs. NavThinker 37.67% (−16.85 pp), SPL 59.41% vs. NavThinker 41.71%.
  • Social-HM3D: SR 62.07% vs. NavThinker 59.46% (+2.6 pp), H-Coll 34.48% vs. NavThinker 39.09% (−4.6 pp); SPL slightly below Falcon (54.54% vs. 52.05% for NavThinker).
  • PSC remains competitive on both benchmarks (92.32% HM3D, 92.96% MP3D) despite optimizing for SR and safety.
  • VLM ablation: fine-tuned 2B model achieves action accuracy within 1% of best baseline (3B+ range) with 33% fewer parameters.
  • Optimal PSC proximity threshold: \(r = 0.80\) m maximizes SR (62.07%) and SPL (54.54%) while minimizing H-Coll (34.48%).
  • Depth JET colormap input outperforms RGB-only and mixed inputs; adding arrows provides no additional gain for depth inputs.

Limitations

  • Environment-freeze assumption during VLM inference: when humans move during the VLM's multi-step action generation, performance degrades from 62% to ~56% SR, which is not realistic for dynamic real-world scenarios.
  • Single-camera depth-only input constrains scene understanding; RGB integration increases latency substantially (>2× ms/call).
  • VLM inference latency (~2,425 ms/call with depth JET) still imposes a real-time constraint—acceptable only because VLM calls are sparse, but the threshold \(r = 0.80\) m limits how early the robot can begin reasoning.
  • Benchmarks are simulation-based (HM3D, MP3D); real-world Mirokaï deployment is qualitative, not quantitatively evaluated.
  • No explicit handling of multi-human simultaneous proximity events or group dynamics beyond minimum-distance triggering.

Relevance to Vision-Language Models

HUMA demonstrates a practical strategy for deploying VLMs in latency-constrained perception-action loops: rather than replacing a fast policy, the VLM serves as a sparse high-level reasoner activated by a learned compliance signal. The LoRA fine-tuning on social navigation QA datasets (SNEI, MUSON) and the custom visual token design (JET colormap + edge overlays) show how domain-specific VLM adaptation can close the gap with much larger models. For VLM researchers, the PSC-gating paradigm generalizes as a template for conditional VLM invocation in any embodied or real-time agent system where full-rate inference is infeasible. The ablation comparing training datasets (action accuracy vs. semantic quality trade-off) is directly informative for instruction-following and embodied VLM fine-tuning work.