ABot-N1: Toward a General Visual Language Navigation Foundation Model

🕒 Published (v1): 2026-07-11 16:21 UTC · Source: HuggingFace · link

Why this paper was selected

Foundation model for Visual Language Navigation; unifies spatial reasoning and embodied versatility

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

ABot-N1 is a Visual Language Navigation foundation model that decouples cognition from control via a slow–fast dual-system architecture, using explicit Chain-of-Thought reasoning and egocentric pixel goals as a universal interface across five navigation tasks. A 4B-parameter slow VLM produces CoT traces and pixel anchors; a 2B-parameter fast VLM consumes these to generate continuous SE(2) waypoints at native control frequency. GRPO-based post-training aligns the slow system's reasoning directly with navigation outcomes rather than token-level supervision.

Problem

Monolithic end-to-end navigation policies suffer from three compounding failures: (1) coordinate drift when SD-map routing shifts targets into non-traversable regions, (2) erosion of pretrained semantic priors during fine-tuning on object-search trajectories, and (3) complete opacity that prevents root-cause attribution of failures to perception, reasoning, or control. Additionally, cross-task generalization across point-goal, object-goal, POI-goal, instruction-following, and person-following has required separate task-specific architectures with disjoint goal interfaces that resist joint training.

Method

ABot-N1 employs an asynchronous slow–fast architecture:

Slow System (Deliberative Reasoner): A 4B-parameter Qwen-3.5-4B VLM operating at low frequency. At each invocation \(n\) it receives a short memory buffer of historical frames \(I_{1:K}^{ref,mem}\), a tri-view (left/front/right) observation \(I_{t_{ri}}^{ref}\), task specification \(g = \ell\) (unified as natural-language instruction for all five tasks), and the prior decision \((C_{n-1}, p_{n-1})\). It outputs: $\(f_{slow}\!\left(I_{1:K}^{ref,mem}, I_{t_{ri}}^{ref}, g, C_{n-1}, p_{n-1}\right) = (C_n, p_n)\)$ where \(C_n\) is a CoT trace and \(p_n\) is a set of pixel goals—2D image-space anchors in two flavors: an Affordance Pixel marking the next safe traversable waypoint (~3 m indoors / ~5 m outdoors), and a Target Pixel marking the visible goal entity. CoT is emitted only for semantically demanding tasks (instruction-following, object-goal); other tasks rely on pixel goals alone.

Fast System (Action Expert): A 2B-parameter lightweight VLM that ingests the dual guidance \((C_n, p_n)\) plus real-time observations. A learnable action query attends to output hidden states via a QFormer module, and an MLP decodes queries into continuous SE(2) waypoints \(a_i = (x_i, y_i, \sin\theta_i, \cos\theta_i, c_i)\) with a binary completion flag \(c_i\).

Training: Pretraining on 30M samples with pixel-grounded CoT trajectories across all five tasks, followed by GRPO-style post-training on the slow system using downstream navigation rewards (goal-arrival and progress signals), not just token-level targets. Balanced sampling across tasks is applied for the GRPO dataset to prevent task imbalance.

Unified Goal Interface: All five task types are expressed as natural-language instruction \(\ell\), reducing heterogeneous goal modalities (metric offsets, route descriptions, category phrases, POI names, person attributes) to a single policy input.

Key Contributions

• Slow–fast VLA architecture with a structured pixel-goal bottleneck that is human-readable, imitation-friendly, and amenable to RL at the reasoning level—distinct from opaque latent vectors used in prior dual-system models.
• GRPO post-training for the slow reasoner, aligning CoT-plus-pixel-goal outputs with actual navigation success rather than linguistic plausibility.
• Unified pixel-goal interface that reduces all five navigation tasks to "track CoT-explained pixels," enabling a single policy with shared parameters across point-goal, object-goal, POI-goal, instruction-following, and person-following.
• ABotN-PointBench: new open-source benchmark for point-goal navigation spanning short/long-range, indoor/outdoor, and hierarchical difficulty levels.
• ABotN-POIBench: new open-source benchmark for POI-goal navigation in realistic commercial districts.
• Real-world edge deployment validated on commodity hardware using only low-fidelity SD maps.

Results

• POI-Goal (ABotN-POIBench): arrival rate boosted by +35.0% to 77.3% over prior best.
• Point-Goal (ABotN-PointBench): 95.4% SR (indoor) and 92.9% SR (outdoor) in complex scenes.
• Instruction-Following (VLN-CE R2R/RxR): state-of-the-art performance (specific numbers not reproduced in provided text excerpt).
• Object-Goal (Short-Horizon OVON): state-of-the-art, outperforming both specialized and multitask baselines.
• Person-Following (EVT-Bench): superior robustness over prior methods.
• Single unified checkpoint achieves leading performance across all five benchmarks simultaneously, exceeding both task-specific specialists and prior generalist foundation models (NavFoM, ABot-N0).

Limitations

• The slow reasoner (4B VLM) introduces latency that necessitates asynchronous decoupling; the system's real-world performance depends on the fast system correctly interpolating between slow-system invocations.
• CoT reasoning is suppressed for point-goal, POI-goal, and person-following tasks—these rely solely on pixel goals, potentially underutilizing the slow system's semantic capacity for those tasks.
• Relies on SD maps for urban navigation; robustness to complete map absence or severe map degradation is not evaluated.
• The provided text is truncated; specific numerical baselines for VLN-CE R2R/RxR and OVON are not available in the supplied excerpt, limiting full reproducibility assessment.
• Pixel-goal accuracy is inherently bounded by the slow VLM's visual grounding capability; long-tail visual semantics may still produce incorrect anchors despite GRPO alignment.
• Scaling analysis (ABot-N1.1) is stated as future work; current data-centric scaling properties are not empirically characterized.

Relevance to Vision-Language Models

ABot-N1 directly demonstrates how a pretrained VLM (Qwen-3.5-4B) can be adapted into an embodied spatial reasoning engine through pixel-goal grounding and GRPO post-training—a clean case study in bridging VLM semantic priors with physical control. The dual-modality interface (CoT + pixel anchors) is a concrete instantiation of visual prompting evolved from passive user marks to active model-generated spatial predictions, which is directly relevant to researchers studying VLM grounding, spatial reasoning, and chain-of-thought for visual tasks. The GRPO alignment of a VLM reasoner with outcome-based rewards (rather than token supervision) is methodologically transferable to any VLM task requiring grounded decision-making. The unified natural-language goal interface that absorbs five heterogeneous task types also advances the understanding of how instruction-following VLMs can be generalized to continuous-action embodied settings.