Active Test-time Vision-Language Navigation¶

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

ATENA (Active TEst-time Navigation Agent) is a test-time adaptation framework for Vision-Language Navigation (VLN) that replaces blind entropy minimization with outcome-conditioned entropy optimization guided by sparse episodic human feedback. It introduces Mixture Entropy Optimization (MEO) and a Self-Active Learning (SAL) mechanism to calibrate uncertainty and reduce human labeling burden during online deployment.

Problem¶

VLN policies trained offline degrade at test time under distributional shift. Existing test-time adaptation (TTA) approaches—principally entropy minimization (e.g., Tent, FSTTA)—are agnostic to navigation outcome: they increase confidence uniformly, which compounds errors in sequential decision-making when the agent overcommits to incorrect actions. There is no principled way to direct entropy optimization without some form of episodic supervision.

Method¶

ATENA operates entirely at test time, adapting a pre-trained VLN policy episode-by-episode via two components:

Mixture Entropy Optimization (MEO): At each step, a mixture action distribution \(q_\text{mix}\) is formed as a convex combination of the policy's predicted action distribution \(\pi_\theta\) and a pseudo-expert distribution \(q_\text{pseudo}\)—a one-hot over the agent's selected action \(a^\text{sel}_t\), i.e., \(q_\text{mix} = \lambda q_\text{pseudo} + (1-\lambda)\pi_\theta\). This sharpens the distribution around the chosen action. Mixture entropy is minimized for successful episodes (reinforcing correct actions) and maximized for failed ones (suppressing incorrect actions). The loss is \(\mathcal{L}_\text{mix} = \mathbb{1}_\text{success} \cdot H'(q_\text{mix}) - (1-\mathbb{1}_\text{success}) \cdot H'(q_\text{mix})\). Because \(q_\text{mix}(a^\text{sel}_t) > \pi_\theta(a^\text{sel}_t)\) for \(\lambda>0\), gradient updates on \(q_\text{mix}\) exert stronger directional influence than optimizing \(\pi_\theta\) alone.

Self-Active Learning (SAL): After each episode, average step-level action entropy is compared to threshold \(\delta\). If above \(\delta\) (uncertain), a human oracle provides a binary success/failure label. If below \(\delta\) (confident), a lightweight self-prediction head \(f_\phi\)—trained online via binary cross-entropy on pooled hidden states \(s_\text{avg}\)—predicts the outcome autonomously. Both labels feed MEO and jointly train \(f_\phi\), so the agent continuously receives adaptation signal without requiring dense human annotation. Total loss: \(\mathcal{L} = \mathcal{L}_\text{mix} + \gamma \mathcal{L}_\text{self}\).

Key Contributions¶

First active learning framework for online VLN TTA, introducing episodic binary human-oracle feedback to guide entropy optimization direction.
MEO: mixture action distribution that amplifies correct-action reinforcement and incorrect-action suppression more than vanilla entropy optimization.
SAL: uncertainty-thresholded strategy that routes uncertain episodes to human feedback and confident episodes to a self-trained prediction head, enabling continuous adaptation with reduced human burden.
Demonstrated plug-in compatibility with multiple pre-trained VLN backbones (HAMT, DUET, BEVBert, ETPNav, GOAT) across discrete and continuous action spaces.

Results¶

REVERIE Val Unseen: ATENA on GOAT improves SR by +25.72% (53.82→67.66), SPL +41.65% (37.52→53.15), RGSPL +47.41% (27.00→39.80) vs. base; DUET SR improves +44.98% (46.98→68.11).
REVERIE Test Unseen: ATENA on GOAT improves OSR +4.59%, SR +7.47%, SPL +15.52%, RGSPL +18.13% over base.
R2R Val Unseen: ATENA on GOAT improves SPL by +2.91% over FSTTA; on DUET achieves same SR as FSTTA but with 11.69% shorter trajectory length.
R2R-CE Val Seen: ATENA on BEVBert improves SPL by +6.7% over base.
MEO vs. AL baselines (Table 4, DUET+REVERIE): MEO with AL achieves Val Unseen SR 63.70 vs. TENT+AL 55.69 and FSTTA+AL 46.95, while requesting human feedback on only ~55% of episodes vs. >85% for baselines.
SAL ablation (Table 5, DUET+REVERIE): Adding SAL over MEO-only improves Val Unseen SR +6.92%, SPL +7.84%, RGSPL +12.32%.
Combination weight: \(\lambda=0.4\) optimal; any \(\lambda>0\) outperforms vanilla entropy (\(\lambda=0\)).
Sampling strategy: Uncertainty-based sampling outperforms random and consecutive baselines, and is competitive with full (100%) feedback while using only ~60%.

Limitations¶

Policy parameters are updated only after episode completion, not during navigation; step-level errors cannot be corrected mid-episode.
Episodic binary feedback is too sparse to drive large gains on R2R variants, where dense step-by-step supervision dominates training signal.
Requires a human oracle to be available during test-time deployment, even if infrequently queried; fully autonomous deployment is approximated but not guaranteed.
The self-prediction head is initialized at test time and must accumulate enough episodes before its labels become reliable, introducing a cold-start period.

Relevance to Vision-Language Models¶

ATENA directly addresses the deployment brittleness of multimodal instruction-following agents—a central concern in VLM-grounded embodied AI—by demonstrating that sparse, binary episodic feedback can substitute for dense expert supervision during online adaptation. The mixture entropy formulation is model-agnostic and applies to any VLN backbone with a softmax action head, making it immediately relevant to transformer-based VLMs used in navigation (HAMT, DUET, GOAT). The work connects to the broader TTA literature for VLMs, showing that outcome-aware entropy calibration outperforms uniform confidence maximization, which has implications for any sequential VLM application where overconfidence causes error accumulation. It also introduces a lightweight human-in-the-loop paradigm compatible with practical latency and expertise constraints, a design consideration relevant to deploying instruction-following VLMs in interactive real-world settings.