Skip to content

Uncertainty-quantified Rollout Policy Adaptation for Unlabelled Cross-domain Video Temporal Grounding

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

URPA adapts a GRPO-trained VLM to new video temporal grounding domains using only 100–200 unlabelled target videos by treating GRPO rollout variance as a proxy for epistemic uncertainty. Pseudo-labels are constructed by averaging predicted timestamps across rollouts, and confidence scores (via exponential decay of rollout std) weight the RL reward during test-time adaptation. This eliminates both the annotation requirement and the need to process the full target dataset.

Problem

Video Temporal Grounding (TG) with VLMs suffers from two compounding problems in cross-domain deployment: (1) GRPO-based fine-grained localisation requires ground-truth temporal boundaries to compute rewards, which are unavailable in target domains; (2) existing unsupervised domain adaptation (UDA) methods require full access to the target dataset for retraining, which is prohibitively slow and storage-intensive for video data.

Method

Source training: Fine-tune Qwen2.5-7B with GRPO using a combined reward R_s = 0.5 × R_format + 0.5 × R_tIoU, where R_format enforces <think>/<answer> structure and R_tIoU uses a relaxed ground-truth interval (boundaries extended by α=0.1 of event duration) to reduce annotation bias.

Target adaptation (URPA): For each of K unlabelled target videos (K=100 or 200), sample G=8 GRPO rollouts. Average predicted start/end timestamps across rollouts to form a pseudo-label. Estimate uncertainty as u = σ({p_start}) + σ({p_end}) across rollouts (analogous to MC Dropout). Convert to confidence c = exp(−γu) and weight the tIoU reward: R_t = 0.5 × R_format + 0.5 × R_tIoU × c. Update the model for 1 epoch; the adapted model is then applied to the full test set without further updates.

Theoretical grounding: Theorem 4.1 proves that rollout std converges (via WLLN + Pinsker's inequality) to the Bayesian predictive std as G→∞ and GRPO training drives KL(π_θ ∥ p*) → 0, establishing rollout variance as a consistent epistemic uncertainty estimator.

Key Contributions

  • A data-efficient cross-domain TG setting where adaptation uses only K=100–200 unlabelled target videos (vs. thousands in prior UDA methods).
  • URPA: the first RL-based self-learning adaptation method combining GRPO pseudo-label generation with uncertainty-weighted rewards, requiring no target annotations.
  • Theoretical proof that GRPO rollout variance approximates Bayesian predictive variance (epistemic uncertainty).
  • Relaxed tIoU accuracy reward for source training that mitigates cross-dataset annotation bias.

Results

Evaluated on six cross-domain transfers among TACoS, ActivityNet Captions, and Charades-STA (R@0.5 and R@0.7 metrics):

  • TACoS→ActivityNet R@0.5: URPA 200-shot = 41.83 vs. UDA-TSL (full-set) = 33.54 — outperforms full-set UDA.
  • TACoS→Charades R@0.5: URPA 200-shot = 55.54 vs. UDA-TSL = 41.39 — outperforms.
  • ActivityNet→Charades R@0.5: URPA 200-shot = 65.12 vs. UDA-TSL = 60.26 — outperforms.
  • Charades→ActivityNet R@0.5: URPA 200-shot = 42.57 vs. UDA-TSL = 49.48 — underperforms full-set UDA.
  • ActivityNet→TACoS R@0.3: URPA 200-shot = 21.97 vs. UDA-TSL = 42.40 — substantially underperforms.
  • Charades→TACoS R@0.3: URPA 200-shot = 16.62 vs. UDA-TSL = 36.42 — substantially underperforms.
  • Ablation (Charades→ActivityNet): uncertainty weighting alone adds +1.48 mIoU over pseudo-label baseline; combined with soft tIoU adds further gain (43.65 vs. 39.86 mIoU).
  • Optimal G=8 rollouts; γ=10 for reward scaling; both are robust to hyperparameter variation.

Limitations

  • Underperforms full-set UDA methods (UDA-TSL) on TACoS-target tasks by large margins (~20 pp R@0.3), suggesting data efficiency comes at a significant cost on certain domain gaps.
  • Requires a labelled source domain — not applicable to fully zero-shot or weakly supervised settings.
  • Pseudo-label quality depends on source model calibration; poor source training propagates noise that uncertainty weighting can only partially mitigate.
  • G=8 rollouts per video still multiplies inference FLOPs by 8× during adaptation; not analysed for on-device latency.
  • Theoretical guarantees require ε = KL(Ï€_θ ∥ p*) → 0, which is an idealized GRPO training assumption.

Relevance to Vision-Language Models

URPA directly engages the practical challenge of deploying VLMs for fine-grained spatio-temporal reasoning under domain shift — a known weakness of holistic VLM representations. By repurposing GRPO's stochastic rollouts as an uncertainty signal rather than a supervised reward mechanism, it extends RL post-training (à la DeepSeek-R1/TimeZero) into the unlabelled adaptation regime, which is the realistic deployment scenario for most video understanding applications. The relaxed tIoU reward and the MC Dropout analogy both address VLM-specific pathologies (annotation bias, overconfident predictions on out-of-distribution video), making this directly relevant to any work on test-time adaptation of VLMs or VLM-based grounding systems.