Skip to content

Eagle 2.5: Boosting Long-Context Post-Training for Frontier Vision-Language Models

๐Ÿ•’ Published (v1): 2025-01-01 ยท Source: NeurIPS ยท Venue: NeurIPS 2025 ยท link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

๐Ÿ’ฌ Ask ChatGPTโœฆ Ask Claude

TL;DR

Eagle 2.5 is an 8B long-context VLM that achieves consistent performance scaling with increasing visual input length โ€” a property most open-source models lack. It introduces two training techniques (Automatic Degradation Sampling and Image Area Preservation) plus a dual-annotated long-video dataset (Eagle-Video-110K). The 8B model reaches 72.4% on Video-MME (512 frames), matching GPT-4o and Qwen2.5-VL-72B.

Problem

Existing long-context VLMs fall into two failure modes: specialized compression modules add overhead and cap capacity, while native context-extension approaches (LongVA, LongVILA, LongViTA) underperform proprietary models and fail to improve consistently as visual input grows. The optimal training strategy and data recipe for state-of-the-art long-context VLMs remain unclear.

Method

Architecture: SigLIP-so400M visual encoder โ†’ MLP projector โ†’ Qwen2.5 LLM, with image tiling for any-resolution inputs. No compression modules.

Information-First Sampling (two components): - Image Area Preservation (IAP): selects tiling grid (rw ร— rh) that maximizes a joint objective penalizing area loss below 60% of original and aspect-ratio deviation โ€” formally argmax min(Anew/Aorig, 0.6) ยท min(rt/rorig, rorig/rt) over candidate grids. - Automatic Degradation Sampling (ADS): fixes text tokens first (complete text retention), then solves a constrained optimization for visual tokens via two-phase degradation โ€” (1) temporal degradation at 2 FPS with minimum frame floor; (2) tiling degradation selects highest tile count from {12,8,6,4,2,1} that fits remaining budget.

Progressive Mixed Post-Training: sequentially trains at Lmax = 32K โ†’ 64K โ†’ 128K, rather than jumping to the target context length. ADS adapts each sample to the current Lmax, and length-balanced packing ensures uniform coverage across short and long sequences.

Eagle-Video-110K Dataset: diversity-driven collection via CLIP similarity thresholding (ฯ„ = 0.5) against existing training data to select novel clips. Dual annotation: - Top-down story-level: human chapter boundaries โ†’ GPT-4o per-segment dense captions โ†’ GPT-4 long-form QA over full narrative. - Bottom-up clip-level: GPT-4o QA per short clip with temporal anchors and context anchors (no answer reveal) for clip-to-video generalization.

Key Contributions

  • Image Area Preservation (IAP): aspect-ratio-faithful tiling that retains โ‰ฅ60% original image area, improving high-resolution image and fine-grained video benchmarks.
  • Automatic Degradation Sampling (ADS): all-context-centric allocation that guarantees full text preservation while dynamically degrading visual resolution/frame count to fill remaining budget.
  • Progressive Mixed Post-Training: incrementally expanding context window that outperforms direct large-Lmax training by avoiding dilution of short-context supervision.
  • Eagle-Video-110K: 110K-sample dataset with chapter-level story annotations and clip-level QA, specifically filling the gap in long-video (>1 hour) training data absent from open-source collections.

Results

  • Video-MME (w/o subs, 512 frames): Eagle2.5-8B 72.4 vs. Qwen2.5-VL-8B 70.6, InternVL2.5-8B 72.1, Qwen2.5-VL-72B 73.3, GPT-4o 71.9.
  • MLVU: 77.6, surpassing InternVL2.5-78B (75.7) and Qwen2.5-VL-72B (74.6).
  • LongVideoBench: 66.4, surpassing InternVL2.5-78B (63.6) and Qwen2.5-VL-72B (61.9).
  • MVBench: 74.8 vs. InternVL2.5-8B 74.0, Qwen2.5-VL-8B 69.6.
  • CG-Bench (clue/long/open/mIoU): 55.8/46.6/45.6/13.4, exceeding Claude-3.5-Sonnet (56.5/40.3/35.6/4.17) and Gemini-1.5-Pro (50.9/37.8/28.7/3.85) on most metrics.
  • HourVideo: 44.5 dev / 41.8 test, surpassing Gemini-1.5-Pro (37.2/37.4).
  • Image benchmarks: DocVQA 94.1, ChartQA 87.5, OCRBench 869, avg 75.6 โ€” competitive with much larger models.
  • Ablations: removing IAP degrades InfoVQA and PerceptionTest significantly; removing ADS causes performance loss from text truncation; progressive 32Kโ†’64K outperforms direct 64K training; Eagle-Video-110K specifically improves โ‰ฅ128-frame inference.

Limitations

  • Training requires 128 H100 GPUs, limiting reproducibility for most academic groups.
  • No compression modules means inference memory scales linearly with input tokens at very long contexts; inference efficiency is unaddressed.
  • Consistent frame-count scaling is demonstrated up to 512 frames (Video-MME) but behaviour beyond 512 or at 128K token contexts is not benchmarked independently.
  • Dataset annotation relies heavily on GPT-4/GPT-4o, inheriting any biases or errors from those models.

Relevance to Vision-Language Models

Eagle 2.5 directly addresses one of the most pressing open problems in VLM research: closing the performance gap between small open-source models and large proprietary systems on long-context tasks. The ADS and IAP techniques are architecture-agnostic and could be applied to other VLM training pipelines. The progressive training schedule offers a practical recipe for safely extending context without sacrificing short-context capability โ€” a persistent failure mode of prior long-context VLMs. Eagle-Video-110K fills a concrete data gap (long videos with hierarchical narrative annotations) that the community can build on.