Skip to content

Self-Taught Agentic Long Context Understanding

🕒 Published (v1): 2025-02-21 20:29 UTC · Source: Arxiv · Venue: ACL 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

AgenticLU introduces a self-taught agentic framework for long-context QA that generates Chain-of-Clarifications (CoC) traces via inference-time tree search, then distills those traces back into the base model via SFT+DPO — eliminating the need for human annotations or a stronger teacher model. The trained model performs full CoC reasoning in a single inference pass. On 7 HELMET long-context benchmarks (8K–128K tokens), AgenticLU-8B outperforms prompting baselines and ProLong-8B.

Problem

LLMs' effective context window is significantly smaller than their nominal context window: Llama3.1-8B-Instruct (128K nominal) degrades sharply on HotPotQA as context grows beyond 8K. Existing solutions either require laborious human annotation, costly 40B-token long-context finetuning (ProLong), or multi-component RAG pipelines. No prior method distills self-generated agentic reasoning traces back into a single model efficiently.

Method

Chain-of-Clarifications (CoC) is a structured agentic loop: at each step the model (1) raises a self-clarification question, (2) executes a pointback — identifying relevant paragraph indices in the long document — and (3) answers the clarification using retrieved paragraphs, then optionally answers the original question.

CoC Path Construction (data generation): a tree search with branching factor \(b=8\) and max depth \(d=3\) is run over NarrativeQA training questions. Each node is a CoC step; candidate paths are scored by a combined objective: RougeL vs. ground truth + binary GPT-4o-mini correctness. This yields 107,550 traces (17M generation tokens, avg. context 67K tokens) with positive/negative path pairs for DPO.

CoC Path Distillation (training): two-stage finetuning on Llama3.1-8B-Instruct — (1) SFT on positive CoC traces to learn decomposition and pointback generation; (2) DPO on positive/negative path pairs to refine output quality. After training, the model generates paragraph index pointbacks directly in one forward pass without iterative querying.

Key Contributions

  • CoC agentic workflow: self-clarification + pointback as a unified, single-model inference loop (no multi-agent pipeline).
  • Self-taught data generation: CoC traces constructed entirely by the base model itself — no human labels, no stronger teacher.
  • Tree-search-to-finetune amortization: high inference-time compute (tree search) is paid once at data collection, then amortized into training weights.
  • 97.8% answer recall on NarrativeQA at depth 3, branching factor 8, during search.
  • Only ~1.93% runtime overhead over direct answering at inference time despite generating 15× more tokens per response (1,205 vs. 76 tokens).

Results

  • Long-context average (128K): AgenticLU-8B +14.7 points over Llama3.1-8B-Instruct (53.4% → 68.1% estimated from table 2 deltas).
  • HotPotQA (128K): 71.1% vs. 40.0% baseline (+31.1 pp); outperforms ProLong-8B.
  • Natural Questions (128K): 77.8% vs. 56.1% baseline (+21.7 pp).
  • TriviaQA (128K): 88.3% vs. 80.6% baseline (+7.7 pp).
  • NarrativeQA (128K): 56.0% vs. 38.0% baseline (+18.0 pp).
  • Short-context benchmarks (ARC, GSM8K, MMLU, MMLU-Pro, MathQA): −0.6 pp average — effectively no regression.
  • Multi-round CoC at inference: adding 2–3 rounds at test time yields further gains (avg 75.7% → 78.4% on RAG tasks).
  • Ablation: removing clarification drops HotPotQA from 71.1% → 57.8%; removing pointback drops it to 53.3%.

Limitations

  • Training data sourced solely from NarrativeQA; generalization to other domains relies on zero-shot transfer without domain-specific CoC traces.
  • Tree search at data collection is compute-intensive (branching factor 8, depth 3, chunk-level paragraph relevance queries); not reproducible on modest hardware.
  • Pointback mechanism requires paragraph-indexed input formatting — may not apply directly to arbitrary document formats.
  • DPO negative examples judged by GPT-4o-mini, introducing dependency on a proprietary model for data quality.
  • Evaluation capped at 128K; behavior beyond that is untested despite existence of 1M+ context models.
  • 92% of questions resolved in one clarification round — the remaining 8% still degrade at depth 3 (only 35% of residual 4% resolved), suggesting hard limits.

Relevance to Harnesses / Meta-Harnesses

AgenticLU exemplifies a self-bootstrapping meta-harness: a harness that uses a model's own inference-time agentic behavior (tree search over CoC steps) to generate the training data that then internalizes the harness into the model weights. This is a concrete instantiation of the "harness-to-weights" distillation pattern — the agentic workflow (clarify → retrieve → reason) is the harness at training time, and the finetuned model is the harness-free deployment artifact. For researchers building meta-harnesses, this paper provides a worked example of how to design preference data collection (positive/negative path pairs) from within an agentic loop, and how SFT+DPO can compress multi-step orchestration into single-pass inference — directly relevant to the cost-amortization problem in production harnesses.