Skip to content

GRASP: GRanularity-Aware Search Policy for Agentic RAG

🕒 Published (v1): 2026-07-11 00:00 UTC · Source: HuggingFace · link

Why this paper was selected

Adobe Research; granularity-aware search policy directly improves agentic RAG decision-making in harnesses

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

GRASP is a reinforcement learning framework that trains a 3B LLM agent to adaptively coordinate three retrieval tools—semantic search, keyword search, and paragraph reading—at varying context granularities during multi-hop question answering. By jointly rewarding answer accuracy, grounded reading, complementary search, and turn efficiency via GRPO, the agent learns emergent skimming-then-scanning behavior without expert demonstrations. It outperforms both prompting-based and RL-based agentic RAG baselines on retrieval recall and QA metrics across three benchmarks.

Problem

Static RAG retrieves a fixed coarse-grained context once, causing hallucinations when irrelevant tokens dilute evidence. Existing agentic RAG either uses a single retriever or fixed chunk granularity, and cannot adapt which retrieval signal (lexical vs. semantic) to apply or how much context to expose at each reasoning step. Errors compound in multi-hop chains when wrong intermediate evidence propagates forward.

Method

GRASP formulates agentic RAG as a finite-horizon MDP \(\mathcal{M} = (\mathcal{X}, \mathcal{A}, P, R, T)\) with three retrieval actions: semantic search \(\tau_s\) (dense retrieval, returns top-\(K\) sentences), keyword search \(\tau_k\) (BM25, returns top-\(K\) sentences), and read paragraph \(\tau_r\) (expands a retrieved sentence to its parent paragraph). The policy (Qwen2.5-3B-Instruct) generates a reasoning segment, selects an action, and updates its interaction history iteratively.

The trajectory-level reward combines four components:

\[R = R_A + \alpha R_R + \beta R_S + \gamma R_E\]
  • \(R_A = \text{F1}(\hat{y}, y)\): token-level answer accuracy
  • \(R_R = \text{F1}(D_r, D_g)\): whether paragraph reads land on gold documents
  • \(R_S = \mathbf{1}[D_s \cap D_g \neq \emptyset \;\wedge\; D_k \cap D_g \neq \emptyset]\): both tools retrieve at least one gold document
  • \(R_E = \frac{T - T_{\text{cur}}}{T} \cdot \mathbf{1}[R_A > 0.5]\): turn efficiency (only credited when answer is already adequate)

Weights are \(\alpha=0.7, \beta=0.15, \gamma=0.15\), capping auxiliary rewards at 1.0 to keep \(R_A\) dominant. The policy is trained with GRPO by sampling \(G\) rollouts per query and computing group-relative advantages. Retrieved document tokens are masked from the policy loss.

Key Contributions

  • Formulates adaptive multi-tool retrieval selection for agentic RAG as an RL problem, enabling dynamic choice among lexical, semantic, and context-expansion actions per reasoning step.
  • Multi-component reward design that explicitly incentivizes complementary tool use, grounded reading of gold evidence, and trajectory efficiency without reward hacking.
  • Empirical demonstration that a 3B open-weight model trained with GRASP surpasses both prompting-based baselines using proprietary models (IRCoT + GPT-5-mini) and prior RL-based baselines (Search-R1) on retrieval recall and QA.
  • Qualitative analysis showing the learned policy exhibits human-like information foraging: \(\tau_s\) for broad exploration → \(\tau_r\) for local verification → \(\tau_k\) for entity-specific refinement.

Results

All results are on 500-sample held-out sets from HotpotQA (in-domain), 2WikiMultiHopQA, and MuSiQue (zero-shot transfer). Policy model: Qwen2.5-3B-Instruct. Baselines use the same or comparable model sizes unless noted.

Retrieval Recall: - GRASP: 0.90 / 0.90 / 0.70 (HotpotQA / 2Wiki / MuSiQue) — best on all three - IRCoT (GPT-5-mini, lexical): 0.91 / 0.83 / 0.62 — second on HotpotQA and 2Wiki - Search-R1 (GRPO, semantic, paragraph): 0.74 / 0.77 / 0.62 - Single-step hybrid (best non-agentic): 0.86 / 0.60 / 0.59

QA (EM / F1 / LLM-judge): - GRASP (HotpotQA): 0.53 / 0.66 / 0.71 vs. Search-R1-GRPO: 0.45 / 0.56 / 0.58; IRCoT: 0.24 / 0.36 / 0.63 - GRASP (2Wiki): 0.52 / 0.60 / 0.63 vs. Search-R1-GRPO: 0.45 / 0.53 / 0.54; IRCoT: 0.27 / 0.40 / 0.58 - GRASP (MuSiQue): 0.23 / 0.33 / 0.34 vs. Search-R1-GRPO: 0.22 / 0.30 / 0.29; IRCoT: 0.08 / 0.15 / 0.25 - Total reward increases from ~0.26 at training start to ~1.22 at convergence

Limitations

  • Trained and indexed on HotpotQA; cross-dataset transfer (2Wiki, MuSiQue) uses a validation-split retrieval corpus, which may not reflect realistic open-domain corpora.
  • Sentence-level indexing and paragraph expansion increase corpus preprocessing complexity; scalability to web-scale corpora is untested.
  • The complementary search reward \(R_S\) requires both \(\tau_s\) and \(\tau_k\) to retrieve gold evidence on every trajectory, which may over-constrain tool use on queries where one signal is sufficient.
  • Only evaluated on two-hop QA; applicability to longer reasoning chains or non-QA tasks is undemonstrated.
  • Base policy is 3B parameters; whether the granularity-aware behavior generalizes to larger models is not studied.

Relevance to Agentic AI / LLM Agents

GRASP directly advances the core agentic RAG problem of deciding when, how, and at what granularity to retrieve, which is a key bottleneck in tool-augmented LLM agents operating over external knowledge. The RL-from-scratch training (GRPO with no expert demonstrations) and the multi-component reward that distinguishes evidence quality, tool diversity, and efficiency offer a reusable design pattern for reward shaping in multi-tool agents. The emergent skimming-scanning behavior mirrors findings on tool-use specialization in multi-step agents, connecting to broader work on Search-R1, IRCoT, and ReAct-style systems. Context window management via fine-grained retrieval is increasingly critical as agents tackle long-horizon tasks, and GRASP's sentence-then-paragraph strategy provides a concrete, trainable mechanism.