Skip to content

ProMSA:Progressive Multimodal Search Agents for Knowledge-Based Visual Question Answering

🕒 Published (v1): 2026-06-26 11:23 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

ProMSA frames KB-VQA as a budgeted progressive search-and-reasoning problem where a single MLLM agent iteratively chooses between image search, text search, or stop, with explicit deduplication to avoid re-retrieving visited pages. The agent is trained via a two-stage pipeline: rejection-sampling SFT for tool-use format learning, then TN-GSPO, a sequence-level RL objective that normalizes policy updates by both generation length and tool-interaction depth. ProMSA achieves SOTA on E-VQA and InfoSeek, outperforming both fixed RAG pipelines and prior search-agent baselines.

Problem

KB-VQA requires combining image understanding with external knowledge (e.g. Wikipedia) under long-tail entity distributions, where entities are rare and models cannot reliably decide whether evidence is trustworthy. Existing methods use a fixed retrieve-then-generate pipeline with a preselected retriever and static top-k, which suffers from three core limitations: (1) tools and retrieval policies are not adaptive — the model cannot choose modality based on current uncertainty; (2) there is no mechanism to recover from failures — when retrieved evidence is wrong, the pipeline forces an answer anyway; (3) static top-k injection cannot support multi-hop evidence chains that require progressive fact-filling.

Method

  • Formulation: KB-VQA is cast as a budgeted sequential decision process. At each step \(t\), the policy \(\pi_\theta\) outputs \((z_t, u_t, \text{args}_t)\) where \(u_t \in \{\texttt{img\_search}, \texttt{text\_search}, \texttt{stop}\}\). Budgets \(B_{\text{img}}, B_{\text{text}}\) cap the maximum tool calls per modality.
  • Deduplication: Maintains a visited-page set \(\mathcal{P}_t\) passed as an exclusion list on every tool call. After retrieval, \(\mathcal{P}_{t+1} = \mathcal{P}_t \cup \{p_i\}\) so the same page is never re-examined.
  • Summarization: Retrieved Wikipedia pages are compressed into question-conditioned short snippets via a separate summarizer (Qwen3-VL-8B), preventing context-length explosion.
  • Training — Stage 1 (SFT cold-start): Samples \(N\) trajectories per question from the initial policy, filters by format validity, executability, and answer correctness via rejection sampling. Retains at most one valid trajectory per question for MLE training.
  • Training — Stage 2 (TN-GSPO RL): Uses a sparse sequence-level reward \(R(\tau) = r_{\text{ans}} + r_{\text{fmt}} - \lambda C(\tau)\) where \(C(\tau)\) is normalized tool-call count. TN-GSPO defines the trajectory likelihood ratio with normalization \(D(\tau) = L(\tau)(1 + c\,H(\tau)^\alpha)\), where \(L(\tau)\) is generated-token count and \(H(\tau)\) is tool-interaction depth. This prevents generation-length bias and aligns update scale with the decision structure of budgeted search. Uses asymmetric clipping (low=0.2, high=0.28) following DAPO.

Key Contributions

  • Proposes ProMSA, a progressive multimodal search agent for KB-VQA that couples retrieval and reasoning through multi-round interaction with tool-choice, deduplication, and budget control.
  • Formulates KB-VQA as a budgeted progressive search-and-reasoning problem where the model learns when to retrieve, which modality to use, and when to stop.
  • Introduces TN-GSPO, a tool-horizon normalized sequence-level policy optimization method that stabilizes search-policy learning by normalizing updates by both generation length and interaction depth.
  • Achieves SOTA on E-VQA and InfoSeek across multiple backbone scales (Qwen2.5-VL-7B, Qwen3-VL-2B/8B).

Results

  • E-VQA (All): ProMSA with Qwen3-VL-8B achieves 52.6%, surpassing REAL (45.5%), CC-VQA (41.4%), MaS-VQA (42.2%), and search-agent baselines MMSearch-R1 (40.6%) and DeepEyesV2 (40.1%).
  • InfoSeek (All): 53.4% vs. next-best REAL (44.1%) and CC-VQA (45.1%).
  • Training progression: Average across E-VQA + InfoSeek improves from 35.1% (base MLLM) → 40.7% (SFT) → 53.0% (RL).
  • TN-GSPO vs. alternatives: TN-GSPO (52.6% E-VQA) outperforms GRPO (44.2%), GRPO with asymmetric clipping (49.7%), and GSPO (49.3%).
  • Ablation — tool modalities: Both tools required — image-only drops to 34.7% (E-VQA), text-only to 27.6%; both together 52.6%.
  • Budget sensitivity: Increasing text+image budgets from (2,2) → (3,3) → (4,4) yields E-VQA scores 48.2 → 52.6 → 52.4, showing diminishing returns beyond (3,3).
  • Inference time: 1.8s per sample (1.5× Echosight) on E-VQA, but with substantially higher accuracy (49.7% vs. 18.8%).

Limitations

  • The agent operates under a fixed small budget (max 7 interaction steps, default 3 per modality) — performance plateaus beyond (3,3), suggesting the budget ceiling is already near-optimal.
  • Inference latency is 1.5× faster methods, though the gap is justified by accuracy gains.
  • Requires a separate summarization model (Qwen3-VL-8B) for content compression, adding infrastructure complexity and a possible bottleneck.
  • The rejection-sampling SFT stage discards incorrect trajectories entirely — the model never learns from negative examples at the SFT level.
  • Budget parameters \(B_{\text{img}}, B_{\text{text}}\) remain manually set hyperparameters rather than learned.

Relevance to Agentic AI / LLM Agents

ProMSA is a direct demonstration of an LLM agent that autonomously selects retrieval tools, manages a limited interaction budget, and decides when to stop searching — a canonical instance of tool-augmented, self-directed reasoning. Its two-stage training (SFT for format + RL for search policy) mirrors the emerging recipe for training LLM agents (e.g., Search-R1, SWE-RL), and its TN-GSPO objective addresses a general challenge in agent RL: how to properly normalize credit assignment across heterogeneous trajectory lengths and interaction depths. For researchers tracking agentic AI, this paper provides a clean case study in tool-augmented reasoning, RL-based search-policy learning, and closed-loop retrieval.