Skip to content

DocPrune:Efficient Document Question Answering via Background, Question, and Comprehension-aware Token Pruning

🕒 Published (v1): 2026-04-24 06:51 UTC · Source: Arxiv · Venue: CVPR 2026 · link

Why this paper was selected

Background/question-aware token pruning for efficient document VQA; CVPR 2026

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

DocPrune is a training-free, progressive token pruning framework for efficient long-document visual question answering. It reduces visual token count through three sequential stages—background removal, question-relevance filtering, and comprehension-adaptive decoder pruning—achieving 3.0×/3.3× throughput gains in encoder/decoder while simultaneously improving F1 by +1.0 on M3DocRAG.

Problem

Document images differ fundamentally from natural images: on average 36% of patches are semantically empty background, and even among content tokens only ~10% are question-relevant. Existing token-reduction methods for natural images and videos exploit spatial redundancy but ignore document structural sparsity, causing text/layout fragmentation when applied naively. Moreover, all prior pruning methods fix pruning layers via static heuristics, ignoring that the model's cross-modal comprehension matures progressively across layers—pruning too early discards tokens before reliable attention signals emerge.

Method

DocPrune is applied on top of a two-stage retrieval-augmented pipeline (retrieval model \(f_\text{RET}\) yields top-\(K\) pages; QA model \(f_\text{QA}\) generates the answer).

Background Token Pruning (BTP). Each image \(I \in \mathbb{R}^{H \times W \times 3}\) is tiled into \(P \times P\) patches. The dominant background intensity \(m\) (pixel mode) is computed globally. A patch \(t_i\) is labeled background if its background ratio

\[R_i = \frac{1}{P^2} \sum_{p=1}^{P^2} \mathbf{1}\!\left[|\hat{t}_i^{(p)} - m| < \tau_e\right]\]

exceeds threshold \(\tau_\text{bg}\). Such patches are discarded before encoding in both retrieval and QA stages.

Question-aware Token Pruning (QTP). Pre-computed ColPali retrieval embeddings (\(E^\text{doc}\), \(E^\text{qst}\)) are used to score each remaining content token via aggregated cosine similarity:

\[s_i = \sum_{j=1}^{N_\text{qst}} \cos(e_i^\text{doc}, e_j^\text{qst}).\]

A bilinear interpolation bridges resolution gaps between retrieval and QA encoders. Tokens below threshold \(\tau_\text{qst}\) are pruned before the QA vision encoder.

Comprehension-aware Token Pruning (CTP). During LLM decoding, the comprehension proxy \(c_l = \|x^l_T\|\) (L2 norm of the last token's hidden state at layer \(l\)) is monitored. Once \(c_l\) exceeds a threshold \(\omega_\text{comp}\)—indicating the model has sufficiently understood the document—top-attention tokens are retained and the rest are pruned. This triggers adaptively per sample: easy samples reach \(\omega_\text{comp}\) in earlier layers than hard samples.

Key Contributions

  • Empirical analysis establishing three document-specific properties: background dominance (36% of patches), extreme question-relevance sparsity (top 10% tokens sustain near-full accuracy), and layer-dependent pruning reliability (attention signals become informative only past ~layer 15).
  • DocPrune: a training-free, three-stage progressive pruning framework that exploits all three properties in sequence.
  • A novel comprehension surrogate metric \(c_l = \|x^l_T\|\) for adaptive pruning-layer selection, replacing fixed-heuristic layer choices.
  • Demonstration that efficiency and accuracy can be simultaneously improved (not traded off) in document VQA without any fine-tuning.

Results

  • On M3DocRAG (multi-modal, multi-page document RAG benchmark):
  • Encoder throughput: 3.0Ă— improvement
  • Decoder throughput: 3.3Ă— improvement
  • FLOPs reduction: 57.3%
  • F1 score: +1.0 vs. base M3DocRAG (30.8 vs. 29.8)
  • Compared to token-reduction baselines applied to the same M3DocRAG backbone (VTW, FastV, DivPrune): DocPrune achieves the highest F1 and lowest TFLOPs among all methods (Figure 1).
  • Retaining only top-10% most-attended tokens (layer 20) incurs only a 1.2-point F1 drop while reducing computation to \(\frac{1}{9}\) FLOPs and achieving 6.46Ă— throughput gain (ablation in Table 1).

Limitations

  • Evaluated on a single benchmark (M3DocRAG); generalization to other document QA benchmarks (DocVQA, MP-DocVQA, etc.) is not shown in the excerpted text.
  • BTP relies on a simple pixel-mode heuristic for background detection, which may fail for colored, scanned, or heavily stylized documents.
  • QTP requires that retrieval and QA stages share compatible embedding spaces; a bilinear interpolation bridge is used for mismatched resolutions but may introduce errors.
  • Threshold hyperparameters (\(\tau_e\), \(\tau_\text{bg}\), \(\tau_\text{qst}\), \(\omega_\text{comp}\)) require tuning and may not transfer across document domains or model architectures.
  • CTP analysis is conducted on Qwen2-VL-based M3DocRAG; layer-comprehension dynamics may differ for other LLM backbones.

Relevance to Vision-Language Models

DocPrune directly addresses the computational bottleneck of deploying VLMs on long, structured documents—a gap that becomes critical as MLLMs are extended beyond natural-image understanding to office documents, PDFs, and multi-page reports. The comprehension-aware pruning mechanism (\(c_l = \|x^l_T\|\)) is a lightweight interpretability probe that reveals how cross-modal alignment evolves across transformer layers, complementing recent work on token merging (ToMe) and attention-based pruning for vision encoders. For VLM researchers, the key takeaway is that document-specific inductive biases (layout sparsity, structural regularity) enable stronger efficiency gains than generic vision token reduction methods, suggesting that domain-aware compression is a productive direction alongside general-purpose MLLM efficiency research.