HPD-Parsing: Hierarchical Parallel Document Parsing

🕒 Published (v1): 2026-07-21 08:25 UTC · Source: Arxiv · link

Why this paper was selected

Hierarchical parallel document parsing with VLMs; speed+accuracy gain for doc-understanding pipelines

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

HPD-Parsing introduces a Hierarchical Parallel Decoding (HPD) paradigm for VLM-based document parsing that replaces full-page autoregressive generation with a two-level structure: a main layout branch coordinates global structure, then dynamically forks concurrent content branches per document block. Combined with Progressive Multi-Token Prediction (P-MTP), it achieves 4,752 tokens/second—3.06× the throughput of its autoregressive baseline—while preserving competitive parsing accuracy.

Problem

Unified VLM-based document parsers generate entire page outputs through a single autoregressive trajectory, creating a sequential bottleneck that scales with document length. Profiling shows decoder latency can exceed encoder latency by ~496× for long outputs, yet existing acceleration strategies (visual token compression, sliding window attention, MTP) do not restructure the fundamental single-trajectory decoding paradigm for full-page generation.

Method

HPD-Parsing is built on InternVL3.5-1B (0.3B InternViT encoder + 0.8B Qwen3-0.6B-derived LLM decoder with GQA, SwiGLU, RMSNorm).

Layout-Coordinated Parallel Decoding: A main layout branch autoregressively generates a structured sequence in reading order; each layout unit produces a <FORK> token that triggers the inference scheduler to spawn an isolated content branch. Content branches receive the shared visual context and structural prefix via KV cache reuse, then append a <CHILD> token and decode the localized region independently and concurrently. Each branch attends only to the shared prefix plus its own incremental KV cache—not the full preceding output—bounding attention cost per branch.

Progressive Multi-Token Prediction (P-MTP): A lightweight residual MLP at each branch position predicts \(k\) future tokens. The joint training loss is: $\(L = \sum_{k=0}^{K} \frac{\sum_{t=0}^{N} \mathcal{M}_{t+k+1} \mathcal{W}_{t,k} \ell(\hat{p}_t^k, y_{t+k+1})}{\sum_{t=0}^{N} \mathcal{M}_{t+k+1}}\)$ where \(k=0\) is standard next-token prediction, \(k \geq 1\) are look-ahead depths, and \(\mathcal{W}_{t,k}\) applies a decay combining distance, path-consistency, and target-consistency signals. Average accepted length is 6.6 tokens per step.

Staged Training (3 stages): 1. Full-page autoregressive pre-training to initialize general parsing capability (including P-MTP). 2. Branch-specific supervision adapting the model to HPD format with hard-case focus; content branches use validity mask \(\mathcal{M}_t^{\text{dec}} = \mathbb{1}(t > t_{\langle\text{CHILD}\rangle})\). 3. Lightweight RL with task-aware rewards (formula quality, table parsing, layout prediction, count consistency).

Automated Difficulty-Aware Data Curation: Four-stage pipeline: feature-based clustering/sampling → multi-model annotation (PaddleOCR-VL-1.5, MinerU-2.5 Pro) + difficulty estimation → VLM-based refinement → distribution balancing across Easy/Medium/Hard samples and document types (formula, table, text, layout).

Key Contributions

• Hierarchical Parallel Decoding (HPD): restructures full-page autoregressive generation into globally coordinated, localized parallel decoding via dynamic branch forking with shared-prefix KV cache reuse.
• P-MTP integration: progressive multi-token prediction applied within each branch, achieving 6.6 average accepted tokens per step, reducing intra-branch decoding steps.
• Staged adaptation strategy: three-stage training that transfers autoregressive parsing capacity to the HPD paradigm without large accuracy loss.
• Automated difficulty-aware data curation: scalable pipeline combining multi-model pseudo-labeling, checkpoint-based difficulty estimation, VLM refinement, and balanced sampling with minimal manual annotation.
• State-of-the-art throughput on OmniDocBench v1.6: 4,752 TPS at batch size 512.

Results

• HPD-Parsing achieves 4,752 TPS (batch size 512) on OmniDocBench v1.6.
• 3.06× throughput over the vanilla autoregressive baseline (InternVL3.5-1B).
• 2.62× throughput over the fastest existing document parsing model (implied: 1.62× stated elsewhere in the paper—the text states "1.62× the throughput of the fastest existing document parsing model" in contributions vs. "2.62×" in the abstract; the abstract figure is 2.62×).
• Maintains competitive parsing accuracy on OmniDocBench v1.6 (exact accuracy scores shown in Figure 1(b) range ~88–98% overall, but individual breakdown numbers are not fully legible in the provided text).
• P-MTP alone contributes an average accepted length of 6.6 tokens per step.
• For long-output samples, decoder latency in the baseline is ~496× encoder latency, confirming decoding as the dominant bottleneck.

Limitations

• The paper does not report wall-clock latency (time-to-first-token or end-to-end latency per page) alongside TPS, so real-world latency improvement is not fully characterized.
• Accuracy numbers are described as "competitive" but not shown to match or exceed all baselines—the paradigm shift to parallel decoding may incur accuracy tradeoffs not fully disclosed in the truncated text.
• The HPD paradigm assumes document content is locally grounded; highly inter-dependent content (e.g., cross-region references, footnotes, mathematical proofs spanning blocks) may violate this locality assumption.
• The 1.62× vs. 2.62× throughput figures cited inconsistently between the contributions section and the abstract suggest some baselines may differ, but the comparison is not fully disambiguated in the provided text.
• Training complexity is substantially increased (3-stage pipeline + RL + large-scale curation) compared to standard supervised fine-tuning baselines.

Relevance to Vision-Language Models

HPD-Parsing directly addresses a core efficiency bottleneck in VLM-based document understanding: the quadratic growth of autoregressive decoding cost with output length. By demonstrating that layout can be separated from content generation within a single unified VLM and exploited for parallelism, it opens a structural alternative to speculative decoding or model compression for the document-parsing use case. For VLM researchers, the shared-prefix KV cache reuse mechanism and the branch-forking inference scheduler are generalizable primitives applicable to any task where outputs decompose into independent subtrees (e.g., structured extraction, multi-question answering). The P-MTP integration also provides a concrete demonstration of non-draft-model speculative decoding embedded directly into a VLM's decoding loop.