Skip to content

Prune4Web: DOM Tree Pruning Programming for Web Agent

🕒 Published (v1): 2025-11-26 13:49 UTC · Source: Arxiv · Venue: AAAI 2026 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

Prune4Web addresses the DOM information-overload bottleneck in LLM-based web agents by replacing LLM-based element selection with LLM-generated Python scoring programs that prune the DOM before the agent ever sees it. A three-stage Planner → Programmatic Element Filter → Action Grounder pipeline, trained with SFT+GRPO on ~5,000 annotated trajectories, raises low-level grounding accuracy from 46.8% to 88.28% on the Multimodal-Mind2Web benchmark.

Problem

Real-world webpage DOMs contain 10,000–100,000 tokens—far exceeding LLM context limits. Existing mitigations either truncate the DOM (losing critical elements), apply rigid rule-based heuristics (poor generalization), or have a separate LLM rank elements directly (still requires feeding the full DOM, causing attention dilution). No prior approach efficiently reduces candidate elements while preserving the ground-truth target.

Method

DOM Tree Pruning Programming (DTPP): Rather than feeding raw HTML to the LLM, a Programmatic Element Filter model receives only the current low-level sub-task text and generates a compact Python scoring function parameterized by task-derived keywords and per-attribute weights. The function is executed outside the LLM against a pre-filtered accessibility tree (interactive tags only), scoring every element via tiered attribute matching (exact > phrase > word > fuzzy) across attribute priority tiers (visible text > aria-label/placeholder > class/id). The Top-N (default N=20) candidates are passed to the downstream Action Grounder, reducing the candidate pool by 25–50×.

Three-stage pipeline: 1. Planner (screenshot + history → low-level sub-task; no HTML access) 2. Programmatic Element Filter (sub-task → Python scoring program → pruned DOM) 3. Action Grounder (sub-task + pruned DOM → executable action)

Training: A two-turn dialogue unified model is fine-tuned from Qwen2.5VL-3B-Instruct. Turn 1 jointly generates the sub-task and scoring parameters (Planner + Filter); Turn 2 receives the executed pruning result and outputs the action (Grounder). SFT on ~5,000 GPT-4o-annotated MM2W trajectories is followed by GRPO-based RFT applied only to the Planner turn, using a hierarchical binary reward: format compliance + whether the ground-truth element survives filtering (intermediate signal) + final grounding success.

Key Contributions

  • DTPP paradigm: LLM generates a scoring program rather than directly ranking elements; traversal and scoring are offloaded to executable Python, avoiding long-context attention dilution.
  • Heuristic Scoring Function Template: Constrains the LLM to generate only keywords and base weights, improving stability while retaining semantic flexibility.
  • Two-turn dialogue unified training strategy: Jointly optimizes all three stages in a single model with SFT + targeted GRPO-RFT; filtering success serves as an intermediate reward signal for the Planner.
  • Automated data annotation pipeline: Uses GPT-4o to backfill intermediate labels (sub-tasks, keyword weights, pruned DOM reasoning) onto the MM2W dataset, producing ~5,000 high-quality step annotations.

Results

  • Low-level grounding accuracy (custom 1,101-step benchmark, given GT sub-tasks):
  • No pruning (Qwen2.5VL-3B FT): 46.80%
  • GPT-4o end-to-end LLM pruning+decision: 70.84% (Recall@20: 85.56%)
  • Prune4Web (Qwen2.5-0.5B FT): 88.28% (Recall@20: 97.64%)
  • Prune4Web (Qwen2.5VL-3B FT): 88.28% (Recall@20: 97.46%)

  • Oracle (GT element guaranteed in top-20, Qwen2.5VL-3B FT): 90.28%

  • Multimodal-Mind2Web offline benchmark (Cross-Task):

  • Prune4Web-3B Separate Models: Ele. Acc 46.0%, Op. F1 83.4%, Step SR 42.2%
  • Prune4Web-3B Two-turn Unified: Ele. Acc 58.4%, Op. F1 84.1%, Step SR 52.4%

  • Best prior open-source baseline (MindAct Flan-T5XL): Ele. Acc 55.1%, Step SR 52.0%

  • Recall@N (fine-tuned 0.5B and 3B vs. GPT-4o): Both fine-tuned models exceed 90% Recall@3 and ~95% Recall@5; GPT-4o reaches only ~72% Recall@3 and ~86% Recall@20.

  • Ablation — programmatic vs. LLM Top-N filtering (online tasks, LLM-Verified Completion Rate): GPT-4o-mini improves from 26.3% → 31.6%; Qwen2.5VL-3B improves from 0.0% → 5.2% with Prune4Web filtering.

  • RFT contribution (Cross-Task Step SR): Separate Models: 37.9% (SFT) → 42.2% (+RFT); Two-turn Unified: 46.5% → 52.4%.

Limitations

  • Grounding accuracy with DTPP still falls ~2 points below oracle pruning (88.28% vs. 90.28%), indicating residual recall loss at N=20.
  • DTPP's scoring template assumes target elements are identifiable by textual/attribute cues; purely visual elements (e.g., unlabeled icon buttons) may evade keyword-based scoring.
  • Online evaluation uses only 30 curated tasks; real-world generalization at scale is not demonstrated.
  • The RFT stage is applied only to the Planner; Filter and Grounder rely solely on SFT, potentially leaving optimization gains on the table.
  • The approach is offline-trained and evaluated primarily on MM2W, which may not fully represent the distribution of live web environments.

Relevance to Agentic AI / LLM Agents

Prune4Web directly attacks a foundational bottleneck for any DOM-grounded web agent: the mismatch between LLM context capacity and real-world HTML size. The shift from LLM-as-ranker to LLM-as-program-generator is a concrete instance of programmatic thinking applied to input preprocessing rather than action sequencing—a distinction with broad applicability to any agent operating over large structured environments (APIs, file trees, databases). The hierarchical reward mechanism that uses intermediate pipeline success (filter recall) as a training signal for an upstream planner is a transferable pattern for credit assignment in multi-stage agentic systems. The result that a fine-tuned 0.5B model matches a 3B model on the filtering subtask suggests DTPP effectively decomposes a hard generalization problem into a learnable program-parameter prediction task.