Verification-Guided Context Optimization for Tool Calling via Hierarchical LLMs-as-Editors¶

🕒 Published (v1): 2025-12-15 19:48 UTC · Source: Arxiv · Venue: AAAI 2026 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

VGCO is a framework that uses LLMs-as-editors to automatically refine tool documentation and knowledge-base context for tool-calling LLMs. It operates in two stages—evaluation (collecting failure cases) and optimization (hierarchical editing at retrieval, tool-description, and parameter-schema levels)—guided by verification signals rather than unconstrained prompt mutations. On xLAM and BFCL benchmarks it consistently outperforms prior context-optimization baselines including DRAFT.

Problem¶

Tool-calling LLM performance degrades when API documentation is written for human readers, not model consumption: schemas are ambiguous, descriptions overlap, and parameter types are underspecified. Existing prompt/context optimizers (DSPy, SAMMO, DRAFT) are stateless, lack a structured action space, and omit domain-specific evaluation signals, making them ineffective at industrial scale with hundreds of overlapping tools.

Method¶

VGCO decomposes tool-calling into three hierarchical subtasks—KB retrieval, tool selection, and parameter filling—and assigns a dedicated LLM editor to each level (Mr, Mt, Mp). The Evaluation stage runs inference on a validation set and collects failure cases per level. The Optimization stage has each editor refine its corresponding context layer (retrieval summaries Dr, tool descriptions Dt, parameter schemas Dp) using a guided instruction prompt encoding four elements: (1) a state space tracking prior edits and outcomes, (2) a structured action space (add/modify/delete at semantic level, not token level), (3) a per-query reward signal grounded in ground-truth tool calls, and (4) in-context learning examples of successful and failed edits. Updates are accepted only if post-edit evaluation improves over the previous checkpoint. Refinement proceeds top-down (Dr → Dt → Dp) for T iterations, with the editors themselves being either large models (via prompt engineering) or small fine-tuned models.

Key Contributions¶

VGCO framework: verification-guided, feedback-driven context optimization using LLMs-as-editors for tool documentation and KB context
Hierarchical LLM editors: three-tier optimization (retrieval → tool description → parameter schema) that respects inter-level dependencies
Guided instruction design: state-aware, action-constrained, domain-aligned editor prompts with ablated ICL examples and requirement constraints
Empirical validation: consistent gains over DRAFT and ReAct on xLAM (100-tool KB) and BFCL across Claude Sonnet 3.5/3.7 and Llama-3.3-70B inference models

Results¶

xLAM (top-100 tools, Claude Sonnet 3.5 inference): - VGCO (Claude-4.5 editor): Final Acc. 61.0%, Retrieval Recall 98.3%, Tool Selection 84.0%, Param Filling 74.0% - DRAFT (Claude-4.5): Final Acc. 56.3%; ReAct: 44.4%; Raw: 32.5% - VGCO (Claude-4 editor): 50.7% vs DRAFT 46.2%, ReAct 38.5% - VGCO (Llama-3.3 editor): 48.7% vs DRAFT 43.1%

BFCL (Claude Sonnet 4 editor, Claude Sonnet 3.7 inference): - VGCO: Tool Selection 96.9%, Parameter Filling 94.7% - DRAFT: 92.1% / 88.3%; ReAct: 85.0% / 74.4%; Raw: 69.1% / 55.8%

Iterative convergence (xLAM top-30 tools, Claude Sonnet 3.5 + Claude Sonnet 4.5 editor): - Training accuracy rises from 0.599 to 0.947 over 3 iterations; testing from ~0.6 to 0.951

Ablation (BFCL, Full Guided Instruction vs. ablated): - Removing ICL examples: Tool Selection drops to 74.89% (from 96.94%), Param Filling to 65.43% (from 94.73%)—the single largest ablation gap

Limitations¶

Experiments capped at 100 tools; generalization to larger/more heterogeneous inventories not yet validated
Robustness under domain shift, unseen tools, or low-resource settings not tested
Evaluated only on single-turn tool-calling; multi-turn and temporal reasoning scenarios are future work
Context window constraints of editor models limit batch-level reward signal application
No online learning loop; all editing is offline before production reintegration

Relevance to Agentic AI / LLM Agents¶

Tool use is a foundational capability for LLM agents, and this work directly addresses a practical bottleneck: documentation quality rather than model reasoning is often the binding constraint in real deployments. VGCO's hierarchical editor architecture—with structured action spaces and verification-grounded feedback—is a concrete instantiation of the broader "LLM-as-optimizer" paradigm applied to the tool-interface layer, complementing work on multi-turn reasoning and ReAct-style agents. For practitioners building production agent systems with large API inventories, VGCO offers a tuning-free path to systematic context improvement without model retraining. The state-aware, action-constrained editor design also connects to ongoing research on constrained prompt optimization and context engineering as a systems-level discipline.