Skip to content

PAACE: A Plan-Aware Automated Agent Context Engineering Framework

🕒 Published (v1): 2025-12-18 12:54 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

PAACE is a plan-aware context engineering framework for long-horizon LLM agents that learns to compress, prune, rewrite, and refine agent context conditioned on the next-k upcoming plan steps. It combines a large-scale synthetic workflow generator (PAACE-Syn, ~1.2M traces, ~9.5B tokens) with a distilled student compressor (PAACE-FT, Qwen3-4B) that recovers 97–98% of teacher performance at a fraction of the cost. On AppWorld, OfficeBench, and multi-hop QA, PAACE improves task accuracy while reducing peak context and cumulative attention dependency versus all baselines.

Problem

LLM agents accumulate rapidly expanding contexts across multi-step workflows, causing "context rot": attention dilution, instruction drift, redundant tool outputs, and quadratic compute growth. Prior compression methods (LLMLingua, Self-RAG, Provence, ACon) optimize for single-step or single-query relevance, ignoring multi-hop plan dependencies, next-k task structure, and the need for joint instruction–context co-refinement. No unified framework existed that jointly performs plan-aware pruning, summarization, rewriting, and instruction refinement over evolving agent state.

Method

PAACE models agent context as a plan-conditioned latent execution state C_t = {I₀, P, Π, H_{0:t}, O_{0:t}, R_{0:t}, M} and learns a compression policy via two components:

  1. PAACE-Syn: Generates ~1.2M synthetic long-horizon workflows (5–30 steps, multi-domain) by running agents without compression, then applying a teacher LLM (GPT-OSS-120B, 65K context window) conditioned on the next-k plan steps Π_{t:t+k} to produce compressed contexts. Successful compressions are selected by cosine similarity between full- and compressed-context final answers (threshold ≥0.85), valid compression ratio (0 < r < 1), and an LLM judge binary label. The teacher's compression prompt p is meta-learned via an LLM-driven evolutionary loop that optimizes a composite score over success rate, semantic equivalence, and compression ratio.

  2. PAACE-FT: Distills the teacher into a compact student (Qwen3-4B-Instruct) via standard causal language modeling on (Π_{t:t+k}, C_t) → C̃_t pairs from successful trajectories only. At inference, the student runs at each agent step without further teacher calls.

The compression objective is non-differentiable and trajectory-level: outcome fidelity (embedding similarity + LLM judge), per-step compression ratio, and implicit next-k plan coverage via conditioning.

Key Contributions

  • PAACE framework: unified plan-aware context engineering combining pruning, summarization, rewriting, re-injection, and instruction co-refinement conditioned on next-k future tasks.
  • PAACE-Syn: scalable synthetic dataset generator producing ~1.2M annotated multi-step workflows with paired full/compressed trajectories and explicit plan structures—no human annotation required.
  • PAACE-FT: distilled compact compressor (Qwen3-4B) achieving 97–98% of teacher performance; adds <8% per-step latency while reducing total input tokens by 35–60%.
  • Next-k relevance modeling: explicit conditioning on multiple future plan steps rather than single-step lookahead, with ablation showing k=2 optimal for tool-centric tasks and k=3 for multi-hop QA.
  • Compression-as-regularization insight: PAACE sometimes surpasses no-compression baselines, demonstrating that removing stale/contradictory context improves reasoning coherence beyond cost savings alone.

Results

AppWorld (multi-application, Easy/Med/Hard): - PAACE: 59.0% acc vs. best baseline ACON UTCO 56.5% (no-compression 56.0%) - Peak context: 6.23M tokens vs. ACON UTCO 7.33M - Cumulative dependency: 3.75M vs. ACON UTCO 4.69M

OfficeBench (document-centric tool chains): - PAACE: 78.1% acc vs. best baseline ACON UT 74.7% - Peak: 4.29M vs. no-compression 7.27M - Dependency: 1.64M vs. no-compression 4.43M

8-Objective QA (multi-hop retrieval): - PAACE: EM 0.402 / F1 0.512 vs. best baseline Prompting EM 0.376 / F1 0.478 - Peak: 4.41M vs. no-compression 10.35M - Dependency: 1.41M vs. ACON UTCO 1.50M

Distillation: PAACE-FT (Qwen3-4B) retains 97–98% of GPT-OSS-120B teacher performance, reducing inference cost by >10×.

Ablation: Removing embedding-similarity or LLM-judge filter each increases failure rates by ~8–12%.

Limitations

  • Compression policies are environment- and workflow-specific; cross-domain generalization is left as future work.
  • No formal semantic equivalence guarantees; relies on embedding similarity and LLM judging as proxies, which can miss safety-relevant omissions.
  • Supervision may inherit teacher LLM biases; outcome-level filtering mitigates but does not eliminate this.
  • Instruction refinement is implicit (emerges from context rewriting) rather than an explicitly optimized objective; standalone instruction optimization is deferred to future work.
  • No formal convergence or optimality guarantees for the evolutionary prompt meta-learning.
  • PAACE is not a safety mechanism; aggressive compression could silently drop safety constraints if deployed outside training conditions.

Relevance to Harnesses / Meta-Harnesses

PAACE is directly a meta-harness component: it wraps an arbitrary agent execution loop with a learned context-shaping policy that operates at the orchestration layer, not inside model weights. The PAACE-Syn pipeline—synthetic workflow generation, teacher rollout, outcome-level filtering, evolutionary prompt optimization, distillation—is precisely the pattern of a harness-over-harness: a meta-system that generates training data for a specialized sub-agent (the compressor) which then plugs into production agent loops. The evolutionary prompt meta-learning loop (population of compression prompts evaluated against real multi-step rollouts) is a concrete instantiation of meta-harness self-improvement. For researchers building multi-agent orchestration harnesses, PAACE offers a principled blueprint for solving context-budget management at the orchestration layer without modifying the backbone LLM, and its PAACE-Syn data generation approach generalizes to other harness subcomponents that require synthetic workflow supervision.