Stop Wasting Your Tokens: Towards Efficient Runtime Multi-Agent Systems

🕒 Published (v1): 2026-01-01 · Source: ICLR · Venue: ICLR 2026 · link

Why this paper was selected

Token-efficient multi-agent systems via misinformation detection and pruning

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

SUPERVISOR AGENT is a lightweight meta-agent framework that attaches to existing Multi-Agent Systems (MAS) to provide real-time, proactive supervision without modifying base agent architecture. It uses an LLM-free adaptive filter to trigger interventions only at high-risk interaction points, achieving a 29.68% average token reduction on GAIA while maintaining task success rates.

Problem

Modern MAS suffer from two compounding failure modes: (1) error propagation, where a single hallucinated output poisons downstream agents through agent-agent, agent-tool, or agent-memory interaction channels; and (2) economic inefficiency from repetitive action loops and excessively long observations flooding context windows. Existing robustness work is post-hoc (failure attribution after the fact) and efficiency work targets static design-time optimization, leaving dynamic runtime inefficiencies unaddressed.

Method

The framework introduces a Supervised MAS (SMAS) defined as a standard MAS augmented with a Supervisor meta-agent that monitors three interaction types (agent-agent, agent-tool, agent-memory) without altering base agent logic.

When to supervise — LLM-free adaptive filter: A heuristic, rule-based filter (no LLM call) detects three trigger conditions: (i) explicit error occurrence (e.g., tool/code exception), (ii) inefficient behavior (e.g., repeated page_down actions), (iii) excessive observation length (e.g., raw HTML). This avoids the cost of monitoring every interaction.

What context — memory-augmented context window: Upon triggering, the Supervisor receives \(W = (N, Q_g, Q_l, T_l, S)\) (agent name, global task, local task, local trace, step summary). For system-wide inefficiency diagnosis, it uses the extended window \(W_\text{ext} = W \cup \{T_g\}\) including the global interaction trace.

How to supervise — tiered action space: The Supervisor selects from four actions keyed to three intervention contexts: - \(c_\text{error}\): correct_observation | provide_guidance | run_verification (invokes a verification sub-agent) - \(c_\text{inefficient}\): approve | provide_guidance - \(c_\text{excessive}\): correct_observation (rewrites/purifies the observation in-place)

Key Contributions

• SUPERVISOR AGENT framework: non-intrusive meta-agent that plugs into any MAS at runtime with no base-agent modification.
• LLM-free adaptive filter: heuristic trigger that avoids LLM overhead on every interaction, making supervision net-efficient.
• Formal SMAS definition: formalizes the three high-risk interaction loci and a structured context window for supervisor decision-making.
• Pareto improvement on GAIA: 29.68% token reduction at pass@1 with no accuracy loss; variance in token consumption reduced by up to 63% on Level 2 tasks.
• Broad generalization: validated on 6 benchmarks (GAIA, AIME, GSM8k-Hard, HumanEval, MBPP, DROP) and 3 foundation model families (GPT-4.1, Gemini-2.5-pro, Qwen3).

Results

• GAIA (pass@1, Smolagent base): 29.68% average token reduction; accuracy unchanged at 50.91%.
• Level 2: −34.64% tokens; Level 3: −29.23% tokens with +7.69% accuracy.
• GAIA (pass@3): −26.45% tokens; +1.21% accuracy (63.03% vs. 61.82%).
• Token variance: GAIA Level 2 variance reduced by ~63% (violin plot comparison).
• AIME 2024: solve rate +6.67% (36.67% vs. 30.00%); tokens −18.92% (47.95K vs. 59.14K).
• HumanEval: accuracy +0.61% (92.68% vs. 92.07%); tokens −23.74% (31.19K vs. 40.91K).
• GSM8k-Hard: accuracy +1.17% (75.50% vs. 74.33%); tokens −8.92%.
• DROP (F1): slight drop −1.28% (79.80% vs. 81.08%); tokens −5.60%.
• MBPP: slight accuracy drop −1.25%; tokens −6.62%.
• Ablation (GAIA): Purification drives most efficiency (disabling it reduces savings from 29.68% to 15.96%); removing Correction or Guidance causes the largest accuracy drops (−3.03% and −2.43% respectively).
• MAS-agnostic: SUPERVISOR AGENT improves both AWorld and OAgents on token-intensive GAIA subsets.

Limitations

• Adaptive observation purification can overcompress long contexts, causing minor accuracy/F1 drops on some benchmarks (DROP, MBPP).
• The LLM-free filter relies on fixed heuristics; novel or domain-specific failure modes not covered by the three predefined triggers will be missed.
• Evaluation of MAS-agnostic generalization uses only the top-10 most token-intensive GAIA tasks per level, not the full benchmark, for AWorld and OAgents.
• The run_verification action introduces a nested sub-agent, adding latency and cost in complex error cases; cost-benefit of this action is not fully analyzed.
• All GAIA experiments use GPT-4.1 as the base model; generalization to open-weight or smaller base models is not directly validated on GAIA.

Relevance to Agentic AI / LLM Agents

SUPERVISOR AGENT directly addresses one of the most pressing practical barriers to MAS deployment — runaway token costs and error propagation — by introducing a runtime process-control layer orthogonal to both static topology design and context compression. Its LLM-free filter design is a practically important insight: the bottleneck for online supervision is not the intervention logic but the trigger overhead, and heuristic detection enables net-positive efficiency. The formal SMAS framework and three-way interaction taxonomy (agent-agent, agent-tool, agent-memory) provide a clean conceptual vocabulary for reasoning about where MAS reliability breaks down. For researchers building agentic pipelines, this work argues that meta-level supervisors — rather than stronger base models or elaborate topologies — can be a cost-effective path to robustness.