Skip to content

Cut the Crap: An Economical Communication Pipeline for LLM-based Multi-Agent Systems

🕒 Published (v1): 2025-01-01 · Source: ICLR · Venue: ICLR 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

AgentPrune is a plug-and-play communication pruning framework for LLM-based multi-agent systems that identifies and eliminates redundant inter-agent messages via one-shot sparsification of a spatial-temporal message-passing graph. It achieves comparable task performance to state-of-the-art topologies at $5.6 vs. $43.7 cost, while also improving adversarial robustness through low-rank graph masking.

Problem

Existing LLM multi-agent pipelines incur 2–11.8× more token consumption than a single-agent chain because their communication topologies (complete graphs, LLM-Debate, etc.) pass a large volume of redundant or noise-carrying messages. No prior work formally defined or systematically pruned this communication redundancy, leaving cost and robustness unaddressed at deployment scale.

Method

AgentPrune models any LLM-MA system as a spatial-temporal graph \(G = (V, E^S \cup E^T)\) where spatial edges \(E^S\) connect agents within a dialogue round (a DAG) and temporal edges \(E^T\) carry dialogue history across rounds. The framework introduces differentiable graph masks \(S^S, S^T \in \mathbb{R}^{|V| \times |V|}\) applied element-wise to the adjacency matrices: \(A(\tilde{G}) = \{A^S \odot S^S,\, A^T \odot S^T\}\).

The masks are optimized over \(K' \ll K\) rounds via two objectives: (1) distribution approximation — maximize system utility \(\varphi\) using a policy-gradient estimator (Equation 9) since \(\varphi\) is non-differentiable; and (2) low-rank sparsity — minimize the nuclear norm \(\|S^X\|_*\) subject to \(\|A^X - S^X\|_F \leq \delta\), promoting sparse and adversarially robust connectivity. After \(K'\) rounds, one-shot magnitude pruning retains the top-\((1-p\%)\) mask entries as a fixed binary mask \(B\), producing the sparse subgraph \(G^{sub}\) used for the remaining \((K - K')\) rounds. A multi-query variant trains the mask on \(Q' \ll Q\) queries then applies it across the full query set.

Key Contributions

  • First formal definition of Communication Redundancy in LLM-MA systems (Definition 1, Equation 5–6), showing that subgraphs exist that match or exceed full-graph utility.
  • AgentPrune: a plug-and-play spatial-temporal graph pruning pipeline combining policy-gradient-based mask optimization with nuclear-norm low-rank regularization and one-shot pruning.
  • Demonstrated seamless integration into AutoGen and GPTSwarm without architecture changes.
  • Low-rank masking is shown to confer adversarial robustness against agent-targeted attacks as a byproduct of sparsification.
  • Closed-form token-savings analysis (Equation 13) quantifying \(\Delta\) as a function of pruning ratio \(p\%\), rounds \(K, K'\), sample count \(M\), and per-message costs \(c^S, c^T, c_q\).

Results

  • MMLU: AgentPrune-C achieves 84.72% vs. state-of-the-art DyLAN 80.16% / PHP 83.45%; cost $5.6 vs. $43.7 for comparable topologies.
  • GSM8K: AgentPrune-C 95.62% vs. PHP 92.45% (next best among temporal methods); AgentPrune-R 95.83%.
  • HumanEval: AgentPrune-R 90.30% vs. DyLAN 89.70%, GPTSwarm 88.49%.
  • Average across 6 benchmarks: AgentPrune-C 89.72%, AgentPrune-R 89.44%, outperforming all baselines (best prior: PHP/LLM-Debate ~87.0).
  • Token reduction: 28.1%–72.8% fewer tokens when integrated into AutoGen and GPTSwarm.
  • Adversarial robustness: 3.5%–10.8% performance improvement under two agent-targeted attack types.
  • Random pruning of 10%–30% of edges already yields up to +2.83% on MMLU, empirically confirming the redundancy claim.

Limitations

  • Requires at least three agents and a non-trivial topology (chain or direct-output structures are excluded as "too simple").
  • One-shot pruning derives a static \(G^{sub}\); the optimal topology may shift across query types or domains not seen during the \(K'\)-round calibration.
  • Policy-gradient optimization is high-variance and requires multiple sampled rollouts (\(M\) samples), adding calibration cost before savings materialize.
  • Experiments use GPT-3.5/GPT-4 only; transferability to open-weight or smaller LLMs is not evaluated.
  • The multi-query training assumes a representative calibration subset \(Q'\); distribution shift between calibration and test queries is not analyzed.

Relevance to Harnesses / Meta-Harnesses

AgentPrune is directly relevant as a meta-layer that wraps around arbitrary multi-agent harnesses (AutoGen, GPTSwarm, LLM-Debate, etc.) and restructures their communication topology without modifying individual agents — exactly the role a meta-harness plays. The spatial-temporal graph formalism provides a unified abstraction for any orchestration topology, which is a foundational primitive for harness designers who need to reason about message flow across agents and rounds. The plug-and-play one-shot pruning pattern — calibrate briefly, fix the wiring, then run at scale — is a deployable efficiency strategy applicable to any harness that controls inter-agent routing. The adversarial-robustness side-effect of low-rank masking also suggests that meta-harnesses could benefit from communication sparsification as a defensive mechanism beyond pure cost reduction.