Skip to content

LLawCo: Learning Laws of Cooperation for Modeling Embodied Multi-Agent Behavior

🕒 Published (v1): 2026-06-26 15:26 UTC · Source: Arxiv · Venue: ICML 2026 · link

Why this paper was selected

ICML 2026; Chuang Gan (MIT-IBM); learning cooperation laws for decentralized embodied multi-agent systems

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

LLawCo is a framework for embodied multi-agent cooperation that autonomously extracts high-level behavioral laws (e.g., "Talk when necessary", "Wait for partner") from failed interaction traces, then uses supervised fine-tuning to embed those laws into LLM-based agents' chains of thought. It achieves average success rate improvements of 4.5% on PARTNR-Dialog and 6.8% on TDW-MAT over the strongest open-source communicative baselines across four backbone LLMs. The authors also introduce PARTNR-Dialog, a large-scale communicative embodied cooperation benchmark extending PARTNR.

Problem

LLM-based communicative embodied agents suffer from two compounding failures: (1) they lack effective mechanisms to align agent behavior with heterogeneous partners and task constraints, producing suboptimal coordination; and (2) they cannot self-improve—existing methods rely either on brittle hand-crafted pipelines capped by the base LLM's capability, or on distillation from stronger models, neither of which allows autonomous improvement through interaction. Existing benchmarks (C-WAH, TDW-MAT) also provide minimal training data (0–24 episodes), bottlenecking learning-based research.

Method

LLawCo wraps a ReAct-style LLM planner extended with a message event buffer, then runs three offline stages before deployment:

  1. Failure-driven law extraction: Given failed episodes \(E_{fail} = \{E_j : \rho(E_j) < \tau\}\), the backbone LLM analyzes each failure trace to produce a reason \(r_j = f_\theta(\tau_j, T_{failure})\). All reasons are aggregated and the LLM generates a compact law set \(\mathcal{P} = \{(n_i, d_i)\}_{i=1}^{K}\) (\(K=5\) in experiments) of named, natural-language behavioral principles.

  2. Law-aligned trace selection: Successful episodes are filtered by prompting the LLM to score whether agent trajectories conform to \(\mathcal{P}\), retaining only episodes that are both successful (\(\rho(E_j) \geq \tau\)) and law-aligned (\(\epsilon_{align}^{(j)} = 1\)).

  3. Law-guided SFT: For each retained episode, every action \(a_k^t\) is paired with a law-referencing thought \(\text{Thought}_k^t = f_\theta(o_k^{\leq t}, a_k^t, \mathcal{P}, T_{reflect})\) that grounds the action in the single most relevant law. This produces dataset \(\mathcal{D}_{law} = \{(o_k^{\leq t}, \text{Thought}_k^t, a_k^t)\}\), used for standard next-token cross-entropy SFT to produce \(f_{\theta'}\).

At inference, \(f_{\theta'}\) autoregressively generates \((Thought_i^t, a_i^t) \sim f_{\theta'}(h_t, \mathcal{P}, T_{inference})\), with laws provided in-prompt and serving as a human-editable control interface.

Key Contributions

  • Law-guided learning framework that extracts interpretable behavioral laws from failure traces and embeds them via SFT—no stronger-model supervision or ground-truth annotations required.
  • PARTNR-Dialog benchmark: large-scale communicative cooperation extension of PARTNR with a Talk action primitive, per-agent event buffers, and distributed vLLM+Habitat parallel evaluation (4 GPU-hours on A100 for LLaMA-3.1-8B validation).
  • Consistent gains across four LLM backbones (LLaMA-3.1-8B, LLaMA-3.1-70B, Gemma-3-12B, Qwen-3-14B); a 14B model matches 70B in certain settings.
  • Human-controllable law specification at inference time: manually editing \(\mathcal{P}\) causes agents to reliably follow updated constraints.

Results

  • PARTNR-Dialog: +4.5% average success rate over the strongest communicative baseline (CommPARTNR) across four backbone LLMs.
  • TDW-MAT: +6.8% average success rate over the strongest baseline (CoELA) across the same backbones.
  • A 14B model (Gemma-3-12B or Qwen-3-14B, implied) achieves performance comparable to LLaMA-3.1-70B in certain settings on these benchmarks.
  • Inference-time law editing demonstrably steers agent behavior, validating controllability.
  • (Full per-model table was truncated in the provided text; precise per-backbone numbers are not fully available.)

Limitations

  • Requires a training split; cannot be applied to benchmarks without one (C-WAH excluded for this reason).
  • Law extraction quality depends on the backbone LLM's ability to introspect failures accurately; weak base models may produce noisy or generic laws.
  • \(K=5\) laws is a hand-set hyperparameter; ablation exists but optimal \(K\) may be task- or environment-dependent.
  • Evaluated only on two-agent scenarios; scalability to larger agent teams is unvalidated.
  • Laws are shared/global across both agents; partner-specific or role-specific law sets are not explored.
  • Self-improvement requires collecting enough failure episodes to extract meaningful patterns—cold-start data quality is a practical dependency.

Relevance to Agentic AI / LLM Agents

LLawCo directly addresses a core open problem in multi-agent LLM systems: how to make agents self-improve through interaction without stronger-model distillation or dense human supervision. The law extraction loop (fail → abstract → filter success → SFT) is a lightweight, reproducible recipe that could generalize beyond embodied settings to any cooperative LLM agent pipeline. The interpretable, human-editable law interface is practically significant for controllable deployment—a standing challenge in agentic AI. The PARTNR-Dialog benchmark also fills a real gap, providing training-scale data for the communicative embodied cooperation setting that prior benchmarks lacked.