LLawCo: Learning Laws of Cooperation for Modeling Embodied Multi-Agent Behavior

🕒 Published (v1): 2026-06-26 15:26 UTC · Source: Arxiv · Venue: ICML 2026 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

LLawCo is a law-guided learning framework for embodied multi-agent systems that automatically extracts high-level behavioral laws (e.g., "Talk when necessary", "Wait for partner") from failed interaction traces, then embeds those laws into LLM agents via supervised fine-tuning. Without relying on stronger teacher models or human annotations, agents self-align to both task objectives and partner behaviors. Evaluated across four LLM backbones, the method achieves +4.5% success rate on PARTNR-Dialog and +6.8% on TDW-MAT over the strongest communicative baseline.

Problem

LLM-based embodied agents in cooperative settings suffer from two gaps: (1) no mechanism to align agent behavior with partner-specific preferences and task constraints, leading to coordination failures; (2) no self-improvement pathway — existing frameworks either rely on hand-engineered prompting pipelines bounded by the base LLM's ceiling, or distill from stronger models, precluding autonomous evolution. Existing benchmarks (C-WAH, TDW-MAT) also lack sufficient training data for learning-based approaches.

Method

LLawCo operates as a modular six-stage pipeline:

1. Failure-Driven Law Extraction: Given failed episodes \(E_{fail} = \{E_j : \rho(E_j) < \tau\}\), an LLM analyzes each trace to produce failure reasons \(r_j = f_\theta(\tau_j, T_{failure})\). All reasons are aggregated and compressed into \(K\) abstract laws \(\mathcal{P} = f_\theta(R, T_{law})\), where each law is a (name, description) tuple. \(K=5\) in experiments.

2. Law-Aligned Trace Selection: From successful episodes, only those verified as law-consistent are retained: \(E_{retain} = \{E_j \mid \epsilon^{(j)}_{succ}=1 \wedge \epsilon^{(j)}_{align}=1\}\), where alignment is scored by prompting the LLM against \(\mathcal{P}\).

3. Law-Guided Data Generation: For each retained action \(a_k^t\), the LLM generates a reasoning trace \(\text{Thought}_k^t = f_\theta(o_k^{\leq t}, a_k^t, \mathcal{P}, T_{reflect})\) that explicitly cites the single most relevant law — producing dataset \(\mathcal{D}_{law} = \{(o_k^{\leq t}, \text{Thought}_k^t, a_k^t)\}\).

4. Supervised Fine-Tuning: Standard next-token prediction SFT on \(\mathcal{D}_{law}\) produces \(f_{\theta'}\), trained to generate law-referencing thoughts before actions.

5. Law-Guided Inference: At each step, the fine-tuned model generates \((\text{Thought}_i^t, a_i^t) \sim f_{\theta'}(h_t, \mathcal{P}, T_{inference})\), where \(h_t\) encodes observation, action, and message history. Laws can be manually overridden at inference time for human control.

The agents run on a ReAct backbone extended with per-agent event buffers that route inter-agent Talk messages. Inference is accelerated via a decoupled vLLM server + parallel Habitat simulators.

Key Contributions

• LLawCo framework: self-improving embodied agents that extract behavioral laws from failure traces and internalize them via SFT, requiring no teacher models or ground-truth annotations.
• Interpretable/controllable laws: laws are human-readable and editable; modifying \(\mathcal{P}\) at inference time reliably changes agent behavior.
• PARTNR-Dialog benchmark: large-scale communicative cooperation benchmark (2,000 train / 1,000 val episodes) extending PARTNR with a Talk action primitive and distributed evaluation infrastructure.
• Cross-backbone generalization: consistent gains across LLaMA-3.1-8B, LLaMA-3.1-70B, Gemma-3-12B, and Qwen-3-14B; a 14B model reaches performance comparable to the 70B model in certain settings.

Results

• PARTNR-Dialog: +4.5% average success rate over the strongest communicative baseline (CommPARTNR) across four LLM backbones.
• TDW-MAT: +6.8% average success rate over the strongest baseline (CoELA) across four LLM backbones.
• Inference-time law modification leads to reliable behavioral adaptation, confirming controllability.
• 14B model (Gemma-3-12B or Qwen-3-14B) matches 70B performance in certain settings after LLawCo training.
• PARTNR-Dialog evaluation of LLaMA-3.1-8B requires only 4 GPU-hours on A100s (episode-level parallelism + vLLM).

(Full numeric tables for Comp./Steps/Replan/Talk are truncated in the provided text.)

Limitations

• Requires a training split; evaluated only on PARTNR-Dialog and TDW-MAT — cannot be applied to C-WAH or other benchmarks lacking training data.
• Law extraction quality depends on the base LLM's ability to summarize failures coherently; weaker base models may produce less useful laws.
• \(K=5\) laws is a hyperparameter requiring tuning (ablation exists but not shown in the provided excerpt).
• Single-law grounding per thought may not handle situations requiring simultaneous compliance with multiple laws.
• Evaluated only in two-agent settings; scalability to larger agent collectives is not demonstrated.
• Communication infrastructure assumes a broadcast model; more complex topologies are not addressed.

Relevance to Harnesses / Meta-Harnesses

LLawCo is a self-bootstrapping meta-harness: it uses an LLM to orchestrate the generation of its own training curriculum (failure analysis → law synthesis → trace filtering → SFT data construction), closing the loop without external supervision. This is structurally analogous to meta-harness patterns where a harness wrapper manages prompt templates, data routing, and fine-tuning pipelines as first-class artifacts. The explicit law set \(\mathcal{P}\) acts as a harness-level control plane that is both machine-generated and human-editable — a concrete realization of programmable behavioral constraints in agentic pipelines. The six-stage modular pipeline and the decoupled inference/simulation architecture also illustrate how production-grade multi-agent harnesses decompose concerns (law extraction, trace curation, SFT, inference serving) to enable scalable, reproducible agent evaluation.