Skip to content

Adaptively Coordinating with Novel Partners via Learned Latent Strategies

🕒 Published (v1): 2025-11-16 19:45 UTC · Source: Arxiv · Venue: NEURIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

TALENTS (Team Adaptation via LatEnt No-regreT Strategies) is a zero-shot coordination method that learns a VAE-based latent strategy space from population agent trajectories, clusters it into discrete partner types, trains a strategy-conditioned cooperator via PPO, and uses a fixed-share regret minimization algorithm to infer and track partner strategy shifts online. It achieves state-of-the-art performance in the Overcooked domain against both agent and human partners without any human data at training time.

Problem

Existing zero-shot coordination methods either (a) rely on population-based training without explicit online strategy inference, producing rigid cooperators that cannot adapt mid-episode, or (b) use agent-modeling methods that depend on hand-crafted strategy sets or struggle with distribution shift between train-time and test-time trajectories. No method unifies continuous latent strategy generation with principled intra-episodic adaptation to non-stationary partners.

Method

TALENTS proceeds in three phases:

  1. Latent strategy learning: A VAE is trained on offline population trajectories to encode agent trajectories into a low-dimensional Gaussian latent space, with a sequential decoder predicting the next H actions given a latent strategy vector z and current observation. The ELBO objective (Eq. 5) with KL regularization encourages a smooth, structured latent space.

  2. Strategy-conditioned cooperator training: K-Means + silhouette analysis partitions the latent space into K clusters. For each training episode, a cluster is sampled (with priority-based weighting), a partner is generated by decoding from the cluster mean, and a cooperator is trained with independent PPO. The cooperator's action logits are additively biased by a learned cluster embedding vector, encoding strategy-specific action preferences.

  3. Online adaptation via fixed-share: At test time, each cluster acts as an "expert." Per step, the decoder predicts each expert's expected partner action; loss is the negative log-likelihood of the observed partner action under each expert. Weights are updated by exponential weighting then a fixed-share redistribution step (weight α is spread uniformly across experts), enabling the agent to track strategy switches mid-episode with bounded tracking regret O(√(T(m ln N + m ln T/m))).

Key Contributions

  • TALENTS framework unifying generative population-based training with online regret-minimization-based strategy inference
  • VAE latent strategy space with K-Means/silhouette clustering, enabling controlled partner generation per strategy type
  • Fixed-share online adaptation replacing direct trajectory encoding, avoiding train/test distribution shift in strategy inference
  • Empirical validation in a time-pressured, multi-recipe Overcooked variant including a 119-participant crowdsourced human-agent study

Results

  • Agent-agent (held-out BP agents): TALENTS outperforms GAMMA and BR across Open, Hallway, and Ring layouts for all three training populations (FCP, MEP, BP); e.g., with BP training: Open 842 vs. GAMMA 574 vs. BR 470; Ring 648 vs. GAMMA 388 vs. BR 640 (competitive)
  • Exception: Forced Coordination layout — BR outperforms both TALENTS and GAMMA (e.g., BP: BR 79 vs. TALENTS 57 vs. GAMMA 48), attributed to sparse reward when paired with low-skill partners skewing priority sampling
  • Fixed-share ablation: Static-regret minimizer (Hedge without weight sharing) fails to update belief after a mid-episode partner swap at t=1200, yielding lower reward in the second half; TALENTS with fixed-share recovers
  • Human-agent study (119 participants, 3 layouts): TALENTS significantly outperforms GAMMA and BR on team score (F(2,166)=5.76, p=.003), team fluency (F(2,122)=4.31, p=.02), and perceived trust (F(2,122)=3.23, p=.04) vs. BR

Limitations

  • Fails on Forced Coordination due to sparse rewards with low-skill partners, which also corrupts priority sampling toward poor partners
  • Generalization is bounded by the VAE's interpolation capacity — novel partners must lie within the behavioral support of the training population
  • Human gameplay rate-limited to 3–4 actions/second while the agent acts at 10 steps/second, requiring an artificial slowdown that may impair long-horizon agent planning
  • Training is computationally heavy: VAE requires 4–6 hours; cooperator requires ~24 hours on a 5×RTX 6000 Ada server

Relevance to Agentic AI / LLM Agents

TALENTS addresses a core challenge for agentic systems deployed alongside humans or other agents: adapting in real time to partners whose behavior is unknown, diverse, and non-stationary — without retraining. The fixed-share online tracking mechanism is directly applicable to any agent that must condition its behavior on an inferred context (partner policy, user intent, task mode) that can shift mid-episode. The framework's separation of offline strategy learning from online inference via regret minimization offers a principled alternative to prompt-conditioned or fine-tuned LLM agents for cooperative tasks. It also demonstrates that human-compatible coordination is achievable without human demonstration data, relevant to scalable agent deployment.