Skip to content

Society of Mind Meets Real-Time Strategy: A Hierarchical Multi-Agent Framework for Strategic Reasoning

🕒 Published (v1): 2025-08-08 05:57 UTC · Source: Arxiv · Venue: COLM 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

HIMA (Hierarchical Imitation Multi-Agent) is a hierarchical framework for StarCraft II strategic play in which \(k=3\) imitation-learning specialists per race generate long-horizon structured action sequences, coordinated by a meta-controller (Strategic Planner, SP) that fuses their proposals via Nominal Group Technique and a temporal Chain-of-Thought (t-CoT). HIMA outperforms all prior LLM-based SC2 SoTAs in win rate while requiring substantially fewer LLM calls.

Problem

Existing LLM-based RTS agents query an LLM at every short timestep, producing invalid build orders (due to unmet prerequisites), redundant commands, and poor long-horizon coherence. A single monolithic model also cannot cover the full strategic space of SC2 (air dominance, ground rush, economic expansion, etc.).

Method

Imitation agents. State-action pairs \(\{S_t, A_t\}\) are extracted from professional SC2 replays (SC2EGSet). GPT-4o-mini annotates each multi-step action window \(A_{t:t+\Delta}\) (\(\Delta=3\) min) with a Tactical Rationale (TR). k-means clustering (\(k=3\) per race) on unit compositions partitions the data into specialization groups; Qwen-2 1.5B is fine-tuned via SFT on each cluster, yielding specialized agents (e.g., Air Superiority Specialist, Ground Superiority Specialist, Infrastructure Specialist).

Strategic Planner (meta-controller). At each decision point, the SP runs a four-stage environment-aware action orchestration: 1. Current assessment — reads resources, units, visible enemies. 2. Advisor strategy resolution — applies the Nominal Group Technique (NGT) to reconcile conflicting Strategic Objectives (SOs) and TRs across agents. 3. Strategy formulation — synthesizes a consolidated plan respecting battlefield constraints. 4. Temporal Chain-of-Thought (t-CoT) — decomposes the plan into immediate, short-term, and long-term action horizons before committing.

A feedback system triggers plan re-evaluation on critical events: failed commands are logged and injected into subsequent prompts; if enemy units exceed threshold \(\tau=10\), the SP discards the current plan and re-queries all imitation agents from scratch.

Benchmark. TEXTSCII-ALL extends TextStarCraft II from 1 matchup to all 9 player Ă— opponent race combinations (Protoss, Terran, Zerg), with race-specific action spaces newly defined for Terran and Zerg.

Key Contributions

  • Hierarchical multi-agent architecture (HIMA) combining imitation-learned specialist agents with an LLM meta-controller.
  • Temporal Chain-of-Thought (t-CoT) decomposing decisions across immediate/short-term/long-term horizons.
  • Structured action sequences over a fixed time window \(\Delta\) with appended Tactical Rationales, reducing invalid commands.
  • TEXTSCII-ALL: first SC2 text-based evaluation testbed covering all 9 race matchups.
  • Empirical demonstration of superior win rate and computational efficiency vs. TextStarCraft II, SwarmBrain, EpicStar, HEP.

Results

  • vs. built-in AI (Protoss vs. Zerg): HIMA 82% at Lv.7, 68% at Lv.8 — vs. HEP 25% at Lv.7 (best prior baseline).
  • vs. built-in AI (Protoss vs. Terran): 68% at Lv.7, 48% at Lv.8.
  • Head-to-head vs. all SoTAs: 100% win rate over 10 games against SwarmBrain, TextStarCraft, and HEP.
  • LLM call efficiency: HIMA requires far fewer total LLM calls per 20-minute game than TextStarCraft II baseline (exact counts shown in Fig. 1b; executable call fraction substantially higher for HIMA).
  • Ablation (Table 4, partially shown): Multi-Agent IL+SP (1.5BĂ—3) outperforms Single-Agent IL+SP (7B), demonstrating that specialization beats model scale.
  • Terran performance noticeably lower (e.g., Terran vs. Protoss: 58%/28%/16%/10% at Lv.4–7), attributed to Terran's higher micro-management demands.

Limitations

  • Terran race performance lags substantially behind Protoss and Zerg; authors attribute this to Terran's micro-management complexity, which their macro-focused design does not fully address.
  • The enemy-surge threshold \(\tau=10\) is heuristically determined, not learned.
  • The time window \(\Delta=3\) min is fixed; the SP cannot dynamically adjust planning frequency except via the binary feedback trigger.
  • Evaluation uses win rate only; other metrics (resource efficiency, build-order quality) are relegated to the appendix with noted low correlation to win rate.
  • Head-to-head evaluations use only 10 games per matchup, limiting statistical power.
  • Zerg mid-to-high difficulty performance drops sharply (e.g., 0% at Lv.7 in Zerg vs. Protoss and Zerg vs. Zerg), suggesting race-specific brittleness.

Relevance to Harnesses / Meta-Harnesses

HIMA is a direct instantiation of the meta-harness pattern: a central orchestrator (Strategic Planner) ingests structured proposals from \(N\) independently fine-tuned sub-agents and synthesizes a final plan, exactly the dispatcher–specialist–aggregator topology central to harness design. The feedback system — logging failures, injecting failure context into re-prompts, and triggering full re-solicitation on threshold events — is a concrete runtime control-flow mechanism that harness designers must implement for fault recovery in agentic pipelines. The t-CoT decomposition (immediate / short-term / long-term) maps to multi-horizon planning interfaces that meta-harnesses need when coordinating agents with different latency characteristics. The use of Nominal Group Technique for inter-agent conflict resolution offers a principled, structured alternative to simple voting or majority-consensus coordination schemes common in existing harnesses.