CoLLMLight: Cooperative Large Language Model Agents for Network-Wide Traffic Signal Control

🕒 Published (v1): 2026-01-01 · Source: ICLR · Venue: ICLR 2026 · link

Why this paper was selected

ICLR 2026; CoLLMLight: cooperative LLM agents for network-wide traffic signal control

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

CoLLMLight is the first cooperative LLM agent framework for network-wide traffic signal control (TSC), addressing the failure of prior LLM-based TSC systems that treat each intersection independently. It decouples slow multi-step spatiotemporal reasoning from fast real-time signal decisions via an asynchronous architecture, and uses a two-stage cost-aware optimization (supervised fine-tuning on concise reasoning traces + PPO with length-penalizing rewards) to balance cooperation quality against inference cost.

Problem

Existing LLM-based TSC agents (e.g., LLMLight) assign one agent per intersection with no inter-intersection communication. This local-only decision-making ignores upstream spillback: an agent may clear its longest queue while blocking an incoming platoon from a neighbor, degrading network-wide throughput. No prior work integrates cooperative reasoning among LLM agents for TSC at the network level.

Method

Asynchronous Cooperative Decision Architecture. At each timestep \(t\), each agent \(i\) collects: current traffic observation \(\mathcal{O}_t^i\) (per-lane queue count, moving count, waiting time, occupancy), spatial neighbor graph \(G^i\), and historical interaction window \(\mathcal{T}_t^i\). Two modules run asynchronously:

1. Spatiotemporal-Aware Cooperative Reasoning (SR): The LLM processes a prompt encoding \((\mathcal{O}_t^i, G^i, \mathcal{T}_t^i)\) through structured reasoning steps—critical lane identification, spatial interaction analysis, temporal pattern analysis, reflection on past signals—and outputs cached cooperative guidance \(Y_t^{SR,i}\).
2. Real-Time Decision (RD): The LLM selects signal \(a_t^i = f_{LLM}^{RD}(\text{Prompt}(\mathcal{O}_t^i, Y^{SR,i}))\) using only the current observation and the latest cached SR output, keeping RD latency below the 3–5 s yellow-light threshold.

Cost-Aware Cooperation Optimization (two stages):

1. Adaptive Reasoning Chain Optimization (AR): GPT-4o generates multiple SR traces of varying lengths in simulation; the shortest SR trace whose paired RD achieves the lowest long-term average queue length is selected. A supervised fine-tuning (SFT) dataset \(\mathcal{D}_{SFT}\) is built from these pairs and the LLM is fine-tuned via negative log-likelihood minimization (Eq. 7).
2. Policy Refinement via RL (PR): PPO is applied with two reward signals. The RD reward is \(R^{RD} \in \{+1,-1\}\) based on whether the chosen signal matches the oracle-optimal signal \(a^*\) (selected by exhaustive simulation). The SR reward is: $\(R^{SR} = R^{RD} \cdot \left[\beta\!\left(1 - \frac{L}{L_{max}}\right) + (1-\beta)U\right]\)$ where \(L\) is reasoning length, \(U \in \{0,1\}\) is the RD module's self-assessed utility of the SR output, and \(\beta\) trades off conciseness vs. quality.

Key Contributions

• First cooperative LLM agent framework for network-wide TSC enabling inter-intersection coordination.
• Asynchronous SR–RD architecture that decouples slow cooperative reasoning from real-time signal selection, keeping decision latency under the yellow-light window.
• Cost-aware cooperation optimization combining SFT on shortest-effective reasoning traces with PPO using length-penalizing, utility-weighted rewards.
• Zero-shot SOTA on four real-world traffic networks (Jinan, Hangzhou, New York 1, New York 2) against 11 RL-based and LLM-based baselines.

Results

• CoLLMLight-8B vs. best RL baseline (Advanced-CoLight): ATT reduced from 1037.9 s → 1000.4 s (NY1), 1428.9 s → 1345.1 s (NY2), 332.9 s → 267.9 s (Jinan), 355.0 s → 308.5 s (Hangzhou); all in zero-shot setting.
• vs. LLMLight-8B (SOTA LLM baseline): ATT reduced from 1187.4 → 1000.4 (NY1), 1599.4 → 1345.1 (NY2), 332.9 → 267.9 (Jinan), 355.0 → 308.5 (Hangzhou).
• Scaling vs. optimization: CoLLMLight-8B outperforms Llama3.1-70B and Qwen3-32B on all datasets, confirming optimization quality over parameter count.
• Ablation — SR removal: Removing SR raises ATT by ~15% on NY1 (1155.1 s) and ~10% on NY2 (1477.3 s); effect is smaller on sparser Jinan/Hangzhou networks.
• Ablation — AR removal: Token usage increases ~52% on NY1 (738.5 vs. 484.2) and ~55% on Jinan (1072.6 vs. 691.3); ATT also rises.
• Latency: CoLLMLight-8B RD latency stays below 3 s across batch sizes \(b \in \{1,5,10\}\), lowest among 8B-scale LLMs tested; SR runs asynchronously without blocking decisions.

Limitations

• Trained and evaluated exclusively in the CityFlow simulator with a synthetic training dataset (Syn-Train); real-world deployment is unvalidated.
• SR benefit is smaller on topologically sparse networks (Jinan, Hangzhou), where local observations suffice, limiting cooperation gains.
• The oracle reward \(a^*\) for PPO requires exhaustive per-step simulation rollouts over all candidate phases, which is computationally expensive and unavailable in true online settings.
• 32B/70B models excluded from full latency evaluation due to cost, leaving the efficiency-performance frontier of larger models unexplored.
• No analysis of failure modes when neighbor observations are noisy or delayed.

Relevance to Agentic AI / LLM Agents

CoLLMLight directly advances multi-agent LLM system design by solving a concrete coordination problem: how to share reasoning across agents without incurring synchronization latency. The asynchronous SR–RD split—running slow deliberative reasoning in the background and caching its output for fast reactive decisions—is a broadly applicable architectural pattern for any real-time multi-agent LLM deployment. The cost-aware RL training (penalizing token length while rewarding task utility) is a practical instantiation of efficiency-aware RLHF for agentic reasoning models, complementing work on thinking-budget control (e.g., o1-style models). The result that an optimized 8B agent outperforms a zero-shot 70B model reinforces the importance of task-specific fine-tuning over raw scale in agentic settings.