Skip to content

Role-aware Multi-agent Reinforcement Learning for Coordinated Emergency Traffic Control

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

RMTC is a multi-agent RL framework for emergency traffic control that jointly coordinates traffic lights, emergency vehicles (EMVs), and regular vehicles (REVs) through dynamically learned role embeddings. It constructs a Heterogeneous Temporal Traffic Graph (HTTG) to model inter-agent relationships and uses mutual information maximization to align agent roles with EMV positions and trajectories. Evaluated at NeurIPS 2025, it outperforms all baselines on both EMV speed and overall traffic efficiency.

Problem

Existing emergency traffic control methods optimize only traffic signal timing, ignoring the navigation behavior of EMVs and REVs. This leaves vehicles without adaptive routing strategies, leading to suboptimal EMV arrival times and unnecessary REV congestion. No prior work jointly coordinates all three agent types (traffic lights, EMVs, REVs) with role-differentiated policies.

Method

RMTC comprises three components:

  1. Heterogeneous Temporal Traffic Graph (HTTG): A dynamic graph updated at each timestep with four edge types (TL-REV, TL-EMV, TL-TL, EMV-REV). EMV-REV edges use cosine similarity between observed states above threshold δ. A two-layer heterogeneous message-passing network produces role embeddings h⁽ᵗ⁾ per agent.

  2. Dynamic Role Learning: Three auxiliary objectives train the role encoder: (a) EMV Position Role Impacting — maximizes mutual information between agent role embeddings and EMV graph features via a MINE-style lower bound; (b) EMV Trajectory Role Impacting — maximizes MI between role embeddings and a GRU-encoded EMV trajectory history; (c) Role Consistency Constraint — contrastive loss pulling consecutive-timestep embeddings of the same agent together and pushing apart temporally distant ones.

  3. Role-aware MARL: Agent policy receives concat(o⁽ᵗ⁾, h⁽ᵗ⁾) as state. A role-aware intrinsic reward proportional to cosine similarity between agent and EMV role embeddings, weighted by EMV extrinsic reward, is added to each agent's environment reward. PPO with shared policy networks per agent class (vehicles, traffic lights) is used for optimization.

Key Contributions

  • Formulates the joint optimization problem over traffic lights, EMVs, and REVs as a MARL problem, the first to coordinate all three types simultaneously.
  • HTTG with four heterogeneous edge types captures spatial and temporal inter-agent dependencies.
  • Dynamic Role Learning with MI-based position/trajectory objectives and temporal consistency constraint.
  • Role-aware intrinsic reward redistributing EMV rewards proportionally to role similarity, enabling soft EMV prioritization.
  • Code released publicly.

Results

Evaluated on SUMO simulator across 2 synthetic (Grid 4×4, Avenue 4×4) and 2 real-world (Cologne8, FengLin) datasets against 8 baselines (FixedTime, MaxPressure, CoLight, IPPO, rMAPPO, X-Light, RECAL, EMVlight):

  • EMV travel time: RMTC reduces over best baseline by 4.27% (Grid4×4), 9.03% (Avenue4×4), 23.40% (Cologne8), 23.52% (FengLin); all at p < 0.01.
  • REV travel time: Simultaneous improvements of 3.96%, 5.46%, 7.84%, 5.19% on the same four datasets.
  • Multi-EMV scaling: With 1–5 concurrent EMVs on Grid4×4, RMTC maintains lower EMV and REV travel times than IPPO and rMAPPO; REV time for competitors rises sharply with EMV count while RMTC remains stable.
  • Ablation: Removing any single component (position MI, trajectory MI, consistency constraint, intrinsic reward) degrades both Temv and Trev on Grid4×4 and FengLin.

Limitations

  • Limitations section is deferred to appendix (not provided in the text); specific limitations are not enumerated in the main paper.
  • Evaluated only in simulation (SUMO); real-world deployment with noisy sensor data and partial observability not validated.
  • Scalability analysis is limited to varying EMV count (1–5) and traffic generation rate; large-scale city-wide networks not tested.
  • Shared policy networks assume role homogeneity within agent classes, which may not hold in heterogeneous real-world fleets.
  • The cosine-similarity threshold δ for EMV-REV edge construction requires manual tuning.

Relevance to Agentic AI / LLM Agents

RMTC is a strong instance of heterogeneous multi-agent coordination where agents must infer latent roles from relational context rather than fixed assignments — a core challenge in agentic systems. The dynamic role learning via mutual information maximization provides a general technique for emergent role specialization applicable beyond traffic, e.g., to LLM agent teams where agents must adapt responsibilities based on the evolving task context of a "priority" agent. The role-aware intrinsic reward mechanism is a principled approach to propagating sparse signals from key agents to supporting agents, relevant to reward shaping in hierarchical or delegative agentic architectures. The HTTG formalism generalizes naturally to any multi-agent setting with heterogeneous entity types and time-varying relational structure.