Controllable Sim Agents with Behavior Latents

🕒 Published (v1): 2026-07-02 17:55 UTC · Source: Arxiv · link

Why this paper was selected

Controllable sim agents with behavior latents; interpretable steering axes for traffic agent simulation

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

CNeVA (Controllable Neural Variational Agents) learns a per-agent Gaussian behavior latent over an explicit reward basis (safety, map compliance, speed, acceleration) via a closed-form conjugate posterior, then conditions a rectified-flow trajectory generator on this latent for steerable traffic simulation. Soft eligibility gates replace hard binary thresholds to preserve gradient signal for near-threshold agents. On the Waymo Open Motion Dataset, CNeVA achieves competitive realism (0.7145 meta-metric) while exposing per-channel controllability absent from higher-ranked imitation models.

Problem

Existing traffic simulation approaches face a three-way tension: imitation learning produces realistic but unsteerable trajectories; VAE/diffusion generative models allow latent resampling but cannot deliberately target specific driving styles; self-play RL is controllable but computationally expensive and must be retrained whenever the reward changes. No prior method provides cheap, interpretable, per-channel behavioral steering from offline data without sacrificing trajectory realism.

Method

CNeVA extends the standard Hidden Markov Model formulation for multi-agent driving with a per-agent latent profile \(\lambda_n \in \mathbb{R}^K\) over \(K=4\) reward channels. The posterior is derived analytically via exponential-family conjugacy:

\[q^\star(\lambda_n) = \mathcal{N}\!\left(\mu_0 + \Sigma_0 G_n,\ \Sigma_0\right)\]

where \(G_n \in \mathbb{R}^K\) is the per-channel discounted return standardized by training-set statistics. This closed-form label is fed as a cross-attention token to a conditional flow-matching (CFM) decoder trained with a four-branch mixed channel-mask curriculum (null / single-channel / two-channel / full, proportions 0.2/0.4/0.2/0.2), enabling classifier-free guidance (CFG) over arbitrary channel subsets at inference. To address sparse reward signals for safety and map channels, hard binary eligibility gates are replaced with smooth exponential decay over pairwise risk (collision clearance \(\tau_c\), time-to-collision \(\tau_t\), road margin \(\tau_m\)), ensuring all valid agents receive non-trivial labels. A context-residual return \(G^{cr}_{n,k} = G_{n,k} - \bar{G}_k(m_n)\) removes scene-difficulty confounds. Contrastive conditioning (paired steered/null forward passes) further improves robustness under closed-loop drift.

Key Contributions

• Closed-form conjugate Gaussian posterior over reward-basis behavior latents, with a tilt-vs-regression interpretive framework predicting which channels are identifiable from the return-shrinkage factor.
• Soft eligibility gates (exponential decay replacing binary thresholds) that restore safety controllability from a statistically-zero CSM diagonal (\(+0.21 \pm 0.22\) hard vs. \(+0.66 \pm 0.10\) soft) without degrading trajectory fidelity.
• Context-residual return labeling to isolate driving style from scenario difficulty.
• Systematic accountability via drift-paired channel steering matrices (CSMs) paired with physical-plausibility guardrails (stall fraction, retained speed, offroad rate) to detect reward-hacking confounds.

Results

• WOSAC 2025 test server: Realism meta-metric 0.7145, minADE 1.80 m (vs. top non-controllable models: TrajTok 0.7852 / SMART-R1 0.7855).
• Open-loop CSM diagonals at \(\rho=1, w=1.5\) (context-residual): Safety \(+0.66 \pm 0.10\), Map \(+0.61 \pm 0.14\), Speed \(+8.15 \pm 0.07\), Accel \(+8.76 \pm 0.07\) vs. unconditional near-zero baselines.
• Hard-eligibility ablation safety CSM collapses to \(+0.21 \pm 0.22\) (indistinguishable from zero).
• Reward-hacking early checkpoint: speed CSM inflated ~6×, stall fraction 76%, retained speed 61% of ground truth; main model retains 95% of ground-truth speed with stall fraction near zero.
• At \(\rho=5\), all channels remain monotonically positive but physical plausibility degrades (speed-steered stall rate \(+4.6\) pp, retained speed drops to 89.1%).

Limitations

• Realism gap vs. top leaderboard entries concentrated in collision and off-road components, attributed to open-loop drift accumulating over the 8 s rollout (offroad rate ~1.3–1.9× logged rate by 8 s).
• Map conformity channel shows weak and coordinate-specific controllability under context-residual returns; improvement requires lane-centerline return alternatives.
• Posterior covariance is fixed at \(\Sigma_0\) (population-level noise), not trajectory-specific—the framework does not recover the agent's underlying utility in the inverse-RL sense.
• Sparse channels (safety, map) depend on CFG amplification (\(w>0\)) to achieve meaningful steering; latent-only control (\(w=0\)) leaves them weak.
• No closed-loop fine-tuning; open-loop drift is the primary remaining failure mode.

Relevance to Agentic AI / LLM Agents

CNeVA directly addresses the challenge of building controllable simulation agents—a prerequisite for scalable testing of any autonomous decision-making system, including LLM-based agents operating in embodied or tool-use settings. The behavior-latent formulation (inferring a compact, interpretable preference vector from reward signals) is a general technique applicable beyond driving to any agent that must be steered along specific behavioral axes without retraining. The soft eligibility and reward-hacking analysis are directly relevant to anyone designing reward signals or RLHF objectives for agentic systems, as they expose how sparse rewards cause preference learning to degrade silently. The mixed channel-mask curriculum enabling partial-channel CFG is transferable to multi-objective agentic scenarios where only a subset of behavioral constraints need to be active at any time.