Skip to content

Out-of-Distribution Generalization with a SPARC: Racing 100 Unseen Vehicles with a Single Policy

🕒 Published (v1): 2025-11-12 20:58 UTC · Source: Arxiv · Venue: AAAI 2026 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

SPARC (Single-Phase Adaptation for Robust Control) collapses the two-stage RMA training procedure into a single joint-training loop, where a context-encoder expert policy and a history-adapter policy are trained simultaneously. Evaluated on Gran Turismo 7 with ~100 held-out car models and wind-perturbed MuJoCo environments, SPARC matches or exceeds two-phase RMA on OOD generalization while being simpler to implement and compatible with continual learning.

Problem

Contextual RL agents must generalize to out-of-distribution (OOD) environments where privileged context (e.g., vehicle mass, friction, wind speed) is unavailable at test time and must be inferred from interaction history. The dominant framework, RMA, requires a two-phase procedure: (1) pre-train a context encoder with privileged information, (2) freeze it and train a history adapter against that fixed checkpoint. This introduces a brittle intermediate checkpoint-selection step and makes continual learning impossible without retraining phase 2.

Method

SPARC trains an expert policy π^ex (with observation encoder + context encoder ψ) and an adapter policy π^ad (with observation encoder + history adapter ϕ) simultaneously using QR-SAC. The history adapter minimizes the MSE between its latent output ẑ = ϕ(h) and the context encoder's output z = ψ(c):

L_ϕ(c, h) = E[(z − ẑ)²]

Unlike RMA, z is a moving target because ψ is still training. A critical design choice: the adapter policy π^ad rolls out in the environment during training (not π^ex), which keeps its learning closer to on-policy and allows error correction before deployment. The expert's critic networks retain context access during training but are not needed at test time — only π^ad is deployed. Black (non-gradient) weight copies from π^ex to π^ad propagate learned observation representations.

Key Contributions

  • SPARC: single-phase training eliminating the phase-1 checkpoint selection problem inherent in RMA.
  • Demonstrated that a non-stationary context-encoding target is tractable for history adapter training.
  • OOD generalization across ~100 unseen car models in Gran Turismo 7 across 3 tracks with highly varied road types.
  • Zero-shot transfer to post-update Gran Turismo physics (different simulator dynamics never seen during training).
  • Ablation studies on history length (H=50 used) and rollout policy choice (Ï€^ad vs. Ï€^ex).

Results

  • Gran Turismo Car Models (OOD, Table 1): SPARC achieves best OOD BIAI ratio on 2/3 tracks and highest aggregate lap completion. On Nürburgring: SPARC BIAI 1.1199 / 89.00% success vs. RMA 1.2995 / 77.99%. On Grand Valley: SPARC 1.0491 / 98.06% vs. RMA 1.0560 / 97.09%.
  • Power & Mass (OOD, Table 4): SPARC BIAI ratio 0.9907 / 99.90% success — outperforms all baselines including Oracle (0.9962 / 99.27%), attributed to history providing useful partial-observability mitigation.
  • MuJoCo Wind Perturbations (OOD, Table 5): SPARC best on HalfCheetah (10017.90 vs. RMA 9033.87, Oracle 7821.42) and Walker2d (2528.25 vs. RMA 2306.23); competitive on Hopper (1348.22 vs. Oracle-best 1710.14).
  • GT7 Physics Update (zero-shot, Figure 4c): SPARC completes laps with >95% of OOD cars; Oracle ~90%, RMA and others degrade significantly.

Limitations

  • Gran Turismo training code is proprietary; only MuJoCo environments are open-sourced.
  • History adapter trains against a non-stationary target — adds a source of instability not present in RMA's fixed-target second phase; the paper shows empirically this works but does not analyze convergence theoretically.
  • Not tested on physical robotic hardware; sim-to-real transfer remains an open step.
  • Privileged context c is still required during training; purely blind training from scratch is not addressed.
  • Oracle comparison is asymmetric: Oracle receives true context but no history, making cross-setting comparisons partially confounded by the history benefit.

Relevance to Agentic AI / LLM Agents

This paper is primarily a robotics/control RL contribution, but it addresses a core agentic challenge: enabling a deployed policy to adapt to unseen environment dynamics using only its own interaction history, without any privileged runtime signal. The SPARC framework — inferring latent context from history to guide behavior — is structurally analogous to how LLM-based agents use in-context trajectory history to adapt their action selection, and the single-phase design improves on RMA's modularity in the same way that end-to-end training often outperforms pipeline approaches in agent architectures. The continual learning compatibility of SPARC (no phase boundary requiring retraining) is directly relevant to persistent agents that must update online in deployment without cloud retraining, a constraint increasingly important in privacy-sensitive agentic settings.