Skip to content

RLSLM: A Hybrid Reinforcement Learning Framework Aligning Rule-Based Social Locomotion Model with Human Social Norms

🕒 Published (v1): 2025-11-14 13:59 UTC · Source: Arxiv · Venue: AAAI 2026 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

RLSLM is a hybrid framework that embeds a psychologically grounded, orientation-sensitive social discomfort field directly into the reward function of a reinforcement learning agent to produce socially compliant navigation in human-populated spaces. By jointly minimizing mechanical energy and social discomfort (via A2C), the agent learns to respect personal space and social groupings within 10,000 training steps. A VR human study (N=30) shows it significantly outperforms prior rule-based baselines on perceived comfort.

Problem

Rule-based socially-aware navigation models are interpretable but brittle and prone to oscillatory paths; data-driven RL methods are flexible but opaque, data-hungry, and poorly aligned with human social intuitions. No prior work had directly encoded empirically validated, human-behavioral-experiment-derived social norms into an RL reward signal.

Method

RLSLM integrates the Social Locomotion Model (SLM) of Zhou et al. 2022—derived from controlled behavioral experiments—as a dense reward term inside an A2C actor-critic loop. The SLM computes an orientation-sensitive, asymmetric discomfort field F over the navigation space via three components: - HRSC (Heading-Relevant Social Component): scales discomfort by cos(θ_h), penalizing the agent more for passing in front of a person facing it. - HISC (Heading-Irrelevant Social Component): constant isotropic discomfort term. - CAC (Collision Avoidance Component): elliptical body-occlusion term with semi-axes a=0.285, b=0.175 m.

Per-person influence is F'_k = min(F_k/K, 1) (clipped at K=10.18), summed over all k persons. The total per-step reward is r_t = R_d + R_e + σ·R_s, where R_d rewards goal progress, R_e penalizes steps (α=1), and R_s is the social influence penalty (σ=0.5). Training uses an MLP (64-128-256-128-64) on OpenAI Gymnasium / Stable-Baselines3, converging in 10,000 steps on an NVIDIA 3090.

Key Contributions

  • Hybrid RL framework that encodes psychologically grounded, empirically fitted social norms as a dense reward signal, not as a post-hoc constraint.
  • Orientation-sensitive social influence field with fitted parameters directly from human behavioral experiments (rather than intuition or dataset statistics).
  • Immersive first-person VR evaluation pipeline (Unreal Engine 5.4, HTC Vive Pro) with 30 participants rating trajectories, establishing a publicly released benchmark dataset.
  • Ablation demonstrating each SLM component's role: removing HRSC causes the agent to pass in front of people 57.76% vs. 11.9% of the time; removing HISC or CAC reduces Maximum Lateral Distance.
  • Sensitivity analysis over σ ∈ {0, 0.5, 1.0, 2.0} showing tunable social conservatism.

Results

  • Comfort rating: RLSLM achieves 4.21/5 vs. best rule-based baseline; Δrating = +1.12 over COMPANION (Bonferroni corrected, P < 0.001).
  • ANOVA: F(2,58) = 219.589, P < 0.001, η²_G = 0.525 — large effect of model type on comfort.
  • RLSLM outperforms both COMPANION and n-Body in single- and multi-human scenarios (all pairwise P < 0.001).
  • RLSLM and n-Body both improve in multi-human vs. single-human scenarios (P < 0.001 and P = 0.008 respectively); COMPANION does not (P = 0.251).
  • Convergence in ≤10,000 training steps in both single- and multi-human conditions.
  • Ablation (HRSC): full model passes in front of a person in 5/42 scenarios vs. 23/42 without HRSC.

Limitations

  • Evaluated only in a static-human setting (pedestrians are stationary); dynamic crowd interactions are not tested.
  • Small VR study (N=30, 11M/19F) from a single university, limiting demographic generalizability.
  • Discrete step-based movement requires Gaussian post-processing to smooth trajectories, introducing a gap from continuous real-robot motion.
  • Scenarios are limited to 15m × 15m with at most three people; scalability to dense crowds is unproven.
  • The fitted SLM parameters (m, n, c, K, a, b) are culture- and context-specific; cross-cultural generalization is unaddressed.
  • LLM or language-grounded social reasoning is entirely absent; social norms are hard-coded from a specific prior experiment.

Relevance to Agentic AI / LLM Agents

This work demonstrates a concrete alignment strategy—embedding empirically derived human behavioral models directly into an agent's reward function—that is directly analogous to challenges in aligning LLM-based agents with human social norms. The approach of using psychologically grounded, structured reward shaping rather than purely human feedback or demonstration data offers a complementary path to RLHF, applicable wherever agent behavior must respect implicit social conventions. For agentic systems operating in physical or social environments (e.g., embodied agents, social robots, or dialogue agents respecting conversational proxemics), RLSLM's decomposition of social influence into interpretable, ablatable components (orientation sensitivity, body avoidance, isotropic comfort) provides a principled template. The VR evaluation pipeline also sets a precedent for high-ecological-validity human-agent interaction benchmarks beyond text-only settings.