Skip to content

EmoAgent-R1: Towards Multimodal Emotion Understanding with Reinforcement Learning-based Dynamic Agent Specialization

🕒 Published (v1): 2026-07-23 07:52 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

EmoAgent-R1 trains a single MLLM to dynamically select and instantiate specialized emotion-reasoning agents at inference time, replacing static monolithic prompting. A two-phase pipeline uses answer-conditioned CoT cold-start training followed by Progressive Group-Relative Policy Optimization (P-GRPO), which adds PMI-inspired token-level modulation to GRPO's uniform credit-assignment scheme. Experiments on MER-UniBench show gains over static-prompt and standard GRPO baselines on sentiment analysis, basic emotion recognition, and fine-grained emotion detection.

Problem

Existing MLLM-based multimodal emotion recognition (MER) systems apply a single static prompt to all inputs, implicitly treating every modality and temporal segment as equally informative ("uniformity bias"). This monolithic design produces brittle optimization and hallucinations because the model cannot dynamically route attention to the most salient cue (e.g., vocal tone vs. facial micro-expression vs. sarcasm). Standard GRPO exacerbates the problem via coarse uniform credit assignment: a scalar sequence-level reward is broadcast identically to every token, diluting gradients for critical reasoning steps.

Method

Architecture — Dynamic Agent Specialization (DSA): A single MLLM backbone plays two roles sequentially. A Router Agent \(\pi_\text{router}(z \mid x)\) reads the multimodal input and selects an expert \(z^* \in \mathcal{A} = \{a_1, \ldots, a_K\}\); each expert is a distinct prompt template activating a specific reasoning strategy (visual-facial, sarcasm-detection, cross-modal contradiction, etc.). A Specialized Agent \(\pi_\text{agent}(o \mid x, z^*)\) then generates a focused Chain-of-Thought trajectory conditioned on the chosen strategy.

Phase I — Cold-Start Training: A Qwen2.5-VL-72B teacher generates \(K{=}8\) answer-conditioned CoT candidates per sample; a GPT-4o verifier filters hallucinated or factually inconsistent trajectories (DeepSeek-style rejection sampling), yielding dataset \(D_\text{reason}\). Router labels are constructed via Empirical Oracle Hindsight Relabeling: all \(K\) agents are run on each instance; the subset \(\mathcal{Z}^* \subset \mathcal{A}\) that correctly predicts the label becomes the supervised routing target, converting unsupervised routing into a classification task. SFT on the combined 124k-sample dataset initializes a robust policy manifold.

Phase II — P-GRPO: Token-level advantages are computed as \(\hat{A}_t^{(g)} = A^{(g)} \cdot M_t^{(g)}\), where the modulation factor \(M_t^{(g)}\) approximates the incremental PMI between token \(o_t\) and the success event. Concretely, normalized rewards are mapped to proxy success probabilities \(p_t^{(g)} = \text{clip}(p_\text{final}^{(g)\,\beta \cdot (t/L_g)}, p_\text{min}, p_\text{max})\) with \(\beta{=}12\). Incremental log-ratios \(\text{PMI}_t^{(g)} = \log(p_t^{(g)}/p_{t-1}^{(g)})\) are exponentiated and layer-normalized to produce \(M_t^{(g)}\). For failed trajectories (\(p_\text{final}^{(g)} \leq \tau\)) a constant attenuation \(\omega_0\) suppresses noisy gradients. The final objective is a PPO-style clipped loss with KL regularization; when \(M_t \equiv 1\), P-GRPO reduces exactly to GRPO. Rewards combine an accuracy component using Emotion Wheel (EW) open-vocabulary metrics and a binary format reward.

Key Contributions

  • EmoAgent-R1 framework: two-stage agentic workflow (router → specialized agent) over a single MLLM backbone, enabling dynamic prompt-strategy selection at inference time.
  • Empirical Oracle Hindsight Relabeling: converts the unsupervised expert-routing problem into supervised classification using hindsight oracle labels derived from parallel agent execution.
  • P-GRPO algorithm: replaces GRPO's uniform token credit assignment with PMI-based progressive token-level modulation, transforming sparse sequence-level rewards into fine-grained learning signals without auxiliary value networks.
  • Cold-start training pipeline: answer-conditioned CoT synthesis with rejection sampling stabilizes RL initialization for the multi-agent policy.

Results

The paper's results tables are not included in the provided text excerpt (truncated before Table 2 / quantitative comparisons). The paper claims: - Superiority over monolithic MLLM baselines on MER-UniBench across sentiment analysis, basic emotion recognition, and fine-grained emotion understanding. - Improved optimization stability relative to standard GRPO. - The full numerical breakdown against specific baselines (e.g., R1-Omni, AffectGPT, DAPO variants) is not available from the provided text.

Limitations

  • The specialized agent library \(\mathcal{A}\) relies on manually designed prompt templates; the number and types of experts are fixed at design time rather than learned end-to-end.
  • Cold-start data quality depends on a proprietary GPT-4o verifier and a large (72B) teacher model, making reproduction expensive.
  • The PMI-inspired modulation assumes monotonically increasing confidence along correct reasoning trajectories (\(\beta{=}12\)), which is a strong prior that may not hold for non-sequential or highly ambiguous emotional cues.
  • Evaluated only on MER-UniBench; generalization to other multimodal reasoning benchmarks is not demonstrated.
  • Results tables were not present in the supplied text, so quantitative claims cannot be independently verified from this excerpt.

Relevance to Harnesses / Meta-Harnesses

EmoAgent-R1 is a concrete instantiation of a trained meta-harness: rather than hard-coding an orchestration strategy, it learns—via RL—when to route to which sub-agent, making the harness itself a policy that adapts to input context. The two-stage workflow (router → specialist) mirrors the harness/tool-agent split seen in LLM orchestration frameworks, but here the routing decision is generated autoregressively by the same model rather than enforced by external glue code. The cold-start relabeling approach (Empirical Oracle Hindsight Relabeling) offers a transferable recipe for bootstrapping any meta-harness that lacks direct supervision over its routing decisions. P-GRPO's token-level credit assignment is directly applicable to any multi-step agentic trajectory where some harness-generated tokens (e.g., tool-call arguments, agent dispatch tokens) are more causally relevant to the final outcome than others.