Skip to content

MMedAgent-RL: Optimizing Multi-Agent Collaboration for Multimodal Medical Reasoning

🕒 Published (v1): 2026-01-01 · Source: ICLR · Venue: ICLR 2026 · link

Why this paper was selected

ICLR 2026; multi-agent RL framework for specialist medical VLM collaboration

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

MMedAgent-RL is a reinforcement learning-based multi-agent framework for multimodal medical reasoning that trains two general-practitioner (GP) agents — a triage doctor and an attending physician — via GRPO on top of Qwen2.5-VL. The key novelty is Curriculum-Based Multi-Agent RL (C-MARL), which dynamically regulates the attending physician's policy entropy according to specialist reliability, enabling it to exploit correct specialist consensus and explore away from misleading ones. On five medical VQA benchmarks it outperforms all single- and multi-agent baselines, exceeding the best SFT baseline by 23.6%.

Problem

Single Med-LVLMs cannot generalize across diverse medical sub-specialties (radiology, pathology, ophthalmology, etc.) due to domain breadth. Prior multi-agent frameworks simulate a GP→Specialist→GP clinical workflow but rely on static, predetermined interaction patterns with no learned optimization, and they offer no mechanism to handle noisy or incorrect specialist outputs — blindly imitating unreliable specialists corrupts attending-physician training.

Method

The pipeline has three learned/fixed stages:

  1. Triage doctor (first GP). Trained via GRPO with format reward \(R_\text{format} \in \{0, 0.5\}\) and accuracy reward \(R_\text{accuracy} \in \{0, 1\}\) to select one of \(k=7\) medical specialties. Ground-truth specialty labels are derived from dataset provenance (e.g., pathology slides → Pathologist).

  2. Specialist doctors. Fixed proprietary LVLMs (GPT-4o) provide \(e=3\) independent opinions per case within the triage-selected department. No cross-specialist interaction occurs to avoid majority-vote suppression of minority correct answers.

  3. Attending physician (second GP) via C-MARL. Training data is partitioned by specialist accuracy \(s = \text{Acc}(y_d, y^*)\) into \(D_\text{easy}\) (\(s=1\)), \(D_\text{medium}\) (\(0 < s < 1\)), \(D_\text{hard}\) (\(s=0\)). GRPO is augmented with an entropy-regularization term: $\(J_\text{C-MARL}(\theta) = \mathbb{E}\!\left[J_\text{GRPO}(\theta) + \gamma_s \cdot H_t(\pi_\theta^\text{attend})\right], \quad H_t = -\sum_{j=1}^V p_{t,j}\log p_{t,j}\)$ where \(\gamma_s\) is curriculum-level-dependent: \(\gamma_\text{easy} \approx 0\) (exploit reliable consensus), \(\gamma_\text{medium} > 0\) (moderate exploration), \(\gamma_\text{hard} \gg \gamma_\text{medium}\) (aggressive exploration to override misleading specialists). Training proceeds in three sequential stages: easy → medium → hard.

Key Contributions

  • MMedAgent-RL: end-to-end RL-optimized multi-agent medical reasoning framework replacing static pipelines with trainable GP agents.
  • C-MARL: curriculum-guided RL for the attending physician with dynamic per-sample entropy coefficients \(\gamma_s\) tied to specialist reliability.
  • Stage-wise GRPO training for both GP agents using rule-based format and accuracy rewards; no critic network required.
  • Theoretical convergence analysis (Theorem 4.1) proving curriculum learning achieves lower parameter error than standard SGD under the same sample budget, with the gap depending on \(\sum_{j=0}^{J-1}\log\|\theta_j^* - \theta_{j+1}^*\|_2^2\).

Results

  • In-domain average (VQA-RAD, SLAKE, PathVQA): MMedAgent-RL 73.3% vs. best multi-agent baseline AFlow 67.5% (+5.8 pp); vs. GPT-4o single-agent 68.6%.
  • Out-of-distribution average (OmniMedVQA, MMMU-Health&Med): MMedAgent-RL 72.6% vs. AFlow 56.6% (+16.0 pp); vs. GPT-4o 69.1%.
  • With test-time scaling (majority voting): 76.1% in-domain, 76.6% OOD.
  • Versus SFT baseline: +23.6% average gain.
  • Per-dataset in-domain gains over best multi-agent baseline: VQA-RAD +4.2 pp, SLAKE +7.3 pp, PathVQA +5.9 pp.

Limitations

  • Specialist doctors are proprietary closed models (GPT-4o), introducing cost, latency, and reproducibility concerns.
  • Only 7 fixed medical specialties; coverage of niche sub-specialties is limited.
  • Evaluation restricted to multiple-choice VQA; generalization to open-ended clinical text generation is untested.
  • Curriculum difficulty labeling requires running specialist inference on all training examples as a preprocessing step, adding pipeline complexity.
  • Triage ground-truth labels are derived from dataset provenance heuristics, which may not reflect realistic clinical ambiguity.

Relevance to Vision-Language Models

MMedAgent-RL extends the single-agent RL reasoning paradigm (e.g., DeepSeek-R1, GRPO) to a structured multi-agent VLM system, demonstrating that policy entropy control is a viable mechanism for managing noisy teacher signals in collaborative inference — a general challenge for any multi-agent VLM pipeline. The C-MARL curriculum shows how difficulty-stratified RL training can outperform both SFT and static multi-agent workflows, offering a blueprint applicable beyond medicine to any domain requiring specialist consultation under uncertainty. For researchers tracking VLMs, the framework bridges clinical workflow simulation with learned optimization, showing that Med-LVLMs benefit more from learned coordination strategies than from simple model scaling.