MAT-Agent: Adaptive Multi-Agent Training Optimization¶

🕒 Published (v1): 2025-10-10 19:41 UTC · Source: Arxiv · Venue: NEURIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

MAT-Agent replaces static training-configuration selection in multi-label image classification (MLIC) with four autonomous DQN-based agents that jointly and adaptively choose data augmentation, optimizer, learning rate schedule, and loss function at each training step, guided by a composite reward signal. The framework achieves state-of-the-art mAP on Pascal VOC (97.4), COCO (92.8), and VG-256 (60.9), while converging ~47% faster than standard training.

Problem¶

Conventional MLIC pipelines fix a global configuration $C^* = \{T^*_{\text{aug}}, O^*, S^*_{lr}, L^*_{\text{loss}}\}$ before training begins. This static paradigm ignores the non-stationarity of training dynamics—evolving label co-occurrence patterns, shifting class difficulty, and changing feature-label mappings—leading to suboptimal strategies during critical learning phases, premature convergence, and a performance ceiling. Joint search over the full combinatorial configuration space is computationally prohibitive and prone to local optima.

Method¶

MAT-Agent formalizes MLIC training as a sequential multi-agent decision problem. Four decentralized agents—$\text{Agent}_{AUG}$, $\text{Agent}_{OPT}$, $\text{Agent}_{LRS}$, $\text{Agent}_{LOSS}$—each independently maintain a DQN policy $\pi_k(a^k \mid I_t; \theta_k)$ over discrete candidate strategies. At each decision step $t$, they observe an extended state $I_t$ encoding performance indicators ($\text{mAP}^{\text{val}}_t$), training dynamics (loss, gradient $L_2$ norm, relative update magnitudes), and dataset descriptors (texture richness). Each agent selects an action via $\epsilon$-greedy with curiosity-driven intrinsic rewards, assembling the joint configuration $C_t = (a^{AUG}_t, a^{OPT}_t, a^{LRS}_t, a^{LOSS}_t)$.

Agents share a global composite reward: $$R_{t+1} = w_{mAP} \cdot f(\Delta \text{mAP}_t) + w_{\text{stab}} \cdot \text{Stability}_t + w_{\text{conv}} \cdot \text{Convergence}_t - w_{\text{pen}} \cdot \text{Penalty}_t$$

Q-networks are trained via TD-learning with experience replay and target networks. The main MLIC model $\Theta_M$ is updated by the selected configuration but its gradient mechanics are untouched—MAT-Agent operates at the hyperparameter/strategy level only.

Key Contributions¶

Reformulates MLIC training configuration selection as a multi-agent sequential decision problem, replacing static search with online adaptive policies.
Four specialized DQN agents jointly control augmentation, optimizer, LR scheduler, and loss function with a shared reward that balances accuracy, rare-class performance ($\Delta\text{mAP}$), training stability, and convergence speed.
Curiosity-driven intrinsic reward via state-transition prediction error to encourage exploration without excessive random sampling.
Dual-rate exponential moving average smoothing and mixed-precision training for robustness and efficiency.
Demonstrated ~47% reduction in epochs to reach equivalent mAP versus standard training on COCO.

Results¶

Pascal VOC: mAP 97.4 / OF1 92.3 / CF1 91.4 (vs. PAT-T: 96.2 / 91.1 / 90.6).
COCO: mAP 92.8 / OF1 88.2 / CF1 87.1 (vs. HSQ-CvN: 92.0 / 87.5 / 86.6).
VG-256: mAP 60.9 / OF1 70.8 / CF1 61.1 (vs. PAT-T: 59.5 / 69.8 / 59.7).
Convergence: reaches 63.8% mAP in 47 epochs vs. ~80 epochs for standard training; at epoch 47 outperforms standard, PBT, and BOHB by 3.5, 2.2, and 1.5 mAP points respectively.
Cross-dataset transfer (trained on COCO): mAP 76.2% on VOC, 62.5% on NUS-WIDE, 53.4% on OpenImages—surpassing DARTS by 2.4–2.8 points on each.
Compute efficiency: converges by $\sim10^2$–$10^3$ GPU-hours while Grid Search, BOHB, and DARTS have not yet converged at $10^3$ GPU-hours.

Limitations¶

Evaluated exclusively on multi-label image classification; generalizability to other task types is undemonstrated.
Action spaces (augmentation policies, optimizers, etc.) are predefined discrete sets—novel strategies outside the candidate list cannot be discovered.
Four separate DQN agents each with replay buffers and target networks add non-trivial overhead on top of the main model; exact overhead is not quantified in the main text.
Reward weight hyperparameters ($w_{mAP}, w_{\text{stab}}, w_{\text{conv}}, w_{\text{pen}}$) and Q-network architectures are relegated to supplementary material, limiting reproducibility from the main text alone.
Cross-dataset experiments use stratified sampling of large datasets rather than full evaluation, potentially underestimating variance.

Relevance to Harnesses / Meta-Harnesses¶

MAT-Agent is a concrete instance of a training meta-harness: an outer control loop that wraps a base model's training process and adaptively reconfigures it in response to observed state, without touching the model's internals. This is structurally analogous to meta-harnesses that orchestrate agent pipelines—the four DQN agents collectively function as an adaptive scheduler/dispatcher deciding which "tools" (augmentation, loss, optimizer) the inner model uses at each step. The shared reward and composite state representation are the harness's observability and feedback layer, a pattern directly applicable to meta-harnesses that must balance competing sub-agent objectives. Work in this space is increasingly relevant as harness designers move from static pipeline configurations to online, feedback-driven orchestration.