MAT-Agent: Adaptive Multi-Agent Training Optimization¶

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

MAT-Agent frames multi-label image classification (MLIC) training as a sequential decision-making problem solved by four autonomous DQN-based agents that dynamically select data augmentation, optimizer, learning rate schedule, and loss function at each training step. Using non-stationary multi-armed bandit-style exploration with a composite reward, it outperforms static and AutoML baselines on Pascal VOC, COCO, and VG-256 while converging ~47% faster than standard training.

Problem¶

Conventional MLIC training relies on a single static configuration C* fixed before training begins, ignoring the non-stationarity of evolving label co-occurrence patterns, class difficulty, and feature-label mappings. Existing AutoML methods (grid/random search, PBT, BOHB) either treat components independently, incur prohibitive search overhead, or cannot adapt in real time to intermediate training feedback.

Method¶

MAT-Agent maintains four independent DQN agents—AUG, OPT, LRS, LOSS—each controlling one training component from a discrete action space (e.g., {SGD, Adam, AdamW, RAdam} for OPT). At each decision step t, every agent observes an extended state It = [s_perf; s_dyn; s_data] encoding validation mAP, loss statistics, gradient norms, and dataset texture descriptors, then selects an action via ε-greedy exploration with curiosity-driven intrinsic rewards. Joint configuration Ct is assembled from the four actions and applied to the MLIC backbone. A shared composite reward Rt+1 = w_mAP·f(ΔmAP) + w_stab·Stability + w_conv·Convergence − w_pen·Penalty propagates back to update all agents via standard DQN with experience replay and target networks. Dual-rate EMA smoothing and mixed-precision training are added for stability and efficiency.

Key Contributions¶

Multi-agent formulation of training-hyperparameter control as decentralized DQN with shared reward, enabling coordinated but independent per-component agents.
Curiosity-driven intrinsic reward augmenting exploration beyond ε-greedy, particularly beneficial for rare-class scenarios.
Composite reward harmonizing accuracy (ΔmAP), stability (loss fluctuation), convergence speed, and configuration cost penalties.
Demonstration that inter-agent coordination (joint reward, shared state) is necessary: ablating coordination alone causes measurable mAP degradation even with all four agents present.

Results¶

Pascal VOC: mAP 97.4 vs. PAT-T 96.2 (+1.2); OF1 92.3 vs. PAT-T 91.1 (+1.2); CF1 91.4 vs. PAT-T 90.6 (+0.8).
MS-COCO: mAP 92.8 vs. HSQ-CvN 92.0 (+0.8); OF1 88.2 vs. HSQ-CvN 87.5 (+0.7); CF1 87.1 vs. HSQ-CvN 86.6 (+0.5).
VG-256 (256-class long-tail): mAP 60.9 vs. PAT-T 59.5 (+1.4); OF1 70.8 vs. PAT-T 69.8 (+1.0).
Convergence: reaches 63.8% mAP on COCO in 47 epochs vs. ~80 for standard training (47% reduction); at epoch 47, outperforms standard training, PBT, BOHB by 3.5, 2.2, 1.5 mAP points respectively.
Zero-shot cross-dataset transfer (COCO→VOC/NUS-WIDE/OpenImages): mAP 76.2/62.5/53.4 vs. DARTS 73.8/59.7/50.8 (+2.4–2.8 points).
Compute efficiency: converges by ~10² GPU-hours while baselines (Grid Search, BO, BOHB, PBT, DARTS) have not converged at 10³ GPU-hours.

Limitations¶

Evaluated exclusively on MLIC; generalizability of the multi-agent training-controller paradigm to other task types (detection, generation) is undemonstrated.
Four agents and associated DQN training add system complexity and overhead not fully quantified in absolute wall-clock terms beyond the GPU-hour convergence curves.
Action spaces for each component are predefined discrete sets; novel augmentations or optimizers outside these sets cannot be discovered.
State representation relies on hand-engineered features (texture richness, gradient norms); sensitivity to these design choices is not ablated.
Cross-dataset experiments use stratified sampling on large datasets rather than full evaluation, limiting generalization claims.

Relevance to Agentic AI / LLM Agents¶

MAT-Agent demonstrates a principled instantiation of multi-agent systems for meta-level control of a learning process—each agent acts as a specialist responsible for one facet of the environment (training pipeline), communicating indirectly through shared state and reward rather than explicit messaging. This is a concrete example of the agent-as-optimizer paradigm, where autonomous agents replace static heuristics in complex, non-stationary pipelines, directly relevant to ongoing work on agentic orchestration and dynamic workflow adaptation. The curiosity-driven exploration and composite reward design are transferable techniques for any multi-agent setup requiring coverage of rare or difficult cases. While the domain is computer vision rather than language, the framework maps naturally onto LLM fine-tuning orchestration, where analogous components (data curriculum, optimizer choice, loss shaping) similarly require adaptive coordination.