Skip to content

Agentic Neural Architecture Search

🕒 Published (v1): 2026-07-08 23:20 UTC · Source: Arxiv · link

Why this paper was selected

Agentic NAS: LLMs autonomously design+expand search spaces; novel meta-agent loop

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

AgentNAS proposes a three-phase pipeline where an LLM generates a high-quality seed architecture, decomposes it into a "slotted architecture" of named interchangeable module slots, and hands the resulting bounded search space to conventional NAS for combinatorial refinement. This cleanly separates LLM-driven design from NAS-driven exploration, enabling the first empirical study of their division of labor. The full pipeline achieves state-of-the-art on 11 of 17 diverse tasks across NAS-Bench-360 and Unseen NAS.

Problem

Every NAS method presupposes a hand-engineered search space requiring domain expertise and per-task reconstruction. LLMs can generate architectures in open-ended spaces but prior work conflates two distinct roles—expanding the architecture space and exploring it—preventing study of when each capability adds value and what the optimal labor division is.

Method

Phase 1 — LLM Seed Search: A Planner–Code Generator–Data Explorer agent loop iterates for up to 160 evaluations. The Planner selects <EXPLORE>, <EXPERIMENT>, or <STOP> actions; each candidate is trained under a 30%-epoch proxy with GPU/VRAM constraints; the top-8 architectures are fully trained after saturation.

Phase 2 — Slotted Architecture Synthesis: A Slot Planner agent decomposes the seed into a slotted architecture with two structurally distinct slot types: - Module-level slots: replaceable nn.Module blocks performing coarse, coherent transformations (e.g., stacked DWP-SE-GELU vs. MBConv-SE-SiLU). - Additive glue slots: optional components interleaved between modules, defaulting to identity so the seed's behavior is preserved unless NAS selects an alternative. - A learning rate slot with five multipliers \(\mu \in \{0.25, 0.5, 1.0, 2.0, 4.0\}\) compensates for parameter-count shifts introduced by module replacement.

A Modularizer Agent first groups primitive sequential operations (conv → norm → activation) into coherent module-level units before slot decomposition, preventing overly fine-grained slots that destabilize search.

Phase 3 — Slotted NAS: Regularized evolution (default), random search, or GDAS explores the discrete categorical slot search space. Top-8 candidates are fully trained using the same proxy protocol as Phase 1. All LLM agents use Claude Sonnet 4.6 at temperature 0.7, with separate context windows per agent.

Key Contributions

  • Slotted architecture mechanism: LLM decomposes its own seed to automatically construct a task-specific, bounded NAS search space without manual engineering.
  • Empirical study of LLM–NAS division of labor, showing the two search mechanisms are broadly complementary: LLM sampling saturates and NAS then delivers additional gains via combinatorial recombination across slots.
  • State-of-the-art on 11/17 tasks across NAS-Bench-360 and Unseen NAS, including tasks with deliberately obfuscated domain metadata.
  • Evidence that Phase 2 constructs a stronger search space than grammar-based alternatives (einspace): fixed-seed + LLM-slotted NAS at 160 evaluations outperforms einspace with a fixed seed at 500 evaluations on Spherical and CIFARTile.

Results

  • Overall: SOTA on 11/17 tasks; AgentNAS-Full average rank 1.50 vs. 3.60 (Expert) on NAS-Bench-360 and 1.71 vs. 2.14 (NAS Best) on Unseen NAS.
  • NAS-Bench-360 (10 tasks): Best on 8/10; e.g., Spherical error 38.71% vs. 66.37% (Expert), Cosmic 1−AUROC 0.051 vs. 0.127 (Expert).
  • Unseen NAS (7 tasks): Average error 13.06% vs. 15.04% for per-task-best conventional NAS; Language task: 0.49% vs. 2.08% (einspace best).
  • LLM seed alone (Phase 1): Already surpasses all published baselines on the majority of tasks before any NAS.
  • NAS additive gain: Spherical 45.08→38.71, CIFARTile 11.20→8.48, Chesseract 33.84→31.39.
  • Complementarity across LLMs: Pattern holds for Claude Haiku 4.5, Sonnet 4.6, and Opus 4.6; seed quality scales with capability, but NAS benefit after LLM saturation is consistent.
  • Search algorithm sensitivity: Differences between regularized evolution, random search, and GDAS within the slotted space are modest.

Limitations

  • Underperforms the per-task-best baseline on 6/17 tasks (NinaPro, PSICOV, AddNIST, MultNIST, CIFARTile, Chesseract—each beaten by a specific conventional NAS method), showing LLM-generated search spaces do not universally dominate hand-designed ones.
  • Darcy Flow Sonnet-seed NAS fails to improve; traced to search space quality rather than seed quality, suggesting Phase 2 decomposition quality is LLM-capability-dependent.
  • Fixed-seed ablation results could not be fully reproduced; results reported with noted discrepancy (consistent with a discrepancy also observed in einspace).
  • Hardware constrained to a single RTX 2080 Ti (11 GB VRAM); fixed-seed Darcy Flow experiment is OOM.
  • Isabella task excluded due to data redistribution rights.
  • Moderate run-to-run variance (standard deviation 0.38–2.10 percentage points across three random seeds).
  • Search algorithm hyperparameters are untuned; future tuning may reveal larger algorithm differences.

Relevance to Agentic AI / LLM Agents

AgentNAS exemplifies a multi-agent pipeline architecture where specialized LLM agents (Planner, Code Generator, Data Explorer, Slot Planner, Modularizer) collaborate to perform a task that is then handed to a non-LLM optimizer—a direct instantiation of the "LLM as architect, algorithm as executor" agentic pattern. The paper's central empirical finding—that LLM sampling and combinatorial NAS are complementary, with LLMs excelling at high-level structural design and NAS at systematic combinatorial exploration—generalizes to broader questions about when to deploy LLM agents versus deterministic search in automated scientific discovery. The slotted architecture mechanism is a concrete example of an LLM agent constructing a structured action/decision space for a downstream optimizer, relevant to any agentic system that must define exploration scopes dynamically. The blind Unseen NAS setting, where agents receive no domain metadata and must discover task structure from data alone, also serves as a hard benchmark for domain-agnostic LLM agent capability.