Skip to content

CASCADE: Cumulative Agentic Skill Creation through Autonomous Development and Evolution

🕒 Published (v1): 2025-12-29 21:50 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

CASCADE is a self-evolving multi-agent framework for scientific research that shifts from static "LLM + tool use" to dynamic "LLM + skill acquisition" by cultivating two meta-skills—continuous learning and self-reflection—that accumulate reusable executable skills across sessions. Evaluated on SciSkillBench (116 materials science/chemistry tasks), it achieves 93.3% success with GPT-5 versus 35.4% for the same model without evolution mechanisms. Skills and memory persist across agents and human collaborators, enabling cumulative scientific capability.

Problem

Current LLM agents rely on predefined, human-curated tool catalogs and task-specific prompts that cannot adapt beyond design-time scope. Autonomous tool generation efforts are limited to Python built-ins, lack sustained skill mastery, and lack memory-based consolidation across sessions. No benchmark existed for evaluating agents' ability to autonomously use diverse domain-specific scientific tools.

Method

CASCADE consists of an Orchestrator agent that routes queries to either a SimpleSolver (rapid code generation + execution for routine tasks) or a DeepSolver (four-agent sequential pipeline: Solution Researcher → Code Agent → parallel Debug Agents (×3, concurrent) → Output Processor). DeepSolver implements two meta-skills:

  • Continuous learning: real-time web search, code extraction from URLs, and memory retrieval to acquire external tool knowledge on-the-fly.
  • Self-reflection: code introspection, runtime probing, knowledge graph exploration, and local package investigation to reason about failures and adapt strategies beyond simple retry loops.

Memory is dual-layered: session-wise memory (conversation history) and consolidated memory (vector + graph databases storing skill sets, API credentials, user preferences, accumulated experience). Successful skills are codified and shared across agents and human scientists. The framework is domain-agnostic at the prompt and tool level; no materials-science-specific system prompts are used.

Key Contributions

  • CASCADE framework: first instantiation of "LLM + skill acquisition" with persistent, shareable, cross-session skill accumulation.
  • SciSkillBench: 116-task benchmark spanning 6 categories (data retrieval, analysis, management, processing, simulation, specialized toolkits) at two difficulty levels (L0/L1), with automated evaluation.
  • DeepSolver architecture: four-agent pipeline with conditional parallel debugging and two explicit meta-skills, outperforming native and search+debug baselines across 9 LLM backbones.
  • Real-world integrations: autonomous lab synthesis/characterization via AlabOS (proprietary API, no docs), hypothesis-driven MLIP error analysis, and multi-turn collaborative battery voltage calculations reproducing published figures.
  • Memory-augmented human-agent collaboration: multi-turn sessions where human corrections are stored and applied in subsequent conversations, demonstrating compounding improvement.

Results

  • DeepSolver (GPT-5): 93.3% overall success vs. 35.4% Native and 89.7% S&D baselines.
  • Best improvement: O3 model gains ~70 percentage points over its Native baseline with DeepSolver.
  • DeepSolver (GPT-4.1-mini) pass@3: 89.66% vs. Claude-Sonnet-4.5 Claude Code baseline 87.93%, at ~7.5Ă— lower input cost and ~9.4Ă— lower output cost; mean completion time 190 s vs. 239 s.
  • Average improvement over S&D: +17.53 percentage points in overall success rate across all models.
  • Ablation: self-reflection alone outperforms the combination of web search + self-debugging (S&D baseline) in most model configurations.
  • Difficult tasks (pass@3): Qwen3-Coder-30B with S&D achieves 0% on difficult tasks; DeepSolver still reaches ~50%.
  • Lab automation: CASCADE's EIS analysis code runs ~60Ă— faster than existing expert-written code and achieves R² = 0.991 vs. 0.989.

Limitations

  • DeepSolver consistently requires more wall-clock time than Native and S&D baselines due to multi-agent orchestration overhead.
  • GPT-4.1-nano and other small models show near-zero performance even with DeepSolver, indicating a backbone capability floor below which meta-skills cannot compensate.
  • Laboratory automation required a human safety check before activating physical devices; full autonomy is not yet realized.
  • SciSkillBench covers only materials science and chemistry; generalizability to other domains (biology, software engineering) is asserted but not evaluated.
  • Memory consolidation and skill sharing infrastructure is described but scalability and conflict resolution across many agents is not formally analyzed.
  • Domain-agnosticism claim rests on omitting domain-specific prompts, but the benchmark itself is domain-specific, limiting cross-domain conclusions.

Relevance to Harnesses / Meta-Harnesses

CASCADE is a direct instantiation of a meta-harness: a framework that does not solve tasks directly but orchestrates a dynamic ensemble of specialized sub-agents (Solution Researcher, Code Agent, Debug Agents, Output Processor) with runtime routing logic (SimpleSolver vs. DeepSolver) and persistent skill storage. The two meta-skills—continuous learning and self-reflection—are themselves orchestration-level abstractions that govern how sub-agents acquire, test, and codify new capabilities, making CASCADE a system that builds harness components at runtime rather than relying on pre-wired tool registries. The SciSkillBench benchmark fills a gap in harness evaluation methodology: a domain-specific, automatically scored suite for measuring autonomous tool-use competence across difficulty tiers, a pattern directly applicable to evaluating other meta-harness designs.