Skip to content

ASPIRE: Agentic /Skills Discovery for Robotics

🕒 Published (v1): 2026-06-30 00:00 UTC · Source: HuggingFace · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

ASPIRE (Agentic Skill Programming through Iterative Robot Exploration) is a continual learning system that autonomously writes, debugs, and refines robot control programs in a code-as-policy paradigm while distilling validated repairs into a growing, reusable skill library. Unlike prior systems that discard per-task experience, ASPIRE compounds debugging knowledge across tasks, achieving up to 77% improvement over the strongest coding-agent baseline on manipulation benchmarks and 31% zero-shot success on long-horizon tasks where prior methods reach only 4%.

Problem

Existing robotic coding agents operate in impoverished execution environments that return only coarse task-level success/failure signals, giving agents no way to attribute failures to specific sub-components (perception, motion planning, grasping, contact dynamics). Additionally, validated repairs from one task are discarded rather than accumulated, so an agent solving its hundredth task has no more prior knowledge than on its first.

Method

ASPIRE has three interlocking components operating in an open-ended learning loop:

  1. Robot Execution Engine: Replaces rollout-level feedback with per-primitive multimodal traces. For each API call (perception, navigation, grasping, planning), the engine logs invoked API, inputs, outputs, return status, RGB keyframes, grasp candidates, object poses, and motion-planning results. Agents selectively inspect traces to localize failure at the sub-primitive level.

  2. Skill Library: A coordinator–actor architecture manages a shared library. Actor coding agents diagnose failures, patch programs, and validate fixes; the coordinator audits reported patterns and admits only reusable, API-policy-compliant repairs. Each skill is stored as compact in-context guidance: failure signature, when-to-apply condition, repair strategy, and an optional code sketch.

  3. Evolutionary Search: To escape local repair loops, ASPIRE proposes a population of \(K\) candidate programs \(\{P_i^k\}_{k=1}^K\) each round, executes them against debug configurations \(S_\text{dbg}\), conditions the next round on the top-3 survivors plus residual failure traces, and terminates when a candidate exceeds threshold \(\theta\) or the budget \((K, T)\) is exhausted. Validated repairs from search are admitted into the skill library post-validation on \(S_\text{val}\).

Key Contributions

  • Per-primitive multimodal trace infrastructure that enables failure attribution at the sub-component level without flooding the agent with raw video.
  • A coordinator-actor architecture that distills transferable repair patterns into a shared in-context skill library, enabling compounding experience across heterogeneous tasks.
  • An evolutionary search procedure over robot programs that escapes local repair loops by maintaining a population of diverse candidate solutions.
  • Empirical demonstration of zero-shot cross-task transfer: skills accumulated on LIBERO-90 transfer to unseen LIBERO-Pro Long tasks with success scaling monotonically with library size.
  • Cross-embodiment sim-to-real transfer: simulation-discovered skills used as in-context guidance for a real bimanual YAM robot (different embodiment and API), substantially reducing token cost and enabling success where naive debugging fails.

Results

  • LIBERO-Pro (10 tasks Ă— 50 held-out seeds, object/goal/spatial perturbation suites): ASPIRE gains +77% on Object, +41.5% on Goal, +42.5% on Spatial over the strongest prior baseline (CaP-Agent0); matches or exceeds human-expert programs on several suites.
  • Robosuite bimanual handover: 20% → 92% (+72 points over CaP-Agent0).
  • BEHAVIOR-1K long-horizon mobile manipulation (navigate-and-pick-up tasks): navigate-and-pick-up-radio task success 56% → 88% (+32 points).
  • Zero-shot transfer (LIBERO-Pro Long): ASPIRE with full \(N=90\) library achieves 23% (position) and 38% (task) perturbation success; CaP-Agent0 and \(\pi_{0.5}\) reach ≤4% on the task axis.
  • Real-robot skill transfer: For open/push-drawer, adding sim-discovered skills reduces total tokens from 334.9M to 81.7M and lifts success from 0/20 to 11/20; for lift-soda-can, from 61.9M to 6.6M tokens with 13/20 → 19/20.

Limitations

  • Evaluation at inference time uses Claude Opus 4.6 with a 1M-token context window; the system is expensive to run and not evaluated with smaller or locally-deployable models.
  • Real-robot skill transfer is demonstrated on only three skills on one bimanual platform; broader cross-embodiment generality is not established.
  • Evolutionary search requires multiple parallel robot executions per round; compute and wall-clock cost at scale are not characterized.
  • The skill library taxonomy emerges inductively; there is no formal analysis of library size scaling laws or of when skills should be retired or merged.
  • BEHAVIOR-1K evaluation uses incremental block execution (next code block generated from current trace), which differs from the single-program evaluation on other benchmarks—making direct cross-benchmark comparisons of the evaluation protocol non-trivial.

Relevance to Agentic AI / LLM Agents

ASPIRE instantiates a concrete architecture for agentic continual learning: rather than relying on test-time reasoning alone, the system accumulates procedural knowledge across episodes into a persistent, retrievable skill library—a direct analogue of tool/memory libraries in LLM agent systems. The coordinator–actor pattern mirrors multi-agent orchestration designs where a central authority manages shared state and dispatches specialized workers, a pattern increasingly central to production agentic pipelines. The evolutionary search over programs with closed-loop execution feedback is a principled alternative to single-pass chain-of-thought repair, relevant to any domain where agents must debug generated code against real environments. The zero-shot and sim-to-real transfer results provide evidence that in-context skill retrieval can substitute for task-specific fine-tuning, which has implications for agentic scaffolds in software engineering, tool use, and embodied settings alike.