Skip to content

GaP: A Graph-as-Policy Multi-Agent Self-Learning Harness For Variational Automation Tasks

🕒 Published (v1): 2026-07-06 00:00 UTC · Source: HuggingFace · link

Why this paper was selected

Berkeley robotics (Fu, Yu); graph-as-policy multi-agent self-learning harness directly builds on agent-harness architecture

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

GaP (Graph-as-Policy) is a multi-agent coding harness that represents robot policies as directed computation graphs composed of modular skill nodes, targeting "Variational Automation" (VA) tasks—commercial/industrial tasks with non-trivial object geometry and pose variation. A self-learning loop rehearses graphs in simulation, iteratively refining structure and parameters before edge-device deployment. GaP achieves success rates near 0.95–0.99 across all pose variation conditions, far outperforming VLA baselines that collapse to ≤0.20 under distribution shift.

Problem

Model-free Vision-Language-Action (VLA) policies achieve high accuracy on fixed benchmarks but fail to maintain commercial-grade reliability when object geometry and pose vary (VA tasks). Code-as-Policy (CaP) approaches suffer from hallucinations, constraint violations, and context window bloat in complex tasks, while single-agent systems lack the modularity needed for persistent, reliable automation. There is no structured framework for agentic coding agents to produce interpretable, self-improving robot programs.

Method

GaP represents a robot policy \(\pi(a \mid x, \mathcal{T})\) as an optimized directed computation graph \(G^* = (V, E)\), where nodes are typed atomic skill primitives (perception, grasp planning, motion planning) drawn from a Modular Open Robot Skill Library (MORSL) with 51 initial skills.

Graph generation (multi-agent harness): 1. An Orchestration Agent decomposes a natural-language VA task specification into functional segments. 2. Skill Agents synthesize localized subgraphs for each segment using MORSL skills. 3. The orchestrator wires subgraphs into a single executable graph \(G_0\).

Self-learning loop (Algorithm 1): Given \(M\) iterations and \(N\) parallel rollouts, GaP samples task instances \(\{\hat{s}_i\}^N_{i=1} \sim \mathcal{B}\) from the belief space, runs parallel rehearsal in NVIDIA Isaac simulator, analyzes failures for geometric root causes, and calls a Graph Update agent that modifies graph topology (node swaps, edge rewiring, parameter changes). The optimized \(G^*\) is deployed to an edge-based interpreter—no LLM required at inference.

The VA task is formalized as \(\mathcal{T} = \langle L, E, R, O, \mathcal{X}, \mathcal{B}, J \rangle\) with reward \(J = w_s \cdot \mathbf{I}(\text{success}) + w_t \cdot \Phi\) where \(\Phi\) is throughput (success rate / cycle time).

Key Contributions

  • VA task class formalization: distinguishes Variational Automation from Fixed Automation and Generalist Robotics, with formal tuple definition.
  • 8 open VA benchmarks: 4 sim + 4 real (Fulfill Grocery Orders, Pack Grocery Items, Make Popcorn, Insert USB-C Cables, Wash Crates) with explicit pose variation conditions.
  • Graph-as-Policy representation: directed computation graphs with typed input/output nodes as the robot policy, enabling modularity, interpretability, and edge deployment.
  • MORSL: 51-skill open library covering perception (SAM2, Grounding DINO, OWL-ViT), grasp planning (GraspGen, Contact GraspNet), motion planning (cuRobo), and utilities.
  • Simulation-based self-learning: rehearsal loop that refines graph structure and parameters autonomously before real-world deployment.
  • Harness design for multi-agent robustness: separation of graph generation from graph testing reduces incentives for agents to "cheat" evaluation metrics.

Results

  • GaP vs. VLA baselines (Benchmark I-a, Fulfill Grocery Orders, 100 instances/cell):
  • \(\pi_{0.5}\): 0.97 (LIBERO no variation) → 0.20 (mixed all); MolmoAct2: 0.22 → 0.20
  • GaP: 0.95 across all variation conditions, 0.97 on mixed all
  • Benchmark II-a, Pack Grocery Items:
  • Baselines: 0.17–0.46; GaP: 0.98–0.99
  • GaP + VLA hybrids (GaP centers wrist camera, then hands off to VLA): improve baselines substantially (e.g., MolmoAct2 w/GaP: 0.66 mixed all vs. 0.20 standalone) but still below GaP alone.
  • TipTop (TAMP baseline): 0.22–0.31 across conditions.
  • CaP-X (single agent, no self-learning): 0.07–0.22 under VA conditions; 0.96 only on LIBERO fixed.
  • Benchmarks IV and V (Insert Cables, Wash Crates): performance data reported but numeric table not included in the provided excerpt.

Limitations

  • Assumes a known, stationary workcell, bounded object set, and calibrated sensors—does not generalize to open-world environments (by design, but a hard constraint).
  • Self-learning relies on simulation fidelity; sim-to-real gaps in perception (e.g., lighting, texture) are not fully addressed.
  • MORSL is hand-curated (51 skills); novel task types may require adding skills before GaP can operate.
  • Self-learning disabled for Benchmarks I and II because the initial graph already performs well—robustness of the learning loop is demonstrated on more complex tasks only.
  • LLM used (Gemini-3.1-Flash-Lite) is a relatively lightweight model; scaling behavior with stronger models is not systematically studied.
  • No ablation of the number of self-learning iterations \(M\) or parallel rollouts \(N\) is presented in the excerpt.

Relevance to Agentic AI / LLM Agents

GaP is a concrete instantiation of structured agentic coding, directly addressing known failure modes of freeform LLM-generated code (hallucination, context window overflow, metric gaming) by imposing a graph schema and separating code generation from code evaluation. The hierarchical multi-agent harness—Orchestration Agent + Skill Agents—demonstrates how role specialization and constrained output formats can make LLM agents reliable enough for physical-world deployment. The self-learning loop (generate → simulate → analyze failure → update graph) is a robotic analogue of the broader "agent-driven iterative refinement" paradigm seen in software engineering agents, providing a grounded empirical case for LLM-based self-improvement beyond text benchmarks. The MORSL skill library and graph-based policy representation offer a reusable architecture pattern for any domain where agentic tool-use must compose modular functions reliably.