UniDomain: Pretraining a Unified PDDL Domain from Real-World Demonstrations for Generalizable Robot Task Planning¶

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

UniDomain pre-trains a unified PDDL symbolic planning domain from 12,393 real-world robot manipulation demonstrations and applies it to zero-shot task planning. It uses VLMs to extract atomic PDDL domains from video keyframes, fuses them via a hierarchical binary-tree merging procedure, and invokes a classical planner at inference time. On 100 unseen real-world tasks it achieves 85% success rate and 83% plan optimality, outperforming the strongest hybrid LLM-PDDL baselines by up to 58% in success and 160% in plan optimality.

Problem¶

LLMs and VLMs exhibit strong commonsense priors but fail at long-horizon, constraint-aware task planning because they cannot accurately model action preconditions/effects or maintain symbolic state. Existing hybrid LLM-PDDL methods either rely on hand-crafted, narrow PDDL domains requiring expert effort or attempt to generate domains on-the-fly from language alone, yielding low-quality domains that fail to generalize across diverse manipulation tasks.

Method¶

Domain Pretraining. Each of 12,393 DROID manipulation videos is segmented into keyframes via energy-based detection (grayscale intensity extrema). A VLM proposes an initial atomic PDDL domain from consecutive keyframe pairs (operators, preconditions, effects, predicates). An LLM revises it for syntactic consistency, then a closed-loop verifier (i) generates K=5 test problems of increasing difficulty and checks PDDL solvability (threshold θ=0.6), and (ii) validates plans against physical commonsense. Up to L=5 refinement iterations are allowed; failing domains are discarded. The resulting atomic domains form a knowledge graph with 3,137 operators, 2,875 predicates, and 16,481 causal edges.

Domain Fusion. For a target task class, relevant atomic domains are retrieved via sentence-embedding similarity to LLM-inferred action names. Retrieved domains are merged bottom-up along a binary tree: predicates are aligned by cosine similarity of text embeddings (τ_p=0.3) and LLM-verified for semantic equivalence before merging; operators are similarly aligned (τ_o=0.3) and merged with union of preconditions/effects.

Online Planning. Given a new task instruction and scene image, a VLM generates an initial PDDL problem. Task-relevant predicates are extracted from it; operators whose preconditions/effects involve those predicates are selected to form a compact filtered domain. A refined problem is generated from this domain, then Fast Downward solves the (domain, problem) pair.

Key Contributions¶

First framework to pre-train a unified general-purpose PDDL domain from large-scale real-world robot demonstrations (12,393 DROID videos).
Closed-loop atomic domain generation combining VLM-based extraction, LLM revision, PDDL solvability checks, and commonsense plan verification—fully automated without human feedback.
Hierarchical binary-tree domain fusion that resolves semantic inconsistencies across atomic domains into a compact, symbolically connected meta-domain.
Task-time predicate grouping and operator filtering that reduce symbolic noise and improve both planning accuracy and LLM efficiency.

Results¶

85% success rate vs. best baseline ISR-LLM (~27% implied from "up to 58% higher") across 100 unseen tasks.
83% optimality rate at K=0 (exact optimal plans); ISR-LLM collapses to near VLM-CoT level at K=0.
160% improvement in plan optimality over strongest baselines.
UniDomain incurs the lowest LLM thinking time and fewest LLM calls among top-performing methods.
Ablation: without domain fusion, success drops from 85% to 19% (perfect syntax/solvability but no compositional generalization).
Ablation: energy-based keyframe extraction achieves 28% single-pass success vs. 15% for similarity-based (SigLIP), and is 80× faster (0.6 s vs. 47.8 s per demo).
Ablation: removing closed-loop verification (w/o CL) causes large domain quality degradation; each component (revision, solvability check, solution verification) contributes incrementally.

Limitations¶

Automatic atomic domain retrieval can return redundant domains, making meta-domain construction time-consuming.
Operates under PDDL 1.0, lacking temporal constraints, numeric fluents, and cost-sensitive planning; extension to PDDL 2.1 is left to future work.
Assumes full observability; occlusion, perceptual noise, and uncertain sensing are not handled (probabilistic planning frameworks such as PPDDL/RDDL are noted as future directions).
Evaluated as a high-level planner only, with perfect low-level control (human teleoperation); end-to-end robot system performance is not quantified in controlled experiments.

Relevance to Vision-Language Models¶

UniDomain is directly relevant to VLM practitioners because it positions VLMs as the grounding interface between raw visual observations and formal symbolic structure: a VLM analyzes keyframe pairs to propose PDDL operators and predicates, and later generates grounded PDDL problem files from scene images at inference time. This reveals both the utility and the limits of current VLMs—they succeed at local perceptual grounding but require symbolic scaffolding (PDDL + classical planners) and LLM-driven closed-loop correction to produce reliable long-horizon plans. The work is part of an emerging line that treats VLMs as perception-to-symbol translators within hybrid neuro-symbolic systems, complementing concurrent VLA work (π0, OpenVLA) by operating at the planning rather than control layer. For researchers tracking VLMs, it highlights a concrete failure mode (action precondition/effect modeling) and a data-driven, demonstration-grounded approach to compensate for it.