Scaling up Memory for Robotic Control via Experience Retrieval¶

🕒 Published (v1): 2026-01-01 · Source: ICLR · Venue: ICLR 2026 · link

Why this paper was selected

Chelsea Finn (Stanford); ICLR 2026; scaling memory for robot policies via retrieval

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

MemER is a hierarchical robot control framework that equips existing vision-language-action (VLA) models with long-term visual memory by training a VLM high-level policy to select and track task-relevant keyframes from its experience history. A fine-tuned Qwen2.5-VL-7B-Instruct manages memory and subtask planning, while a fine-tuned π0.5 handles low-level motor control. The system achieves >95% task progress on three real-world manipulation tasks requiring minutes of memory, matching a human high-level oracle.

Problem¶

Existing robot policies either operate without visual history (making partial-observability tasks impossible) or naively extend context with long frame sequences, which is computationally prohibitive at deployment and causes covariate shift because suboptimal policies generate histories diverging from expert demonstrations. Prior memory-extending methods handle at most ~24 frames; tasks requiring hundreds of frames across several minutes remain unsolved.

Method¶

MemER decomposes the policy hierarchically: \(\pi(A_t | o_{0:t}) = \pi_l(A_t | I_t, q_t, l'_t)\,\pi_h(l'_t, J_t | I_{t-N+1:t}, K_t)\).

High-level policy (\(\pi_h\)): Fine-tuned Qwen2.5-VL-7B-Instruct receives the last \(N\) recent frames \(R_t\), a task instruction \(l_t\), and up to 8 previously selected keyframes \(K_t \subseteq I_{0:t-N+1}\). It predicts the current language subtask \(l'_t\) and candidate keyframes \(J_t \subseteq R_t\) to potentially memorize.

Keyframe filter: All nominated candidate frame indices across the trajectory are pooled into a temporally ordered list. A 1D single-linkage clustering groups indices within distance \(d\) frames; the median index per cluster becomes the representative keyframe stored in \(K_{t+1}\). Duplicate nominations bias the median toward frequently nominated frames. Clusters with indices well outside the recent window need not be recomputed, keeping filtering \(O(1)\) per step.

Low-level policy (\(\pi_l\)): Fine-tuned π0.5 (initialized from DROID checkpoint) executes subtasks at ~2 Hz given the current image, joint states, and the latest \(l'_t\); requires only 50 long-horizon demonstrations per task.

Data annotation: Keyframe ground truth is generated semi-automatically by defining one rule per subtask (e.g., "keep last frame of 'look inside bin'") and applying it to all demonstrations — a one-time, per-subtask decision rather than per-frame labeling.

Deployment: \(\pi_h\) runs at ~1 Hz, \(\pi_l\) at ~2 Hz asynchronously on separate servers. Weight interpolation between the fine-tuned and base Qwen2.5-VL weights improves performance.

Key Contributions¶

Hierarchical memory framework (MemER) that adds persistent, streaming keyframe memory to any VLA without modifying the low-level policy architecture.
1D single-linkage keyframe filter that consolidates redundant nominations into representative frames with negligible inference overhead, supporting continuous streaming.
Semi-automatic keyframe annotation procedure requiring only ~3 demo reviews and one rule per subtask.
First real-world robotic benchmark requiring reasoning over hundreds of frames (~minutes), with three distinct memory types: object location recall, action counting, and state tracking.

Results¶

All numbers from 20 trials per task:

MemER vs. No History / Short History (N=8) / Long History (N=32) / Human HL on average task progress:
MemER: ~96–97% across tasks; Human HL: ~95–97%; Long History: ~52–62%; Short History: ~35–58%; No History: ~26–55%.
MemER outperforms Long History by 34% on average; Long History also incurs ~1 s inference latency, near closed-loop limit.
Object Search: MemER retrieves 59/60 objects correctly, uses optimal path 57/60 times; No History: 32 retrievals, 25 optimal paths.
Counting Scoops: MemER: 1 wrong scoop total across 20 trials; No History: 61 wrong scoops.
Dust & Replace: MemER: 18–20/20 correct per subtask; No History: 4–7/20.
Offline trajectory accuracy vs. API VLMs (Object Search / Counting / Dust & Replace): MemER 0.80 / 0.67 / 0.87; GPT-5 0.15 / 0.43 / 0.67; Gemini Robotics-ER 1.5 0.21 / 0.13 / 0.19.
Fine-tuned open-source VLM substantially outperforms proprietary API-based VLMs on memory-aware robotics tasks.

Limitations¶

Memory is purely visual; text-based memory representation was tested and performed worse (high-level policy over-indexes on text tokens, missing visual detail).
Keyframe clustering uses a fixed distance threshold \(d\); task-agnostic tuning may not generalize perfectly to all scenarios.
Low-level policy failures remain the primary failure mode at >95% task progress, suggesting a ceiling imposed by \(\pi_l\) quality and the limited 50-demo fine-tuning set.
Wrist-mounted camera requires the robot to actively move to observe objects; spatial map-based memory is inapplicable because the robot modifies the environment.
All experiments on a single DROID robot setup (Franka arm); generalization to other hardware is untested.
Cross-episodic memory (Object Search) requires the high-level policy to preserve keyframes across separately issued instructions within one episode; long episodes with many instruction changes may strain the \(|K_t| \leq 8\) keyframe budget.

Relevance to Vision-Language Models¶

MemER demonstrates a concrete recipe for adapting general-purpose video-understanding VLMs (Qwen2.5-VL-7B) to embodied, streaming settings by fine-tuning for joint subtask prediction and keyframe nomination — a novel dual-output VLM training objective. It directly benchmarks fine-tuned open-source VLMs against GPT-5 and Gemini Robotics-ER, showing that task-specific fine-tuning of 7B-scale models drastically outperforms large proprietary models when domain-specific visual reasoning (robot affordances, manipulation states) is required. The work also shows that visual tokens in memory are more informative than text summaries for grounding robot decisions, a finding relevant to the broader VLM debate on modality priority. For VLM researchers, the keyframe selection mechanism provides a lightweight, streaming-compatible alternative to full-video long-context inference, relevant to efficient VLM deployment in real-time settings.