Skip to content

Bridging Creative Intent and Visual Quality: Creator-Driven Recurrent Video Generation with Agentic Feedback Loops

🕒 Published (v1): 2026-06-17 01:39 UTC · Source: Arxiv · Venue: ICML 2026 · link

Why this paper was selected

ICML 2026 acceptance; agentic feedback loop for coherent long-form video generation

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

CHIEF is a human-AI co-creation framework for iterative video generation that replaces autonomous closed-loop refinement with creator-gated iteration supported by persona-conditioned multimodal LLM feedback agents. It addresses the collapse of narrative coherence and creative direction in AI-generated long-form video by placing the creator at the center of the refinement loop. Evaluated via a student-produced 10-minute short film, it achieves a live-audience rating of 4.1/5 vs. 2.4/5 for the unrefined baseline.

Problem

Existing AI video generation frameworks inherit an autonomous closed-loop structure from coding AI (e.g., self-refinement via test signals), which produces videos with high visual fidelity but poor narrative coherence and creative direction — problems that worsen severely at longer durations (>1 minute). Feedback signals in prior work aggregate generic preferences or use narrow reward models, failing to capture real viewer sentiment or enable genuine creator expression.

Method

CHIEF is a modular agentic pipeline with three components:

  1. Video Generator: A divide-and-conquer approach splits a creator-written script into 8-second clip descriptions. A Prompt Orchestrator generates keyframe prompts (text-to-image via Imagen 4.0), clip prompts (video interpolation via Veo 3.1), and music prompts (ElevenLabs Music v1), anchoring character/environment consistency across all clips. Keyframes serve as lightweight proxies for full video, enabling early creator assessment without full generation cost.

  2. Feedback Agents: Multimodal LLM agents (Gemini 2.5 Flash) conditioned on synthetic personas watch generated keyframes/clips and produce subjective critique. Personas are generated from real YouTube comment histories (30 comments per user) via a Self-Refine loop. Two agent categories operate: audience-persona agents (grounded artifact/coherence critique) and movie-critic agents (narrative structure, cinematic quality). Feedback generation is parallelizable across personas.

  3. Feedback Translator: Decomposes raw agent output into structured issue tuples \((description, category, urgency)\) where \(category \in \{\text{narrative, pacing, character, visual, technical}\}\) and \(urgency \in \{\text{low, medium, high}\}\). Issues are aggregated per category, ranked by support count, and surfaced to the creator. A Refiner translates creator natural-language revisions into updated prompts; a Planner sub-component (autonomous mode only) outputs a two-stage structural-then-stylistic refinement plan.

Two operating modes: Autonomous Refinement (closed-loop for short clips, creator monitors) and Creator-Driven Film Generation (creator gates every regeneration decision, with prior keyframe as visual anchor for consistency across long arcs).

Key Contributions

  • A human-in-the-loop iterative video generation framework (CHIEF) that positions the creator as the primary driver of narrative direction, not the automation.
  • Persona-conditioned multimodal feedback agents that simulate real audience sentiment (YouTube comments → personas via Self-Refine; Rotten Tomatoes/The Guardian → critic personas), producing diverse subjective critique unavailable from single-model self-evaluation.
  • Demonstration of end-to-end long-form AI video creation (10-minute short film) by non-expert high school students, with live audience evaluation.
  • A structured issue-extraction and ranking pipeline (tuples with urgency scores) that prevents feedback summarization from losing high-priority detail.

Results

  • Autonomous refinement (1-minute clips, 5 iterations, 20 critic + 30 audience agents): Qualitative keyframe improvements including artifact removal (floating objects, erroneous flashlight), crowd density correction, and motion blur addition; no quantitative benchmark reported (authors explicitly decline VBench/VBench 2.0/VideoReward-based evaluation as misaligned with the co-creation setting).
  • Creator-driven film (10-minute student film, live audience, \(n=20\)): CHIEF-refined version scored 4.1/5 vs. baseline 2.4/5 on a 1–5 scale; written feedback cited improved coherence and narrative flow, while continuity artifacts (character appearance, dialogue) remained.

Limitations

  • No quantitative benchmark evaluation; results rely entirely on qualitative case studies and a small live-audience rating (\(n=20\)), making direct comparison to prior video generation systems impossible.
  • Audience evaluation conducted at a single venue with a non-random convenience sample; no statistical significance reported.
  • Autonomous refinement is limited to short-form (~1-minute) video; structural changes in longer content require creator gating to avoid unintended cascading keyframe inconsistencies.
  • Prompt specificity creates a tradeoff: highly specified prompts can introduce inconsistencies elsewhere when refined; under-specified prompts are less susceptible but provide less control.
  • Persona generation relies on the availability of YouTube comment histories and external review corpora; quality of feedback is contingent on persona diversity and comment volume (30 comments per user required).
  • Cross-clip character consistency remains an open problem even with keyframe anchoring, as noted by audience feedback.

Relevance to Vision-Language Models

CHIEF demonstrates a concrete application of multimodal LLMs as agentic evaluators — Gemini 2.5 Flash watches generated keyframes/clips and produces language critique conditioned on persona context, functioning as a VLM-based reward signal in a human-AI loop. The persona-conditioned feedback architecture is a novel use of VLMs for simulating subjective human viewer sentiment rather than objective quality scoring, extending prior VLM-as-judge work into the affective and narrative domain. For researchers tracking VLMs, CHIEF's feedback agents demonstrate that VLMs can disaggregate along persona axes (critic vs. audience) and provide semantically distinct, complementary critiques — relevant to VLM alignment and multi-perspective evaluation research. The system also illustrates an emerging class of VLM-augmented creative pipelines where the model serves not as the generator but as the evaluative intermediary bridging user intent and generated output.