GroundShot: Visually Consistent Multi-Shot Long Video Generation via Entity-Grounded Shot Scheduling¶

🕒 Published (v1): 2026-06-18 18:00 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

GroundShot is a training-free, model-agnostic agentic framework that wraps any text-to-video backbone with an orchestration layer managing entity-level visual memory, quality-aware shot scheduling, and VLM-driven verification to prevent cross-shot identity drift in multi-shot long video generation. It decouples generation order from narrative order, generating shots most likely to yield clean entity references first, then conditioning later shots against those canonical references. A companion benchmark, GroundBench, evaluates consistency at the entity level across controlled challenge dimensions.

Problem¶

Existing multi-shot video generation methods maintain consistency via frame-level memories or nearby-frame conditioning in narrative order. Two compounding failures result: (1) whole-frame references entangle characters, objects, and backgrounds, forcing the generator to implicitly disentangle entities from mixed visual context; and (2) narrative-order generation may lock in a weak early view (partially occluded face, distant establishing shot) as the identity anchor, causing cumulative drift as later shots condition on an already-degraded reference.

Method¶

GroundShot is a five-stage pipeline that wraps any compatible T2V or Ref2V backbone:

1. Script Parsing. An LLM parses the full script and optional global caption $C$ to build a global entity registry (characters, objects, locations), links co-referring mentions across shots, and outputs the entity set $E_i$ for each shot $s_i$ together with a priority ordering (character > object > location).

2. Quality-Aware Shot Scheduling. For each recurring entity $e$ and each shot $s_i$ where it appears, a reference-quality estimator predicts $$q_{\text{src}}(s_i, e) = Q_{\text{ref}}(s_i, e;\, C)$$ scoring expected reference usefulness from shot text, entity description, and global caption. The highest-scoring shot per entity becomes its source shot. All source→dependent constraints are encoded as a directed graph; cycles are broken by preserving constraints from higher-priority entities with larger quality gains, then defaulting to narrative order. A topological sort of the resulting DAG yields the generation order $\pi$.

3. Entity-Level Visual Memory $\mathcal{R}$. Per entity, memory stores a protected canonical reference $r_e^*$ (first sufficiently clear appearance) and a compact set of canonical-consistent supplementary references. After each generated shot, open-vocabulary grounding localizes characters and objects as foreground crops; locations are handled by masking all foreground entities and running an instruction-following inpainting model to reconstruct the background. Candidates pass a type-specific quality threshold $\tau_{\text{type}(e)}$ before admission. Supplementary references must satisfy both a canonical-consistency gate $\text{sim}(c, r_e^*) \ge \theta_{\text{id}}$ and a diversity gate $\max_{r \in \mathcal{R}_e} \text{sim}(c, r) \le \theta_{\text{div}}$.

4. Shot Generation. At each scheduled step $t$, GroundShot retrieves references $I_i$ from $\mathcal{R}_{t-1}$ (canonical by default; supplementary appended when a VLM retriever scores one as better covering the target framing/expression), then selects T2V (no reference available) or Ref2V (references present) generation. A VLM critic verifies the output via structured issue detection (identity mismatch, missing entities, style drift, motion artifacts, etc.), using $$\text{pass}(\tilde{v}_i) = Q(\tilde{v}_i) \ge \tau_{\text{pass}} \;\wedge\; \neg\,\text{Severe}(\tilde{v}_i).$$ Failed shots trigger targeted repairs (stronger conditioning, prompt augmentation, new seed) within a fixed retry budget; the highest-scoring attempt is kept if no attempt passes.

5. Experience Accumulation. A generation experience log $\mathcal{X}$ records which generation and repair strategies succeeded per shot context, and is queried before each new shot to inform mode selection and repair choices.

Finally, generated clips are reordered back to narrative order.

Key Contributions¶

GroundShot: training-free, model-agnostic agentic meta-harness that reformulates multi-shot consistency as a reference-management problem, decoupling generation order from narrative order.
Quality-aware shot scheduling via predicted reference-quality scores, DAG construction, and cycle-aware topological sort.
Entity-level visual memory with canonical protection, diversity-gated supplementary references, type-specific quality filters, and VLM-driven grounding and verification.
GroundBench: diagnostic benchmark measuring consistency at the entity level, organized into controlled challenge modules for fine-grained failure analysis.

Results¶

GroundShot outperforms strong multi-shot baselines on GroundBench with no additional training or model modification (specific numbers are not included in the provided text excerpt).
Ablations confirm the critical role of each agentic component (scheduling, grounding, verification, memory management).

Limitations¶

Specific benchmark numbers and baseline comparisons are not present in the provided text, making quantitative assessment impossible from this excerpt.
Retry budget is finite; if no generated attempt passes verification, the highest-scoring failed attempt is used, introducing a quality floor that may still contain drift.
Cycle resolution in the scheduling DAG may sacrifice some entity constraints when preferences conflict, potentially leaving certain entities under-referenced.
Location grounding depends on inpainting quality; artifacts in reconstructed backgrounds can propagate to all shots conditioned on that location reference.
The framework's pipeline length (parse → schedule → generate → verify → ground → update memory per shot) may be slow for long multi-shot scripts.

Relevance to Harnesses / Meta-Harnesses¶

GroundShot is a canonical example of an agentic meta-harness: it wraps an unmodified base model (T2V or Ref2V) with a full orchestration layer—script parsing, DAG-based scheduling, memory CRUD, grounding, VLM verification, targeted repair, and experience logging—matching exactly the harness pattern of "loop over work items, maintain shared state, verify and repair outputs." Algorithm 1 makes the harness structure explicit and is directly reusable as a template. The framework demonstrates how quality-aware scheduling (deciding in what order to invoke a base model) and feedback-driven repair loops can substitute for architectural changes or fine-tuning—a key design principle for meta-harness builders. The separation between the orchestration layer and the backbone also illustrates the model-agnostic composition pattern central to the meta-harness literature.