Sol Video Inference Engine: Agent-Native Full-Stack Acceleration Framework for Efficient Video Generation¶

🕒 Published (v1): 2026-06-21 17:23 UTC · Source: Arxiv · link

Why this paper was selected

Agent-native full-stack inference acceleration; practical for deploying harness-wrapped generative pipelines

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

Sol Video Inference Engine (Sol VIE) is an agent-native, training-free acceleration framework for video diffusion inference that treats deployment optimization as an instance-specific tuning problem. Parallel skill agents locally optimize five acceleration techniques (cache, sparse attention, token pruning, quantization, kernel fusion), an integrator agent composes them into a global stack, and a human validator gates quality. Applied to Cosmos3-Super (64B), LTX-2.3 (22B), and SANA-Video (2B), it achieves >2× end-to-end speedup with near-lossless VBench quality.

Problem¶

Acceleration recipes for video diffusion models are highly instance-specific: the optimal combination of caching policy, sparsity pattern, pruning budget, quantization scheme, and kernel fusion depends jointly on model architecture, hardware memory hierarchy, and inference configuration (resolution, duration, fps). The joint (model × hardware × config) space is too large for manual tuning, yet existing work treats each technique in isolation and assumes transferability across instances.

Method¶

Sol VIE decomposes the acceleration problem into a local-to-global agentic workflow:

Parallel skill agents each independently search one technique family's configuration space for the target deployment instance. A cache agent selects among TeaCache / EasyCache / TaylorSeer and tunes skip-step schedules; a sparse-attention agent selects a backend (PISA, SparseVideoGen, SpargeAttention) and tunes layer selection and sparsity density; a token-pruning agent tunes budgets and temporal smoothing; a kernel-fusion agent profiles per-kernel cost and targets fusions (GEMM+GELU epilogue, RoPE+norm); a quantization agent selects precision (e.g., NVFP4 on B200, FP8 elsewhere) and scope (FFN layers only).
Integrator agent receives the locally optimal candidates and performs global search over their composition, explicitly resolving inter-technique interactions (e.g., accumulated approximation error when lossy cache and lossy sparse attention are both active).
Human validator inspects a fixed validation set of generated videos and provides perceptual quality feedback, overriding pixel-level metrics (PSNR) that misalign with visual quality; this feedback drives the next optimization round.

The five technique families operate at three levels: algorithm-level (cross-step cache), model-level (sparse attention, token pruning), and kernel-level (fusion, quantization).

Key Contributions¶

Reframes video diffusion acceleration as instance-specific tuning across the (model, hardware, serving config) joint space rather than a one-shot transferable optimization.
A full-stack acceleration technique library covering cross-step cache, sparse attention, token pruning, quantization, and kernel fusion, integrated under a unified quality constraint.
An agent-native workflow with parallel skill agents + integrator agent + human-in-the-loop validator, replacing multi-team manual coordination.
Empirical demonstration across three architecturally diverse models (64B Cosmos3-Super, 22B LTX-2.3, 2B SANA-Video) showing >2× speedup with near-lossless quality.

Results¶

Cosmos3-Super (64B): cumulative stack reduces latency from baseline through diffusion cache (1.39×) → +sparse attention & token pruning (1.90×) → +kernel fusion & quantization (2.38×).
LTX-2.3 (22B): 97.8 s baseline → 58.5 s (after cache, 1.39×) → 41.0 s (+sparse/pruning, then 2.38× relative to baseline with full stack).
SANA-Video (2B): denoise 94.5 s → 52.4 s (+cache, 1.90×→ 2.27× combined → ~2.77× with kernel optimization).
All results reported as near-lossless on VBench quality metrics.
No single technique accounts for the full gain; the integrator's composition is essential.

(Note: exact per-model final speedup figures are partially conveyed via Figure 4 decomposition; paper states "roughly 2×–3× end-to-end acceleration" across the three models.)

Limitations¶

Human validator remains a bottleneck: perceptual quality feedback is not automated, requiring human inspection per candidate stack.
Only three model instances evaluated; generalization claims to other architectures (e.g., linear-attention variants beyond SANA-Video) are not empirically verified.
The paper text provided is truncated; quantitative VBench scores and ablations on individual agent contributions are not available in the excerpt.
Token pruning was found inapplicable on at least one model, indicating technique applicability is not universal and requires per-instance determination.
The integrator's global search procedure is not fully specified in the available text; scalability to larger technique libraries is unaddressed.

Relevance to Harnesses / Meta-Harnesses¶

Sol VIE is a concrete instantiation of the harness-as-runtime-substrate pattern: the agent workflow (skill agents → integrator → validator loop) is the harness, and the acceleration techniques are the pluggable tools it orchestrates. It explicitly cites AI Harness Engineering [45] and aligns with CUDA-LLM/CudaForge's iterative agentic compilation-and-validation loops, extending the pattern from code generation to multi-technique systems optimization. The local-to-global decomposition—parallel specialist agents feeding a composer agent under a human-gated quality oracle—is a reusable meta-harness architecture applicable beyond video inference to any domain where a large technique combination space must be tuned under black-box quality constraints.