Personalization as Inverse Planning: Learning Latent Design Intents for Agentic Slide Generation via Structural Denoising¶

🕒 Published (v1): 2026-07-01 00:00 UTC · Source: HuggingFace · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

Spire frames page-level slide personalization (PSP) as an inverse planning problem where a latent design intent must be inferred from reference slides rather than explicit templates or verbose instructions. It trains a critic and a planner jointly via structural denoising—corrupting slides at the element level and using RL to recover them—bypassing the need to differentiate through a black-box rendering executor. A 7B-parameter Spire model outperforms GPT-based baselines on both in-distribution and OOD slide benchmarks.

Problem¶

Existing agentic slide-generation pipelines handle page-level layout and styling passively: they either follow rigid templates or demand long explicit user instructions, failing to infer the user's latent design intent from a small set of reference slides. Formal optimization of the rendering likelihood $p(s^*\mid z)$ is intractable because the executor (e.g., a coding agent writing python-pptx) is a non-differentiable black box, and pixel-level image similarity is an unreliable proxy for discrete structural slide quality.

Method¶

Spire (Structural Planning via Inverse REconstruction) converts the intractable PSP objective into a verifiable structural denoising task.

Perturbation process. Given a gold slide $s^*$ with $N$ visual elements $\{e_i\}$, element-level actions are sampled independently: $$q(\mathcal{A}\mid s^*) = \prod_{i=1}^{N} q_{\mathcal{T}(e_i)}(a_i \mid e_i)$$ Perturbations operate along three axes—spatial layout (coordinate shifts), visual hierarchy (size distortions), and stylistic coherence (color alterations)—producing a corrupted slide $\tilde{s}$ and a ground-truth discrepancy list $\mathcal{A}_\text{diff}$.

Critic (Structural Discrimination). A Qwen2.5-VL-7B critic $C_\phi$ observes $(\tilde{s}, x, \mathcal{D}_\text{ref})$ and generates element-level critique $c$. It is rewarded by element-level verification accuracy $R_\text{vfy}(c,\mathcal{A}) = \frac{1}{N}\sum_i y_i$ where $y_i \in \{0,1\}$ is judged by GPT-4o-mini against the recorded action. The critic is trained with DAPO.

Planner (Structural Planning). A Qwen2.5-VL-7B planner $\pi_\theta$ generates or revises an executable plan $z$ conditioned on $(x, \mathcal{D}_\text{ref}, c\text{ or }\emptyset, \tilde{z}\text{ or }\emptyset)$. The reward $R_\text{pvm}(z,s^*,\tilde{s})$ is a plan–visual matching score from Claude Opus 4.5 asked to pick which visual better matches $z$, with presentation order swapped to mitigate positional bias. Oracle plans and critiques for training are generated by GPT-5.

At inference, the planner proposes a plan, the critic iteratively refines it, and a black-box coding executor (GPT-o4-mini + python-pptx) renders the final slide.

Key Contributions¶

Formal probabilistic formulation of PSP as latent-variable inverse planning, decoupling the design intent from the specific executor.
Spire: a structural denoising framework that creates scalable self-supervised training data by controlled element-level slide corruption, eliminating need for manual plan annotations.
Theorem 1 (Surrogate Validity): Spire's gradient approximates the true PSP gradient up to an explicit error bound $\|\nabla_\theta \hat{\mathcal{J}}_\text{SP}(\theta,\phi) - \frac{\beta}{4}\nabla_\theta \mathcal{J}(\theta)\| \le \epsilon_\text{tot}$.
Theorem 2 (Variance Reduction): Two-agent decomposition satisfies $\mathrm{Var}(\hat{g}_\text{2a}) \le \mathrm{Var}(\hat{g}_\text{e2e})$, with strict inequality when executor noise is nonzero.
7B-level models trained with Spire surpass GPT-o4-mini-based and GPT-based pipeline baselines.

Results¶

All comparisons use visual similarity (SSIM, CLIP) and VLM-as-Judge (faithfulness, color, layout, aesthetics). Backbone is Qwen2.5-VL-7B vs. GPT-based baselines.

In-distribution test pages (visual similarity AVG / VLM-judge AVG): - Spire (7B): 0.7576 / 0.7333 - PSP (o4-mini): 0.8062 / 0.4784 — higher pixel similarity but far weaker on judge metrics - AutoPresent [GPT-based]: 0.5247 / 0.5069 - PPTAgent [GPT-based]: 0.5309 / 0.3758 - PSP (Base, no fine-tuning): 0.3448 / 0.1937

OOD pages (SlideBench): - Spire achieves 0.6565 VLM-judge AVG vs. AutoPresent 0.5201 and PPTAgent 0.3831. - PSP (o4-mini) was not evaluated on OOD (results absent in paper text).

Spire is the only 7B-level model to consistently beat all GPT-level baselines on judge metrics across both test splits.

Limitations¶

Structural perturbations operate on known element types (text box, image, shape, chart, table); element parsing errors or unusual slide formats may degrade training signal quality.
Oracle data (suboptimal plan $\tilde{z}$ and gold critique $c^*$) are generated by GPT-5, introducing a dependency on a proprietary frontier model during training.
Evaluation relies on VLM-as-Judge with GPT-4o-mini and Claude Opus 4.5, which may embed their own aesthetic biases into reward signals.
The executor (GPT-o4-mini + python-pptx) is fixed; generalization to other executors (Beamer, image generators) is claimed theoretically but not empirically validated.
Train/test split is from a single source (Zenodo10k, 200 decks); scale of personalization corpus is limited.

Relevance to Agentic AI / LLM Agents¶

Spire is a concrete instantiation of multi-agent RL for a creative, tool-using domain: a critic and planner co-train without ever back-propagating through the downstream tool (the slide renderer), which is the standard situation for LLM agent pipelines that call external APIs or code interpreters. The theoretical results—surrogate validity and variance reduction from agent decomposition—generalize beyond slides and speak directly to the challenge of training agentic systems where the environment is a non-differentiable black box. The structural denoising paradigm offers a blueprint for generating scalable self-supervised reward signals in any domain with discrete, inspectable artifacts (code, structured documents, UI layouts), sidestepping the need for human preference annotations.