Skip to content

P2P: Automated Paper-to-Poster Generation and Fine-Grained Benchmark

🕒 Published (v1): 2026-01-01 · Source: ICLR · Venue: ICLR 2026 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

P2P is a three-agent LLM pipeline that converts PDF research papers into HTML/CSS academic posters via sequential specialised agents, each guarded by a checker-reflection loop. The authors simultaneously release P2P EVAL (1,738 human-curated checklist items, 121 paper-poster pairs) with a dual objective-fidelity/subjective-quality evaluation framework, and P2P INSTRUCT (30,460 instruction–response pairs) for fine-tuning. Evaluation across 35 models shows closed-source MLLMs outperform open-source ones on both fine-grained fidelity and universal aesthetic scoring.

Problem

Manual poster creation is time-consuming and requires simultaneous content distillation, visual-textual integration, and 2-D layout design. Existing template/rule-based approaches cannot capture semantic richness; no standardised benchmark exists that jointly measures scientific fidelity (objective) and aesthetic holistic quality (subjective), making systematic comparison of poster-generation systems impossible.

Method

P2P is a model-agnostic, sequential multi-agent orchestration pipeline formalised as \(P = A_{\text{Orch}}(A_{\text{Sec}}(D, F), F)\) where \(F = A_{\text{Fig}}(D)\):

  1. Figure Agent — uses DocLayout-YOLO for layout detection to extract figures/tables, then an MLLM (Figure Describer) to generate captions; produces \(F_d = \{(v_i, c_i, \text{desc}_i)\}\). A Figure Checker validates dedup, completeness, and pairing via an adaptive confidence threshold.
  2. Section Agent — infers a JSON structural schema \(S\) from the paper, then generates poster text \(P_{\text{poster\_text}} = M_{\text{text}}(D, S, F_d)\) with Markdown-style figure references embedded at contextually optimal positions. A Section Checker validates coherence, completeness, faithfulness, and reference correctness; failures trigger a reflection-revision loop.
  3. Orchestrate Agent — the HTML Generator renders \(P_{\text{poster\_text}}\) + raw visuals into HTML/CSS following three principles: content-structure decoupling (modular CSS), institutional identity alignment, and responsive CSS-flexbox layout. A Poster Checker triggers iterative layout adjustments.

P2P EVAL provides two evaluation tracks: (a) Fine-Grained: LLM-as-a-Judge (GPT-4o) scores each human-annotated checklist item; the normalised score is \(S_{\text{fine}} = \frac{\sum_{i=1}^n s_i}{\sum_{i=1}^n M_i} \times 100\); (b) Universal: an LLM scores 10 design criteria \(U_1\)–\(U_{10}\); an XGBoost model (200 trees, 10-fold CV, \(R^2 = 0.92\)) maps these to a holistic 0–50 score matching human ratings.

Key Contributions

  • P2P multi-agent framework with per-agent checker-reflection loops, producing HTML-rendered posters from raw PDFs.
  • P2P INSTRUCT: 30,460 instruction–response pairs (16,848 figure-description + 13,612 text-generation) derived from P2P pipeline intermediates, averaging >3,300 tokens/response.
  • P2P EVAL: 121 paper-poster pairs (ACL 2022–2024 + SciPostLayout), 1,738 human-annotated checklist items (avg. importance 4.07/5), with a dual Fine-Grained + Universal evaluation framework.
  • XGBoost holistic scorer achieving \(R^2 = 0.92\) on 1,701 human ratings, bridging LLM criterion scoring and human aesthetic judgment.
  • Comparative benchmark across 35 LLMs/MLLMs plus three fine-tuned P2P-INSTRUCT models (Qwen3-P2P, Qwen2.5-VL-P2P, InternVL3-P2P).

Results

  • P2P (Claude-3.7-Sonnet backbone) achieves Fine-Grained score 65.40 and Universal score 37.25, the top result among all tested configurations.
  • Claude-3.7-Sonnet with reasoning scores Fine-Grained 65.88 / Universal 35.51, slightly higher fidelity but lower universal.
  • Seed-Thinking-v1.5-RT (reasoning mode): Fine-Grained 61.96, Universal 34.69.
  • GPT-4o: Fine-Grained 55.44, Universal 34.39.
  • Deepseek-R1 (MoE): Fine-Grained 62.50, Universal 33.97.
  • InternVL3-P2P (8B fine-tuned on P2P INSTRUCT): Fine-Grained 51.97, Universal 31.62 — competitive vs. much larger open-source models.
  • Output format ablation: HTML (65.40 / 37.25) > LaTeX (56.88 / 25.26) > SVG (52.74 / 30.66) on Fine-Grained/Universal respectively.
  • Human pairwise preference: P2P preferred or tied vs. YuanBao (Tencent) in 83.05% of cases; P2P preferred or tied vs. original author posters in 57.63%.

Limitations

  • Evaluation set is limited to 121 pairs, concentrated in ACL/NLP (Computer Science: 49 pairs) with sparse coverage of other domains.
  • Fine-Grained evaluation relies on GPT-4o as judge, inheriting its biases and cost; the checklist is static ground-truth from official posters, not necessarily the optimal poster for a given paper.
  • P2P INSTRUCT construction uses Claude as Figure Describer even when the backbone model is a different provider, introducing distribution mismatch.
  • HTML rendering quality depends on browser/renderer; cross-platform visual fidelity is not evaluated.
  • The XGBoost Universal scorer is trained on 1,701 ratings from a narrow annotator pool, limiting generalisation of aesthetic preferences across cultures and fields.

Relevance to Harnesses / Meta-Harnesses

P2P is a concrete instantiation of a document-transformation meta-harness: a model-agnostic orchestration layer that chains specialised sub-agents (Figure, Section, Orchestrate) with per-stage checker-reflection loops, analogous to how research harnesses chain fetch → digest → verify → publish stages. The checker-reflection pattern—where each stage emits output, a dedicated validator assesses it against criteria, and failure triggers a revision loop—is directly transferable to any multi-stage automated pipeline and mirrors quality-gate idioms common in paper-digest harnesses. Additionally, P2P INSTRUCT's derivation from pipeline intermediates demonstrates how a harness can self-generate fine-tuning data as a byproduct of normal operation, a technique relevant to self-improving meta-harnesses. The dual-evaluation framework (objective checklist fidelity + learned holistic scorer) offers a reusable blueprint for evaluating complex harness outputs where no single metric suffices.