Skip to content

Crafter: A Multi-Agent Harness for Editable Scientific Figure Generation from Diverse Inputs

🕒 Published (v1): 2026-05-28 00:00 UTC · Source: HuggingFace · link

Why this paper was selected

Multi-agent harness for editable scientific figure generation from diverse inputs

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

CRAFTER is a multi-agent harness that wraps any image-generation backend with structured planning, critique, and revision loops to produce publication-quality scientific figures across diverse figure types and input conditions. A companion system, CRAFTEDITOR, applies the same harness pattern to convert raster outputs into editable SVGs. Together they form the first end-to-end generation-to-editing pipeline for scientific figures.

Problem

Existing scientific figure generators are narrow: each targets a single figure type (methodology diagrams, posters, etc.) under text-only input, and produces static raster outputs that cannot be locally edited. Iterative free-text repair degrades quality due to contradictory prompt accumulation, and stronger generator backbones do not fix structured-layout failures such as garbled labels or misaligned connectors.

Method

The core abstraction is a harness loop over a shared evolving specification \(S\), formalized as four roles iterated per round \(t\):

\[p_t = D(\text{input}, S_{t-1}), \quad a_t = E(p_t), \quad d_t = V(a_t, \text{input}, S_{t-1}), \quad S_t = R(d_t, S_{t-1})\]

where \(D\) is a designer/planner, \(E\) is a pluggable executor (image generator or SVG code generator), \(V\) is a directive critic emitting per-dimension diagnostics rather than scalar scores, and \(R\) writes typed edits (structured operations: add layout constraint, ban artifact category, resize named element) into \(S\) rather than appending free text. Three mechanisms specialize the harness for CRAFTER: (1) diversity-driven plan exploration — \(D\) proposes \(K\) candidate visual plans in parallel, the best is selected before refinement begins; (2) structured corrective layer — \(R\) converts diagnostics into typed edits on \(S\), keeping the specification internally consistent; (3) verify-then-refine with directive critic — up to \(T=3\) refinement rounds with best-so-far reversion for non-monotonic regressions.

CRAFTEDITOR instantiates the same four-role loop for raster-to-SVG conversion via three sequential phases: (a) instruction-driven canvas cleaning (VLM designer + instructable image editor + verifier, \(T=3\)); (b) per-element captioning/grounding/classification; (c) iterative SVG assembly with a hybrid critic combining a VLM for global layout fidelity and programmatic checkers for text overflow, arrow-endpoint accuracy, and element overlap (\(T=4\)).

CRAFTBENCH is a 279-sample benchmark spanning three figure types (academic, poster, infographic) and four input conditions (text-to-image, mask-completion, key-element composition, sketch-conditioned), curated from arXiv, conference posters, and research blogs with unanimous graduate-annotator agreement on all reference-conditioned samples. Evaluation uses a per-image (non-pairwise) Gemini 3.5 Flash judge scoring task-specific aspects, reporting win-rate against human-drawn targets.

Key Contributions

  • A unified harness abstraction with formal four-role loop (Designer, Executor, Verifier, Reviser) instantiated in CRAFTER (figure generation) and CRAFTEDITOR (raster-to-SVG), forming the first end-to-end generation-to-editing pipeline for scientific figures.
  • Three harness mechanisms: diversity-driven parallel plan exploration, structured corrective layer (typed edits vs. free-text accumulation), and directive critic with verify-then-refine loop.
  • CRAFTBENCH: 279-sample benchmark covering 3 figure types × 4 input conditions with human quality annotation and a bias-reduced VLM-as-judge protocol.
  • CRAFTEDITOR's hybrid critic combining VLM + programmatic checkers for SVG structural validation.

Results

  • On PaperBanana-Bench, CRAFTER (w/ Nano Banana 2) scores 64.21% overall vs. 61.68% for the strongest agentic baseline PaperBanana (+2.53 pt); gap over standalone Nano Banana 2 is +16.26 pt overall (up to +41.43 pt on conciseness).
  • On CRAFTBENCH overall, CRAFTER (w/ Nano Banana 2) scores 50.20% vs. 28.00% for PaperBanana (+22.20 pt); PaperBanana's gain over its backbone collapses from 22.60 pt on PaperBanana-Bench to 8.10 pt on CRAFTBENCH, and it falls below its backbone on the sketch task.
  • CRAFTER is the only method that improves over its backbone uniformly across every quality dimension and every task on both benchmarks.
  • Ablations: removing any single component causes a 5.04–8.90 point drop.
  • Backbone swap (Nano Banana 2 → Pro) shifts CRAFTER's overall score by only 0.34 pt (PaperBanana-Bench) and 2.10 pt (CRAFTBENCH), confirming executor-agnosticism.
  • Open-source standalone generators (GLM-Image, Qwen-Image) score 0.00% overall on both benchmarks; GPT-Image-2 returns valid outputs for only 260/279 CRAFTBENCH inputs.

Limitations

  • CRAFTER is not uniformly successful (failure analysis deferred to appendices K/L, not reproduced in the provided text).
  • Evaluation relies on VLM judges (Gemini 3.5 Flash), introducing potential model-specific biases despite the non-pairwise redesign.
  • CRAFTBENCH's reference-conditioned tasks (mask, sketch, key-element) are drawn entirely from academic figures, limiting poster/infographic coverage for those conditions.
  • The \(K\) candidate plans and \(T\) refinement rounds introduce multiplicative inference cost; adaptive \(K\) is mentioned but cost analysis is not provided in the excerpt.
  • The harness depends on an "instructable image editor" for canvas cleaning in CRAFTEDITOR; reliability of this component under diverse poster layouts is unquantified.

Relevance to Harnesses / Meta-Harnesses

This paper is a direct instantiation and formal articulation of the harness pattern as an engineering primitive: it explicitly cites and extends [Young 2025, Pan et al. 2026, Si et al. 2026] as the harness lineage, and frames its central argument as "what is needed is not a better generator but a harness." The four-role loop (Designer, Executor, Verifier, Reviser) with a shared evolving structured specification is a reusable abstraction that generalizes across both generation (CRAFTER) and post-processing (CRAFTEDITOR) tasks without architectural change — a concrete demonstration that harness design subsumes task-specific tuning. The typed-edit structured corrective layer directly addresses a known failure mode in iterative LLM harnesses (free-text prompt accumulation causing contradictions), offering a transferable design pattern. CRAFTBENCH and the executor-swap experiments provide empirical evidence that harness design quality dominates executor quality, a key hypothesis for meta-harness research.