From EduVisBench to EduVisAgent: A Benchmark and Multi-Agent Framework for Reasoning-Driven Pedagogical Visualization¶

🕒 Published (v1): 2026-01-01 · Source: ICLR · Venue: ICLR 2026 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

EduVisBench is a 1,154-question STEM benchmark evaluating foundation models on pedagogically grounded visual explanation generation, scored across five dimensions (0–100 scale). EduVisAgent is a five-agent orchestration framework designed to close the observed gap, achieving 81.6% average vs. 58.2% for the strongest single-model baseline (v0), a 40.2% relative improvement. The core finding is that decomposing pedagogical reasoning into specialized agent roles is necessary because no single model handles the full task adequately.

Problem¶

Foundation models—including state-of-the-art LVLMs and diffusion models—fail to produce pedagogically effective step-by-step visual explanations for STEM problems. Existing benchmarks focus on textual reasoning; there is no systematic evaluation framework for the quality of AI-generated instructional visualizations aligned with human cognitive processes. Single-model approaches score below 50/100 on average, exhibiting semantic misalignment between text and visuals, omitted reasoning steps, and structural code inconsistencies.

Method¶

EduVisBench curates 1,154 STEM questions (Math, Physics, Chemistry; easy/medium/hard) from public datasets (C-MHChem-Benchmark, MATH-500, Illustrative Mathematics, high-school-physics). Models generate visualizations (SVG, HTML/Next.js, or images); outputs are canonicalized to raster images via headless browser rendering. GPT-4o scores each output across five dimensions—Context Visualization, Diagram Design, Text–Graphic Integration, Thought Guidance, Interactivity—on 0–5 scales, summed to 25 and normalized to 100%.

EduVisAgent operates in two stages: 1. Instructional Flow Construction: A Task Planning Agent decomposes the problem into subgoals, clarifies expected reasoning per step, aligns steps to domain principles, and anticipates misconceptions. 2. Collaborative Solution Generation: Four downstream agents execute sequentially— - Conceptual Mapping Agent: classifies problem components via the CRA (Concrete–Representational–Abstract) model. - Reasoning Decomposition Agent: applies the FOPS strategy (Find type, Organize, Plan, Solve) to generate step-specific instructional guidance. - Metacognitive Reviewer: generates reflective prompts grounded in metacognitive theory to support self-correction. - Visualization Agent: produces abstract pedagogical visuals (number lines, schematics, interactive sliders) rendered via Vercel v0. A synthesis module integrates all agent outputs into an interactive learning webpage.

Key Contributions¶

EduVisBench: first multi-domain, multi-level benchmark (1,154 STEM Qs, 15 sub-domains, 3 difficulty levels) specifically targeting AI-generated pedagogical visualization quality.
Five-dimensional rubric grounded in pedagogical theory for assessing visualization correctness, clarity, cognitive alignment, reasoning support, and interactivity.
GPT-4o as automated judge validated against human raters (cosine similarity 0.9655, MSE 0.5702 across subjects).
EduVisAgent: modular five-agent pipeline embedding CRA, FOPS, and metacognitive theory directly into agent roles.
Systematic benchmarking of diffusion models, LVLMs, and a specialized agent (v0) establishing strong baselines.

Results¶

EduVisAgent: 81.6% average across all subjects and difficulty levels.
Best single-model baseline (v0): 58.2% → EduVisAgent improves by +23.4 pp (40.2% relative).
Best LVLM: Claude 3.7 Sonnet (Webpage format) at 54.6%.
GPT-4o (Webpage): 38.1%; (SVG): 26.3%.
Gemini 2.0 Flash: 43.6%.
Diffusion models: Flux.1-dev 13.8%, SD3.5 21.5%, SDXL 21.8% — uniformly lowest.
Most LVLMs score 17.5%–30.2%; Webpage generation consistently outperforms SVG for the same model.
Human–GPT judge agreement: cosine similarity 0.9655, MSE 0.5702 (average across Chemistry, Math, Physics).

Limitations¶

EduVisBench is limited to three STEM subjects (Math, Physics, Chemistry); generalization to humanities, social sciences, or non-STEM domains is untested.
GPT-4o as judge introduces potential model-specific bias despite human validation on only 150 samples (50 per subject).
Evaluation canoncializes interactive webpages to static screenshots, which may underrepresent interactivity quality.
EduVisAgent relies on Vercel v0 for rendering, creating an external infrastructure dependency not reproducible without that service.
No ablation on the contribution of individual agents is presented in the excerpted text, making it unclear which components drive the largest gains.
The benchmark is derived from pre-existing datasets not designed for this task; difficulty levels inherited from sources may not calibrate consistently across subjects.

Relevance to Harnesses / Meta-Harnesses¶

EduVisAgent is a textbook example of a meta-harness: a central orchestrator (Task Planning Agent) decomposes a complex task and routes work to specialized sub-agents, each instantiating a distinct cognitive theory (CRA, FOPS, metacognitive regulation), with a synthesis module integrating outputs into a final artifact. The 40.2% relative gain over the best single-model baseline quantifies the meta-harness premium—the same underlying models, reorganized into an orchestration layer, dramatically outperform their solo performance. For researchers building harnesses, this paper provides a domain-specific case study on how embedding structured domain knowledge (pedagogical theory) directly into agent role definitions, rather than leaving it to prompt engineering of a single model, is a key design lever. The GPT-4o-as-judge evaluation pipeline is also a reusable harness pattern for assessing open-ended generative outputs at scale.