Skip to content

K12-KGraph: A Curriculum-Aligned Knowledge Graph for Benchmarking and Training Educational LLMs

🕒 Published (v1): 2026-07-23 00:00 UTC · Source: HuggingFace · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

K12-KGraph is a curriculum-aligned heterogeneous knowledge graph extracted from official Chinese K–12 textbooks (People's Education Press), covering four subjects with nine node types including Figure and VisualElement for visual grounding. From this single graph the authors derive K12-Bench, a 23,640-question multi-select benchmark probing structural "curriculum cognition," and K12-Train, a 7,335-sample KG-guided SFT corpus including 5,068 multimodal VQA pairs. Even the strongest evaluated model (Gemini-3-Flash) reaches only 57% exact match on K12-Bench, while domain-specific KG-guided fine-tuning substantially closes the gap with fewer samples than general-purpose baselines.

Problem

Existing K–12 benchmarks (C-Eval, CMMLU, GaokaoBench, EduEval) measure only factual recall on exam questions. They do not test curriculum cognition: prerequisite dependency chains, concept taxonomies, experiment–concept evidence links, or where and how knowledge is visually grounded in textbook figures. No current instruction-tuning corpus explicitly teaches this structural and visual understanding, creating a bottleneck for educational AI systems beyond mere question-answering.

Method

Construction proceeds in five stages: 1. OCR parsing: MinerU converts textbook PDFs to structured Markdown, preserving formulas, headings, and image assets. 2. TOC segmentation: a table-of-contents parser produces per-section files and associates images with section text. 3. LLM extraction: GPT-5.2 is prompted with a schema-aware instruction to emit nodes and edges as structured JSON with confidence scores and textbook-evidence citations for every edge. 4. Hierarchical merge: per-section graphs are merged bottom-up, assigning globally unique IDs, deduplicating same-name concepts across books within a subject (e.g., "velocity" in 8th vs. 9th-grade physics). 5. DAG validation: depth-first cycle detection on \(\text{is\_a}\) and \(\text{prerequisites\_for}\) subgraphs; violations resolved manually.

The resulting K12-KGraph has 9 node types (Book, Chapter, Section, Concept, Skill, Experiment, Exercise, Figure, VisualElement) and 14 directed edge types. The multimodal component includes edges such as illustrates, refers_to, requires_figure, and supports_edge linking figures/visual elements to curriculum concepts.

K12-Bench derives 23,640 multi-select questions across five task families by graph traversal: Ground (tests_concept/tests_skill), Prereq (prerequisites_for), Neighbor (is_a / relates_to), Evidence (verifies), and Locate (appears_in / leads_to). Distractors are sampled from structurally proximate but non-answer nodes (e.g., 2-hop neighborhoods), filtered by 3-gram cosine similarity and a GPT-5.2 pedagogical filter.

K12-Train synthesizes SFT data via three paths: (i) node-grounded QA prompted from Qwen3-235B-A22B on typed node properties; (ii) edge-grounded QA with relation-specific templates (e.g., "Why must one learn A before B?" for prerequisites_for; "How does figure F explain concept C?" for illustrates); (iii) deterministic templates for factually unambiguous tests_concept/tests_skill edges. The multimodal subset (K12-Train-MM, 5,068 pairs) requires an accompanying figure or visual element to answer.

Key Contributions

  • K12-KGraph: first large-scale, multi-subject, curriculum-aligned KG from Chinese K–12 official textbooks with explicit visual grounding (Figure and VisualElement nodes; 9,077 contains_visual_element edges, 10,352 refers_to edges across subjects).
  • K12-Bench: 23,640-question multi-select benchmark with graph-derived distractors and 98.4% human-verified accuracy (Fleiss' \(\kappa = 0.84\)); fully reproducible from the graph.
  • K12-Train: 7,335-sample KG-guided SFT corpus (K12-Train-Text: 2,267; K12-Train-MM: 5,068) demonstrating superior sample efficiency over eight mainstream instruction-tuning datasets.
  • Reproducible end-to-end pipeline (graph, benchmark, training data, and construction code released).

Results

  • K12-Bench baseline: Gemini-3-Flash 57% exact match; Gemma-4-31B-IT 46%; Prereq and Neighbor are the hardest task families across all models.
  • LLM SFT (2,300-sample budget): K12-Train-Text outperforms all eight baselines (OpenHermes, Infinity, UltraChat, WizardLM, DataFlow, LMSYS, SmolTalk, Tulu-3) on GaokaoBench and EduEval; e.g., +24.1/+32.4 over the strongest SFT baseline, and +114.6/+221.0 over official instruction-tuned model variants on GaokaoBench.
  • VLM SFT: K12-Train-Full (7,335 samples) achieves best overall performance on Gaokao-MM, MDK12-medium, and K12Vista, surpassing full DataFlow (10,000 samples) and full WizardLM (142,759 samples).
  • K12-Train-Full consistently outperforms both K12-Train-Text and K12-Train-MM in isolation, confirming complementarity of textual and visual supervision.

Limitations

  • Scope is limited to Chinese K–12 (People's Education Press); not directly transferable to other national curricula or languages without full re-extraction.
  • Only four subjects covered (mathematics, physics, chemistry, biology); no social sciences, history, or language arts.
  • Graph extraction depends on GPT-5.2, introducing potential hallucination in edge confidence scores despite mitigation via evidence fields and human verification.
  • K12-Train-MM at 5,068 pairs is small relative to the scale of mainstream VQA corpora; generalization beyond Chinese educational content is untested.
  • K12-Bench probes parametric knowledge only (no graph context provided at inference), which may penalize models that could reason correctly with retrieval access.

Relevance to Vision-Language Models

K12-KGraph's multimodal component—with Figure and VisualElement nodes connected via illustrates, refers_to, requires_figure, and supports_edge edges—provides a structurally grounded source of educational VQA data that explicitly links visual content to curriculum concepts. The finding that K12-Train-Full (combining textual and visual supervision) outperforms larger general-purpose VQA corpora on Gaokao-MM, MDK12-Bench, and K12Vista demonstrates that KG-structured visual grounding is more sample-efficient than unstructured visual instruction data. For researchers tracking VLMs, this work introduces a reproducible method for constructing domain-specific multimodal training sets from knowledge graphs, and provides a benchmark family (Evidence and Locate tasks with visual grounding) that exposes a capability gap current VLMs do not address.