Perovskite-LLM: Knowledge-Enhanced Large Language Models for Perovskite Solar Cell Research

🕒 Published (v1): 2025-02-18 09:19 UTC · Source: Arxiv · Venue: EMNLP 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

Perovskite-LLM is a domain-specialized system for perovskite solar cell (PSC) research that combines a knowledge graph (Perovskite-KG), two instruction-tuning datasets generated via a multi-agent pipeline, and two fine-tuned LLMs optimized for knowledge retrieval and scientific reasoning respectively. The fine-tuned 7B reasoning model achieves 43.95 on GPQA Diamond and 44.49 on Minerva using only 2,000 training examples, competitive with 32B models trained on orders of magnitude more data.

Problem

Existing AI systems for materials science focus on narrow prediction tasks or general scientific knowledge and lack the specialized capability to handle the complex synthesis-process-property relationships, fabrication ontologies, and performance metrics unique to PSC research. Simultaneously, creating high-quality instruction-tuning data for niche domains is expensive and prone to hallucination when done naively.

Method

Knowledge Graph Construction (Perovskite-KG): A three-stage LLM-driven pipeline processes 1,517 papers against an expert-defined schema of three ontologies (fabrication, parameters, performance), each decomposed into typed sub-ontologies \(\{soi^{(j)}, d_i^{(j)}, t_i^{(j)}\}\). Stage I filters documents per sub-ontology: \(D_{\text{filtered}}^{(i,j)} = \{D_k \in D \mid soi^{(j)} \subset D_k\}\). Stage II extracts knowledge via zero-shot LLM prompting: \(K = \text{search}\,\text{LLM}(f_{\text{prompt}}(soi^{(j)}, d_i^{(j)}, t_i^{(j)}); \theta)\). Stage III applies entity disambiguation and relationship deduplication before loading into a graph database.

Dataset Generation: A three-agent pipeline — Information Extraction Agent \(A_{\text{extract}}\), Quality Validation Agent \(A_{\text{validate}}\), Document Summarization Agent \(A_{\text{summarize}}\) — processes 2,214 high-impact papers guided by 7 expert categories and 21 research questions to produce Perovskite-Chat (55,101 QA pairs). Perovskite-Reasoning (2,217 problems) is curated from materials science textbooks with multi-step solution paths generated by DeepSeek-R1 and O1.

LLM Fine-Tuning: Full-parameter fine-tuning (not LoRA) of LLaMA-3.1-8B-Instruct and Qwen-2.5-7B-Instruct on the respective datasets, with KG/RAG augmentation as an additive option.

Key Contributions

• Perovskite-KG: first PSC-specific knowledge graph, 23,789 entities and 22,272 relationships from 1,517 papers, with citation links to reduce hallucination
• Multi-agent data generation framework producing Perovskite-Chat (55,101 QA pairs) and Perovskite-Reasoning (2,217 problems) with automated quality control
• Perovskite-Chat-LLM achieving Rouge-L 41.25 and LLM-Judge 2.97, outperforming GPT-4o (11.36 / 1.41) on Perovskite QA
• Perovskite-Reasoning-LLM reaching 43.95 GPQA / 44.49 Minerva with only 2K training examples vs. 800K for R1-Qwen2.5-7B
• Evaluation datasets: Perovskite-QA and Perovskite-MCQ (1,103 instances each), with difficulty stratified by LLaMA-3.1-8B zero-shot performance

Results

• Perovskite QA (Rouge-L): Perovskite-Chat-LLM 41.25 vs. GPT-4o 11.36, LLaMA-3.1-8B 13.18
• Perovskite QA (LLM-Judge): 2.97 vs. GPT-4o 1.41; Human Expert score 3.12 vs. GPT-4o 1.73
• Perovskite MCQ (All accuracy): 87.22 vs. GPT-4o 84.68; Hard subset 62.86 (highest among all tested models)
• +KG augmentation: MCQ All 89.21, QA Rouge-L 43.10, LLM-Judge 3.15
• GPQA Diamond (7B scale): Perovskite-Reasoning-LLM 43.95 vs. OpenThinker-7B 42.90 (114K examples), R1-Qwen2.5-7B 44.49 (800K examples)
• Minerva (7B scale): 44.49, competitive with LIMO-32B 44.90 and approaching S1-32B 47.79

Limitations

• Full fine-tuning requires substantial compute; PEFT (LoRA) exploration deferred to future work
• Perovskite-KG construction relies on LLM zero-shot extraction — accuracy of relationship extraction not independently benchmarked
• GPQA performance lags larger models (32B+), indicating model-size sensitivity for graduate-level reasoning
• Domain coverage is PSC-centric; generalization to adjacent materials domains not evaluated
• Evaluation dataset drawn from the same publication pool as training data, risking distribution overlap

Relevance to Harnesses / Meta-Harnesses

The three-agent pipeline (extraction → validation → summarization) is a textbook domain-specific harness: a deterministic orchestration layer that coordinates specialized agents to transform raw literature into structured training data at scale, directly analogous to meta-harness patterns that automate dataset construction pipelines. The schema-driven document filtering stage — where typed sub-ontologies gate which documents feed which extraction prompts — demonstrates parameterized harness configuration, a key design pattern for reusable orchestration systems. The KG construction pipeline (3-stage with quality control loops) and the reasoning dataset generation pipeline (textbook ingestion → multi-model solution synthesis → expert validation) each constitute independent harnesses composable within a larger meta-harness. For researchers tracking this topic, this paper illustrates how multi-agent harnesses can replace manual annotation at domain-specific scale while maintaining quality via structured validation checkpoints.