Skip to content

MARS: Multi-hop Adaptive Retrieval and SPARQL Generation for KGQA

🕒 Published (v1): 2026-07-16 04:40 UTC · Source: Arxiv · link

Why this paper was selected

Multi-hop adaptive retrieval + SPARQL; grounded LLM reasoning over knowledge graphs

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

MARS is a training-free KGQA system that answers natural-language questions over RDF knowledge graphs by iteratively retrieving triple patterns and generating SPARQL queries. Rather than open-ended agentic tool selection, it uses a structured loop that filters and ranks candidate predicates at each hop, giving it more predictable behavior than fully agentic alternatives. It achieves state-of-the-art results on three Wikidata benchmarks across up to 10 languages without any fine-tuning.

Problem

LLM-based KGQA systems either rely on fine-tuning (limiting generalization and requiring costly retraining) or on fully agentic frameworks that iteratively call tools without structural constraints, making them vulnerable to hallucination error propagation. Text-based RAG cannot represent relational multi-hop structure explicitly, and subgraph-retrieval methods blow up context windows or require expensive offline graph summarization. MARS addresses the gap between brittle fine-tuned pipelines and hallucination-prone open-ended agents.

Method

MARS operates as a three-stage pipeline:

  1. Entity linking & text augmentation — named entities are linked to the KG via a GRASP-based linker; an instruction-tuned multilingual LLM generates an entity-type-augmented rewrite \(q^{\text{aug}}_{\text{text}}\) alongside the original (possibly non-English) question.

  2. Pattern retrieval & filtering — for each linked entity set \(E\), triple patterns \(P = \{(s,p,\_)\mid s\in E\}\cup\{(\_,p,o)\mid o\in E\}\) are extracted from the KG. Patterns are verbalized and embedded with a sparse mixture-of-experts embedding model (\(d=768\)); semantic similarity (dot product) to the question vector ranks them, and the top-\(N\) patterns \(P_{\text{top}}\) are kept.

  3. Enrichment & SPARQL reasoning loop — each selected pattern is enriched with instance values (up to \(i=10\)), instance frequency \(n=|I_p|\), and typological schema information (domain/range labels). The enriched verbalization \(V^{\text{enriched}}_{P_{\text{top}}}\) plus the original question is fed to a SPARQL Reasoner LLM, which either emits a final SPARQL query or selects a pattern subset \(P_{\text{sel}}\) to expand. Expansion extracts new frontier entities \(E_{\text{next}} = E_{\text{sel}} \setminus E_{\text{vis}}\), re-invokes pattern retrieval for next-hop facts, and accumulates \(P_{\text{top}} \leftarrow P_{\text{top}} \cup P^{\text{next}}_{\text{top}}\). The loop terminates when a query is generated or a hop limit \(m_{\text{hop}}\) is reached.

For multilingual queries, a translation LLM converts the question to English for entity linking while the original-language text is preserved for semantic pattern matching (hybrid strategy).

Key Contributions

  • A fine-tuning-free KGQA pipeline that controls LLM context size via ranked triple-pattern retrieval rather than raw subgraph retrieval.
  • An iterative multi-hop expansion mechanism that adapts retrieval depth per question, avoiding fixed hop counts.
  • A hybrid multilingual setup supporting 10 languages (including 4 low-resource: Bashkir, Armenian, Belarusian, Ukrainian) by combining machine translation with native-language semantic matching.
  • Release of a specific Wikidata snapshot and updated benchmark datasets to address reproducibility gaps (prior QALD-10 dump omits property qualifiers needed to answer some queries).
  • Ablation study and error analysis covering result truncation, boolean ASK queries, aggregation/projection failures, and conservative abstention behavior.

Results

  • QALD-10 (most complex; ~30% multi-hop): MARS achieves Macro F1 of 66.29 (en), 62.02 (de), 58.49 (ru), 55.04 (zh) — best on all four languages, outperforming GRASP (62.29 / 59.34 / 52.40 / 51.11) and all other baselines.
  • QALD-9-plus (9 languages): MARS wins on French (71.93 vs. GRASP 12.38), Armenian (65.96 vs. 10.14), and Bashkir (36.32 vs. 26.18); GRASP leads on English (65.47 vs. 58.21), German, Spanish, Russian, Belarusian, Ukrainian.
  • LC-QuAD2.0 (English, 4624 test): MARS 40.83 Macro F1 vs. GRASP 38.36 and UniQ-Gen 34.56; no fine-tuning on this dataset's template-generated queries.
  • MARS beats GRASP in 8 of 14 head-to-head per-language comparisons across all three datasets.
  • Fine-tuned baselines (DeepPavlov, MST5, UniQ-Gen) lag substantially on heterogeneous QALD benchmarks, with Macro F1 generally 10–30 points below MARS/GRASP.

Limitations

  • Result truncation cap (1,000 results per query) reduces recall on open-ended questions with large gold answer sets (e.g., "Show me all Czech movies" gold: 8,376).
  • Boolean ASK questions are often answered with numeric/entity evidence rather than true/false.
  • Aggregation queries (COUNT, comparative projections) are not consistently generated; the system retrieves correct facts but fails to project through COUNT or select the winning entity.
  • Conservative abstention on temporal literals ("What year was X?") trades recall for precision, leaving some queries with empty predictions.
  • Class selection is occasionally too narrow for broad "Give me all X" queries.
  • Evaluation on open-weight LLMs only; GPT-4.1 replaced by GPT-OSS in GRASP baseline, which may affect fair comparison.
  • The KG schema (TBox) must be available; the approach is not directly applicable to KGs without explicit domain/range axioms.

Relevance to Agentic AI / LLM Agents

MARS is directly relevant as a case study in structured vs. open-ended agentic retrieval: it deliberately trades the flexibility of dynamic tool selection (as in GRASP, SPINACH, AgentIct2s) for a deterministic pattern-guided loop, and empirically shows this tradeoff reduces hallucination propagation while remaining competitive or superior on multi-hop benchmarks. For researchers studying LLM agent reliability, the paper quantifies when a fixed retrieval pipeline beats a fully agentic system (complex multi-hop, low-resource languages) and when it does not (high-resource, simpler hop patterns). It also contributes a reproducibility framework — releasing KG snapshots and using open-weight models only — that is directly transferable to agentic KGQA benchmarking. The hybrid multilingual strategy and the adaptive hop-depth mechanism are engineering patterns applicable to any agent that must traverse structured external knowledge stores.