SAGA: Schema-Aware Grounding for Agentic Text-to-SPARQL Generation¶

🕒 Published (v1): 2026-07-16 02:17 UTC · Source: Arxiv · link

Why this paper was selected

Schema-aware grounding for agentic SPARQL; structured KG query generation via LLMs

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

SAGA is a training-free framework that addresses type-blind grounding in LLM-based agentic SPARQL generation, where agents select KB properties without checking entity-type compatibility against property domain/range declarations. By maintaining a persistent bidirectional type state and filtering incompatible property candidates before they reach the LLM controller, SAGA achieves the highest F1 on all nine benchmark settings across Wikidata and Freebase. The approach modifies the agent–KB interface rather than the model or its decoding process.

Problem¶

Interactive agentic KBQA systems (e.g., SPINACH, Interactive-KBQA) alternate between reasoning and KB tool calls to construct SPARQL queries incrementally, but their property-grounding step is type-blind: candidates are ranked by lexical relevance or pruned by an extra LLM call without consulting entity types, property domains ($\delta(p)$), or ranges ($\rho(p)$). This produces syntactically valid but semantically incompatible triple patterns that execute to empty results, and wastes interaction turns on repair. Existing schema-use approaches either inject schema into the prompt (model must apply it) or validate post-hoc — none filter at the grounding interface itself.

Method¶

SAGA operates in three stages:

Offline schema index: For each property $p$, record $\delta(p)$ (domain) and $\rho(p)$ (range) from formal sources (Wikidata P2302 type constraints; Freebase fb_roles). When formal declarations are absent, soft evidence is induced by sampling subject–object pairs and recording the most frequent observed types. Properties with no source are left unconstrained ($\delta(p) = \rho(p) = \emptyset$) rather than rejected.
Persistent bidirectional type state $\Theta$: During the interaction, $\Theta[x] \subseteq C$ accumulates types for every entity or SPARQL variable $x$ discovered via lookups or explicit type patterns in partial queries (forward direction). Before the first tool call, the LLM controller also predicts an expected answer type from the question, which propagates backward as soft priority guidance (never a hard filter, to allow intermediate variables of different types).
Type-constrained grounding at lookup time: When lookup(e) is called, the property set $P(e)$ is filtered to: $$P_{\tau(e)} = \{p \in P(e) \mid \delta(p) = \emptyset \;\vee\; \delta(p) \cap \tau(e) \neq \emptyset\}$$ Only properties with a known domain conflict are removed; unknown-domain properties are always retained (permissive under incompleteness). When $\tau(e) = \emptyset$ (no type known), filtering is skipped and the system falls back to LLM-based neighborhood pruning. Retained properties are returned in a compact schema-annotated format (PID: label — description; domain: ...; range: ...), replacing a separate LLM pruning call.

Trace-local evidence (types observed from non-empty execution results) further refines the state within an interaction without modifying the offline index.

Key Contributions¶

Identifies type-blind grounding as a distinct, systematic failure mode at the agentic KBQA interface, separate from prompting or decoding issues.
Introduces SAGA, a training-free framework that constrains the property-grounding action space at construction time using KG schema, with a persistent bidirectional type state and permissive handling of schema incompleteness.
Demonstrates consistent gains across nine benchmark settings on two KGs (Wikidata, Freebase) with two backbone LLMs, while also reducing empty-result rates and eliminating the extra LLM pruning call.

Results¶

SAGA (gpt-oss-120B) vs. SPINACH (next-strongest baseline) on Wikidata (F1 / EM):
QALD-7: 64.89 / 53.33 vs. 44.56 / 37.78 (+20.3 F1)
QALD-9+: 55.66 / 48.44 vs. 43.54 / 36.65 (+12.1 F1)
QALD-10: 53.04 / 48.04 vs. 41.64 / 39.69 (+11.4 F1)
WWQ-test: 51.49 / 45.56 vs. 34.36 / 32.21 (+17.1 F1)
SPINACH-test: 20.60 F1 vs. 17.00 (+3.6 F1)
LC-QuAD 2.0: 28.25 / 25.39 vs. 25.83 / 23.58 (+2.4 F1)
SAGA (gpt-oss-120B) on Freebase (F1 / EM), vs. best prior baseline (SPINACH or GRASP):
WebQSP: improvement of +0.2–11.2 F1 points over strongest non-SAGA baseline
GrailQA: GRASP was strongest at 43.05 / 40.60; SAGA results partially shown in truncated text
Highest F1 on all 9 settings; highest EM on 8/9.
Reduces empty-result query rate on all five Wikidata settings for which this diagnostic is reported.
LLaMA-3.1-70B variant also evaluated across all nine settings with consistent but lower absolute performance.

Limitations¶

When an entity's type $\tau(e) = \emptyset$ (unknown), filtering is skipped entirely and the system degrades to LLM-based pruning — offering no schema benefit in those cases.
Schema index completeness varies by KG: Freebase's fb_roles and Wikidata's P2302 constraints are incomplete, requiring soft empirical induction whose quality depends on triple sampling.
The framework is designed for SP-based KBQA with SPARQL endpoints; it does not address IR-based or embedding-based KBQA approaches.
Evaluation is restricted to Wikidata and Freebase; generalization to other KGs (e.g., DBpedia, domain-specific KGs) is untested.
Full paper text was truncated — ablation analysis and complete Freebase results tables are not visible in the excerpt.

Relevance to Agentic AI / LLM Agents¶

SAGA directly advances the design of multi-turn tool-calling agents that operate over structured environments: it shows that constraining the action space at construction time — rather than relying on the LLM to self-prune post-retrieval — yields significant reliability and accuracy gains. The concept of a persistent bidirectional type state (forward from observations, backward from goal) is a principled, reusable pattern for cross-turn context management in any agent that builds structured outputs incrementally. The finding that type-blind grounding is a grounding interface problem (not merely a prompting problem) has broader implications: agent frameworks that expose unfiltered tool outputs to LLM controllers may systematically suffer from analogous action-space pollution in other structured-retrieval domains. SAGA's training-free, schema-exploiting approach is also directly composable with stronger LLM controllers without retraining.