Skip to content

AutoLink: Autonomous Schema Exploration and Expansion for Scalable Schema Linking in Text-to-SQL at Scale

🕒 Published (v1): 2025-11-21 12:12 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

AutoLink reformulates schema linking for text-to-SQL as an iterative, agent-driven discovery process where an LLM agent explores a database without ever ingesting the full schema. It achieves state-of-the-art strict schema recall rates of 97.4% on Bird-Dev and 91.2% on Spider-2.0-Lite while consuming significantly fewer tokens than competing methods. The approach scales to schemas exceeding 3,000 columns where existing methods degrade sharply.

Problem

Industrial-scale databases contain hundreds to thousands of columns, making full-schema prompting infeasible due to LLM context window limits and irrelevant noise. Existing schema linking methods face a three-way bottleneck: (1) element-level scorers require \(O(|S|)\) passes per question, (2) database-level LLM methods exceed context limits on large schemas, and (3) high-recall retrieval approaches reintroduce noise by returning large candidate sets. No prior method simultaneously achieves high strict recall rate (SRR), low token usage, and scalability to industrial-scale schemas.

Method

AutoLink models schema linking as a sequential decision process: an LLM agent \(\pi\) iteratively constructs \(S_{\text{linked}} \subset S_{\text{full}}\) via multi-turn interaction with two pre-built environments per database:

  • \(\mathcal{E}_{DB}\) (Database Environment): executes exploratory SQL against the live database, returning truncated results or error feedback.
  • \(\mathcal{E}_{VS}\) (Schema Vector Store): dense-retrieval index of all columns (encoded with bge-large-en-v1.5), supporting ANN queries with exclusion sets to avoid redundancy.

The agent begins with an initial schema \(S_{\text{linked}}^{(0)} = \mathcal{E}_{VS}(Q, n, \emptyset)\) (top-\(n\) columns by semantic similarity, \(n \in [5, 100]\)) plus the full table name list. Each turn produces a <think> reasoning trace and a structured <action> block from a five-action space:

  1. @explore_schema — SQL queries against \(\mathcal{E}_{DB}\) for structural metadata or sample values.
  2. @retrieve_schema — targeted ANN query to \(\mathcal{E}_{VS}\) using agent-inferred concepts (top-\(m = 3\)), excluding already-seen columns.
  3. @verify_schema — executes a draft answer SQL via \(\mathcal{E}_{DB}\); errors directly signal missing schema elements.
  4. @add_schema — updates \(S_{\text{linked}} \leftarrow S_{\text{linked}} \cup S_{\text{added}}\); must be paired with a feedback action.
  5. @stop_action — terminates the loop; maximum 10 turns as a safeguard.

DeepSeek-V3 is the agent policy (DeepSeek-R1 was excluded due to instruction-following degradation on structured action formats). SQL generation downstream uses self-consistency sampling + majority voting over DeepSeek-R1/V3; it is not the paper's focus.

Key Contributions

  • Reframes schema linking as an interactive sequential decision-making problem, avoiding full-schema ingestion entirely.
  • Dual-environment design (\(\mathcal{E}_{DB}\) + \(\mathcal{E}_{VS}\)) enabling complementary exploration: structural SQL queries and semantic vector retrieval.
  • Agent-inferred retrieval queries (@retrieve_schema) that go beyond user-question rephrasing to reason about missing column semantics from partial schema context.
  • Demonstrated scalability to schemas with >3,000 columns, where full-schema methods collapse.
  • State-of-the-art SRR (97.4% Bird-Dev, 91.2% Spider-2.0-Lite) with lowest average token consumption on Spider-2.0-Lite (21.2K vs. 171.9K for SQL-to-Schema).

Results

  • SRR (Bird-Dev): 97.4% vs. 93.3% (RSL-SQL, 2nd best); avg. columns in \(S_{\text{linked}}\): 35.8.
  • SRR (Spider-2.0-Lite overall): 91.2% vs. 64.0% (SQL-to-Schema, 2nd best) — a +27.2% absolute improvement.
  • SRR by dialect (Spider-2.0-Lite): BigQuery 93.7%, Snowflake 85.7%, SQLite 95.8%.
  • Token consumption (Spider-2.0-Lite): 21.2K avg. vs. 168.9K (MCS-SQL), 171.9K (SQL-to-Schema), 29.2K (RSL-SQL) — 87.7% reduction vs. the next lowest full-schema method.
  • Execution accuracy (Bird-Dev): 68.71% (Gemini-1.5-Pro backbone), 66.36% (DeepSeek-V3); competitive with CHESS 68.31% at far lower token cost.
  • Execution accuracy (Spider-2.0-Lite): 34.92% (DeepSeek-R1), ranking 2nd on the official leaderboard; vs. REFORCE 37.84% (o3, 81.1K tokens) at 38.0K tokens.
  • Ablation confirms @retrieve_schema is the most critical action for recall on large schemas.

Limitations

  • Maximum turn limit of 10 is a hard cap; complex schemas may require more exploration than this allows.
  • DeepSeek-R1 (reasoning model) was excluded from the agent policy due to instruction-following degradation with structured action formats, limiting use of stronger reasoning models.
  • Top-\(n\) for initial schema retrieval is a sensitive hyperparameter (varied 5–100), requiring per-dataset tuning to balance recall vs. noise.
  • Token savings are measured vs. LLM inference tokens; \(\mathcal{E}_{VS}\) construction and vector indexing costs are not reported.
  • Evaluation is limited to two benchmarks; generalization to proprietary industrial schemas beyond Bird and Spider-2.0-Lite is not directly demonstrated.

Relevance to Harnesses / Meta-Harnesses

AutoLink is a clear instance of a tool-augmented agentic harness: a controller LLM orchestrates a structured action space over two external environments (live DB and vector store) in a multi-turn loop, with explicit turn budgeting (max 10) and state accumulation across iterations. The design pattern — initial broad retrieval seeding the loop, targeted tool calls per turn, hypothesis-driven verification via @verify_schema, and an explicit stopping condition — maps directly to the harness design principles of initialization, iterative refinement, and termination control. The finding that reasoning-optimized models (DeepSeek-R1) degrade on structured action formats is a practically important constraint for harness builders choosing agent backbone models. The scalability result (graceful degradation vs. catastrophic collapse of full-schema methods) demonstrates that agentic harnesses can outperform static pipelines specifically because they defer context construction to runtime, a design advantage relevant to any harness operating over large structured environments.