An Agentic Retrieval Framework for Autonomous Context-Aware Data Quality Assessment

🕒 Published (v1): 2026-05-15 11:24 UTC · Source: Arxiv · link

Why this paper was selected

Agentic retrieval framework for autonomous context-aware data quality assessment

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

The paper proposes a multi-agent framework that takes a natural-language description of intended data usage and autonomously derives context-specific data quality assessment plans, generates executable validation logic (SQL/programmatic checks), and runs a feasibility-gated execution loop to prevent non-executable rules from reaching the data layer. Results show that assessment outcomes adapt to usage context and that the feasibility gate reduces unrealistic rule generation.

Problem

Existing data quality tools apply static, context-free rules that cannot adapt to different intended uses (e.g., exploratory analysis vs. regulatory auditing). LLM-based approaches that generate validation logic via prompting are sensitive to prompt formulation, lack reproducibility, and may produce non-executable or hallucinated rules. The gap between high-level, context-aware assessment planning and reliable automated execution remains unaddressed.

Method

The framework is structured in three layers:

1. Interaction Layer: Ingests dataset schema metadata (types, cardinalities, missing ratios) and a natural-language usage description.
2. Autonomous Reasoning Layer — four cooperating agents in sequence:
3. Context Interpreter Agent: infers assessment intent from usage description + dataset metadata.
4. Assessment Planner Agent: selects relevant quality dimensions and priorities.
5. Rule Generation Agent: maps the plan to concrete candidate validation specifications (rule templates, thresholds, attribute mappings).
6. Feasibility Validator Agent: checks whether each generated specification is realistic and executable against the known schema/environment; failures trigger structured feedback back to the Rule Generation Agent for iterative refinement.
7. Deterministic Execution Layer: Accepted specifications are run deterministically (e.g., SQL queries), guaranteeing reproducibility and auditability.

RAG grounds each agent's reasoning in a curated external knowledge store (quality dimensions, domain standards), reducing hallucination. The probabilistic/generative components are strictly separated from the deterministic execution path.

Key Contributions

• A unified agentic-RAG architecture that bridges context-aware assessment planning and executable validation in a single end-to-end pipeline.
• An explicit feasibility validation gate inside the agentic loop that prevents non-executable or operationally inconsistent rules from reaching execution, with structured feedback for iterative refinement.
• A principled separation of probabilistic reasoning (agents) from deterministic execution (query/code runner), ensuring reproducibility and auditability.
• An end-to-end prototype evaluated across multiple usage scenarios on the same dataset, demonstrating context-sensitivity of outcomes.

Results

• Assessment outcomes adapt meaningfully when the same dataset is evaluated under different intended-use descriptions (e.g., different quality dimensions and thresholds are selected per scenario).
• Feasibility-gated execution reduces unrealistic or non-executable rule generation compared to ungated prompting baselines.
• No quantitative benchmark numbers are reported in the provided text (paper appears to be truncated before Section 6); evaluation is qualitative/demonstrative across multiple usage scenarios on a single dataset.

Limitations

• Evaluation uses only a single dataset across multiple scenarios; generalizability to heterogeneous schemas or domains is not demonstrated.
• No quantitative comparison against prior LLM-based or RAG-only baselines is visible in the provided text.
• The feasibility validator is itself LLM-driven, so it may inherit prompt sensitivity issues it is designed to mitigate.
• Scalability of the multi-agent chain (latency, cost) under large datasets or many concurrent scenarios is not addressed.
• The prototype does not appear to handle schema evolution or streaming/incremental data contexts.

Relevance to Harnesses / Meta-Harnesses

This paper is a direct instance of a domain-specific multi-agent harness: it orchestrates a fixed pipeline of specialized sub-agents (Interpreter → Planner → Generator → Feasibility Validator → Executor), injects RAG-retrieved context at each stage, and uses a feedback loop for iterative refinement — the canonical structure of a meta-harness. The feasibility-gated execution pattern — where a control agent intercepts generated artifacts and routes failures back for refinement before any irreversible action — is directly applicable to harness design for code generation, workflow synthesis, or tool-call validation. The explicit separation of probabilistic generation from deterministic execution is a clean architectural principle that harness designers can adopt to bound blast radius and guarantee auditability.