Skip to content

RAGalyst: Automated Human-Aligned Agentic Evaluation for Domain-Specific RAG

🕒 Published (v1): 2025-11-06 16:22 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

RAGalyst is an end-to-end agentic framework for evaluating domain-specific RAG systems that combines automated synthetic QA generation with prompt-optimized LLM-as-a-Judge metrics validated against human annotations. It addresses the dual gap of unreliable synthetic benchmarks and unvalidated LLM judge prompts for specialized domains. Applied to military operations, cybersecurity, and bridge engineering, it shows no single RAG configuration dominates across domains.

Problem

Existing RAG evaluation frameworks rely on lexical metrics (BLEU, ROUGE) that miss semantic variation, or LLM-as-a-Judge formulations that have never been validated against human judgment. Automated QA generation pipelines (e.g., RAGAS) lack rigorous quality filtering, producing unreliable benchmark datasets. Specialized domains (classified, safety-critical) cannot practically source manually annotated QA datasets, yet domain-specific variables (jargon, chunk granularity, document structure) make cross-domain generalization invalid.

Method

RAGalyst operates in three orchestrated stages:

  1. Preprocessing: LangChain parses PDF/Markdown/plain text into token-based chunks (default: 800 tokens, 400-token overlap), vectorized into a retrieval store.

  2. Agentic QA generation pipeline: (a) Random context chunk sampling; (b) a "user" agent generates a question grounded in the chunk, then a "subject matter expert" agent generates the ground-truth answer; (c) a validation/filtering gate applies Answerability, Faithfulness, and Answer Relevance thresholds — QA pairs below threshold are discarded.

  3. Evaluation module: Custom LLM-as-a-Judge metrics — Answer Correctness (continuous 0–1 semantic alignment of generated vs. reference answer) and Answerability (binary: is the question fully supported by the context alone). Standard retrieval metrics MRR and Recall@K are also included. Both judge prompts are refined via DSPy optimizers (COPRO, MIPROv2, LabeledFewShot) using Spearman correlation with human annotations as the objective. Answer Correctness is validated on STS-B; Answerability on SQuAD 2.0.

Key Contributions

  • End-to-end agentic harness integrating document preprocessing, synthetic QA generation with automatic quality filtering, and modular RAG evaluation.
  • Novel prompt-optimized LLM-as-a-Judge formulations for Answer Correctness and Answerability that outperform RAGAS baselines on human alignment.
  • Multi-domain evaluation (military operations, cybersecurity, bridge engineering) exposing domain-specific performance trade-offs.
  • Empirical analysis of the most common failure modes driving low Answer Correctness in RAG.

Results

  • Answer Correctness (GPT-4o-mini, unoptimized): \(\rho_s = 0.874\) (SE = 0.053) vs. RAGAS \(\rho_s = 0.836\) and Cosine Similarity (Qwen3-Embedding-8B) \(\rho_s = 0.622\).
  • Answer Correctness (MIPROv2 + LabeledFewShot, k=8): \(\rho_s = 0.894\) — best configuration.
  • Answerability (Gemini 2.5 Pro, unoptimized): \(\rho_s = 0.752\) (SE = 0.051) — best single-model result.
  • DSPy instruction optimizers (COPRO, MIPROv2) decrease Answerability correlation to \(\rho_s \approx 0.670\); LabeledFewShot provides no improvement over the handcrafted Answerability prompt.
  • Across domains, no single embedding model, generator LLM, or hyperparameter configuration is universally optimal.

Limitations

  • Answerability metric resists automatic prompt optimization; the handcrafted prompt outperforms all DSPy-optimized variants, suggesting the metric's binary nature limits gradient signal.
  • Absolute Answerability correlation (\(\rho_s = 0.752\)) is moderate — meaningful alignment gaps with human judgment remain.
  • Framework depends on proprietary LLMs (GPT-4o-mini, Gemini 2.5 Pro) for evaluation and optimization, creating cost and reproducibility concerns in closed-domain deployments.
  • Optimal chunk size is domain-dependent (>20% variation reported) but no automated tuning procedure for chunk hyperparameters is proposed.
  • Human annotation ground truth for metrics validation was sourced from general-domain benchmarks (STS-B, SQuAD 2.0), not the target specialized domains.

Relevance to Harnesses / Meta-Harnesses

RAGalyst is a concrete example of a multi-stage evaluation meta-harness: it orchestrates independent agents (user-role QA generator, SME-role answerer, LLM judges) into a self-regulating pipeline where internal quality filters gate outputs before they propagate downstream. The prompt optimization loop — running DSPy optimizers against held-out human-annotated corpora to harden judge prompts — is itself a meta-harness pattern: a harness that tunes the components of an inner harness. For researchers tracking harness architectures, the paper is notable for its explicit separation of the QA generation harness from the evaluation harness, and for quantifying where automated prompt optimization adds value (continuous metrics like Answer Correctness) versus where it fails (binary metrics like Answerability).