An Exploration of Agentic Information Fusion for Test Maintenance Prediction

🕒 Published (v1): 2026-07-06 08:20 UTC · Source: Arxiv · link

Why this paper was selected

Agentic information fusion for test maintenance; novel multi-signal agent coordination pattern

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

MAST (Multi-Agent Multi-Source Test Maintenance) is a LangGraph-based multi-agent harness that predicts which test cases require maintenance after a production code change by fusing three parallel analyses—static (call graph), lexical (BM25), and semantic (RAG)—through an LLM fusion agent followed by an LLM post-check agent. Evaluated on 21 industrial Java repositories from Ericsson AB, MAST achieves higher precision, accuracy, F1, and F2 than a prior semantic-only baseline, at some cost to recall.

Problem

Identifying which tests need modification following a production code change ("test localization") is error-prone and labor-intensive. Prior LLM-based approaches suffer from one or more of: (1) assuming a pre-existing one-to-one production-to-test mapping, (2) restricting analysis to single-method diffs, (3) ignoring negative cases where no maintenance is actually needed, or (4) relying on a single analysis modality that misses cross-cutting signals.

Method

MAST is a pre-planned directed-graph agent workflow implemented in LangGraph, with state shared via a common memory bus. The pipeline:

1. Test Maintenance Prediction Agent — receives a git diff chunk, generates an NL summary of the change, and binary-classifies whether maintenance is needed. If yes, it also extracts the changed class and method names. Early-exit if no maintenance predicted.
2. Three parallel analysis nodes:
3. Semantic Analysis Agent: embeds test case NL summaries with bge-m3, stores in FAISS; retrieves tests with cosine similarity > 0.50 to the change summary, then applies a second filter retaining only those above \(\text{mean} + 0.25 \cdot \text{std.dev.}\) of the retrieved set.
4. Lexical Analysis: tokenizes test code (lowercased, punctuation-removed, "test" keyword split), indexes with BM25; retrieves up to 5 tests with score > 50.00.
5. Static Analysis: parses the repository with tree-sitter to build a call graph mapping each production method to all tests that directly invoke it; looks up tests for the methods extracted in step 1.
6. Fusion Agent — an LLM that receives all three candidate lists with source explanations (without source trustworthiness hints to avoid bias) and produces a single merged candidate list with rationale.
7. Post-Check Analysis Agent — fetches source code for each fused candidate and compares it directly against the diff, pruning tests lacking clear relevance and providing per-test rationale.

All LLM agents use Qwen3-Coder-480B-A35B-Instruct-FP8 deployed on Ericsson's internal infrastructure.

Key Contributions

• Multi-source fusion of static, lexical, and semantic analysis via an LLM agent, first applied to test maintenance.
• LLM post-check agent as a second-stage false-positive filter after fusion.
• Repository-level analysis processing full git diffs (multiple chunks, multiple files), not single-method changes.
• No assumption of a pre-existing production-to-test mapping; relationships are inferred.
• Explicit evaluation on both positive (maintenance required) and negative (no maintenance required) commits.
• Industrial evaluation on 21 Ericsson Java repositories across five teams.

Results

• Positive cases (RQ1): MAST achieves substantially higher precision than the semantic-only baseline, translating to higher accuracy, F1, and F2; baseline achieves higher recall.
• Negative cases (RQ2): MAST produces far fewer false positives than the baseline, yielding higher accuracy.
• Ablation (RQ3): Each of the three analyses and the fusion/post-check procedure contributes distinctly; removing any degrades the combined metric. Post-check prunes false positives at some recall cost; inspecting the pre-post-check fusion output is recommended if recall is prioritized over precision.
• Evaluated across 21 industrial Java repositories (Ericsson AB); specific per-repo precision/recall/F1 numbers are in sections not included in the provided excerpt.

Limitations

• Precision–recall trade-off: the baseline retains higher recall; MAST is not appropriate when recall must be maximized unconditionally.
• Evaluation limited to Java with JUnit; cross-language generalizability is asserted conceptually but not empirically demonstrated.
• Workflow is fully pre-planned (static directed graph); no dynamic agent-determined routing or adaptive replanning.
• Relies on Ericsson-approved model availability; the choice of Qwen3-Coder-480B was constrained by internal approval, not independent selection.
• Call graph captures only direct invocations; transitive or reflective dependencies may be missed.
• Paper text is truncated before full quantitative results tables are presented.

Relevance to Harnesses / Meta-Harnesses

MAST is a textbook example of a structured agent harness: a LangGraph directed graph orchestrates specialized sub-agents in a fixed topology (predict → parallel-analyze → fuse → post-check), with a shared state bus acting as inter-agent memory. The fusion+post-check pattern—where a meta-agent synthesizes the outputs of peer agents before a second meta-agent filters the synthesis—is a reusable harness idiom applicable beyond test maintenance. The implementation illustrates how LangGraph's node-edge model maps to the harness concept, and the ablation study provides rare empirical evidence for how each harness component contributes to end-to-end quality, which is directly useful for harness designers weighing component inclusion costs.