Explainable Model Routing for Agentic Workflows

🕒 Published (v1): 2026-04-04 00:11 UTC · Source: Arxiv · link

Why this paper was selected

Explainable model routing for agentic workflows; cost-quality tradeoff analysis

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

Topaz is an interpretable model-routing framework for agentic workflows that assigns subtasks to LLMs by matching task skill requirements against model capability profiles derived from public benchmarks, with routing logic traced through explicit numerical computations rather than opaque learned mechanisms. It provides both local (per-subtask) and global (cross-task) natural-language explanations grounded in actual match scores and cost penalties, enabling developers to audit, verify, and iteratively steer cost-quality tradeoffs.

Problem

Current LLM routing systems treat model assignment as a black-box optimization—confidence thresholds, learned decision boundaries, or latent preference signals—leaving developers unable to distinguish legitimate efficiency gains (capability saturation justifies cheaper model) from silent quality regressions (budget pressure degrades a critical subtask). Aggregate benchmark scores further mask model capability variance across skill dimensions, and standard post-hoc XAI techniques surface optimization internals rather than actionable rationale about model-task fit.

Method

Topaz operates in three stages:

1. Skill-based profiling. A shared skill taxonomy \(S = \{s_1, \ldots, s_k\}\) (eight skills: math, logic, code, tool use, factual knowledge, writing, instruction following, summarization) is defined with natural-language descriptions. Each public benchmark \(b\) is profiled by an LLM to yield \(L_1\)-normalized skill weights \(w_{b,s}\). Model capability scores are: $\(C_{m,s} = \frac{\sum_b \tilde{S}_{m,b} \cdot w_{b,s}}{\sum_b w_{b,s}}\)$ where \(\tilde{S}_{m,b}\) is the 0-max normalized leaderboard score. Each incoming subtask \(t\) is likewise profiled for skill requirements \(R_{t,s}\), complexity \(k_t\), token skew \(\sigma^{i/o}_t\), and quality sensitivity \(q_t\).

2. Cost-aware routing. Skill match is capped at requirement satisfaction: $\(\text{Match}_{m,t} = \sum_{s \in S} \min\!\left(1, \frac{C_{m,s}}{k_t \cdot R_{t,s}}\right) \cdot R_{t,s}\)$ Two routing algorithms are provided: (i) Objective-based: per-subtask argmax over a weighted quality-cost objective parameterized by global cost sensitivity \(c_{\text{global}}\) and per-task \(q_t\); (ii) Budget-based: dynamic programming over the subtask sequence to maximize quality-weighted match subject to a hard dollar budget \(B\), with assignments recovered by backtracing.

3. Explanation generation. A structured log of user configuration, intermediate match and cost scores, and final assignments is passed to an LLM to generate natural-language explanations. Because the LLM reads actual numerical traces rather than post-hoc rationalizing from outputs, explanations are faithful to the decision logic. A feedback loop lets developers adjust \(q_t\) per subtask to shift future routing.

Key Contributions

• Formal critique of agentic routing's XAI deficit: routing XAI requires explaining orchestration decisions (why this model, this task, this cost), not just single-model inference.
• Domain-agnostic benchmark synthesis pipeline that maps heterogeneous public leaderboard scores into per-skill capability profiles via LLM-profiled skill weights (Eq. 1).
• Two fully traceable routing algorithms (objective-based multi-objective optimization and DP budget-based) whose intermediate traces directly ground explanations.
• Developer-facing local and global explanation layer derived from real routing traces, plus a closed-loop feedback mechanism for iterative quality-sensitivity tuning.

Results

All results are qualitative case studies; no aggregate quantitative benchmark comparisons to competing routers are reported.

• Five-model pool (Claude Opus 4.5 at $5/$25 per M tokens through Mistral Small 3.1 at $0.10/$0.30) spans a 50–250× cost range.
• At \(c_{\text{global}}=0.0\): Claude Opus 4.5 assigned to Technical Diagnosis and Knowledge Base Search; Gemini 3 Pro to math/writing tasks.
• At \(c_{\text{global}}=0.5\): Gemini 3 Pro replaces Claude on Technical Diagnosis (comparable skill coverage at <50% cost); Mistral handles extraction subtasks where its capability fully satisfies lower-complexity requirements.
• At \(c_{\text{global}}=1.0\): Gemini 3 Pro retained only for Technical Diagnosis (high complexity, strong logic/tool-use requirement); Mistral assigned everywhere else.
• Generated explanations correctly attribute downgrade decisions to capability saturation (sufficient match) rather than quality loss for low-sensitivity subtasks.

Limitations

• No quantitative evaluation against competing routers (RouterBench, FrugalGPT, BELLA); efficacy is demonstrated only through a single case study workflow with six subtasks.
• Skill taxonomy and benchmark-to-skill mappings are produced by an LLM profiler (Gemini Flash), introducing potential profiling error that propagates into all downstream routing and explanation quality.
• Token count estimation (particularly output length) is acknowledged as unreliable; the relative cost model is a workaround, not a solution.
• Budget-based DP routing is demonstrated only in an appendix; the main case study covers only objective-based routing.
• Topaz explains model selection, not downstream model behavior: whether the assigned model actually performs as its skill profile predicts on the specific instance is unmonitored.
• Feedback loop adjusts \(q_t\) but provides no automated signal for when quality was actually insufficient—developers must notice failures manually.

Relevance to Harnesses / Meta-Harnesses

Topaz is directly a meta-harness component: it provides the orchestration layer that decides which model handles each subtask in a multi-model pipeline, making it the policy engine that meta-harnesses need when mixing specialist models across workflow stages. The skill-taxonomy-based profiling and traceable routing algorithms are reusable primitives for any harness that must dynamically allocate tasks across a heterogeneous model pool under budget constraints. The feedback loop and per-task quality sensitivity parameters mirror the kind of human-in-the-loop steering that robust meta-harnesses require for continuous calibration. The explicit distinction between local (per-subtask) and global (cross-workflow) explanations maps directly onto the audit and debugging needs of harness maintainers who need to diagnose failures at both the individual step level and the pipeline architecture level.