BiasBusters: Uncovering and Mitigating Tool Selection Bias in Large Language Models

🕒 Published (v1): 2026-01-01 · Source: ICLR · Venue: ICLR 2026 · link

Why this paper was selected

Tool selection bias in LLM agents—fairness issue in tool marketplaces

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

LLMs used as agents in tool marketplaces exhibit systematic bias in selecting among functionally equivalent APIs, favoring certain providers due to superficial metadata or prompt position rather than true utility. This paper introduces a benchmark to measure this bias, identifies semantic alignment in tool descriptions as the dominant driver, and proposes a filter-then-uniform-sample mitigation that substantially reduces bias.

Problem

When LLMs act as agents and must choose among functionally equivalent tools from a marketplace (e.g., multiple weather APIs), they do not select uniformly—they systematically favor certain providers. This creates unfair market concentration, degrades user experience (slow/unreliable providers get disproportionate calls), and distorts competition. No prior work had formally studied or quantified tool-selection bias in LLM agents.

Method

Benchmark construction: 10 clusters of 5 functionally interchangeable APIs each (from RapidAPI via ToolLLM), with 100 provider-agnostic user queries per cluster (1,000 cluster-query pairs total). API selection is measured across cyclic rotations of prompt order to decouple positional from provider-level bias.

Bias metrics: Cluster-level API bias \(\delta_\text{API} = \text{TV}(P^\text{API}, \mathcal{U})\) and positional bias \(\delta_\text{pos} = \text{TV}(P^\text{pos}, \mathcal{U})\), where \(\mathcal{U}\) is the uniform distribution and TV is total variation distance. Combined: \(\delta_\text{model} = (\delta_\text{API} + \delta_\text{pos})/2\).

Root-cause analysis: (1) Pearson correlation + linear regression + random forest regressor on 7 API-level features (semantic similarity, parameter count, description length, readability, promotional wording, etc.) against empirical selection rates; (2) 8 controlled metadata perturbation experiments (name scramble, description scramble, parameter scramble, targeted scrambles, description swap); (3) biased continued pre-training (CPT) on Qwen3-8B using ~3.5M tokens saturated with one endpoint's metadata.

Mitigation: Filter candidate tools to a relevant subset, then sample uniformly among them.

Key Contributions

• First empirical study and formal benchmark of tool-selection bias in LLMs, covering 7 models (GPT-3.5-turbo, GPT-4.1-mini, Claude 3.5 Sonnet, DeepSeek-V3.2-Exp, Gemini 2.5 Flash, ToolLLaMA-2-7B, Qwen3 1.7B–235B).
• TV-distance metrics (\(\delta_\text{API}\), \(\delta_\text{pos}\), \(\delta_\text{model}\)) that decompose bias into provider-level vs. positional components.
• Causal attribution: description-level semantic alignment is the strongest predictor of tool choice (\(R^2 < 0.4\) for linear models, suggesting additional unexplained variance); name-only perturbations have inconsistent effects; CPT on biased corpora amplifies preferences (~20× relative increase in target endpoint selection).
• Lightweight mitigation: filter-then-uniform-sample, reducing \(\delta_\text{model}\) substantially while preserving task coverage.
• Public benchmark and code at https://github.com/thierry123454/tool-selection-bias.

Results

• All 7 models show \(\delta_\text{model} \approx 0.3\)–\(0.4\), meaning 30–40% of selection probability mass must be redistributed to achieve fairness.
• GPT-4.1 mini is most biased (\(\delta_\text{model} = 0.377\)); Qwen3-235B is least biased (\(\delta_\text{model} = 0.249\)).
• Two regimes observed: high API bias + low positional bias, or low API bias + high positional bias; models default to positional cues when no API semantically dominates.
• Most frontier models (GPT-4.1-mini, Claude 3.5 Sonnet, Gemini, DeepSeek, Qwen3-235B) share correlated bias patterns (aligned in which APIs they prefer), while GPT-3.5 and ToolLLaMA diverge.
• Description scrambling causes the largest shift in selection (TV vs. base \(\approx 0.45\) for Gemini); name-only scrambles cause smaller, higher-variance shifts; parameter-only scrambles are least impactful.
• Biased CPT increased target endpoint selection from 0.006 to 0.128 (>20× relative) within one epoch, with most gain realized in the first third of training.
• Raising temperature modestly reduces bias; model scale (larger = less biased) is a notable factor.

Limitations

• Benchmark covers only 10 API clusters from RapidAPI; coverage may not generalize to all tool domains or enterprise toolsets.
• Linear regression explains less than 40% of variance in selection rates (\(R^2 < 0.4\)), indicating significant unexplained drivers that the feature set does not capture.
• Biased CPT experiments use only Qwen3-8B on one target cluster; generalization to other models or multi-endpoint contamination is untested.
• Mitigation (filter-then-sample) is evaluated on coverage but full task-success tradeoffs against bias reduction are not completely reported in the provided text.
• Root-cause analysis is largely correlational; causal mechanisms within model internals (attention patterns, memorization circuits) are not mechanistically identified.

Relevance to Agentic AI / LLM Agents

Tool-augmented agents that select from API marketplaces are a core LLM agent paradigm, and this work reveals a structural fairness failure in that paradigm: agents do not behave as neutral dispatchers but carry systematic preferences that distort provider competition. The finding that description-level semantic alignment is the primary driver—rather than name familiarity or prompt position—has direct implications for how tool registries should be designed and how tool descriptions should be normalized to reduce inadvertent bias. The biased CPT result also alerts practitioners that pre-training data composition shapes not just knowledge but marketplace behavior, raising concerns for any agent system trained on web data containing disproportionate API mentions. The proposed filter-then-sample mitigation is immediately deployable in existing agentic pipelines.