Skip to content

The Idola Tribus of AI: Large Language Models tend to perceive order where none exists

🕒 Published (v1): 2025-10-10 02:51 UTC · Source: Arxiv · Venue: EMNLP 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

LLMs—including state-of-the-art thinking models (o3, o4-mini, Gemini 2.5 Flash Thinking)—systematically hallucinate patterns in structureless number sequences, a bias the authors term the AI equivalent of Francis Bacon's Idola Tribus (Idols of the Tribe). The bias persists even when a model's own self-evaluation judges the fabricated pattern incorrect. A simple prompt modification allowing models to declare a series random substantially mitigates the effect for most models.

Problem

Agentic and RAG frameworks assume LLMs maintain logical self-consistency and can form reliable inductive hypotheses. Prior benchmarks (ReClor, LogiQA, MATH) test deductive reasoning with fixed answers, not open-ended inductive abstraction free from knowledge-recall confounds. No systematic study had shown whether LLMs fabricate patterns in genuinely structure-free inputs, which would undermine the premise of agent orchestration that relies on grounded reasoning.

Method

  • Dataset: 724 integer sequences across 8 categories: arithmetic, geometric, difference (second-order), quasi-ordered variants (one-term ±1 perturbation), random-increasing, and random. Each series provided as 5 terms.
  • Regularity identification: Five LLMs (GPT-4.1, o3, o4-mini, Gemini 2.5 Flash Thinking, Llama 3.3) each received a zero-shot user-only prompt asking for a one-sentence regularity explanation.
  • Evaluation: LLM-as-a-Judge using o3 classified each response into four options: (1) correct + category-aligned, (2) correct but alternative rule, (3) incorrect, (4) declared random. Options 1, 2, and 4 (for non-clearly-ordered categories) counted as success.
  • Ablation: A second prompt variant explicitly permitted the model to answer "random series," and the resulting success/random-declaration rates were compared against the baseline.

Key Contributions

  • First systematic empirical demonstration that LLMs generate internally inconsistent pattern explanations for structureless sequences—distinct from factual hallucination.
  • Shows the bias survives extended chain-of-thought: o3 self-evaluated only 52/100 random-category responses as valid, yet still output those invalid patterns.
  • Identifies "creative over-reach" in thinking models (o3, o4-mini fabricate atomic-number, tarot, and biblical interpretations for random data).
  • Demonstrates that a single-sentence prompt change ("you may respond that it is a random series") raises random-category success dramatically (e.g., o4-mini: 43% → 97%, o3: 52% → 94%).
  • Frames the phenomenon through Bacon's Idola Tribus as a structural AI-cognition bias, not merely a data or knowledge gap.

Results

  • Arithmetic/geometric: 100% success across all models—LLMs handle clearly structured series reliably.
  • Difference series: o3/o4-mini/Gemini2.5 ≈ 99–100%; GPT-4.1 drops to 36%; Llama3.3 93%.
  • Random category (original prompt): o3 52%, o4-mini 43%, Gemini2.5 31%, GPT-4.1 8%, Llama3.3 0%.
  • Random category (random-allowing prompt): o4-mini 97%, o3 94%, Llama3.3 86%, GPT-4.1 83%, Gemini2.5 30%.
  • Overall success (original prompt): o3 73.1% > Gemini2.5 70.6% > o4-mini 67.8% > Llama3.3 39.9% > GPT-4.1 33.0%.
  • Overall success (random-allowing prompt): o3 87.7%, o4-mini 88.3%, GPT-4.1 55.0%, Llama3.3 56.6%, Gemini2.5 68.9%.
  • Gemini 2.5 shows minimal improvement with the modified prompt, suggesting model-specific internal suppression of the "random" declaration.

Limitations

  • Only five sequences per series shown to the model; richer context might reduce false pattern detection.
  • LLM-as-a-Judge (o3) is itself fallible; the paper acknowledges no perfect autonomous evaluator exists.
  • Experiment confined to positive integers ≤100 (except geometric); generalization to other domains (text, relational data) untested.
  • The random-allowing prompt fix is manually engineered; no systematic prompt optimization was performed.
  • Quasi-ordered categories remain hard even with the improved prompt (e.g., o3 quasi-arithmetic only 66.7%), suggesting the bias is deeper than simple output suppression.
  • Self-evaluation disconnect (CoT vs. final output) is observed but mechanistically unexplained.

Relevance to Harnesses / Meta-Harnesses

Harnesses and meta-harnesses that orchestrate multi-step LLM pipelines often treat reasoning model outputs as reliable signals for branching, verification, or self-correction loops. This paper provides concrete evidence that even the strongest thinking models (o3, o4-mini) may assert conclusions their own intermediate reasoning rejects—directly undermining harness designs that assume chain-of-thought self-consistency. The finding that a single prompt-level constraint ("you may say random") substantially recovers reliability suggests harnesses can partially mitigate this failure mode through structured output contracts or explicit null-hypothesis affordances, without requiring model retraining. The self-evaluation disconnect also implies that meta-harnesses using LLM-as-a-Judge for quality control may silently pass fabricated intermediate results if the judge and generator share the same bias.