Skip to content

The Truth Stays in the Family: Enhancing Contextual Grounding via Inherited Truthful Heads in Model Lineages

🕒 Published (v1): 2026-06-14 13:39 UTC · Source: Arxiv · Venue: ICML 2026 · link

Why this paper was selected

ICML 2026; inherited truthful heads across model lineages offers new grounding tool against hallucination

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

Attention heads specialized for context-grounded truthfulness are preserved when a base LLM is fine-tuned into multimodal descendants, forming inheritable "truthful head" distributions within model lineages. The authors quantify this with per-head Truth Scores via linear probing and exploit it with TruthProbe, a soft-gating mechanism that amplifies high-scoring heads at inference time. Transfer of base-LLM Truth Scores to MLLMs reduces hallucination on HaluEval, POPE, and CHAIR without retraining.

Problem

Hallucination mitigation strategies treat models as isolated instances and re-derive interventions for each variant independently. It is unknown whether foundational LLMs and their fine-tuned multimodal descendants (e.g., LLaMA2 → Vicuna → LLaVA) share a structural basis for truthfulness that could be exploited systematically across an entire model family rather than model-by-model.

Method

Truth Score computation. For each attention head \(h\) in layer \(l\), a binary linear probe is trained on activations extracted at the final answer token from inputs structured as \(x = \{x_\text{context}, x_\text{question}, x_\text{answer}\}\), where \(x_\text{answer}\) is either truthful or hallucinated. Probe validation accuracy defines the head's Truth Score \(S_h^l\)—a measure of how linearly decodable context-faithful signals are from that head.

Inheritance analysis. Truth Score vectors across all heads are correlated between base LLMs and their fine-tuned LLM/MLLM descendants (single-dataset and cross-dataset settings). Parameter-level drift is measured via the Frobenius norm of weight differences between base and derived models to mechanistically explain score preservation.

TruthProbe (soft gating). At each transformer layer \(l\), the multi-head attention output is modulated before the residual addition: $\(x^{l+1} = x^l + \text{Concat}_{h=1}^{H}\left(g_h^l \cdot o_h^l\right)\)$ $\(g_h^l = 1 + \lambda \cdot \text{norm}(S)\)$ where \(g_h^l\) scales each head's output proportionally to its normalized Truth Score \(S\), with \(\lambda\) as a scalar hyperparameter. All heads remain active; only their contribution magnitudes are modulated. Truth Scores from the base LLM can be used directly as the gate for downstream MLLMs (plug-and-play transfer).

Key Contributions

  • Definition and measurement of per-head Context-Truthfulness Scores via linear probing on final-answer-token activations.
  • Empirical demonstration that Truth Scores are strongly inherited within model lineages (Vicuna/LLaVA, Qwen2.5/Qwen2.5-VL) and collapse to near-zero correlation across unrelated families (Vicuna vs. Mistral).
  • Mechanistic evidence: within-family weight drift (Frobenius norm \(\approx 0.03\)) vs. cross-family drift (\(\approx 1.01\)) explains why truthful heads survive fine-tuning; high-scoring heads cluster in middle-to-deep layers (\(>80\%\) of top-20 heads in layers 10–31 of LLaVA-1.5).
  • TruthProbe: inference-time soft-gating that transfers base-LLM Truth Scores to MLLMs without per-model re-probing.
  • Attention-overlay analysis showing context-truthful heads attend to query-relevant visual evidence, while low-scoring heads produce semantically diffuse attention.

Results

  • HaluEval (LLM truthfulness): Vicuna-7B F1 improves from 13.37 → 29.15; Qwen2.5 F1 improves from 36.69 → 46.54 with TruthProbe\(_\text{LLM}\).
  • POPE-COCO (object presence, MLLMs): LLaVA-1.5 Recall improves 79.1 → 80.1 (TruthProbe\(_\text{LLM}\)); LLaVA-NeXT 78.8 → 80.9; Qwen2.5-VL-Omni 75.0 → 77.7.
  • POPE-A-OKVQA: LLaVA-1.5 Acc improves 87.8 → 90.1; LLaVA-NeXT 86.8 → 89.7; Qwen2.5-VL-Omni 84.7 → 87.1.
  • CHAIR\(_I\) (image captioning hallucination, lower is better): LLaVA-1.5 6.99 → 5.36; LLaVA-NeXT 6.91 → 4.94 with TruthProbe\(_\text{LLM}\); notably, TruthProbe\(_\text{LLM}\) outperforms TruthProbe\(_\text{MLLM}\) on CHAIR for LLaVA models.
  • Truth Score correlations (single-dataset): within-family correlations range 0.77–0.98; cross-family (Vicuna vs. Mistral) 0.04–0.08.

Limitations

  • Soft-gating hyperparameter \(\lambda\) and normalization method (centered vs. min-max) differ across benchmarks, suggesting sensitivity to these choices.
  • Probing requires labeled truthful/hallucinated answer pairs; constructing reliable probing datasets for new domains or modalities adds overhead.
  • Evaluated on 7B-scale models; whether inheritance scales to larger parameter counts (e.g., 70B) is addressed only in the Appendix.
  • The transfer mechanism depends on architectural identity (shared layer structure) between base and derived models; models with architectural modifications during fine-tuning may break the assumption.
  • Results are on standard hallucination benchmarks (HaluEval, POPE, CHAIR); broader factual accuracy and open-ended generation quality are not evaluated in the main paper.

Relevance to Vision-Language Models

This work directly addresses multimodal hallucination—a central failure mode of VLMs—by showing that the truthfulness circuitry of a base LLM is preserved when it is extended into a vision-language model. For researchers tracking VLMs, the key insight is that hallucination mitigation can be inherited rather than re-derived per model: probing Vicuna-7B once yields gates transferable to LLaVA-1.5 and LLaVA-NeXT. The attention-overlay analysis connecting high Truth Score heads to visual evidence grounding provides mechanistic grounding for why VLMs hallucinate—low-scoring heads with diffuse attention dominate the residual stream—and suggests head-level interventions as a lightweight alternative to full fine-tuning or RLHF-based alignment for MLLMs.