Skip to content

TrustGen: A Platform of Dynamic Benchmarking on the Trustworthiness of Generative Foundation Models

🕒 Published (v1): 2026-01-01 · Source: ICLR · Venue: ICLR 2026 · link

Why this paper was selected

ICLR 2026; TrustGen: dynamic benchmarking platform for generative model trustworthiness

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

TrustGen is a dynamic, modular benchmarking platform for evaluating the trustworthiness of generative foundation models (GenFMs) across text-to-image, LLM, and vision-language modalities. It covers 25+ fine-grained trust dimensions via three automated pipeline modules and evaluates 39 models. Key findings: most top models exceed a 70/100 trust score but show persistent gaps in hallucination, fairness, and privacy; open-source models now rival proprietary ones.

Problem

Existing trustworthiness evaluations are fragmented by modality (T2I, LLM, VLM evaluated in isolation), rapidly outdated by static benchmarks that models can memorize, and lack a unified taxonomy or extensible architecture for cross-model comparison. There is no standard, continuously evolving system that covers diverse trust dimensions at scale.

Method

TrustGen provides three composable pipeline modules:

  1. Metadata Curator — populates evaluation metadata via three strategies: (a) a web-browsing agent powered by LLMs to retrieve current information, (b) a dataset pool maintainer that ingests existing structured datasets (CSV/JSON), and (c) model-based data generators that produce metadata components (not full test cases, to avoid self-enhancement bias).

  2. Test Case Builder — constructs test instances using either model-based operations (LLM-guided fuzzification, attribute-guided generation, principle-guided generation, paraphrasing) or programmatic operations (noise perturbation, template instantiation, scene graphs). Ground-truth labels are generated deterministically; LLMs only paraphrase, preventing label hallucination.

  3. Contextual Variator — diversifies prompts by transforming question format (open-ended / multiple-choice / true-false), adjusting sentence length, and paraphrasing vocabulary/syntax, all validated for semantic consistency via human evaluation.

Evaluation uses LLM-as-a-Judge / VLM-as-a-Judge against ground-truth labels, with dimension-specific metrics (accuracy for hallucination, refuse-to-answer rate for jailbreak, win rate for robustness). All metrics are normalized to \([0, 100]\), inverted where lower is better, and averaged hierarchically (task → sub-dimension → dimension → overall score).

Seven top-level trust dimensions are assessed: Truthfulness, Safety, Fairness, Robustness, Privacy, Machine Ethics, and Advanced AI Risk, with coverage varying by modality (e.g., Machine Ethics is not applied to T2I models).

Key Contributions

  • First dynamic benchmarking platform spanning T2I, LLM, and VLM modalities under a unified 25+ dimension taxonomy
  • Three-module pipeline (Metadata Curator, Test Case Builder, Contextual Variator) enabling automated, iterative, and memorization-resistant evaluation dataset generation
  • Comprehensive evaluation of 39 GenFMs (8 T2I, 21 LLM, 10 VLM) with per-dimension trustworthiness scores
  • Public dataset on Hugging Face and open-source toolkit with leaderboard integration

Results

  • Majority of evaluated models across all three categories achieve overall trustworthiness scores >70/100, indicating broad alignment with key trust dimensions
  • Open-source models rival proprietary ones: CogView-3-Plus (open-source T2I) outperforms DALL-E 3; Llama-3.2-70B matches GPT-4o on trustworthiness metrics
  • Convergence trend: score differences among top-performing models are generally <10 points, narrowed compared to prior iterations
  • Persistent bottlenecks remain in hallucination resistance, fairness, and privacy preservation across modalities
  • T2I models show marked variation in NSFW generation resistance; privacy leakage rates differ substantially between models and between organizational vs. individual content types
  • Some LLMs exhibit exaggerated safety (over-refusal on benign queries), negatively impacting helpfulness

Limitations

  • Detailed per-model and per-dimension numerical results are deferred to the appendix (not in the main text)
  • Self-enhancement bias is mitigated but not fully eliminated when LLMs generate paraphrases of test cases for models from the same family
  • Not all trust dimensions apply to all modalities; cross-modality comparisons are inherently asymmetric
  • Model-as-judge evaluation introduces its own reliability concerns, despite human validation of judge quality
  • Dynamic generation relies on LLM web-browsing agents whose retrieved metadata quality is not fully controlled

Relevance to Vision-Language Models

TrustGen includes a dedicated VLM evaluation track covering hallucination, jailbreak, robustness, privacy, and stereotype/disparagement for 10 state-of-the-art VLMs, making it directly applicable to researchers tracking VLM safety and alignment. The dynamic benchmark design is especially relevant to VLMs, which rapidly gain multimodal capabilities and expose new attack surfaces (e.g., visual jailbreaks, cross-modal privacy leakage) that static benchmarks cannot track. The VLM-as-a-Judge evaluation methodology also contributes to the emerging literature on using VLMs for automated assessment. Findings on the narrowing trustworthiness gap between open-source and proprietary VLMs inform deployment decisions in safety-critical settings.