Social Agents: Collective Intelligence Improves LLM Predictions

🕒 Published (v1): 2026-01-01 · Source: ICLR · Venue: ICLR 2026 · link

Why this paper was selected

Social multi-agent collective intelligence improves LLM prediction accuracy [ICLR 2026]

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

Social Agents is a multi-agent framework that operationalizes the Wisdom of Crowds by instantiating \(N\) diverse LLM persona agents (varied by demographic and psychographic attributes), having each independently score a stimulus, and aggregating via mean. Evaluated across 11 behavioral prediction tasks, it yields average improvements of 30.5% on low-level and 9.9% on high-level tasks over a single-LLM expert baseline, with gains robust across nine model backbones.

Problem

LLMs typically produce a single definitive answer, collapsing the subjective variance that characterizes real human populations. Existing approaches either rely on expensive human crowd-sourcing or train task-specific models on large labeled corpora (e.g., LCBM on 168M tweets), neither of which scales cheaply to new domains. Simple repeated sampling without persona conditioning reduces within-group noise but misses the systematic between-group variation that gives crowd wisdom its predictive power.

Method

A Moderator selects \(N\) (default 10) diverse persona agents from a Persona Agent Factory, where each persona is defined by demographic attributes (age, gender, location) and psychographic attributes (interests, values, lifestyle). Agents are prompted independently using the same backbone LLM but conditioned on distinct profiles — preventing groupthink. Each agent follows a chain-of-thought protocol: generate a rationale from its persona's viewpoint, then output a quantitative prediction. For regression tasks the aggregate is the simple mean \(\hat{S} = \frac{1}{N}\sum_{i=1}^{N} s_i\); for classification, majority vote. A neutral unconditioned expert LLM synthesizes the rationales into a single collective explanation. For CTR tasks, ad embeddings are used to retrieve top-\(K\) semantically similar ads as few-shot examples per agent.

Key Contributions

• Social Agents framework: multi-agent operationalization of Wisdom of Crowds (diversity, independence, decentralization, aggregation) using persona-conditioned LLM ensembles.
• Model-agnostic evaluation: consistent gains across nine proprietary and open-source LLMs/VLMs on 11 tasks spanning Construal Level Theory (CLT) low/medium/high construals.
• Empirical superiority over repeated sampling: persona conditioning yields 78.4% KDE overlap with human predictions on webpage likability vs. 61.5% for No-Persona (Mean of 10 Trials), demonstrating structured variation > stochastic noise.
• Synthetic dataset release: persona-conditioned predictions, rationales, and definitions across all 11 behavioral tasks.

Results

• GPT-4o webpage likability: +164.2% Pearson \(r\) over No-Persona baseline.
• GPT-4o long-term memorability: +24.2% over No-Persona baseline.
• GPT-4o Ad CTR (Creative): −34.7% MAPE; ROAS (Real Estate): −39.8% MAPE.
• Averaged across model backbones: +30.5% on low-level tasks, +9.9% on high-level tasks, +21.5% overall.
• Task-specific comparisons: Social Agents surpasses LCBM (trained on 40k videos + 168M tweets), XGBoost, and Behavior-LLaVA (730k instruction pairs) on most tasks; only Henry outperforms on memorability (though Social Agents still beats the No-Persona baseline there).
• Individual persona Pearson correlations with human judgments reach up to \(r = 0.71\).
• Both Social Agents and No-Persona (Mean of 10 Trials) plateau after ~10–20 calls per datum.

Limitations

• Gains on high-level tasks (abstract reasoning) are smaller (9.9% avg) than on low-level tasks (30.5%), suggesting diminishing returns as task complexity increases beyond surface perception.
• Henry outperforms Social Agents on long-term memorability despite Social Agents' improvements over baseline, indicating specialized trained models retain an edge on certain high-construal tasks.
• Persona conditioning introduces correlated (not fully independent) outputs because agents share the same backbone model, violating the strict independence assumption of classical crowd wisdom.
• Fixed \(N=10\) personas by default; the paper notes performance plateaus around 10–20 calls, so the approach cannot easily scale to crowd sizes that would further reduce variance.
• Psychographic diversity is template-driven and synthetic; whether the coverage maps faithfully to real population distributions is not validated.

Relevance to Harnesses / Meta-Harnesses

Social Agents is a concrete instance of a meta-harness pattern: a moderator layer orchestrates an ensemble of homogeneous sub-agents (same backbone, different conditioning), collecting their outputs and applying a deterministic aggregation rule. The Persona Agent Factory and Moderator roles directly parallel the dispatcher/router and reducer components common in meta-harness architectures. The key design insight — that structured prompt diversity across agents yields better aggregate outcomes than repeated sampling with the same prompt — is directly applicable to any harness that uses ensembles of LLM calls, suggesting that persona or role conditioning is a lightweight and model-agnostic mechanism for improving ensemble-based harness outputs without architectural changes.