Social Agents: Collective Intelligence Improves LLM Predictions

🕒 Published (v1): 2026-01-01 · Source: ICLR · Venue: ICLR 2026 · link

Why this paper was selected

Collective intelligence from social LLM agent ensembles outperforms individual models

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

Social Agents is a multi-agent framework that operationalizes the "Wisdom of Crowds" by instantiating an ensemble of LLM-based personas with diverse demographic and psychographic attributes, each independently scoring a stimulus before their outputs are aggregated via mean. Across 11 behavioral prediction tasks, the approach outperforms single-LLM baselines by an average of 30.5% on low-level tasks and 9.9% on high-level tasks, and often exceeds task-specific fine-tuned expert models without any task-specific training.

Problem

Standard LLM inference produces a single deterministic answer, ignoring the structured variation in human judgments that arises from demographic and psychographic diversity. Existing multi-call ensembles (repeated sampling) reduce variance but miss systematic between-group differences — the very structure that gives crowd wisdom its predictive power. Task-specific expert models require large labeled corpora and cannot generalize across domains.

Method

A Moderator selects \(N\) personas from a Persona Agent Factory, where each persona is defined by demographic attributes (age, gender, location) and psychographic attributes (interests, values, lifestyle). All \(N\) persona agents share the same LLM backbone but are conditioned on distinct profiles and prompted independently (no inter-agent communication). Each agent first generates a chain-of-thought rationale grounded in its persona, then outputs a quantitative prediction \(s_i\). The collective estimate is:

\[\hat{S} = \frac{1}{N} \sum_{i=1}^{N} s_i\]

For interpretability, a neutral unconditioned expert LLM synthesizes all rationales into a single collective explanation. Few-shot exemplars are retrieved via embedding similarity for applicable tasks. Default \(N=10\); ablations show diminishing returns beyond 10–20 calls per datum.

Tasks are stratified by Construal Level Theory (CLT): low-construal (perceptual/immediate: CTR, tweet engagement, webpage likability, surface attribute classification), medium-construal (ROAS prediction), and high-construal (long-term memorability, latent attribute classification: reason, persuasion, emotion).

Key Contributions

• Social Agents framework: multi-agent ensemble that instantiates the four Wisdom of Crowds pillars (diversity, independence, decentralization, aggregation) via persona-conditioned LLM agents.
• Task-agnostic generalization: consistent improvements across 11 heterogeneous behavioral prediction tasks without task-specific fine-tuning.
• Model-agnostic gains: 21.5% average improvement across 9 different LLMs/VLMs (proprietary and open-source, varying scales).
• Synthetic dataset release: persona-conditioned predictions, rationales, and definitions across all 11 behavioral tasks (ad CTR, webpage likability, etc.).

Results

• Webpage likability (Pearson \(r\), GPT-4o): +164.2% over No-Persona baseline; 78.4% KDE overlap with human distribution vs. 61.5% for No-Persona (Mean of 10 Trials).
• Ad CTR MAPE (GPT-4o, Creative): 34.7% reduction; ROAS MAPE (Real Estate): 39.8% reduction.
• Long-term video memorability (GPT-4o): +24.2% over No-Persona.
• Low-level tasks average: +30.5% across model backbones; high-level tasks average: +9.9%.
• Overall average across 9 models and 11 tasks: +21.5% over No-Persona baseline.
• Tweet engagement: +21.75% average; Ad CTR MAPE: 28.2% average reduction; ROAS MAPE: 27.9% average reduction; webpage likability Pearson \(r\): +97.95% average.
• Social Agents surpasses fine-tuned expert baselines (LCBM, XGBoost, Behavior-LLaVA) on most tasks; exception: Henry outperforms on memorability, though Social Agents still beats the No-Persona baseline there.
• Individual persona Pearson correlations with human judgments reach up to 0.71.

Limitations

• Gains are smaller on high-level/abstract reasoning tasks (9.9%) than low-level perceptual tasks (30.5%), suggesting persona diversity helps less when tasks require deep semantic elaboration.
• Performance plateaus after ~10–20 agents, limiting scalability benefits; experiments default to \(N=10\).
• Persona conditioning induces correlations through the shared model backbone, violating the strict independence assumption of classical Wisdom of Crowds theory.
• Evaluation is confined to behavioral/marketing prediction (ads, webpages, tweets); generalizability to other domains is undemonstrated.
• Henry still outperforms Social Agents on long-term memorability despite Social Agents' gains over the baseline, indicating task-specific training retains advantages in some high-level constructs.
• No analysis of when persona diversity actively hurts performance (e.g., tasks where crowd variance is noise, not signal).

Relevance to Agentic AI / LLM Agents

This paper directly demonstrates a practical multi-agent architecture where agent heterogeneity — enforced through persona conditioning rather than architectural differences — is the primary source of performance gain, distinguishing it from naive repeated sampling. It provides strong empirical evidence that structured diversity among agents improves aggregate accuracy in a model-agnostic way, a generalizable design principle for any multi-agent system that must approximate population-level judgments. The Construal Level Theory framing also offers a principled taxonomy for predicting when collective agent inference will yield large vs. modest gains, useful for deciding whether to invest in multi-agent orchestration for a given task. For researchers building agentic pipelines around human behavioral modeling, this work validates persona simulation as a scalable substitute for human annotation or fine-tuning.