Skip to content

DialogGuard: Multi-Agent Psychosocial Safety Evaluation of Sensitive LLM Responses

🕒 Published (v1): 2025-12-01 23:53 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

DialogGuard is a model-agnostic multi-agent evaluation framework that wraps arbitrary LLMs with four distinct LLM-as-a-judge pipelines to assess psychosocial safety across five dimensions (privacy, discrimination, psychological harm, mental manipulation, insulting behaviour). Evaluated against human annotations from PKU-SafeRLHF, multi-agent mechanisms consistently outperform single-agent and non-LLM baselines. Dual-agent correction and majority voting achieve the best accuracy/alignment trade-off; multi-agent debate yields higher recall but over-flags borderline cases.

Problem

Existing LLM safety evaluation is dominated by broad content-policy categories (toxicity, hate speech) using single-judge architectures, and does not systematically cover fine-grained psychosocial risks in sensitive conversational settings (mental health, crisis intervention). No prior work compares alternative judging architectures—single-agent, corrective-pair, debate, and majority vote—on these dimensions, nor provides a model-agnostic evaluation wrapper deployable around black-box APIs without retraining.

Method

DialogGuard defines a three-level (0–2) scoring rubric for each of five psychosocial safety dimensions, shared by human annotators and LLM judges. Four evaluation pipelines are implemented:

  1. Single-agent scoring: \(s = A(x; \theta)\) — one judge, no feedback.
  2. Dual-agent correction: Agent \(A_1\) scores, Agent \(A_2\) sees \(A_1\)'s score and reasoning and issues a correction; final score is a weighted combination \(s^* = w_1 s_1 + w_2 s_2\), \(w_1 + w_2 = 1\).
  3. Multi-agent debate: Two LLM debaters (\(D_{\text{aff}}\), \(D_{\text{chal}}\)) argue over \(R\) rounds with randomized speaking order; an impartial judge \(J\) scores after each round and early-stops when score std across \(J_e\) votes falls below threshold \(\tau\); final score is the sample median.
  4. Stochastic majority voting: \(K\) independent calls to the same model at nonzero temperature yield \(s_k \in [0,1]\); binary votes are thresholded and majority-aggregated: \(\hat{y} = \mathbb{I}[\bar{y} \geq \tfrac{1}{2}]\).

Experiments use PKU-SafeRLHF (200 risky + 200 safe examples per dimension, \(N=400\)) with three diverse judge LLMs (DeepSeek-V3.2-Exp, GPT-4o-mini, Qwen-Plus). Baselines are a lexicon rule-based classifier and zero-shot BART-MNLI.

Key Contributions

  • Five operationalised psychosocial safety dimensions with a unified 0–2 rubric for both human and LLM annotation, grounded in PKU-SafeRLHF harm categories.
  • Systematic empirical comparison of four LLM-as-a-judge architectures plus two non-LLM baselines across all five dimensions, revealing a consistent performance hierarchy.
  • Open-source DialogGuard framework (code + graphical web interface) exposing per-dimension risk scores and natural-language rationales.
  • Formative usability study with 12 practitioners validating that the framework supports prompt design, auditing, and decision-making in sensitive web deployments.

Results

  • Macro-averaged accuracy per dimension (best mechanism): Privacy Violation 82.7%, Mental Manipulation 84.7%, Psychological Harm 76.0%, Discriminatory Behaviour 77.2%, Insulting Behaviour 90.2% / 90.0% (figures in paper).
  • Multi-agent mechanisms consistently outperform both the rule-based lexicon baseline and BART-MNLI zero-shot baseline.
  • Dual-agent correction and majority voting (\(K=20\)) achieve the best trade-off across accuracy, Spearman \(\rho\), Pearson \(r\), and F1.
  • Multi-agent debate achieves higher recall but over-flags borderline cases (lower precision).
  • Single-agent scoring shows mild but consistent accuracy degradation as sampling temperature increases (robustness analysis on DeepSeek-V3.2-Exp).

Limitations

  • Evaluation dataset is balanced to 400 examples per dimension by subsampling, which may not reflect real-world class distributions.
  • Responses in PKU-SafeRLHF are generated by LLaMA-family models; generalizability to modern instruction-tuned models' outputs is untested.
  • Multi-agent debate and majority voting are substantially more expensive computationally than single-agent scoring; cost/accuracy trade-offs are not quantified.
  • Robustness analysis (temperature/weighting) uses only one model (DeepSeek-V3.2-Exp).
  • The five dimensions are a curated subset of 19 harm categories, omitting macro-level harms (economic crime, national security).
  • Formative practitioner study is qualitative (\(n=12\)) with no formal reliability measure.

Relevance to Harnesses / Meta-Harnesses

DialogGuard is structurally a meta-harness: a model-agnostic evaluation wrapper that can be attached to any LLM without modifying the base model, instantiating multiple judge sub-pipelines and aggregating their outputs. The four architectures it compares—single-agent, corrective-pair, debate, and majority-vote—directly map to a design space for harness orchestration strategies, with concrete empirical evidence for when each topology is preferable. The framework's modular, plug-in design (fixed LLM + swappable judge module + shared rubric) mirrors the broader harness pattern of separating task execution from evaluation/monitoring layers. For practitioners building harnesses over LLMs deployed in sensitive domains, DialogGuard provides both an empirically validated architecture choice and a reusable evaluation component.