Skip to content

Personalized Attacks of Social Engineering in Multi-turn Conversations: LLM Agents for Simulation and Detection

🕒 Published (v1): 2025-03-18 19:14 UTC · Source: Arxiv · Venue: COLM 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

SE-VSim is a dual-agent simulation harness that generates multi-turn chat-based social engineering (CSE) conversations by independently modeling attacker and victim LLM agents conditioned on attack goals and Big Five personality traits. The resulting 1,350-conversation dataset is used to evaluate and motivate SE-OmniGuard, a proof-of-concept personalized detector that outperforms all baselines. The work demonstrates that LLM-simulated CSE dynamics reproduce known personality-susceptibility correlations documented in behavioral psychology.

Problem

Existing CSE detection systems (1) are not grounded in established SE effect-mechanism frameworks (e.g., Wang et al. 2021), producing simulated conversations that diverge from real-world dynamics, and (2) treat sensitive-information exchange as the sole success signal, ignoring trust-building phases that precede extraction. Victim personality heterogeneity is also neglected, yielding one-size-fits-all defenses.

Method

SE-VSim frames conversation generation as a two-agent pipeline:

  • Attacker agent: conditioned via in-context learning on goal \(G_{se} = A_{intent} \oplus A_{role}\); response at turn \(i\) is \(A^i_{res} = F(C^{i-1}_{prior}, G_{se})\). Three attacker roles (Funding Agency, Journalist, Recruiter) targeting PII, financial data, or intellectual property.
  • Victim agent: conditioned on a Big Five personality profile \(P_{trait}\); \(V^i_{res} = H(C^{i-1}_{prior}, P_{trait})\).
  • Conversation pipeline: alternating turns updating shared context \(C^{t+1}_{prior} = C^t_{prior} + A^t_{res} + V^t_{res}\), terminated at a budget of \(i=10\) turns. Benign conversations generated by stripping malicious intent: \(G_{benign} = G_{se} - A_{intent}\).
  • Labels are injected deterministically from the generation process (no external annotation needed for malicious/benign). Attack success (1–3 scale) is validated with two human annotators plus GPT-4o-mini as a third annotator (Fleiss' \(\kappa = 0.796\)). Attack strategy multi-labels (Persuasion, Social Influence, Cognition/Attitude/Behavior) are assigned by GPT-4o-mini as LLM judge.

SE-OmniGuard (proof-of-concept detector) uses prior knowledge of victim personality, monitors evolving attacker strategy, and tracks information exchange dynamics to classify conversations as malicious or benign.

Key Contributions

  • SE-VSim: open dual-agent harness grounded in SE effect-mechanism theory, supporting personality-varied victim simulation.
  • Dataset: 1,350 multi-turn LinkedIn-style conversations (900 malicious, 450 benign) across 3 attacker roles Ă— 3 info types Ă— 5 Big Five trait profiles, with success and strategy annotations.
  • Empirical mapping of Big Five traits to attack susceptibility, validating simulated dynamics against behavioral research.
  • SE-OmniGuard: personality-aware, strategy-aware detector demonstrating the value of personalized defense.

Results

Evaluated on per-role splits (AF/JO/RE); baselines trained and tested within each role:

  • SE-OmniGuard: 0.813 overall accuracy, 0.815 overall F1 (vs. best baseline ConvoSentinel GPT-4o: 0.680 accuracy, 0.640 F1).
  • ConvoSentinel GPT-4o: 0.710/0.610/0.690 accuracy across AF/JO/RE; 0.710/0.540/0.660 F1.
  • GPT-4o Few-Shot: 0.668 accuracy, 0.517 F1 overall.
  • Llama-3 8B Few-Shot: 0.580 accuracy, 0.679 F1 overall.
  • LLM-human annotation agreement: Fleiss' \(\kappa = 0.796\) (GPT-4o-mini vs. two human annotators on 900 conversations).
  • SE-OmniGuard accuracy advantage is largest on fully successful attacks (see Figure 3).

Limitations

  • SE-OmniGuard is a proof-of-concept; no detailed architecture or training procedure is provided.
  • Dataset is entirely synthetic; no validation against real-world CSE corpora.
  • Conversation budget fixed at 10 turns; real CSE attacks may span many more exchanges.
  • Attack strategy annotation relies solely on GPT-4o-mini (no human ground truth) due to budget constraints.
  • Only LinkedIn-type professional contexts are covered; other platforms are out of scope.
  • Causal claims between personality traits and susceptibility are not established—correlational only.

Relevance to Harnesses / Meta-Harnesses

SE-VSim is a multi-agent simulation harness: it orchestrates two independently conditioned LLM agents (attacker, victim) through a shared context-update pipeline with deterministic termination criteria and automated label injection—a textbook harness pattern applied to adversarial conversation generation. The LLM-judge layer (GPT-4o-mini as third annotator for both success scores and strategy multi-labels) exemplifies a meta-harness annotation loop, where one model evaluates the outputs of the simulation pipeline. The work also demonstrates harness-as-dataset-factory: by varying agent conditioning (role, personality, intent), the harness enumerates a structured combinatorial space of scenarios, analogous to how meta-harnesses generate evaluation suites in other domains.