Skip to content

From Visual Perception to Deep Empathy: An Automated Assessment Framework for House-Tree-Person Drawings Using Multimodal LLMs and Multi-Agent Collaboration

🕒 Published (v1): 2025-12-23 09:26 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

This paper builds an automated assessment framework for the House–Tree–Person (HTP) projective drawing test using multimodal LLMs and a four-role multi-agent pipeline. Study 1 validates that Qwen-VL-Plus achieves ~0.75 cosine similarity with human expert interpretations across 307 drawings. Study 2 extends this with a multi-agent system that decouples visual feature extraction from psychological inference, adds social context, and produces empathic clinical reports.

Problem

HTP scoring has no unified standard: the same symbol (e.g., a chimney) is interpreted differently across clinical schools, and even the 39 drawing features associated with mental disorders lack agreed measurement frameworks. Single MLLMs produce generic, hallucination-prone outputs and lack empathy or sociocultural grounding—limiting clinical utility.

Method

Study 1: 307 anonymised HTP drawings with expert interpretations are fed to Qwen-VL-Plus via a standardised prompt. Both model outputs and expert texts are embedded with Doubao-embedding-vision (2048-dim vectors); cosine similarity measures AI–expert alignment.

Study 2: A four-role multi-agent system built on the proprietary Qingliu platform: - Observer — extracts >150 structured visual features (line quality, spatial layout, proportions) as JSON. - Interpreter — maps JSON features to psychological meanings via Buck/Hammer theory and developmental psychology, building evidential chains. - Zeitgeist Observer — situates findings in macro social/cultural context using real statistics to reduce pathologisation. - Listener — synthesises a warm, action-oriented report using positive psychology framing.

An additional fusion layer runs three independent MLLMs (Gemini-3-Pro-Preview, Qwen3-VL-Plus, Doubao-Seed-1-6-Vision) in parallel, then uses DeepSeek-v3.1-Terminus to merge outputs under four principles (Directness, Evidence, Caution, Practicality), classifying consensus findings vs. unique observations.

Key Contributions

  • Empirical baseline showing MLLMs achieve ~0.75 cosine similarity with HTP expert interpretations at scale (n=307), with structuralist expert data reaching 0.85.
  • A role-decomposed multi-agent architecture that separates feature recognition, psychological inference, social contextualisation, and empathic reporting into discrete agents.
  • A multi-model fusion protocol using three vision-language models + one text model to cross-validate and robustify clinical interpretations.
  • Qualitative demonstration across three cases (IT worker, doctoral student, mental-health practitioner) showing how the system corrects hallucinations and avoids over-pathologisation.

Results

  • Overall mean cosine similarity (307 cases): 0.748 (SD=0.058); >82% of samples exceed 0.70.
  • Structuralist expert (Wang Long, n=178): mean similarity 0.787, multiple cases >0.85.
  • Imagistic experts (Yan Hu, Zhang Tongyan): means 0.698 and 0.674, respectively; some cases as low as 0.556.
  • No held-out quantitative benchmark for the multi-agent system; evaluation is qualitative (ecological validity, internal coherence, hallucination correction) across ~30 real-world cases.

Limitations

  • Qwen-VL-Plus degrades on abstract or highly artistic drawings; domain fine-tuning has not been performed.
  • Dataset is adult, Chinese-population only; generalisability across ages and cultures is untested.
  • The multi-model fusion and multi-agent scheduling add complexity with no efficiency analysis or latency/cost reporting.
  • No ground-truth clinical outcomes (diagnoses, therapeutic outcomes) to validate psychological inferences; similarity to expert text is a proxy, not a clinical validity measure.
  • Study 2 evaluation is case-study-only; no systematic quantitative comparison between single-model and multi-agent outputs.

Relevance to Harnesses / Meta-Harnesses

This paper is a domain-specific instance of a multi-agent meta-harness: it orchestrates a fixed pipeline of specialised agents (Observer → Interpreter → Zeitgeist Observer → Listener) where each agent's output is the next agent's structured input, closely analogous to harness patterns that chain tools with typed intermediate representations. The additional parallel-model fusion layer—running N independent models then invoking a meta-model to merge and classify consensus vs. divergence—is a direct example of ensemble orchestration logic found in meta-harness designs. For researchers tracking harnesses, the Qingliu platform's role definitions, workflow orchestration, context management, and parallel task processing are the operative engineering contribution, not the psychology domain itself. The explicit decomposition of "what to observe" from "how to infer" from "what report to generate" mirrors separation-of-concerns patterns in production harness architectures.