Skip to content

AI Hiring with LLMs: A Context-Aware and Explainable Multi-Agent Framework for Resume Screening

🕒 Published (v1): 2025-04-01 12:56 UTC · Source: Arxiv · Venue: CVPR 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

This paper presents a four-agent pipeline (extractor → evaluator → summarizer → score formatter) for automated resume screening, augmented with RAG to inject company-specific hiring criteria without fine-tuning. The framework is evaluated against HR-professional scores on 105 anonymized resumes, where the RAG-augmented multi-agent system outperforms single-LLM baselines on Pearson/Spearman correlation and MAE.

Problem

Existing LLM-based resume screeners are monolithic single-call systems: all parsing, scoring, and feedback generation are coupled, making it expensive to adapt scoring criteria across roles/companies and opaque to recruiters who cannot trace why a score was assigned. Rule-based and embedding-only systems lack contextual adaptability to evolving job markets.

Method

A sequential four-agent pipeline is implemented with CrewAI + LangChain:

  1. Resume Extractor — LLM parses unstructured résumé text into six structured fields (position, self-evaluation, skills, work experience, basic info, education).
  2. Resume Evaluator — RAG pipeline (OpenAI embeddings + ChromaDB, cosine similarity threshold \(\tau = 0.3\)) retrieves company/role-specific documents and constructs a prompt \(P = \text{concat}(Q, J, C)\) to score five dimensions producing \(S^J = \{S^J_S, S^J_K, S^J_W, S^J_B, S^J_E\}\). Final score: \(S^J_{\text{final}} = \sum w_i S^J_i\) with fixed weights \(\sum w_i = 1\).
  3. Resume Summarizer — three sub-agents (CEO, CTO, HR) debate candidate strengths/weaknesses and produce structured natural-language feedback.
  4. Score Formatter — normalizes raw scores to a uniform numerical array for downstream ranking.

Key Contributions

  • Modular four-agent pipeline that decouples extraction, evaluation, summarization, and formatting, improving traceability and independently reconfigurable.
  • RAG-augmented evaluator enabling real-time, fine-tuning-free adaptation to company-specific hiring criteria (certifications, university rankings, past-hire profiles).
  • Role-specific contextual scoring: the same résumé receives different scores depending on applied position level (e.g., intern vs. director).
  • Ablation demonstrating that the extraction agent alone measurably boosts downstream evaluation quality.
  • Discussion of bias, fairness, and regulatory considerations for AI hiring.

Results

  • Best configuration: RAG-LLM with DeepSeek-V3 backbone.
  • \(PC_{10} = 0.84\), \(SC_{10} = 0.74\) (p < 0.001), \(\text{MAE} = 0.90\) vs. HR ground truth.
  • Single-LLM DeepSeek-V3 baseline: \(PC_{10} = 0.70\), \(SC_{10} = 0.71\), \(\text{MAE} = 1.08\).
  • RAG-LLM consistently outperforms single-LLM across all percentile thresholds (top/bottom 10%, 15%, 20%).
  • Score distribution alignment: HR mean = 7.68, LLM mean = 7.76.
  • Ablation (Table 2): adding the extraction agent improves \(PC_{20}\) from 0.65 → 0.70 and reduces \(\text{MAE}\) for DeepSeek-V3.

Limitations

  • Dataset is small (105 résumés), domain-narrow (HR roles only), and single-source, limiting generalizability.
  • Ground-truth labels are themselves subjective HR scores; inter-rater agreement is not reported.
  • Fixed criterion weights \(w_i\) across all job roles are hand-set, not learned.
  • Cosine similarity threshold \(\tau = 0.3\) is set by fiat with no ablation.
  • Evaluation focuses on top/bottom percentile candidates; borderline candidates are explicitly excluded from the primary analysis.
  • Privacy constraints in real deployments may require local LLM substitution (Ollama suggested but not evaluated).

Relevance to Harnesses / Meta-Harnesses

This paper is a domain-specific instance of the sequential multi-agent harness pattern: a fixed topology of specialized agents, each with a narrow contract, wired together by a coordinator (CrewAI), with RAG serving as a pluggable external-knowledge injection layer. The four-agent pipeline mirrors structural choices seen in general-purpose meta-harnesses — separation of extraction, reasoning, synthesis, and formatting — making it a concrete case study for how harness topology affects output quality and explainability. The ablation isolating the extractor agent's contribution is directly analogous to ablations in meta-harness research that evaluate individual stage value. The RAG-as-adapter pattern (replacing fine-tuning with retrieval to swap domain context at runtime) is a recurring design principle relevant to building adaptable harnesses without retraining.