VulnLLM-R: Specialized Reasoning LLM with Agent Scaffold for Vulnerability Detection

🕒 Published (v1): 2025-12-08 13:06 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

VulnLLM-R is a 7B specialized reasoning LLM for vulnerability detection, trained via knowledge distillation from DeepSeek-R1 and QwQ-32B with a custom pipeline of data selection, rejection sampling, and constitution-based correction. An agent scaffold wraps the model with CodeQL-based context retrieval to scale from function-level to project-level analysis. The 7B model outperforms much larger commercial reasoning models and discovers 15 zero-day vulnerabilities in actively maintained repositories.

Problem

General-purpose reasoning LLMs are too large, closed-source, or lack domain-specific security knowledge for practical vulnerability detection. Classical ML/DL models degrade to pattern matching, fail on OOD CWEs and languages, and cannot handle project-scale context. Static tools (CodeQL, AFL++) remain the de facto standard despite limited coverage.

Method

Training recipe (SFT-based distillation): 1. Data selection: curates 6 datasets (Juliet 1.3, ARVO, PrimeVul, SecCodePLT, SVEN, SecLLMHolmes) with CWE coverage and scale diversity (function-level to multi-file); deduplication via 20-token n-gram matching; LLM-assisted label validation. 2. Reasoning data generation: queries DeepSeek-R1 and QwQ-32B (8 samples each per datum) as dual teacher models to diversify reasoning structures. 3. Filtering and correction: rejection sampling (keep shortest correct-answer chain); for all-fail cases, applies 73 manually written per-CWE "constitutions" as additional teacher prompts, recovering ~30% more training samples. 4. Summary-based training: two-stage SFT — first on full reasoning chains, then on DeepSeek-R1-summarized chains — to reduce verbosity. 5. Testing-phase optimization: truncated generation (append "Final Answer" token at fixed length); policy-based generation (4 preliminary samples yield a CWE candidate set; a final query constrains output to that set).

Agent scaffold: Wraps VulnLLM-R with a context-retrieval tool. For each function, three randomly sampled call-graph paths from the project entry to that function are prepended as context; the model can call a tool to retrieve additional function bodies by name within a bounded number of rounds. The model is then fine-tuned further on agentic traces generated by o3 (with rejection sampling) to learn tool-calling behavior from real project repositories.

Key Contributions

• First specialized reasoning LLM (7B) for vulnerability detection with a complete training recipe.
• Dual-teacher distillation (DeepSeek-R1 + QwQ-32B) to diversify reasoning and balance FP/FN bias.
• Constitution-based correction: CWE-specific guidance injected into teacher prompts to recover rejected training samples.
• Summary-based training for concise reasoning without performance loss.
• Policy-based generation as a testing-phase strategy reducing CWE classification complexity.
• Agent scaffold with CodeQL-based context retrieval, fine-tuned on agentic traces for project-level analysis.
• Discovery of 15 zero-day vulnerabilities in 5 actively maintained repositories.

Results

• Function-level F1: VulnLLM-R (7B) exceeds GPT-o3, DeepSeek-R1, QwQ-32B, Claude-3.7-Sonnet, and gpt-o3-oss-120B on the combined Python/C/C++/Java benchmark (exact F1 scores visible in Figure 1 only; paper states SOTA among all compared models).
• OOD generalization: generalizes to unseen CWEs and Java, whereas the non-reasoning base model (Qwen2.5-7B-Instruct) fails — confirming the gain comes from reasoning training, not base model knowledge.
• Testing-phase scaling: unlike general coding/math, longer reasoning chains and parallel sampling do not improve performance; truncated generation and policy-based generation are preferred.
• Project-level agent: outperforms CodeQL, AFL++, RepoAudit, and G2 Fuzz on 5 real-world C/C++ and Java projects.
• Zero-day discovery: 15 previously unknown vulnerabilities confirmed in the latest versions of 5 popular repositories.
• Ablation: each component of the training recipe (dual teachers, rejection sampling, constitution correction, summary training) contributes positively to final F1.

Limitations

• Constitution writing is manual: 73 constitutions across 19 CWEs require human security expertise to author and maintain.
• RL training was attempted but abandoned due to reward hacking (model defaults to predicting "benign" to avoid penalty for wrong CWE labels); left to future work.
• Agent scaffold bounds retrieval rounds for speed, which may miss deep call-chain context.
• Project-level agentic training data sourced from a small set of C/C++ and Java repos; generalization to other languages/ecosystems is unverified.
• Testing-phase scaling (parallel samples, long reasoning) provides no benefit — limits the applicability of inference-time compute scaling used in other domains.
• Paper text truncated before full quantitative results tables are presented.

Relevance to Harnesses / Meta-Harnesses

VulnLLM-R's agent scaffold is a concrete example of a task-specific harness: a thin orchestration layer that wraps a core model with tool retrieval (call-graph context), enforces round limits, and produces structured outputs for downstream triage. Critically, the scaffold is not just inference infrastructure — the model is co-trained on agentic traces, making the harness a first-class component of the training pipeline, which mirrors the meta-harness pattern of generating specialized training data through orchestrated model-tool interactions. The dual-teacher distillation loop (generate → filter → correct → re-query) is itself a data-generation meta-harness that produces high-quality domain-specific reasoning chains at scale. For researchers tracking harness design, this paper demonstrates that domain adaptation requires both a tailored inference harness and harness-aware fine-tuning rather than treating the two as independent concerns.