ProjAgent: Procedural Similarity Retrieval for Repository-Level Code Generation¶

🕒 Published (v1): 2026-07-09 16:50 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

ProjAgent augments repository-level code generation by introducing procedural similarity as an explicit retrieval signal: it projects LLM hidden states onto a reasoning subspace to find functions that implement analogous computational procedures regardless of naming or domain vocabulary. It combines this with conventional lexical/semantic retrieval and an iterative static-analysis repair loop. On REPOCOD it achieves 41.14% Pass@1, outperforming existing retrieval-based baselines.

Problem¶

Existing retrieval methods for repository-level code generation (BM25, dense embeddings, structural/data-flow analysis) rely on surface-level lexical or semantic similarity. Critical helper functions that implement the same computational pattern as the target function—input validation, state transformation, unit conversion—are missed when they share no lexical overlap (e.g., BM25 similarity 0.38) and low semantic similarity (embedding similarity 0.59) despite sharing a guard-clause procedure. The gap is that no prior system treats procedural logic as a first-class retrieval dimension.

Method¶

Offline preprocessing.
For each repository, ProjAgent selects the top-20 files by benchmark target density, decomposes their functions into grounded logical steps (LLM-generated description + code snippet, validated via substring match → ROUGE-L ≥ 0.7 → LLM judgement, and snippet cosine similarity ≥ 0.75 via google/embedding-gemma-300m), and computes a reasoning-subspace projection for each step:

\[\text{proj}_R(h_l) = V_R^\top \cdot h_l\]

where \(V_R\) are the right singular vectors of the unembedding weight matrix \(W_\text{unemb} = U\Sigma V^\top\) corresponding to the low-energy (non-dominant) singular values (energy threshold 0.98, targeting ≈5% of hidden-state dimensionality). To counteract anisotropy, a PCA-based debiasing step removes the mean \(\mu\) and first principal component \(\text{PC}_1\) estimated by iterative stabilization (relative change \(\leq 10^{-3}\) for \(\mu\); \(|\cos(\text{PC}_{1,\text{old}}, \text{PC}_{1,\text{curr}})| \geq 0.99\)):

\[\tilde{h} = (\text{proj}_R(h_l) - \mu) - (\text{proj}_R(h_l) - \mu)^\top \cdot \text{PC}_1 \cdot \text{PC}_1\]

Procedural retrieval (online).
The target function is similarly decomposed into steps and projected. For each target step, candidate context steps are ranked by cosine similarity (threshold 0.75, top-20 retained), then expanded by an agentic search loop (up to 5 iterations) that validates and uses an LLM plan-confirmation step to judge whether sufficient procedural context has been collected.

Semantic retrieval. BM25 and dense retrieval capture APIs, class definitions, and utility functions not recoverable from procedural similarity.

Code generation and repair. A conservative static-analysis feedback loop (compiler + linter) iteratively repairs generated code until it passes or a budget is exhausted. Backbone: Qwen2.5-Coder-14B-Instruct.

Key Contributions¶

Introduces procedural similarity as a novel retrieval dimension for repository-level code generation, complementary to lexical and semantic signals.
Repurposes LLM reasoning-subspace projections (originally used for hallucination detection) to represent and compare the procedural logic of function steps.
Proposes a PCA-based debiasing technique to resolve anisotropy in projection-based cosine similarity.
Designs ProjAgent: a multi-stage agentic pipeline (offline projection construction → iterative procedural retrieval → semantic retrieval → generation + static-analysis repair).
Evaluates component contributions via ablation on REPOCOD.

Results¶

41.14% Pass@1 on REPOCOD with Qwen2.5-Coder-14B-Instruct backbone, outperforming all retrieval-augmented generation baselines evaluated.
(Full per-baseline comparison numbers were not present in the provided text excerpt.)

Limitations¶

Preprocessing cost scales with repository size (mitigated by top-20 file heuristic, but Astropy has >16,000 functions); the selection heuristic may discard relevant files.
Cosine similarity threshold (0.75) and file count (top-20) were calibrated empirically on five Astropy target steps; generalizability across repositories is not established.
Evaluation is restricted to a single benchmark (REPOCOD) and a single backbone model (Qwen2.5-Coder-14B-Instruct, Python only).
The reasoning-subspace energy threshold (0.98 → ≈5% dimensionality) is model-dependent and must be re-tuned for other LLM families.
The agentic loop has a hard cap of five iterations; failure cases where procedural context is never confirmed are not deeply analyzed.

Relevance to Harnesses / Meta-Harnesses¶

ProjAgent is itself a multi-stage agentic harness: offline preprocessing feeds into an iterative retrieval agent loop, which feeds into a semantic retrieval stage, which feeds into a generation-and-repair loop—each stage is a distinct orchestrated component with its own validation and termination logic. The iterative procedural-retrieval loop (seed → agentic expansion → LLM plan confirmation, up to 5 rounds) is a canonical harness pattern where an orchestrator decides dynamically whether sub-task completion criteria are met before advancing. For researchers tracking harness architectures, ProjAgent is notable for combining offline index construction, online agentic retrieval with dynamic stopping, and post-generation static-analysis repair into a single coordinated pipeline, demonstrating how domain-specific retrieval signals can be surfaced through harness orchestration rather than requiring end-to-end model training.