ProjAgent: Procedural Similarity Retrieval for Repository-Level Code Generation

🕒 Published (v1): 2026-07-09 16:50 UTC · Source: Arxiv · link

Why this paper was selected

ProjAgent procedural similarity retrieval for repo-level codegen; practical agent tool-use

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

ProjAgent introduces procedural similarity as a retrieval signal for repository-level code generation, using LLM hidden-state projections in the reasoning subspace to find functions that implement similar computational procedures regardless of naming or domain vocabulary. It combines this with lexical/semantic retrieval and a static-analysis repair loop. On REPOCOD it achieves 41.14% Pass@1, outperforming retrieval-augmented baselines.

Problem

Existing retrieval methods for repository-level code generation (BM25, dense embeddings, dependency graphs) capture lexical, semantic, or structural similarity but miss functions that implement the same computational procedure under different identifiers or in unrelated modules. Because real repositories contain useful guard-clause, validation, or transformation patterns scattered across files that share no surface overlap with the target function, critical context goes unretrieved, causing LLMs to hallucinate APIs or violate project conventions.

Method

ProjAgent operates in three stages on top of Qwen2.5-Coder-14B-Instruct:

Offline preprocessing. A ranked subset of repository files is selected. Each context function is decomposed by the LLM into logical steps (description + code snippet). Steps are validated via docstring substring match, ROUGE-L ≥ 0.7, or LLM judgment, and code snippets are verified by embedding cosine similarity (threshold 0.75 using google/embedding-gemma-300m). For each valid grounded step, a reasoning-subspace projection is computed: the model reasons about the step, hidden states from the final layer of response tokens are projected onto the reasoning subspace \(\mathcal{S}_{\text{Reasoning}}\) obtained by SVD of the unembedding weight matrix \(W_{\text{unemb}} = U\Sigma V^\top\): $\(\text{proj}_R(h_l) = V_R^\top \cdot h_l\)$ To correct for anisotropy, PCA debiasing removes the mean \(\mu\) and first principal component \(\text{PC}_1\) from each projection: $\(\tilde{h} = (\text{proj}_R(h_l) - \mu) - (\text{proj}_R(h_l) - \mu)^\top \cdot \text{PC}_1 \cdot \text{PC}_1\)$ \(\mu\) and \(\text{PC}_1\) are estimated incrementally until stable (relative change \(\leq 10^{-3}\) for \(\mu\); cosine similarity \(\geq 0.99\) for \(\text{PC}_1\)).

Procedural retrieval. The target function is decomposed into target steps; each gets a projection. An iterative agentic workflow (max 5 rounds) retrieves top-20 candidate steps by cosine similarity (threshold 0.75), expands them to neighboring steps within context functions, validates them, and uses an LLM plan-confirmation step to decide if sufficient procedural context has been collected.

Semantic retrieval + generation. Conventional BM25/dense retrieval adds API and structural context. A conservative static-analysis feedback loop (compiler + static analyzer) iteratively repairs generated code.

Key Contributions

• Introduces procedural similarity as an explicit, previously unexplored retrieval dimension for repository-level code generation.
• Repurposes LLM hidden-state reasoning-subspace projections (originally proposed for hallucination detection) as procedural similarity representations, with a novel PCA debiasing step to address anisotropy.
• An agentic retrieval workflow that validates, expands, and confirms procedurally similar context across repository files.
• A conservative static-analysis feedback loop for iterative code repair post-generation.
• End-to-end evaluation on REPOCOD with ablations across all retrieval components.

Results

• ProjAgent: 41.14% Pass@1 on REPOCOD, outperforming existing retrieval-augmented generation baselines (specific baseline numbers not fully reproduced in the provided text excerpt).
• Procedural retrieval is shown via ablation (RQ3) to contribute meaningfully beyond semantic retrieval alone.
• Projection similarity (RQ2) effectively identifies procedurally related context across naming and domain boundaries where BM25 similarity was ~0.38 and embedding similarity ~0.59 in the illustrative example.

Limitations

• Offline preprocessing is computationally expensive; only the top-20 ranked repository files are preprocessed, potentially excluding relevant context in large repositories (e.g., Astropy has >16,000 functions).
• The projection similarity threshold (0.75) and energy threshold (0.98) were calibrated on a small sample (five target steps from Astropy), limiting generalizability.
• Relies on Qwen2.5-Coder-14B-Instruct for all LLM calls; procedural similarity representations are model-specific and may not transfer across architectures.
• Evaluated on a single benchmark (REPOCOD); broader validity across languages and repository styles is unverified.
• Agentic retrieval loop (max 5 iterations) introduces latency and non-determinism from repeated LLM calls.

Relevance to Agentic AI / LLM Agents

ProjAgent exemplifies an agentic RAG architecture where an LLM actively orchestrates multi-stage retrieval with validation and plan-confirmation loops, directly relevant to the design of tool-using coding agents. The core insight — that LLM reasoning-subspace representations encode transferable procedural knowledge useful for retrieval — extends beyond code generation and informs how agent memory or context banks could be indexed by behavioral similarity rather than surface text. It also contributes a concrete technique for reducing hallucination in agentic code execution by grounding generation in procedurally matched repository context, complementing work on iterative self-repair agents. The static-analysis feedback loop exemplifies the broader agentic pattern of tool-augmented iterative refinement.