FHE-Coder: Benchmarking Secure Agentic Code Generation for Fully Homomorphic Encryption

🕒 Published (v1): 2026-01-01 · Source: ICLR · Venue: ICLR 2026 · link

Why this paper was selected

Security-focused agentic code generation benchmark for fully homomorphic encryption

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

FHE-Coder is a three-phase agentic framework for generating cryptographically secure Fully Homomorphic Encryption (FHE) code from natural-language specifications. It addresses the failure of baseline LLM agents that produce functionally correct but cryptographically insecure code by combining formal parameter derivation, expert-enriched RAG retrieval, and an automated security verifier. Evaluated on 10 TFHE tasks across 4 LLMs, it achieves near-perfect Pass@1(func sec) scores while baselines score near zero.

Problem

Baseline LLM agents applied to FHE code generation suffer three distinct failure modes: (1) semantic ambiguity in translating natural-language intent to cryptographic constraints, (2) API hallucination and misuse due to sparse FHE-specific training data, and (3) cryptographic insecurity — agents frequently emit plaintext implementations that pass functional unit tests (Pass@k(func)) while violating ciphertext-only computation requirements. Standard evaluation metrics like Pass@k do not capture security, masking this failure entirely.

Method

FHE-Coder wraps an LLM agent in a ReAct loop with three integrated components:

1. FHE Prompt Formalizer: Extracts user intent, invokes the Lattice Estimator (Albrecht et al., 2015) to mathematically derive the security parameter \(\lambda\) and compatible LWE lattice dimensions, then uses a second LLM to produce a formal specification with Dafny pseudocode and ensure statements that enforce parameter constraints and ciphertext invariants.

2. FHE API RAG Retriever: An offline-prepared knowledge base of scheme-specific API documentation (TFHE/CKKS) converted to Doxygen format with structured @objective tags, enabling semantic retrieval of security-compliant code snippets (chunk size 600, overlap 120, embedded with text-embedding-3-small).

3. FHE Security Verifier: A four-stage automated pipeline (Compile → Functional → Security → Latency) that enforces LWE-based parameter bounds, mandates exclusive use of homomorphic APIs (no plaintext leakage), and verifies input encryption. Failures produce a structured Formal Error Report fed back to the agent; the loop runs up to 10 iterations.

For compositional tasks (e.g., MLP, CNN, Transformer), a structured decomposition strategy is applied: verified secure primitives (e.g., dot product) are generated first, then a "composer" agent assembles them into the target architecture.

Key Contributions

• FHE-Coder agentic framework with three novel components: Prompt Formalizer (Lattice Estimator-grounded), Expert-Enriched RAG Retriever, and automated Security Verifier
• New evaluation metric Pass@1(func sec): a program is secure iff it (i) uses only FHE APIs, (ii) configures parameters within LWE-secure bounds, and (iii) encrypts all inputs before use
• Benchmark of 10 FHE tasks spanning TFHE primitives (AND, ReLU, Adder, Multiplier), linear algebra (vector/matrix ops), and deep learning components (MLP, CNN, Softmax, Attention, Transformer)
• Demonstration that security deficiency of baseline agents is model-agnostic across GPT-5, Gemini-2.5-Pro, Deepseek-V3.1, and Qwen3-Coder-480B
• Cross-scheme generalization: plug-and-play adaptation from TFHE to CKKS by swapping the documentation corpus and parameter estimator

Results

• Security (baseline): BAS and COT achieve Pass@1(func sec) \(\approx 0.0\) across all 10 tasks and all 4 LLMs
• Security (FHE-Coder): Near-perfect Pass@1(func sec) on primitive and linear algebra tasks for all models; GPT-5 and Gemini-2.5-Pro outperform Deepseek-V3.1 and Qwen3-Coder on complex tasks
• Structured decomposition (TFHE): Pass@1(func sec) of 0.70 for matrix-vector multiplication, 0.60 for CNN, ~0.35 for MLP; without decomposition, scores are near zero on these tasks
• Non-linear TFHE architectures: Perfect Pass@1(func sec) on Softmax and Attention for GPT-5; Transformer achieves ~0.40 for GPT-5; DSK fails on Softmax
• CKKS generalization: Perfect Pass@1(func sec) on Attention for both GPT-5 and Gemini; near-zero for all baselines
• Latency overhead: Compile and Security checks complete in milliseconds; functional check runtime is dominated by FHE circuit execution (seconds to minutes), not framework overhead

Limitations

• Maximum 10 ReAct iterations; complex tasks may not converge within this budget
• Performance ceiling is partially determined by the base LLM's intrinsic reasoning — Transformer-level FHE circuits remain partially unsolved (GPT-5 ~0.40 Pass@1(func sec))
• Benchmark covers only TFHE and CKKS; BGV and other schemes are not evaluated
• RAG corpus requires one-time human expert annotation (Doxygen enrichment) per scheme, adding a manual preparation cost
• Latency evaluation is relative to expert-written reference code, and FHE-Coder inherits the execution cost of the generated FHE circuits themselves
• Results are averaged over 5 runs but at temperature 0.5, leaving some variance uncharacterized

Relevance to Agentic AI / LLM Agents

FHE-Coder is a case study in domain-specialized agentic code generation where standard functional metrics are insufficient and a custom verifier must close the feedback loop — a pattern broadly applicable to any security- or correctness-critical synthesis task. The work demonstrates that ReAct-based agents augmented with tool-grounded parameter derivation (Lattice Estimator) and schema-enriched RAG can reliably operate in domains where naive LLMs hallucinate or violate hard constraints, extending the known frontier of what agentic systems can be trusted to generate. The structured decomposition strategy — verified sub-task composition — is a reusable technique for compositional agentic tasks where monolithic generation fails. The introduction of Pass@1(func sec) as a dual-axis correctness metric (functional AND property-verified) is a methodological contribution relevant to any agentic evaluation framework targeting safety or security properties.