FHE-Agent: Automating CKKS Configuration for Practical Encrypted Inference via an LLM-Guided Agentic Framework

🕒 Published (v1): 2025-11-23 23:26 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

FHE-Agent is an LLM-guided agentic framework that automates the notoriously expert-heavy task of configuring CKKS fully homomorphic encryption (FHE) for privacy-preserving neural network inference. It wraps an existing FHE compiler (Orion) in a deterministic tool suite and orchestrates a hierarchy of specialized LLM agents through a three-phase, multi-fidelity search that treats expensive encrypted evaluations as a scarce resource. The result is a system that finds feasible 128-bit-secure configurations where one-shot heuristics fail.

Problem

Configuring CKKS-based encrypted inference requires jointly tuning ring dimension \(\log N\), modulus chain \(\log Q\), scale schedule, bootstrapping plan, and per-layer packing layouts — a tightly coupled, nonlinear space that requires deep cryptographic expertise. Existing FHE compilers (CHET, Orion, Fhelipe) emit a single configuration via fixed heuristics, often severely over-provisioning latency or failing entirely for deeper networks. No prior system exposes the backend diagnostics needed for automated, resource-aware search over this joint space.

Method

FHE-Agent decomposes the problem into three layers:

1. Deterministic tool suite — wraps Orion/Lattigo into four backend-agnostic tools: StaticAnalyzer (validates depth/security without running inference), LayerProfiler (cleartext simulation producing per-layer profile \(\text{prof}_i = (s_i, p_i, n_i)\) covering runtime fraction, slot utilization, rotation counts, noise margin), BootstrapScheduler (emits a depth/noise mask forbidding depth-increasing patches), and CostModel (\(\text{cost}(C) = \sum_i [\alpha \cdot \text{mul}_i + \beta \cdot \text{rot}_i + \gamma \cdot \text{boot}_i + \delta \cdot \text{mem\_cost}_i]\) with Phase B-calibrated coefficients).

2. Hierarchical multi-agent controller — five specialized LLM agents: InitAgent (proposes diverse global regime candidates), RegimeAgent (prunes via static/cleartext checks), GlobalTradeoffAgent (proposes scheme-level patches and identifies bottleneck layer set \(\mathcal{I}\)), LayerwiseAgent (fine-tunes only local packing of layers in \(\mathcal{I}\), forbidden from touching global \(C_{\text{global}}\)), and PatchGateAgent (admission control: gates expensive encrypted evaluations on static validity + precision gates + CostModel rank). Agents select from a fixed vocabulary of discrete optimization directions rather than freely editing configs.

3. Three-phase multi-fidelity workflow — Phase A: static + cleartext regime exploration; Phase B: calibrate CostModel coefficients with sparse FHE_LIGHT encrypted runs; Phase C: admission-controlled refinement around the calibrated baseline under a strict FHE evaluation budget. A persistent trace repository stores configs, metrics, and decisions for few-shot context retrieval on future runs.

Key Contributions

• Frames CKKS configuration for encrypted inference as a hierarchical, resource-constrained search problem \(C = (C_{\text{global}}, \{C^{(i)}_{\text{local}}\}_i)\) with a budget on expensive oracle calls.
• Deterministic, backend-agnostic tool suite that exposes per-layer diagnostics previously buried inside compiler passes.
• Multi-agent controller with enforced role separation: global agents cannot be overridden by local agents; local agents are constrained by the depth/noise mask.
• Three-phase multi-fidelity workflow that prunes the search space with cheap static/cleartext signals before committing encrypted evaluations.
• Bottleneck scoring function (Eq. 6) combining runtime fraction, slot utilization, rotation count, and noise margin to focus agent attention on high-impact layers.

Results

• On standard benchmarks (MLP, LeNet, LoLa) and a deeper architecture (AlexNet), FHE-Agent consistently finds feasible 128-bit-secure configurations.
• Baseline one-shot heuristics (Orion defaults) and one-shot LLM prompting both fail to produce a valid configuration for AlexNet; FHE-Agent succeeds.
• FHE-Agent achieves better precision and lower encrypted inference latency than naïve FHE parameter sweeps across all tested models.
• (Note: the paper text provided is truncated before the quantitative results table; specific latency numbers are not available in the supplied excerpt.)

Limitations

• Evaluated only on Orion+Lattigo; generalization to other CKKS backends (e.g., OpenFHE, SEAL directly) is not demonstrated.
• The discrete patch vocabulary must be hand-crafted per backend; novel backends require new vocabulary design.
• The CostModel is a linear approximation (Eq. 5); it may mis-rank candidates when interaction effects between primitives are nonlinear.
• Targets semi-honest threat model only; does not address side-channel or traffic-analysis attacks.
• The text is truncated, so full ablation results, wall-clock optimization times, and cost breakdowns are not assessable from the provided excerpt.

Relevance to Harnesses / Meta-Harnesses

FHE-Agent is a textbook domain-specific meta-harness: it wraps an existing black-box compiler (Orion) in a structured tool suite, then places an LLM orchestration layer on top that manages evaluation fidelity, budget allocation, and agent role separation — the same architectural concerns that define general-purpose meta-harnesses. The three-phase multi-fidelity loop (cheap static → simulation → expensive execution) is a concrete instance of the harness pattern of guarding costly oracle calls behind progressively tighter filters. The trace repository for few-shot reuse across deployments is analogous to harness-level memory, relevant to anyone designing harnesses that amortize expensive evaluations. For researchers tracking harness design, this paper demonstrates how to enforce hierarchical search structure and discrete action vocabularies to make LLM agent decisions auditable and safe within an automated pipeline.