Energy per Successful Goal: Goal-Level Energy Accounting for Agentic AI Systems¶

🕒 Published (v1): 2026-05-20 22:55 UTC · Source: Arxiv · link

Why this paper was selected

Goal-level energy accounting metric; new efficiency unit for multi-step agentic AI

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

Current AI energy benchmarks normalize by inference invocation, which systematically undercounts the true cost of agentic workflows where retries, planning steps, and failure-recovery cycles multiply energy consumption. This paper introduces A-LEMS, a measurement framework redefining the fundamental unit as Energy per Successful Goal (EpG), and demonstrates that agentic workflows consume 4.33× more energy per successful goal than linear baselines. Crucially, the overhead is driven by orchestration structure—not inference compute—and inverts below 1.0× for tool-augmented tasks where agentic dispatch eliminates costly token generation.

Problem¶

Energy-per-inference is a unit-misaligned metric for multi-step agentic systems: inference count is an implementation artifact (determined at runtime by retry sequences, conditional tool calls, and failure recovery), not a task property. This causes four systematic measurement failures: (1) the unit problem—retries are invisible, so a 5-retry system looks identical to a 1-retry system; (2) the boundary problem—post-task framework teardown inflates linear baseline energy more than agentic, artificially compressing the overhead ratio toward 1.0×; (3) the attribution problem—raw RAPL readings conflate idle draw and concurrent processes with workload energy; (4) the reproducibility problem—absent hardware/environment binding, cross-paper comparison is meaningless. No published agentic benchmark normalizes energy by successfully completed goals.

Method¶

A-LEMS (Agentic LLM Energy Measurement System) is a cross-layer framework with four interacting components:

EpG definition. For a batch of $N_\text{success}$ completed goals, each comprising $n_i$ attempts: $$\text{EpG} = \frac{\sum_{i} E^{(i)}_\text{attr}}{N_\text{success}}, \quad E^{(i)}_\text{workflow} = \sum_{k=1}^{n_i} E_\text{attempt,k}$$ Failed workflow units contribute energy to the numerator but zero to $N_\text{success}$.

Temporal boundary model. Three anchors $t_0/t_1/t_2$ define the attribution window $[t_0, t_1]$: $t_0$ is set at start_measurement() (RAPL counter read), $t_1$ at executor return (after all retries complete), and $t_2$ after ETL/teardown. $E_\text{pre}$ and $E_\text{post}$ are recorded diagnostically and excluded from EpG.

Five-layer observation pipeline (L0–L4): - L0: Raw RAPL package energy $E_\text{pkg} = E_\text{core} + E_\text{uncore} + E_\text{dram}$ - L1: Baseline isolation — $E_\text{dyn} = \max(0, E_\text{pkg} - P_\text{base} \cdot \Delta t)$ using a $2\sigma$ idle-window baseline - L2: Process attribution — $E_\text{attr} = f_\text{cpu} \cdot E_\text{dyn}$, where $f_\text{cpu} = \Delta\text{ticks}_\text{pid}/\Delta\text{ticks}_\text{total}$ - L3: Phase decomposition — $E_\text{attr} = E_\text{plan} + E_\text{exec} + E_\text{syn} + E_\text{gap}$ at 100 Hz RAPL sampling with coverage metric $C \geq 95\%$ (gold tier) - L4: Goal aggregation → EpG

Orchestration Overhead Index (OOI). A normalized ratio comparing agentic vs. linear workflows under identical task definitions and success criteria, isolating orchestration cost from inference substrate.

Three-hash reproducibility protocol. $H_\text{hw}$ (hardware fingerprint) + $H_\text{env}$ (software environment) + $H_\text{run}$ (execution state) jointly bind each measurement to its exact context.

Key Contributions¶

EpG: goal-level energy unit aggregating all attempts (including failures) normalized by successful completions, resolving the unit-misalignment problem
Temporal boundary model ($t_0/t_1/t_2$) with explicit $E_\text{pre}$/$E_\text{post}$ exclusion, eliminating four boundary failure modes (truncation, inflation, overlap, idle conflation)
Five-layer attribution hierarchy (L0–L4) mapping raw RAPL signals through baseline subtraction and CPU-fraction isolation to workflow-level energy with per-layer provenance tagging (measured / calculated / inferred)
OOI (Orchestration Overhead Index): comparative ratio requiring a matched linear baseline, analogous to PUE for datacenters
Three-hash reproducibility protocol binding measurements to hardware, environment, and runtime state
Empirical validation across 827 agentic goals spanning 5 reasoning and 3 tool-augmented task families with RAPL hardware profiling

Results¶

Agentic workflows: 4.33× higher mean EpG vs. linear baselines (888.1 J vs. 205.3 J) across 827 goals, 5 reasoning task families
Overhead is not driven by increased inference compute but by orchestration structure (retries, intermediate planning, recovery)
OOI inverts below 1.0× for tool-augmented tasks where agentic dispatch replaces costly token generation — confirms the metric is directionally correct, not an upward-biased artifact
Canonical GSM8K trace (exp. 946, llama_cpp, failure_injection): failed attempt consumed 2,256 J vs. 1,358 J for the successful attempt; inference-level accounting misses 62.4% of true cost; EpG = 3,614 J
RAPL sampler achieves ~103.0 samples/second at 100 Hz; gold-tier coverage ($C \geq 95\%$) leaves at most 332.6 ms unresolved across 2,228 runs
During LLM API wait phase, system power drops to 1.0 W vs. 0.2 W during active planning — invisible without phase-level measurement

Limitations¶

Current implementation measures local CPU package only; GPU, NIC, and remote inference energy are excluded from the empirical results despite being included in the full EpG definition
OOI requires a matched linear baseline, which may not always be constructible for tasks without a natural single-pass equivalent
RAPL counters are unavailable on some hardware (certain cloud VMs, non-Intel architectures); portability of the measurement stack is not evaluated
The reproducibility protocol binds to specific hardware configurations, potentially complicating comparisons across heterogeneous infrastructure
Empirical scope is limited to reasoning and tool-augmented task families; code generation, multi-turn dialogue, and embodied agent tasks are not covered
Binary success criterion (no partial credit at goal level) may not capture meaningful variance in output quality for open-ended tasks

Relevance to Harnesses / Meta-Harnesses¶

A-LEMS is directly a meta-harness measurement framework: it wraps arbitrary agentic workflow executors and instruments them at the goal level, making it a cross-cutting observability layer over multi-step orchestration systems. The five-layer attribution pipeline and temporal boundary model are precisely the kind of harness-level instrumentation hooks that meta-harnesses need to expose—any harness managing retries, tool dispatch, and failure recovery is exactly the system whose energy the EpG unit captures. The OOI metric provides a rigorous way to compare harness designs (e.g., eager-retry vs. backoff, monolithic vs. decomposed planning) on energy grounds, independent of the underlying model. For harness designers, the key finding that orchestration structure (not inference substrate) dominates energy cost means that harness-level architectural choices—retry budgets, planning depth, tool dispatch strategies—are first-class energy optimization targets.