Skip to content

Compiler-R1: Towards Agentic Compiler Auto-tuning with Reinforcement Learning

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

Compiler-R1 is the first RL-driven framework that trains LLM agents to perform LLVM compiler pass-sequence optimization via tool-augmented, environment-interactive learning. A two-stage pipeline (SFT cold-start + RL policy optimization) achieves 8.46% average IR instruction count reduction over -Oz across seven benchmarks, outperforming SFT-only baselines and classical autotuners in both quality and speed.

Problem

LLM-based compiler auto-tuning is hindered by two gaps: (1) no high-quality reasoning datasets exist that combine chain-of-thought, tool invocation, and environment feedback for the compiler domain; (2) prior SFT-only approaches do not interact adaptively with the compilation environment, limiting generalization to unseen programs.

Method

Compiler-R1 uses a two-stage pipeline on top of Qwen2.5-Instruct backbones (1.5B–7B):

Stage 1 — SFT cold-start. A 19,603-sample dataset is constructed by (a) identifying synergistic LLVM pass pairs per program, (b) building a Global Synergy Graph, (c) selecting optimal sequences via graph-guided random walks evaluated by OverOz (IR reduction vs. -Oz), and (d) wrapping results in simulated <think>/<tool_call>/<tool_response>/<answer> CoT trajectories. Programs are represented as 56-dimensional AutoPhase statistical features (not raw IR) to fit LLM context limits. Two tools are exposed: instrcount (evaluates a candidate sequence) and find_best_pass_sequence (searches for better sequences when OverOz ≤ 0).

Stage 2 — RL policy optimization. The SFT-initialized agent interacts with a simulated compilation environment. A composite reward R_final = w_f · R_format + w_a · R_answer is used: R_format (1.5 if trajectory structure is valid, else 0) and R_answer = α · OverOrig (IR reduction vs. unoptimized baseline, chosen over OverOz for denser reward signal). Training uses PPO, GRPO, or REINFORCE++ (RPP) for 40 steps over 19k interactive episodes.

Key Contributions

  • First RL-based framework specifically targeting LLM-driven LLVM compiler pass-sequence optimization.
  • A curated 19,603-sample reasoning dataset with CoT + tool-use trajectories derived from synergy-graph-guided pass analysis.
  • Two-stage SFT+RL pipeline that enables autonomous environment exploration via instrcount and find_best_pass_sequence tool calls.
  • Empirical demonstration that AutoPhase 56-dim features match raw LLVM IR performance while drastically reducing input length.

Results

  • GRPO-7B: 8.46% average OverOz across 7 benchmarks (blas, cbench, chstone, mibench, npb, opencv, tensorflow); 96.71% task success rate; 26s/program.
  • Vs. SFT-only baselines (N=40 attempts): GRPO-7B (8.46%) outperforms SFT-Qwen-7B (4.66%) with fewer inference attempts and lower runtime (26s vs. 55s).
  • Vs. traditional autotuners: Outperforms OpenTuner (6.09% avg, 200s), GA (5.17%, 561s), CompTuner (3.52%, 9803s), and AutoPhase PPO variants (which degrade to negative OverOz on several benchmarks).
  • GRPO-1.5B: 3.87% OverOz at 20s/program, exceeding SFT-Qwen-1.5B (3.15% at 29s).
  • SFT-only sensitivity: SFT-Qwen-7B requires N=40 samples to reach 4.66%; at N=1 it achieves −14.78%, underscoring the benefit of interactive RL.
  • AutoPhase vs. raw IR (SFT Qwen-1.5B, N=40): 0.4837 vs. 0.4795 OverOrig — comparable performance with shorter input.

Limitations

  • IR instruction count is a proxy metric; it does not directly measure runtime performance (cache effects, ILP, and hardware-specific behavior can cause shorter sequences to run slower).
  • AutoPhase's 56-dim features lose control-flow and data-flow semantics, potentially limiting detection of complex pass interactions.
  • The agent learns procedural tool-use protocols rather than deep causal understanding of why specific pass sequences are effective.
  • Dataset and training are tied to LLVM 10.0.0; cross-version compatibility requires intersection heuristics and is not robustly addressed.

Relevance to Agentic AI / LLM Agents

Compiler-R1 is a concrete instantiation of the tool-augmented LLM agent paradigm applied to a well-defined combinatorial optimization domain, demonstrating that DeepSeek-R1-style RL training transfers beyond math/code to systems-level tasks requiring iterative environment feedback. The two-stage SFT-cold-start + outcome-based RL pipeline generalizes the pattern emerging across domains (OS tuning, graph RAG, KBQA) where SFT alone fails to produce agents capable of adaptive exploration. The framework's emphasis on protocol compliance (thought–tool–answer trajectory) and reward shaping for sparse-feedback environments is directly relevant to designing reliable LLM agents in any domain with external tool dependencies. It also highlights a recurring challenge: agents can achieve high end-task performance (OverOz) while violating interaction protocols, raising open questions about how to jointly optimize for correctness and compliance.