Skip to content

SVGym (SciVerseGym): An Environment for Reinforcement Learning and Bayesian Optimization in Crystal Discovery

🕒 Published (v1): 2026-06-21 10:24 UTC · Source: Arxiv · link

Why this paper was selected

SVGym: unified RL/BO environment harness for crystal discovery; shows how to scaffold closed-loop scientific agent pipelines

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

SVGym (SciVerseGym) is a Gymnasium-compatible environment that wraps crystal structure discovery as a Markov decision process, providing a unified action schema, evaluator adapters, and reward interface so that RL, Bayesian optimization, evolutionary search, and language-agent workflows can be benchmarked under identical physical assumptions. It decouples search-algorithm logic from materials-specific infrastructure (structure manipulation, MLIP backends, thermodynamic reference data) via a standard obs, reward, terminated, truncated, info transition protocol. The core contribution is an integration layer, not a new force field or optimizer.

Problem

Closed-loop crystal search algorithms are routinely implemented as isolated, bespoke pipelines: structure modification, relaxation, scoring, constraint handling, and bookkeeping are entangled with method-specific code. This fragmentation makes it impossible to fairly compare RL, BO, evolutionary, and language-agent approaches because differences in results may reflect implementation details rather than search strategy. MLIP backends introduce further heterogeneity (model checkpoints, reference energies, convex-hull conventions) that every method must handle separately.

Method

SciVerseGym formalizes crystal discovery as a finite-horizon MDP registered as CrystalDiscovery-v0 under the Gymnasium API. At each step \(t\):

\[s_{t+1}, r_t, d_t, \tau_t, i_t = \texttt{Env.step}(a_t)\]

Action space — five bounded edit families applied to an ASE Atoms object: element replacement, lattice perturbation, atomic displacement, vacancy creation, atom insertion. Each action is a typed dictionary; validity rules (index clipping, bounded displacements, fractional-coordinate insertion, minimum-one-atom guard) are enforced at the environment level, not by the agent.

Evaluation backends — pluggable ASE-compatible calculators; built-in adapters for SevenNet, MatterSim, and ORB. Formation energy per atom is:

\[E_{\text{form}} = \frac{E - \sum_i n_i \mu_i}{N}\]

Energy above hull is computed against same-backend reference phases; phonon stability via Phonopy finite-displacement calculations (optional). Missing reference data returns None rather than fabricating values.

Default reward — transparent linear combination:

\[r = 0.80\, r_{\text{stability}} + 0.10\, r_{\text{abundance}} + 0.10\, r_{\text{novelty}}\]

Reward is replaceable without touching the action schema or evaluator interface.

Agent protocols — BO agents treat environment steps as structured black-box evaluations with UCB acquisition \(\alpha_t(a) = \mu_t(\phi(s_t,a)) + \kappa_t \sigma_t(\phi(s_t,a))\); RL agents maximize discounted return \(G_t = \sum_{k=0}^{T-t} \gamma^k r_{t+k}\) using any policy architecture.

Key Contributions

  • Gymnasium-compatible MDP abstraction for sequential crystal editing (CrystalDiscovery-v0).
  • Five-family bounded action schema shared across RL, BO, evolutionary, MC, and language-agent methods.
  • Backend-agnostic evaluator layer with built-in MLIP adapters (SevenNet, MatterSim, ORB) and any ASE calculator.
  • Rich info dictionary (formation energy, hull distance, phonon indicators, relaxation status) separate from the scalar reward.
  • Configurable, replaceable reward function with a transparent default (stability + abundance + novelty).
  • Default dataset (ALEX-MP-20) with support for user-mounted structure pools.
  • Open-source Python package; no bespoke wrappers needed to plug in existing RL/BO libraries.

Results

The paper is an environment/framework description paper; it does not report benchmark numbers comparing search algorithms head-to-head. No quantitative performance results, discovery rates, or ablation numbers appear in the provided text.

Limitations

  • No empirical benchmark comparing RL vs. BO vs. evolutionary methods within the framework is presented in this paper.
  • Phonon stability is excluded from the default reward and is computationally expensive; its role in evaluation is optional and incomplete.
  • Validity rules reduce but do not eliminate physically unrealistic candidates; chemical plausibility is not guaranteed.
  • Convex-hull and formation-energy computation require same-backend elemental references; when these are unavailable, thermodynamic quantities are silently suppressed rather than approximated.
  • The framework currently registers a single environment (CrystalDiscovery-v0); multi-property or multi-fidelity discovery tasks require user extension.
  • Language-model agents are cited as supported but are described only conceptually; no LM-agent baseline or integration is demonstrated.

Relevance to Harnesses / Meta-Harnesses

SVGym is a domain-specific harness for materials-discovery agents: it provides the standard integration layer (action schema, evaluator adapters, reward interface, diagnostic telemetry) that allows heterogeneous search methods — RL, BO, evolutionary, Monte Carlo, and LLM-based agents — to be plugged in and compared without reimplementing infrastructure. This is structurally analogous to software meta-harnesses that wrap diverse agent backends behind a common protocol, managing environment state, bookkeeping, and feedback normalization. For researchers tracking harness design patterns, SVGym illustrates how a well-scoped environment abstraction (MDP + Gymnasium) can eliminate the fragmentation problem in a scientific domain, and demonstrates the reward-function-as-plugin pattern that meta-harnesses often need for task-specific objective injection.