In this paper
AbstractIntroductionRelated WorkMethodsMolecular optimization setupEmpirical latent geometry and the on-sphere scaffoldGeoWalk: geodesic local search on the empirical sphereGeoCMA: covariance learning in a lifted spherical subspaceExperimentsRandom search on the shell and the external baselinesWhat a surrogate adds on the shellCandidate support and acquisition maximizationWhen covariance learning is usefulDiscussionAcknowledgments and Disclosure of FundingDetailed pseudocode and settings for GeoWalk and GeoCMAHyperparameter sensitivity of GeoWalk and GeoCMAPer-task breakdown of GeoWalk vs the GP on the shellPer-task convergence on the GuacaMol benchmarkScope of the codec comparisonBudget scaling beyond LassoBench per dimensionBaseline implementation detailsShell radius and alternative candidate supportsThe five contrasts of the factorial experimentCompute and assetsNeurIPS Paper ChecklistTheory assumptions and proofsExperimental result reproducibilityOpen access to data and codeExperimental setting/detailsExperiment statistical significanceExperiments compute resourcesCode of ethicsBroader impactsSafeguardsLicenses for existing assetsNew assetsCrowdsourcing and research with human subjectsInstitutional review board (IRB) approvals or equivalent for research with human subjectsDeclaration of LLM usageReferencesWorksComputer Science
Match the Geometry Before the Surrogate: Extreme Low-Budget Optimization in High Dimensions
Abstract
Low-budget molecular optimization is often framed as Bayesian optimization in a learned latent space, but standard benchmarks typically allow hundreds or thousands of evaluations, whereas a realistic wet-lab round may allow only a few dozen. We revisit this regime using a SELFIES molecular variational autoencoder (VAE) with a Gaussian prior and GuacaMol objectives at N = 30 evaluations. The main finding is not a new surrogate model, but an interaction in candidate generation. The encoder means of a reference set of molecules concentrate near a narrow spherical shell, while many high-dimensional optimizers propose candidates from boxes or box-shaped trust regions. We study a simple constraint, the on-sphere scaffold, that restricts candidate generation to this shell. We then introduce two minimal optimizers to probe what remains useful once the support is fixed. GeoWalk performs geodesic local search on the shell without a surrogate. After 30 evaluations, it shows no detected difference from a Gaussian-process (GP) surrogate on the shell, which is ahead earlier in the budget. GeoCMA adds covariance learning in a random subspace. It does not improve on GeoWalk in the molecular benchmark. On LassoBench it has a lower mean loss than GeoWalk, but full-dimensional CMA-ES remains stronger on most tasks. A factorial experiment with a single GP crosses the candidate support (shell or box) with the acquisition maximizer (a finite candidate set or a continuous optimizer). In this benchmark, shell candidates outperformed box candidates when acquisition was maximized over a candidate set, but not with continuous maximization by L-BFGS-B. These results come from a controlled study of one Gaussian-prior VAE benchmark and do not recommend VAEs for molecular optimization. The practical lesson is to control the candidate support and the acquisition maximizer before attributing a gain to the surrogate.
Introduction
Molecular and protein-design loops are often constrained by assays rather than by computation. A single wet-lab round may evaluate only a few dozen variants, each requiring substantial cost and weeks of turnaround time. This is far from the usual evaluation setting for high-dimensional Bayesian optimization (HD-BO) [1, 2, 3], where benchmark budgets often range from hundreds to thousands of oracle calls. In principle, Bayesian optimization is well suited to such expensive black-box objectives: it fits a cheaper surrogate model, typically a Gaussian process (GP) or another learned predictor, and uses it to choose the next candidate to evaluate. However, the budget mismatch raises a simple yet consequential question: In the extreme low-budget scenario (30 evaluations), what should the optimizer actually do?
We study this question in a widely used molecular-design pipeline built around a SELFIES [4] Transformer VAE, released with local latent space Bayesian optimization (LOL-BO) [2] and pretrained by its authors on the GuacaMol training set. The VAE has latent dimension , and we evaluate it on the GuacaMol goal-directed tasks [5]. This benchmark underlies several recent latent-space BO comparisons, and keeping its generator fixed lets us isolate the optimization layer. When the molecules from the ZINC-250k reference set [6, 7] are encoded by the VAE, their encoder means concentrate near a narrow spherical shell rather than filling the surrounding box. Many HD-BO methods, however, generate candidates inside axis-aligned boxes or box-shaped trust regions.
A Bayesian optimizer picks its next point by maximizing an acquisition function, a score of how promising a point is under the surrogate. It can score a finite set of randomly drawn candidates and take the best one, or it can start from a point and climb the acquisition function with a gradient-based optimizer, which we call continuous maximization. Here that optimizer is L-BFGS-B on both supports. In this Gaussian-prior VAE benchmark, shell candidates outperformed box candidates when acquisition was maximized over a candidate set, but not with continuous maximization by L-BFGS-B. The GP is fitted in the same way on both supports, but box candidates are rescaled per coordinate to before the GP, and shell candidates are not. With candidate-set maximization, the shell is ahead on 9 of the 11 tasks of our protocol.
The main object of study is therefore not a new surrogate model, but candidate generation: where candidates come from, and how an acquisition function is maximized over them. Instead of proposing candidates from an axis-aligned box, the on-sphere scaffold places them on the empirical spherical shell occupied by the encoder means of the reference set. Formally, candidates have the form
where is the mean encoder norm of the run’s cold-start molecules and is a unit direction in the -dimensional latent space. This constrains only the norm of a candidate, not its direction. We introduce two deliberately simple optimizers, GEOWALK and GEOCMA, as probes of what remains useful once candidate generation is restricted to the encoder-mean shell. The first, GEOWALK, samples the shell uniformly and then takes geodesic steps from the best point found so far. Despite having no surrogate, no learned covariance, and no acquisition function, GEOWALK shows no detected difference from a GP on the shell after thirty evaluations, but the GP is ahead earlier in the budget. The second optimizer, GEOCMA, is positioned not as a competitor to GEOWALK on the molecular VAE benchmark but as a complementary regime probe. It keeps the same on-sphere scaffold but replaces both phases of GEOWALK with covariance learning in a random low-dimensional subspace, started from the best cold-start point, to mark where adding such structure becomes useful. On the molecular VAE benchmark itself GEOCMA does not improve over GEOWALK. On two benchmarks outside chemistry, LassoBench [8] and NAS-Bench-101 [9], it has better means than GEOWALK. Full-dimensional CMA-ES, however, has the lower loss on most LassoBench tasks, and on NAS-Bench-101 GEOCMA is not detectably ahead of random search.
The paper makes three contributions. First, a factorial experiment shows that, in this benchmark, the advantage of shell candidates over box candidates depends on how the acquisition is maximized. Second, we introduce GEOWALK and GEOCMA as minimal diagnostic optimizers that show what remains of this advantage without a GP. Third, all baselines are compared on one matched set, with every known deviation from the published methods reported (Appendix H and the method cards in the repository), because comparisons between latent optimizers can be confounded by candidate generation.
Practical lesson. When a learned latent space is optimized with few evaluations, first check where the candidates lie, for example by comparing their norms with those of encoded data, and how the acquisition is maximized. Otherwise, a gain attributed to the surrogate may come from candidate generation instead. A search without a surrogate on the same support, such as GEOWALK in this paper, shows how much the surrogate adds.
Related Work
Latent-space molecular optimization often follows a VAE-based pipeline: a variational autoencoder maps discrete molecules to continuous latent vectors, the latent space is searched with a Gaussian-process model of the objective, and selected latent points are decoded into molecules for scoring [7]. Later methods refined different parts of this pipeline. LOL-BO [2] introduced local trust regions in latent space. CoBO [10] regularized the VAE so that nearby latents have similar objective values. InvBO [3] addressed the encoder–decoder misalignment by inverting the decoder so that latent codes reconstruct their inputs consistently. NF-BO [11] replaced the VAE altogether with an autoregressive normalizing flow that admits an exact left-inverse and removes the reconstruction gap. EULBO [12] jointly optimizes the ELBO of its variational GP surrogate and a utility-based acquisition objective so that the surrogate is fitted with optimization in mind. COWBOYS [13] decouples the GP surrogate from the generative model entirely, fitting them separately and combining them via a Bayesian update. The authors of COWBOYS also observed that molecular VAE latents occupy a thin shell, and argued that box-shaped search spaces cannot target it. These methods form the natural background for our study, but most of them are evaluated with budgets far larger than those available in a wet-lab round. We focus on the opposite regime: What remains reliable when the entire optimization loop consists of only a few dozen decoded and scored candidates?
High-dimensional BO provides several mechanisms that could, in principle, help in this low-budget regime. TuRBO [1] uses local trust regions, BAxUS [14] adapts nested low-dimensional embeddings, SAASBO [15] introduces a sparse axis-aligned subspace prior on the GP lengthscales, and REMBO [16] and HeSBO [17] search through random embeddings. A related linear-embedding line [18] uses Mahalanobis-style kernels on random subspaces; like GEOCMA, it restricts search to a low-dimensional subspace, but remains box-supported in the ambient space. Covariance Matrix Adaptation Evolution Strategy (CMA-ES) [19, 20] provides a complementary evolutionary baseline that does not rely on a probabilistic surrogate.
Linear BO [21] applies an inverse stereographic projection inside the kernel of a linear GP; this changes the geometry seen by the model, while the search domain stays a cube. The on-sphere scaffold instead restricts where candidates are proposed. A complementary line of work designs Gaussian processes that respect a Riemannian manifold structure. GaBO [22] originally proposed Bayesian optimization on Riemannian manifolds, including the sphere, using a geodesic squared-exponential kernel. The Riemannian Matérn variant [23] uses Matérn Gaussian processes on Riemannian manifolds [24].
Recent work has also shown that Gaussian processes can remain competitive in high dimensions when their priors, lengthscales, and kernel defaults are chosen carefully [25, 26, 27]. These methods provide a strong basis for comparison. Our goal is not to show that Gaussian-process optimization cannot work in high dimensions, but to test how its results depend on where candidates come from and on how the acquisition is maximized. This view is also consistent with recent work emphasizing strong non-BO baselines [28, 29]: in the VAE studied here, the appropriate random baseline is not arbitrary sampling from a box, but sampling from the encoder-mean shell. We note that direct discrete-space methods such as graph genetic algorithms [28] bypass the VAE entirely and are highly competitive at larger budgets. We restrict scope to the question of how to use a learned latent space when one is already available, since this is the setting in which a candidate support must be chosen. We make no claim about which molecular model to use.
Methods
Molecular optimization setup
The molecular objectives are the 12-task subset of the GuacaMol goal-directed benchmark [5], combining multi-property optimization (amlodipine, fexofenadine, osimertinib, perindopril, ranolazine, sitagliptin, zaleplon), deco hop, scaffold hop, valsartan SMARTS, and median-molecule (median molecule 1, median molecule 2) objectives. We adopt this 12-task subset directly from the LOL-BO code [30]. The valsartan SMARTS task requires generating a molecule that matches a strict SMARTS pattern. No method tested produced a match at , so the per-task score is 0 for every row. We exclude it from cross-task aggregation rather than letting an all-zero column dominate noise statistics, leaving 11 effective tasks. Because every row scores 0 on it, leaving it out multiplies every cross-task mean by and changes no comparison. Cross-task means are means of per-task means. On a matched set with the same number of seeds in every task this equals the pooled mean, and the two differ only where a task has missing cells. A cell is one run of a method on one task with one seed. Each GuacaMol table, or block of a table, uses one matched set of cells, those completed by every row, and prints its size. A row that covers only part of that set is marked, and it uses the part it covers. The LassoBench tables print the coverage of every cell instead. Cells in which a failed GP fit forced a fallback step are included, and the repository counts them per method.
Each run is limited to evaluations. The first 5 evaluations are fixed cold-start molecules shared by all optimizers for the same seed, and the optimizer then makes exactly 25 proposals. A proposal that decodes to an already scored molecule, or does not decode, uses one of the 25. The cold-start set is drawn without replacement from a fixed file of the first 10,000 ZINC-250k molecules using the seed as random-number-generator state, so all methods see identical cold-start molecules within a seed but different molecules across seeds. They are the same for every task and method, are encoded by the encoder mean, and are scored directly. SELFIES tokens outside the VAE’s vocabulary (stereo tokens) are removed before encoding; this affects 63.1 % of the ZINC-250k molecules and every method alike. We report the best objective value found within these 30 evaluations. For the main benchmark, each method is run on the 11 non-degenerate GuacaMol tasks and on 40 random seeds, numbered 42–81. The runs of NAS-Bench-101, of two further codecs and with (§ 4.4 and Appendices § 11 and § 12) are stored runs from an earlier code version and were not rerun. The BAxUS row of Table 1 has 439 of the 440 cells, because the GP fit failed in one cell of the Zaleplon task.
| Method | Cross-mean | W/L | |||
| CMA-ES [19, 20] | 0.4026 | +0.0200 | 253/148 | 0.12 | 0.25 |
| TuRBO [1] | 0.3790 | +0.0437 | 270/123 | 0.032 | 0.43 |
| BAxUS‡ [14] | 0.3907 | +0.0320 | 283/101 | 0.024 | 0.34 |
| REMBO [16] | 0.4092 | +0.0135 | 238/144 | 0.002 | 0.23 |
| HeSBO [17] | 0.4120 | +0.0106 | 225/158 | 0.032 | 0.17 |
| Vanilla BO [25] | 0.4008 | +0.0218 | 238/159 | 0.083 | 0.26 |
| Papenmeier MSR [26] | 0.3899 | +0.0327 | 281/109 | 0.019 | 0.40 |
| Xu Standard GP [27] | 0.3951 | +0.0275 | 270/120 | 0.042 | 0.37 |
| Linear BO [21] | 0.4101 | +0.0125 | 226/172 | 0.032 | 0.17 |
| GaBO [22] | 0.4153 | +0.0074 | 231/165 | 0.17 | 0.13 |
| Riemannian Matérn GP [23] | 0.4059 | +0.0167 | 230/152 | 0.014 | 0.26 |
| This paper, on the empirical shell (Eq. (1)): | |||||
| Random Search (shell) | 0.4118 | +0.0108 | 194/64 | 0.00098 | 0.30 |
| GEOCMA | 0.4158 | +0.0068 | 225/169 | 0.054 | 0.11 |
| GEOWALK | 0.4226 | – | – | – | – |
Table 1. Cross-task means and paired statistics for GEOWALK versus all comparators at on one matched set of (task, seed) cells of the 11 GuacaMol tasks. REMBO and HeSBO propose on the empirical shell; GaBO and Riemannian Matérn GP propose on the sphere manifold; all other comparators propose off the shell. is the improvement of GEOWALK over the comparator. W/L counts the matched cells where GEOWALK wins or loses, excluding ties. is the exact two-sided Wilcoxon test on the 11 per-task mean differences, not corrected. is Cohen’s of the paired cell differences. Bold marks the highest mean. Notes are in Appendix H.
The primary test of every comparison is the exact two-sided Wilcoxon signed-rank test [31] on the per-task mean differences (11 tasks on GuacaMol, six on LassoBench), reported with the number of tasks on which a method is ahead. NAS-Bench-101 has a single task, so it is tested over seeds. The task is the unit because the cold-start molecules of a seed are shared across tasks, so the cells of one seed are not independent. Cell-level Wilcoxon tests are secondary, and so is Cohen’s , which we report as a cell-level effect size, computed as the mean paired difference divided by the standard deviation of paired differences. Benjamini–Hochberg correction [32] is applied within families: the five contrasts of the factorial experiment, the three recipe rows of Table 2, the tests of the published recipes (Table 15), and the LassoBench tests of the published methods. The secondary cell-level tests of the appendix tables are corrected within each block. The per-task comparison of GEOWALK with the GP on the shell (Appendix § 9) uses Holm correction over the 11 tasks. The task-level in Table 1 and Table 4 is uncorrected, and we call a result significant only within a corrected family.
Table 2: Box versus empirical shell within each GP recipe. Each GP on the empirical shell (candidate-set argmax) and in the box (L-BFGS-B), on one matched set of cells; cross-means as in Table 1. Tasks counts the tasks, of the 11, on which the shell version has the higher mean. is the task-level test of the shell version against the box version, corrected within the three recipes. The two DSP rows are the GP on the shell and the GP in the box of § 4.2 and § 4.3.
| Variant | Candidate support | Cross-mean | Tasks shell higher | |
| Matérn-5/2 GP (DSP prior), shell | Empirical shell | 0.4228 | 9/11 | 0.081 |
| Matérn-5/2 GP (DSP prior), box | 0.3989 | |||
| Matérn-5/2 GP (), shell | Empirical shell | 0.4176 | 9/11 | 0.081 |
| Matérn-5/2 GP (), box | 0.3845 | |||
| Matérn-5/2 GP (), shell | Empirical shell | 0.4149 | 9/11 | 0.083 |
| Matérn-5/2 GP (), box | 0.3957 |
Table 2.
| Benchmark | Dir. | GeoWalk | GeoCMA | Best other method | ||||
| GuacaMol mol VAE (256D) | 0.423 | 0.416 | 0.007 | 0.11 | 0.054 | 440 | GP on the shell, 0.423 | |
| NAS-Bench-101 graph VAE (16D) | 0.937 | 0.942 | 0.004 | 0.94 | 10 | random search, 0.941 | ||
| LassoBench, , normalized | 0.796 | 0.495 | 0.300 | 0.37 | 0.094 | 240 | CMA-ES, 0.428 |
Table 4. GeoWalk and GeoCMA across benchmark regimes. . Arrows give the optimization direction, and bold marks the better of the two means. GuacaMol: mean of per-task means on the matched cells. LassoBench: each method’s loss divided by the random-search mean loss of the task, averaged over the six tasks. is the number of paired cells, or of paired seeds on NAS-Bench-101. Best other method: the best mean among the other methods that cover the row’s matched set. is paired and favors the better method, is Cohen’s of the paired cells, and is the task-level test of § 3.1, uncorrected across the rows; marks the paired test over seeds, because NAS-Bench-101 has a single task.
Table 15: Published recipes against GEOWALK and the GP on the shell at (baseline family). All rows use one matched set of cells. Mean and are means of per-task means of best@30, and = the reference minus the recipe. The reference is GEOWALK in the left group of columns and the GP on the shell in the right group. Tasks = tasks, of the 11, on which the reference has the higher mean. = exact two-sided Wilcoxon test on the 11 per-task differences, the primary test. It is corrected by Benjamini–Hochberg within the 12 tests of this table. Bold marks . The last row is the other run of the Hvarfner recipe, outside the corrected family; Table 14 describes both runs.
| Recipe | Reference: GEOWALK Mean | Reference: GEOWALK | Reference: GEOWALK Tasks | Reference: GEOWALK | Reference: GEOWALK | Reference: GP on the shell | Reference: GP on the shell Tasks | Reference: GP on the shell | Reference: GP on the shell |
| Vanilla BO | 0.4008 | +0.0218 | 9/11 | 0.083 | 0.091 | +0.0220 | 7/11 | 0.15 | 0.15 |
| Papenmeier MSR | 0.3899 | +0.0327 | 9/11 | 0.019 | 0.056 | +0.0329 | 10/11 | 0.0098 | 0.039 |
| Papenmeier MLE (scaled) | 0.3892 | +0.0334 | 9/11 | 0.0098 | 0.039 | +0.0335 | 10/11 | 0.0098 | 0.039 |
| Xu Standard GP | 0.3951 | +0.0275 | 9/11 | 0.042 | 0.063 | +0.0277 | 9/11 | 0.054 | 0.064 |
| Linear BO | 0.4101 | +0.0125 | 10/11 | 0.032 | 0.063 | +0.0127 | 9/11 | 0.054 | 0.064 |
| TuRBO | 0.3790 | +0.0437 | 9/11 | 0.032 | 0.063 | +0.0438 | 8/11 | 0.042 | 0.063 |
| Vanilla BO, all observations as baseline (the other run of the same recipe) | 0.3993 | +0.0233 | 8/11 | 0.12 | – | +0.0234 | 8/11 | 0.10 | – |
Table 15.
The GuacaMol tasks have very different absolute score ranges (per-task means range from below 0.1 to above 0.7, Table 8). The standard error of the unweighted cross-task mean is therefore dominated by between-task heterogeneity rather than by between-method differences and is not informative for method comparison. We therefore base statistical claims on per-task differences and do not attach standard-error bands to cross-task means.
| Task | GeoWalk | GeoCMA | Random Search (shell) | GP on the shell | GP in the box | Linear BO port (shell) | Linear BO port (box) | |
| adip | 40 | 0.501 | 0.499 | 0.498 | 0.504 | 0.459 | 0.500 | 0.474 |
| dhop | 40 | 0.571 | 0.569 | 0.568 | 0.571 | 0.561 | 0.570 | 0.560 |
| fexo | 40 | 0.676 | 0.669 | 0.662 | 0.675 | 0.628 | 0.666 | 0.628 |
| med1 | 40 | 0.129 | 0.121 | 0.120 | 0.122 | 0.154 | 0.121 | 0.156 |
| med2 | 40 | 0.153 | 0.152 | 0.151 | 0.151 | 0.150 | 0.153 | 0.150 |
| osmb | 40 | 0.714 | 0.719 | 0.708 | 0.725 | 0.702 | 0.719 | 0.714 |
| pdop | 40 | 0.484 | 0.463 | 0.469 | 0.477 | 0.378 | 0.475 | 0.405 |
| rano | 40 | 0.539 | 0.516 | 0.507 | 0.532 | 0.477 | 0.531 | 0.502 |
| shop | 40 | 0.440 | 0.440 | 0.436 | 0.441 | 0.427 | 0.438 | 0.424 |
| siga | 40 | 0.099 | 0.076 | 0.089 | 0.112 | 0.128 | 0.098 | 0.125 |
| zale | 40 | 0.344 | 0.349 | 0.323 | 0.339 | 0.323 | 0.340 | 0.349 |
| mean | 440 | 0.423 | 0.416 | 0.412 | 0.423 | 0.399 | 0.419 | 0.408 |
Table 8. Per-task means at . All columns use one matched set of completed cells, as in Table 1. Task codes as in Table 7. Last row: mean of per-task means. Bold: best per task. Box = [, ]. The GP on the shell and the GP in the box are the two DSP rows of Table 2; the Linear BO ports are those of Table 9.
Empirical latent geometry and the on-sphere scaffold
The motivation for the on-sphere scaffold comes from the empirical geometry of the VAE latent space. When the 249,455 molecules of the ZINC-250k reference set, which is not the VAE’s training set, are encoded by the VAE, their encoder mean vectors do not fill the surrounding ambient space uniformly. Instead, their norms concentrate near a relatively thin spherical shell, with empirical mean and standard deviation and 5th–95th percentile range . This is well below the norm of a typical draw from the prior. The covariance of the encoder means has effective rank (participation ratio) , and 66 principal components account for 90 % of the total variance (Appendix § 15). We never sample posterior noise during optimization, so we measure the shell radius on encoder means. These statistics describe the shell. They do not guarantee that it is the best place to search for every objective. Relatedly, Arvanitidis et al. [33] show that the latent space of a deep generative model is in general not flat Euclidean and that ignoring this structure can substantially mislead distance-based queries. The empirical concentration of VAE latents on a sphere is consistent with a separate line of work on hyperspherical VAEs, which makes this geometry explicit by placing the latent distribution on the unit sphere [34, 35, 36]. Our observation can be read as the implicit version of the same phenomenon: rather than redesigning the codec, the on-sphere scaffold uses the implicit shell that this Gaussian-prior VAE already exhibits as a constraint on candidate generation. Every newly proposed candidate is placed on this shell, the empirical sphere,
where is the empirical average norm of the encoded cold-start latents within each run and is a unit direction in the -dimensional latent space. This is an estimate of the shell radius reported above, and Appendix § 15 compares it with other estimators and with fixed radii. A uniform sample from this scaffold is obtained by drawing and returning . The scaffold constrains the norm of a candidate; it does not assume that the latent distribution is isotropic. Restricting the directions to the principal subspace of the encoder means did not improve on uniform directions (Appendix I). Random search, GEOWALK, GEOCMA, and the on-sphere GP variants use the scaffold in Eq. (1), whereas the external baselines use their own candidate generation (Appendix § 14).
Acquisition and its maximization. The GP of the factorial experiment uses the log expected improvement (qLogEI); the acquisition of every other row is given in its label or, where the label has none, in Appendix § 14. We compare two ways of maximizing it. With candidate-set argmax, each step draws 2,048 fresh uniform candidates from the support and proposes the one with the highest acquisition value. With continuous maximization, the acquisition is optimized by L-BFGS-B, the quasi-Newton optimizer of BoTorch [37], from 10 starts chosen among 512 raw samples, on both supports. In the box the raw samples are Sobol points of the box, and the optimizer is bounded by the box. On the shell the raw samples are random directions , left unbounded; the acquisition is optimized over and evaluated at , so every iterate lies on the shell. The box spans, in each coordinate, the 1st to 99th percentile of 50,000 encoded ZINC-250k molecules. The kernel and its prior are the same on both supports: a Matérn-5/2 kernel with the dimension-scaled lengthscale prior (DSP) of Hvarfner et al. [25]. Box candidates are rescaled per coordinate to before the GP, and shell candidates are not.
GeoWalk: geodesic local search on the empirical sphere
GeoWalk is a deliberately minimal optimizer for the on-sphere scaffold. It is designed to test how far one can get without fitting a surrogate model once candidate generation is restricted to the encoder-mean shell. Like every optimizer on the molecular benchmark, GeoWalk starts from the same cold-start points. The remaining budget is evaluations. GeoWalk first uses of them for uniform exploration of the empirical sphere, where is the exploration ratio. The remaining evaluations are used for local geodesic refinement from the best latent point found so far. The two phases balance exploration and exploitation: the uniform samples explore the shell, and the geodesic steps exploit the best point found so far. At the default budget , we use , which gives exploration samples and 9 refinement steps.
The refinement phase is a spherical variant of the standard (1 + 1) evolution strategy. In each refinement step, the algorithm keeps the best point found so far, proposes one new candidate near it on the sphere, and updates the reference point only if the new candidate improves the objective. Let be the current best latent and let be its unit direction. To propose a nearby candidate without leaving the sphere, we draw , remove its radial component, normalize the resulting tangent direction, and take a geodesic step:
The angle decays linearly from to radians during the refinement phase. This creates a simple spherical analogue of a shrinking local search region, but without fitting a GP, learning a covariance matrix, or optimizing an acquisition function. We also tested an adaptive step-size variant related to the classical 1/5 success rule [38, 39]. That rule raises the step size when more than about a fifth of recent steps improve the objective, and lowers it otherwise. Unlike that rule, this variant updates the angle after every step: it starts at 0.30 radians, is multiplied by 1.2 after an improving step and by 0.85 otherwise, and stays within . The adaptive variant underperforms the fixed decreasing schedule, and so does a constant angle of 0.15, while constant angles of 0.60 and 0.35 show no detected difference from the fixed schedule. We therefore use the fixed angular schedule as a deliberate low-budget design choice rather than as a replacement for full evolution-strategy step-size theory. A full sensitivity analysis of and the angle schedule is reported in Appendix § 8.
GeoCMA: covariance learning in a lifted spherical subspace
GeoCMA is designed to test when the geodesic random walk of GeoWalk is no longer sufficient. It keeps the same on-sphere scaffold from Eq. (1), but replaces both phases of GeoWalk with covariance learning. The motivation is that some task spaces may contain directional structure that can be learned even from a small budget, so that the sampling distribution on the sphere can be refined toward promising directions. To do this at low budget, GEOCMA runs Covariance Matrix Adaptation Evolution Strategy (CMA-ES) [19, 20] in a random low-dimensional subspace rather than in the full -dimensional latent space. We draw a Gaussian random matrix and compute an orthonormal basis by QR decomposition, with in our experiments. CMA-ES works in the projected coordinates of this subspace. There is no uniform exploration phase: its search distribution starts at the projection of the best cold-start point, and the projected cold-start points with their scores are its first update. Each proposal is normalized to the unit sphere in the subspace, lifted to the ambient latent space by , whose orthonormal columns preserve the norm, and scaled to the empirical radius:
On the molecular benchmark, CMA-ES runs with population size and initial step size , and every GEOCMA proposal lies on the same empirical shell as the proposals of GEOWALK, restricted to the directions of the subspace. After this normalization only the direction of matters: the covariance learned by CMA-ES shapes which directions are proposed, and its step size sets how widely they spread around the current mean of its search distribution. The two other benchmarks differ in the support. On LassoBench, the CMA-ES sample is lifted without either normalization and clipped to the box , so there GEOCMA does not search a sphere (§ 4.4). On NAS-Bench-101, the arch2vec latent space [40] has 16 dimensions, so the subspace is the whole space and GEOCMA is CMA-ES on the sphere. Full pseudocode for both methods is provided in Appendix § 7 (Algorithms 1 and 2), and Appendix § 8 compares and with the default.
Experiments
The experiments are organized around four questions. (1) How do established optimizers compare with uniform random search on the shell (§ 4.1)? (2) On the shell, does a search without a surrogate differ from a GP (§ 4.2)? (3) Does the advantage of the shell come from the support, from the acquisition maximizer, or from both, when the two are varied separately (§ 4.3)? (4) When does the random-walk behavior of GEOWALK stop being sufficient, so that covariance learning in GEOCMA becomes useful (§ 4.4)?
Random search on the shell and the external baselines
The first experiment tests whether the empirical latent sphere is already a strong search support, even before adding any surrogate model or adaptive local structure. We compare uniform random search on the scaffold in Eq. (1) with classical high-dimensional BO and evolutionary baselines, as adapted to this benchmark (Appendix § 14), on one matched set (Table 1). The comparators are TuRBO [1], BaxUS [14] and full-dimensional CMA-ES, the random-embedding methods REMBO and HeSBO, and four published GP recipes (Vanilla BO, Pappenmeier MSR, Xu Standard GP and Linear BO), which recent work reports as strong in high dimensions (§ 2). We keep the VAE fixed, so methods that update it during the search, such as LOL-BO [2] and InvBO [3], are not compared. Random search is intentionally minimal: at each step it draws a fresh Gaussian vector, normalizes it to the unit sphere, scales it by , decodes the latent point and evaluates the objective.
The bottom rows of Table 1 show that this simple on-sphere random search already has a higher mean at than every comparator that proposes candidates off the shell. GaBO and the Riemannian Matérn GP also propose at radius , with kernels defined on the sphere, so they are not counted among the rows off the shell. Random search is ahead on most tasks against each of these comparators. With the task as the unit of the test (§ 3.1), none of these leads is significant. Table 1 does not print this test, because its column tests each row against GEOWALK, not against random search. The higher mean of random search should not be read as a general criticism of these methods: most were designed for larger budgets or for direct parameter spaces. Appendix § 14 lists the restrictions under which each method ran.
Against the published recipes, GEOWALK has the higher mean in every case, but the difference is significant only against one of the two recipes of Pappenmeier et al., MLE (scaled) (Table 15 in Appendix § 14). The closest of them is Linear BO [21], which runs in its authors’ box from the shared cold-start molecules. GEOWALK is higher on 10 of the 11 tasks, and the difference is not significant. Appendix § 10 also has our port of Linear BO to the shell.
What a surrogate adds on the shell
We next ask whether a Gaussian-process surrogate still provides a substantial advantage once candidate generation is restricted to the same on-sphere scaffold. We compare GEOWALK with uniform random search on the shell and with the GP of Section 3.2 with candidate-set argmax, which we call the GP on the shell. It adapts the lengthscale prior of Vanilla BO [25] to the shell. It is not a baseline but the scaffold with a surrogate, so Table 2 prints it beside its box version, the GP in the box, next to the recipes adapted from Papenmeier MSR and Xu Standard GP. The cross-mean of the GP on the shell is above that of every method in Table 1, GEOWALK included, but its lead over GEOWALK is not significant, so it is the main comparator of GEOWALK here.
Earlier in the budget, the GP on the shell is ahead of GEOWALK, and after thirty evaluations we detect no difference between them. At that budget, GEOWALK is higher on 6 of the 11 tasks. The task-level test does not separate the two (), and no per-task difference is significant (Appendix § 9). After twenty evaluations of the same runs, the GP on the shell is ahead of GEOWALK on every task (Appendix § 10). At that point GEOWALK is still in its uniform exploration phase, which ends only after its 21st evaluation (§ 3.3), so until then its runs coincide with random search. The gain of GEOWALK over uniform random search on the shell at is small but consistent: GEOWALK is higher than random search on every task (Table 8). The memory and the geodesic refinement steps of GEOWALK therefore add value beyond the scaffold alone.
Candidate support and acquisition maximization
Table 2 compares three GP recipes on the box and on the shell: a Matérn-5/2 GP with the DSP prior [25], one with the lengthscale initialized at as in MSR [26], and one initialized at as in Xu et al. [27]. Each recipe uses L-BFGS-B in the box and candidate-set argmax on the shell, so these rows change the acquisition maximizer together with the support and cannot isolate the support. The box row also changes the acquisition function (Appendix § 14). A factorial experiment with the DSP-prior GP therefore separates the two factors (Table 3). It crosses the support, shell or box, with the maximizer: candidate-set argmax or continuous maximization by L-BFGS-B, the same on both supports (Section 3.2). Two of its four cells are the two DSP rows of Table 2. The five contrasts of the factorial, with their task counts and values, are in Appendix § 16.
Table 3: The support acquisition-maximization factorial: cross-mean of each cell. One Matérn-5/2 GP (DSP prior) on matched cells; continuous maximization is L-BFGS-B on both supports. Cross-means as in Table 1. The five contrasts of these cells, with their task counts and corrected values, are in Table 19 (Appendix § 16).
| Support | Candidate-set argmax | Continuous (L-BFGS-B) |
| Empirical shell | 0.4228 | 0.3999 |
| Box | 0.3925 | 0.3989 |
Table 3.
The shell helps when the acquisition is maximized over a finite candidate set, and not when it is maximized continuously by L-BFGS-B, which on the shell optimizes over directions and stays on it. Table 3 gives the cross-mean of each cell, and Table 19 in Appendix § 16 the five contrasts, each with the number of tasks on which it is positive and its value. With candidate-set argmax, the cross-mean is 0.4228 on the shell and 0.3925 in the box. The shell is ahead on 9 of the 11 tasks, and this difference is significant. With L-BFGS-B, the shell is ahead on 6 of the 11 tasks, and the difference is not significant. The interaction, the difference between these two support effects, is also significant: on 9 of the 11 tasks, the lead of the shell over the box is larger under candidate-set argmax than under L-BFGS-B. The maximizer acts in opposite directions on the two supports. On the shell, candidate-set argmax is ahead of L-BFGS-B on most tasks. In the box, L-BFGS-B is ahead of candidate-set argmax on nearly every task. Both of these contrasts are significant.
| Contrast | Tasks | ||
| shell minus box, with candidate-set argmax | +0.0303 | 9/11 | 0.017 |
| shell minus box, with L-BFGS-B | +0.0010 | 6/11 | 0.90 |
| candidate-set argmax minus L-BFGS-B, on the shell | +0.0229 | 9/11 | 0.017 |
| candidate-set argmax minus L-BFGS-B, in the box | −0.0064 | 1/11 | 0.012 |
| interaction: the first row minus the second | +0.0294 | 9/11 | 0.012 |
Table 19. The five contrasts of the factorial experiment of § 4.3. Differences of the cross-means of Table 3 on the same matched cells. is the mean of per-task means, Tasks counts the tasks with a positive contrast, and is the task-level test, corrected within the five contrasts; bold marks . The cell-level statistics and the bootstrap intervals are in the full table in the repository.
The recipe rows of Table 2 point the same way. For each recipe, the shell version is ahead of the box version on 9 of the 11 tasks, but none of these differences is significant.
When covariance learning is useful
Once candidate generation is restricted to the empirical shell, a simple geodesic random walk shows no detected difference from the GP on the shell after thirty evaluations. On the molecular benchmark, GeoCMA does not improve on GeoWALK, which is higher on 9 of the 11 tasks (Table 4). To find where a random walk stops being sufficient, we compare GeoWALK and GeoCMA on NAS-Bench-101 and LassoBench.
On these two benchmarks we make no claim about a shell, and the sphere is only a fixed-radius search domain. The radius and the clipping of each benchmark are in Appendix § 15. NAS-Bench-101 [9] is searched through the 16-dimensional latent space of an arch2vec graph VAE [40] over about 423,000 trained architectures, with CIFAR-10 validation accuracy as the objective. LassoBench [8] is a sparse-regression benchmark with no decoder. We use six of its tasks of increasing dimension: four synthetic tasks, DNA and leukemia. There, GeoCMA clips its points to the box without normalizing them, so it does not search a sphere. The column Best other method of Table 4 is the best of the comparators of Table 1 and the GP on the shell on GuacaMol, and the best row of Table 12 on LassoBench. On NAS-Bench-101 it is the best of random search, TuRBO, a GP and full-dimensional CMA-ES.
| Method | ||||||
| GEOCMA | 0.805(40) | 0.351(40) | 0.891(40) | 0.126(40) | 0.127(40) | 0.672(40) |
| GEOWALK | 0.566(40) | 0.687(40) | 0.974(40) | 0.748(40) | 0.868(40) | 0.931(40) |
| Random search | 1.000(40) | 1.000(40) | 1.000(40) | 1.000(40) | 1.000(40) | 1.000(40) |
| RBF GP (DSP prior) | 2.160(40) | 1.246(40) | 0.968(40) | 0.687(40) | 0.676(40) | 0.869(40) |
| Matérn-5/2 GP () | 1.141(40) | 0.793(40) | 0.994(40) | 0.890 (38/40) | 0.865(40) | 0.916 (36/40) |
| Matérn-5/2 GP () | 0.758(40) | 0.492(40) | 1.004 (39/40) | 0.730 (25/40) | 0.711 (38/40) | n.c. |
| Linear BO port | 0.879(40) | 0.243(40) | 0.896(40) | 0.138(40) | 0.133(40) | n.c. |
| Published methods | ||||||
| Vanilla BO | 1.710(40) | 0.856(40) | 0.954(40) | 0.754(40) | 0.685(40) | 0.899(40) |
| Pappenmeier MSR | 0.692(40) | 1.011(40) | 0.994(40) | 0.819(40) | 0.717 (17/40) | n.c. |
| Papenmeier MLE (scaled) | 0.616(40) | 0.519(40) | 0.998(40) | 0.834(40) | 0.946 (29/40) | n.c. |
| Xu Stand GP | 0.400(40) | 0.356(40) | 1.016(40) | 0.806(40) | n.a. | n.a. |
| Linear BO | 0.759(40) | 0.271(40) | 0.898(40) | 0.125(40) | 0.131(40) | n.c. |
| TuRBO | 1.298(40) | 0.789(40) | 0.962(40) | 0.759(40) | 0.963(40) | 1.148(40) |
| BAxUS | 0.997(40) | 0.399(40) | 0.873(40) | 0.115(40) | 0.124(40) | 0.709(40) |
| CMA-ES (LassoBench recipe) | 0.478(40) | 0.287(40) | 0.874(40) | 0.109(40) | 0.125(40) | 0.692(40) |
Table 12. LassoBench: final loss divided by the task’s random-search mean loss, per dimension. Each value is the mean over seeds of this ratio after 30 evaluations. A value of 1 equals random search, and a value above 1 is worse than random search. Lower is better. The tasks are synthetic simple, synthetic medium, DNA, synthetic high, synthetic hard, leukemia. In parentheses: the seeds with a finite loss ( where fewer than 40 seeds are 42–81 have one). n.c.: no seed finished with a finite loss. n.a.: not applicable at this dimension, so no run. The coverage and time-outs of each method, and each method minus GEOCMA, are in the full table in the repository.
Table 4 summarizes the boundary. GeoWalk remains ahead of GeoCMA in mean on the molecular VAE benchmark, where local random movement on the empirical sphere is sufficient. Outside the molecular benchmark, GeoCMA has the better mean of the two, but neither method is detectably ahead of the best other method there. On LassoBench, GeoCMA has a lower normalized loss than GeoWalk on 5 of the six tasks, but the task-level test does not detect this difference. The best method there is full-dimensional CMA-ES started at the origin (Appendix § 14): it has a lower normalized loss than GeoCMA on five of the six tasks. It also has a lower loss than GeoWalk on all six tasks, but this sweep is not significant once corrected within the published methods (Table 13). On NAS-Bench-101, which has a single task and is therefore tested over seeds, GeoCMA improves over GeoWalk. Random search, however, is ahead of GeoWalk there, with a Wilcoxon of 0.037 over the matched seeds, and GeoCMA does not differ detectably from random search.
| Method | (GEOWALK − method) | Tasks GEOWALK lower | |||
| Vanilla BO | −0.181 | 3/6 | 240 | 1.00 | 1.00 |
| Papenmeier MSR | −0.077 | 4/5 | 177 | 0.44 | 0.70 |
| Papenmeier MLE (scaled) | −0.012 | 4/5 | 189 | 0.62 | 0.71 |
| Xu Standard GP | +0.099 | 2/4 | 160 | 0.62 | 0.71 |
| Linear BO | +0.332 | 1/5 | 200 | 0.19 | 0.44 |
| TuRBO | −0.191 | 5/6 | 240 | 0.094 | 0.38 |
| BAxUS∥ | +0.259 | 1/6 | 240 | 0.22 | 0.44 |
| CMA-ES (LassoBench recipe)∥ | +0.368 | 0/6 | 240 | 0.031 | 0.25 |
Table 13. LassoBench, continued from Table 12: GEOWALK against each published method. Same runs and normalization as Table 12. These methods form the baseline family of LassoBench. Here a task is a dimension. The difference is GEOWALK minus the method, a mean over tasks of the per-task mean normalized losses. A positive means that GEOWALK has the higher, worse loss. Tasks counts the tasks on which GEOWALK has the lower mean loss, of those where both have runs. The column gives the (task, seed) pairs that both completed. The column is the exact two-sided Wilcoxon test on the per-task mean differences. The column is corrected by Benjamini–Hochberg over the 8 methods. Bold marks values below 0.05. With tasks the exact two-sided test cannot go below , so a method with fewer than 6 tasks cannot reach 0.05. The mark ∥ denotes the two arms that ran only on LassoBench; Table 1 prints the same method names under other settings. This family has 8 methods, and the family of Table 15 has 6 recipes.
Discussion
The experiments support a simple interpretation: in the studied molecular VAE benchmark, where the candidates came from mattered more than which surrogate scored them, as long as the acquisition was maximized over a candidate set. Candidates on the encoder-mean shell then outperformed candidates from the box. All three methods on the shell are the on-sphere scaffold with a different search inside it. The GP on the shell is the scaffold with a surrogate, GeoWalk is the scaffold without one, and GeoCMA is the scaffold with covariance learning. The GP on the shell has the highest cross-mean of any method on this benchmark (Table 1 and Table 2), and GeoWalk is level with it after thirty evaluations with no surrogate at all. At this budget of thirty evaluations the surrogate is therefore superfluous once the candidates lie on the shell. Earlier in the budget it is ahead, while GeoWalk is still exploring. This does not make surrogates unnecessary in general: on LassoBench and NAS-Bench-101, GeoCMA has better means than GeoWalk, while on the molecular benchmark covariance learning does not improve on the walk at . Comparisons of latent optimizers should therefore control and report the support and the acquisition maximizer before a gain is attributed to the surrogate.
Limitations. The advantage of the shell is shown only for candidate sets drawn uniformly from the whole support; local candidate sets, as in trust-region methods, were not tested. The evidence comes from this Gaussian-prior VAE and the 11 GuacaMol tasks of the protocol. In a separate run of the same factorial experiment on nine further GuacaMol tasks, the shell was ahead under candidate-set argmax on only three of them (not shown in the tables). On a second VAE, Chembed 1, whose encoder means also form a shell, moving the GP to the shell gave no detected gain (Appendix § 11). Locating a shell is therefore no guarantee that searching on it helps.
Budget scaling stops at , in stored runs on ten seeds (Appendix § 12), so results should not be read asymptotically. The cross-domain evidence for GEOCMA rests mainly on LassoBench, because the NAS-Bench-101 row has only 10 matched seeds. Other latent spaces were not tested: text VAEs, normalizing flows, diffusion models and the soft-prompt spaces of multimodal language models. Finally, better low-budget molecular optimization could lower experimental cost, but it has potential dual-use risks if applied to harmful objectives.
Conclusion. In this Gaussian-prior VAE benchmark, shell candidates outperformed box candidates when acquisition was maximized over a candidate set, but not with continuous maximization by L-BFGS-B. The GP on the shell, our scaffold with a surrogate, had the highest cross-mean of any method tested. A surrogate-free geodesic walk on the shell was level with it after thirty evaluations, so at this budget the surrogate is superfluous. Earlier in the budget the surrogate was ahead. This is a controlled study of a Gaussian-prior VAE benchmark, not a recommendation to use VAEs for molecular optimization. The lesson of this study is to control the candidate support and the acquisition maximizer before attributing a gain to the surrogate.
Code and data availability. The code, run configurations, result files and the generator of every table, figure and number are available at https://github.com/michalprusek/geometry-before-surrogates-neurips2026. The NAS-Bench-101 embedding is not available; its result files are included. The project page, with the paper, the poster and the results, is https://staff.utia.cas.cz/novozada/geo.
Acknowledgments and Disclosure of Funding
This project started in the Legal Data Analysis Lab, a master’s practical course at the Technical University of Munich; we thank Matthias Grabmair and Mohamed Elganayni for their guidance during the course. This work was supported by the Czech Science Foundation, grant GA25-15933S, and by the Grant Agency of the Czech Technical University in Prague, grant SGS24/141/OHK4/3T/14. The experiments were run on the computing servers of the Institute of Information Theory and Automation, Czech Academy of Sciences. The authors declare no competing interests.
Detailed pseudocode and settings for GeoWalk and GeoCMA
Algorithms 1 and 2 restate the GeoWalk and GeoCMA procedures described in § 3.3 and § 3.4 as explicit pseudocode. The cold-start, on-sphere scaffold (Eq. (1)), and GeoCMA lift step (Eq. (4)) match the main-text description; the inner CMA-ES loop (sample population → evaluate → tell) is the standard generation scheme of Hansen and Ostermeier [19], written here in population-per-generation form. The deferred-tell variant used in our implementation evaluates one candidate per step and updates CMA-ES once a full population of size has accumulated, which is mathematically equivalent. Table 5 lists the settings of GeoWalk and GeoCMA in the GuacaMol experiments.
| Setting | Value |
| GEOWALK | |
| Cold-start points | 5 |
| Evaluations after the cold start | 25 |
| Budget | 30 |
| Exploration ratio | 0.64 |
| Exploration steps | 16 uniform proposals on the shell |
| Refinement steps | 9 geodesic steps |
| Angle | 0.60 0.15 rad |
| Angle schedule | linear in the refinement step, over intervals |
| Anchor of a refinement step | best evaluated point so far, cold starts included |
| Radius | mean norm of the cold-start latents |
| GEOCMA | |
| Subspace dimension | 16 |
| Basis | QR decomposition of a Gaussian matrix |
| Population size | 5 |
| Initial step size | 0.5 |
| Initial mean | of the best cold start |
| Pre-conditioning | the cold starts, projected by , told as the first generation |
| Update | deferred: one evaluation per step; a generation is told once evaluations have accumulated |
| Lift | normalize in the subspace, map by , scale to |
| CMA-ES library and options | pycma 4.4.4; population , verbosity , seed drawn from NumPy’s global stream (seeded by the run seed) |
| Shared protocol (GuacaMol) | |
| Tasks | 11 GuacaMol tasks |
| Seeds | 42–81 (40 seeds) |
| Cold start | molecules drawn without replacement from the first 10,000 ZINC molecules (NumPy RandomState of the seed), encoded by the encoder mean, scored directly |
| Decoding | greedy, at most 80 tokens |
| Time cap per proposal | decoding capped at 30 s, scoring not time-limited |
| Time limit per run | 3,600 s for both methods, a guard against hangs, not a setting of the methods |
Table 5. Settings of GEOWALK and GEOCMA on GuacaMol. These settings hold for every task and seed. The settings that differ on LassoBench and NAS-Bench-101 are given in § 4.4 and Appendix § 15.
Algorithm 1 GeoWalk: geodesic local search on the empirical sphere
Require: Encoder mean , decoder , oracle , latent dimension , budget , cold-start molecules , exploration ratio , angles
Encode the cold-start molecules to encoder means and score them directly: and for
Set and and
for to do
Draw and set
Evaluate and add to the evaluated set
end for
for to do
Let be the best evaluated point so far and set
Draw and project it onto the tangent space:
Normalize the tangent direction:
Set
Propose
Evaluate and add to the evaluated set
end for
return The best evaluated candidate
Algorithm 2 GeoCMA: covariance learning in a lifted spherical subspace
Require: Encoder mean , decoder , oracle , latent dimension , budget , cold-start molecules , subspace dimension , CMA-ES population size , initial step size
Encode the cold-start molecules to encoder means and score them directly: and for
Set
Draw a Gaussian random matrix and form the orthonormal basis via the QR decomposition
Project the cold-start latents into the subspace: for
Let be the projected cold-start point with the highest oracle score
Initialize CMA-ES with mean , step size , and population size , then pre-condition it on the cold-start data
Set and
while do
Sample a new population from CMA-ES and set
for to do
Normalize within the subspace:
Lift to the ambient space: (the columns of are orthonormal, so )
Evaluate and add to the evaluated set
end for
if then
Update CMA-ES with the evaluated population:
end if
end while
return The best evaluated candidate
Hyperparameter sensitivity of GeoWalk and GeoCMA
GeoWalk has two design choices. The first is the exploration ratio (default 0.64, giving uniform-sphere samples and geodesic refinement steps out of the post-cold-start budget). The second is the angle schedule, which decays linearly from to radians during the refinement phase. Table 6 reports a paired comparison between the default schedule and six variants: three constant-angle variants (covering the start, middle, and end of the default schedule), two exploration-ratio variants ( and ), and the success-based rule of § 3.3. All variants are evaluated on the same 11 non-degenerate GuacaMol tasks, on one matched set of 440 cells.
| Variant | Mean | Tasks row higher | ||||
| constant angle 0.60 | 0.4228 | +0.0001 | 0.00 | 0.11 | 4/11 | 0.46 |
| constant angle 0.35 | 0.4228 | +0.0001 | 0.00 | 0.76 | 6/11 | 0.97 |
| constant angle 0.15 | 0.4189 | 0.006 | 2/11 | 0.024 | ||
| 8 exploration steps () | 0.4202 | 0.76 | 7/11 | 0.97 | ||
| 24 exploration steps () | 0.4133 | 0/11 | 0.00098 | |||
| success-based angle rule (start 0.30, / , clamp [0.05, 0.50]) | 0.4176 | 0.005 | 1/11 | 0.0098 | ||
| default (0.60 0.15, 16 exploration steps) (reference) | 0.4226 | – | – | – | – | – |
| Mean | ||||||
| (default) | 0.4158 | – | – | – | ||
| 0.4086 | 0.22 | |||||
| 0.4184 | +0.0026 | 0.04 | 0.22 |
Table 6. Sensitivity of GEOWALK and GEOCMA to their settings. Top: six variants of the step schedule and the exploration of GEOWALK, each against the default (angle 0.60 0.15, 16 exploration steps). Bottom: the subspace dimension of GEOCMA, 8 and 64 against the default 16. Both blocks use the same completed cells, as in Table 1. Means are means of per-task means. Bold marks the highest mean of the top block. The difference is the variant minus the default, computed from unrounded means. The effect size is Cohen’s of the cell differences. The column is the two-sided Wilcoxon signed-rank test over the cells. It is Benjamini–Hochberg corrected within each block, over 6 and 2 rows. Tasks counts the tasks on which the variant has a higher mean than the default. The column is the exact two-sided Wilcoxon test on the 11 per-task mean differences, not corrected. The task-level test of the bottom block is given in the text.
Of the three constant-angle variants, only (the end of the schedule) underperforms the default linear schedule: it is ahead of the default on only 2 of the 11 tasks. The constants (the beginning of the schedule) and (the middle) show no detected difference from the default. The exploration ratio shows an asymmetric pattern: raising from 0.64 to (almost all uniform exploration) puts the variant behind the default on all 11 tasks, while reducing to (less exploration, more refinement) shows no detected difference. The success-based rule is ahead of the default on only 1 of the 11 tasks. Overall, no variant is detectably better than the default, and three are worse.
Subspace dimension of GeoCMA. The subspace dimension is a low-budget design choice. With population size , the 25 evaluations give five CMA-ES generations whatever is, so sets how many directions CMA-ES adapts, not how often it updates. The bottom block of Table 6 compares and with the default, and neither difference is detected (task-level and 0.70).
Per-task breakdown of GeoWalk vs the GP on the shell
Section 4.2 detects no difference between GEOWALK and the GP on the shell after thirty evaluations. Their paired difference is at matched task-seed cells (last row of Table 7). Table 7 shows the per-task breakdown across the 11 non-degenerate GuacaMol tasks, using the same 40 seeds as Table 1.
| Task | GeoWalk | GP on the shell | |||||
| adip | 40 | 0.5013 | 0.5037 | 0.92 | 1.00 | ||
| dhop | 40 | 0.5706 | 0.5713 | 0.82 | 1.00 | ||
| fexo | 40 | 0.6760 | 0.6753 | 0.83 | 1.00 | ||
| med1 | 40 | 0.1287 | 0.1221 | 0.024 | 0.27 | ||
| med2 | 40 | 0.1525 | 0.1512 | 0.33 | 1.00 | ||
| osmb | 40 | 0.7142 | 0.7254 | 0.44 | 1.00 | ||
| pdop | 40 | 0.4838 | 0.4769 | 0.47 | 1.00 | ||
| rano | 40 | 0.5393 | 0.5322 | 0.72 | 1.00 | ||
| shop | 40 | 0.4400 | 0.4406 | 0.82 | 1.00 | ||
| siga | 40 | 0.0988 | 0.1123 | 0.78 | 1.00 | ||
| zale | 40 | 0.3438 | 0.3393 | 0.97 | 1.00 | ||
| all tasks | 440 | 0.4226 | 0.4228 | 0.00 | 0.50 | – |
Table 7. GeoWalk versus the GP on the shell, per task. = GeoWalk minus the GP on the shell (Table 2). Per task, , Cohen’s and the two-sided Wilcoxon are paired over the seeds of the task’s completed cells, as in Table 1. is Holm-corrected over the 11 tasks. Last row: all matched cells, as a mean of per-task means. The of the last row is the cell-level test over all cells. That test is not one of the per-task tests, so it has no (–). Task codes: adip = Amlodipine MPO, dhop = Deco Hop, fexo = Fexofenadine MPO, med1 = Median molecules 1, med2 = Median molecules 2, osmb = Osimertinib MPO. Further codes: pdop = Perindopril MPO, rano = Ranolazine MPO, shop = Scaffold Hop, siga = Sitagliptin MPO, zale = Zaleplon MPO. MPO = multi-property objective.
Per-task absolute differences are small: across all 11 tasks. The largest differences favor the GP on the shell, on Sitagliptin and Osimertinib; the largest in favor of GEOWALK is on Ranolazine. The signs are balanced across tasks (6/11 favor GEOWALK). Only one per-task paired Wilcoxon test reaches (Median 1, ), and it does not survive Holm correction over the 11 tasks (). Cohen’s values are small to moderate in either direction (range [, +0.40]). The cross-task mean is therefore a representative summary of the per-task picture rather than an artifact of cancellation.
Per-task means across the main baselines. Table 8 reports per-task means at across the seven most directly comparable methods (GEOWALK, GEOCMA and five shell/box baselines). On most tasks, the five methods on the shell (GEOWALK, GEOCMA, random search on the shell, the GP on the shell, the Linear BO port to the shell) are clustered closely together within each row, while the GP in the box trails (median per-task gap to GEOWALK is ). On Median Molecules 1 and Sitagliptin, however, the two box columns have the highest means.
Per-task convergence on the GuacaMol benchmark
Figure 1 shows the full 30-evaluation convergence traces for each of the 11 non-degenerate GuacaMol tasks, complementing the aggregate cross-task numbers in Table 1. Table 9 aggregates the same runs over tasks. It names its rows as Table 1 and Figure 1 do. The Linear BO port to the shell, which Table 1 does not print (§ 4.1), changes the support, the acquisition (qLogEI instead of LogEI) and its maximization, so we do not read the difference between the port and the published recipe as an effect of the support alone. The kernel bounds of the port were calibrated after a first run (Appendix § 14). The first 16 proposals of GEOWALK are uniform draws under the same seed as random search, so the two rows of Table 9 agree at and . The GP on the shell is ahead of GEOWALK at on all 11 tasks, and GEOWALK catches up only in its last evaluations (Table 9, bottom). In the aggregate, the GP rows on the shell are above the GP rows in the box at each budget of Table 9. These differences were not tested.
| Method | Support | |||
| GEOWALK | shell | 0.3612 | 0.3978 | 0.4226 |
| GP on the shell | shell | 0.3673 | 0.4067 | 0.4228 |
| Linear BO port to the shell (calibrated bounds) | shell | 0.3681 | 0.4066 | 0.4192 |
| Random Search (shell) | shell | 0.3612 | 0.3978 | 0.4118 |
| Linear BO port, box | 0.3499 | 0.3887 | 0.4079 | |
| Linear BO | box | 0.3479 | 0.3907 | 0.4101 |
| GP in the box | 0.3435 | 0.3784 | 0.3989 | |
| TuRBO | trust region in | 0.3350 | 0.3607 | 0.3790 |
| BAxUS‡ | nested subspaces of | 0.3336 | 0.3662 | 0.3907 |
| GEOWALK − the GP on the shell, paired on the same cells | ||||
| −0.0062 | −0.0090 | −0.0001 | ||
| −0.07 | −0.13 | 0.00 | ||
| 0.27 | 0.002 | 0.50 | ||
| Tasks, GEOWALK higher | 2/11 | 0/11 | 6/11 |
Table 9. Best-so-far after , 20 and 30 evaluations on one matched set. All rows except the one marked ‡ use the same completed cells, as in Table 1. The row marked ‡ covers of them (one run did not complete). Values are means of per-task means of the best score after evaluations: 5 shared cold-start molecules and proposals. Bold marks the highest mean in the column. Linear BO is the published recipe of Table 15; the two ports are ours (Appendix § 14). Bottom block: GEOWALK minus the GP on the shell, paired on the same cells. The effect size is Cohen’s of the cell differences. The column is the two-sided Wilcoxon signed-rank test over the cells. It is Benjamini–Hochberg corrected over the 3 budgets , 20 and 30.
Figure 1. Thirty-evaluation convergence on the 11 non-degenerate GuacaMol tasks. Curves show the best score found so far over 30 evaluations, averaged over the seeds of one matched set of 440 cells; bands indicate standard error. The grey vertical line marks the end of the five shared cold-start molecules. The three curves drawn heavier are GEOWALK, the GP on the shell and random search on the shell, which § 4.2 compares. The legend uses the names of § 4.2 and Table 1 and Table 2. The two ports of Linear BO appear in the appendix only (Table 8, Table 9, Table 12 and Table 14).
Scope of the codec comparison
The main experiments use the SELFIES Transformer VAE released with LOL-BO [2]. Stored runs exist for two further molecular codecs: Chembed [41], which is a second pre-trained Transformer VAE, and a Power-Spherical VAE [36]. For each codec we evaluated four methods at the same budget. Table 10 reports the cross-task means.
| Codec | normalized in decoder; decoding | GEOWALK | GEOCMA | GP on the shell | GP in the box | ||||
| SELFIES VAE | no, deterministic | 440 | 0.4226 | 0.4158 | 0.4228 | 0.3989 | +0.0239 | 0.30 | < 0.001 |
| Chembed (Transformer VAE) | no, deterministic | 440 | 0.3765 | 0.3734 | 0.3860 | 0.3810 | +0.0050 | 0.06 | 0.26 |
| Power-Spherical VAE | yes, stochastic | 440 | 0.3739 | 0.3706 | 0.3689 | 0.3594 | +0.0094 | 0.11 | 0.052 |
Table 10. Scope of the codec comparison (Appendix E). The GP on the shell uses candidate-set argmax and the GP in the box L-BFGS-B, so shell versus box also changes the maximizer. Each codec has its own box in its own latent space. Each codec row uses the matched cells of its four methods (completed cells, as in Table 1). Means are means of per-task means. = the GP on the shell minus the GP in the box. = Cohen’s of the cell differences. = two-sided Wilcoxon signed-rank test over the cells, not corrected. The task-level test is given in the text.
The comparison does not extend the main result beyond the SELFIES VAE. The Power-Spherical VAE normalizes its latent inside the decoder and decodes stochastically, so it cannot test where candidates are placed radially. Chembed’s encoder means also form a shell (the norm statistics on ZINC-250k are in the results bundle). Even so, on Chembed the GP on the shell is not detectably ahead of the GP in the box: it is ahead on 6 of the 11 tasks (task-level ). On Chembed the GP on the shell also has a higher mean than GEOWALK. GEOWALK is higher on only 4 of the 11 tasks, and the difference is not significant. A systematic study across more diverse codecs (e.g., diffusion-based or normalizing-flow latent spaces) is left to future work.
Budget scaling beyond
The main experiments focus on the low-budget regime motivated by wet-lab cost structures, but it is useful to ask whether the same pattern persists when more evaluations are available. Table 11 reports stored runs with , read at budgets on the same 11 non-degenerate GuacaMol tasks, using seeds 42–51. The table is descriptive; the goal is not to claim asymptotic dominance. GEOWALK and the GP on the shell remain close across all budgets, with matched differences of at most +0.008 for and −0.002 at . GEOCMA and random search on the shell also improve steadily with budget, but remain below GEOWALK and, except for GEOCMA at and , below the GP on the shell. The box row, a Thompson-sampling GP in the bounding box of the cold-start latents, is below every method on the shell at every budget. It differs from them in more than the support, so these runs carry no claim about the support.
| Method | ||||||
| GEOWALK | 0.424 | 0.445 | 0.463 | 0.479 | 0.497 | 0.509 |
| GP on the shell | 0.426 | 0.443 | 0.459 | 0.471 | 0.489 | 0.501 |
| GEOCMA | 0.414 | 0.439 | 0.461 | 0.474 | 0.487 | 0.495 |
| Random Search (shell) | 0.409 | 0.424 | 0.444 | 0.460 | 0.478 | 0.491 |
| GP in the cold-start box | 0.380 | 0.395 | 0.420 | 0.430 | 0.446 | 0.458 |
| GEOWALK − GP on the shell | −0.002 | +0.002 | +0.004 | +0.008 | +0.008 | +0.008 |
| GEOWALK − GP in the cold-start box | +0.044 | +0.050 | +0.043 | +0.049 | +0.051 | +0.051 |
Table 11. Best-so-far of the stored runs at intermediate budgets (seeds 42–51). One matched set of of the 110 (task, seed) cells. Each column is the best value after evaluations of the same runs. The column is therefore a prefix of these runs, not the results of Table 1. GP in the cold-start box: a Thompson-sampling GP in the bounding box of the five cold-start latents, not the box version of Table 2. Bold: the highest mean in the column. The rows are paired differences on the matched cells.
LassoBench per dimension
Table 12 accompanies the LassoBench discussion in § 4.4 by showing the per-dimension performance behind the aggregate row in Table 4. The GP rows other than TuRBO and BAxUS maximize the acquisition continuously with L-BFGS-B over the box . TuRBO and BAxUS draw a Thompson sample over a finite candidate set inside a trust region. For BAxUS, the trust region lies in its embedded subspace. GEOCMA has a lower normalized loss than GEOWALK at 5 of the six dimensions. It has the lowest loss of all methods only at . At the other dimensions, the CMA-ES recipe of LassoBench (all dimensions, started at the origin) has a lower mean loss than GEOCMA. No difference between GEOWALK and a published method is significant after correction within the published methods (Table 13). The runs also reveal a practical limitation of full-dimensional GPs at very low budget. Several GP methods lack seeds at some dimensions, because a model fit failed or, for MSR and MLE (scaled) at , because runs exceeded the limit of one hour. Both GEOWALK and GEOCMA avoid the full-dimensional GP fitting problem altogether.
Baseline implementation details
This appendix summarizes every method of Table 1 and Table 12 as it ran. Table 14 gives one row per method: its implementation, surrogate, acquisition and maximizer, and candidate support. The box version of the recipe of Table 2 uses the acquisition of MSR, analytic LogEI with sample-around-best restarts (RAASP in Table 14), so that row differs from its shell version in the acquisition as well as in the support and the maximizer. We print that row because it is the published call. A box run of this recipe with qLogEI, which has a higher mean than the printed box row, is in the full table in the repository. Neither row enters the factorial of Table 3, which uses the DSP recipe with the same maximizer on both supports.
| Method | Implementation | Surrogate | Acquisition, maximizer | Support, trust region |
| Methods of this paper (GuacaMol, LassoBench and NAS-Bench-101) | ||||
| GeoWALK and its variants | this paper | none | 16 uniform draws, then geodesic steps, angle 0.60 to 0.15 rad | shell of radius (NAS: mean embedding norm; LassoBench: clipped) |
| GeoCMA and its variants | this paper; CMA-ES from pycma | none (CMA-ES) | 16-dim. subspace, population 5, step 0.5 (NAS: sd of cold starts) | shell of radius (NAS: whole space, mean norm; LassoBench: box) |
| Random search, on the shell and on other supports | this paper | none | uniform draws; Gaussian fit: draws; NAS: 30 table entries | shell, PCA sphere, convex combinations of latents, Gaussian; LassoBench: |
| GP recipes adapted in this repository | ||||
| Matérn-5/2 GP (DSP prior) | own BoTorch GP; adapts Hvarfner et al. 2024 | Matérn-5/2, ARD, DSP lengthscale prior, MAP fit | qLogEI; argmax of 2,048 draws or L-BFGS-B | shell, box , or cube directions at |
| Matérn-5/2 GP () | own BoTorch GP; adapts Papenmeier et al. 2025 | Matérn-5/2, ARD, start , no lengthscale/outputscale prior | qLogEI, argmax or L-BFGS-B; one row LogEI with RAASP | shell or box ; LassoBench: |
| Matérn-5/2 GP () | own BoTorch GP; adapts Xu et al. 2025 | Matérn-5/2, ARD, start ; no lengthscale/outputscale prior | qLogEI; argmax over 2,048 candidates or L-BFGS-B | shell or box ; LassoBench: |
| Linear BO port, box | kernel copied from Doumont et al. | spherical linear kernel, LogNormal() prior | LogEI; L-BFGS-B, 4 restarts; sample-around-best | box (recaled); LassoBench: |
| Linear BO port to the shell (calibrated bounds) | kernel copied from Doumont et al. | as the box port; kernel bounds calibrated to the shell | qLogEI; argmax over 2,048 candidates on the shell | shell; the kernel-bound rule was set after a first run on the shell |
| RBF GP (DSP prior) | own BoTorch GP; adapts Hvarfner et al. 2024 | RBF, ARD, LogNormal(), MAP | qLogNEI; L-BFGS-B, 4 restarts, sample-around-best | (LassoBench only) |
| Published recipes | ||||
| Method | Implementation | Surrogate | Acquisition, maximizer | Support, trust region |
| Vanilla BO | authors’ code (Ax 0.3.7) rebuilt on BoTorch | RBF, ARD, LogNormal(, ), MAP | qLogNEI (in-box baseline); BoTorch, 4 restarts | box ; LassoBench: |
| Vanilla BO, all observations as baseline (the other run of the same recipe) | as the row above | as the row above | qLogNEI, all observations as baseline; BoTorch | box |
| Papenmeier MSR | authors’ released code, rebuilt | Matérn-5/2, ARD, start , learned noise | LogEI; 5 restarts, RAASP (sigma 0.001) | box ; LassoBench: ; no trust region |
| Papenmeier MLE (scaled) | authors’ released code, rebuilt | Matérn-5/2, ARD, start , learned noise | LogEI; 5 restarts, no sampling around the best | box ; LassoBench: |
| Xu Standard GP | authors’ released code, rebuilt | Matérn-5/2, ARD, Uniform() prior, start | UCB (beta 1.5); 10 restarts, 1,000 raw samples | box ; LassoBench: , not applicable (n.a.) on the two largest tasks |
| Linear BO | authors’ released code; kernel copied | spherical linear kernel, LogNormal() prior | LogEI; L-BFGS-B, 4 restarts, sample-around-best | authors’ box ; LassoBench: |
| TuRBO | authors’ turbo_1.py and gp.py, ported | Matérn-5/2, ARD, ExactGP trained by Adam (50 steps) | Thompson sampling of the predictive over Sobol candidates | TuRBO-1 trust region in ; LassoBench: |
| BAxUS (LassoBench) | BAxUS tutorial of BoTorch (tutorials/baxus) | BoTorch default RBF GP on the embedding | Thompson sampling; argmax of one posterior sample | nested count-sketch embeddings, BAxUS trust region |
| BAxUS (GuacaMol) | BAxUS tutorial of BoTorch (tutorials/baxus) | BoTorch default RBF GP on the embedding | Thompson sampling; argmax of one posterior sample | GuacaMol: nested count-sketch embeddings of the box , BAxUS trust region |
| CMA-ES (LassoBench recipe) | LassoBench’s CMA-ES example | none (pycma 4.4.4) | CMA-ES, population 20, step size 0.1, | (LassoBench only) |
| Other baselines implemented in this repository | ||||
| REMBO and HeSBO | re-implements REMBO (Wang et al.), HeSBO (Nayebi et al.) | BoTorch default RBF GP on the 16-dim. embedding | Thompson sampling; argmax over 100 points of the box | 16-dim. embedding lifted to the shell |
| GaBO | vendored GaBOtorch (Jaquier et al.) at 884f64a | sphere Gaussian kernel, Gamma(2, 0.15) outputscale prior | EI; Riemannian trust regions; uniform draw on failure | unit sphere , proposals scaled to |
| Riemannian Matérn GP | vendored MaternGaBO (Jaquier et al.) at 84bffda | Riemannian Matérn kernel on the sphere, nu = 2.5 | EI; robust Riemannian trust regions, 5 restarts | unit sphere , proposals scaled to |
| CMA-ES (population 5 or 20) | this repository; CMA-ES from pycma | none (CMA-ES) | CMA-ES in , step size 0.5, population 5 or 20 | , no bounds |
| GuacaMol stored runs GP in the cold-start box ( runs) | this repository; not the recipe of Havrner et al. | Matérn-5/2, ARD, gpytorch defaults, no explicit priors | Thompson sampling; argmax over 2,560 Sobol points | bounding box of the five cold-start encodings |
Table 14.
The full card of every method is in the repository (docs/baselines/BASELINE_CARDS.md). Each card has ten fields: the software version, kernel, acquisition and maximizer, lengthscale initialization, lengthscale prior, candidate support, trust region or RAASP settings, seeds, repository of the published method, and every deviation from it.
Notes to Table 1. All rows are evaluated on one matched set of (task, seed) cells. It is the intersection of the completed cells of the rows of the full version of the table, except the rows that cover only part of it. A completed cell is a run that made all 25 proposals after the 5 cold starts. In every row printed here, such a run reached all 30 evaluations. Runs in which a surrogate fit failed and a step fell back to a uniform proposal are included. The row marked ‡ covers only part of that set and uses the effect that it covers (BAxUS: , one run did not complete). is GeoWalk minus the row on the same cells, as a mean of per-task means of the paired differences. W/L counts the cells won and lost by GeoWalk. A cell with is a tie and is excluded from W/L. The effect size is the mean of the cell differences divided by their standard deviation. The column is the exact two-sided Wilcoxon test on the per-task mean differences. It is the primary test of the paper and is printed here without correction. The full version of this table is released with the code (docs/baselines/TABLE1_FULL.md and docs/baselines/tab_main_full.tex). It has all 31 comparator rows under the names of their implementation, and the cell-level Wilcoxon test, Benjamini–Hochberg corrected over the 31 rows. The rows not printed here are in the full version. Table 2 compares the GP recipes on the box and on the shell. TuRBO, Vanilla BO, Papenmeier MSR, Xu Standard GP and Linear BO are the published recipes, rebuilt from the authors’ released code; Table 15 compares them with GeoWalk and the GP on the shell. The other rows adapt the cited method to this benchmark. Their deviations from the published methods are summarized in Table 14 and listed in full in the method cards released with the code (docs/baselines/).
The full version names each row by its implementation: CMA-ES is “CMA-ES in , , population 5”; BAxUS is “GP in nested count-sketch subspaces, Thompson sampling in a trust region”; REMBO is “GP in a random linear embedding, Thompson sampling”; HeSBO is “GP in a hashed embedding, Thompson sampling”; GaBO is “sphere Gaussian-kernel GP, EI, Riemannian trust-region maximizer”; Riemannian Matérn GP is “Riemannian Matérn GP, EI, Riemannian trust-region maximizer”. The GP on the shell of § 4.2, which Table 2 prints with its box version, is “Matérn-5/2 GP (DSP prior), qLogEI, candidate-set argmax”. Box rows use the box, except Linear BO, which runs in . Table 14 gives the support of every row. Abbreviations: DSP prior is the dimension-scaled lengthscale prior LogNormal() of Hvarfner et al. The value is the initial lengthscale before the maximum-likelihood fit. EI, LogEI and qLogEI are expected improvement, its logarithm (analytic) and its Monte Carlo batch version. Candidate-set argmax: the acquisition is maximized over a finite set of random candidates on the support. Sample-around-best: L-BFGS-B restarts include points sampled around the best observations. MSR and MLE (scaled) are the two recipes of Papenmeier et al., with maximum-likelihood lengthscales initialized at . MSR adds RAASP perturbations (random axis-aligned subspace perturbations) in the acquisition maximizer, and MLE (scaled) does not. The box spans the per-coordinate 1st to 99th percentile of encoded ZINC latents. The radius of the shell is the mean norm of the run’s five cold-start latents. The set is the unit sphere.
Table 14: Overview of every printed method, one row per method and its variants. Implementation: where the code comes from. Surrogate: model and fit. Acquisition, maximizer: how a proposal is chosen. Support, trust region: where the candidates are drawn (: mean norm of the run’s five cold-start latents). Abbreviations: ARD = one lengthscale per input dimension. MAP = maximum a posteriori fit of the hyperparameters. RAASP = random axis-aligned subspace perturbations. qLogNEI = Monte Carlo log noisy expected improvement. UCB = upper confidence bound. PCA sphere = the sphere of radius in the span of the top 64 principal components of 3,000 encoded ZINC latents. The two BAxUS rows run the BoTorch tutorial, not the authors’ code, so Table 1 counts that method as an adapted one. Our adapted Matérn-5/2 GP uses the DSP prior with location , as in the paper of Hvarfner et al. Their released configuration, which the published recipe follows, uses 1.4 in place of . GaBO fell back to a uniform draw in 4,420 proposals, in 254 of its 440 runs. The full card of every method and its run status are released with the code (docs/baselines/).
Shell radius and alternative candidate supports
Table 16 and Figure 2 describe the shell of the encoder means on ZINC-250k. Let and be the covariance and the mean of the encoder means, and . The second-order approximation of the mean norm, , differs from the measured mean norm by . It is computed on the same encodings, so it describes the shell rather than predicting it. By default, stereo tokens outside the vocabulary of the VAE are dropped before encoding, and the affected atoms are lost. Every run of the paper uses this default pipeline (§ 3.1). Stripping stereochemistry with RDKit before encoding keeps these atoms, and the mean norm then changes by .
Figure 2. Norms of the encoder means on ZINC-250k. Histogram of for the 249,455 molecules, encoded with the default pipeline (out-of-vocabulary stereo tokens dropped) and with stereochemistry stripped before encoding. The marks show the mean norm over ZINC-250k and , the typical norm of a sample from the prior with .
| Quantity | ZINC-250k (full set) | ZINC-250k, stereo stripped |
| Molecules encoded | 249,455 | 249,455 |
| Molecules with dropped tokens (count, share) | 157,337 (63.1 %) | 5 (0.0 %) |
| 7.9834 ± 0.709 | 7.9751 ± 0.692 | |
| 5th and 95th percentile of | 6.81; 9.14 | 6.83; 9.11 |
| and (two values) | 59.10; 5.13 | 59.37; 4.71 |
| Participation ratio | 64.7 | 65.5 |
| Principal components for 90 % and 99 % of the variance | 66; 92 | 67; 93 |
| Model radius (error vs , %) | 7.9861 (+0.03) | 7.9766 (+0.02) |
| Model width | 0.678 | 0.676 |
| Relative width vs | 0.0888 vs 0.0879 | 0.0868 vs 0.0874 |
| 0.499 | 0.499 |
Table 16. Shell geometry of the encoder means of the VAE (), on ZINC-250k. is the covariance of and , both with the normalization. SD is the standard deviation of , also with . The model radius and width are second-order (Gaussian) delta-method values, with . They are computed on the same encodings, so they describe the shell and do not predict it. Dropped tokens: SELFIES tokens outside the vocabulary of the VAE (stereo tokens) are removed before encoding.
Radius, estimator and thickness. The three blocks of Table 17 vary how GEOWALK sets the radius of its proposals. Fixed radii from 6 to 12 show no detected difference from at the task level. A radius of 4 is ahead on only 3 of the 11 tasks, and a radius of 16 on only 3. Both have lower means, but neither difference is significant at the task level. Setting the radius to the median of the five cold-start norms, instead of their mean, gives a lower mean score. This variant is ahead on only 2 of the 11 tasks (task-level ). A fixed radius equal to the mean norm over ZINC-250k and the three thick shells show no detected difference from the default.
| Variant | Mean | Tasks row higher | ||||
| Fixed radius (reference ) | ||||||
| 0.4124 | −0.0103 | −0.19 | 0.009 | 3/11 | 0.24 | |
| 0.4187 | −0.0040 | −0.08 | 0.20 | 5/11 | 0.64 | |
| 0.4204 | −0.0022 | −0.04 | 0.12 | 5/11 | 0.64 | |
| 0.4199 | −0.0028 | −0.05 | 0.064 | 5/11 | 0.70 | |
| 0.4117 | −0.0110 | −0.15 | < 0.001 | 3/11 | 0.37 | |
| per-run (default) | 0.4226 | 0.0000 | 0.00 | 0.99 | 5/11 | 1.00 |
| (reference) | 0.4227 | – | – | – | – | – |
| Radius estimator (reference: mean of the five norms) | ||||||
| median of the five cold-start norms | 0.4207 | −0.0019 | −0.07 | 0.074 | 2/11 | 0.083 |
| fixed 7.98 (ZINC mean norm) | 0.4223 | −0.0004 | −0.01 | 0.82 | 5/11 | 0.52 |
| mean of the five cold-start norms (reference) | 0.4226 | – | – | – | – | – |
| Thick shells (reference: single shell ) | ||||||
| 0.4183 | −0.0043 | −0.08 | 0.099 | 3/11 | 0.054 | |
| 0.4195 | −0.0032 | −0.05 | 0.13 | 3/11 | 0.17 | |
| 0.4204 | −0.0022 | −0.04 | 0.13 | 4/11 | 0.21 | |
| single shell (default) (reference) | 0.4226 | – | – | – | – | – |
Table 17. Table 17: Radius of GEOWALK proposals. Three blocks, each against its own reference row. Fixed radius: every proposal is placed at radius . The default estimates per run from the five cold-start latents. Radius estimator: the statistic of the five cold-start norms that sets . Thick shells: the radius of each proposal is drawn independently from the stated range. The bounds and are the 5th and 95th percentiles of the encoder-mean norms of ZINC-250k. All blocks use the same completed cells, as in Table 1. Means are means of per-task means. Bold marks the highest mean of each block. The difference is the row minus the block’s reference on the same cells. The effect size is Cohen’s of the cell differences. The column is the two-sided Wilcoxon signed-rank test over the cells. It is Benjamini–Hochberg corrected within each block: 6 rows for the fixed radius, 2 for the estimator, 3 for the thick shells. Tasks counts the tasks on which the row has the higher mean. The column is the exact two-sided Wilcoxon test on the 11 per-task mean differences, not corrected.
Supports inside the latent cloud and directions. Every convex sampler and the Gaussian fit of Table 18 has a lower mean than uniform sampling on the shell (the convex samplers draw Dirichlet combinations of random latents, not uniform samples of the convex hull), with differences from −0.0209 to −0.0295. With the task as the unit, none of these differences is significant. Averaging, which moves candidates inside the shell, does not explain this gap: the sampler with draws encoded molecules themselves, whose mean norm equals the shell radius, and it is about as weak as the sampler with . Two controls change the directions at the shell radius (Table 18). Random search restricted to the top-64 principal subspace has a lower mean than random search on the full shell, ahead on only 3 of the 11 tasks (task-level ). The GP with candidate-set argmax over directions drawn from a cube centered at the origin and rescaled to shows no detected difference from the GP on the shell (task-level ). These directions come from that cube, not from the per-coordinate box of § 4.3. These controls suggest that, among the tested alternatives, matching the radial scale is more important than explicitly imposing the observed directional structure.
Table 18: Samplers inside the latent cloud, and direction controls. First block: surrogate-free samplers against random search, which samples uniformly on the shell. The samplers draw convex combinations of latents with Dirichlet(1) weights, or draw from a Gaussian fitted to the latents, a sample of 3,000 encoded ZINC-250k molecules. is the mean norm of the candidates. For random search it is , the radius of each run. Second block: two direction controls. Random search on the sphere in the span of the top 64 principal components is compared with random search on the full shell. The GP (DSP prior) with qLogEI argmax over cube directions rescaled to is compared with the same GP on the uniform shell. All rows use the same completed cells, as in Table 1. Means are means of per-task means. Bold marks the highest mean of the first block. The difference is the row minus its reference on the same cells. The effect size is Cohen’s of the cell differences. The column is the two-sided Wilcoxon signed-rank test over the cells. It is Benjamini–Hochberg corrected within each block, over 8 rows and 2 comparisons. Tasks counts the tasks on which the row has the higher mean, of the 11. The column is the exact two-sided Wilcoxon test on the 11 per-task mean differences, not corrected.
| Variant | Mean | Tasks row higher | |||||
| convex, | 0.3838 | 7.98 | −0.0280 | −0.31 | < 0.001 | 3/11 | 0.12 |
| convex, | 0.3888 | 6.62 | −0.0230 | −0.27 | < 0.001 | 4/11 | 0.24 |
| convex, | 0.3909 | 5.83 | −0.0209 | −0.24 | < 0.001 | 3/11 | 0.32 |
| convex, | 0.3829 | 5.32 | −0.0289 | −0.31 | < 0.001 | 3/11 | 0.17 |
| convex, | 0.3893 | 4.24 | −0.0225 | −0.23 | < 0.001 | 4/11 | 0.37 |
| convex, | 0.3855 | 3.46 | −0.0263 | −0.26 | < 0.001 | 3/11 | 0.32 |
| convex, | 0.3823 | 2.62 | −0.0295 | −0.27 | < 0.001 | 3/11 | 0.32 |
| Gaussian fit | 0.3840 | 7.99 | −0.0278 | −0.33 | < 0.001 | 3/11 | 0.12 |
| uniform on the shell (random search) (reference) | 0.4118 | — | — | — | — | — |
Direction controls at the shell radius (reference: the same method with uniform directions on the shell)
| Comparison | Variant | Shell | Tasks variant higher | |||||
| Random search: top-64 PCA sphere vs full sphere | 440 | 0.4069 | 0.4118 | −0.0049 | −0.08 | 0.046 | 3/11 | 0.17 |
| GP on the shell: cube directions rescaled to vs uniform directions | 440 | 0.4220 | 0.4228 | −0.0008 | −0.01 | 0.91 | 6/11 | 0.90 |
Table 18.
Radius and clipping on NAS-Bench-101 and LassoBench. On NAS-Bench-101, both methods search a sphere whose radius is the mean norm of the embeddings, and GEOCMA starts with a step size equal to the standard deviation of its projected cold-start points. On LassoBench, GEOWALK uses the mean norm of its cold-start points as the radius and clips its proposals to the box . GEOCMA clips its lifted points to the same box without normalizing them, so there it does not search a sphere.
The five contrasts of the factorial experiment
Table 19 lists the five contrasts of the factorial experiment of § 4.3: the support effect under each maximizer, the maximizer effect on each support, and their interaction, the difference between the two support effects. Each contrast is the difference of two cross-means of Table 3, tested at the task level and corrected within the five contrasts (§ 3.1). § 4.3 quotes the task counts of the factorial from this table.
Compute and assets
Machines. Servers A and B each have two NVIDIA L40S GPUs (48 GB each) and two Intel Xeon Gold 6542Y CPUs. Server A ran the GuacaMol runs other than the stored runs and the two sphere-manifold GPs. Server B ran these two GPs, all LassoBench runs and the checks of the published recipes. All of these runs used only the CPUs of server B. The stored runs were recorded on server B and on server C. Server C has two Intel Xeon Gold 5315Y CPUs, an NVIDIA A100 GPU (80 GB) and an NVIDIA RTX A5000 GPU.
Compute. A table in the repository lists, for each block of runs, its machine, its process-hours and the median time of one run. A process-hour is one hour of wall-clock time as recorded in a result file, and the totals sum these hours over the runs of a machine. The runs other than the stored runs took about 647 process-hours on server A and about 4,372 on server B. Of the hours on server B, about 4,185 were runs that checked the published recipes; the outcome of each check is in the results bundle, with the method cards. The stored runs that the printed tables read took about 1,289 process-hours. This total counts only the runs whose result files record a time. The project also used compute for preliminary and failed runs that the paper does not report.
Assets and licenses. Every external asset is cited where it is used. The list below gives the license of each asset and its version where it has one. The third-party notices of the repository give the commit of every asset and list every third-party file with its license.
Molecules and models. The SELFIES [4] VAE and code released with LOL-BO [2, 30] are MIT. GuacaMol [5] 0.5.2 is MIT; its data are CC BY 4.0, derived from ChEMBL under CC BY-SA 3.0. ZINC-250k [6], in the subset of Gómez-Bombarelli et al. [7], is free to use without a formal license; the file comes from the Apache-2.0
chemical_vaerepository. The code of Chembed [41] is MIT. The weights of Chembed declare no license, and our repository does not redistribute them. We trained the Power-Spherical VAE of Appendix § 11 with thepower_sphericalpackage of De Cao and Aziz [36], which is MIT.
Benchmarks. LassoBench [8] is MIT for the code and the synthetic tasks; its real-world DNA and leukemia data fall under the terms of the LIBSVM website. NAS-Bench-101 [9] and arch2vec [40] are Apache-2.0.
Software. PyTorch [42] 2.11.0, pycma [20] 4.4.4 and RDKit [43] 2026.03.1 are BSD-3. BoTorch [37] 0.17.2 and GPyTorch [44] 1.15.2 are MIT, and selfies 2.2.0 is Apache-2.0. The released code of Linear BO [21], GaBO [22] and the Riemannian Matérn GP [23] is MIT.
Ported code. Our port of the official TuRBO code [1] (https://github.com/uber-research/TuRBO) is used for every TuRBO row of the paper and for the TuRBO curve of Figure 1. That code is under the TuRBO license, which is non-commercial, so the port is distributed under the same license, with the change notice it requires. The BAXUS row of Table 12 ports the BoTorch BAXUS tutorial, which is MIT; this port shares its file with the TuRBO port and is distributed with it under the TuRBO license.
Our code is released under the MIT license, except the file with the two ports above.
NeurIPS Paper Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
Answer: [Yes]
Justification: The claims are supported in Section 4: Table 1 compares the methods on one matched set (its full version, with every row and the cell-level statistics, is in the repository), Table 2 gives the three recipe rows, Table 3 the factorial experiment whose contrasts (Table 19, Appendix § 16) carry the central claim, and Table 4 compares GEOWALK and GEOCMA across benchmarks; Appendices § 7–§ 16 give the details. The primary test is the exact task-level Wilcoxon test over the 11 tasks, with Benjamini–Hochberg correction within families. The abstract and conclusion state that this is a controlled study of a Gaussian-prior VAE benchmark, not a recommendation to use VAEs for molecular optimization; § 3.1 and the Limitations restrict it to and the 11 GuacaMol tasks of the protocol.
Guidelines:
The answer [N/A] means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A [No] or [N/A] answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: The Limitations paragraph of Section 5 states that the advantage of the shell is shown only for candidate sets drawn uniformly from the whole support; that the evidence comes only from the Gaussian-prior VAE benchmark studied and the 11 tasks of the protocol; that on nine further tasks the shell was ahead on three; that a second VAE (Chembed) gave no detected gain; that budgets above come from stored runs; and that flows, diffusion models and language-model spaces were not tested.
Guidelines:
The answer [N/A] means that the paper has no limitation while the answer [No] means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate “Limitations” section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren’t acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [N/A]
Justification: The paper is empirical and presents no theorems. The shell statistics of Section 3.2 and Appendix § 15 (mean norm, effective rank, a second-order description of the radius by two moments) are measured on the encoder means of the ZINC-250k reference set and describe the shell on the same encodings; they are not derived as a guarantee.
Guidelines:
The answer [N/A] means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross-referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: Algorithms 1 and 2 give complete pseudocode for GEOWALK and GEOCMA, with their constants in § 3.3 and § 3.4. § 3.1 specifies the tasks, seeds, cold-start protocol, budget accounting and statistical protocol, and Section 3.2 defines the acquisition, both maximizers, the box and the GP constructor. Appendix § 14 summarizes every baseline as it ran; the method cards in the repository give every deviation from the published method. The code, the run configurations, the result files and the generator of every table, figure and number are released (“Code and data availability”, Section 5). The SELFIES VAE weights are the public release accompanying LOL-BO [2].
Guidelines:
The answer [N/A] means that the paper does not include experiments.
If the paper includes experiments, a [No] answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general, releasing code and data
is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example (a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm. (b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully. (c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset). (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The code, the run configurations and the result files are released in the public repository given in the “Code and data availability” paragraph of Section 5, with instructions and the generator that writes every table, figure and number from the result files. GuacaMol, LassoBench, NAS-Bench-101 and the SELFIES VAE released with LOL-BO are public. ZINC-250k is the public subset distributed by Gómez-Bombarelli et al. [7]; it serves here as a reference set, not as the VAE’s training set.
Guidelines:
The answer [N/A] means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://neurips.cc/public/guides/CodeSubmissionPolicy) for more details.
While we encourage the release of code and data, we understand that this might not be possible, so [No] is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https://neurips.cc/public/guides/CodeSubmissionPolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer) necessary to understand the results? Answer: [Yes]
Justification: § 3.1 specifies the tasks, seeds, cold-start protocol and statistical procedure, and Section 3.2 the acquisition, the maximizers, the box and the GP. § 3.3 and § 3.4 and Table 5 give the settings of GEOWALK and GEOCMA, and Appendix § 8 their sensitivity. Baselines are adaptations of the published methods run in our harness, summarized in Appendix § 14, with every known deviation listed in the method cards of the repository; the factorial experiment is described in § 4.3.
Guidelines:
The answer [N/A] means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: The primary test of every comparison is the exact two-sided Wilcoxon signed-rank test on the 11 per-task mean differences, reported with the number of tasks on which a method is ahead (§ 3.1). Benjamini–Hochberg correction is applied within families: the five factorial contrasts, the three recipe rows of Table 2, the published-recipe tests of Table 15, and the LassoBench tests of the published methods; the task-level of Table 1 and Table 4 is uncorrected. The full version of Table 19 in the repository also reports a percentile bootstrap interval over tasks for each contrast. Cell-level Wilcoxon tests and Cohen’s are secondary, and every table prints the size of its matched set. Bands in Figure 1 are standard errors over seeds.
Guidelines:
The answer [N/A] means that the paper does not include experiments.
The authors should answer [Yes] if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g., negative error rates).
If error bars are reported in tables or plots, the authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Answer: [Yes] Justification: Appendix K names the machines and gives the total process-hours; a table in the repository gives the process-hours and the median time of one run for every block of runs, recomputed from the times recorded in the result files by a script of the repository. The runs other than the stored runs took about 647 process-hours on a server with two NVIDIA L40S GPUs and about 4,372 on the CPUs of a second server, most of them for checks of the published recipes. The stored runs that printed tables read took about 1,289 process-hours. The project also used compute for preliminary and failed runs that the paper does not report.
Guidelines:
The answer [N/A] means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn’t make it into the paper).
Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines?
Answer: [Yes]
Justification: The research uses publicly available benchmarks (GuacaMol, LassoBench, NAS-Bench-101) and does not involve human subjects, private data, or personally identifiable information. Dual-use risks of molecular optimization are acknowledged in Section 5.
Guidelines:
The answer [N/A] means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer [No], they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).
Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The Limitations paragraph of Section 5 notes that better low-budget molecular optimization could lower experimental cost (positive) and that it has dual-use risks if applied to harmful objectives (negative).
Guidelines:
The answer [N/A] means that there is no societal impact of the work performed.
If the authors answer [N/A] or [No], they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate Deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pre-trained language models, image generators, or scraped datasets)?
Answer: [N/A]
Justification: The paper releases optimization code, run configurations and result files only; the underlying SELFIES VAE and oracle implementations are pre-existing public assets released by their original authors. No new generative model, pre-trained language model, or large-scale scraped dataset is released.
Guidelines:
The answer [N/A] means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.
Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Every external asset is cited, and Appendix K gives its license and version. Our port of the official TuRBO code [1] (https://github.com/uber-research/TuRBO) is distributed under the non-commercial TuRBO license, with the change notice it requires. Third-party files keep their own licenses, listed in the repository’s third-party notices.
Guidelines:
The answer [N/A] means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset’s creators.
New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: GEOWALK, GEOCMA, the experiment scripts, the run configurations and the result files are released in the public repository given in the “Code and data availability” paragraph of Section 5. It contains installation instructions, the generator that reproduces every table, figure and number of the paper, the full card and the run status of every method, a license file (MIT for the authors’ code), and the notices of the third-party and ported files under their own licenses.
Guidelines:
The answer [N/A] means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [N/A]
Justification: This research does not involve crowdsourcing or human subjects.
Guidelines:
The answer [N/A] means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [N/A]
Justification: This research does not involve human subjects.
Guidelines:
The answer [N/A] means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigor, or originality of the research, declaration is not required.
Answer: [N/A]
Justification: The proposed methods (GEOWALK and GEOCMA) contain no LLM component. As declared on OpenReview, LLMs were used for data processing and filtering, implementing standard methods, editing, and drafting sections of the paper; this does not affect the core methodology.
Guidelines:
The answer [N/A] means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy in the NeurIPS handbook for what should or should not be described.
References
- [1]David Eriksson, Michael Pearce, Jacob R. Gardner, Ryan D. Turner, and Matthias Polocz ek. Scalable global optimization via local Bayesian optimization. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL https://proceedings.neurips.cc/paper_files/paper/2019/hash/6c990b7aca7bc7058f5e98ea909e924b-Abstract.html.arxiv.org/abs/1910.01739
- [2]Natalie Maus, Haydn T. Jones, Juston S. Moore, Matt J. Kusner, John Bradshaw, and Jacob R. Gardner. Local latent space Bayesian optimization over structured inputs. In Advances in Neural Information Processing Systems, volume 35, pages 34505–34518. Curran Associates, Inc., 2022. doi: 10.52202/068431-2500.DOI
- [3]Jaewon Chu, Jinyoung Park, Seunghun Lee, and Hyunwoo J. Kim. Inversion-based latent Bayesian optimization. In Advances in Neural Information Processing Systems, volume 37, pages 68258–68286. Curran Associates, Inc., 2024. doi: 10.52202/090717-2180.arxiv.org/abs/2411.05330
- [4]Mario Krenn, Florian Häse, AkshatKumar Nigam, Pascal Friederich, and Alan Aspuru-Guzik. Self-referencing embedded strings (SELFIES): A 100% robust molecular string representation. Machine Learning: Science and Technology, 1(4):045024, 2020. doi: 10.1088/2632-2153/aba947.DOI
- [5]Nathan Brown, Marco Fiscato, Marwin H. S. Segler, and Alain C. Vaucher. GuacaMol: Benchmarking models for de novo molecular design. Journal of Chemical Information and Modeling, 59(3):1096–1108, 2019. doi: 10.1021/acs.jcim.8b00839.DOI
- [6]John J. Irwin, Teague Sterling, Michael M. Mysinger, Erin S. Bolstad, and Ryan G. Coleman. ZINC: A free tool to discover chemistry for biology. Journal of Chemical Information and Modeling, 52(7):1757–1768, 2012. doi: 10.1021/ci3001277.DOI
- [7]Rafael Gómez-Bombarelli, Jennifer N. Wei, David Duvenaud, José Miguel Hernández-Lobato, Benjamín Sánchez-Lengeling, Dennis Sheberla, Jorge Aguilera-Iparraguirre, Timothy D. Hirzel, Ryan P. Adams, and Alán Aspuru-Guzik. Automatic chemical design using a data-driven continuous representation of molecules. ACS Central Science, 4(2):268–276, 2018. doi: 10.1021/acscentsci.7b00572.
- [8]Kenan Šehić, Alexandre Gramfort, Joseph Salmon, and Luigi Nardi. LassoBench: A high-dimensional hyperparameter optimization benchmark suite for Lasso. In Proceedings of the First International Conference on Automated Machine Learning, volume 188 of Proceedings of Machine Learning Research, pages 2/1–24. PMLR, 2022. URL https://proceedings.mlr.press/v188/sehic22a.html.
- [9]Chris Ying, Aaron Klein, Eric Christiansen, Esteban Real, Kevin Murphy, and Frank Hutter. NAS-Bench-101: Towards reproducible neural architecture search. In Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 7105–7114. PMLR, 2019. URL https://proceedings.mlr.press/v97/ying19a.html.arxiv.org/abs/1902.09635
- [10]Seunghun Lee, Jaewon Chu, Sihyeon Kim, Juyeon Ko, and Hyunwoo J. Kim. Advancing Bayesian optimization via learning correlated latent space. In Advances in Neural Information Processing Systems, volume 36, pages 48906–48917. Curran Associates, Inc., 2023. doi: 10.52202/075280-2124.DOI
- [11]Seunghun Lee, Jinyoung Park, Jaewon Chu, Minseo Yoon, and Hyunwoo J. Kim. Latent Bayesian optimization via autoregressive normalizing flows. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=ZCUwwRAaE1.arxiv.org/abs/2504.14889
- [12]Natalie Maus, Kyurae Kim, Geoff Pleiss, David Eriksson, John P. Cunningham, and Jacob R. Gardner. Approximation-aware Bayesian optimization. In Advances in Neural Information Processing Systems, volume 37, pages 21114–21140. Curran Associates, Inc., 2024. doi: 10.52202/079017-0665. 52202/079017-0665.DOI
- [13]Henry Moss, Sebastian W. Ober, and Tom Diethe. Return of the latent space COWBOYS: Rethinking the use of VAEs for Bayesian optimisation of structured spaces. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pages 44956–44970. PMLR, 2025. URL https://proceedings.mlr.press/v267/moss25a.html.arxiv.org/abs/2507.03910
- [14]Leonard Papenmeier, Luigi Nardi, and Matthias Poloczek. Increasing the scope as you learn: Adaptive Bayesian optimization in nested subspaces. In Advances in Neural Information Processing Systems, volume 35, pages 11586–11601. Curran Associates, Inc., 2022. doi: 10.52202/068431-0842.DOI
- [15]David Eriksson and Martin Jankowiak. High-dimensional Bayesian optimization with sparse axis-aligned subspaces. In Proceedings of the Thirty-Seventh Conference on Uncertainty in Artificial Intelligence, volume 161 of Proceedings of Machine Learning Research, pages 493–503. PMLR, 2021. URL https://proceedings.mlr.press/v161/eriksson21a.html.arxiv.org/abs/2103.00349
- [16]Ziyu Wang, Frank Hutter, Masrour Zoghi, David Matheson, and Nando de Freitas. Bayesian optimization in a billion dimensions via random embeddings. Journal of Artificial Intelligence Research, 55:361–387, 2016. doi: 10.1613/jair.4806.DOI
- [17]Amin Nayebi, Alexander Munteanu, and Matthias Poloczek. A framework for Bayesian optimization in embedded subspaces. In Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 4752–4761. PMLR, 2019. URL https://proceedings.mlr.press/v97/nayebi19a.html.
- [18]Ben Letham, Roberto Calandra, Akshara Rai, and Eytan Bakshy. Re-examining linear embeddings for high-dimensional Bayesian optimization. In Advances in Neural Information Processing Systems, volume 33, pages 1546–1558. Curran Associates, Inc., 2020. URL https://proceedings.neurips.cc/paper_files/paper/2020/hash/10fb6cfa4c990d2bad5ddef4f70e8ba2-Abstract.html.arxiv.org/abs/2001.11659
- [19]Nikolaus Hansen and Andreas Ostermeier. Completely derandomized self-adaptation in evolution strategies. Evolutionary Computation, 9(2):159–195, 2001. doi: 10.1162/106365601750190398.
- [20]Nikolaus Hansen, Youhei Akimoto, and Petr Baudis. CMA-ES/pycma on Github. Zenodo, 2019. URL https://doi.org/10.5281/zenodo.2559634.
- [21]Colin Doumont, Donney Fan, Natalie Maus, Jacob R. Gardner, Henry Moss, and Geoff Pleiss. We still don’t understand high-dimensional Bayesian optimization. In Proceedings of The 29th International Conference on Artificial Intelligence and Statistics, volume 300 of Proceedings of Machine Learning Research, pages 2629–2637. PMLR, 2026. URL https://proceedings.mlr.press/v300/doumont26a.html.arxiv.org/abs/2512.00170
- [22]Noémie Jaquier, Leonel Rozo, Sylvain Calinon, and Mathias Bürger. Bayesian optimization meets Riemannian manifolds in robot learning. In Proceedings of the Conference on Robot Learning, volume 100 of Proceedings of Machine Learning Research, pages 233–246. PMLR, 2020. URL https://proceedings.mlr.press/v100/jaquier20a.html.arxiv.org/abs/1910.04998
- [23]Noémie Jaquier, Viacheslav Borovitskiy, Andrei Smolensky, Alexander Terenin, Tamim Asfour, and Leonel Rozo. Geometry-aware Bayesian optimization in robotics using Riemannian Matérn kernels. In Proceedings of the 5th Conference on Robot Learning, volume 164 of Proceedings of Machine Learning Research, pages 794–805. PMLR, 2022. URL https://proceedings.mlr.press/v164/jaquier22a.html.arxiv.org/abs/2111.01460
- [24]Viacheslav Borovitskiy, Alexander Terenin, Peter Mostowsky, and Marc Peter Deisenroth. Matérn Gaussian processes on Riemannian manifolds. In Advances in Neural Information Processing Systems, volume 33, pages 12426–12437. Curran Associates, Inc., 2020. URL https://proceedings.neurips.cc/paper_files/paper/2020/hash/92bf5e6240737e0326ea59846a83e076-Abstract.html.arxiv.org/abs/2006.10160
- [25]Carl Hvarfner, Erik Orm Hellsten, and Luigi Nardi. Vanilla Bayesian optimization performs great in high dimensions. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 20793–20817. PMLR, 2024. URL https://proceedings.mlr.press/v235/hvarfner24a.html.arxiv.org/abs/2402.02229
- [26]Leonard Papenmeier, Matthias Poloczek, and Luigi Nardi. Understanding high-dimensional Bayesian optimization. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pages 47902–47923. PMLR, 2025. URL https://proceedings.mlr.press/v267/papenmeier25a.html.arxiv.org/abs/2502.09198
- [27]Zhitong Xu, Haitao Wang, Jeff M. Phillips, and Shandian Zhe. Standard Gaussian process is all you need for high-dimensional Bayesian optimization. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=kX8h23UG6v.arxiv.org/abs/2402.02746
- [28]Austin Tripp and José Miguel Hernández-Lobato. Genetic algorithms are strong baselines for molecule generation. arXiv preprint, 2023. URL https://arxiv.org/abs/2310.09267.
- [29]Miguel González-Duque, Richard Michael, Simon Bartels, Yevgen Zainchkovskyy, Søren Hauberg, and Wouter Boomsma. A survey and benchmark of high-dimensional Bayesian optimization of discrete sequences. In Advances in Neural Information Processing Systems, volume 37, pages 140478–140508. Curran Associates, Inc., 2024. doi: 10.52202/079017-4459.
- [30]Natalie Maus. Local latent Bayesian optimization (LOLBO). GitHub repository, official implementation of Local Latent Space Bayesian Optimization over Structured Inputs, 2022. URL https://github.com/nataliemaus/lolbo. Commit 0876147, 31 October 2022.
- [31]Frank Wilcoxon. Individual comparisons by ranking methods. Biometrics Bulletin, 1(6):80–83, 1945. doi: 10.2307/3001968.DOI
- [32]Yoav Benjamini and Yosef Hochberg. Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B (Methodological), 57(1):289–300, 1995. doi: 10.1111/j.2517-6161.1995.tb02031.x.DOI
- [33]Georgios Arvanitidis, Lars Kai Hansen, and Søren Hauberg. Latent space oddity: On the curvature of deep generative models. In International Conference on Learning Representations, 2018. URL https://openreview.net/forum?id=SJzRZ-WCZ.arxiv.org/abs/1710.11379
- [34]Tim R. Davidson, Luca Falorsi, Nicola De Cao, Thomas Kipf, and Jakub M. Tomczak. Hyperspherical variational auto-encoders. In Proceedings of the Thirty-Fourth Conference on Uncertainty in Artificial Intelligence, pages 856–865. AUAI Press, 2018. URL https://www.auai.org/uai2018/proceedings/papers/309.pdf.arxiv.org/abs/1804.00891
- [35]Jiacheng Xu and Greg Durrett. Spherical latent spaces for stable variational autoencoders. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4503–4513. Association for Computational Linguistics, 2018. doi: 10.18653/v1/D18-1480.DOI
- [36]Nicola De Cao and Wilker Aziz. The Power Spherical distribution. In ICML 2020 Workshop on Invertible Neural Networks, Normalizing Flows, and Explicit Likelihood Models (INNF+), 2020. URL https://arxiv.org/abs/2006.04437.
- [37]Maximilian Balandat, Brian Karrer, Daniel R. Jiang, Samuel Daulton, Ben Letham, Andrew Gordon Wilson, and Eytan Bakshy. BoTorch: A framework for efficient Monte-Carlo Bayesian optimization. In Advances in Neural Information Processing Systems, volume 33, pages 21524–21538. Curran Associates, Inc., 2020. URL https://proceedings.neurips.cc/paper_files/paper/2020/hash/f5b1b89d98b728673128a5fb112cb9a-Abstract.html.
- [38]Ingo Rechenberg. Evolutionsstrategie: Optimierung technischer Systeme nach Prinzipien der biologischen Evolution. Number 15 in Problemata. Frommann-Holzboog, Stuttgart, 1973. ISBN 3772803733.
- [39]Jens Jägersküpper. Algorithmic analysis of a basic evolutionary algorithm for continuous optimization. Theoretical Computer Science, 379(3):329–347, 2007. doi: 10.1016/j.tcs.2007.02.042.
- [40]Shen Yan, Yu Zheng, Wei Ao, Xiao Zeng, and Mi Zhang. Does unsupervised architecture representation learning help neural architecture search? In Advances in Neural Information Processing Systems, volume 33, pages 12486–12498. Curran Associates, Inc., 2020. URL https://proceedings.neurips.cc/paper_files/paper/2020/hash/937936029af671cf479fa893db91cbdd-Abstract.html.
- [41]Hugo Talibart and Dimitri Gilis. Learning a chemistry-aware latent space for molecular encoding and generation with a large-scale transformer variational autoencoder. Journal of Cheminformatics, 18(1):123, 2026. doi: 10.1186/s13321-026-01243-0.DOI
- [42]Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zachary DeVito, Martin Raison, Aly Khan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. PyTorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL https://proceedings.neurips.cc/paper_files/paper/2019/hash/bdbca288fee7f92f2bfa9f7012727740-Abstract.html.arxiv.org/abs/1912.01703
- [43]RDKit: Open-source cheminformatics. Release 2026.03.1, https://doi.org/10.5281/zenodo.19250388, 2026. URL https://www.rdkit.org.
- [44]Jacob R. Gardner, Geoff Pleiss, Kilian Q. Weinberger, David Bindel, and Andrew Gordon Wilson. GPyTorch: Blackbox matrix-matrix Gaussian process inference with GPU acceleration. In Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018. URL https://proceedings.neurips.cc/paper_files/paper/2018/hash/27e8e17134dd7083b050476733207ea1-Abstract.html.arxiv.org/abs/1809.11165

