WorksComputer Science
HapBO: Harnessed - Bayesian Optimization via Expert Natural Language Priors
Abstract
Bayesian Optimization (BO) can be substantially more sample-efficient when informed by prior knowledge about the problem. In practice, however, such knowledge is often available only in natural language, e.g., expert intuitions or documentation, rather than as the explicit probability distribution landscape that prior-guided BO methods may require. Leveraging it effectively demands both an expressive mapping from language to priors and robustness to information that may be misleading. We propose HapBO (Harnessed - Bayesian Optimization), which elicits beliefs once from a human expert, a large language model (LLM), or both, and deterministically compiles a structured specification into a reusable belief function (), with a rich representation not restricted to a Gaussian family. Building on -BO, HapBO applies this belief function through a bounded positive multiplier on any nonnegative acquisition, preserving positive acquisition support and requiring no LLM calls during optimization. We prove that, for the Expected Improvement acquisition function (EI), HapBO preserves the standard simple-regret rate up to a bounded factor, even under adversarial beliefs. We evaluate HapBO on 40 HPOBench tasks, a synthetic suite in 6-15 dimensions and two real-world applications: cortical neurostimulation and constrained solar design. HapBO achieves the lowest aggregate HPOBench regret and strong real-world task performance, with zero in-loop token cost. In real-world applications it improves the optimization outcome even at budgets of 12-20 queries, and at a small fraction of the wall-clock and token costs of other LLM-based methods.
Introduction
Bayesian optimization (BO) is widely used for optimizing expensive black-box functions. An evaluation may for instance require patient time, hours of computation, or a physical experiment. A surrogate model, typically a Gaussian process (GP), approximates the objective, and an acquisition function selects the next query [37, 28, 18]. In most real-world scenarios, BO never truly starts the search from zero, since the practitioner usually has additional knowledge about the problem. For example, a machine learning engineer knows the plausible range of each hyperparameter of the network she is designing [15], a neuroscientist knows which nervous system region controls a movement [5], and a solar thermal engineer can have an estimation of which design parameters must be balanced against capital cost [3]. Used well, such knowledge buys the same result in fewer evaluations, and the gain is largest where evaluations are scarcest: at small budgets, where the surrogate alone (the internal model used by BO) is not mature yet [25, 20]. In real BO applications, such knowledge is most naturally expressed in language rather than as a probability distribution: an expert can describe where good configurations lie, and an LLM can produce a comparable description from a task specification alone.
To take advantage of the prior belief, prior-guided BO techniques have been developed. These techniques allow an expert to inject a prior over the location of the optimum of the objective function. Since experts can make mistakes, these techniques may sometimes incorporate mechanisms that limit the damage when the prior is misleading [25, 20, 27] (Souza et al. 2021). However, these techniques assume the prior is already available in numerical form, which practitioners are rarely equipped to provide.
Here, our fundamental question is whether LLMs can bridge the gap between knowledge in its many native forms (e.g., a sentence by an expert, knowledge acquired during pretraining, or documents retrieved by web search) and a compiled belief function that is directly compatible with the landscape of GP models and BO acquisition functions.
Large Language Models (LLMs) trained on internet-scale datasets have demonstrated broad knowledge and reasoning capabilities, so the BO community has tried to incorporate them within the BO loop. Current LLM-guided BO algorithms may use LLMs to 1) warm-start the search, 2) narrow the search space, or 3) replace core BO components (surrogate model or the acquisition function) during optimization [35, 14] (Yuan et al. 2026). Our working hypothesis is that the GP-BO machinery is already well suited to provide high performance in most optimization tasks, from low to high dimension [26]. Moreover, in-loop LLM methods may incur substantial per-iteration cost, while lacking the calibrated uncertainty mechanisms developed for prior-guided BO techniques [31, 21]. Methods that elicit the belief once avoid this per-iteration cost, but most use it in one of two limited ways: to warm-start the search, so that the belief shapes only the first few evaluations [35], or to prune the search space, so that any region it excludes can never be revisited [33]. The first may discard the belief too early; the second may trust it too much. Patel et al. [38] distill the LLMs knowledge once into a utility function, learned from its stated preferences, that guides BO toward promising regions. HAPBO shares this elicit-once design, but compiles the belief deterministically from a typed language and injects it through a bounded multiplier.
To overcome these gaps, we introduce HAPBO (Harnessed - Bayesian Optimization). HAPBO elicits beliefs once from a human expert, an LLM, or both, and deterministically compiles a structured specification into a belief function (; Figure 1). Our contributions are as follows:
Figure 1. HAPBO (blue components). Left to right: an LLM reads the task card, which may include information stated by a human expert, and the PSL vocabulary and produces a document in the typed Prior Specification Language (PSL); a compiler turns the document into a prior landscape and a trust level . The prior is injected in the BO loop at the acquisition function level. Right: shaping re-ranks candidates toward plausible regions but cannot zero a positive base acquisition.
(i) To turn natural-language knowledge into a BO-compatible landscape, we designed the Prior Specification Language (PSL, v1.0), a vocabulary built for versatility and completeness: its layered tokens express the structure experts may want to indicate (baseline trends, marginal effects, pairwise interactions), and anchors let it approximate any landscape to arbitrary precision (Theorem 1). A PSL document compiles deterministically into a landscape, which is also compatible with traditional prior-injection methods (a prior on the GP mean or on the acquisition function).
(ii) We choose to build upon -BO, which allows HAPBO to apply this belief function through a bounded positive multiplier on any nonnegative acquisition, preserving positive acquisition support and requiring no LLM calls during optimization. We prove that, for the Expected Improvement acquisition function (EI), HAPBO preserves the standard simple-regret rate up to a bounded factor, even under adversarial beliefs.
(iii) To evaluate HAPBO in practice, we ask whether HAPBO can improve sample efficiency of State-Of-The-Art (SOTA) methods: 1) one-shot LLM strategies that warm-start or constrain the search space, 2) methods that keep the LLM inside the BO loop, and 3) traditional prior-free search. We specifically ask how robust HAPBO is against misleading LLM-elicited beliefs, a potential roadblock that its component -BO is designed to alleviate.
(iv) We also present a series of ablations to interrogate the value of each component equipped to HAPBO by our chain of design choices. Our questions include whether results vary with the injection site, the belief representation, and the source of the belief.
Related Work
Prior-guided BO. A substantial corpus of work focuses on allowing practitioners to inject a prior over the location of the optimum. The resulting methods differ in how they limit the damage of a misleading belief, which they do in one of four ways.
First, by a schedule. -BO weights the acquisition function by the prior landscape , raised to an exponent that decays with the number of iterations, regardless of the observed data (Hvarfner et al., 2022). Second, by accumulating evidence. The prior’s influence shrinks as observations contradict it, either through the model, as in BOPRO, which combines the prior with a data-driven model into a pseudo-posterior ((Souza et al., 2021)), and COLABO, which conditions the Gaussian process (GP) on the belief ([27]), or through the sampling policy, as in PRIORBAND, which adapts its reliance on prior-based samples in multi-fidelity hyperparameter optimization (HPO) ([36]), and HYPBO, which distributes effort between expert-specified hypothesis regions and the full space ([13]). Third, by explicitly testing the belief. DYNABO combines decay with a surrogate-based safeguard test that rejects flagged priors ([17]), and PSH proves a no-harm guarantee for prior-guided successive halving ([23]). Fourth, by bounding the prior’s influence structurally. -BO (although descending from -BO) does not insert a fading schedule, but rather places a fixed floor on the acquisition value, which results in quickly incorporating “good” priors, while drowning misleading ones, without a decay schedule and without deciding whether the belief is wrong ([20]). We build the HAPBO framework on the latter mechanism, but we modify the normalization of this construction to maintain a direct correspondence with our theoretical formulation.
A common challenge across all methods, model-based and acquisition-based alike, is that they require beliefs in a formal probabilistic form that the surrogate or the acquisition function can absorb. We here add the language interface that produces it.
LLMs for BO. Another line of work incorporates LLMs within the BO loop. These approaches vary mainly in the frequency with which the LLM is queried during optimization. Some approaches query the LLM before the search as a warm-starting strategy, either to select initial points to initialize the surrogate model ([35, 14]), or to produce a prior that persists throughout the search ([33]). Others query the LLM at every iteration of the optimization by using LLMs to replace core components of the BO loop such as the surrogate model or the acquisition function (AF), among others ([35, 10, 2] (Suwandi et al. 2025; Yuan et al. 2026)), or query the LLM on demand to guide the AF in the selection of the next point to sample ([14, 11]). Our working hypothesis is that warm-starting initial points alone may not exploit the full capacity of BO priors. Also, available methods that produce priors that persist during the search may miss a key mechanism for exploiting it effectively, and in consequence can stop their action at producing the prior, and use it to constrain the search space rather than to weight it. Finally, using LLMs in the loop introduces additional challenges of uncalibrated uncertainty, volatility, which can cause having these methods consistently outperformed by classical surrogate-based methods. They also incur a query cost for knowledge that helps only in domain-specific settings ([31, 21, 8, 30]). HapBO therefore proposes to query the LLM before the search, leaving core BO components unchanged. Rather than warm-starting the surrogate model, or constraining the search, we give the acquisition function a prior to use through the entire search.
Belief elicitation. Outside of BO, AUTOELICIT generates Gaussian priors over the parameters of a linear model [7]. Also, LLM-PRIOR converts rich, unstructured context, e.g. natural-language descriptions, data, or figures into analytically convenient probability distributions suitable for Bayesian inference [24]. HAPBO differs from these methods, since both stop at the prior, whereas ours is carried into the BO loop.
Methodology
Belief elicitation
The first step of HAPBO is eliciting a belief in natural language. This stage requires a prompt to an LLM. The second step is to provide the vocabulary to the LLM to transcribe the elicited belief into a document that is then compiled deterministically. Two cards are used to drive this step: the task card provides the LLM’s context, and the vocabulary card harnesses what the belief document can contain.
Task card. The task card gives the context needed by the LLM to write the belief document. At minimum, it shall contain three sections: the domain, which describes the system under study; the search space, which specifies each parameter; and a description of the optimization problem. The card can further contain human expert knowledge, pertinent data, and a scope of the search process (e.g., deep research, looking for literature sources, access to data). The design of the task card is open to the practitioner implementing HAPBO in their research, and benefits from prompt engineering skills developed with common LLM use. In one experiment, we distinguish between cards written from three knowledge sources, whose variants differ only in what the card contains. LLM-only cards ask the LLM to act as a domain expert and form the belief from its own knowledge; human-expert cards carry substantial expert’s belief, including available data from previous experiments; and mixed cards carry the expert’s belief written in their own words and ask the LLM to complement it with its own knowledge.
Vocabulary card. The vocabulary card is fixed by HAPBO design, and defines the Prior Specification Language (PSL, v1.0, see Appendix § 12), a closed grammar for writing beliefs, and restricts the LLM to translation: the LLM writes the belief in PSL and never computes a prior. PSL expresses per-dimension shapes, pairwise interactions, penalized regions, and anchor points. The resulting belief is joint over the search space and not restricted to a Gaussian family.
Belief document. Both cards are sent together in a single LLM call, which returns a belief document containing the typed search space, the list of beliefs, and a qualitative trust block. Each shaping belief carries a weight (its relative importance) and a provenance tag (its source), both used by the compiler (see § 3.2). An example of an elicited document is given in Appendix § 12.
Deterministic compilation
The compiler maps the belief document to the two quantities used by --BO: a prior landscape and a trust level (where a higher value means more trust in the prior). The compiler first validates the belief document against the PSL vocabulary, rejecting any unknown PSL token, field, or out-of-range value. It then translates each token into a predefined formula (see Table 2 and Algorithm 1, Appendix § 12), and computes from the trust block and the provenance tags by a fixed rule (Eq. (1)). It involves no LLM call and no randomness, so the same belief document always yields the same prior.
| PSL Token | Fields | Semantics |
| Base layer | ||
baseline | level [0, 1] (default 0.1); tilt per-dim [−1, 1] | default plausibility floor plus optional global slope |
| Marginal shapes (one dimension) | ||
bump | (dim, mode); ci; skew [−1, 1]; sign {peak, trough} | single peak or trough; split-Gaussian, from the CI |
trend | (dim, direction {increase, decrease}); knee | monotone lean, optional saturation at the knee |
threshold | (dim, at, active {above, below}); sharpness {soft, sharp} | regime change at a boundary (logistic step) |
periodic | (dim, period); phase | oscillatory structure (raised cosine) |
categorical | (dim, preferences level→ [0, 1]) | multiplicative preference gate over unordered levels |
| Interactions (two dimensions) | ||
joint_region | (dims, mode per-dim); ci per-dim | localized joint optimum (product of per-dim Gaussians) |
ridge | (dims, relation {positive, negative}); width {narrow, medium, wide}; through | trade-off line through two dimensions |
conditional | (when {dim, region}, then nested belief); mode {gate, synergy} | one dimension’s shape activates within a region of another |
| Regions and exact knowledge | ||
plateau | (region per-dim interval) | indifference band (flat raise) |
avoid | (region box or dim/mode/ci) | soft penalized region (subtractive; can never remove a candidate) |
anchors | (points [{coords, value [0, 1]}]); bandwidth (> 0, default 0.15) | exact values at known points, applied last |
| Meta tokens (objective properties, not location beliefs) | ||
smoothness | level / per-dim {very smooth . . . very rough} | suggests GP lengthscale |
noise | (level {low . . . very high}); repeats recommended | suggests GP noise floor |
modality | expectation {unimodal, few, many}; n optima | suggests exploration appetite |
| Trust block (document level) | ||
trust | (overall confidence {very low, low, medium, high, very high}); notes | source’s stated confidence |
Table 2. Declaration of the PSL vocabulary. PSL v1.0: Required fields in parentheses; all shaping tokens additionally require weight and provenance . CI fields are 90% intervals (half-width = ).
Building the prior . Each PSL token has a predefined closed-form shape on the search space, encoded to the unit cube so that all dimensions share the same scale (Table 2, Figure 7). The compiler proceeds to build the prior by sequentially adding the weighted shapes to a baseline, subtracting penalties, clamping the sum at zero, multiplying it by the preference gates, and rescaling it to . The anchors are then used to set exact specified values at the anchor points. Eqs. (4)–(7) and Algorithm 1 in Appendix § 12 give the full procedure; Figure 8 visualizes the compiler algorithm.
Figure 7. PSL components as belief functions. Each panel is a minimal PSL document on a two-dimensional unit square ( horizontal, vertical), compiled by Algorithm 1 into . In the categorical panel has five unordered levels; in the anchors panel the anchor values override a tilted baseline at the circled points. The last panel stacks several components in one document. Meta tokens (smoothness, noise, modality) and the trust block describe the objective rather than the optimum’s location, so they have no panel.
Figure 8. Compiling one PSL document. The sample PSL document (top) refers to the XGBoost task on kc1 of App. B. It lives on the space of two XGBoost parameters: . Step 1 Irregular document candidates (dots) become a regular grid. Step 2 Closed forms are applied. Step 3 A signed sum is performed, and clamped at zero. Step 4 Rescaling on (dots: Halton points; crosses: declared points); is attained at the declared mode. Step 5 Anchors (circles: , labelled with ) will override the field within radius (dashed: support of ). Top right: from the card to Eq. 1.
Determining the trust level . We opted for an automatic simplistic rule, that allows us to illustrate two possible practitioner choices at once. We combine the document’s LLM-stated confidence $c_{\mathrm{overall}} with the provenance of each belief,
where is the average weighted by the belief weights. A higher means more trust in the prior. Capping at 0.9 bounds the worst-case constant of our guarantee at 10 (Proposition 2 in App. § 23.§ 2). In our implementation, we set the belief stated by the expert to a higher trust level than one supplied by the LLM. This idea is set to showcase at once the combination of an LLM-stated confidence, with a more ‘human’ fixed trust rule. Nevertheless, a practitioner using HAPBO could effectively substitute our simplified -determination module with an LLM decision-maker, or even with a human-tuned cursor.
Shaping the acquisition function
HAPBO leaves the BO loop (presented in Appendix § 11) unchanged except for the final ranking. At each query it selects
where is the raw base acquisition, e.g., EI:
with ; (App. § 11).
The prior landscape and the trust are compiled once before the run (§ 3.2).
Following Guay-Hottin et al. [20], the belief acts as a bounded multiplier (which their experiments found to be more robust to misleading priors than -BO); unlike their formulation, which min–max normalizes the acquisition before multiplying, we apply to the raw acquisition, maintaining a direct correspondence with our theoretical formulation (Definition 1 in App. § 23). When comparing with Vanilla BO, the surrogate, the base acquisition, and the candidate set are kept identical, and both start from the same random point. Because the belief enters only through , the rule applies to any nonnegative acquisition (Appendix § 18).
Theory guarantees
The PSL harness itself is free of expressive bottlenecks, because 1) anchors-only documents can reproduce any belief table on a finite candidate set exactly, and 2) approximate any continuous belief uniformly with error in the number of anchors over continuous dimensions (Theorem 1).
Regarding the - harness, since , the multiplier satisfies , so no candidate with positive acquisition is ever ruled out, and the selected point is always within a factor 10 of the raw acquisition’s maximum (Proposition 2). For noiseless Expected Improvement with a fixed kernel, shaping preserves the classical simple-regret rate of Bull [6] up to a constant , where is the prior’s miss at the optimum (Theorem 2, Corollary 2); a prior that is correct at the optimum costs nothing, however wrong it is elsewhere. For noisy GP-UCB, the regret bound gains an additive term that depends on the belief only through its value at the optimum (Theorem 3).
Full statements, assumptions, and proofs are given in Appendix § 23.
Experimental setup
Benchmarks. HPOBench contributes 40 tabular hyperparameter-optimization tasks spanning five learner families ((Espensperger et al., 2021)). The NeuroMOSAICS Cortical panel optimizes evoked muscle responses across 40 rat-M1 electrode tasks (eight hindlimb muscles recorded in six rats) ([29]). SOLAR MAXEFF_RE is a constrained solar-design black box ([3]). We further designed an analytic panel, containing high-dimensional synthetic functions: Hartmann-6, Styblinski–Tang-7, Levy-10, and Ackley-15 ([22, 34, 1] (Styblinski & Tang 1990)). For this panel, we generate three controlled prior fidelities per experiment, by manipulating the synthetic functions to create alternate functions within the same family but having a diverse landscape (correlation to , from poor to accurate). We then feed the manipulated function to the elicitor, informing the LLM that this is our expert belief for the desired landscape. The elicited prior is tested against the true synthetic function, with shifted optima to exclude possible memorization effects (e.g. Ackley optimum is no longer at the origin). We further discuss LLM memorization leak control in Appendix § 21.§ 1.
Baselines. We compare HAPBO against an array of methods. Some are prior-free: vanilla BO, random search and TuRBO (Eriksson et al., [16]). Other methods are prior-guided, like CoLABO (Hvarfner et al., [27]), which conditions the GP posterior on a user belief, which we provide via an LLM query following BORA’s LLM-prior recipe (Cissé et al., [14]), and we particularly focus on other methods incorporating a LLM-in-the-loop approach: LLAMBO (Liu et al., [35]), which uses the LLM as surrogate and candidate sampler; BORA (Cissé et al., [14]), which lets the LLM propose regions during the search; and LGBO (Yuan et al., 2026), which shifts the GP mean by an LLM preference at each query. The in-loop methods receive the same task card as HAPBO, through their own interface, and query the same locally served LLM. Where a baseline admits more than one faithful implementation, we report the strongest variant we could construct (Appendix § 20). Further methods, CAKE (Suwandi et al., 2025), CALIPER (Chen et al., [9]), PRIORBAND (Mallik et al., [36]), PRIMO (Basu et al., [4]), HYPBO (fed LLM hypotheses) (Cissé et al., [13]), and LLM-EI (Patel et al., [38]), are reported in Appendix § 22.
Protocol. Every baseline runs with the same locally served LLM (Qwen3.5-122B; Qwen Team, 2026) and the same task descriptions, served through its native interface. HAPBO and Vanilla BO share the surrogate, acquisition, candidate set, and initial point (same as random search), and differ only in the presence or absence of the prior.
For HAPBO, priors are elicited once per task at temperature 0 for HPOBench, cortical and SOLAR, and as three independent single-call draws per card for the synthetic panel (default temperature 0.6): the elicitor receives a de-identified task card and must answer in the closed PSL vocabulary (Appendix § 12), with validator errors fed back for up to four corrective turns (Appendix § 13). Per-benchmark normalizations, SOLAR’s constraint handling, and per-query wall-clock and token costs are detailed in Appendix § 14.
Metrics. We evaluate query-wise best-so-far performance. Simple regret is used when a reference optimum is known; SOLAR reports best feasible efficiency because its optimum is unknown. Means and uncertainty bands are computed over tasks or subjects after averaging 10 paired seeds.
Results
HPOBench and real-world tasks
We first compared HAPBO to the SOTA and traditional baselines on HPOBench. HAPBO has the lowest grand-average regret curve from the second query on (Figure 2). Its mean raw endpoint regret, averaged within and then across the five model families, is 0.0026, compared with 0.0039 for CoLaBO, 0.0145 for its matched random-search control, 0.0074 for LLAMBO, 0.0092 for its matched vanilla-BO control, 0.0053 for LGBO, and 0.0048 for BORA. The rank panel in Figure 2 positions HAPBO ahead of every baseline in the performance ranking across HPOBench problem families. Two further SOTA methods, CAKE and CALIPER, are competitive under a faithful prior but do not overtake HAPBO (Appendix § 22, Figure 20). PRIORBAND and PRIMO are less performant under the same good LLM-elicited prior and substantially on par under an adversarial prior (Figure 21). HYPBO with LLM hypotheses and LLM-EI also trail HAPBO under both faithful and adversarial priors (Figure 22).
Figure 2. HPOBench. Average rank and normalized simple regret as a grand mean over 40 tasks for five model families. Per-family panels in App. G; bands are ±SEM across tasks after averaging seeds.
Figure 20. CAKE and CALIPER vs. HAPBO on HPOBench. Normalized simple regret (lower is better) per model family under a faithful prior (top) and an adversarial prior (middle), with grand means over the 40 tasks (bottom; faithful left, adversarial right). All arms share the GP surrogate and matched random initialization across ten seeds; CAKE is prior-agnostic, so the same curve appears in both conditions.
Figure 21. PriorBand and PriMO vs. HAPBO on HPOBench. Normalized simple regret (lower is better) per model family under a faithful prior (top) and an adversarial prior (middle), with grand means over the 40 tasks (bottom; faithful left, adversarial right); ten seeds. PriorBand and PriMO are plotted against full-fidelity-equivalent cost (budget 16), the other arms against queries.
Figure 22. HypBO and LLM-EI vs. HAPBO on HPOBench. Normalized simple regret (lower is better) per model family under a faithful prior (top) and an adversarial prior (middle), with grand means over the 40 tasks (bottom; faithful left, adversarial right). All arms run for ten seeds; HypBO and LLM-EI read the same task card as HAPBO.
In two real-world experiments, pertaining to cortical neurostimulation and solar design, HAPBO is the most performant method (second best to LLAMBO on cortical). However, unlike LLAMBO, it never fails completion. Table 1 collects the endpoints.
Table 1: Endpoint performance per task family. HPOBench simple regret for each of the five model families (lr, rf, xgb, nn, svm; lower is better), cortical exploration performance (higher is better — the same unit as Figure 3A), and SOLAR best feasible efficiency (higher); regrets are comparable across HPO families because all are validation error. fail% is the pooled fraction of task–seed runs that did not complete across every family (HPO, cortical, and SOLAR). HapBO uses matched random initialization on HPO, cortical, and SOLAR alike. Bold marks the best value in each performance column.
| Method | lr | rf | xgb | nn | svm | cortical | SOLAR | fail% |
| HapBO | 0.0022 | 0.0015 | 0.0020 | 0.0052 | 0.0021 | 0.785 | 4881.2 | 0.0 |
| ColaBO | 0.0014 | 0.0031 | 0.0022 | 0.0102 | 0.0028 | 0.735 | 3848.9 | 0.0 |
| LLAMBO | 0.0017 | 0.0201 | 0.0021 | 0.0090 | 0.0041 | 0.819† | 4874.9† | 1.9 |
| LGBO | 0.0023 | 0.0088 | 0.0020 | 0.0075 | 0.0060 | 0.785 | 4789.6 | 0.0 |
| BORA | 0.0022 | 0.0074 | 0.0020 | 0.0081 | 0.0042 | 0.737 | 4858.9 | 0.0 |
| vanilla BO | 0.0020 | 0.0243 | 0.0020 | 0.0119 | 0.0059 | 0.766 | 4877.9 | 0.0 |
| random | 0.0029 | 0.0387 | 0.0023 | 0.0094 | 0.0191 | 0.690 | 4687.3 | 0.0 |
Table 1. Endpoint performance per task family. HPOBench simple regret for each of the five model families (lr, rf, xgb, nn, svm; lower is better), cortical exploration performance (higher is better — the same unit as Fig. 3A), and SOLAR best feasible efficiency (higher); regrets are comparable across HPO families because all are validation error. fail% is the pooled fraction of task–seed runs that did not complete across every family (HPO, cortical, and SOLAR). HapBO uses matched random initialization on HPO, cortical, and SOLAR alike. Bold marks the best value in each performance column.
Figure 3. Real-world tasks. A: NeuroMOSAICS cortical exploration performance (higher is better); n=40 technical replicates (muscle outputs), seeds averaged within each map. B: SOLAR MAXEFF best feasible efficiency; zero denotes that no feasible point has yet been found.
Cost. The efficiency gains are bought with modest compute overhead. HAPBO pays the LLM once, at setup (~15,000 tokens to elicit and compile one HPO prior), then runs as a pure GP loop at 0.20 s and zero tokens per query, the same as vanilla BO. Conversely, SOTA baselines have larger expenditures, with LLAMBO spending ~79,700 tokens and 28.6 s per query, CoLaBO 104 s per query for its prior-conditioned sampling. BORA and LGBO, which engage in more compute time per query, manage to mitigate the tokens/query figure, sitting at few hundreds of tokens per query on average (App. Table 4).
Robustness in the accurate-to-poor prior spectrum
We next ask what price an adversarial misbelief would carry within the optimization process. An enticing feature of prior-based BO methods is the capacity to disregard prior information that is proven non-performant. Eventually, the ideal method would be expected to close the gap, query after query, with prior-free variants under these conditions.
We also propose a synthetic function experiment, where the prior fidelity (as a modified function expressed in natural language with an equation) is controlled for exact correlation levels with respect to the true target function to be optimized. Our experiment makes the cost of prior quality visible for every method on one display (Figure 4). At all values, prior-using methods have an advantage with respect to prior-free controls (except LLAMBO, which seems to only offer acceptable performance at ). This advantage modulates with the quality of information received: it reduces as falls to 0.5 and 0.1. Figure 4 shows that HAPBO’s performance is always best across all conditions.
Figure 4. Synthetic prior-fidelity sweep. Grand-average simple regret under trustworthy (), medium (), and poor () priors on 4 synthetic functions. Each function is evaluated at its own budget and aligned only by fraction of budget for this summary.
We then tested HAPBO and SOTA methods performance under matched adversarial priors for HPO (prior placed at each task’s empirical anti-optimum), with the same hostile description passed to every baseline. HAPBO initially pays for the misleading belief but recovers better than any SOTA alternative (except BORA) toward the strongest prior-free curves of random search (Figure 5).
Figure 5. Adversarial HPOBench. Average task-wise rank (left; over the task grid complete for every method) and grand-mean normalized simple regret over 40 tasks (right) when every method receives the same hostile task description through its native interface. Per-family panels in App. G; bands are SEM across tasks after averaging ten seeds.
Elicitation modalities
Knowledge sources. The same compiled interface accepts LLM-only, human-only, and combined beliefs. We show an example of this three-way usage of the LLM elicitor interface on the NeuroMOSAICS Cortical panel (Appendix § 16). We elicit the prior with 1) no hint (LLM only), 2) a modest injection of human-level expertise in the form of a single informative sentence (LLM+human), and 3) substantial prior information in the form of previous experimental data not included in the benchmark dataset (human only). The LLM+human and human-only priors take less than half the total number of queries to hit performance levels aligned with the final one of vanilla BO; the LLM-only prior does not improve on vanilla BO because of lacking domain knowledge (Figure 13G–I). Just a single line of human expert information thus improves HapBO performance in this niche application.
Figure 13. Prior source and cortical optimization, complete panel. A–C: compiled belief functions for LLM-only, LLM+human, and human-only knowledge. D–F: three example ground-truth response maps from the NeuroMOSAICS dataset, each normalized independently to expose spatial structure; they are not paired with the priors above. G–I: each prior mode against matched vanilla BO over twelve queries (budget 12); mean ± SEM clustered over six rats after averaging muscles and ten seeds.
Elicitor LLM. We further test different elicitor LLMs, including commercial frontier models (App. I). Leveraging more potent models can result in a more accurate modeling of the target function by PSL, albeit with modest gains in regret in the BO phase.
Ablations
Injection site. In this ablation we ask whether changing the injection site for a prior leads to a difference in performance, as expected from literature (Hvarfner et al., [25]; Guay-Hottin et al., 2025). We fixed the representation and moved the site: 1) the bounded acquisition multiplier of HAPBO, 2) the no-floor -BO, 3) a prior to the mean of the surrogate (prior-to-mean), or 4) the initial queries (prior-warm-start), with a prior-only (non-BO) greedy arm and no-prior controls. Across HPOBench and the synthetic functions, HAPBO is the strongest landscaped method (vs prior-to-mean), with equal final performance to -BO on good priors but more robust than -BO on misleading or intermediate priors. -BO is known to be less robust to low-fidelity priors across benchmarks [20]. As expected, all landscaped algorithms avoid dramatic collapse of performance when the prior is misleading (Figure 9). There, HAPBO’s loss is modulated by its assigned trust weight (Figure 11).
Figure 9. Prior injection site ablation. One compiled prior (, ) per task; every arm shares the GP surrogate, base EI, and curves are query-wise mean SEM over the 40 HPOBench tasks (16 queries) and the four analytic functions aligned by fraction of budget. A faithful HPO prior, B adversarial HPO prior, C synthetic , grand mean over the four analytic functions.
Figure 11. Trust weight of HAPBO. HAPBO (-) with the trust weight fixed at , 0.7, 0.5, 0.3, or 0.1 instead of the LLM-compiled ; 40 HPOBench tasks 10 seeds, 16 queries, matched random start; mean SEM over tasks. Left: good prior. Middle: misleading prior. Right: misleading prior, final regret against . Colors run from the highest (darkest) to the lowest trust; dashed: vanilla BO (); LLM-compiled : thick line (left, middle) and outlined marker (right).
Representation. In this ablation we ask whether PSL holds a substantial advantage with respect to three more naive prior-building languages (point-only, direct LLM elicitation and gaussian-only). We show that all structured representations outperform direct LLM scoring when a good prior is available. Full PSL is best on the synthetic suite at , and always aligned with the other best prior-based arms on the good prior (point-only) and adversarial prior (gaussian-only and direct-LLM). Of note, direct-LLM elicitation in high-dimension PSL is not possible in cases such as our synthetic suite, since the fixed anchor set degenerates in 7–15D (Figure 6; ranks and per-function and per-family panels in App. Fig. 10).
Figure 6. Prior representation ablation. The injection site is fixed at the bounded BO multiplier; only how the belief becomes varies: full PSL (ours), point-only, Gaussian-only, direct-LLM elicitation, and vanilla BO (no prior). Grand-mean normalized simple regret on A: the synthetic suite at (four analytic functions, aligned by fraction of budget), B: HPOBench with the good prior, and C: HPOBench with the adversarial prior. Direct-LLM is absent from the synthetic suite, because its fixed anchor set degenerates in 7–15D. Ten seeds per task (synthetic prior arms: three documents ten seeds per function); mean SEM across tasks; lower is better. Average ranks and per-function and per-family panels in App. Fig. 10.
BO variants. In this ablation we ask whether the bounded multiplier depends on the BO machinery around it, since it acts only on the acquisition ranking. We repeat the matched prior-on/prior-off contrast under EI, PI, and UCB, and under sequential versus batch selection (Kriging Believer [19], ), and three GP kernels. Across all three acquisitions and kernels, shaping reduces final normalized HPOBench regret by 2.5–12×, with the largest cortical gains at low budget, and a boost of early performance is present under batch selection (Appendix § 18, Figure 16).
Figure 16. Bounded shaping across BO variants (matched arms; only the stated ingredient varies; details and conditions in the text of this appendix). A–C, acquisition functions: matched HapBO vs. vanilla BO under EI, PI, and UCB; top rows HPOBench (normalized simple regret, lower is better; 40 tasks), bottom rows cortical (normalized performance, higher is better; technical replicates (muscle outputs); human+LLM prior, budget 12 queries); ten seeds. D–E, batch selection: the same matched EI contrast under sequential () and batch (, Kriging Believer) selection; D reproduces the EI panels of A. F–H, GP kernel: EI-based contrast with a Matérn-5/2 (the kernel of A–E), RBF, or Matérn-1/2 kernel.
Knowledge leak control
The objective of this paper is to determine, when LLM knowledge about a given problem exists, whether this can be integrated, by proper harnessing, in BO. However, there is a type of existing knowledge that is less desirable when testing any LLM-prior method on public benchmarks, which is contamination: a strong prior might reflect recall of the final benchmark answer rather than transferable expertise. We address this with two complementary controls: a direct probe of what the elicitor natively knows, and an additional objective whose optimum cannot have been memorized (Appendix § 21).
First, we show that no elicitor recalls an optimum on any real task: the blind guess recovers 48–68% of the head-room on HPOBench and falls below the map average on the cortical grid, while naming a synthetic function outright recovers its optimum exactly. Second, we build a bespoke CNN benchmark whose response surface is thus completely unseen to the model. There, HAPBO again reaches the lowest simple regret (Figure 19).
Figure 19. Leak-control benchmark. (A) the bespoke convolutional encoder optimized as an un-memorizable objective, with the four searched hyperparameters highlighted. (B) query-wise simple regret (log axis, mean SEM over ten seeds) on the 750-configuration grid. The optimum is a property of our network, grid, and training protocol and appears in no public corpus, so no method’s prior can reflect memorization.
Conclusion and future work
We presented HAPBO, a framework that takes natural-language beliefs, from a human expert, an LLM, or both, and converts them into a structured specification, which is then deterministically compiled into a reusable belief function with a rich representation palette. This belief function can be used by any BO method admitting a landscaped prior; we instantiate it with -πBO, which is robust to erroneous beliefs. Since the belief function is compiled once before optimization, the resulting pipeline requires no LLM calls during optimization, and rather admits a traditional (shaped) BO loop. Results on 40 hyperparameter optimization problems, a synthetic suite in 6–15 dimensions, and two real-world applications show that HAPBO outperforms methods that keep the LLM in the loop under small budgets, in terms of performance, optimization time, and token cost; our experiments also show that HAPBO is very robust to misleading or poor priors, since such priors tend to only waste a small budget, and no region of the search space is ruled out by the prior-injection technique.
Several limitations remain. First, the quality of the compiled belief depends on how information is presented to the LLM when converting natural language into a specification; more broadly, each component of the pipeline is modular and further research could improve some of our design choices. Elicitation design in particular is a promising direction for future work. In this regard, PSL was built for versatility and completeness: its layered vocabulary covers order-0, marginal and pairwise structure, and anchors make it dense in the space of landscapes (Theorem 1). Future work could optimize PSL itself, for instance by refining its token set, the compiled shapes, or the guidance given to the elicitor, to further improve the toolbox available for elicitation. Second, HAPBO relies on one-shot elicitation: the belief function is compiled once before optimization and cannot be updated with what the search reveals. A natural extension is a hybrid approach in which the LLM occasionally intervenes during optimization to refine the belief. Finally, a misleading prior can still degrade performance relative to vanilla BO under small budgets; further limiting this negative effect is an additional available direction for future work.
Overall, we here advocate for a black-box optimization approach that uses the LLM offline to construct an auditable belief, then let an ordinary BO loop decide how much the data flow should trust or overturn it.
AI USE STATEMENT
In this work, the authors used generative AI tools to formulate mathematical claims, provide critical ingredients for proving mathematical claims, assist in the writing of proofs, implement methods, and assist with translation. The authors did not use generative AI tools to help develop theoretical models or conceptual frameworks, propose or refine hypotheses, design or provide feedback on research methodology or experiments, or interpret results. Supporting qualitative and thematic data analysis is not applicable to this work. Additionally, the authors used generative AI tools to generate synthetic datasets (limited to the synthetic function experiment and the bespoke CNN experiment in appendix), reformat datasets (limited to SOLAR, allowing a tabular format for testing on an offline batch run), create or modify scientific figures or images, create or edit software code, draft parts of the paper, and edit the paper to improve readability. In each case, the authors verified that the scientific figures were accurate and free of errors, two authors verified and tested the code generated by AI coding agents, and the authors revised the drafted text. The authors take responsibility for the final content of this work, including any text, claims, or artifacts produced with the aid of generative AI.
ETHICS STATEMENT
The cortical experiments use a public rodent neurostimulation dataset [29]; no animal experiments were conducted for this work.
REPRODUCIBILITY STATEMENT
In order to make HapBO results reproducible, we provide the source code for this project at https://github.com/mbonizzato/HapBO. There, readers have access to documentation with instructions on how to run each of the experiments presented in the main text and the appendix. We also provide readers with the pipeline that allows them to generate their own priors by modifying the appropriate prompt_card.md. The data processing used for each dataset and instructions on how to download the datasets can be found in benchmark/README.md. For the theoretical results, we also provide explanations of any assumptions and a complete proof of the claims in the appendix.
ACKNOWLEDGMENTS
We thank the Digital Research Alliance of Canada (Calcul Québec) for providing computational resources for several arms of this study. This research was supported by the New Frontiers in Research Fund – Transformation project RE-MOVE (NFRFT-2024-00209), the Natural Sciences and Engineering Research Council of Canada (NSERC; RGPIN-2023-04370), the equipment used was procured through a grant from Canada Foundation for Innovation.
Gaussian-process Bayesian optimization
BO seeks to find for an expensive black-box function in as few evaluations as possible [37, 28, 18]. Starting from initial observations with a possibly noisy , each iteration 1) fits a probabilistic surrogate to , 2) maximizes an acquisition function that scores candidates by the surrogate’s mean and uncertainty, 3) evaluates the maximizer, and 4) augments . The incumbent is the best response observed so far. The surrogate throughout this paper is a Gaussian process, : conditioned on under a Gaussian likelihood with noise variance , the posterior at a test point is Gaussian with
where and is the covariance matrix of the observed inputs. The kernel sets the correlation structure; we use the Matérn-5/2 kernel, with distance and lengthscale , giving .
Every query maximizes the acquisition over the candidate set (the grid of options for each task).
The three standard acquisitions used in this paper are
where , with and the standard normal CDF and PDF and the confidence parameter of GP-UCB (Srinivas et al., 2010). HaPBO leaves every element of this loop unchanged except the final ranking, which it multiplies by the bounded factor of equation 2.
The Prior Specification Language (PSL)
A belief is one JSON document with three parts:
a
schema(the typed search space restated by the source),a list of
beliefs,a
trustblock.
Table 2 declares the vocabulary (PSL v1.0, Figure 7). Three primitives (bump, joint_region, anchors) form a complete basis, exact on finite candidate sets and dense in continuous landscapes, with an explicit anchor-budget rate (App. § 23). The remaining tokens are macros that name recurring patterns one token each; adding macros cannot reduce expressivity (App. § 23). Every shaping belief carries a weight and a provenance tag; anchors carries no weight (its values are exact by definition), and the meta tokens describe the objective rather than the optimum’s location.
Compilation of . In plain terms, the compiler draws every stated shape on the encoded unit cube, adds them up (penalties subtract), keeps the total nonnegative, applies the preference gates, rescales the result to , and finally lets exact anchors override the field wherever the source knew the answer. Algorithm 1 gives the procedure; each and is the closed form declared in Table 2.
The normalization extrema are computed over a fixed quasirandom reference set augmented with the anchors and every belief’s declared mode or centre, so a narrow joint peak cannot be missed by the normalizer. The anchor blend uses compactly supported (Wendland-type) weights with default bandwidth 0.15 ( vanishes beyond distance bandwidth from , as distances are normalized by ) and is exact at anchored points, which is what makes the anchor basis exact and dense (App. § 23). Compilation is deterministic.
The compiler draws every stated shape on the encoded unit cube and combines them in five steps. Let and be the baseline level and tilt, and the weight of belief .
(i) Encoding. Each of the dimensions is mapped onto (log-spaced ranges on the log scale, ordinal and categorical levels by rank), so that a configuration becomes .
(ii) Shapes. Each belief is drawn as its closed form (Table 2).
(iii) Sum, clamp and gates. The shapes are added, each scaled by its weight; penalties (avoid and troughs) subtract:
The score is clamped at zero and multiplied by the gates (categorical preferences and synergy conditionals):
(iv) Normalization. The score is rescaled with its extrema over the fixed reference set :
(v) Anchors. Anchor points override the field locally through compactly supported weights :
so that exactly. Algorithm 1 gives the same procedure; each and is the closed form declared in Table 2.
Algorithm 1 Compilation of the belief function from a validated PSL document.
Require: validated document ; query points on the encoded unit cube
Ensure: belief function
▷ base layer
for all additive beliefs do
▷ avoid and troughs subtract
end for
▷ clamp
for all gates do
▷ gates take values in
end for
unscrambled Halton points ▷ reference set
▷ min–max normalization
for all anchor sets with points and weights do
▷ applied last; exactly
end for
return $\pi
Compilation of ((1) of the main text). represents the overall confidence assigned to . Choosing the most appropriate for a given task can be a standalone research question. The current research is designed in a modular way, and we decided to opt for a simplification for the choice of assigning . In this paper, the overall confidence maps to for the five levels; each belief’s provenance maps to for:
user_stated(highest confidence),user_confirmed,literature,llm_domain,llm_default(lowest confidence).
with the average weighted by the belief weights. In practice, this leads to a medium-confidence document built entirely on model-internal knowledge, compiling to in most of the present work; only a user-stated, very-high-confidence document raises this value (e.g., the cortical priors sourced by a human expert).
Several different choices are possible and the HAPBO method is designed to accommodate a future selection module.
The original -BO work [20] suggested a default for all priors, which would be a reasonable choice here that would not affect the results qualitatively. The present work uses the more nuanced of Eq. (1) to illustrate that the PSL expressivity can be modulated by design, by specific choice rules (our implementation), or even by LLM feedback.
Meta tokens and the surrogate. smoothness, noise, and modality can configure the GP (per-dimension lengthscales, noise floor, exploration). For a fixed surrogate, the compiled field enters the noiseless EI guarantee through a multiplicative constant. The separate noisy GP-UCB bound also contains an additive misplaced-trust term, which can grow linearly under static trust (App. § 23). In all campaigns reported in this paper the meta-token channel is disabled. Future work can investigate the opportunity to further guide the BO approach by having the LLM condition the GP properties. The authors have considered this capacity to be beyond the scope of the current research.
A real elicited document. The prior compiled for XGBoost on the kc1 task, as returned by the elicitor (numeric values rounded for display):
{ "schema": { "objective": "maximize",
"dimensions": [ {"name": "colsample_bytree", ...},
{"name": "eta", ...},
{"name": "max_depth", ...},
{"name": "reg_lambda", ...} ] },
"beliefs": [
{ "token": "baseline", "level": 0.1 },
{ "token": "joint_region", "dims": ["eta", "max_depth"],
"mode": { "eta": 0.021, "max_depth": 8 },
"ci": { "eta": [0.0098, 0.099], "max_depth": [5, 13] },
"weight": 1.0, "provenance": "llm_domain",
"note": "Standard sweet spot for XGBoost: moderate learning
rate (~0.01–0.1) with moderate depth (5–10) balances
bias/variance effectively on tabular data." },
{ "token": "bump", "dim": "colsample_bytree", "mode": 0.6,
"ci": [0.4, 0.8], "weight": 0.8, "provenance": "llm_domain",
"note": "Feature subsampling around 0.5–0.7 typically reduces
overfitting without losing too much signal." },
{ "token": "bump", "dim": "reg_lambda", "mode": 0.46,
"ci": [0.099, 10.1], "weight": 0.6, "provenance": "llm_domain",
"note": "Moderate L2 regularization helps stability on
imbalanced datasets; very high values underfit." },
{ "token": "smoothness", "level": "medium" },
{ "token": "noise", "level": "medium", "repeats_recommended": false },
{ "token": "modality", "expectation": "few" } ],
"trust": { "overall_confidence": "high",
"notes": "Based on extensive empirical literature regarding
XGBoost hyperparameter tuning on tabular classification tasks." } }
Task cards and elicitation protocol
Card structure. Every source receives the same three-part card: a domain description, the typed search space, and the optimization description (objective, direction, budget, known constraints), preceded by a fixed instruction to express the belief only in the closed PSL vocabulary. Cards are de-identified, they do not name the public benchmark or dataset (the domain is described only by model class and task type), and they carry no hyperparameter guidance or expectations about the response surface. The elicitor must place the prior from its own knowledge. The card for the XGBoost defect-prediction task (parameter value lists abridged):
Domain. System under study: hyperparameter optimization of an XGBoost gradient-boosted-tree classifier on a software-defect-prediction task (about 2,000 records, about 20 numeric features, 2 strongly imbalanced classes). The four knobs are the learning rate eta (log), max_depth (log), colsample_bytree (per-tree feature-subsampling fraction), and the L2 leaf regularization reg_lambda (log). General gradient-boosted-tree hyperparameter-tuning knowledge applies.
Search space. colsample_bytree: ordinal, 10 linear values in ; eta: ordinal, 10 log-spaced values in ; max_depth: ordinal, ; reg_lambda: ordinal, 10 log-spaced values in .
Optimization description. Objective: mean validation accuracy (fraction correct) on a held-out split, averaged over several random seeds, higher is better. Direction: maximize. Evaluation budget: about 8 to 32 evaluations (low-budget regime). Known constraints or prior observations: none.
Human expertise enters as text on the same card. For the cortical mixed (LLM+human) prior, the card’s third section embeds the expert’s statement verbatim:
A domain expert who has run similar experiments on this preparation states: “Usually the top row is the most effective, and the two outer columns of the array are the most effective.”
The provenance tags of the resulting beliefs (user_stated vs. llm_domain) record which parts came from the human and set the compiled trust through Eq. (1); there is no separate numeric merge step.
For the human-only prior, the elicitor is given access to a summary of data from 17 subjects that are not contained in the dataset (as in [5]), transcribing expert maps directly.
Adversarial priors. The matched adversarial condition replaces the compiled belief with a programmatically inverted one, which was selected to be a joint_region bump placed at the task’s empirical anti-optimum (the worst grid configuration), same weight and provenance as the faithful prior. So the adversarial arm consumes exactly the same interface with exactly misplaced content. Baselines that consume text instead of a compiled prior receive a hostile task description through their native interface.
Elicitation protocol. The priors are elicited from a locally served open-weights model (Qwen3.5-122B) at temperature 0, one prior per task (synthetic panel: three draws per card at temperature 0.6; elicitor comparison: Appendix I), offline, before any optimization. The first attempt uses schema-guided decoding against the PSL grammar; if validation or compilation fails, the exact validator error is fed back for up to four corrective turns (typical cost: one to two calls, 15,000 tokens total per HPO task).
Experimental details and cost
Surrogate and acquisition. All internal arms (HapBO, its ablation variants, vanilla BO) share a Matérn-5/2 GP (Appendix § 18 also tests RBF and Matérn-1/2 kernels). The acquisition is EI with , except in Appendix § 18, which tests EI/PI/UCB; already-queried candidates are excluded from selection in the noiseless benchmarks. The single random point per seed is identical across HAPBO, vanilla, and random search. Seeds 1001–1010. Prior warm-starting is disabled everywhere except the injection-site ablation arm that studies it and -BO, whose first initial point is the prior mode as in Hvarfner et al. [25] (App. § 15).
Estimands and normalizations. HPOBench: simple regret against the tabulated grid optimum; figure curves normalize per task by the grid span (optimum minus mean grid score); the adversarial figure normalizes by vanilla’s mean first-query regret. Cortical: ground truth response of the electrode currently considered best divided by the optimum one, averaged over ten seeds; comparisons use all 40 rat–muscle maps, with mean SEM over the maps (technical replicates); budget 12 queries. Synthetic: per-function regret normalized by vanilla’s mean first-query regret and aligned by fraction of each function’s budget (Hartmann-6: 40, Styblinski–Tang-7: 50, Levy-10: 50, Ackley-15: 150 queries). SOLAR: best feasible efficiency, zero until the first feasible design.
Synthetic prior fidelity. For each analytic function the elicitor receives a de-named card carrying a perturbed closed-form model of the objective whose correlation with the truth is controlled (, 0.5, 0.1), and transcribes it into PSL under a maximum-fidelity instruction set. Prior quality is therefore an experimental variable; the compiled trust of Eq. (1) is used unchanged.
Detail of prior elicitation quality in the synthetic experiment. Table 3 shows, for each function and card fidelity, how closely the compiled prior follows the card’s model and the true objective , and the value of at the global optima of each (, ). At every fidelity the elicited prior has a peak aligned with the card model that elicited it. The card model’s optimizer does not coincide with the true one. The global agreement stays well below one with the Qwen3.5-122B elicitor used throughout; see Appendix § 19 for how stronger elicitors transcribe the card model with more accuracy and detail, and improve global agreement.
| Function | |||||
| , all functions | 0.90 | 0.38 | 0.41 | 0.50 | 0.90 |
| Hartmann-6 | 0.91 | 0.41 | 0.39 | 0.08 | 0.72 |
| Styblinski–Tang-7 | 0.90 | 0.15 | 0.12 | 0.00 | 0.86 |
| Levy-10 | 0.90 | 0.41 | 0.51 | 0.91 | 1.00 |
| Ackley-15 | 0.90 | 0.55 | 0.63 | 1.00 | 1.00 |
| , all functions | 0.50 | 0.34 | 0.30 | 0.62 | 0.98 |
| Hartmann-6 | 0.49 | 0.56 | 0.45 | 0.79 | 0.93 |
| Styblinski–Tang-7 | 0.51 | 0.21 | −0.05 | 0.01 | 0.99 |
| Levy-10 | 0.51 | 0.35 | 0.40 | 0.75 | 1.00 |
| Ackley-15 | 0.50 | 0.24 | 0.39 | 0.93 | 1.00 |
| , all functions | 0.11 | 0.15 | 0.08 | 0.14 | 1.00 |
| Hartmann-6 | 0.12 | 0.03 | 0.08 | 0.14 | 1.00 |
| Styblinski–Tang-7 | 0.11 | 0.20 | −0.15 | 0.13 | 1.00 |
| Levy-10 | 0.11 | 0.33 | 0.23 | 0.11 | 1.00 |
| Ackley-15 | 0.11 | 0.06 | 0.16 | 0.20 | 1.00 |
Table 3. Prior elicitation quality in the synthetic experiment. : the closed-form model printed on the card; : the true objective; : the compiled PSL prior (Qwen3.5-122B; mean over the three elicitation draws). : Pearson correlation over 20,000 uniform points of the search box. , : value of at the global optimizer of and of .
SOLAR constraints. HAPBO fits one GP per constraint and multiplies the (shaped) acquisition by the product of per-constraint feasibility probabilities ; simulator crashes are modeled by an additional GP whose probability-of-success factor multiplies the score, and crashed observations are excluded from the objective and constraint models. The incumbent is the best feasible, non-crashed observation.
Cost accounting. Table 4 reports per-query wall-clock and token costs on HPOBench. Wall-clock is the median budget-16 run divided by budget on the archived hardware; the LLAMBO and BORA token figures are measured on a 10-run subset rerun and LGBO’s from run telemetry; hardware differs across some archived external runs, so these are order-of-magnitude costs, not a controlled benchmark. The locally served model has no marginal provider fee; its compute and energy costs are nonzero.
| Method | wall-clock/query (s) | setup tokens (once) | tokens/query |
| HapBO | 0.20 | 15,001 | 0 |
| ColaBO | 104.30 | 1,770 | 0 |
| LLAMBO | 28.59 | – | 79,650 |
| LGBO | 12.44 | – | 566 |
| BORA | 4.08 | – | 266 |
| vanilla BO | 0.20 | – | – |
| random | 0.20 | – | – |
Table 4. Per-query cost on HPOBench (Qwen3.5-122B served locally), averaged across the five families. Setup tokens is the one-off prior elicitation, compiled once per task; tokens/query is the in-loop LLM cost, zero when the model is consulted only offline.
Fixed-belief ablations: trajectories and endpoints
This appendix reports the two ablations summarized in § 5.4. Both hold the belief fixed, so that differences can be attributed to how a prior is used rather than to what it says: every arm consumes one compiled prior per task and shares the same GP surrogate, base EI, and ten seeds; only the stated axis differs.
Injection site (Figure 9). With the representation fixed, the prior enters through the bounded acquisition multiplier of HAPBO, the floor-free -BO multiplier, the surrogate mean (prior-to-mean), or the initial queries (prior-warm-start), bracketed by a prior-only greedy arm and no-prior controls. The -BO arm follows Hvarfner et al. [25] as published ( with the number of BO iterations). Its first initial point is the mode of the prior.
Representation (Figure 6 and Figure 10). With the site fixed at the bounded multiplier, only the language through which the belief becomes varies: PSL against point-only, Gaussian-only, and direct-LLM elicitation. PSL’s richness pays off on trustworthy high-dimensional synthetic priors, in particular. Direct-LLM elicitation is reported on the finite benchmarks only, since a fixed coarse anchor grid degenerates in 7–15 dimensions. We evaluate the five arms as in the main evaluation (grand mean, Figure 6; average task-wise rank, Figure 10A; per-function/family, Figure 10B–D) on the synthetic suite at and on the 40 HPOBench tasks with the good and adversarial prior. Full PSL wins on the synthetic suite. On HPOBench with the good prior, full PSL is aligned with point-only and above the other options. With the inverted-map prior, full PSL performs better than point-only, and similarly to the other options.
Figure 10. Prior representation ablation. The injection site is fixed at the bounded multiplier; only how the belief becomes varies: full PSL (ours), point-only, Gaussian-only, direct-LLM elicitation, and vanilla BO (no prior). Grand means in Figure 6. A: average task-wise rank (1 = best) in the same three conditions (synthetic suite at , aligned by fraction of budget; HPOBench with the good and with the adversarial prior). B–D: per synthetic function (B), and per HPOBench model family with the good (C) and with the adversarial prior (D). Direct-LLM is absent from the synthetic suite, because its fixed anchor set degenerates in 7–15D. Ten seeds per task (synthetic prior arms: three documents ten seeds per function); mean SEM across tasks; lower is better.
Trust weight (Figure 11). How important is the confidence stated by the LLM? It enters HAPBO through the trust weight (equation (1)). We keep each task’s compiled and replace the compiled by a fixed on the 40 HPOBench tasks (ten seeds, 16 queries). With the good prior (task card), every gives about 55% lower final regret than vanilla EI (grid-span normalized regret at query 16: vanilla 0.133, 0.063), and is indistinguishable from the compiled . With the misleading prior, regret at query 16 rises with . The trust weight thus bounds the loss from a misleading prior. In different optimization problems, regret might decrease with increasing for good priors ([20])
Note on optimization grids in continuous domains (Figure 12). On the finite benchmarks the acquisition is maximized over the whole candidate grid, so the multiplier scores every option, the prior’s maximizers included. A continuous domain has no grid: each query maximizes the acquisition from a finite set of candidate points (2048 scrambled Sobol points, the best eight refined by L-BFGS-B). In 7–15 dimensions such a set almost never falls inside a narrow prior mode, so on the synthetic benchmarks (Figure 4 and Figure 10) every prior arm also receives the maximizers of its own as candidates. Including the prior’s peaks among the candidate points is the obvious choice (since they’re likely candidates for final acquisition function argmax), but to prove that it does not constitute an additional mechanism by which the prior is transmitted, we show that vanilla BO given the same candidates does not improve (Figure 12).
Figure 12. Prior-peak candidates alone do not transmit the prior (synthetic panel). Normalized simple regret as in Figure 4 under trustworthy (A), medium (B), and poor (C) priors. HAPBO against vanilla BO given the same prior-maximizer candidates with an unweighted acquisition, and vanilla BO.
Prior sources: compiled fields, example maps, and trajectories
This appendix reports the prior-source comparison on the NeuroMOSAICS Cortical panel, together with the objects the trajectories consume. The three compiled belief functions (Figure 13A–C) show how differently the same interface renders the three knowledge sources: the LLM-only prior encodes a smooth somatotopic gradient, the LLM+human prior transcribes the expert’s rostral-row and edge-column preferences, and the human-only prior reconciles data from previous subjects (rats tested under the same conditions but not included in the dataset). The example ground-truth response maps (Figure 13D–F) illustrate the spatial structure and cross-subject variability of the underlying objective; they are shown for context and are not paired with the priors above them. Figure 13G–I shows each prior mode against its matched vanilla BO trajectory.
HPOBench: per-family trajectories
Figure 14 expands the compact comparison of the main text (Figure 2) into its five per-family panels. Matched HAPBO consumes the compiled HPO prior from the same random start as its controls, and every method shows its own curve.
Figure 14. HPOBench, per model family. Normalized simple regret for the five families (top), average task-wise rank and grand mean over 40 tasks (bottom), using matched HAPBO and every method’s native budget-16 curve. Regret is normalized per task by the full-grid span (optimum minus mean grid score); bands are ±SEM across tasks after averaging ten seeds within each task. Every method completes all 400 task–seed runs. The family panels show the aggregate gain of the main text is not driven by one model class.
Figure 15 expands the compact adversarial comparison of the main text (Figure 5) into its five per-family panels. Each method receives the same hostile task description through its native interface, and matched HAPBO consumes the compiled adversarial prior (a bump at each task’s empirical anti-optimum) from the same random start as its controls.
Figure 15. Adversarial HPOBench, per model family. Normalized simple regret for the five families (top), average task-wise rank and grand mean over 40 tasks (bottom), using matched HAPBO and the same hostile description through each method’s native interface. The rank panel uses the task grid complete for every method; regret is normalized by vanilla’s mean first-query regret; bands are ±SEM across tasks after averaging ten seeds. BORA completes 400/400, LLAMBO 400/400, and ColaBO 400/400 task–seed endpoints.
HapBO across BO variants
The bounded multiplier attaches to the acquisition landscape (it fundamentally re-sorts its ranking) and nothing else, so it should be indifferent to the BO machinery around it: which acquisition is shaped, which kernel the GP surrogate uses, and whether queries are issued one at a time or in batches. This appendix verifies all three on matched pairs; Figure 16 collects the evidence (panels A–C: acquisition functions; D–E: batch selection; F–H: GP kernel).
Acquisition functions (Figure 16A–C). The bounded multiplier applies to any nonnegative base acquisition. Panels A–C repeat the matched prior-on/prior-off contrast under EI, PI, and UCB on HPOBench and the cortical tasks. Every arm shares the GP surrogate, candidate set, and matched seeded initialization (); only the prior differs. Raw UCB is an absolute score that can fall below the incumbent, so its – arm shapes the clipped improvement over the incumbent , keeping its behavior consistent with the nonnegative EI and PI improvements. Because the paired arms share the same seeded first query, their query-1 values coincide exactly (normalized regret 0.099 on HPOBench, normalized performance 0.433 on cortical), so every sub-
sequent divergence is attributable to shaping alone. Across all three acquisitions the shaped arm improves low-budget sample efficiency: on HPOBench the final normalized regret has a 3–10 reduction from vanilla to HAPBO; on cortical the advantage is largest early, but seen throughout.
Batch selection (Figure 16D–E). The multiplier is likewise indifferent to whether queries are issued one at a time or in parallel batches (an often likely setting when several electrodes or configurations are evaluated together). We keep the matched EI contrast of panels A–C and replace sequential selection with a Kriging-Believer batch of size : each round takes the shaped-EI argmax, fantasizes its outcome as the posterior mean, refits, and repeats until the batch is full, then evaluates all three points. The first member of a batch sees exactly the sequential arm’s data, so divergence appears only inside a batch, where the surrogate is frozen across picks. The shaped arm’s advantage is within the early queries, in both settings.
GP kernel (Figure 16F–H). Every arm above uses a Matérn-5/2 kernel. Panels F–H change only the GP covariance: the Matérn-5/2 default (F, which reproduces the EI panels of A), the infinitely smooth squared-exponential kernel (RBF, G), and the rough Matérn-1/2 (exponential) kernel (H).
On HPOBench HAPBO has a , , and reduction of regret. On cortical all HAPBO arms display an early lead.
Elicitation comparison: elicitor quality vs. downstream performance
Does a stronger elicitor produce a more useful prior? We compare five elicitors of increasing capability, the open-weights Qwen3.5-122B used throughout the paper and four commercial models (Claude Haiku 5.5, Sonnet 5.5, Opus 5.5 and Fable 5.1), on the synthetic panel with the accurate cards (). Each elicitor transcribes the same card into PSL by two routes: a single API call and an agentic coding session in which the model can compile and test its own document before submitting it (with a token cap of , where is the number of dimensions). We measure how faithfully the compiled prior reproduces the card model (precision and agreement), what the elicitation costs, which PSL tokens the elicitor uses, and the downstream regret of HAPBO on the true function, with every other setting held fixed (Figure 17, Table 5).
| Elicitor | final regret |
| Qwen3.5 | 0.048 ± 0.015 |
| Haiku 5.5 | 0.048 ± 0.016 |
| Sonnet 5.5 | 0.044 ± 0.012 |
| Opus 5.5 | 0.045 ± 0.012 |
| Fable 5.1 | 0.044 ± 0.014 |
| vanilla BO | 0.228 ± 0.160 |
Table 5. Elicitor comparison: downstream regret Final normalized simple regret of HAPBO on the true function at the end of the budget (lower is better), mean ± SEM of data in Fig. 17. Bold marks the best value.
Figure 17. Downstream performance and elicitor quality. Synthetic panel, accurate cards (); five elicitors and two elicitation routes (solid: one API call; hatched: agentic session). A Two-dimensional slice of the card model (Styblinski–Tang-7, vs. , through its optimum, white cross) and the compiled of each elicitor’s single-call document on the same slice. B Precision (rank correlation between and the card model) and C agreement (overlap of the top 10% of points); bars: mean over the four functions, dots: functions. D Tokens and E list-price cost per elicitation (the local model has no marginal fee). F Number of distinct PSL token types used and G share of each token type in the single-call documents. H Normalized simple regret of HapBO on the true function vs. fraction of budget, per elicitor (average of single-call and agentic curves, SEM); vanilla BO dashed.
Stronger elicitors transcribe the card model more faithfully (Figure 17A): the weaker models draw a coarse, mostly separable structure, while the frontier models reproduce the correct landscape of the card model, including precise depiction of favorable and unfavorable regions. Precision and agreement increase with the capability of the model, and the agentic route improves both at every level (B, C). These gains come at a price that grows with both factors: the agentic route requires far more tokens and a correspondingly higher cost than a single call, and more capable models cost more per elicitation (D, E). Moving up in model quality, elicitors tend to use a wider variety of PSL tokens (F) and change what the tokens encode (G): weaker models rely mostly on single-dimension bumps, whereas stronger ones spread the document over joint regions, periodic and trend components, and explicit avoid regions, stating also where the optimum is not. However, in this type of problem (H), elicitation with Qwen3.5 already captures the key features of the search space robustly enough that all elicitors yield nearly overlapping regret curves, well below vanilla BO: benefits in performance, however present, are limited. In other optimization problems, for instance when the true optimum lies at the second-, third- or lower-ranked candidate of the available model, further precision in the representation may have a higher impact than in our synthetic problem suite.
Targeted choices in baseline implementations
Prior-guided and in-loop baselines admit more than one faithful implementation, and the choice can move a curve as much as a change of method. Methods like LLAMBO and LGBO are designed to be modular and modified in future work. This is the case also for the current HAPBO, as discussed above, since the choice of and metatokens for the surrogate is subject to welcome variations from future work. For the two baselines LLAMBO and LGBO where we experienced the most open ambiguity at the time of integration, we performed a direct comparison in a restricted HPOBench setting, holding the harness identical (same local model, task cards, GP where used, candidate set, initialization, seeds, budget). Then, we adopted and reported in the main text the variant that performed best, so every headline comparison is against the strongest competitor we could construct (Figure 18).
Figure 18. Baseline-implementation sensitivity. LGBO and LLAMBO, in two possible implementations, each shown with the matched HAPBO curve. Left, LGBO: the paper’s mean-shift equation (solid, used in the main text) vs. the public release’s confidence operator (dashed). Right, LLAMBO: random initialization, mode 1 (dashed) vs. zero-shot warm-start, mode 2 (solid, main text). Top: standard prior, grid-span-normalized log regret. Bottom: adversarial prior, vanilla-query-1-normalized regret (log). Bands are ±SEM across tasks after averaging ten seeds.
LLAMBO: initialization mode. The original paper presents random initialization (mode 1) and zero-shot warm-starting with five LLM-proposed points (mode 2). In our experiments, Mode 2 is markedly more sample-efficient through the low-budget regime; we therefore report mode 2.
LGBO: confidence operator. No public code reproduces the LGBO (as defined in the original paper) on these benchmarks, so we reconstruct it, and two readings exist: the paper’s stated construction lifts the GP mean by exactly posterior standard deviations (nonnegative, capped), whereas the public release maps confidence through , which is negative for . This suppresses the very region the model prefers, and is unbounded as . We port both into the identical harness; the paper-equation version is stronger on the standard prior and comparable on the adversarial prior; we therefore report our paper-equation version.
Leak control
The objective of this paper is to determine, when LLM knowledge about a given problem exists, whether this can be integrated, by proper harnessing, in BO. However, there is a type of existing knowledge that is less desirable when testing any LLM-prior method on public benchmarks, which is contamination: a strong prior might reflect recall of the final benchmark answer rather than transferable expertise. We address this with two complementary controls: a direct probe of what the elicitor natively knows, and an additional objective whose optimum cannot have been memorized.
Does the elicitor natively know the optimum?
For every task we present the task card and ask the model, from prior knowledge alone, for its single best guess of the optimum’s location. We score the guess with the metric recovered % , the fraction of the head-room between the grid mean and the optimum that the blind guess captures. We read the reasoning trace and flag recall only when the model both names the underlying benchmark and states its optimum. For the analytic functions we probe the de-named perturbed-model cards at all three fidelities, score against the optimum of the true (hidden-shift) objective, and add a positive control that names the function outright.
No elicitor natively knows the optima from the cards alone (Table 6–Table 8). On HPOBench the blind guess recovers 48–68% of the head-room; on the cortical grid it lands below the map average (−12%); no trace on any real task shows recall. On the analytic functions the guess tracks the fidelity of the supplied model and the de-named cards are not recognized, with one exception: the elicitor recognizes the Ackley family at every fidelity yet defers to the perturbed card rather than to a memorized optimum, so its guess still degrades with . The named-function control recovers 100% in every case, confirming the knowledge exists but surfaces only when explicitly requested. Across the four elicitors of Table 8 the picture holds on all five HPO families: a stronger model guesses marginally better blind (grand-mean recovered 60% to 76%) but none of the 400 elicitations recalls an optimum.
| Problem (family) | recovered % | raw % of best | recall evidence | partial recall evidence |
| HPOBench lr | 60±11 | 98±1 | no recall evidence (0/8) | 0/8 |
| HPOBench rf | 48±9 | 94±1 | no recall evidence (0/8) | 0/8 |
| HPOBench xgb | 66±9 | 99±0 | no recall evidence (0/8) | 0/8 |
| HPOBench nn | 68±9 | 96±2 | no recall evidence (0/8) | 0/8 |
| HPOBench svm | 55±16 | 89±5 | no recall evidence (0/8) | 0/8 |
| NeuroMOSAICS cortical | −12±4 | 37±6 | no recall evidence (0/16) | 0/16 |
| SOLAR | 98 | 98 | no recall evidence (0/1) | 0/1 |
Table 6. Probing native recall of benchmark optima. Recovered and raw % of the blind best-guess optimum per benchmark (local elicitor, greedy decoding), with reasoning-trace recall evidence. HPO values are means over the eight tasks per family (±SEM); cortical over the sixteen rat–muscle maps; SOLAR is a single problem scored against the best feasible pool design.
| Elicitor | lr | rf | xgb | nn | svm | grand mean | recall evidence | partial recall evidence |
| Qwen3.5-122B | no recall evidence (0/40) | 0/40 | ||||||
| Claude Opus | no recall evidence (0/120) | |||||||
| Claude Sonnet | no recall evidence (0/120) | |||||||
| Claude Haiku | no recall evidence (0/120) | 0/120 |
Table 8. Probing native recall of benchmark optima across models. Recovered % of the blind best-guess per HPOBench family (Claude elicitors at temperature 1.0, three draws per task; local elicitor greedy). No elicitation on any family recalls an optimum.
An un-memorizable CNN benchmark
We define a small bespoke convolutional encoder (Figure 19A) and compute its hyperparameter-response surface ourselves: a 4-D grid of configurations over learning rate, base channel width, dropout, and weight decay, each network trained from scratch and scored by seed-averaged validation accuracy on a fixed small image subsample. This exact architecture-and-grid response map exists in no public corpus, so the optimum is not recoverable from memory; a competent elicitor can still place a reasonable prior from general machine-learning intuition, which is the ability we mean to isolate. On this objective HAPBO reaches the lowest simple regret, ahead of LGBO, BORA, whereas prior-free controls, LLAMBO and ColaBO clearly trail.
| Function | card | recovered % | dist. to true | dist. to textbook | recall evidence |
| Hartmann-6 | 86 | 0.47 | 0.49 | follows provided model, no family recognition | |
| 7 | 0.23 | 0.27 | follows provided model, no family recognition | ||
| -3 | 0.31 | 0.37 | follows provided model, no family recognition | ||
| named control | 100 | 0.00 | 0.00 | names Hartmann, states | |
| Styblinski–Tang-7 | 98 | 0.03 | 0.11 | follows provided model, no family recognition | |
| 41 | 0.44 | 0.50 | follows provided model, no family recognition | ||
| -370 | 0.53 | 0.54 | follows provided model, no family recognition | ||
| named control | 100 | 0.00 | 0.00 | names Styblinski–Tang, states | |
| Levy-10 | 99 | 0.03 | 0.20 | follows provided model, no family recognition | |
| 73 | 0.17 | 0.30 | follows provided model, no family recognition | ||
| -118 | 0.41 | 0.45 | follows provided model, no family recognition | ||
| named control | 100 | 0.00 | 0.00 | names Levy, states | |
| Ackley-15 | 97 | 0.00 | 0.23 | recognizes Ackley family, guess follows the provided (shifted) model | |
| 6 | 0.20 | 0.31 | recognizes Ackley family, guess follows the provided (shifted) model | ||
| -1 | 0.51 | 0.50 | recognizes Ackley family, guess follows the provided (shifted) model | ||
| named control | 100 | 0.00 | 0.00 | names Ackley, states |
Table 7. Probing native recall of benchmark optima, analytic functions. De-named cards supply a perturbed model at three fidelities ; the guess is scored on the true (hidden-shift) objective (recovered % and normalized RMS distance to the true and to the textbook ; 0 = exact). The named control names the function outright.
Further baselines
Two recent LLM-driven methods keep the model in the loop in ways orthogonal to the prior-weighting baselines of the main text; we run both in our harness on the HPOBench suite (five families, ten seeds), under the faithful and the adversarial prior. CAKE (Suwandi et al., 2025) uses the LLM as a crossover/mutation operator that evolves the GP kernel expression at each query; it consumes no compiled belief, so its curve is identical in both prior conditions. CALIPER (Chen et al., [9]) elicits per-role priors once and admits them through a discrete counterfactual evidence gate; we adapt it to single-objective HPO (three objective personas; EI in place of the hypervolume acquisition) and, absent public code, reimplement it from the paper. HAPBO attains the lowest grand-mean regret on the good prior, slightly ahead of CALIPER, which however drops largely with an adversarial prior (Figure 20) whereas the bounded multiplier keeps HAPBO robust; CAKE always performs in line with vanilla BO.
Two expert-prior HPO methods take the belief as one value and one confidence level per hyperparameter, which we elicit with the same LLM from the same cards as HAPBO. PriorBand [36] and PriMO [4] (official NePS 0.17.0 implementations, used with original papers’ settings) are multi-fidelity and the only methods in this paper with access to cheap low-fidelity evaluations: they also evaluate cheaper versions of each task (fewer iterations or trees, a data subsample), each charged of a full evaluation within the same budget of 16, and are scored by their best full-fidelity evaluation. Both are less performant than HAPBO under the same good LLM-elicited prior and substantially on par under an adversarial prior (Figure 21).
Two further methods use the LLM’s knowledge as something other than a weighted prior. HypBO [13] searches boxes around optimum guesses that the LLM states with a confidence, elicited as in BORA [14]; we run its public code unmodified, with one initial point (its authors use five). LLM-EI [38] fits a preference GP to the LLM’s answers to a pairwise survey of each card (4–8k questions per task) and restricts EI to the top 15% of the resulting utility. Both trail HAPBO under both faithful and adversarial priors (Figure 22).
Theory
This appendix contains the formal analysis behind HAPBO: the expressivity and organization of the Prior Specification Language (PSL), the acquisition-level guarantee of the bounded multiplier, and two separate convergence analyses: 1) noiseless fixed-kernel EI and 2) noisy finite-domain GP-UCB. The two convergence regimes are analyzed independently.
Notation. The search space is a typed product: each coordinate is a compact interval after the schema’s scale transform (e.g. log), a finite ordinal set, or a finite categorical set; is compact, and a finite-candidate implementation acts on a grid . The objective is , maximized unless stated otherwise; (nonempty on compact for continuous , always on finite ).
Three regret quantities. We distinguish:
the instantaneous regret of the point queried at round ;
the best-sampled latent simple regret , the regret of the best point queried within the budget, evaluated on the latent objective (under noise, is not observed exactly, so is not directly observable);
the recommendation regret of a point recommended from a noisy history.
Each result below controls exactly one of these: the EI theorem controls best-sampled simple regret in a noiseless setting (where it is observable); the UCB theorem controls cumulative instantaneous regret , and through the best-sampled latent simple regret.
PSL compiler: expressivity and vocabulary structure
The results of this subsection characterize the compiler independently of any optimizer.
Typed domain and compiler contract
A PSL document (typed schema; primitives: anchors, bumps, joint regions, trends, thresholds, periodic, ridges, conditionals, avoid regions, plateaus, categorical weights; trust metadata) compiles as , where is the prior belief function, is the compiled trust, and is the feasible set. The semantics clauses used by the theory:
(S1) Finite anchors. On finite , an anchor at with value sets the compiled prior at to exactly , with no renormalization or clipping; candidates not covered by any anchor take the baseline value.
(S2) Continuous anchors. On continuous coordinates, for any finite anchor set forming a -cover, the compiler realizes a continuous partition of unity subordinate to that cover: with , , and only when .
(S3–S6) Macro semantics. The baseline compiles to an affine function; a trend to a piecewise-linear saturating ramp; a periodic token to a sinusoidal envelope mapped into ; a bump or avoid to a signed Gaussian with weight ; a joint region to the corresponding multi-dimensional product.
(S7) Feasibility. No belief primitive removes a candidate. Hard limits are part of defining the search space: they enter through the schema’s typed per-dimension ranges (and, for a residual joint limit, a caller-supplied feasibility filter), fixing before any belief is compiled. The soft avoid primitive subtracts from the score, which is then clamped to ; it can lower a region’s plausibility but never removes it from .
Theorem 1 (PSL expressivity). Let the compiler satisfy (S1)–(S2).
(a) Finite exactness. For every on a finite candidate set there is an anchors-only PSL document with for all $i.
(b) Continuous density. For every continuous (categorical dimensions as finite strata) and every there is an anchors-only document with .
(c) Quantitative rate. If the continuous block of a stratum satisfies and is -Lipschitz there, accuracy is achieved with anchors per stratum; equivalently, the sup-error in the anchor budget is .
Proof. (a) One anchor per candidate with value ; clause (S1) returns the table exactly, and each needs no clipping.
(b) Fix a continuous stratum (compact). Uniform continuity gives with . Take a finite -cover of and the partition of unity of clause (S2); set anchor values . Then
since forces and the sum to one. Repeat per categorical stratum, gating anchor sets by the categorical assignment. The interpolant is a convex combination of values in , hence a valid prior field.
(c) Partition the bounding box into axis-aligned cells of side with ; each cell has diameter , so anchors placed one per cell meeting form a -cover of size at most the stated product. Running the argument of (b) with this cover bounds the sup-error by ; inverting the cell count gives the form.
Proposition 1 (vocabulary organization). Let be a product probability space and . Define recursively and . Then
and the order-2 truncation satisfies
PSL’s layers mirror this decomposition: the baseline is the order-0 component, the marginal primitives the order-1 components, the interaction primitives the order-2 components, and anchors the residual complete basis.
Proof. The recursion gives for every strict subset ; the product structure of then yields orthogonality across distinct index sets, and the recursive sum telescopes to . For the truncation, and orthogonality gives the Pythagorean identity.
Limitation. Proposition 1 explains the vocabulary’s organization; it does not show that low-order structure dominates any particular elicited prior.
Remark 1 (macros cannot reduce expressivity). Enlarging the vocabulary with macro tokens cannot reduce what is expressible: any document written in a sub-vocabulary is a document of the full vocabulary, so exactness and density (Theorem 1) are retained. The macro tokens exist for economy of expression, not for additional expressive power.
Common bounded-shaping results
Definition 1 (raw bounded shaping). Let be a base acquisition at round . The shaped acquisition is
with and . The default is a run-fixed field and scalar static trust compiled before the run; time-varying trust is introduced only for the decay results below. The implementation clamps , so . We apply the multiplier to the raw acquisition, unlike Eq. 10 of Guay-Hottin et al., [20], which min–max normalizes the acquisition before multiplying. These generally define different selection rules: raw shaping preserves positive acquisition support, while min–max normalization can assign zero to candidates with positive raw acquisition. Nevertheless, the bounded-influence and EI regret guarantees also hold for the normalized formulation (Corollary 1).
Proposition 2 (bounded influence on acquisition selection). For as in Eq. (8):
for all and . Consequently, if on the feasible set and , then
Proof. with gives the bounds. For the selection statement, for every , using , optimality of , and .
Regime I: noiseless fixed-kernel expected improvement
This subsection follows the minimization convention of the classical EI analysis (Bull [6]): is minimized (equivalently is maximized); the statements are orientation-free. denotes the nonnegative best-sampled simple regret after evaluations, observable here because observations are noiseless.
Assumption 1 (EI regime). *(i) Observations are deterministic and noiseless; (ii) the domain is compact; (iii) the kernel, lengthscale, and scale are fixed during the run, with smoothness ; (iv) the objective lies in the RKHS ball ; (v) the acquisition is raw EI, shaped as in Definition 1 with a run-fixed multiplier , continuous in so the shaped argmax exists; (vi) no sequential GP hyperparameter refitting. In addition, the two selection-dependent ingredients of the classical machinery hold for any selection sequence: the covering argument of Bull [6] (Lemma 7) bounds the posterior standard deviation at all but at most rounds by , and the running-optimum telescoping bounds the incumbent decrement at no more than rounds; a pigeonhole over the window yields the index satisfying both bounds simultaneously. Finally, EI satisfies the two-sided sandwich (Bull [6], Lemma 8): with , , and ,
For a run-fixed multiplier define
using and ; note always.
Theorem 2 (shaped-EI rate preservation). Under Assumption 1, the best-sampled simple regret of shaped EI satisfies, for with past the initial design,
where denotes the Bull-style theoretical upper bound on the loss of unshaped EI—an upper bound, not the realized loss of a separately executed vanilla run. In particular : bounded shaping preserves the classical rate, with a constant determined by the multiplier at an optimizer.
Proof. Fix past the initial design and let be the pigeonhole index of Assumption 1. Let be a global minimizer maximizing over . By the lower branch of equation (9) at , with ,
By optimality of the shaped selection at step , , hence
By the upper branch of equation (9) at and the two pigeonhole bounds,
Chaining the three displays and using monotonicity for (in particular for , since ) gives equation (11).
Corollary 1 (Min–max-normalized acquisition shaping). Let , , and define and . For , consider the normalized selection rule
Then , as in Proposition 2. Moreover, for EI under Assumption 1, with the raw selection rule in (v) replaced by this normalized rule, the conclusion of Theorem 2 holds with the same constant .
Proof. By optimality of the normalized selection, for every ,
Taking and using gives
since .
For EI, let be a global optimizer maximizing over . The same optimality inequality yields
Since and ,
This is exactly the selection inequality used in the proof of Theorem 2; all subsequent steps are unchanged. When , both acquisition comparisons hold trivially, irrespective of the selected point.
Corollary 2 (scalar prior-quality constant). For scalar trust and run-fixed prior with miss at the optimizer,
Proof. and in equation 10.
Corollary 3 (correct at an optimizer). If at some global optimizer, then : the shaped run satisfies the same theoretical upper bound as unshaped EI, at any trust level, regardless of the belief elsewhere. This is a statement about the bound; it does not assert that the realized shaped trajectory dominates a realized vanilla-EI trajectory.
Proof. and give in equation 10.
Corollary 4 (elicitation–compilation error). Let be the source’s latent belief landscape with miss at a global optimizer, and suppose the elicitation-plus-compilation pipeline produces with . Then for scalar trust ,
The pipeline enters the constant through two scalars added together—the source’s miss at the optimizer and the sup-norm compilation error—and the penalty never exceeds the ceiling .
Proof. gives , and always; insert into equation 10 via .
Remark 2 (time-varying trust). If the trust schedule satisfies , the same argument applied at the pigeonhole iteration shows that the effective constant approaches one asymptotically for any prior. Decay is not required for convergence in this noiseless EI regime, the rate is preserved at any fixed .
Regime II: noisy finite-domain raw GP-UCB
Back to the maximization convention.
Assumption 2 (UCB regime). (i) Finite candidate set with ; kernel with , fixed during the run, noise model fixed; (ii) observations with noise variance in the GP update; (iii) the acquisition is raw GP-UCB, with nondecreasing—no incumbent-relative and no min–max-normalized variant; (iv) shaped selection with history-measurable ; (v) re-querying of candidates is allowed (no exclusion mask), so the optimum remains selectable at every step; (vi) , a normalization treated as part of the algorithm (satisfied when responses are nonnegative, or after subtracting a known lower bound; the multiplicative rule is not shift-invariant, so the shift is part of the method’s definition). Two standard facts are imported. Confidence event: there are standard choices of such that with probability at least ,
(for sampled from the GP, , Srinivas et al., 2010; for in the kernel’s RKHS with sub-Gaussian noise, the self-normalized construction of Chowdhury & Gopalan, [12], valid for any adapted selection sequence). Call the event in equation (12) . Information-gain bound: for any selection sequence, with nondecreasing and ,
where is the maximum information gain after rounds (Srinivas et al., 2009, Lemma 5.4).
Lemma 1 (finite-domain information gain). On finite with :
Proof. The information gain of any observations is with the kernel matrix of the queried points. is positive semidefinite of rank at most (all columns lie in the span of the candidate feature vectors), so it has at most nonzero eigenvalues with . Then by concavity of (Jensen), which is at most .
Define the per-step misplaced-trust penalty at the optimum
and .
Theorem 3 (noisy shaped GP-UCB regret). Under Assumption 2, on the event , the cumulative regret of shaped GP-UCB satisfies
The second term is the usual information-gain term of unshaped GP-UCB, inflated by at most ; the first, additive term is the cumulative price of misplaced trust and depends on the belief only through its value at the optimum: confident but wrong beliefs about regions that do not contain the optimum contribute nothing to the bound.
Proof. Fix and work on . Let attain the minimum in .
Step 1 (optimism through the multiplier). By equation (12), . By shaped optimality and ,
so , and , using and .
Step 2 (value at the query). By equation (12), .
Step 3 (per-step regret). Subtracting, . By equation (12) again, , so
Step 4 (sum). Sum over , bound , and apply Cauchy–Schwarz with equation (13):
Corollary 5 (correct at an optimizer). If for all at some optimizer, then , , and shaped GP-UCB satisfies the vanilla GP-UCB bound on —at any trust level, however wrong the belief is elsewhere.
Proof. collapses equation (14).
Corollary 6 (decaying trust). If , then and, on ,
the additional cumulative term is for any prior, however adversarial; cumulative regret is sublinear under the usual information-gain conditions (on finite , by Lemma 1); and the best-sampled latent simple regret vanishes, since .
Proof. ; ; and . The simple-regret statement is .
Proposition 3 (static misplaced trust can cost linear cumulative regret). For every , there is a two-point problem in the noisy regime of Assumption 2 and a valid run-fixed multiplier such that, on the confidence event , shaped GP-UCB has
Thus static shaping can incur linear cumulative regret even with positive observation noise.
Proof. Let , , and , where . Use the fixed kernel , a zero GP prior mean, and independent Gaussian observation noise of variance , with the same variance in the posterior update. Set and . These multipliers are realized by , , and . Use the shaped selection rule from the first round and allow re-querying.
The objective belongs to the kernel’s RKHS, with . Choose the nondecreasing RKHS confidence schedule from Assumption 2; on this fixed two-point domain it can be chosen with and , since . Work on its confidence event .
Write for the number of observations of before round . The independent-arm kernel gives
Whenever the shaped rule selects , optimality and confidence imply
Consequently,
Using monotonicity of and accounting for the last selection of (with if it is never selected), we obtain
Each query of incurs regret , whereas a query of incurs zero regret. Therefore
Three comments on Proposition 3. First, it establishes that linear cumulative regret can occur within the positive-noise regime of Theorem 3. The linear dependence on is therefore a real worst-case possibility, although the construction does not establish sharpness of the coefficient in the upper bound. Second, Corollaries 5 and 6 give sufficient conditions for recovering sublinear cumulative regret: correctness at an optimizer or decaying trust. They are not necessary conditions; for example, a positive multiplier that is constant across candidates leaves the vanilla GP-UCB selection rule unchanged. The default HAPBO configuration keeps trust static within a run and accepts the resulting worst-case trade-off, because deployment campaigns of 8–30 queries are judged by the best configuration found rather than by cumulative regret. Third, the proposition does not imply a failure of discovery. In this construction, must eventually be queried on : otherwise , while repeated observations of give , contradicting the shaped selection rule. Hence the best-sampled latent simple regret is zero after the first query of , despite linear cumulative regret. This does not establish a recommendation guarantee from noisy observations.
References
- [1]David Ackley. A connectionist machine for genetic hillclimbing. Springer science & business media, 2012.
- [2]Dhruv Agarwal, Manoj Ghuhan Arivazhagan, Rajarshi Das, Sandesh Swamy, Sopan Khosla, and Rashmi Gangadharaiah. Searching for optimal solutions with LLMs via Bayesian optimization. In Y. Yue, A. Garg, N. Peng, F. Sha, and R. Yu (eds.), International Conference on Learning Representations, volume 2025, pp. 67180–67201, 2025. URL https://proceedings.iclr.cc/paper_files/paper/2025/file/a79237d6c47feef24c1df723cb41f674-Paper-Conference.pdf.
- [3]Nicolau Andrés-Thió, Charles Audet, Miguel Diago, Aïmen E. Gheribi, Sébastien Le Digabel, Xavier Lebeuf, Mathieu Lemyre Garneau, and Christophe Tribes. Solar: a solar thermal power plant simulator for blackbox optimization benchmarking. Optimization and Engineering, 26(3): 1815–1861, 2025. doi: 10.1007/s11081-024-09952-x.
- [4]Soham Basu, Nastaran Alipour, Frank Hutter, and Danny Stoll. Multi-objective hyperparameter optimization with expert priors, 2025. URL https://arxiv.org/abs/2511.08371.
- [5]Marco Bonizzato, Rose Guay Hottin, Sandrine L. Côté, Elena Massai, Léo Choinière, Uzay Macar, Samuel Laferrière, Parikshat Sirpal, Stephan Quessy, Guillaume Lajoie, Marina Martinez, and Numa Dancause. Autonomous optimization of neuroprosthetic stimulation parameters that drive the motor cortex and spinal cord outputs in rats and monkeys. Cell Reports Medicine, 4(4): 101008, 2023. doi: 10.1016/j.xcrm.2023.101008.DOI
- [6]Adam D. Bull. Convergence rates of efficient global optimization algorithms. Journal of Machine Learning Research, 12(88):2879–2904, 2011. URL http://jmlr.org/papers/v12/bull11a.html.arxiv.org/abs/1101.3501
- [7]Alexander Capstick, Rahul G. Krishnan, and Payam Barnaghi. AutoElicit: Using large language models for expert prior elicitation in predictive modelling. In Aarti Singh, Maryam Fazel, Daniel Hsu, Simon Lacoste-Julien, Felix Berkenkamp, Tegan Maharaj, Kiri Wagstaff, and Jerry Zhu (eds.), Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pp. 6746–6777. PMLR, 2025. URL https://proceedings.mlr.press/v267/capstick25a.html.arxiv.org/abs/2411.17284
- [8]Chih-Yu Chang, Milad Azvar, Chinedum Okwudire, and Raed Al Kontar. LLINBO: Trustworthy LLM-in-the-loop Bayesian optimization, 2025. URL https://arxiv.org/abs/2505.14756.
- [9]Jiangyu Chen, Yi Ban, and Tianfan Fu. Evidence-gated LLM priors for multi-objective Bayesian optimization, 2026a. URL https://arxiv.org/abs/2606.01730.
- [10]Qi Chen, Fabio Ramos, Alán Aspuru-Guzik, and Florian Shkurti. Informing acquisition functions via foundation models for molecular discovery, 2025. URL https://arxiv.org/abs/2512.13935.
- [11]Zhuo Chen, Xinzhe Yuan, Jianshu Zhang, Jinzong Dong, Ruichen Zhou, Yingchun Niu, Tian-hang Zhou, Yu Yang Fredrik Liu, Yuqiang Li, Nanyang Ye, and Qinying Gu. LABO: LLM-accelerated Bayesian optimization through broad exploration and selective experimentation. In Proceedings of the 43rd International Conference on Machine Learning, volume 306 of Proceedings of Machine Learning Research, pp. 17979–18005. PMLR, 2026b. URL https://proceedings.mlr.press/v306/chen26fx.html.arxiv.org/abs/2605.22054
- [12]Sayak Ray Chowdhury and Aditya Gopalan. On kernelized multi-armed bandits. In Doina Precup and Yee Whye Teh (eds.), Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pp. 844–853. PMLR, 06–11 Aug 2017. URL https://proceedings.mlr.press/v70/chowdhury17a.html.
- [13]Abdoulatif Cissé, Xenophon Evangelopoulos, Sam Carruthers, Vladimir V. Gusev, and Andrew I. Cooper. HypBO: Accelerating black-box scientific experiments using experts’ hypotheses. In Kate Larson (ed.), Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI-24, pp. 3881–3889. International Joint Conferences on Artificial Intelligence Organization, 8 2024. doi: 10.24963/ijcai.2024/429. URL https://doi.org/10.24963/ijcai.2024/429. Main Track.
- [14]Abdoulatif Cissé, Xenophon Evangelopoulos, Vladimir V. Gusev, and Andrew I. Cooper. Language-based Bayesian optimization research assistant (BORA). In James Kwok (ed.), Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence, IJCAI-25, pp. 4967–4975. International Joint Conferences on Artificial Intelligence Organization, 8 2025. doi: 10.24963/ijcai.2025/553. URL https://doi.org/10.24963/ijcai.2025/553. Main Track.
- [15]Katharina Eggensperger, Philipp Müller, Neeratyoy Mallik, Matthias Feurer, René Sass, Aaron Klein, Noor Awad, Marius Lindauer, and Frank Hutter. HPOBench: A collection of reproducible multi-fidelity benchmark problems for HPO. In J. Vanschoren and S. Yeung (eds.), Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks, volume 1, 2021. URL https://datasets-benchmarks-proceedings.neurips.cc/paper_files/paper/2021/file/93db85ed909c1383ff95ccfa94cebd9-Paper-round2.pdf.arxiv.org/abs/2109.06716
- [16]David Eriksson, Michael Pearce, Jacob Gardner, Ryan D Turner, and Matthias Poloczek. Scalable global optimization via local bayesian optimization. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL https://proceedings.neurips.cc/paper_files/paper/2019/file/6c990b7aca7bc7058f5e98ea909e924b-Paper.pdf.arxiv.org/abs/1910.01739
- [17]Lukas Fehring, Marcel Wever, Maximilian Spliethover, Leona Hennig, Henning Wachsmuth, and Marius Lindauer. Dynamic priors in Bayesian optimization for hyperparameter optimization, 2025. URL https://arxiv.org/abs/2511.02570.
- [18]Peter I. Frazier. A tutorial on Bayesian optimization, 2018. URL https://arxiv.org/abs/1807.02811.
- [19]David Ginsbourger, Rodolphe Le Riche, and Laurent Carraro. Kriging is well-suited to parallelize optimization. In Computational intelligence in expensive optimization problems, pp. 131–162. Springer, 2010.
- [20]Rose Guay-Hottin, Lison Kardassevitch, Hugo Pham, Guillaume Lajoie, and Marco Bonizzato. Robust prior-biased acquisition function for human-in-the-loop Bayesian optimization. Knowledge-Based Systems, 311:113039, 2025. ISSN 0950-7051. doi: 10.1016/j.knosys.2025.113039. URL https://www.sciencedirect.com/science/article/pii/S0950705125000863.DOI
- [21]Rushi Gupta, Jason Hartford, and Bang Liu. LLMs for Bayesian optimization in scientific domains: Are we there yet? In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng (eds.), Findings of the Association for Computational Linguistics: EMNLP 2025, pp. 15482–15510, Suzhou, China, November 2025. Association for Computational Linguistics. doi: 10.18653/v1/2025.findings-emnlp.838. URL https://aclanthology.org/2025.findings-emnlp.838/.
- [22]James K. Hartman. Some experiments in global optimization. Naval Research Logistics Quarterly, 20(3):569–576, 1973. doi: https://doi.org/10.1002/nav.3800200316. URL https://onlinelibrary.wiley.com/doi/abs/10.1002/nav.3800200316.
- [23]Leona Hennig, Jasmin Brandt, Lukas Fehring, Barbara Hammer, Marius Lindauer, and Marcel Wever. Provably reduced sample cost in prior-guided hyperparameter optimization, 2026. URL https://arxiv.org/abs/2606.04866.
- [24]Yongchao Huang. LLM-prior: A framework for knowledge-driven prior elicitation and aggregation, 2025. URL https://arxiv.org/abs/2508.03766.
- [25]Carl Hvarfner, Danny Stoll, Artur Souza, Marius Lindauer, Frank Hutter, and Luigi Nardi. πBO: Augmenting acquisition functions with user beliefs for Bayesian optimization. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=MMAeCXIa89.arxiv.org/abs/2204.11051
- [26]Carl Hvarfner, Erik Orm Hellsten, and Luigi Nardi. Vanilla Bayesian optimization performs great in high dimensions. In Forty-first International Conference on Machine Learning, 2024a. URL https://openreview.net/forum?id=OfT8MgIqHT.arxiv.org/abs/2402.02229
- [27]Carl Hvarfner, Frank Hutter, and Luigi Nardi. A general framework for user-guided Bayesian optimization. In B. Kim, Y. Yue, S. Chaudhuri, K. Fragkiadaki, M. Khan, and Y. Sun (eds.), International Conference on Learning Representations, volume 2024, pp. 42393–42410, 2024b. URL https://proceedings.iclr.cc/paper_files/paper/2024/file/b95f3f8b6cff9e7ac6553a56b895cf70-Paper-Conference.pdf.arxiv.org/abs/2311.14645
- [28]Donald R Jones, Matthias Schonlau, and William J Welch. Efficient global optimization of expensive black-box functions. Journal of Global Optimization, 13(4):455–492, 1998. doi: 10.1023/A:100836431147.DOI
- [29]Lison Kardassevitch, Davide Burchielli, Numa Dancause, and Marco Bonizzato. NeuroMOSAICS: A collection of neurostimulation datasets - multi-scale open-source across interfaces conditions & species. NeuroLibre Reproducible Preprints, 33, 2025. doi: 10.55458/neurolibre.00033. URL https://doi.org/10.55458/neurolibre.00033. Data archive: https://doi.org/10.5281/zenodo.1569258.
- [30]Kasia Kobalczyk, Zhiyuan Jerry Lin, Benjamin Letham, Zhuokai Zhao, Maximilian Balandat, and Eytan Bakshy. LILO: Bayesian optimization with natural language feedback. In Proceedings of the 43rd International Conference on Machine Learning, volume 306 of Proceedings of Machine Learning Research, pp. 59643–59693. PMLR, 2026. URL https://proceedings.mlr.press/v306/kobalczyk26a.html.arxiv.org/abs/2510.17671
- [31]Agustinus Kristiadi, Felix Strieth-Kalthoff, Marta Skreta, Pascal Poupart, Alán Aspuru-Guzik, and Geoff Pleiss. A sober look at LLMs for material discovery: Are they actually good for Bayesian optimization over molecules? In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.), Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 25603–25622. PMLR, 2024. URL https://proceedings.mlr.press/v235/kristiad24a.html.arxiv.org/abs/2402.05015
- [32]H. J. Kushner. A New Method of Locating the Maximum Point of an Arbitrary Multipeak Curve in the Presence of Noise. Journal of Basic Engineering, 86(1):97–106, March 1964. ISSN 0021-9223. doi: 10.1115/1.3653121. URL https://doi.org/10.1115/1.3653121.
- [33]Jiale Lao, Yibo Wang, Yufei Li, Jianping Wang, Yunjia Zhang, Zhiyuan Cheng, Wanghu Chen, Mingjie Tang, and Jianguo Wang. GPTuner: A manual-reading database tuning system via GPT-guided Bayesian optimization. Proceedings of the VLDB Endowment, 17(8):1939–1952, April 2024. ISSN 2150-8097. doi: 10.14778/3659437.3659449. URL http://dx.doi.org/10.14778/3659437.3659449.
- [34]Alejandro Velasco Levy and Antonio Montalvo. The tunneling algorithm for the global minimization of functions. SIAM Journal on Scientific and Statistical Computing, 6(1):15–29, 1985.
- [35]Tennison Liu, Nicolás Astorga, Nabeel Seedat, and Mihaela van der Schaar. Large language models to enhance Bayesian optimization. In B. Kim, Y. Yue, S. Chaudhuri, K. Fragiadaki, M. Khan, and Y. Sun (eds.), International Conference on Learning Representations, volume 2024, pp. 31252–31284, 2024. URL https://proceedings.iclr.cc/paper_files/paper/2024/file/84b8d9fcb4e262fcd429544697e1e720-Paper-Conference.pdf.arxiv.org/abs/2402.03921
- [36]Neeratyoy Mallik, Edward Bergman, Carl Hvarfner, Danny Stoll, Maciej Janowski, Marius Lindauer, Luigi Nardi, and Frank Hutter. PriorBand: Practical hyperparameter optimization in the age of deep learning. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing Systems, volume 36, pp. 7377–7391. Curran Associates, Inc., 2023. doi: 10.52202/075280-0323. URL https://proceedings.neurips.cc/paper_files/paper/2023/file/1704fe7aeaff33a54802b83a016050ab8-Paper-Conference.pdf.DOI
- [37]J. Mockus. On the Bayes methods for seeking the extremal point. IFAC Proceedings Volumes, 8(1, Part 1):428–431, 1975. ISSN 1474-6670. doi: 10.1016/S1474-6670(17)67769-3. URL https://www.sciencedirect.com/science/article/pii/S1474667017677693. 6th IFAC World Congress (IFAC 1975) - Part 1: Theory, Boston/Cambridge, MA, USA, August 24-30, 1975.DOI
- [38]Roshan A. Patel, Mingxuan Li, Chin-Fei Chang, Louis De Lescure, Paul Chauvin, Alan Cherney, Saeed Moayedpour, Sven Jager, and Yasser Jangjou. Distilling and exploiting quantitative insights from large language models for enhanced Bayesian optimization of chemical reactions. Digital Discovery, 5:2042–2051, 2026. doi: 10.1039/D6DD00052E.DOI





















