In this paper
AbstractIntroductionRelated WorkTransfer Learning for Bayesian OptimizationLearning Query Policies for Black-Box OptimizationLearned Priors and In-Context PredictionMethodProblem Setting and OverviewHistory-Conditioned Practice FunctionsFeedback-Conditioned Candidate AcquisitionLearning from Terminal Returns and Candidate ResponsesExperimentsExperimental SetupSynthetic Task FamiliesEngineering Simulation TasksConclusionAI use statementMethod ImplementationPractice-Function ConstructionOnline Representation, Candidates, and TrainingExperimental ProtocolTask Archives and Evaluation BudgetsSynthetic Task ConstructionEngineering Simulation TasksBaseline Information and TrainingSynthetic Evaluation and Complete TrajectoriesComplete Engineering Simulation TrajectoriesReferencesWorksComputer Science
CAP: Learning to Optimize from Sparse Observations of Related Tasks
Abstract
In experiment-driven black-box optimization, each evaluation can consume materials, equipment time, and other resources, limiting the number of experiments available for a new task. Records from completed related optimization campaigns could help, but learning a sequential query policy remains challenging when task-specific experience consists of only a few sparse histories. We propose CAP (Candidate Acquisition Policy), which models shared structure and task-specific variation in fixed historical records to sample interactive practice functions. On these functions, CAP learns a feedback-conditioned query policy aimed at final optimization performance under a fixed evaluation budget. At deployment, its parameters remain fixed while each new observation informs the next query. Across five synthetic task families spanning 6 and 20 dimensions and two engineering simulation benchmarks, BOPTEST and SU2, CAP attains the lowest median final-budget value among the reported methods on four synthetic families and both engineering benchmarks. These results indicate that sparse records from completed related tasks can support adaptive query decisions on new tasks, making more effective use of limited, costly experimental evaluations.
Introduction
Experiment-driven black-box optimization seeks high-quality solutions with a limited number of objective evaluations. Each evaluation may consume materials or equipment time, or require a time-consuming engineering simulation, making effective use of every query on a new task important [16, 7]. Bayesian optimization (BO) uses observations and predictive uncertainty to guide subsequent queries under such budgets [7]. Yet a new task need not start entirely from scratch: completed optimization campaigns on related tasks leave records that may contain useful experience. This paper asks how that experience can be turned into effective query decisions on a new task.
We study a setting with limited historical experience: only sparse optimization records from a small number of related tasks are available, and these completed tasks cannot be evaluated further. The goal is to learn a query policy from the records that uses incoming feedback to optimize a new, related task within a fixed evaluation budget. Related tasks can share useful structure without having identical response functions, so configurations that worked well in the past cannot simply be reused. A policy must draw on historical experience while continually revising its decisions in light of observations from the current task.
Transfer BO methods can use historical observations to improve predictions or query selection on a new task [19]. Learned optimizers go further by learning query behavior from related tasks [17, 9]. Training such behavior requires tasks or approximations that can answer newly proposed queries; these interactions need not be physical experiments. In our setting, however, task-family-specific experience is limited to sparse, fixed records of completed campaigns. The records cannot directly answer queries proposed during policy training, and the available observations do not uniquely determine the response surfaces of related tasks. The central challenge is therefore not merely to identify promising locations from historical data, but to construct grounded interactive practice within this information boundary and learn a sequential strategy that adapts to feedback on a new task.
We propose CAP (Candidate Acquisition Policy). CAP models structure shared across historical tasks alongside task-specific variation, then samples interactive practice functions from the fitted distribution. These functions provide sequential optimization experience without further evaluations of the historical tasks. On them, CAP learns candidate-selection behavior aimed at final performance under a fixed evaluation budget. When deployed on a new related task, the policy parameters remain fixed: decisions use the growing observation context and candidate information from a frozen pretrained predictor, and each new evaluation informs the next query. Historical experience is thus distilled into reusable query behavior, while adaptation to a particular target task occurs through feedback rather than policy fine-tuning.
We evaluate CAP on five synthetic task families spanning 6 and 20 dimensions and on two engineering simulation benchmarks, building control with BOPTEST ([2]) and airfoil design with SU2 ([4]), against single-task, transfer, and learned optimizers. CAP attains the lowest median final-budget value among the reported methods on four synthetic families and both engineering benchmarks; FSAF leads on Levy20. These results indicate that sparse records from completed related tasks can support adaptive queries on new tasks, making more effective use of limited, costly evaluations.
Our main contributions are:
We develop a way to construct interactive practice functions from sparse, fixed records of related tasks, enabling sequential query-policy training from completed campaigns.
We introduce CAP, which combines record-derived practice with policy learning aimed at final performance under a fixed budget. At deployment, the policy adapts its decisions through the growing observation context without updating its parameters.
We evaluate the complete optimizer across synthetic task families with different landscapes and dimensions and two engineering simulation benchmarks, demonstrating its final-budget optimization performance.
Related Work
Work on query efficiency under limited evaluation budgets either improves search and acquisition rules or uses experience from related tasks to learn predictions and decisions. In the former direction, TuRBO organizes search within local trust regions ([5]), while FunBO discovers reusable acquisition functions ([1]). CAP focuses on a more constrained setting in the latter direction: only fixed records from a small number of related tasks are available, yet the goal is to train a policy that selects queries using feedback from a new task. We discuss related work on cross-task transfer, query policy learning, and in-context prediction.
Transfer Learning for Bayesian Optimization
Transfer BO typically uses historical experience in the surrogate or acquisition decisions for a new task. RGPE weights GP models according to the ranking performance of historical models on current-task observations ([6]); ABLR relates regression models across tasks through a shared representation ([13]); and FSBO meta-learns a deep-kernel surrogate that adapts to new-task observations ([19]). At the acquisition level, TAF combines historical-task information with expected improvement on the new task ([20]). CAP also uses cross-task relatedness, but the historical task model generates practice functions for policy training rather than predicting candidate responses at deployment; training incorporates historical experience into the policy parameters.
Learning Query Policies for Black-Box Optimization
MetaBO trains a neural acquisition function with reinforcement learning [17]; FSAF connects acquisition functions to action values [9]; and NAP jointly learns predictive representations and acquisition decisions [10]. OptFormer takes another route, learning query behavior from existing hyperparameter optimization trajectories [3].
Interactive training of the first three methods requires responses at points selected by the policy: MetaBO uses evaluable tasks or inexpensive approximations, FSAF uses sampleable functions, and NAP uses candidate responses available from source tasks. In our setting, only sparse records from a small number of completed tasks are available; if a policy selects an unrecorded point, the records cannot provide feedback, so subsequent training trajectories cannot proceed as in the original setups. Thus, the limitation is not in these policy models themselves, but in the inability to meet their original training conditions directly from fixed records. CAP constructs queryable computational practice from the records, then learns a feedback-conditioned policy aimed at final performance under a fixed budget.
Learned Priors and In-Context Prediction
Prior-data fitted networks (PFNs) make predictions conditioned on an observation context [11]; TabPFN applies this approach to prediction on small tabular datasets [8], and PFNs4BO studies its use in Bayesian optimization [12]. CAP uses a frozen pretrained predictor to provide candidate features for the current task and trains the query policy separately. This predictor differs from the practice-function model fitted to historical records: the former supports current decisions, whereas the latter provides an interactive environment only during training.
Method
Problem Setting and Overview
We consider related black-box minimization tasks with comparable decision variables, mapped to a common domain . The available historical experience consists of fixed optimization records from completed tasks,
The tasks need not have identical response functions. Training may use these records and quantities derived from them, but cannot obtain further evaluations of the historical objectives or interact with a new target task.
On a new related task , an initial design of evaluations is followed by adaptive queries within a total budget . Let contain the initial observations and the first adaptive evaluations, and let be the best value observed so far. The objective is a small terminal value ; when a task optimum is available to the evaluator, it can also compute terminal simple regret. Neither the optimum nor unqueried target responses are available to the policy. Historical record length and target budget have distinct roles, even when an experimental protocol assigns them the same numerical scale.
CAP turns the fixed records into interactive computational practice and learns sequential query behavior over complete budgets. For a given historical archive and budget condition, training yields one policy whose parameters remain frozen when deployed on new tasks associated with that archive. The method is fitted and trained separately for different archives; a single set of weights is not assumed to transfer across unrelated task families. Adaptation to a new task comes from its growing observation context and the resulting query decisions, not from retraining the policy. Figure 1 summarizes this process.
Figure 1. Overview of CAP. Fixed historical records fit an approximate distribution of coherent practice functions on which complete trajectories train a candidate-selection policy. At each step, the menu and represented observations inform a frozen, externally pretrained TabPFN and the actor–critic; only the selected response updates the observation context. Full trajectories yield PPO returns, while responses of all practice candidates supply auxiliary labels only to the training loss. At deployment, one frozen policy per historical archive and budget condition uses current-task feedback and pays for only the selected target query.
History-Conditioned Practice Functions
Fixed records provide responses only at previously queried locations and cannot directly answer new queries proposed during policy training. To obtain complete sequential training trajectories, we fit an approximate, history-conditioned task distribution and sample
Each draw defines a function path fixed throughout an optimization episode. Interacting with it is computation derived from the records, not another evaluation of a historical or target objective.
Shared structure and task variation. For the continuous part of the responses, we model historical observations with shared spatial structure and task-specific variation:
Here transforms responses for historical modeling, is a common mean, captures structure shared across tasks, captures task-specific spatial variation, and is an optional task-level offset. The kernels and describe the shared and task-specific processes, respectively; is a fitting-noise term for historical observations. All records contribute to the joint fit. In a new practice task, the shared structure is conditioned on the records, while its task-specific process and offset are drawn anew. A practice function therefore need not reproduce any particular historical response surface.
We approximate prior function paths with random Fourier features and condition whole paths on the historical observations [14, 18]. The path and any applicable discrete response component are fixed at the start of an episode; repeated queries at the same location do not resample fitting noise. The resulting is an approximation constrained by sparse records, not an exact recovery of the true task distribution. Response-transformation families and whether a discrete component is enabled are specified for each experimental condition before target-task evaluation; their numerical parameters are fitted only from the corresponding historical records. These response-modeling choices do not change the subsequent policy architecture or training objective. Path construction and the response choices are detailed in Appendix § 6.
Feedback-Conditioned Candidate Acquisition
The historical archive influences query behavior through the trained policy parameters, but is not supplied as a direct input at each target-task decision. The policy instead uses the current task’s evaluated observations, candidate locations, predictions based only on those observations, and the decision stage. The practice generator’s latent variables and unqueried responses are unavailable.
Current-task response representation. To separate response modeling for practice-task generation from online decision features, define a context-dependent representation
Unlike , which is used to construct practice functions, depends only on the current task’s evaluated observations and is recomputed after every query. The synthetic experiments use the identity representation; the engineering experiments use within-context normal scores based on response ranks, so wide response ranges do not compress input differences among favorable observations. Each representation is held fixed for its training condition and used identically at deployment; it is not selected using target-task outcomes. The representation affects only predictor and policy inputs. Incumbents, training returns, auxiliary labels, and reported objective values retain the original response units.
Candidate menu and online predictor. At step , a fixed-size menu combines a persistent global Sobol pool with local candidates regenerated around spatially separated, favorably observed locations. Selected global points are removed from the pool. Local sampling uses a predefined budget-dependent radius schedule, while its centers respond to the observations. The policy learns which available candidate to query; it does not learn the menu’s allocation or radius rule.
A frozen, externally pretrained TabPFN-v2 regressor predicts candidate means and standard deviations from [8]. These predictions are in the representation space. TabPFN supplies current-task decision features, whereas the historical model in Eq. (3) supplies training environments. Task-family-specific practice and policy learning come from the fixed records, distinct from TabPFN’s external pretraining.
Query decisions and feedback. A dual-stream attention actor–critic encodes observation and candidate tokens separately and uses candidate-to-context cross-attention to form feedback-conditioned candidate representations. A learned step embedding indicates the stage of the fixed query horizon. The actor assigns a logit to each candidate and defines
where comprises the represented observations, candidate features, and the stage within the fixed budget. The critic shares the encoder and estimates state value. Training samples an action from ; deployment selects its highest-scoring candidate. Only that candidate’s response is added to , after which the representation, predictions, and menu are rebuilt. The policy can thus revise subsequent decisions in response to new feedback, but learns selection within the predefined menu rather than its allocation or radius rule.
Learning from Terminal Returns and Candidate Responses
Practice functions provide two complementary training signals: terminal improvement over complete query trajectories and additional responses at the current candidates. The former targets fixed-budget optimization; the latter provides denser supervision at visited states, but reflects immediate response quality only.
Terminal objective. For each practice function , a fixed reference set determines a positive response scale before the episode. The scale remains constant within the episode and is used only for training. With computed from raw practice responses, define
For a fixed practice function and initial design, the undiscounted return equals its scale-normalized terminal improvement; across functions, training optimizes the expectation of this quantity. No exact function optimum is needed for the reward.
We update the actor–critic with clipped PPO [15], using returns and advantages from complete sampled trajectories. A query without immediate improvement can therefore receive a learning signal through later improvements in its trajectory. Optimization and numerical settings are given in Appendix A.
State-adaptive candidate supervision. At each visited state, the sampled practice function can also answer queries for the other candidates. Their signed, immediate utilities are
We retain negative values so that candidates can still be ordered when none improves the incumbent. Sort utilities as and form a soft label with a state-dependent temperature:
Here is the candidate rank used to set the temperature, controls its scale, supplies a numerical floor, and mixes the labels with a uniform distribution. We detach from gradient computation and minimize
These labels describe immediate response quality, not the long-term information value of a query. They complement, rather than replace, the complete-trajectory objective.
Joint training and deployment. The training loss combines the clipped PPO policy loss, a value loss, entropy regularization, and candidate supervision:
Here is the PPO actor loss, trains the critic to predict complete-trajectory returns, denotes policy entropy, and , , are loss weights. Responses of unselected candidates are used only to construct training labels; they do not enter the policy state, TabPFN context, or subsequent observation trajectory. These extra evaluations are computational queries to practice functions, not target-task evaluations.
After training, deployment retains the frozen predictor, candidate construction, and one fixed policy for the historical archive and budget condition. It does not use the practice generator or candidate labels. Each adaptive step pays for only the selected target query and returns the best point observed within the budget.
Experiments
We ask whether a policy learned from a small number of fixed historical optimization records can improve budget-limited optimization on new, related tasks. We first compare the complete optimizer across five synthetic task families with different landscapes and dimensions, then examine whether its performance extends to two engineering simulation benchmarks.
Experimental Setup
Tasks and budgets. We evaluate five synthetic families—Hartmann6, Ackley6, Levy6, Ackley20, and Levy20—each with 20 historical tasks ( fixed observations per task) and 100 distinct target tasks; their construction is outlined in Appendix § 7.§ 2. BOPTEST building control and SU2 airfoil design each have 20 historical and 20 target tasks ([2, 4]). All methods use the same target tasks, shared initial design (), and total budget ( evaluations) within each benchmark. CAP trains one policy per historical archive and freezes it on target tasks. Further protocol is in Appendix § 7; synthetic scoring is in Appendix § 8.
Baselines. Single-task optimizers are EI, PI, UCB, RolloutEI-2, FunBO, and TuRBO; FunBO uses a discovered acquisition function and TuRBO uses local trust regions (Aglietti et al., [1]; Eriksson et al., [5]). Record-based transfer methods are TAF-ranking, RGPE, ABLR, and FSBO (Wistuba et al., [20]; Feurer et al., [6]; Perrone et al., [13]; Wistuba & Grabocka, [19]). FSAF, MetaBO, and NAP learn query policies through interaction (Hsieh et al., [9]; Volpp et al., [17]; Maraval et al., [10]). We also include publicly pretrained PFNs4BO as an external-prior reference (Müller et al., [12]). Each retains its native surrogate, candidate construction, and decision rule; objective-evaluation budgets are matched.
FSAF, MetaBO, and NAP need training tasks that answer new queries, but the historical records cannot. We therefore train them on the same record-derived, queryable practice-function distribution as CAP, within the same historical-information boundary. They share practice feedback, not CAP’s predictor, architecture, candidates, or loss. The distinct roles of CAP’s frozen TabPFN predictor and the externally pretrained PFNs4BO model are explained in § 3.3.
Evaluation metric. For synthetic tasks, we report median normalized simple regret across targets, at the final budget and along the trajectory. Since engineering optima are unknown, we report median best-observed objective values in original units. Lower is better. Appendix § 8 details scoring and aggregation.
Synthetic Task Families
Table 1 compares final regret across all methods; Figure 2 shows trajectories for a fixed cross-category selection. All trajectories appear in Appendix § 8.
| Method | Hartmann6 | Ackley6 | Levy6 | Ackley20 | Levy20 |
| CAP | 0.0267 | 0.1514 | 0.0220 | 0.1806 | 0.1798 |
| FSAF | 0.7936 | 0.6786 | 0.0384 | 0.7808 | 0.1589 |
| MetaBO | 0.1511 | 0.6052 | 0.0721 | 0.7808 | 0.3311 |
| NAP | 0.2606 | 0.1672 | 0.0686 | 0.7407 | 0.3460 |
| TAF-ranking | 0.1365 | 0.2643 | 0.0985 | 0.7248 | 0.2804 |
| RGPE | 0.1201 | 0.3821 | 0.0778 | 0.7125 | 0.2747 |
| ABLR | 0.1765 | 0.5680 | 0.0616 | 0.7361 | 0.3212 |
| FSBO | 0.0900 | 0.2479 | 0.0639 | 0.6822 | 0.2308 |
| PFNs4BO | 0.1610 | 0.3717 | 0.0877 | N/A | N/A |
| TuRBO | 0.1070 | 0.3065 | 0.0876 | 0.4816 | 0.2254 |
| EI | 0.4870 | 0.5060 | 0.1290 | 0.7167 | 0.2839 |
| PI | 0.4588 | 0.5230 | 0.1165 | 0.7155 | 0.2833 |
| UCB | 0.4840 | 0.5291 | 0.1372 | 0.7133 | 0.2872 |
| FunBO | 0.5010 | 0.5145 | 0.1219 | 0.7144 | 0.2840 |
| RolloutEI-2 | 0.5063 | 0.5350 | 0.1146 | 0.7225 | 0.2824 |
Table 1. Final median normalized simple regret on 100 target tasks per family (lower is better; best in bold). PFNs4BO supports at most 18 dimensions and is omitted in 20D.
Figure 2. Median normalized simple regret across 100 target tasks per family (lower is better). Curves begin after the initial design, included in the evaluation count; panel scales differ. Appendix C shows all methods.
CAP has the lowest median final regret among the reported methods in four of the five families, including Ackley20 (Table 1). On Levy20, FSAF has the lowest median (0.1589), followed by CAP (0.1798). Thus the endpoint benefit spans different landscapes and dimensions, but does not extend to every baseline on every family. The trajectories in Figure 2 show how the methods improve with target-task feedback, complementing the fixed-budget comparison in the table.
Engineering Simulation Tasks
Synthetic families allow controlled variation between related tasks, whereas the engineering experiments use specific simulator objectives and operating conditions. BOPTEST varies operating time windows within a shared building and control space; SU2 varies flight conditions within a shared airfoil-design space. BOPTEST has and a budget of 30 simulator evaluations per target task; SU2 has and a budget of 50. Policy training uses only the fixed historical records and their derived practice tasks, not simulator calls on target tasks.
Table 2 reports the median final best-observed objective for all reported methods on both benchmarks. Figure 3 shows their progress within the simulation budget; trajectories for every reported method appear in Appendix § 9.
Figure 3. Median best-observed objective versus simulator evaluations (lower is better). Vertical axes use each benchmark’s original scale. The SU2 panel zooms to evaluations 10–50 and objective values 0.066–0.080; earlier evaluations still count toward the budget. Appendix D shows all methods.
| Method | BOPTEST (6D) | SU2 (10D) |
| CAP | 0.1651 | 0.06787 |
| FSAF | 0.1710 | 0.07072 |
| MetaBO | 0.2078 | 0.07048 |
| NAP | 0.1878 | 0.07065 |
| TAF-ranking | 0.1773 | 0.06935 |
| RGPE | 0.1926 | 0.06988 |
| ABLR | 0.4497 | 0.07018 |
| FSBO | 0.2786 | 0.07024 |
| PFNs4BO | 0.1695 | 0.06988 |
| TuRBO | 0.1801 | 0.06901 |
| EI | 0.2057 | 0.06944 |
| PI | 0.1943 | 0.06963 |
| UCB | 0.1808 | 0.07007 |
| FunBO | 0.1935 | 0.06952 |
| RolloutEI-2 | 0.1814 | 0.06950 |
Table 2. Final median best-observed objective across 20 target tasks per engineering benchmark (lower is better). Values use each benchmark’s original scale; bold marks the best result.
CAP obtains the lowest median final best-observed objective among the reported methods on both BOPTEST and SU2 (Table 2). The trajectories show how this performance develops across simulator evaluations (Figure 3). Together with the synthetic results, these experiments demonstrate its budget-limited optimization performance across the evaluated task families.
The two engineering benchmarks both show modest final-median margins. On BOPTEST, the final medians of CAP (0.1651), PFNs4BO (0.1695), and FSAF (0.1710) are close. Several methods improve rapidly early in the budget and approach similar values near its end. On SU2, CAP reaches 0.06787 after 50 evaluations, while the next lowest median among the reported methods is TuRBO’s 0.06901. The zoomed trajectory shows a continuing but small late-budget improvement. These comparisons are within each benchmark: BOPTEST and SU2 retain their own objective scales, and the reported medians do not imply a win on every target task.
Conclusion
We introduced CAP to turn fixed, sparse records of completed related tasks into interactive computational practice and learn a sequential query policy for a fixed evaluation budget. On a new task, the policy requires no further training: it adjusts subsequent queries as observations accumulate. Across five synthetic families
and two engineering simulation benchmarks, CAP achieves the lowest median final-budget value among the compared methods on four synthetic families and both engineering benchmarks, while FSAF leads on Levy20. These results show that even sparse, non-extendable records can provide useful experience for feedback-conditioned optimization on new tasks, helping make better use of limited, costly objective evaluations.
This setting matters when past campaigns cannot be extended: the practice used for policy learning is generated computationally from their fixed records, while target-task evaluations are reserved for deployment. The policy then uses its learned parameters together with the observations collected on the current task, without updating those parameters between queries. Our evidence concerns new tasks related to the recorded ones and sharing their decision space; it does not establish performance on unrelated task families. Within that scope, the results support learning a reusable query policy from a small historical archive rather than requiring fresh interactions with the historical objectives.
AI use statement
Generative AI tools were used to assist with language polishing of the manuscript, code optimization, and organization of figures and research materials. The authors reviewed the AI-assisted work and take responsibility for the final manuscript and research results.
Method Implementation
Practice-Function Construction
Flatten the continuous-response historical observations and let identify the task associated with record . The model in Eq. (3) gives the covariance
Both spatial kernels are isotropic Matérn-5/2 kernels. A new task shares only with historical tasks; its task-specific process and offset are sampled anew. Model parameters are fitted using the corresponding historical records, without choosing them from target-task responses or results.
We approximate prior paths with random Fourier features and condition whole paths on the historical observations. Let be the transformed historical responses, the prior draw at historical inputs (including the historical tasks’ processes, offsets, and fitting noise), and the new task’s prior latent path. With , the continuous practice path, conditional on a mean and response-transform component, is
Applying the inverse response transformation, and any applicable fixed discrete response component, yields . The path, transformation component, and discrete component remain fixed within a draw; historical fitting noise is not resampled on practice queries.
The joint model and path-conditioning procedure are shared across experiments. Response transformations and an optional discrete component accommodate different response distributions in the fixed historical records. Synthetic experiments use an affine transformation; engineering experiments use record-fitted monotone transformations, with a discrete component where repeated responses occur. These choices are fixed before target-task evaluation, with numerical parameters fitted only from the corresponding records; they do not alter the candidate policy, training objective, or deployment procedure.
Online Representation, Candidates, and Training
The online representation in Eq. (4) serves decision inputs, whereas serves practice-function construction. Synthetic runs use . In engineering runs, let and let be the one-based average rank of among the observed responses, with ties sharing their average rank. Then
This representation is recomputed from current observations at every step, with the same rule used in training and deployment. It leaves raw-response incumbents, rewards, and evaluation unchanged. The wide range of historical engineering responses motivates representing their relative differences without introducing hidden task parameters.
Each decision uses candidates: 128 points from an initial persistent scrambled Sobol pool and local points filling the remaining slots. Selected persistent points are removed. Local points are generated around up to three spatially separated, favorably observed locations. Their centers change with feedback, while the radius and allocation follow a predefined budget-dependent rule and are not policy actions. Context tokens contain locations and response-order features; candidate tokens contain locations, TabPFN predictive means and standard deviations in the online representation space, and a persistent-pool indicator. Separate attention streams encode context and candidates before candidate-to-context attention forms action scores.
The candidate construction, network design, and training objective are shared across experiments. The auxiliary-label parameters in Eq. (8) are , , , and ; the value, entropy, and auxiliary coefficients in Eq. (10) are , , and . PPO uses complete-trajectory returns without discounting.
Experimental Protocol
This appendix expands the experimental setup in § 4.1. The practice-function sampler and policy implementation are described in Appendix § 6; the benchmark task generators and evaluators are separate from that training procedure.
Task Archives and Evaluation Budgets
Each benchmark provides a fixed archive of related historical tasks, with observed input–response pairs per task. The five synthetic families each have 100 target tasks with identities distinct from those in the corresponding historical archive. BOPTEST and SU2 each have 20 historical and 20 target tasks. Decision variables are mapped to within each benchmark. The dimensions are for Hartmann6, Ackley6, Levy6, and BOPTEST; for SU2; and for Ackley20 and Levy20.
At each target task, the methods receive the same initial design of scrambled Sobol points and a total budget of objective evaluations, including that design. Thus is , , or for 6D, 10D, or 20D tasks, respectively. The reported target-task runs use evaluation seed 42. Every target-task objective evaluation, including the initial design, counts toward ; constructing and querying practice functions during training are computational operations. Once an archive is frozen, no method can request additional evaluations of a historical task. The synthetic reference values used for scoring are withheld from all optimizers and are defined in Appendix § 8.
Synthetic Task Construction
Hartmann6, Ackley6, Levy6, Ackley20, and Levy20 denote related-task families, not repeated optimization of a single unmodified benchmark function. All tasks are defined on . A benchmark generator samples each task’s parameters and provides objective responses to the historical logger or the target-task evaluator; neither the parameters nor unqueried responses are available to CAP training. Historical and target tasks have distinct identities, and all 100 sampled targets in each family are retained without selection by method performance.
Task variation. Hartmann6 retains the four basin centers and base precision coefficients of the Hartmann function. For task parameters , its objective has the form
Here is a boundary-preserving invertible warp, while , , and vary basin depth, width, and coordinate anisotropy. The base depths are . Warp shifts, scales, and pairwise coupling angles range over , , and , respectively; depth factors range over . Width and anisotropy receive smaller task-specific perturbations. The generator-only factor keeps the numerical reference minimum on the base Hartmann scale. Neither nor the task parameters are supplied to an optimizer.
For Ackley and Levy, a task combines a base function, the same type of invertible warp, and a localized smooth perturbation. Let map unit-cube coordinates to the corresponding base-function domain, coordinatewise. Then
Specifically, for Ackley and for Levy, giving base-function domains and , respectively; is the corresponding base optimum in unit-cube coordinates. The localized perturbation uses 32 cosine features with task-specific coefficients and envelope width . Its amplitude is 0.3 times the family reference scale for Ackley and 0.1 for Levy. Warp shifts, scales, and coupling angles range over , , and , respectively. These variations produce distinct related tasks within a common decision space.
Engineering Simulation Tasks
BOPTEST evaluates a six-parameter building-control problem using the multizone_office_simple_air FMU in BOPTEST v0.9.0. Tasks correspond to different operating windows in the same building and Chicago representative weather year. Each window has a seven-day warm-up followed by three scored days at one-hour control intervals. The six decision variables are occupied heating and cooling setpoints (18–22°C and 22–26°C), unoccupied heating and cooling setpoints (12–18°C and 26–32°C), supply-air temperature (12–18°C), and duct static pressure (100–350 Pa). A shared coordinate mapping enforces heating–cooling deadbands of at least 2°C when occupied and 4°C when unoccupied. The minimized scalar objective is ; it is not normalized by a task-specific optimum.
SU2 evaluates ten-dimensional RAE2822 airfoil designs with SU2 v8.5.0, an SA turbulence model, fixed target lift coefficient, and cold-start RANS evaluations from a shared mesh. The design coordinates control five Hicks–Henne deformations on each of the upper and lower surfaces, each within chord. Tasks vary flight conditions: Mach number in , Reynolds number in , and target lift coefficient in . The benchmark minimizes a prespecified scalar objective: feasible simulations use when and maximum thickness ; constraint violations receive , where , and classified geometry or numerical failures receive 2. The original scalar objectives, rather than regret to an unknown optimum, are reported for both simulators.
Baseline Information and Training
The single-task methods EI, PI, UCB, RolloutEI-2, FunBO, and TuRBO use the current target’s observations and their native surrogates and search procedures. TAF-ranking, RGPE, ABLR, and FSBO use the fixed historical records through their transfer mechanisms. FSAF, MetaBO, and NAP are trained on the same record-derived, queryable practice-function distribution as CAP, while retaining their own predictors, candidates, policies, and learning objectives; these practice queries do not access historical or target simulators. PFNs4BO uses its released externally pretrained model. CAP uses a separate frozen, externally pretrained TabPFN predictor for online features and learns its candidate policy from the historical archive. All methods retain their native target-task decision procedures, and only actual target-objective evaluations are budget-matched.
Synthetic Evaluation and Complete Trajectories
For target task , let be the lowest objective value observed through evaluation . Given the task’s reference minimum and maximum for evaluation, normalized simple regret is
The reference values are used only for offline scoring and are not supplied to the optimizers; reference minima obtained by finite search are not claimed to be certified global optima. At each evaluation count, we take the median of across the 100 target tasks in a family. The horizontal axis includes evaluations used by the initial design, so trajectories begin once that design is complete. Table 1 reports final-budget values; Figure 4–Figure 7 give the complete trajectories by method category under the protocol in Appendix § 7.
Figure 4. Median normalized simple regret for CAP and the learned query policies FSAF, MetaBO, and NAP across the five synthetic families. Lower is better; vertical scales differ by family.
Figure 7. Median normalized simple regret for CAP, FunBO, and RolloutEI-2. Each point is a cross-task median at the indicated evaluation count.
Figure 5. Median normalized simple regret for CAP, TAF-ranking, RGPE, ABLR, FSBO, and PFNs4BO. The released PFNs4BO model supports at most 18 input dimensions, so it has no 20D curves.
Figure 6. Median normalized simple regret for CAP, TuRBO, EI, PI, and UCB. The evaluation count includes the initial design.
Complete Engineering Simulation Trajectories
The engineering benchmarks have no known global optimum from which to compute regret, so we use the original objective, as in the main text. For target task , let be the objective value at simulator evaluation and the best value observed through evaluation . At each , the curves report the median of over the 20 target tasks; lower is better. Evaluation counts include the initial designs of three points for BOPTEST and five for SU2. The SU2 panels display evaluations 10–50 on a linear, truncated vertical axis spanning 0.067–0.075 to resolve the late-budget differences; earlier evaluations still count toward the budget. Table 2 gives the final-budget comparison, while Figure 8–11 show every reported method by category.
Figure 8. Median best-observed objective for CAP and the learned query policies FSAF, MetaBO, and NAP on BOPTEST and SU2.
Figure 9. Median best-observed objective for CAP, TAF-ranking, RGPE, ABLR, FSBO, and PFNs4BO on both engineering benchmarks.
Figure 10. Median best-observed objective for CAP, TuRBO, EI, PI, and UCB on both engineering benchmarks.
Figure 11. Median best-observed objective for CAP, FunBO, and RolloutEI-2 on both engineering benchmarks.
References
- [1]Virginia Aglietti, Ira Ktena, Jessica Schrouff, Eleni Sgouritsa, Francisco J. R. Ruiz, Alan Malek, Alexis Bellot, and Silvia Chiappa. Funbo: Discovering acquisition functions for bayesian optimization with funsearch, 2024. URL https://arxiv.org/abs/2406.04824.
- [2]David Blum, Javier Arroyo, Sen Huang, Ján Drgona, Filip Jorissen, Harald Taxt Walnum, Yan Chen, Kyle Benne, Draguna Vrabie, Michael Wetter, et al. Building optimization testing framework (boptest) for simulation-based benchmarking of control strategies in buildings. Journal of Building Performance Simulation, 14(5):586–610, 2021.DOI
- [3]Yutian Chen, Xingyou Song, Chansoo Lee, Zi Wang, Qiuyi Zhang, David Dohan, Kazuya Kawakami, Greg Kochanski, Arnaud Doucet, Marc’aurelio Ranzato, Sagi Perel, and Nando de Freitas. Towards learning universal hyperparameter optimizers with transformers, 2022. URL https://arxiv.org/abs/2205.13320.
- [4]Thomas D Economon, Francisco Palacios, Sean R Copeland, Trent W Lukaczyk, and Juan J Alonso. Su2: An open-source suite for multiphysics simulation and design. AIAA Journal, 54(3):828–846, 2016.DOI
- [5]David Eriksson, Michael Pearce, Jacob Gardner, Ryan D Turner, and Matthias Poloczek. Scalable global optimization via local bayesian optimization. Advances in neural information processing systems, 32, 2019.arxiv.org/abs/1910.01739
- [6]Matthias Feurer, Benjamin Letham, and Eytan Bakshy. Scalable meta-learning for bayesian optimization. stat, 1050(6):7, 2018.
- [7]Peter I Frazier. A tutorial on bayesian optimization. arXiv preprint arXiv:1807.02811, 2018.arxiv.org/abs/1807.02811
- [8]Noah Hollmann, Samuel Müller, Lennart Purucker, Arjun Krishnakumar, Max Körfer, Shi Bin Hoo, Robin Tibor Schirrmeister, and Frank Hutter. Accurate predictions on small data with a tabular foundation model. Nature, 01 2025. doi: 10.1038/s41586-024-08328-6. URL https://www.nature.com/articles/s41586-024-08328-6.DOI
- [9]Bing-Jing Hsieh, Ping-Chun Hsieh, and Xi Liu. Reinforced few-shot acquisition function learning for bayesian optimization. Advances in Neural Information Processing Systems, 34:7718–7731, 2021.arxiv.org/abs/2106.04335
- [10]Alexandre Maraval, Matthieu Zimmer, Antoine Grosnit, and Haitham Bou Ammar. End-to-end meta-bayesian optimisation with transformer neural processes. Advances in Neural Information Processing Systems, 36:11246–11260, 2023.arxiv.org/abs/2305.15930
- [11]Samuel Müller, Noah Hollmann, Sebastian Pineda Arango, Josif Grabocka, and Frank Hutter. Transformers can do bayesian inference. arXiv preprint arXiv:2112.10510, 2021.arxiv.org/abs/2112.10510
- [12]Samuel Müller, Matthias Feurer, Noah Hollmann, and Frank Hutter. Pfns4bo: In-context learning for bayesian optimization. In International Conference on Machine Learning, pp. 25444–25470. PMLR, 2023.arxiv.org/abs/2305.17535
- [13]Valerio Perrone, Rodolphe Jenatton, Matthias W Seeger, and Cédric Archambeau. Scalable hyperparameter transfer learning. Advances in neural information processing systems, 31, 2018.
- [14]Ali Rahimi and Benjamin Recht. Random features for large-scale kernel machines. Advances in neural information processing systems, 20, 2007.
- [15]John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms, 2017. URL https://arxiv.org/abs/1707.06347.
- [16]Benjamin J Shields, Jason Stevens, Jun Li, Marvin Parasram, Farhan Damani, Jesus I Martinez Alvarado, Jacob M Janey, Ryan P Adams, and Abigail G Doyle. Bayesian reaction optimization as a tool for chemical synthesis. Nature, 590(7844):89–96, 2021.DOI
- [17]Michael Volpp, Lukas Fröhlich, Kirsten Fischer, Andreas Doerr, Stefan Falkner, Frank Hutter, and Christian Daniel. Meta-learning acquisition functions for transfer learning in bayesian optimization. arXiv preprint arXiv:1904.02642, 2019.arxiv.org/abs/1904.02642
- [18]James T. Wilson, Viacheslav Borovitskiy, Alexander Terenin, Peter Mostowsky, and Marc Peter Deisenroth. Efficiently sampling functions from gaussian process posteriors, 2020. URL https://arxiv.org/abs/2002.09309.
- [19]Martin Wistuba and Josif Grabocka. Few-shot bayesian optimization with deep kernel surrogates. arXiv preprint arXiv:2101.07667, 2021.arxiv.org/abs/2101.07667
- [20]Martin Wistuba, Nicolas Schilling, and Lars Schmidt-Thieme. Scalable gaussian process-based transfer surrogates for hyperparameter optimization. Machine Learning, 107(1):43–78, 2018.DOI










