WorksComputer Science
Investigating Composite Modeling in Multi-Objective Bayesian Optimization for Scientific Design
Abstract
Scientific design often requires balancing competing objectives that are expensive to evaluate through experiments or simulations. Multi-objective Bayesian optimization (MOBO) addresses this challenge by using probabilistic models to guide the selection of promising designs. In many applications, tunable inputs map to observable intermediate responses, which are then combined through known functions to compute the final objectives. Prior work on Bayesian optimization of composite functions shows that learning from these intermediate responses, which we call Composite modeling, can substantially improve sampling efficiency in single-objective optimization, compared to methods that condition only on black-box observations of the objective function. The benefits of Composite modeling in MOBO, however, remain understudied. In this work, we study the benefits of Composite modeling in MOBO and investigate the factors driving variation in its gains. We first establish single-objective conditions under which intermediate observations provide no additional predictive information beyond objective values, where composite and black box methods use the same warped Gaussian process prior. Controlled numerical experiments with versions of four MOBO methods across eight synthetic and application-inspired scientific-design benchmarks examine how these conditions relate to optimization gains in the multi-objective setting, where the black box approach models the objective functions with a standard multi-output Gaussian process. Both investigations support the claim that composite modeling tends to help more when the function mapping intermediate responses to final objectives is non-bijective and non-affine.
Introduction
Scientific design often involves balancing competing goals. A material, for example, may need to perform well while also being inexpensive. Evaluating a design can require a costly experiment. We formulate this problem as , where is the design space. Multi-objective Bayesian optimization (MOBO) seeks favorable tradeoffs with few evaluations [1, 2], using Gaussian process (GP) models [3] and an acquisition function to select designs.
In many scientific applications, tunable inputs map to observable intermediate responses, which are then combined through known functions to compute the final objectives. These objectives have the composite form for where is an expensive black-box function returning intermediate responses and is known and inexpensive to evaluate. For single objectives, we write , and in place of , and . For example, in color formulation, dye concentrations determine measured color coordinates, which known functions convert into color differences from a target [4]. Matching under different lighting can create competing objectives [5].
Bayesian optimization (BO) of composite functions [6] models intermediate responses and propagates posterior samples through the known mappings. This can substantially improve single-objective sampling efficiency, with extensions to high-dimensional outputs and function networks [7, 8, 9, 10]. Composite modeling has also been applied to MOBO through smooth Tchebycheff scalarization (STCH) [11, 12], but the factors governing its benefits remain understudied.
We investigate the benefits of composite modeling through three inference approaches. Composite places a GP prior on the intermediate responses and conditions on their observations. BlackBox-GP places a GP prior directly on the objectives and conditions on objective observations. BlackBox-Induced uses the objective prior induced by Composite’s prior through the known mappings, but conditions only on objective observations. BlackBox-GP corresponds directly to how BO is usually implemented in practice while BlackBox-Induced supports a precise comparison of the benefits provided by the additional information provided by intermediate responses while holding the prior distribution fixed.
We find evidence that composite modeling tends to perform as well or better than black box approaches and provides the largest benefits when the functions are non-bijective and non-affine. First, we analyze the single-objective case and show that Composite and BlackBox-Induced create equivalent posterior distributions when is bijective or when is affine and the prior on has separable covariance. This suggests composite approaches are equal to black box ones in these settings. Second, experiments comparing Composite with BlackBox-GP on both single and multi-objective problems show that Composite provides a positive mean final-hypervolume gain in 15 of 16 comparisons and that the problems where Composite provides the biggest benefits are non-affine and non-bijective .
Problem Setting
Consider objectives , , where is unknown and is known. The intermediate responses may be shared across objectives. We consider both single-objective () and multi-objective () settings. For the single-objective analysis, we omit the subscript and write .
We consider three ways of performing inference in this setting, one that conditions on observations of and two that condition only on observations of . The first method, called Composite, aligns with composite BO [6]: we put a multi-output GP on and condition on observations of . The second method, called BlackBox-GP, aligns with classical BO [13, 14]: we put a GP on and condition on observations of . This treats as a black box, supposing that we cannot observe the inner function . The third method, called BlackBox-Induced, supports theoretical analysis: we start with the same GP prior on as Composite, induce a corresponding prior on through the known mapping , and condition on observations of . Thus, this method uses the same prior as Composite but conditions on the same black box information as BlackBox-GP. This is a form of warped GP prior [15] with a fixed warping function. Appendix A describes how to perform posterior inference under this induced prior.
Below, using theory and experiments, we analyze whether observing (in the Composite method) provides a benefit over observing only (the two BlackBox methods) and how this depends on the choice of . We focus specifically on whether is a bijection or is affine, finding support for the claim that conditioning on does not provide a benefit when is a bijection or is affine, and often does provide a benefit when is neither bijective nor affine.
Theoretical Analysis
We first compare Composite and BlackBox-Induced in a single-objective case theoretically.
Proposition 1 (Posterior equivalence). At fixed evaluation locations , let , and . Suppose is drawn from a GP prior with fixed known mean function and kernel. Suppose also that either (1) is bijective on the possible intermediate responses, so each objective value uniquely determines its intermediate response; or (2) for some fixed and , and the GP prior on $h has separable covariance. Then, for every fixed and , with probability one.
Technical conditions and the proof are provided in Appendix § 9. Under the Proposition 1, which applies to Identity and Cubic experiments below, Composite and BlackBox-Induced yield the same posterior over when they share a GP prior on . Consequently, acquisition rules determined solely by the posterior produce the same decisions and final posterior distribution. In these problems, differences observed between Composite and BlackBox-GP are due to the prior distribution, Monte Carlo noise, or small implementation differences, since BlackBox-GP and BlackBox-Induced condition on the same information.
Experimental Analysis
Controlled Synthetic Experiments: We fix scalar intermediate functions and vary the known outer mapping (See Appendix § 10 for construction). We compare BlackBox-GP and Composite methods, each of which fits its own GP models and share the initial design, acquisition rule, and evaluation budget. Single-objective experiments use LogEI [13, 16] and we report mean percentage of final simple regret (Final-RG) reduction for Composite relative to BlackBox-GP with standard error (SE). For multi-objective, we use , and , where and are independent scalar intermediates, modeled with independent GPs. The identity mapping isolates composition in . We use LogEHVI [17, 16] and report mean percentage of final hypervolume (Final-HV) gain against a shared reference point for Composite relative to BlackBox-GP with SE. Higher metrics favor Composite. Single-objective starts with 5 initial points and two-objective starts with 10; both then sequentially evaluate 40 additional points based on their acquisition rules. Appendix D details comparison metrics. Four function draws and five initial designs per draw yield 20 paired runs.
Table 1 shows significant single-objective gains for all four selected non-bijective mappings, while for multi-objective cases, only square and three-root cubic yield significant hypervolume gains. Neither selected bijective mapping shows a significant gain in either setting. Square gives the largest gain of the six in both settings, but the two scales are not comparable: its single-objective gain is a 99.8% reduction in final regret, against a hypervolume gain under 3%. Hypervolume optimization requires discovering a set of trade-offs rather than a single optimum, which may limit Composite’s gains under a fixed evaluation budget. Larger-budget experiments could test this explanation.
| Outer mapping | Bijective? | Single objective | Multi-objective | ||
| Final-RG reduction (%) | Final-HV gain (%) | ||||
| Identity | Yes | ||||
| Cubic | Yes | ||||
| Square | No | ||||
| Shifted square | No | ||||
| Three-root cubic | No | ||||
| Cosine | No |
Table 1. Controlled synthetic experiments. Final regret reductions (single-objective) and final hypervolume gains (multi-objective) for Composite relative to BlackBox-GP, both in percentage (%) (mean SE over 20 paired runs). Positive values favor Composite. Bold marks unadjusted (one-sided paired -tests for improvement).
Scientific-Design Experiments: We compare BlackBox-GP and Composite versions of four MOBO methods on eight benchmarks as listed in Table 2 (see Appendix § 13 for details). All have at least one non-bijective outer mapping; only truss design and power flow have affine outer mappings. Low-dimensional problems use LogEHVI [17, 16] and STCH [11, 12] methods with 5 initial and 40 adaptive evaluations; high-dimensional problems use STCH with a spherical kernel (sp-STCH) [25] and MORBO [26] with 20 initial and 100 adaptive evaluations. Appendices § 14 and § 15 detail methods and implementation. Composite models intermediates with independent GPs. Two methods share the initial design, acquisition rule, reference point, and budget. We report mean percentage gains in Final-HV ± SE over 10 different trials for all problems, except Synthetic 1, which uses 80 trials.
| Low-dimensional | High-dimensional | ||||||
| Benchmark | Method | Final-HV gain (%) | Benchmark | Method | Final-HV gain (%) | ||
| Color formulation [5] | LogEHVI | +13.8 ± 1.6 | < 0.01 | Synthetic 2 [18] | Sp-STCH | +3.6 ± 1.8 | 0.038 |
| STCH | +8.7 ± 1.7 | < 0.01 | MORBO | +5.3 ± 2.6 | 0.036 | ||
| Synthetic 1 [19] | LogEHVI | +12.7 ± 0.6 | < 0.01 | Lake management [20] | Sp-STCH | +8.7 ± 0.7 | < 0.01 |
| STCH | +7.5 ± 0.4 | < 0.01 | MORBO | +4.1 ± 1.5 | 0.012 | ||
| Truss design (Affine) [21] | LogEHVI | +0.8 ± 0.1 | < 0.01 | Power flow (Affine) [22] | Sp-STCH | +0.2 ± 0.2 | 0.17 |
| STCH | +0.7 ± 0.3 | 0.022 | MORBO | +1.1 ± 0.2 | < 0.01 | ||
| Nanoparticle design [23] | LogEHVI | +0.2 ± 0.2 | 0.17 | ||||
| STCH | +0.3 ± 0.4 | 0.24 | |||||
| Reaction optimization [24] | LogEHVI | +0.2 ± 0.2 | 0.17 | ||||
| STCH | −0.4 ± 0.9 | 0.67 |
Table 2. Scientific-design experiments. Final hypervolume gains (%, mean ± SE) for Composite relative to BlackBox-GP over 10 paired trials per benchmark (80 for Synthetic 1). Positive values favor Composite. Bold marks unadjusted (one-sided paired -tests for improvement). Affine marks affine outer mappings.
Table 2 shows that Composite improved mean final hypervolume in 15 of 16 comparisons, with the largest gains on color formulation (8.7–13.8%) and Synthetic 1 (7.5–12.7%). Benefits extend to high-dimensional problems: Synthetic 2 (3.6–5.3%) and lake management (4.1–8.7%). Composite provides statistically significant gains on the two affine benchmarks. This might seem to contradict Proposition 1: for affine , BlackBox-Induced also places a GP prior on , yet observing intermediates provides no additional predictive information under that shared prior. However, the GP priors used by BlackBox-Induced and BlackBox-GP need not coincide. In our implementation, each intermediate has its own fitted kernel, so the proposition’s separable-covariance condition generally fails. Combining the kernels fitted to individual intermediates induces a richer kernel family over than BlackBox-GP allows, potentially explaining these gains. Nanoparticle design and reaction optimization show small gains despite non-affine, non-bijective mappings, so these properties alone do not explain performance. Both have correlated intermediates that we model with independent GPs; modeling these dependencies may improve performance. Synthetic 2 also shows modest gains despite non-affine mappings. Limited data in this high-dimensional problem may hinder both approaches. Independently varying input dimension and evaluation budget could clarify their effects.
Conclusion
We investigate when modeling intermediate responses improves MOBO through theory and experiments. Under a shared prior, intermediate observations add no predictive information when the outer mapping is bijective or affine with separable covariance. In the controlled experiments, large single-objective regret reductions did not translate into comparably large hypervolume gains. In scientific-design benchmarks, Composite improved mean final hypervolume in almost all cases, with the largest gains on problems with non-affine, non-bijective outer mappings, although these properties alone do not ensure improvement. Further work should examine how modeling choices such as kernels and dependencies among responses, and input dimension relative to evaluation budget jointly determine these gains.
Acknowledgement
We would like to thank the anonymous reviewers for their constructive feedback and suggestions. This material is based on work supported by the AI Research Institutes program supported by the NSF and Intel Corporation under NSF award DMR-2433348. PB gratefully acknowledges financial support from the Development and Promotion of Science and Technology Talents Project (DPST) scholarship program of the Institute for the Promotion of Teaching Science and Technology (IPST), Ministry of Education, Thailand.
Appendix
Induced Prior and Inference for BlackBox-Induced
At fixed evaluation locations , let , and . BlackBox-Induced and Composite share the vector-valued GP prior , where is the mean vector and is the matrix-valued covariance kernel. This prior induces a prior on . BlackBox-Induced conditions only on the objective observations . Its posterior conditions on . For bijective , this reduces to standard GP conditioning on . For general non-bijective , inference integrates over intermediate responses consistent with and need not have a closed form. One numerical approach is to repeatedly draw the latent intermediate responses from their conditional distribution given under the shared GP prior. For each draw of , condition the GP on those responses, sample at a new input , and apply to obtain a posterior predictive sample of . Averaging these samples estimates the posterior mean .
For affine , the induced prior of is a GP with mean and kernel . Inference therefore uses standard GP conditioning directly on . If the intermediate responses are modeled as independent GPs, then is diagonal. Its th diagonal entry, , is the covariance kernel of the th intermediate response . The induced kernel of therefore simplifies to
A sum of Matérn or RBF kernels with different lengthscales is generally not a single kernel of the same family. Thus, separately fitted BlackBox-GP and BlackBox-Induced priors need not coincide.
Proof of Proposition 1
Throughout, the evaluation locations are fixed, and and contain noiseless observations. Both conditional distributions are computed under a single prior on , with ; in particular, the prior on is the one induced by this construction.
For the bijective case, we additionally assume that is measurable and bijective on a measurable set containing the intermediate responses with probability one, and that its inverse on the corresponding image is measurable. For the affine case, the mean function and covariance parameters of the GP prior on are fixed and known.
Proof for the bijective case. Since is measurable, the objective observations
are a measurable function of . Conversely, since is bijective with measurable inverse,
is a measurable function of . Thus, each dataset can be recovered from the other, and neither provides additional information when the other is known.
Consequently, under the same prior on , for every and ,
Hence the conditional CDFs coincide.
Proof for the affine case. Suppose and has a GP prior with mean and separable covariance
where is a scalar covariance kernel and is a fixed positive semidefinite matrix. Then is a GP with mean and covariance
If , is deterministic under the prior, so the result holds immediately. Assume henceforth that .
Fix a prediction location and define
Write for the Moore–Penrose pseudoinverse of , which equals when is nonsingular.
Conditioning the vector-valued GP on gives
Applying therefore yields a Gaussian conditional distribution for with mean and variance
where .
Alternatively, conditioning the scalar GP directly on gives
using .
Both conditional distributions are Gaussian with identical means and variances. Hence, for every ,
Details of the construction of h function in controlled experiments
In the controlled experiments, for each , we randomly generate using 1024 random Fourier features approximating a Matern-5/2 GP with lengthscale 0.5. For each problem, this function is randomly generated four times.
Comparison metrics
Single-objective metric
Let be the optimum, computed numerically for every controlled mapping, and let
be the simple regret of variant in paired trial within evaluation budget . Since we minimize, smaller values indicate better performance. Regret varies over many orders of magnitude, so we average it on the scale:
where is the BlackBox-GP or Composite column of Table E.1. We measure final performance by
Positive reductions favor Composite.
Multi-objective metric
Let be the set of nondominated objective vectors observed so far for a minimization problem. A vector dominates if for every and the inequality is strict for at least one objective. Given a fixed reference point worse than every vector in , the dominated hypervolume (HV) is
where denotes -dimensional volume. Larger HV indicates better objective values or broader coverage of favorable tradeoffs. When an observed vector is not componentwise better than the reference point, it contributes no volume. We use the same fixed reference point for all methods and trials in a given setting. Let denote the dominated hypervolume after total evaluations for variant in paired trial . The mean HV across trials is
We measure final performance by the relative improvement in mean HV at the total evaluation budget :
Significance
The two arms share the function draw, the initial design, the acquisition rule and the evaluation budget, so runs are paired. All reported -values are one-sided paired -tests for a positive mean gain, computed on the per-run differences and . All tables report mean one standard error over the paired runs. Eq. (D.1) and Eq. (D.3) are functions of the arm means rather than averages of per-run percentages, so their standard errors are obtained from the standard error of the mean paired difference by the delta method.
Additional results for controlled experiments
Table 1 of the main text reports six representative outer mappings. This appendix gives every mapping we considered, under the metrics of Appendix D. Table E.1 reports results for the single-objective cases while Table E.2 reports results for multi-objective (two-objective) cases. Both settings use four function draws and five initial designs per draw, giving 20 paired runs per problem, and within each comparison the two methods (BlackBox-GP and Composite) share the initial design, the acquisition rule and the evaluation budget.
Single objective. Of the five mappings that are bijective on the range attains, none reaches significance; of the eleven that are not, every one does, with throughout. Bijectivity therefore separates the two groups perfectly in this setting.
Percentage reductions near 100% can obscure substantial differences in final regret. For the two-root quadratic and four-root quartic, reductions of 99.96% and 99.53% correspond to approximately 2600-fold and 210-fold reductions relative to BlackBox-GP, respectively. Table E.1 therefore also reports gains in regret to distinguish improvements of different orders of magnitude. The 100% ceiling also explains the narrow standard errors on the saturated rows.
Multi-objective (two objectives). The pattern transfers only partially. None of the five bijective mappings reaches significance, as in the single-objective case, but only five of the fourteen non-bijective settings do, and the effect sizes are percent-level rather than orders of magnitude. The ordering also fails to carry over: the three- and five-root polynomials give the two weakest single-objective gains but both yield significant hypervolume gains, including the largest, while the four-root quartic and the shifted squares at and all exceed +99.5% in the single-objective grid yet fall below +0.5% here. Hypervolume optimization requires discovering and refining a set of tradeoffs rather than locating a single optimum, and under a fixed budget that additional difficulty appears to dilute the benefit of modelling intermediates.
| Outer mapping | BlackBox-GP | Composite | Reduction | Final-RG reduction (%) | |
| Bijective on the range attains | |||||
| Exponential | −2.59 ± 0.09 | −2.74 ± 0.14 | +0.15 ± 0.12 | +28.91 ± 19.01 | 0.11 |
| Identity | −1.03 ± 0.18 | −1.11 ± 0.14 | +0.09 ± 0.16 | +18.10 ± 29.24 | 0.29 |
| Cubic | +0.09 ± 0.11 | +0.01 ± 0.10 | +0.08 ± 0.11 | +17.10 ± 21.07 | 0.23 |
| Hyperbolic sine | +0.24 ± 0.11 | +0.16 ± 0.09 | +0.08 ± 0.10 | +17.00 ± 19.58 | 0.22 |
| Quintic | +0.75 ± 0.15 | +0.79 ± 0.08 | −0.05 ± 0.16 | −11.62 ± 40.30 | 0.62 |
| Not bijective | |||||
| Two-root quadratic | −4.97 ± 0.28 | −8.39 ± 0.30 | +3.42 ± 0.41 | +99.96 ± 0.04 | <0.01 |
| Shifted square | −5.36 ± 0.24 | −8.22 ± 0.24 | +2.86 ± 0.30 | +99.86 ± 0.09 | <0.01 |
| Square | −5.14 ± 0.25 | −7.95 ± 0.26 | +2.81 ± 0.40 | +99.84 ± 0.14 | <0.01 |
| Shifted square | −5.62 ± 0.22 | −8.40 ± 0.28 | +2.78 ± 0.35 | +99.83 ± 0.13 | <0.01 |
| Shifted square | −5.40 ± 0.28 | −7.81 ± 0.14 | +2.41 ± 0.25 | +99.61 ± 0.22 | <0.01 |
| Shifted square | −4.69 ± 0.36 | −7.05 ± 0.49 | +2.36 ± 0.46 | +99.56 ± 0.46 | <0.01 |
| Four-root quartic | −4.80 ± 0.21 | −7.13 ± 0.17 | +2.33 ± 0.20 | +99.53 ± 0.22 | <0.01 |
| Cosine | −4.36 ± 0.41 | −5.79 ± 0.43 | +1.43 ± 0.48 | +96.31 ± 4.06 | <0.01 |
| Cosine | −4.94 ± 0.32 | −6.00 ± 0.20 | +1.06 ± 0.35 | +91.27 ± 7.08 | <0.01 |
| Five-root quintic | +1.69 ± 0.04 | +1.27 ± 0.16 | +0.42 ± 0.14 | +62.36 ± 12.33 | <0.01 |
| Three-root cubic | +0.54 ± 0.09 | +0.28 ± 0.11 | +0.26 ± 0.13 | +45.32 ± 16.44 | 0.03 |
Table E.1. Controlled single-objective experiments, all 16 outer mappings. Values in BlackBox-GP and Composite columns are the final simple regret from each method; Reduction is their paired difference, so positive favors Composite. Final-RG reduction is that same difference as a percentage, via Eq. (D.1). All values are means ± one standard error over the 20 paired runs, the percentage column by the delta method, and is a one-sided paired -test for positive mean reduction. Bold marks significance at the unadjusted 0.05 level.
Table E.2: Controlled two-objective experiments, all 19 distinct settings of . Values in BlackBox-GP and Composite columns are the final hypervolume; Gain is their paired difference, so positive favors Composite. Final-HV gain is that same difference as a percentage, via Eq. (D.3). All values are means one standard error over the 20 paired runs, the percentage column by the delta method, and is a one-sided paired -test for positive mean gain. The BlackBox-GP and Composite columns have the largest standard errors because each mapping pools four function draws whose hypervolumes differ substantially; that spread cancels in the paired Gain. Bold marks significance at the unadjusted 0.05 level.
| Outer mapping | BlackBox-GP | Composite | Gain | Final-HV gain (%) | |
| Bijective on the range attains | |||||
| Cubic | 0.29 | ||||
| Exponential | 0.33 | ||||
| Quintic | 0.43 | ||||
| Identity | 0.40 | ||||
| Hyperbolic sine | 0.69 | ||||
| Not bijective | |||||
| Five-root quintic | 0.01 | ||||
| Square | |||||
| Shifted square | 0.01 | ||||
| Three-root cubic | 0.03 | ||||
| Two-root quadratic | 0.06 | ||||
| Cosine | 0.04 | ||||
| Shifted square | 0.12 | ||||
| Three-root, spread 0.5 | 0.18 | ||||
| Four-root, spread 0.5 | 0.20 | ||||
| Shifted square | 0.30 | ||||
| Shifted square | 0.36 | ||||
| Six-root, spread 1.0 | 0.48 | ||||
| Four-root quartic | 0.51 | ||||
| Cosine | 0.95 |
Table E.2.
Benchmark definitions
This section specifies the exact deterministic oracle used in the experiments. All design vectors below are normalized to , and all final objectives are minimized. Here is the input dimension, is the number of scalar intermediate responses retained by the implementation, and is the number of final objectives.
Multi-illuminant color formulation: , ,
This analytical simulator represents six hypothetical dye loadings in an opaque textile coating. It is motivated by multi-objective color formulation under multiple illuminants [5], but is not a measured dataset and does not reproduce that paper’s objective equations. Wavelengths are nm. Define the Gaussian spectral bump of center and width
The substrate absorption-to-scattering ratio is . The six fixed dye increments are
For design , the dye concentrations and total absorption-to-scattering ratio are
The opaque Kubelka–Munk inverse [27] gives
Let denote the analytic CIE 1931 color-matching functions of Wyman et al. [28], and write nm. For meters, define the normalized blackbody spectrum
where m K. For illuminant , with K and K, define
The tristimulus coordinates and perfect-reflector white point are
The corresponding CIE Lab coordinates are
where and
The objective-specific intermediate responses are
These contain six distinct retained scalar responses because the daylight coordinates are shared.
The unattainable target pigment has
The target ratio is converted to reflectance using Eq. (F.4), then to illuminant-specific XYZ and CIE Lab coordinates using the equations above. The resulting target vectors are for . For a generic argument , define
The objectives are . Thus is daylight color error and is a metameric-mismatch proxy: it penalizes how the sample-minus-target Lab error changes between illuminants. The nonlinear norms erase the direction of the six-dimensional color errors. The hypervolume reference is .
DTLZ2 (Synthetic 1): , ,
The synthetic two-objective DTLZ2 problem [19] is
The repeated deliberately gives each objective an independently modeled component group, avoiding implicit multi-task transfer. The Pareto set has for and the front satisfies , . For reference point the exact maximum hypervolume is .
Four-bar truss RE21: , ,
Four bars are joined into a frame carrying a load at one joint, and the design chooses how thick each bar is. Thicker bars sag less but use more material, giving the problem a tradeoff. This problem follows the RE21 formulation in the real-world multi-objective suite [21]. The four cross-sectional areas are
in , where each entry of sets that bar’s area as a fraction of the way between its minimum and maximum. The lower bounds are the thinnest each bar can be without exceeding the material’s stress limit of under the load , so every design in the box is already safe. Each bar is long and the material’s stiffness (elastic modulus) is . The intermediate responses are
Each entry corresponds to a bar’s contribution to total material in , and to the sag at the loaded joint in . Raw volume and loaded-joint displacement are
They are affinely normalized using exact extrema over the box:
The final objectives are . The reference is . Unlike the color and DTLZ2 reductions, the outer map is essentially a signed sum plus affine scaling, consistent with the small, though significant, gains in Table 2.
Latent Langermann–Ackley (Synthetic 2): , ,
Let have fixed orthonormal rows generated once with seed 18,271, and define the dense latent vector
The intermediate responses are objective-specific copies, . Thus the implementation retains scalar responses, which prevents its independent GP implementation from sharing the six latent observations across objectives. For centers
define and . The two known mappings are
The objectives are . is a bounded Langermann-style landscape and is normalized Ackley. Their optima conflict: Ackley is minimized at , whereas the Langermann centers are displaced. The reference point is .
Shallow-lake management: , ,
A town releases phosphorus into a nearby lake each year for 100 years. Releasing more is economically useful, but phosphorus builds up in the lake, and past a threshold the lake flips into a polluted state that is very hard to reverse. The design chooses the 100 yearly release amounts. Each in controls annual phosphorus release over 100 years. The lake is simulated under 100 fixed common-random-number inflow scenarios (). For scenario , writing for the phosphorus in the lake water at year , we have and for ,
where was drawn once with seed 20,240,820. The first term is the carryover of phosphorus from last year, the third term is the amount the town chose to release last year, and the fourth is uncontrolled inflow from nature. The second term accounts for phosphorus that settled into the lake bed and later returns to the water. Below a critical level the lake clears itself faster than this recycling refills it; above that level it does not, and pollution keeps rising even if releases stop. That threshold in the lake’s phosphorus level is the tipping point. Fixing scenarios makes the oracle deterministic while retaining scenario-averaged variability in the physical response. Let . The first intermediate response contains 20 five-year block peaks,
The second intermediate response is normalized discounted utility,
Thus and . The known mappings, minimized objectives, and reference point are
This is a deterministic two-objective slice of the shallow-lake dynamics of Carpenter et al. [29] as used in multi-objective robustness studies [20]; it is not the original four-objective stochastic benchmark. The last decision contributes to utility but, following the implemented indexing, not to the 100 recorded lake states. The composite representation retains which five-year period controls water quality before the outer maximum discards that information.
RGB-selective multilayer nanoparticle: , ,
Light partially reflects at every boundary between two different materials, so a particle whose layers alternate between two materials produces several reflections that interfere, reinforcing at some wavelengths and canceling at others. Choosing the layer thicknesses therefore chooses which colors the particle scatters, and the goal is a particle that scatters one target color band and little else. This electromagnetic design problem chooses a core radius and five concentric shell thicknesses. For normalized , the implemented dimensions are nm. The layers alternate between silica, with relative permittivity 2.04, and a wavelength-dependent titanium-dioxide model. For wavelength measured in nanometers,
This formula returns the relative permittivity of titanium dioxide at wavelength , which increases toward the blue end of the spectrum. The surrounding medium has relative permittivity 1.77. The simulator evaluates the concentric-sphere Mie calculation for orders 1–12 and both polarizations on the discrete wavelength grid
Mie theory gives the exact scattering from a layered sphere as an infinite series, which the implementation truncates at twelve orders. Given the layer thicknesses , the layer and medium permittivities, and the wavelength grid , the simulator returns a scattering value at each wavelength. The design enters through the layer radii , which determine the interference conditions at each interface. The resulting scattering spectrum therefore depends on the design through the interface radii and on the material properties through . Specifically, for silica, , and represents the surrounding medium; is the boundary between regions and . It is normalized by the particle cross-sectional area . For brevity, we denote this spectrum by below. For target bands , , and nm, the implemented in-band and out-of-band responses are the discrete sums
Here is the scattering that lands inside the target band and is everything else, so the objective below is the fraction of the particle’s scattering that misses the target band, and smaller values mean a more color-selective particle. The intermediate response, known mapping, and minimized objective for band are
The particle parameterization and scattering simulator follow Kim et al. [23]; the three RGB-selectivity objectives are constructed in this work. When is applied to GP posterior samples, the implementation clamps both components to be nonnegative and floors their sum at . Each objective therefore owns a disjoint component pair. The ratio is nonlinear and scale-invariant, but retaining the three doubles the independent output models from three to six without exposing a component shared across objectives. The reference point is .
Summit SNAr reaction: , ,
This case implements the mechanistic reaction benchmark distributed with Summit [24]. The starting material is a molecule carrying two replaceable fluorine atoms, and the reagent pyrrolidine displaces one of them. Because either fluorine can be replaced, the reaction yields the desired product, a variant in which the substitution occurred at the other position, and—if a second pyrrolidine reacts—a byproduct with both fluorines replaced.
Four normalized decisions set how long the mixture stays in the reactor, min; how much pyrrolidine is supplied per unit of starting material, moles per mole; the concentration of starting material entering the reactor, mol L; and the temperature, °C.
The state holds the molar concentrations of starting material, pyrrolidine, product, variant, and byproduct, with molar masses g mol. From , a fixed 256-step RK4 solve integrates the five-species plug-flow model (a tube reactor without back-mixing) over the residence time. The four rate constants follow the Arrhenius law, , with activation energy in kJ mol, gas constant kJ mol K, , and K; the pairs for , , , are , , , and . With
the concentration derivatives are
Let mL min be the flow through the 5 mL reactor, and let denote the outlet concentrations produced by the RK4 integration. The retained intermediate responses are
Thus product concentration and flow occur in both intermediate groups and are modeled twice. For , the first known mapping is one minus normalized space-time yield,
For , define
where is the outlet waste-mass concentration. The E-factor is
The final objectives are . The implementation clamps modeled concentrations to nonnegative values, floors modeled flow at , and floors the E-factor denominator at before applying these mappings.
The composition is nonlinear and mechanistic but offers the surrogate little: both outer maps are smooth over the attained states, product concentration and flow are modeled twice, and flow does not affect at all. The reference point is .
RCM40 power flow: , ,
RCM40 is the two-objective optimal-power-flow problem from the real-world constrained multi-objective suite [22]. It involves setting the voltages on an electrical grid so that both wasted power and reactive loading are minimized. We only use the implemented hypercube constraints and not the original inequality constraints.
The grid is the standard IEEE-14 test network; its 14 nodes are called buses. The optimizer receives . Its first 26 coordinates are mapped affinely to and used as the real and imaginary parts of the voltages at buses 2–14. Bus 1 is the slack bus and is fixed at . The final eight coordinates are not read by the implemented oracle, leaving 26 active dimensions.
The admittance matrix encoding how strongly each pair of buses is electrically connected, is known in advance. The real matrices and contain the conductive and reactive parts of that connection. Multiplication yields the per-bus current and power vectors from the voltage vector :
where denotes entrywise multiplication, denotes complex conjugation, and and return the real and imaginary parts. Thus and are the real- and reactive-power injections at the buses in per-unit coordinates.
The intermediate quantities we retain and their known maps are
Here sums the real-power injections, giving total real-power loss. The second mapping gives ; the retained value receives zero weight. Both mappings are linear with constant Jacobians. Their coefficients are all one for and are , , or for . The reference point is .
Considered MOBO methods
For objectives , let denote the evaluated designs, their objective observations, and the retained scalar intermediate responses, collected in . The two modeling approaches differ in the posterior samples supplied to the acquisition rule. BlackBox-GP fits independent GPs to the final objectives, giving samples
Composite instead fits independent GPs to the retained scalar intermediate responses and propagates their posterior samples through the known mappings:
Here and denote the corresponding observed output columns, and collects the responses used by . Shared or duplicated responses are retained as specified in Appendix § 13. The induced Composite objective posterior need not be Gaussian. Both variants use the same acquisition rule and observed objectives to measure improvement; MORBO applies these modeling choices locally within trust regions. All definitions below use the minimization convention.
Log Expected Hypervolume Improvement (LogEHVI)
Expected hypervolume improvement (EHVI) [17] selects designs according to their expected contribution to the hypervolume of the observed Pareto front. Let denote the nondominated objective vectors observed so far and the fixed reference point. For our minimization problems, the hypervolume improvement from evaluating is
EHVI averages this improvement over the posterior distribution of . Its logarithmic counterpart targets
Taking the logarithm preserves the maximizers whenever EHVI is positive. However, directly computing EHVI and then taking its logarithm can suffer from numerical underflow and vanishing gradients. We therefore use BoTorch’s qLogEHVI implementation [16], which evaluates a numerically stable, smooth approximation in log space. We set , selecting one design at each iteration.
Smooth Tchebycheff scalarization (STCH)
Smooth Tchebycheff scalarization combines the objectives into the single scalar objective , where specifies the tradeoff among the objectives. For weights with , reference vector , and temperature , define
Let be the smallest scalarized value observed in the branch for weight . Its LogEI acquisition targets
where and the expectation is over the BlackBox-GP or Composite objective posterior. Maximizing this acquisition selects the next design for that weight. All STCH variants use temperature and five fixed weights. For two objectives, is equally spaced from 0.05 to 0.95 and ; for more objectives, deterministic normalized exponential draws cover the simplex. The five scalarizations with different weights share one Sobol initial design, evaluated only once, and then evolve independently. The BlackBox-GP STCH variant applies the negative of the scalarization in Eq. (G.5), , to each objective posterior sample . The Composite STCH variant first constructs and then applies the same . Both variants maximize qLogEI () [16] relative to the best scalarized value observed for that weight vector. Low-dimensional branches receive eight adaptive evaluations each, while the high-dimensional STCH branches receive twenty. For reporting only, their points are interleaved in round-robin order so the HV trace does not depend on an arbitrary ordering of the five weights.
Spherical-linear STCH (Sp-STCH)
Sp-STCH retains the STCH weights, independent branches, composition, and qLogEI steps above, but replaces every standard output GP with the following spherical-linear model. For each output, the implementation centers at , divides coordinates by learned ARD scales, and applies a learned global radial scale before constructing
with the zero-radius direction defined as zero. A softmax constrains and . The GP has a constant mean, a learned Gaussian likelihood with a log-normal noise prior, and standardized outputs, following the original spherical-linear implementation [25]. Posterior objective samples from these models are scalarized using Eq. (G.5), and the next design maximizes the LogEI acquisition in Eq. (G.6). Thus, only the surrogate changes relative to STCH.
Multi-Objective Regional Bayesian Optimization (MORBO)
MORBO maintains multiple adaptive trust regions and fits local GP models using observations near each region. For a candidate pool , a Thompson sample assigns a sampled objective vector to each candidate. The sampled hypervolume improvement relative to a current nondominated set is
Coordinated batch selection uses these sampled improvements across regions, accounting for candidates already selected for the batch. Successful regions expand, while unsuccessful regions shrink or restart.
The BlackBox-GP MORBO variant fits local GPs to final objectives, whereas the Composite MORBO variant fits local GPs to the retained scalar entries of and maps each posterior sample through before scoring candidate hypervolume improvement. Both variants begin at the same points and use the same region state, shared past observations, and batch size. There are five trust regions, each initialized with side length 0.8, minimum 0.01, and maximum 1.6. Candidate pools contain 512 raw points. Each local output GP uses a Matérn- ARD kernel with length scales constrained to , output scale, and a Gaussian likelihood with a Gamma noise prior. Candidates are then selected via lower-tail winsorization and coordinated Thompson Sampling/LogEHVI.
Optimization configurations
Protocol. Each matched BlackBox-GP/Composite pair on the benchmarks uses the same trial seed and an identical scrambled-Sobol initial design, and every benchmark comparison is repeated over 10 paired trials, with 80 trials for Synthetic 1. One evaluation at a design returns all objectives, or all retained components, at once; applying the known composition to those components does not consume a further evaluation. Hypervolume is recomputed after each recorded design against the benchmark’s fixed reference point.
Models. All models are double-precision BoTorch and GPyTorch GPs [30]. Standard and spherical models are fit with fit_gpytorch_mll using exact or summed marginal log likelihoods; the configured kernel and likelihood priors enter that same fitting objective.
Surrogate models. The LogEHVI and STCH output GPs use a scaled Matérn 5/2 kernel with ARD in the controlled experiments and a dimension-scaled RBF kernel with ARD in the scientific-design benchmarks. Sp-STCH and MORBO set their covariance explicitly, as Appendices § 14.§ 3 and § 14.§ 4 describe. Table H.1 gives both configurations; absolute gains should be compared within a setting rather than across the two.
| Controlled experiments | Scientific-design benchmarks | |
| BoTorch | 0.9.5 | 0.18.0 |
| GPyTorch | 1.11 | 1.15.2 |
| PyTorch | 2.11.0 | 2.12.0 |
| Covariance | ScaleKernel(Matern 5/2), ARD | RBFKernel, ARD, no outputscale |
| Lengthscale prior | ||
| Lengthscale constraint | — | |
| Outputscale prior | — |
Table H.1. Surrogate and software configuration for the LogEHVI and STCH output GPs in each setting. Both fit hyperparameters by fit_gpytorch_mll in double precision with a Standardize outcome transform.
Evaluation budgets. Table H.2 gives the budget for each regime. The scalarization methods spend their adaptive budget across five fixed weight vectors rather than in one sequence, which is why the per-weight counts multiply out to the same totals as the sequential methods.
| Regime | Method | Initial | Adaptive | Adaptive budget spent as |
| Low-dimensional | LogEHVI | 5 | 40 | sequential, one candidate per step |
| Low-dimensional | STCH | 5 | 40 | 5 weights 8 evaluations |
| High-dimensional | Sp-STCH | 20 | 100 | 5 weights 20 evaluations |
| High-dimensional | MORBO | 20 | 100 | 20 batches 5 candidates |
Table H.2. Initial design size and adaptive evaluation budget per trial. The high-dimensional regime covers L–A, Lake, and RCM40.
Acquisition optimization. LogEHVI, STCH, and Sp-STCH maximize their acquisition function with BoTorch’s optimize_acqf, which seeds a multistart search from raw Sobol candidates and refines it with SciPy’s L-BFGS-B [30]. LogEHVI and STCH use 128 raw candidates with 8 restarts, and Sp-STCH uses 256 raw candidates with 10 restarts. MORBO does not use L-BFGS-B: it sequentially selects twenty batches of five candidates from a discrete Sobol pool of 512 points drawn inside its trust regions.
References
- [1]Abhijith M Gopakumar, Prasanna V Balachandran, Dezhen Xue, James E Gubernatis, and Turab Lookman. Multi-objective optimization for materials discovery via adaptive design. Scientific reports, 8(1):3738, 2018.DOI
- [2]Joshua Knowles. Parego: A hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems. IEEE transactions on evolutionary computation, 10(1):50–66, 2006.DOI
- [3]Carl Edward Rasmussen and Christopher KI Williams. Gaussian processes for machine learning. Number 3. MIT press Cambridge, MA, 2006.DOI
- [4]Tobias Ginsburg, Kyle Hippe, Ryan Lewis, Aileen Cleary, Doga Ozgulbas, Rory Butler, Casey Stone, Abraham Stroka, Rafael Vescovi, and Ian Foster. Exploring benchmarks for self-driving labs using color matching. In Proceedings of the SC’23 Workshops of the International Conference on High Performance Computing, Network, Storage, and Analysis, pages 2147–2152, 2023.arxiv.org/abs/2310.00510
- [5]Sabrine Chaouch, Ali Moussa, and Neji Ladhari. Color formulation of cotton fabrics using multi-objective ant colony optimization. Journal of Natural Fibers, 19(17):1549–1547, 2022.DOI
- [6]Raul Astudillo and Peter Frazier. Bayesian optimization of composite functions. In International Conference on Machine Learning, pages 354–363. PMLR, 2019.arxiv.org/abs/1906.01537
- [7]Wesley J Maddox, Maximilian Balandat, Andrew G Wilson, and Eytan Bakshy. Bayesian optimization with high-dimensional outputs. Advances in neural information processing systems, 34:19274–19287, 2021.arxiv.org/abs/2106.12997
- [8]Wesley Maddox, Qing Feng, and Max Balandat. Optimizing high-dimensional physics simulations via composite bayesian optimization. arXiv preprint arXiv:2111.14911, 2021.arxiv.org/abs/2111.14911
- [9]Raul Astudillo and Peter Frazier. Bayesian optimization of function networks. Advances in neural information processing systems, 34:14463–14475, 2021.arxiv.org/abs/2112.15311
- [10]P. Buathong, J. Wan, R. Astudillo, S. Daulton, M. Balandat, and P. I. Frazier. Bayesian optimization of function networks with partial evaluations. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 4752–4784. PMLR, 2024.arxiv.org/abs/2311.02146
- [11]Xi Lin, Xiaoyuan Zhang, Zhiyuan Yang, Fei Liu, Zhenkun Wang, and Qingfu Zhang. Smooth tchebycheff scalarization for multi-objective optimization. arXiv preprint arXiv:2402.19078, 2024.arxiv.org/abs/2402.19078
- [12]TM Nogueira Pires, RP Cardoso Coelho, and FM Andrade Pires. Composite bayesian optimisation for multi-objective problems with smooth tchebycheff scalarisation. Structural and Multidisciplinary Optimization, 69(2):34, 2026.DOI
- [13]Donald R Jones, Matthias Schonlau, and William J Welch. Efficient global optimization of expensive black-box functions. Journal of Global optimization, 13(4):455–492, 1998.DOI
- [14]Peter I Frazier. Bayesian optimization. In Recent advances in optimization and modeling of contemporary problems, pages 255–278. Informs, 2018.DOI
- [15]Edward Snelson, Zoubin Ghahramani, and Carl Rasmussen. Warped gaussian processes. Advances in neural information processing systems, 16, 2003.
- [16]Sebastian Ament, Samuel Daulton, David Eriksson, Maximilian Balandat, and Eytan Bakshy. Unexpected improvements to expected improvement for bayesian optimization. Advances in neural information processing systems, 36:20577–20612, 2023.arxiv.org/abs/2310.20708
- [17]Samuel Daulton, Maximilian Balandat, and Eytan Bakshy. Differentiable expected hypervolume improvement for parallel multi-objective bayesian optimization. Advances in neural information processing systems, 33:9851–9864, 2020.
- [18]S. Surjanovic and D. Bingham. Virtual library of simulation experiments: Test functions and datasets. Retrieved November 8, 2026, from http://www.sfu.ca/~ssurjano.
- [19]Kalyanmoy Deb, Lothar Thiele, Marco Laumanns, and Eckart Zitzler. Scalable multi-objective optimization test problems. In Proceedings of the 2002 congress on evolutionary computation. CEC’02 (Cat. No. 02TH8600), volume 1, pages 825–830. IEEE, 2002.DOI
- [20]Victoria I Ward, Riddhi Singh, Patrick M Reed, and Klaus Keller. Confronting tipping points: Can multi-objective evolutionary algorithms discover pollution control tradeoffs given environmental thresholds? Environmental Modelling & Software, 73:27–43, 2015.DOI
- [21]Ryoji Tanabe and Hisao Ishibuchi. An easy-to-use real-world multi-objective optimization problem suite. Applied Soft Computing, 89:106078, 2020.DOI
- [22]Abhishek Kumar, Guohua Wu, Mostafa Z Ali, Qizhang Luo, Rammohan Mallipeddi, Ponnuthu-rai Nagaratnam Suganthan, and Swagatam Das. A benchmark-suite of real-world constrained multi-objective optimization problems and some baseline results. Swarm and Evolutionary Computation, 67:100961, 2021.DOI
- [23]Samuel Kim, Peter Y Lu, Charlotte Loh, Jamie Smith, Jasper Snoek, and Marin Soljačić. Deep learning for bayesian optimization of scientific problems with high-dimensional structure. arXiv preprint arXiv:2104.11667, 2021.arxiv.org/abs/2104.11667
- [24]Kobi C Felton, Jan G Rittig, and Alexei A Lapkin. Summit: benchmarking machine learning methods for reaction optimisation. Chemistry-Methods, 1(2):116–122, 2021.DOI
- [25]Colin Doumont, Donney Fan, Natalie Maus, Jacob R Gardner, Henry Moss, and Geoff Pleiss. We still don’t understand high-dimensional bayesian optimization. arXiv preprint arXiv:2512.00170, 2025.arxiv.org/abs/2512.00170
- [26]Samuel Daulton, David Eriksson, Maximilian Balandat, and Eytan Bakshy. Multi-objective bayesian optimization over high-dimensional search spaces. In Uncertainty in Artificial Intelligence, pages 507–517. PMLR, 2022.arxiv.org/abs/2109.10964
- [27]Paul Kubelka and Franz Munk. Ein beitrag zur optik der farbanstriche. Z. tech. Phys, 12:593–601, 1931.
- [28]Chris Wyman, Peter-Pike Sloan, and Peter Shirley. Simple analytic approximations to the cie xyz color matching functions. J. Comput. Graph. Tech, 2(2):1–11, 2013.
- [29]Stephen R Carpenter, Donald Ludwig, and William A Brock. Management of eutrophication for lakes subject to potentially irreversible change. Ecological applications, 9(3):751–771, 1999.DOI
- [30]Maximilian Balandat, Brian Karrer, Daniel Jiang, Samuel Daulton, Ben Letham, Andrew G Wilson, and Eytan Bakshy. Botorch: A framework for efficient monte-carlo bayesian optimization. Advances in neural information processing systems, 33:21524–21538, 2020.arxiv.org/abs/1910.06403