betaFeedback

WorksComputer Science

Investigating Composite Modeling in Multi-Objective Bayesian Optimization for Scientific Design

Abstract

Scientific design often requires balancing competing objectives that are expensive to evaluate through experiments or simulations. Multi-objective Bayesian optimization (MOBO) addresses this challenge by using probabilistic models to guide the selection of promising designs. In many applications, tunable inputs map to observable intermediate responses, which are then combined through known functions to compute the final objectives. Prior work on Bayesian optimization of composite functions shows that learning from these intermediate responses, which we call Composite modeling, can substantially improve sampling efficiency in single-objective optimization, compared to methods that condition only on black-box observations of the objective function. The benefits of Composite modeling in MOBO, however, remain understudied. In this work, we study the benefits of Composite modeling in MOBO and investigate the factors driving variation in its gains. We first establish single-objective conditions under which intermediate observations provide no additional predictive information beyond objective values, where composite and black box methods use the same warped Gaussian process prior. Controlled numerical experiments with versions of four MOBO methods across eight synthetic and application-inspired scientific-design benchmarks examine how these conditions relate to optimization gains in the multi-objective setting, where the black box approach models the objective functions with a standard multi-output Gaussian process. Both investigations support the claim that composite modeling tends to help more when the function mapping intermediate responses to final objectives is non-bijective and non-affine.

Introduction

Scientific design often involves balancing competing goals. A material, for example, may need to perform well while also being inexpensive. Evaluating a design can require a costly experiment. We formulate this problem as min⁡x∈Xf(x)=(f1(x),…,fN(x))\min_{x \in\mathcal{X}} f(x) = (f_1(x), \ldots, f_N(x)), where X⊆Rd\mathcal{X} \subseteq\mathbb{R}^{d} is the design space. Multi-objective Bayesian optimization (MOBO) seeks favorable tradeoffs with few evaluations [1, 2], using Gaussian process (GP) models [3] and an acquisition function to select designs.

In many scientific applications, tunable inputs map to observable intermediate responses, which are then combined through known functions to compute the final objectives. These objectives have the composite form fi(x)=gi(hi(x))f_i(\mathbf{x}) = g_i(\mathbf{h}_i(\mathbf{x})) for i=1,…,Ni = 1, \ldots, N where hi:X→Rki\mathbf{h}_i : \mathcal{X} \to\mathbb{R}^{k_i} is an expensive black-box function returning kik_i intermediate responses and gi:Rki→Rg_i : \mathbb{R}^{k_i} \to\mathbb{R} is known and inexpensive to evaluate. For single objectives, we write ff, gg and hh in place of f1f_1, g1g_1 and h1h_1. For example, in color formulation, dye concentrations determine measured color coordinates, which known functions convert into color differences from a target [4]. Matching under different lighting can create competing objectives [5].

Bayesian optimization (BO) of composite functions [6] models intermediate responses and propagates posterior samples through the known mappings. This can substantially improve single-objective sampling efficiency, with extensions to high-dimensional outputs and function networks [7, 8, 9, 10]. Composite modeling has also been applied to MOBO through smooth Tchebycheff scalarization (STCH) [11, 12], but the factors governing its benefits remain understudied.

We investigate the benefits of composite modeling through three inference approaches. Composite places a GP prior on the intermediate responses and conditions on their observations. BlackBox-GP places a GP prior directly on the objectives and conditions on objective observations. BlackBox-Induced uses the objective prior induced by Composite’s prior through the known mappings, but conditions only on objective observations. BlackBox-GP corresponds directly to how BO is usually implemented in practice while BlackBox-Induced supports a precise comparison of the benefits provided by the additional information provided by intermediate responses while holding the prior distribution fixed.

We find evidence that composite modeling tends to perform as well or better than black box approaches and provides the largest benefits when the functions gig_i are non-bijective and non-affine. First, we analyze the single-objective case and show that Composite and BlackBox-Induced create equivalent posterior distributions when gg is bijective or when gg is affine and the prior on hh has separable covariance. This suggests composite approaches are equal to black box ones in these settings. Second, experiments comparing Composite with BlackBox-GP on both single and multi-objective problems show that Composite provides a positive mean final-hypervolume gain in 15 of 16 comparisons and that the problems where Composite provides the biggest benefits are non-affine and non-bijective gig_i.

Problem Setting

Consider objectives fi(x)=gi(hi(x))f_i(\mathbf{x}) = g_i(\mathbf{h}_i(\mathbf{x})), i=1,…,Ni = 1, \ldots, N, where hi:X→Rki\mathbf{h}_i : \mathcal{X} \to\mathbb{R}^{k_i} is unknown and gi:Rki→Rg_i : \mathbb{R}^{k_i} \to\mathbb{R} is known. The intermediate responses may be shared across objectives. We consider both single-objective (N=1N = 1) and multi-objective (N>1N > 1) settings. For the single-objective analysis, we omit the subscript and write f(x)=g(h(x))f(\mathbf{x}) = g(\mathbf{h}(\mathbf{x})).

We consider three ways of performing inference in this setting, one that conditions on observations of h(x)\mathbf{h}(\mathbf{x}) and two that condition only on observations of f(x)f(\mathbf{x}). The first method, called Composite, aligns with composite BO [6]: we put a multi-output GP on h(x)\mathbf{h}(\mathbf{x}) and condition on observations of h(x)\mathbf{h}(\mathbf{x}). The second method, called BlackBox-GP, aligns with classical BO [13, 14]: we put a GP on f(x)f(\mathbf{x}) and condition on observations of f(x)f(\mathbf{x}). This treats f(x)f(\mathbf{x}) as a black box, supposing that we cannot observe the inner function h(x)\mathbf{h}(\mathbf{x}). The third method, called BlackBox-Induced, supports theoretical analysis: we start with the same GP prior on h(x)\mathbf{h}(\mathbf{x}) as Composite, induce a corresponding prior on f(x)f(\mathbf{x}) through the known mapping gg, and condition on observations of f(x)f(\mathbf{x}). Thus, this method uses the same prior as Composite but conditions on the same black box information as BlackBox-GP. This is a form of warped GP prior [15] with a fixed warping function. Appendix A describes how to perform posterior inference under this induced prior.

Below, using theory and experiments, we analyze whether observing h(x)\mathbf{h}(\mathbf{x}) (in the Composite method) provides a benefit over observing only f(x)f(\mathbf{x}) (the two BlackBox methods) and how this depends on the choice of gg. We focus specifically on whether gg is a bijection or is affine, finding support for the claim that conditioning on h(x)\mathbf{h}(\mathbf{x}) does not provide a benefit when gg is a bijection or is affine, and often does provide a benefit when gg is neither bijective nor affine.

Theoretical Analysis

We first compare Composite and BlackBox-Induced in a single-objective case theoretically.

Proposition 1 (Posterior equivalence). At fixed evaluation locations x1,…,xnx_1, \ldots, x_n, let H=(h(x1),…,h(xn))H = (h(x_1), \ldots, h(x_n)), and F=(f(x1),…,f(xn))F = (f(x_1), \ldots, f(x_n)). Suppose hh is drawn from a GP prior with fixed known mean function and kernel. Suppose also that either (1) gg is bijective on the possible intermediate responses, so each objective value uniquely determines its intermediate response; or (2) g(z)=a⊤z+bg(\mathbf{z}) = \mathbf{a}^{\top}\mathbf{z} + b for some fixed a\mathbf{a} and bb, and the GP prior on $h has separable covariance. Then, for every fixed x∈X\mathbf{x} \in\mathcal{X} and t∈Rt \in\mathbb{R}, P(f(x)≤t∣H)=P(f(x)≤t∣F)\mathbb{P}(f(\mathbf{x}) \le t \mid H) = \mathbb{P}(f(\mathbf{x}) \le t \mid F) with probability one.

Technical conditions and the proof are provided in Appendix § 9. Under the Proposition 1, which applies to Identity and Cubic experiments below, Composite and BlackBox-Induced yield the same posterior over ff when they share a GP prior on hh. Consequently, acquisition rules determined solely by the posterior produce the same decisions and final posterior distribution. In these problems, differences observed between Composite and BlackBox-GP are due to the prior distribution, Monte Carlo noise, or small implementation differences, since BlackBox-GP and BlackBox-Induced condition on the same information.

Experimental Analysis

Controlled Synthetic Experiments: We fix scalar intermediate functions h:[0,1]6→Rh : [0, 1]^{6} \to\mathbb{R} and vary the known outer mapping gg (See Appendix § 10 for hh construction). We compare BlackBox-GP and Composite methods, each of which fits its own GP models and share the initial design, acquisition rule, and evaluation budget. Single-objective experiments use LogEI [13, 16] and we report mean percentage of final log⁡10\log_{10} simple regret (Final-RG) reduction for Composite relative to BlackBox-GP with standard error (SE). For multi-objective, we use f1(x)=g(h1(x))f_1(\mathbf{x}) = g(h_1(\mathbf{x})), and f2(x)=h2(x)f_2(\mathbf{x}) = h_2(\mathbf{x}), where h1h_1 and h2h_2 are independent scalar intermediates, modeled with independent GPs. The f2f_2 identity mapping isolates composition in f1f_1. We use LogEHVI [17, 16] and report mean percentage of final hypervolume (Final-HV) gain against a shared reference point for Composite relative to BlackBox-GP with SE. Higher metrics favor Composite. Single-objective starts with 5 initial points and two-objective starts with 10; both then sequentially evaluate 40 additional points based on their acquisition rules. Appendix D details comparison metrics. Four function draws and five initial designs per draw yield 20 paired runs.

Table 1 shows significant single-objective gains for all four selected non-bijective mappings, while for multi-objective cases, only square and three-root cubic yield significant hypervolume gains. Neither selected bijective mapping shows a significant gain in either setting. Square gives the largest gain of the six in both settings, but the two scales are not comparable: its single-objective gain is a 99.8% reduction in final regret, against a hypervolume gain under 3%. Hypervolume optimization requires discovering a set of trade-offs rather than a single optimum, which may limit Composite’s gains under a fixed evaluation budget. Larger-budget experiments could test this explanation.

Outer mapping ggBijective?Single objectiveMulti-objective
Final-RG reduction (%)ppFinal-HV gain (%)pp
IdentityYes+18.1±29.2+18.1 \pm29.20.290.29+0.2±0.9+0.2 \pm0.90.400.40
Cubic h3h^3Yes+17.1±21.1+17.1 \pm21.10.240.24+0.9±1.6+0.9 \pm1.60.290.29
Square h2h^2No+99.8±0.1+99.8 \pm0.1<0.01< 0.01+2.9±1.0+2.9 \pm1.0<0.01< 0.01
Shifted square (h−3)2(h - 3)^2No+99.6±0.5+99.6 \pm0.5<0.01< 0.01+1.0±0.8+1.0 \pm0.80.120.12
Three-root cubicNo+45.3±16.4+45.3 \pm16.40.030.03+2.5±1.2+2.5 \pm1.20.030.03
Cosine cos⁡h\cos hNo+96.3±4.1+96.3 \pm4.1<0.01< 0.01−1.5±0.9-1.5 \pm0.90.950.95

Table 1. Controlled synthetic experiments. Final regret reductions (single-objective) and final hypervolume gains (multi-objective) for Composite relative to BlackBox-GP, both in percentage (%) (mean ±\pm SE over 20 paired runs). Positive values favor Composite. Bold marks unadjusted p<0.05p < 0.05 (one-sided paired tt-tests for improvement).

Scientific-Design Experiments: We compare BlackBox-GP and Composite versions of four MOBO methods on eight benchmarks as listed in Table 2 (see Appendix § 13 for details). All have at least one non-bijective outer mapping; only truss design and power flow have affine outer mappings. Low-dimensional problems use LogEHVI [17, 16] and STCH [11, 12] methods with 5 initial and 40 adaptive evaluations; high-dimensional problems use STCH with a spherical kernel (sp-STCH) [25] and MORBO [26] with 20 initial and 100 adaptive evaluations. Appendices § 14 and § 15 detail methods and implementation. Composite models intermediates with independent GPs. Two methods share the initial design, acquisition rule, reference point, and budget. We report mean percentage gains in Final-HV ± SE over 10 different trials for all problems, except Synthetic 1, which uses 80 trials.

Low-dimensionalHigh-dimensional
BenchmarkMethodFinal-HV gain (%)ppBenchmarkMethodFinal-HV gain (%)pp
Color formulation [5]LogEHVI+13.8 ± 1.6< 0.01Synthetic 2 [18]Sp-STCH+3.6 ± 1.80.038
STCH+8.7 ± 1.7< 0.01MORBO+5.3 ± 2.60.036
Synthetic 1 [19]LogEHVI+12.7 ± 0.6< 0.01Lake management [20]Sp-STCH+8.7 ± 0.7< 0.01
STCH+7.5 ± 0.4< 0.01MORBO+4.1 ± 1.50.012
Truss design (Affine) [21]LogEHVI+0.8 ± 0.1< 0.01Power flow (Affine) [22]Sp-STCH+0.2 ± 0.20.17
STCH+0.7 ± 0.30.022MORBO+1.1 ± 0.2< 0.01
Nanoparticle design [23]LogEHVI+0.2 ± 0.20.17
STCH+0.3 ± 0.40.24
Reaction optimization [24]LogEHVI+0.2 ± 0.20.17
STCH−0.4 ± 0.90.67

Table 2. Scientific-design experiments. Final hypervolume gains (%, mean ± SE) for Composite relative to BlackBox-GP over 10 paired trials per benchmark (80 for Synthetic 1). Positive values favor Composite. Bold marks unadjusted p<0.05p < 0.05 (one-sided paired tt-tests for improvement). Affine marks affine outer mappings.

Table 2 shows that Composite improved mean final hypervolume in 15 of 16 comparisons, with the largest gains on color formulation (8.7–13.8%) and Synthetic 1 (7.5–12.7%). Benefits extend to high-dimensional problems: Synthetic 2 (3.6–5.3%) and lake management (4.1–8.7%). Composite provides statistically significant gains on the two affine benchmarks. This might seem to contradict Proposition 1: for affine gg, BlackBox-Induced also places a GP prior on ff, yet observing intermediates provides no additional predictive information under that shared prior. However, the GP priors used by BlackBox-Induced and BlackBox-GP need not coincide. In our implementation, each intermediate has its own fitted kernel, so the proposition’s separable-covariance condition generally fails. Combining the kernels fitted to individual intermediates induces a richer kernel family over ff than BlackBox-GP allows, potentially explaining these gains. Nanoparticle design and reaction optimization show small gains despite non-affine, non-bijective mappings, so these properties alone do not explain performance. Both have correlated intermediates that we model with independent GPs; modeling these dependencies may improve performance. Synthetic 2 also shows modest gains despite non-affine mappings. Limited data in this high-dimensional problem may hinder both approaches. Independently varying input dimension and evaluation budget could clarify their effects.

Conclusion

We investigate when modeling intermediate responses improves MOBO through theory and experiments. Under a shared prior, intermediate observations add no predictive information when the outer mapping is bijective or affine with separable covariance. In the controlled experiments, large single-objective regret reductions did not translate into comparably large hypervolume gains. In scientific-design benchmarks, Composite improved mean final hypervolume in almost all cases, with the largest gains on problems with non-affine, non-bijective outer mappings, although these properties alone do not ensure improvement. Further work should examine how modeling choices such as kernels and dependencies among responses, and input dimension relative to evaluation budget jointly determine these gains.

Acknowledgement

We would like to thank the anonymous reviewers for their constructive feedback and suggestions. This material is based on work supported by the AI Research Institutes program supported by the NSF and Intel Corporation under NSF award DMR-2433348. PB gratefully acknowledges financial support from the Development and Promotion of Science and Technology Talents Project (DPST) scholarship program of the Institute for the Promotion of Teaching Science and Technology (IPST), Ministry of Education, Thailand.

Appendix

Induced Prior and Inference for BlackBox-Induced

At fixed evaluation locations x1,…,xn\mathbf{x}_1,\ldots,\mathbf{x}_n, let H=(h(x1),…,h(xn))H=(\mathbf{h}(\mathbf{x}_1),\ldots,\mathbf{h}(\mathbf{x}_n)), and F=(f(x1),…,f(xn))F=(f(\mathbf{x}_1),\ldots,f(\mathbf{x}_n)). BlackBox-Induced and Composite share the vector-valued GP prior h∼GP(mh,Kh)\mathbf{h}\sim\mathcal{GP}(\mathbf{m}_h,K_h), where mh(x)=E[h(x)]\mathbf{m}_h(\mathbf{x})=\mathbb{E}[\mathbf{h}(\mathbf{x})] is the mean vector and Kh(x,x′)=Cov⁡(h(x),h(x′))K_h(\mathbf{x},\mathbf{x}')=\operatorname{Cov}(\mathbf{h}(\mathbf{x}),\mathbf{h}(\mathbf{x}')) is the matrix-valued covariance kernel. This prior induces a prior on f=g∘hf=g\circ h. BlackBox-Induced conditions only on the objective observations FF. Its posterior conditions on g(h(xj))=Fjg(\mathbf{h}(\mathbf{x}_j))=F_j. For bijective gg, this reduces to standard GP conditioning on h(xj)=g−1(Fj)\mathbf{h}(\mathbf{x}_j)=g^{-1}(F_j). For general non-bijective gg, inference integrates over intermediate responses consistent with FF and need not have a closed form. One numerical approach is to repeatedly draw the latent intermediate responses HH from their conditional distribution given FF under the shared GP prior. For each draw of HH, condition the GP on those responses, sample h(x∗)\mathbf{h}(\mathbf{x}_*) at a new input x∗\mathbf{x}_*, and apply gg to obtain a posterior predictive sample of f(x∗)f(\mathbf{x}_*). Averaging these samples estimates the posterior mean E[f(x∗)∣F]\mathbb{E}[f(\mathbf{x}_*)\mid F].

For affine g(z)=aTz+bg(\mathbf{z})=\mathbf{a}^{\mathsf{T}}\mathbf{z}+b, the induced prior of ff is a GP with mean aTmh(x)+b\mathbf{a}^{\mathsf{T}}\mathbf{m}_h(\mathbf{x})+b and kernel aTKh(x,x′)a\mathbf{a}^{\mathsf{T}}K_h(\mathbf{x},\mathbf{x}')\mathbf{a}. Inference therefore uses standard GP conditioning directly on FF. If the intermediate responses are modeled as independent GPs, then Kh(x,x′)K_h(\mathbf{x},\mathbf{x}') is diagonal. Its jjth diagonal entry, kj(x,x′)=[Kh(x,x′)]jjk_j(\mathbf{x},\mathbf{x}')=[K_h(\mathbf{x},\mathbf{x}')]_{jj}, is the covariance kernel of the jjth intermediate response hjh_j. The induced kernel of ff therefore simplifies to

aTKh(x,x′)a=∑jaj2kj(x,x′).\mathbf{a}^{\mathsf{T}}K_h(\mathbf{x},\mathbf{x}')\mathbf{a}=\sum_j a_j^2 k_j(\mathbf{x},\mathbf{x}').

A sum of Matérn or RBF kernels with different lengthscales is generally not a single kernel of the same family. Thus, separately fitted BlackBox-GP and BlackBox-Induced priors need not coincide.

Proof of Proposition 1

Throughout, the evaluation locations x1,…,xn\mathbf{x}_1,\ldots,\mathbf{x}_n are fixed, and HH and FF contain noiseless observations. Both conditional distributions are computed under a single prior on hh, with f=g∘hf=g\circ h; in particular, the prior on ff is the one induced by this construction.

For the bijective case, we additionally assume that gg is measurable and bijective on a measurable set containing the intermediate responses with probability one, and that its inverse on the corresponding image is measurable. For the affine case, the mean function and covariance parameters of the GP prior on hh are fixed and known.

Proof for the bijective case. Since gg is measurable, the objective observations

F=(g(h(x1)),…,g(h(xn)))F=\left(g(\mathbf{h}(\mathbf{x}_1)),\ldots,g(\mathbf{h}(\mathbf{x}_n))\right)

are a measurable function of HH. Conversely, since gg is bijective with measurable inverse,

H=(g−1(f(x1)),…,g−1(f(xn)))H=\left(g^{-1}(f(\mathbf{x}_1)),\ldots,g^{-1}(f(\mathbf{x}_n))\right)

is a measurable function of FF. Thus, each dataset can be recovered from the other, and neither provides additional information when the other is known.

Consequently, under the same prior on hh, for every x∈Xx\in\mathcal{X} and t∈Rt\in\mathbb{R},

P(f(x)≤t∣H)=P(f(x)≤t∣H,F)=P(f(x)≤t∣F)almost surely.\begin{aligned} \mathbb{P}(f(x)\leq t\mid H)=\mathbb{P}(f(x)\leq t\mid H,F) \\ &=\mathbb{P}(f(x)\leq t\mid F)\quad\text{almost surely}. \end{aligned}

Hence the conditional CDFs coincide. □\square

Proof for the affine case. Suppose g(z)=aTz+bg(\mathbf{z})=\mathbf{a}^{\mathsf{T}}\mathbf{z}+b and h\mathbf{h} has a GP prior with mean mh(x)\mathbf{m}_h(\mathbf{x}) and separable covariance

Cov⁡(h(x),h(x′))=kh(x,x′)B,\operatorname{Cov}(\mathbf{h}(\mathbf{x}),\mathbf{h}(\mathbf{x}'))=k_h(\mathbf{x},\mathbf{x}')B,

where kk is a scalar covariance kernel and BB is a fixed positive semidefinite matrix. Then ff is a GP with mean mf(x)=aTmh(x)+bm_f(\mathbf{x}) = \mathbf{a}^{\mathsf{T}}m_h(\mathbf{x}) + b and covariance

kf(x,x′)=ckh(x,x′),c=aTBa.k_f(\mathbf{x},\mathbf{x}') = c k_h(\mathbf{x},\mathbf{x}'), \qquad c = \mathbf{a}^{\mathsf{T}}B\mathbf{a}.

If c=0c = 0, ff is deterministic under the prior, so the result holds immediately. Assume henceforth that c>0c > 0.

Fix a prediction location xx and define

(Kh)ij=kh(xi,xj),kh=(kh(x1,x),…,kh(xn,x))T.(K_h)_{ij} = k_h(\mathbf{x}_i,\mathbf{x}_j), \qquad\mathbf{k}_h = (k_h(\mathbf{x}_1,\mathbf{x}),\ldots,k_h(\mathbf{x}_n,\mathbf{x}))^{\mathsf{T}}.

Write Kh†K_h^{\dagger} for the Moore–Penrose pseudoinverse of KhK_h, which equals Kh−1K_h^{-1} when KhK_h is nonsingular.

Conditioning the vector-valued GP on HH gives

E[h(x)∣H]=mh(x)+∑i=1n(Kh†kh)i(h(xi)−mh(xi)),\mathbb{E}[\mathbf{h}(\mathbf{x}) \mid H] = \mathbf{m}_h(\mathbf{x}) + \sum_{i=1}^{n}(K_h^{\dagger}\mathbf{k}_h)_i(\mathbf{h}(\mathbf{x}_i) - \mathbf{m}_h(\mathbf{x}_i)),
Cov⁡(h(x)∣H)=(kh(x,x)−khTKh†kh)B.\operatorname{Cov}(\mathbf{h}(\mathbf{x}) \mid H) = (k_h(\mathbf{x},\mathbf{x}) - \mathbf{k}_h^{\mathsf{T}}K_h^{\dagger}\mathbf{k}_h)B.

Applying gg therefore yields a Gaussian conditional distribution for f(x)f(\mathbf{x}) with mean and variance

μH(x)=mf(x)+khTKh†(F−mf(X)),\mu_H(\mathbf{x}) = m_f(\mathbf{x}) + \mathbf{k}_h^{\mathsf{T}}K_h^{\dagger}(F - m_f(X)),
vH(x)=c(kh(x,x)−khTKh†kh),v_H(\mathbf{x}) = c(k_h(\mathbf{x},\mathbf{x}) - \mathbf{k}_h^{\mathsf{T}}K_h^{\dagger}\mathbf{k}_h),

where mf(X)=(mf(x1),…,mf(xn))Tm_f(X) = (m_f(\mathbf{x}_1),\ldots,m_f(\mathbf{x}_n))^{\mathsf{T}}.

Alternatively, conditioning the scalar GP ff directly on FF gives

μF(x)=mf(x)+(ckh)T(cKh)†(F−mf(X))=μH(x),\mu_F(\mathbf{x}) = m_f(\mathbf{x}) + (c\mathbf{k}_h)^{\mathsf{T}}(cK_h)^{\dagger}(F - m_f(X)) = \mu_H(\mathbf{x}),
vF(x)=ckh(x,x)−(ckh)T(cKh)†(ckh)=vH(x),v_F(\mathbf{x}) = c k_h(\mathbf{x},\mathbf{x}) - (c\mathbf{k}_h)^{\mathsf{T}}(cK_h)^{\dagger}(c\mathbf{k}_h) = v_H(\mathbf{x}),

using (cKh)†=c−1Kh†(cK_h)^{\dagger} = c^{-1}K_h^{\dagger}.

Both conditional distributions are Gaussian with identical means and variances. Hence, for every t∈Rt \in\mathbb{R},

P(f(x)≤t∣H)=P(f(x)≤t∣F)almost surely.\mathbb{P}(f(\mathbf{x}) \leq t \mid H) = \mathbb{P}(f(\mathbf{x}) \leq t \mid F) \quad\text{almost surely}.

Details of the construction of h function in controlled experiments

In the controlled experiments, for each hh, we randomly generate hh using 1024 random Fourier features approximating a Matern-5/2 GP with lengthscale 0.5. For each problem, this hh function is randomly generated four times.

Comparison metrics

Single-objective metric

Let f⋆=min⁡xf(x)f^\star= \min_x f(x) be the optimum, computed numerically for every controlled mapping, and let

rTv,s=(min⁡1≤t≤Tf(xtv,s))−f⋆r_T^{v,s} = \left(\min_{1 \leq t \leq T} f(x_t^{v,s})\right) - f^\star

be the simple regret of variant v∈{BlackBox-GP,Composite}v \in\{\mathrm{BlackBox\text{-}GP}, \mathrm{Composite}\} in paired trial ss within evaluation budget TT. Since we minimize, smaller values indicate better performance. Regret varies over many orders of magnitude, so we average it on the log⁡10\log_{10} scale:

rˉTv=10mTv,mTv=1S∑s=1Slog⁡10rTv,s,\bar{r}_T^v = 10^{m_T^v}, \qquad m_T^v = \frac{1}{S}\sum_{s=1}^{S}\log_{10}r_T^{v,s},

where mTvm_T^v is the BlackBox-GP or Composite column of Table E.1. We measure final performance by

Final-RG reduction⁡(%)=100rˉTBlackBox-GP−rˉTCompositerˉTBlackBox-GP.(1)\operatorname{Final\text{-}RG\ reduction}(\%) = 100 \frac{\bar{r}_T^{\mathrm{BlackBox\text{-}GP}}-\bar{r}_T^{\mathrm{Composite}}}{\bar{r}_T^{\mathrm{BlackBox\text{-}GP}}}. \tag*{(1)}

Positive reductions favor Composite.

Multi-objective metric

Let P⊂RN\mathcal{P} \subset\mathbb{R}^{N} be the set of nondominated objective vectors observed so far for a minimization problem. A vector y\mathbf{y} dominates y′\mathbf{y}' if yi≤yi′y_i \le y_i' for every ii and the inequality is strict for at least one objective. Given a fixed reference point r\mathbf{r} worse than every vector in P\mathcal{P}, the dominated hypervolume (HV) is

HV⁡(P;r)=λN(⋃y∈P∏i=1N[yi,ri]),(2)\operatorname{HV}(\mathcal{P};\mathbf{r}) = \lambda_N \left( \bigcup_{\mathbf{y}\in\mathcal{P}} \prod_{i=1}^{N} [y_i,r_i] \right), \tag*{(2)}

where λN\lambda_N denotes NN-dimensional volume. Larger HV indicates better objective values or broader coverage of favorable tradeoffs. When an observed vector is not componentwise better than the reference point, it contributes no volume. We use the same fixed reference point for all methods and trials in a given setting. Let HV⁡tv,s\operatorname{HV}_t^{v,s} denote the dominated hypervolume after tt total evaluations for variant v∈{BlackBox-GP,Composite}v \in\{\mathrm{BlackBox\text{-}GP},\mathrm{Composite}\} in paired trial ss. The mean HV across SS trials is

HV⁡‾tv=1S∑s=1SHV⁡tv,s.\overline{\operatorname{HV}}_t^v = \frac{1}{S}\sum_{s=1}^{S}\operatorname{HV}_t^{v,s}.

We measure final performance by the relative improvement in mean HV at the total evaluation budget TT:

Final-HV gain⁡(%)=100HV⁡‾TComposite−HV⁡‾TBlackBox-GPHV⁡‾TBlackBox-GP.(3)\operatorname{Final\text{-}HV\ gain}(\%) = 100 \frac{\overline{\operatorname{HV}}_T^{\mathrm{Composite}}-\overline{\operatorname{HV}}_T^{\mathrm{BlackBox\text{-}GP}}}{\overline{\operatorname{HV}}_T^{\mathrm{BlackBox\text{-}GP}}}. \tag*{(3)}

Significance

The two arms share the function draw, the initial design, the acquisition rule and the evaluation budget, so runs are paired. All reported pp-values are one-sided paired tt-tests for a positive mean gain, computed on the per-run differences log⁡10rTBlackBox-GP,s−log⁡10rTComposite,s\log_{10} r_T^{\mathrm{BlackBox\text{-}GP},s} - \log_{10} r_T^{\mathrm{Composite},s} and HV⁡TComposite,s/HV⁡TBlackBox-GP,s\operatorname{HV}_T^{\mathrm{Composite},s}/\operatorname{HV}_T^{\mathrm{BlackBox\text{-}GP},s}. All tables report mean ±\pm one standard error over the paired runs. Eq. (D.1) and Eq. (D.3) are functions of the arm means rather than averages of per-run percentages, so their standard errors are obtained from the standard error of the mean paired difference by the delta method.

Additional results for controlled experiments

Table 1 of the main text reports six representative outer mappings. This appendix gives every mapping we considered, under the metrics of Appendix D. Table E.1 reports results for the single-objective cases while Table E.2 reports results for multi-objective (two-objective) cases. Both settings use four function draws and five initial designs per draw, giving 20 paired runs per problem, and within each comparison the two methods (BlackBox-GP and Composite) share the initial design, the acquisition rule and the evaluation budget.

Single objective. Of the five mappings that are bijective on the range hh attains, none reaches significance; of the eleven that are not, every one does, with p≤0.030p \le0.030 throughout. Bijectivity therefore separates the two groups perfectly in this setting.

Percentage reductions near 100% can obscure substantial differences in final regret. For the two-root quadratic and four-root quartic, reductions of 99.96% and 99.53% correspond to approximately 2600-fold and 210-fold reductions relative to BlackBox-GP, respectively. Table E.1 therefore also reports gains in log⁡10\log_{10} regret to distinguish improvements of different orders of magnitude. The 100% ceiling also explains the narrow standard errors on the saturated rows.

Multi-objective (two objectives). The pattern transfers only partially. None of the five bijective mappings reaches significance, as in the single-objective case, but only five of the fourteen non-bijective settings do, and the effect sizes are percent-level rather than orders of magnitude. The ordering also fails to carry over: the three- and five-root polynomials give the two weakest single-objective gains but both yield significant hypervolume gains, including the largest, while the four-root quartic and the shifted squares at (h−1)2(h-1)^2 and (h−2)2(h-2)^2 all exceed +99.5% in the single-objective grid yet fall below +0.5% here. Hypervolume optimization requires discovering and refining a set of tradeoffs rather than locating a single optimum, and under a fixed budget that additional difficulty appears to dilute the benefit of modelling intermediates.

Outer mapping ggBlackBox-GPCompositeReductionFinal-RG reduction (%)pp
Bijective on the range hh attains
Exponential ehe^h−2.59 ± 0.09−2.74 ± 0.14+0.15 ± 0.12+28.91 ± 19.010.11
Identity hh−1.03 ± 0.18−1.11 ± 0.14+0.09 ± 0.16+18.10 ± 29.240.29
Cubic h3h^3+0.09 ± 0.11+0.01 ± 0.10+0.08 ± 0.11+17.10 ± 21.070.23
Hyperbolic sine sinh⁡h\sinh h+0.24 ± 0.11+0.16 ± 0.09+0.08 ± 0.10+17.00 ± 19.580.22
Quintic h5h^5+0.75 ± 0.15+0.79 ± 0.08−0.05 ± 0.16−11.62 ± 40.300.62
Not bijective
Two-root quadratic−4.97 ± 0.28−8.39 ± 0.30+3.42 ± 0.41+99.96 ± 0.04<0.01
Shifted square (h−1)2(h-1)^2−5.36 ± 0.24−8.22 ± 0.24+2.86 ± 0.30+99.86 ± 0.09<0.01
Square h2h^2−5.14 ± 0.25−7.95 ± 0.26+2.81 ± 0.40+99.84 ± 0.14<0.01
Shifted square (h−2)2(h-2)^2−5.62 ± 0.22−8.40 ± 0.28+2.78 ± 0.35+99.83 ± 0.13<0.01
Shifted square (h−0.5)2(h-0.5)^2−5.40 ± 0.28−7.81 ± 0.14+2.41 ± 0.25+99.61 ± 0.22<0.01
Shifted square (h−3)2(h-3)^2−4.69 ± 0.36−7.05 ± 0.49+2.36 ± 0.46+99.56 ± 0.46<0.01
Four-root quartic−4.80 ± 0.21−7.13 ± 0.17+2.33 ± 0.20+99.53 ± 0.22<0.01
Cosine cos⁡h\cos h−4.36 ± 0.41−5.79 ± 0.43+1.43 ± 0.48+96.31 ± 4.06<0.01
Cosine cos⁡2h\cos2h−4.94 ± 0.32−6.00 ± 0.20+1.06 ± 0.35+91.27 ± 7.08<0.01
Five-root quintic+1.69 ± 0.04+1.27 ± 0.16+0.42 ± 0.14+62.36 ± 12.33<0.01
Three-root cubic+0.54 ± 0.09+0.28 ± 0.11+0.26 ± 0.13+45.32 ± 16.440.03

Table E.1. Controlled single-objective experiments, all 16 outer mappings. Values in BlackBox-GP and Composite columns are the final log⁡10\log_{10} simple regret from each method; Reduction is their paired difference, so positive favors Composite. Final-RG reduction is that same difference as a percentage, via Eq. (D.1). All values are means ± one standard error over the 20 paired runs, the percentage column by the delta method, and pp is a one-sided paired tt-test for positive mean reduction. Bold marks significance at the unadjusted 0.05 level.

Table E.2: Controlled two-objective experiments, all 19 distinct settings of g1g_1. Values in BlackBox-GP and Composite columns are the final hypervolume; Gain is their paired difference, so positive favors Composite. Final-HV gain is that same difference as a percentage, via Eq. (D.3). All values are means ±\pm one standard error over the 20 paired runs, the percentage column by the delta method, and pp is a one-sided paired tt-test for positive mean gain. The BlackBox-GP and Composite columns have the largest standard errors because each mapping pools four function draws whose hypervolumes differ substantially; that spread cancels in the paired Gain. Bold marks significance at the unadjusted 0.05 level.

Outer mapping g1g_1BlackBox-GPCompositeGainFinal-HV gain (%)pp
Bijective on the range hh attains
Cubic h3h^371.49±11.1771.49 \pm11.1772.12±11.2272.12 \pm11.22+0.63±1.11+0.63 \pm1.11+0.88±1.56+0.88 \pm1.560.29
Exponential ehe^h71.17±11.2271.17 \pm11.2271.58±11.0971.58 \pm11.09+0.41±0.92+0.41 \pm0.92+0.58±1.29+0.58 \pm1.290.33
Quintic h5h^571.95±11.1171.95 \pm11.1172.21±10.9772.21 \pm10.97+0.26±1.44+0.26 \pm1.44+0.36±2.00+0.36 \pm2.000.43
Identity hh70.93±11.1570.93 \pm11.1571.10±11.3071.10 \pm11.30+0.16±0.65+0.16 \pm0.65+0.23±0.92+0.23 \pm0.920.40
Hyperbolic sine sinh⁡h\sinh h71.46±11.2571.46 \pm11.2571.19±11.2171.19 \pm11.21−0.27±0.53-0.27 \pm0.53−0.38±0.75-0.38 \pm0.750.69
Not bijective
Five-root quintic72.13±11.2772.13 \pm11.2780.51±10.0480.51 \pm10.04+8.38±3.23+8.38 \pm3.23+11.62±4.48+11.62 \pm4.480.01
Square h2h^269.88±10.8769.88 \pm10.8771.91±11.1671.91 \pm11.16+2.03±0.70+2.03 \pm0.70+2.90±1.01+2.90 \pm1.01<0.01<0.01
Shifted square (h−0.5)2(h - 0.5)^270.98±11.0370.98 \pm11.0372.94±11.2872.94 \pm11.28+1.96±0.80+1.96 \pm0.80+2.76±1.13+2.76 \pm1.130.01
Three-root cubic70.72±11.1270.72 \pm11.1272.52±11.0472.52 \pm11.04+1.80±0.86+1.80 \pm0.86+2.54±1.22+2.54 \pm1.220.03
Two-root quadratic71.56±11.2371.56 \pm11.2372.88±11.2372.88 \pm11.23+1.32±0.81+1.32 \pm0.81+1.85±1.13+1.85 \pm1.130.06
Cosine cos⁡2h\cos2h70.54±11.0570.54 \pm11.0571.39±11.0871.39 \pm11.08+0.85±0.45+0.85 \pm0.45+1.20±0.64+1.20 \pm0.640.04
Shifted square (h−3)2(h - 3)^270.79±11.1770.79 \pm11.1771.50±11.1871.50 \pm11.18+0.70±0.58+0.70 \pm0.58+1.00±0.83+1.00 \pm0.830.12
Three-root, spread 0.570.78±10.9570.78 \pm10.9571.31±10.9571.31 \pm10.95+0.53±0.57+0.53 \pm0.57+0.75±0.80+0.75 \pm0.800.18
Four-root, spread 0.571.15±11.0871.15 \pm11.0871.63±11.2271.63 \pm11.22+0.47±0.55+0.47 \pm0.55+0.66±0.78+0.66 \pm0.780.20
Shifted square (h−1)2(h - 1)^271.52±11.2971.52 \pm11.2971.87±11.1971.87 \pm11.19+0.35±0.65+0.35 \pm0.65+0.49±0.90+0.49 \pm0.900.30
Shifted square (h−2)2(h - 2)^271.09±11.0271.09 \pm11.0271.39±11.1971.39 \pm11.19+0.30±0.84+0.30 \pm0.84+0.42±1.18+0.42 \pm1.180.36
Six-root, spread 1.071.61±11.1971.61 \pm11.1971.66±11.1471.66 \pm11.14+0.05±1.06+0.05 \pm1.06+0.07±1.48+0.07 \pm1.480.48
Four-root quartic71.97±11.4071.97 \pm11.4071.95±11.2871.95 \pm11.28−0.01±0.85-0.01 \pm0.85−0.02±1.18-0.02 \pm1.180.51
Cosine cos⁡h\cos h71.86±11.1871.86 \pm11.1870.81±11.1570.81 \pm11.15−1.05±0.63-1.05 \pm0.63−1.46±0.87-1.46 \pm0.870.95

Table E.2.

Benchmark definitions

This section specifies the exact deterministic oracle used in the experiments. All design vectors x\mathbf{x} below are normalized to [0,1]d[0,1]^d, and all final objectives are minimized. Here dd is the input dimension, KK is the number of scalar intermediate responses retained by the implementation, and NN is the number of final objectives.

Multi-illuminant color formulation: d=6d=6, K=6K=6, N=2N=2

This analytical simulator represents six hypothetical dye loadings in an opaque textile coating. It is motivated by multi-objective color formulation under multiple illuminants [5], but is not a measured dataset and does not reproduce that paper’s objective equations. Wavelengths are λ∈{400,410,…,700}\lambda\in\{400, 410, \ldots, 700\} nm. Define the Gaussian spectral bump of center μ\mu and width σ\sigma

B(λ;μ,σ)=exp⁡[−12(λ−μσ)2].(4)B(\lambda;\mu,\sigma)=\exp\left[-\frac{1}{2}\left(\frac{\lambda-\mu}{\sigma}\right)^2\right]. \tag*{(4)}

The substrate absorption-to-scattering ratio is q0(λ)=0.012+0.008B(λ;430,75)q_0(\lambda)=0.012+0.008B(\lambda;430,75). The six fixed dye increments are

q1(λ)=1.75B(λ;610,34)+0.22B(λ;445,52),q2(λ)=1.55B(λ;535,31)+0.18B(λ;650,42),q3(λ)=1.65B(λ;448,28)+0.16B(λ;585,65),q4(λ)=1.20B(λ;495,48)+0.38B(λ;632,27),q5(λ)=1.35B(λ;575,41)+0.24B(λ;420,24),q6(λ)=0.55+0.22B(λ;510,115).(5)\begin{aligned} q_1(\lambda)&=1.75B(\lambda;610,34)+0.22B(\lambda;445,52),\\ q_2(\lambda)&=1.55B(\lambda;535,31)+0.18B(\lambda;650,42),\\ q_3(\lambda)&=1.65B(\lambda;448,28)+0.16B(\lambda;585,65),\\ q_4(\lambda)&=1.20B(\lambda;495,48)+0.38B(\lambda;632,27),\\ q_5(\lambda)&=1.35B(\lambda;575,41)+0.24B(\lambda;420,24),\\ q_6(\lambda)&=0.55+0.22B(\lambda;510,115). \tag*{(5)} \end{aligned}

For design x\mathbf{x}, the dye concentrations and total absorption-to-scattering ratio are

cm(x)=0.85xm,q(λ;x)=q0(λ)+∑m=16cm(x)qm(λ).(6)c_m(\mathbf{x})=0.85x_m,\qquad q(\lambda;\mathbf{x})=q_0(\lambda)+\sum_{m=1}^{6}c_m(\mathbf{x})q_m(\lambda). \tag*{(6)}

The opaque Kubelka–Munk inverse [27] gives

R(λ;x)=1+q(λ;x)−q(λ;x)2+2q(λ;x).(7)R(\lambda;\mathbf{x})=1+q(\lambda;\mathbf{x})-\sqrt{q(\lambda;\mathbf{x})^2+2q(\lambda;\mathbf{x})}. \tag*{(7)}

Let cˉ(λ)=(xˉ(λ),yˉ(λ),zˉ(λ))\bar{\mathbf{c}}(\lambda)=(\bar{x}(\lambda),\bar{y}(\lambda),\bar{z}(\lambda)) denote the analytic CIE 1931 color-matching functions of Wyman et al. [28], and write Λc={400,410,…,700}\Lambda_c=\{400,410,\ldots,700\} nm. For λm=10−9λ\lambda_m=10^{-9}\lambda meters, define the normalized blackbody spectrum

I~T(λ)=1λm5[exp⁡(c2λmT)−1],IT(λ)=I~T(λ)max⁡λ′∈ΛcI~T(λ′).(8)\widetilde{I}_T(\lambda)=\frac{1}{\lambda_m^5\left[\exp\left(\frac{c_2}{\lambda_mT}\right)-1\right]},\qquad I_T(\lambda)=\frac{\widetilde{I}_T(\lambda)}{\max_{\lambda'\in\Lambda_c}\widetilde{I}_T(\lambda')}. \tag*{(8)}

where c2=1.438776877×10−2c_2=1.438776877\times10^{-2} m K. For illuminant j∈{D,A}j\in\{D,A\}, with TD=6504T_D=6504 K and TA=2856T_A=2856 K, define

κj=[∑λ∈ΛcITj(λ)yˉ(λ)]−1.(9)\kappa_j=\left[\sum_{\lambda\in\Lambda_c}I_{T_j}(\lambda)\bar{y}(\lambda)\right]^{-1}. \tag*{(9)}

The tristimulus coordinates and perfect-reflector white point are

(Xj(x),Yj(x),Zj(x))=κj∑λ∈ΛcR(λ;x)ITj(λ)cˉ(λ),(10)\left(X_j(\mathbf{x}),Y_j(\mathbf{x}),Z_j(\mathbf{x})\right)=\kappa_j\sum_{\lambda\in\Lambda_c}R(\lambda;\mathbf{x})I_{T_j}(\lambda)\bar{\mathbf{c}}(\lambda), \tag*{(10)}
(Xn,j,Yn,j,Zn,j)=κj∑λ∈ΛcITj(λ)c(λ).(11)\left(X_{n,j},Y_{n,j},Z_{n,j}\right)=\kappa_j\sum_{\lambda\in\Lambda_c}I_{T_j}(\lambda)\mathbf{c}(\lambda). \tag*{(11)}

The corresponding CIE Lab coordinates are

Lj∗(x)=116ϕ(Yj(x)Yn,j)−16,(12)L_j^*(\mathbf{x}) = 116\phi\left(\frac{Y_j(\mathbf{x})}{Y_{n,j}}\right) - 16, \tag*{(12)}
aj∗(x)=500[ϕ(Xj(x)Xn,j)−ϕ(Yj(x)Yn,j)],(13)a_j^*(\mathbf{x}) = 500\left[\phi\left(\frac{X_j(\mathbf{x})}{X_{n,j}}\right) - \phi\left(\frac{Y_j(\mathbf{x})}{Y_{n,j}}\right)\right], \tag*{(13)}
bj∗(x)=200[ϕ(Yj(x)Yn,j)−ϕ(Zj(x)Zn,j)],(14)b_j^*(\mathbf{x}) = 200\left[\phi\left(\frac{Y_j(\mathbf{x})}{Y_{n,j}}\right) - \phi\left(\frac{Z_j(\mathbf{x})}{Z_{n,j}}\right)\right], \tag*{(14)}

where δ=6/29\delta= 6/29 and

ϕ(t)={t1/3,t>δ3,t/(3δ2)+4/29,t≤δ3.(15)\phi(t) = \begin{cases} t^{1/3}, & t > \delta^3, \\ t/(3\delta^2) + 4/29, & t \le\delta^3. \end{cases} \tag*{(15)}

The objective-specific intermediate responses are

h1(x)=(LD∗(x),aD∗(x),bD∗(x)),\mathbf{h}_1(\mathbf{x}) = (L_D^*(\mathbf{x}), a_D^*(\mathbf{x}), b_D^*(\mathbf{x})),
h2(x)=(LD∗(x),aD∗(x),bD∗(x),LA∗(x),aA∗(x),bA∗(x)).(16)\mathbf{h}_2(\mathbf{x}) = (L_D^*(\mathbf{x}), a_D^*(\mathbf{x}), b_D^*(\mathbf{x}), L_A^*(\mathbf{x}), a_A^*(\mathbf{x}), b_A^*(\mathbf{x})). \tag*{(16)}

These contain six distinct retained scalar responses because the daylight coordinates are shared.

The unattainable target pigment has

q⋆(λ)=q0(λ)+0.92B(λ;482,19)+0.74B(λ;622,25)+0.18B(λ;550,72).(17)q_\star(\lambda) = q_0(\lambda) + 0.92B(\lambda;482,19) + 0.74B(\lambda;622,25) + 0.18B(\lambda;550,72). \tag*{(17)}

The target ratio q⋆(λ)q_\star(\lambda) is converted to reflectance using Eq. (F.4), then to illuminant-specific XYZ and CIE Lab coordinates using the equations above. The resulting target vectors are tj=(L⋆,j∗,a⋆,j∗,b⋆,j∗)\mathbf{t}_j = (L_{\star,j}^*, a_{\star,j}^*, b_{\star,j}^*) for j∈{D,A}j \in\{D,A\}. For a generic argument y\mathbf{y}, define

g1(y)=∥y1:3−tD∥260,g2(y)=∥(y4:6−tA)−(y1:3−tD)∥218.(18)g_1(\mathbf{y}) = \frac{\lVert\mathbf{y}_{1:3} - \mathbf{t}_D \rVert_2}{60}, \qquad g_2(\mathbf{y}) = \frac{\lVert(\mathbf{y}_{4:6} - \mathbf{t}_A) - (\mathbf{y}_{1:3} - \mathbf{t}_D) \rVert_2}{18}. \tag*{(18)}

The objectives are fi(x)=gi(hi(x))f_i(\mathbf{x}) = g_i(\mathbf{h}_i(\mathbf{x})). Thus f1f_1 is daylight color error and f2f_2 is a metameric-mismatch proxy: it penalizes how the sample-minus-target Lab error changes between illuminants. The nonlinear norms erase the direction of the six-dimensional color errors. The hypervolume reference is (1.05,1.05)(1.05, 1.05).

DTLZ2 (Synthetic 1): d=6d=6, K=4K=4, N=2N=2

The synthetic two-objective DTLZ2 problem [19] is

q(x)=∑j=26(xj−0.5)2,θ(x)=πx12,(19)q(\mathbf{x}) = \sum_{j=2}^{6}(x_j - 0.5)^2, \qquad\theta(\mathbf{x}) = \frac{\pi x_1}{2}, \tag*{(19)}
h1(x)=(q(x),cos⁡θ(x)),h2(x)=(q(x),sin⁡θ(x)),(20)\mathbf{h}_1(\mathbf{x}) = (q(\mathbf{x}), \cos\theta(\mathbf{x})), \qquad \mathbf{h}_2(\mathbf{x}) = (q(\mathbf{x}), \sin\theta(\mathbf{x})), \tag*{(20)}
g1(y)=(1+y1)y2,g2(y)=(1+y1)y2,fi(x)=gi(hi(x)).(21)g_1(\mathbf{y}) = (1+y_1)y_2, \qquad g_2(\mathbf{y}) = (1+y_1)y_2, \qquad f_i(\mathbf{x}) = g_i(\mathbf{h}_i(\mathbf{x})). \tag*{(21)}

The repeated qq deliberately gives each objective an independently modeled component group, avoiding implicit multi-task transfer. The Pareto set has xj=0.5x_j = 0.5 for j=2,…,6j = 2,\ldots,6 and the front satisfies f12+f22=1f_1^2 + f_2^2 = 1, f1,f2≥0f_1, f_2 \ge0. For reference point (2.5,2.5)(2.5,2.5) the exact maximum hypervolume is 2.52−π/42.5^2 - \pi/4.

Four-bar truss RE21: d=4d=4, K=8K=8, N=2N=2

Four bars are joined into a frame carrying a load at one joint, and the design chooses how thick each bar is. Thicker bars sag less but use more material, giving the problem a tradeoff. This problem follows the RE21 formulation in the real-world multi-objective suite [21]. The four cross-sectional areas are

a(x)=amin⁡+x⊙(amax⁡−amin⁡),amin⁡=(1,2,2,1),amax⁡=(3,3,3,3)(22)\mathbf{a}(\mathbf{x}) = \mathbf{a}^{\min} + \mathbf{x} \odot(\mathbf{a}^{\max} - \mathbf{a}^{\min}), \qquad \mathbf{a}^{\min} = (1,\sqrt{2},\sqrt{2},1), \qquad \mathbf{a}^{\max} = (3,3,3,3) \tag*{(22)}

in cm2\mathrm{cm}^2, where each entry of xx sets that bar’s area as a fraction of the way between its minimum and maximum. The lower bounds are the thinnest each bar can be without exceeding the material’s stress limit of 10 kN/cm210\ \mathrm{kN}/\mathrm{cm}^2 under the load F=10 kNF=10\ \mathrm{kN}, so every design in the box is already safe. Each bar is L=200 cmL=200\ \mathrm{cm} long and the material’s stiffness (elastic modulus) is E=2×105 kN/cm2E=2\times10^5\ \mathrm{kN}/\mathrm{cm}^2. The intermediate responses are

h1(x)=(2a1(x),2a2(x),a3(x),a4(x)),h2(x)=(2a1(x),22a2(x),22a3(x),2a4(x)).(23)\begin{aligned} \mathbf{h}_1(\mathbf{x})&=\left(2a_1(\mathbf{x}),\sqrt{2}a_2(\mathbf{x}),\sqrt{a_3(\mathbf{x})},a_4(\mathbf{x})\right),\\ \mathbf{h}_2(\mathbf{x})&=\left(\frac{2}{a_1(\mathbf{x})},\frac{2\sqrt{2}}{a_2(\mathbf{x})},\frac{2\sqrt{2}}{a_3(\mathbf{x})},\frac{2}{a_4(\mathbf{x})}\right). \tag*{(23)} \end{aligned}

Each entry corresponds to a bar’s contribution to total material in h1\mathbf{h}_1, and to the sag at the loaded joint in h2\mathbf{h}_2. Raw volume and loaded-joint displacement are

V(y)=L∑j=14yj,D(y)=FLE∑j=14yj.(24)V(\mathbf{y})=L\sum_{j=1}^{4}y_j,\qquad D(\mathbf{y})=\frac{FL}{E}\sum_{j=1}^{4}y_j. \tag*{(24)}

They are affinely normalized using exact extrema over the box:

Vmin⁡=1237.8414,Vmax⁡=2994.9383,(25)V_{\min}=1237.8414,\qquad V_{\max}=2994.9383, \tag*{(25)}
Dmin⁡=0.00276142,Dmax⁡=0.05057191,(26)D_{\min}=0.00276142,\qquad D_{\max}=0.05057191, \tag*{(26)}
g1(y)=V(y)−Vmin⁡Vmax⁡−Vmin⁡,g2(y)=D(y)−Dmin⁡Dmax⁡−Dmin⁡.(27)g_1(\mathbf{y})=\frac{V(\mathbf{y})-V_{\min}}{V_{\max}-V_{\min}},\qquad g_2(\mathbf{y})=\frac{D(\mathbf{y})-D_{\min}}{D_{\max}-D_{\min}}. \tag*{(27)}

The final objectives are fi(x)=gi(hi(x))f_i(\mathbf{x})=g_i(\mathbf{h}_i(\mathbf{x})). The reference is (1.05,1.05)(1.05,1.05). Unlike the color and DTLZ2 reductions, the outer map is essentially a signed sum plus affine scaling, consistent with the small, though significant, gains in Table 2.

Latent Langermann–Ackley (Synthetic 2): d=600d=600, K=12K=12, N=2N=2

Let A∈R6×600A\in\mathbb{R}^{6\times600} have fixed orthonormal rows generated once with seed 18,271, and define the dense latent vector

z(x)=8A(x−121).(28)\mathbf{z}(\mathbf{x})=8A\left(\mathbf{x}-\frac{1}{2}\mathbf{1}\right). \tag*{(28)}

The intermediate responses are objective-specific copies, h1(x)=h2(x)=z(x)∈R6\mathbf{h}_1(\mathbf{x})=\mathbf{h}_2(\mathbf{x})=\mathbf{z}(\mathbf{x})\in\mathbb{R}^{6}. Thus the implementation retains K=12K=12 scalar responses, which prevents its independent GP implementation from sharing the six latent observations across objectives. For centers

C=[1.6−0.90.71.2−1.40.5−1.31.5−0.80.61.0−1.10.81.11.4−1.5−0.50.9−0.7−1.61.01.40.8−0.61.20.6−1.5−0.91.31.0],c=(1,2,5,2,3),(29)C= \begin{bmatrix} 1.6 & -0.9 & 0.7 & 1.2 & -1.4 & 0.5\\ -1.3 & 1.5 & -0.8 & 0.6 & 1.0 & -1.1\\ 0.8 & 1.1 & 1.4 & -1.5 & -0.5 & 0.9\\ -0.7 & -1.6 & 1.0 & 1.4 & 0.8 & -0.6\\ 1.2 & 0.6 & -1.5 & -0.9 & 1.3 & 1.0 \end{bmatrix},\qquad\mathbf{c}=(1,2,5,2,3), \tag*{(29)}

define rℓ(y)=∥y−Cℓ∥2r_\ell(\mathbf{y})=\lVert\mathbf{y}-C_\ell\rVert_2 and CΣ=∑ℓ=15cℓ=13C_\Sigma=\sum_{\ell=1}^{5}c_\ell=13. The two known mappings are

g1(y)=CΣ−∑ℓ=15cℓexp⁡[−rℓ(y)/π]cos⁡[πrℓ(y)]2CΣ,(30)g_1(\mathbf{y})=\frac{C_\Sigma-\sum_{\ell=1}^{5}c_\ell\exp[-r_\ell(\mathbf{y})/\pi]\cos[\pi r_\ell(\mathbf{y})]}{2C_\Sigma}, \tag*{(30)}
g2(y)=−20exp⁡[−0.216∑jyj2]−exp⁡[16∑jcos⁡(2πyj)]+20+e20+e−e−1.(31)g_2(\mathbf{y})=\frac{-20\exp\left[-0.2\sqrt{\frac{1}{6}\sum_j y_j^2}\right]-\exp\left[\frac{1}{6}\sum_j\cos(2\pi y_j)\right]+20+e}{20+e-e^{-1}}. \tag*{(31)}

The objectives are fi(x)=gi(hi(x))f_i(\mathbf{x})=g_i(\mathbf{h}_i(\mathbf{x})). f1f_1 is a bounded Langermann-style landscape and f2f_2 is normalized Ackley. Their optima conflict: Ackley is minimized at z=0\mathbf{z}=0, whereas the Langermann centers are displaced. The reference point is (1.1,1.1)(1.1,1.1).

Shallow-lake management: d=100d = 100, K=21K = 21, N=2N = 2

A town releases phosphorus into a nearby lake each year for 100 years. Releasing more is economically useful, but phosphorus builds up in the lake, and past a threshold the lake flips into a polluted state that is very hard to reverse. The design chooses the 100 yearly release amounts. Each xtx_t in x\mathbf{x} controls annual phosphorus release ut=0.1xtu_t = 0.1x_t over 100 years. The lake is simulated under 100 fixed common-random-number inflow scenarios (ss). For scenario ss, writing Pt(s)P_t^{(s)} for the phosphorus in the lake water at year tt, we have P0(s)=0P_0^{(s)} = 0 and for t=1,…,99t = 1, \ldots, 99,

Pt(s)=0.58Pt−1(s)+(Pt−1(s))21+(Pt−1(s))2+ut−1+ξt−1(s)(32)P_t^{(s)} = 0.58P_{t-1}^{(s)} + \frac{\left(P_{t-1}^{(s)}\right)^2}{1 + \left(P_{t-1}^{(s)}\right)^2} + u_{t-1} + \xi_{t-1}^{(s)} \tag*{(32)}

where ξt(s)∼LogNormal⁡(−3.52,0.1052)\xi_t^{(s)} \sim\operatorname{LogNormal}(-3.52, 0.105^2) was drawn once with seed 20,240,820. The first term is the carryover of phosphorus from last year, the third term is the amount the town chose to release last year, and the fourth is uncontrolled inflow from nature. The second term accounts for phosphorus that settled into the lake bed and later returns to the water. Below a critical level the lake clears itself faster than this recycling refills it; above that level it does not, and pollution keeps rising even if releases stop. That threshold in the lake’s phosphorus level is the tipping point. Fixing scenarios makes the oracle deterministic while retaining scenario-averaged variability in the physical response. Let Pˉt=100−1∑sPt(s)\bar{P}_t = 100^{-1}\sum_s P_t^{(s)}. The first intermediate response contains 20 five-year block peaks,

h1,b(x)=12.4max⁡t=5(b−1),…,5b−1Pˉt,b=1,…,20.(33)h_{1,b}(\mathbf{x}) = \frac{1}{2.4}\max_{t=5(b-1),\ldots,5b-1}\bar{P}_t,\quad b = 1,\ldots,20. \tag*{(33)}

The second intermediate response is normalized discounted utility,

h2,1(x)=∑t=0990.98tut0.1∑t=0990.98t.(34)h_{2,1}(\mathbf{x}) = \frac{\sum_{t=0}^{99}0.98^t u_t}{0.1\sum_{t=0}^{99}0.98^t}. \tag*{(34)}

Thus h1(x)=(h1,1,…,h1,20)\mathbf{h}_1(\mathbf{x}) = (h_{1,1},\ldots,h_{1,20}) and h2(x)=(h2,1)\mathbf{h}_2(\mathbf{x}) = (h_{2,1}). The known mappings, minimized objectives, and reference point are

g1(y)=max⁡b=1,…,20yb,g2(y)=1−y1,fi(x)=gi(hi(x)),r=(1.05,1.05).(35)g_1(\mathbf{y}) = \max_{b=1,\ldots,20} y_b,\quad g_2(\mathbf{y}) = 1-y_1,\quad f_i(\mathbf{x}) = g_i(\mathbf{h}_i(\mathbf{x})),\quad\mathbf{r} = (1.05,1.05). \tag*{(35)}

This is a deterministic two-objective slice of the shallow-lake dynamics of Carpenter et al. [29] as used in multi-objective robustness studies [20]; it is not the original four-objective stochastic benchmark. The last decision u99u_{99} contributes to utility but, following the implemented indexing, not to the 100 recorded lake states. The composite representation retains which five-year period controls water quality before the outer maximum discards that information.

RGB-selective multilayer nanoparticle: d=6d = 6, K=6K = 6, N=3N = 3

Light partially reflects at every boundary between two different materials, so a particle whose layers alternate between two materials produces several reflections that interfere, reinforcing at some wavelengths and canceling at others. Choosing the layer thicknesses therefore chooses which colors the particle scatters, and the goal is a particle that scatters one target color band and little else. This electromagnetic design problem chooses a core radius and five concentric shell thicknesses. For normalized x∈[0,1]6\mathbf{x} \in[0,1]^6, the implemented dimensions are tj(x)=30+40xjt_j(\mathbf{x}) = 30 + 40x_j nm. The layers alternate between silica, with relative permittivity 2.04, and a wavelength-dependent titanium-dioxide model. For wavelength λ\lambda measured in nanometers,

ϵTiO2(λ)=5.913+0.244110−6λ2−0.0803.(36)\epsilon_{\mathrm{TiO}_2}(\lambda) = 5.913 + \frac{0.2441}{10^{-6}\lambda^2 - 0.0803}. \tag*{(36)}

This formula returns the relative permittivity of titanium dioxide at wavelength λ\lambda, which increases toward the blue end of the spectrum. The surrounding medium has relative permittivity 1.77. The simulator evaluates the concentric-sphere Mie calculation for orders 1–12 and both polarizations on the discrete wavelength grid

Λs={350+2m:m=0,…,200} nm.(37)\Lambda_s = \{350 + 2m : m = 0,\ldots,200\}\ \mathrm{nm}. \tag*{(37)}

Mie theory gives the exact scattering from a layered sphere as an infinite series, which the implementation truncates at twelve orders. Given the layer thicknesses tj(x)t_j(\mathbf{x}), the layer and medium permittivities, and the wavelength grid Λs\Lambda_s, the simulator returns a scattering value at each wavelength. The design enters through the layer radii rj(x)=∑k<jtk(x)r_j(\mathbf{x}) = \sum_{k<j} t_k(\mathbf{x}), which determine the interference conditions at each interface. The resulting scattering spectrum S(λ;r(x),ϵ(λ))S(\lambda; \mathbf{r}(\mathbf{x}), \boldsymbol{\epsilon}(\lambda)) therefore depends on the design through the interface radii r(x)=(r1(x),…,r6(x))\mathbf{r}(\mathbf{x}) = (r_1(\mathbf{x}), \ldots, r_6(\mathbf{x})) and on the material properties through ϵ(λ)=(ϵ1(λ),…,ϵ7(λ))\boldsymbol{\epsilon}(\lambda) = (\epsilon_1(\lambda), \ldots, \epsilon_7(\lambda)). Specifically, ϵ1=ϵ3=ϵ5=2.04\epsilon_1 = \epsilon_3 = \epsilon_5 = 2.04 for silica, ϵ2(λ)=ϵ4(λ)=ϵ6(λ)=ϵTiO2(λ)\epsilon_2(\lambda) = \epsilon_4(\lambda) = \epsilon_6(\lambda) = \epsilon_{\mathrm{TiO}_2}(\lambda), and ϵ7=1.77\epsilon_7 = 1.77 represents the surrounding medium; rj(x)r_j(\mathbf{x}) is the boundary between regions jj and j+1j+1. It is normalized by the particle cross-sectional area πr6(x)2\pi r_6(\mathbf{x})^2. For brevity, we denote this spectrum by S(λ;x)S(\lambda; \mathbf{x}) below. For target bands B1=[400,500)B_1 = [400, 500), B2=[500,600)B_2 = [500, 600), and B3=[600,700)B_3 = [600, 700) nm, the implemented in-band and out-of-band responses are the discrete sums

Ib(x)=∑λ∈Λs∩BbS(λ;x),Ob(x)=∑λ∈Λs∖BbS(λ;x).(38)I_b(\mathbf{x}) = \sum_{\lambda\in\Lambda_s \cap B_b} S(\lambda; \mathbf{x}), \qquad O_b(\mathbf{x}) = \sum_{\lambda\in\Lambda_s \setminus B_b} S(\lambda; \mathbf{x}). \tag*{(38)}

Here IbI_b is the scattering that lands inside the target band and ObO_b is everything else, so the objective below is the fraction of the particle’s scattering that misses the target band, and smaller values mean a more color-selective particle. The intermediate response, known mapping, and minimized objective for band bb are

hb(x)=(Ib(x),Ob(x)),gb(y)=y2y1+y2,fb(x)=gb(hb(x)),b=1,2,3.(39)\mathbf{h}_b(\mathbf{x}) = (I_b(\mathbf{x}), O_b(\mathbf{x})), \qquad g_b(\mathbf{y}) = \frac{y_2}{y_1+y_2}, \qquad f_b(\mathbf{x}) = g_b(\mathbf{h}_b(\mathbf{x})), \qquad b = 1, 2, 3. \tag*{(39)}

The particle parameterization and scattering simulator follow Kim et al. [23]; the three RGB-selectivity objectives are constructed in this work. When gbg_b is applied to GP posterior samples, the implementation clamps both components to be nonnegative and floors their sum at 10−1210^{-12}. Each objective therefore owns a disjoint component pair. The ratio is nonlinear and scale-invariant, but retaining the three hb\mathbf{h}_b doubles the independent output models from three to six without exposing a component shared across objectives. The reference point is (2.5,2.5,2.5)(2.5, 2.5, 2.5).

Summit SNAr reaction: d=4d = 4, K=8K = 8, N=2N = 2

This case implements the mechanistic reaction benchmark distributed with Summit [24]. The starting material is a molecule carrying two replaceable fluorine atoms, and the reagent pyrrolidine displaces one of them. Because either fluorine can be replaced, the reaction yields the desired product, a variant in which the substitution occurred at the other position, and—if a second pyrrolidine reacts—a byproduct with both fluorines replaced.

Four normalized decisions set how long the mixture stays in the reactor, τ∈[0.5,2.0]\tau\in[0.5, 2.0] min; how much pyrrolidine is supplied per unit of starting material, e∈[1,5]e \in[1, 5] moles per mole; the concentration of starting material entering the reactor, cin∈[0.1,0.5]c_{\mathrm{in}} \in[0.1, 0.5] mol L−1^{-1}; and the temperature, T∈[30,120]T \in[30, 120] °C.

The state C=(C0,…,C4)\mathbf{C} = (C_0, \ldots, C_4) holds the molar concentrations of starting material, pyrrolidine, product, variant, and byproduct, with molar masses M=(159.09,71.12,210.21,210.21,261.33)\mathbf{M} = (159.09, 71.12, 210.21, 210.21, 261.33) g mol−1^{-1}. From C(0)=(cin,ecin,0,0,0)\mathbf{C}(0) = (c_{\mathrm{in}}, e c_{\mathrm{in}}, 0, 0, 0), a fixed 256-step RK4 solve integrates the five-species plug-flow model (a tube reactor without back-mixing) over the residence time. The four rate constants follow the Arrhenius law, kj=0.6kjrefexp⁡[−EjR−1(TK−1−Tref−1)]k_j = 0.6 k_j^{\mathrm{ref}} \exp[-E_j R^{-1}(T_K^{-1} - T_{\mathrm{ref}}^{-1})], with activation energy EjE_j in kJ mol−1^{-1}, gas constant R=8.314×10−3R = 8.314 \times10^{-3} kJ mol−1^{-1} K−1^{-1}, TK=T+273.71T_K = T + 273.71, and Tref=363.71T_{\mathrm{ref}} = 363.71 K; the (kref,E)(k^{\mathrm{ref}}, E) pairs for aa, bb, cc, dd are (57.9,33.3)(57.9, 33.3), (2.70,35.3)(2.70, 35.3), (0.865,38.9)(0.865, 38.9), and (1.63,44.8)(1.63, 44.8). With

ra=kaC0C1,rb=kbC0C1,rc=kcC1C2,rd=kdC1C3,(40)r_a = k_a C_0 C_1, \qquad r_b = k_b C_0 C_1, \qquad r_c = k_c C_1 C_2, \qquad r_d = k_d C_1 C_3, \tag*{(40)}

the concentration derivatives are

C˙0=−ra−rb,C˙1=−ra−rb−rc−rd,C˙2=ra−rc,C˙3=rb−rd,C˙4=rc+rd.(41)\begin{aligned} \dot{C}_0 &= -r_a-r_b, & \dot{C}_1 &= -r_a-r_b-r_c-r_d, \\ \dot{C}_2 &= r_a-r_c, & \dot{C}_3 &= r_b-r_d, \\ \dot{C}_4 &= r_c+r_d. \tag*{(41)} \end{aligned}

Let F(x)=5/τ(x)F(\mathbf{x}) = 5/\tau(\mathbf{x}) mL min−1^{-1} be the flow through the 5 mL reactor, and let Cout(x)\mathbf{C}^{\mathrm{out}}(\mathbf{x}) denote the outlet concentrations produced by the RK4 integration. The retained intermediate responses are

h1(x)=(C2out(x),F(x)),h2(x)=(C0out(x),…,C4out(x),F(x)).(42)\mathbf{h}_1(\mathbf{x}) = \left(C^{\mathrm{out}}_2(\mathbf{x}), F(\mathbf{x})\right), \qquad\mathbf{h}_2(\mathbf{x}) = \left(C^{\mathrm{out}}_0(\mathbf{x}),\ldots,C^{\mathrm{out}}_4(\mathbf{x}),F(\mathbf{x})\right). \tag*{(42)}

Thus product concentration and flow occur in both intermediate groups and are modeled twice. For y=(y1,y2)∈R2\mathbf{y}=(y_1,y_2)\in\mathbb{R}^2, the first known mapping is one minus normalized space-time yield,

g1(y)=1−clip⁡(12M2y1y213000,0,1).(43)g_1(\mathbf{y}) = 1-\operatorname{clip}\left(\frac{12M_2y_1y_2}{13000},0,1\right). \tag*{(43)}

For y=(y1,…,y6)∈R6\mathbf{y}=(y_1,\ldots,y_6)\in\mathbb{R}^6, define

W(y)=M0y1+M1y2+M3y4+M4y5,(44)W(\mathbf{y}) = M_0y_1+M_1y_2+M_3y_4+M_4y_5, \tag*{(44)}

where WW is the outlet waste-mass concentration. The E-factor is

E(y)=0.789y6+10−3y6W(y)10−3M2y3y6,g2(y)=min⁡{E(y),1000}500.(45)E(\mathbf{y}) = \frac{0.789y_6+10^{-3}y_6W(\mathbf{y})}{10^{-3}M_2y_3y_6}, \qquad g_2(\mathbf{y}) = \frac{\min\{E(\mathbf{y}),1000\}}{500}. \tag*{(45)}

The final objectives are fi(x)=gi(hi(x))f_i(\mathbf{x})=g_i(\mathbf{h}_i(\mathbf{x})). The implementation clamps modeled concentrations to nonnegative values, floors modeled flow at 10−810^{-8}, and floors the E-factor denominator at 10−1210^{-12} before applying these mappings.

The composition is nonlinear and mechanistic but offers the surrogate little: both outer maps are smooth over the attained states, product concentration and flow are modeled twice, and flow does not affect f2f_2 at all. The reference point is (2.5,2.5)(2.5,2.5).

RCM40 power flow: d=34d=34, K=29K=29, N=2N=2

RCM40 is the two-objective optimal-power-flow problem from the real-world constrained multi-objective suite [22]. It involves setting the voltages on an electrical grid so that both wasted power and reactive loading are minimized. We only use the implemented hypercube constraints and not the original inequality constraints.

The grid is the standard IEEE-14 test network; its 14 nodes are called buses. The optimizer receives x∈[0,1]34\mathbf{x}\in[0,1]^{34}. Its first 26 coordinates are mapped affinely to [−1,1][-1,1] and used as the real and imaginary parts of the voltages at buses 2–14. Bus 1 is the slack bus and is fixed at V1=1V_1=1. The final eight coordinates are not read by the implemented oracle, leaving 26 active dimensions.

The admittance matrix Y=G+iB∈C14×14Y=G+iB\in\mathbb{C}^{14\times14} encoding how strongly each pair of buses is electrically connected, is known in advance. The real matrices GG and BB contain the conductive and reactive parts of that connection. Multiplication yields the per-bus current and power vectors I,S∈C14I,S\in\mathbb{C}^{14} from the voltage vector V∈C14V\in\mathbb{C}^{14}:

I=YV,S=V⊙I‾,P=ℜS,Q=ℑS,(46)I=YV,\qquad S=V\odot\overline{I},\qquad P=\Re S,\qquad Q=\Im S, \tag*{(46)}

where ⊙\odot denotes entrywise multiplication, ⋅‾\overline{\cdot} denotes complex conjugation, and ℜ\Re and ℑ\Im return the real and imaginary parts. Thus P∈R14P\in\mathbb{R}^{14} and Q∈R14Q\in\mathbb{R}^{14} are the real- and reactive-power injections at the buses in per-unit coordinates.

The intermediate quantities we retain and their known maps are

h1(x)=(P1,…,P14),g1(y)=∑j=114yj,h2(x)=(ℑI2,Q1,…,Q14),g2(y)=−y1+∑j=315yj,fi(x)=gi(hi(x)),i=1,2.(47)\begin{aligned} \mathbf{h}_1(\mathbf{x})&=(P_1,\ldots,P_{14}), & g_1(\mathbf{y})&=\sum_{j=1}^{14}y_j,\\ \mathbf{h}_2(\mathbf{x})&=(\Im I_2,Q_1,\ldots,Q_{14}), & g_2(\mathbf{y})&=-y_1+\sum_{j=3}^{15}y_j,\\ f_i(\mathbf{x})&=g_i(\mathbf{h}_i(\mathbf{x})), & &i=1,2. \tag*{(47)} \end{aligned}

Here g1g_1 sums the real-power injections, giving total real-power loss. The second mapping gives f2=−ℑI2+∑j=214Qjf_2=-\Im I_2+\sum_{j=2}^{14}Q_j; the retained value Q1Q_1 receives zero weight. Both mappings are linear with constant Jacobians. Their coefficients are all one for g1g_1 and are −1-1, 00, or 11 for g2g_2. The reference point is (200,500)(200,500).

Considered MOBO methods

For objectives fi(x)=gi(hi(x))f_i(\mathbf{x}) = g_i(h_i(\mathbf{x})), let Xt=(x1,…,xt)X_t = (\mathbf{x}_1,\ldots,\mathbf{x}_t) denote the evaluated designs, Ft=(f(xj))j=1tF_t = (\mathbf{f}(\mathbf{x}_j))_{j=1}^{t} their objective observations, and Ht=(h(xj))j=1tH_t = (\mathbf{h}(\mathbf{x}_j))_{j=1}^{t} the retained scalar intermediate responses, collected in h\mathbf{h}. The two modeling approaches differ in the posterior samples supplied to the acquisition rule. BlackBox-GP fits independent GPs to the final objectives, giving samples

f~i(⋅)∼pi(fi(⋅)∣Xt,Ft,i),i=1,…,N.(48)\widetilde{f}_i(\cdot) \sim p_i(f_i(\cdot) \mid X_t,F_{t,i}), \qquad i=1,\ldots,N. \tag*{(48)}

Composite instead fits independent GPs to the KK retained scalar intermediate responses and propagates their posterior samples through the known mappings:

h~k(⋅)∼pk(hk(⋅)∣Xt,Ht,k),k=1,…,K,f~i(x)=gi(h~i(x)).(49)\widetilde{h}_k(\cdot) \sim p_k(h_k(\cdot) \mid X_t,H_{t,k}), \qquad k=1,\ldots,K, \qquad\widetilde{f}_i(\mathbf{x}) = g_i\left(\widetilde{\mathbf{h}}_i(\mathbf{x})\right). \tag*{(49)}

Here Ft,iF_{t,i} and Ht,kH_{t,k} denote the corresponding observed output columns, and h~i\widetilde{\mathbf{h}}_i collects the responses used by gig_i. Shared or duplicated responses are retained as specified in Appendix § 13. The induced Composite objective posterior need not be Gaussian. Both variants use the same acquisition rule and observed objectives to measure improvement; MORBO applies these modeling choices locally within trust regions. All definitions below use the minimization convention.

Log Expected Hypervolume Improvement (LogEHVI)

Expected hypervolume improvement (EHVI) [17] selects designs according to their expected contribution to the hypervolume of the observed Pareto front. Let P\mathcal{P} denote the nondominated objective vectors observed so far and r\mathbf{r} the fixed reference point. For our minimization problems, the hypervolume improvement from evaluating x\mathbf{x} is

ΔHV(x)=HV(P∪{f(x)};r)−HV(P;r).(50)\Delta\mathrm{HV}(\mathbf{x}) = \mathrm{HV}(\mathcal{P} \cup\{\mathbf{f}(\mathbf{x})\};\mathbf{r}) - \mathrm{HV}(\mathcal{P};\mathbf{r}). \tag*{(50)}

EHVI averages this improvement over the posterior distribution of f(x)\mathbf{f}(\mathbf{x}). Its logarithmic counterpart targets

αLogEHVI(x)=log⁡E[ΔHV(x)].(51)\alpha_{\mathrm{LogEHVI}}(\mathbf{x}) = \log\mathbb{E}[\Delta\mathrm{HV}(\mathbf{x})]. \tag*{(51)}

Taking the logarithm preserves the maximizers whenever EHVI is positive. However, directly computing EHVI and then taking its logarithm can suffer from numerical underflow and vanishing gradients. We therefore use BoTorch’s qLogEHVI implementation [16], which evaluates a numerically stable, smooth approximation in log space. We set q=1q = 1, selecting one design at each iteration.

Smooth Tchebycheff scalarization (STCH)

Smooth Tchebycheff scalarization combines the NN objectives into the single scalar objective sμ(f(x);w)s_\mu(\mathbf{f}(\mathbf{x});\mathbf{w}), where w\mathbf{w} specifies the tradeoff among the objectives. For weights wi≥0w_i \ge0 with ∑i=1Nwi=1\sum_{i=1}^{N} w_i = 1, reference vector z∗\mathbf{z}^*, and temperature μ>0\mu> 0, define

sμ(f(x);w)=μlog⁡[∑i=1Nexp⁡(wi(fi(x)−zi∗)μ)].(52)s_\mu(\mathbf{f}(\mathbf{x});\mathbf{w}) = \mu\log\left[\sum_{i=1}^{N} \exp\left(\frac{w_i(f_i(\mathbf{x}) - z_i^*)}{\mu}\right)\right]. \tag*{(52)}

Let swbests_{\mathbf{w}}^{\mathrm{best}} be the smallest scalarized value observed in the branch for weight w\mathbf{w}. Its LogEI acquisition targets

αLogEI(x;w)=log⁡E[(swbest−sμ(f~(x);w))+].(53)\alpha_{\mathrm{LogEI}}(\mathbf{x};\mathbf{w}) = \log\mathbb{E}\left[\left(s_{\mathbf{w}}^{\mathrm{best}} - s_\mu(\widetilde{\mathbf{f}}(\mathbf{x});\mathbf{w})\right)_{+}\right]. \tag*{(53)}

where (a)+=max⁡{a,0}(a)_{+} = \max\{a,0\} and the expectation is over the BlackBox-GP or Composite objective posterior. Maximizing this acquisition selects the next design for that weight. All STCH variants use temperature μ=0.05\mu= 0.05 and five fixed weights. For two objectives, w1w_1 is equally spaced from 0.05 to 0.95 and w2=1−w1w_2 = 1 - w_1; for more objectives, deterministic normalized exponential draws cover the simplex. The five scalarizations with different weights share one Sobol initial design, evaluated only once, and then evolve independently. The BlackBox-GP STCH variant applies the negative of the scalarization in Eq. (G.5), −sμ(f~;w)-s_{\mu}(\tilde{\mathbf{f}};\mathbf{w}), to each objective posterior sample f~\tilde{\mathbf{f}}. The Composite STCH variant first constructs f~=(gi(h~i))i=1d\tilde{\mathbf{f}}=\left(g_i(\tilde{\mathbf{h}}_i)\right)_{i=1}^{d} and then applies the same −sμ(f~;w)-s_{\mu}(\tilde{\mathbf{f}};\mathbf{w}). Both variants maximize qLogEI (q=1q=1) [16] relative to the best scalarized value observed for that weight vector. Low-dimensional branches receive eight adaptive evaluations each, while the high-dimensional STCH branches receive twenty. For reporting only, their points are interleaved in round-robin order so the HV trace does not depend on an arbitrary ordering of the five weights.

Spherical-linear STCH (Sp-STCH)

Sp-STCH retains the STCH weights, independent branches, composition, and qLogEI steps above, but replaces every standard output GP with the following spherical-linear model. For each output, the implementation centers x\mathbf{x} at (1/2,…,1/2)(1/2,\ldots,1/2), divides coordinates by learned ARD scales, and applies a learned global radial scale before constructing

P(u)=[b0,b1u∥u∥2],k(u,u′)=P(u)TP(u′).(54)P(\mathbf{u})=\left[\sqrt{b_0},\sqrt{b_1}\frac{\mathbf{u}}{\lVert\mathbf{u}\rVert_2}\right],\qquad k(\mathbf{u},\mathbf{u}')=P(\mathbf{u})^{\mathsf{T}}P(\mathbf{u}'). \tag*{(54)}

with the zero-radius direction defined as zero. A softmax constrains b0+b1=1b_0+b_1=1 and b0,b1>0b_0,b_1>0. The GP has a constant mean, a learned Gaussian likelihood with a log-normal noise prior, and standardized outputs, following the original spherical-linear implementation [25]. Posterior objective samples from these models are scalarized using Eq. (G.5), and the next design maximizes the LogEI acquisition in Eq. (G.6). Thus, only the surrogate changes relative to STCH.

Multi-Objective Regional Bayesian Optimization (MORBO)

MORBO maintains multiple adaptive trust regions Tℓ⊆XT_{\ell}\subseteq\mathcal{X} and fits local GP models using observations near each region. For a candidate pool Cℓ⊂TℓC_{\ell}\subset T_{\ell}, a Thompson sample f~ℓ\tilde{\mathbf{f}}_{\ell} assigns a sampled objective vector to each candidate. The sampled hypervolume improvement relative to a current nondominated set P\mathcal{P} is

ΔHVℓ(x;P)=HV(P∪{f~ℓ(x)};r)−HV(P;r),x∈Cℓ.(55)\Delta\mathrm{HV}_{\ell}(\mathbf{x};\mathcal{P})=\mathrm{HV}\left(\mathcal{P}\cup\{\tilde{\mathbf{f}}_{\ell}(\mathbf{x})\};\mathbf{r}\right)-\mathrm{HV}(\mathcal{P};\mathbf{r}),\qquad\mathbf{x}\in C_{\ell}. \tag*{(55)}

Coordinated batch selection uses these sampled improvements across regions, accounting for candidates already selected for the batch. Successful regions expand, while unsuccessful regions shrink or restart.

The BlackBox-GP MORBO variant fits local GPs to final objectives, whereas the Composite MORBO variant fits local GPs to the retained scalar entries of hi(x)\mathbf{h}_i(\mathbf{x}) and maps each posterior sample h~i\tilde{\mathbf{h}}_i through gig_i before scoring candidate hypervolume improvement. Both variants begin at the same points and use the same region state, shared past observations, and batch size. There are five trust regions, each initialized with side length 0.8, minimum 0.01, and maximum 1.6. Candidate pools contain 512 raw points. Each local output GP uses a Matérn-5/25/2 ARD kernel with length scales constrained to [0.05,4][0.05,4], output scale, and a Gaussian likelihood with a Gamma noise prior. Candidates are then selected via lower-tail winsorization and coordinated Thompson Sampling/LogEHVI.

Optimization configurations

Protocol. Each matched BlackBox-GP/Composite pair on the benchmarks uses the same trial seed and an identical scrambled-Sobol initial design, and every benchmark comparison is repeated over 10 paired trials, with 80 trials for Synthetic 1. One evaluation at a design returns all objectives, or all retained components, at once; applying the known composition to those components does not consume a further evaluation. Hypervolume is recomputed after each recorded design against the benchmark’s fixed reference point.

Models. All models are double-precision BoTorch and GPyTorch GPs [30]. Standard and spherical models are fit with fit_gpytorch_mll using exact or summed marginal log likelihoods; the configured kernel and likelihood priors enter that same fitting objective.

Surrogate models. The LogEHVI and STCH output GPs use a scaled Matérn 5/2 kernel with ARD in the controlled experiments and a dimension-scaled RBF kernel with ARD in the scientific-design benchmarks. Sp-STCH and MORBO set their covariance explicitly, as Appendices § 14.§ 3 and § 14.§ 4 describe. Table H.1 gives both configurations; absolute gains should be compared within a setting rather than across the two.

Controlled experimentsScientific-design benchmarks
BoTorch0.9.50.18.0
GPyTorch1.111.15.2
PyTorch2.11.02.12.0
CovarianceScaleKernel(Matern 5/2), ARDRBFKernel, ARD, no outputscale
Lengthscale priorGamma⁡(3.0,6.0)\operatorname{Gamma}(3.0, 6.0)LogNormal⁡(2+12log⁡d,3)\operatorname{LogNormal}(\sqrt{2} + \frac{1}{2}\log d, \sqrt{3})
Lengthscale constraint—≥0.025\ge0.025
Outputscale priorGamma⁡(2.0,0.15)\operatorname{Gamma}(2.0, 0.15)—

Table H.1. Surrogate and software configuration for the LogEHVI and STCH output GPs in each setting. Both fit hyperparameters by fit_gpytorch_mll in double precision with a Standardize outcome transform.

Evaluation budgets. Table H.2 gives the budget for each regime. The scalarization methods spend their adaptive budget across five fixed weight vectors rather than in one sequence, which is why the per-weight counts multiply out to the same totals as the sequential methods.

RegimeMethodInitialAdaptiveAdaptive budget spent as
Low-dimensionalLogEHVI540sequential, one candidate per step
Low-dimensionalSTCH5405 weights ×\times 8 evaluations
High-dimensionalSp-STCH201005 weights ×\times 20 evaluations
High-dimensionalMORBO2010020 batches ×\times 5 candidates

Table H.2. Initial design size and adaptive evaluation budget per trial. The high-dimensional regime covers L–A, Lake, and RCM40.

Acquisition optimization. LogEHVI, STCH, and Sp-STCH maximize their acquisition function with BoTorch’s optimize_acqf, which seeds a multistart search from raw Sobol candidates and refines it with SciPy’s L-BFGS-B [30]. LogEHVI and STCH use 128 raw candidates with 8 restarts, and Sp-STCH uses 256 raw candidates with 10 restarts. MORBO does not use L-BFGS-B: it sequentially selects twenty batches of five candidates from a discrete Sobol pool of 512 points drawn inside its trust regions.

References

  1. [1]Abhijith M Gopakumar, Prasanna V Balachandran, Dezhen Xue, James E Gubernatis, and Turab Lookman. Multi-objective optimization for materials discovery via adaptive design. Scientific reports, 8(1):3738, 2018.DOI
  2. [2]Joshua Knowles. Parego: A hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems. IEEE transactions on evolutionary computation, 10(1):50–66, 2006.DOI
  3. [3]Carl Edward Rasmussen and Christopher KI Williams. Gaussian processes for machine learning. Number 3. MIT press Cambridge, MA, 2006.DOI
  4. [4]Tobias Ginsburg, Kyle Hippe, Ryan Lewis, Aileen Cleary, Doga Ozgulbas, Rory Butler, Casey Stone, Abraham Stroka, Rafael Vescovi, and Ian Foster. Exploring benchmarks for self-driving labs using color matching. In Proceedings of the SC’23 Workshops of the International Conference on High Performance Computing, Network, Storage, and Analysis, pages 2147–2152, 2023.arxiv.org/abs/2310.00510
  5. [5]Sabrine Chaouch, Ali Moussa, and Neji Ladhari. Color formulation of cotton fabrics using multi-objective ant colony optimization. Journal of Natural Fibers, 19(17):1549–1547, 2022.DOI
  6. [6]Raul Astudillo and Peter Frazier. Bayesian optimization of composite functions. In International Conference on Machine Learning, pages 354–363. PMLR, 2019.arxiv.org/abs/1906.01537
  7. [7]Wesley J Maddox, Maximilian Balandat, Andrew G Wilson, and Eytan Bakshy. Bayesian optimization with high-dimensional outputs. Advances in neural information processing systems, 34:19274–19287, 2021.arxiv.org/abs/2106.12997
  8. [8]Wesley Maddox, Qing Feng, and Max Balandat. Optimizing high-dimensional physics simulations via composite bayesian optimization. arXiv preprint arXiv:2111.14911, 2021.arxiv.org/abs/2111.14911
  9. [9]Raul Astudillo and Peter Frazier. Bayesian optimization of function networks. Advances in neural information processing systems, 34:14463–14475, 2021.arxiv.org/abs/2112.15311
  10. [10]P. Buathong, J. Wan, R. Astudillo, S. Daulton, M. Balandat, and P. I. Frazier. Bayesian optimization of function networks with partial evaluations. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 4752–4784. PMLR, 2024.arxiv.org/abs/2311.02146
  11. [11]Xi Lin, Xiaoyuan Zhang, Zhiyuan Yang, Fei Liu, Zhenkun Wang, and Qingfu Zhang. Smooth tchebycheff scalarization for multi-objective optimization. arXiv preprint arXiv:2402.19078, 2024.arxiv.org/abs/2402.19078
  12. [12]TM Nogueira Pires, RP Cardoso Coelho, and FM Andrade Pires. Composite bayesian optimisation for multi-objective problems with smooth tchebycheff scalarisation. Structural and Multidisciplinary Optimization, 69(2):34, 2026.DOI
  13. [13]Donald R Jones, Matthias Schonlau, and William J Welch. Efficient global optimization of expensive black-box functions. Journal of Global optimization, 13(4):455–492, 1998.DOI
  14. [14]Peter I Frazier. Bayesian optimization. In Recent advances in optimization and modeling of contemporary problems, pages 255–278. Informs, 2018.DOI
  15. [15]Edward Snelson, Zoubin Ghahramani, and Carl Rasmussen. Warped gaussian processes. Advances in neural information processing systems, 16, 2003.
  16. [16]Sebastian Ament, Samuel Daulton, David Eriksson, Maximilian Balandat, and Eytan Bakshy. Unexpected improvements to expected improvement for bayesian optimization. Advances in neural information processing systems, 36:20577–20612, 2023.arxiv.org/abs/2310.20708
  17. [17]Samuel Daulton, Maximilian Balandat, and Eytan Bakshy. Differentiable expected hypervolume improvement for parallel multi-objective bayesian optimization. Advances in neural information processing systems, 33:9851–9864, 2020.
  18. [18]S. Surjanovic and D. Bingham. Virtual library of simulation experiments: Test functions and datasets. Retrieved November 8, 2026, from http://www.sfu.ca/~ssurjano.
  19. [19]Kalyanmoy Deb, Lothar Thiele, Marco Laumanns, and Eckart Zitzler. Scalable multi-objective optimization test problems. In Proceedings of the 2002 congress on evolutionary computation. CEC’02 (Cat. No. 02TH8600), volume 1, pages 825–830. IEEE, 2002.DOI
  20. [20]Victoria I Ward, Riddhi Singh, Patrick M Reed, and Klaus Keller. Confronting tipping points: Can multi-objective evolutionary algorithms discover pollution control tradeoffs given environmental thresholds? Environmental Modelling & Software, 73:27–43, 2015.DOI
  21. [21]Ryoji Tanabe and Hisao Ishibuchi. An easy-to-use real-world multi-objective optimization problem suite. Applied Soft Computing, 89:106078, 2020.DOI
  22. [22]Abhishek Kumar, Guohua Wu, Mostafa Z Ali, Qizhang Luo, Rammohan Mallipeddi, Ponnuthu-rai Nagaratnam Suganthan, and Swagatam Das. A benchmark-suite of real-world constrained multi-objective optimization problems and some baseline results. Swarm and Evolutionary Computation, 67:100961, 2021.DOI
  23. [23]Samuel Kim, Peter Y Lu, Charlotte Loh, Jamie Smith, Jasper Snoek, and Marin Soljačić. Deep learning for bayesian optimization of scientific problems with high-dimensional structure. arXiv preprint arXiv:2104.11667, 2021.arxiv.org/abs/2104.11667
  24. [24]Kobi C Felton, Jan G Rittig, and Alexei A Lapkin. Summit: benchmarking machine learning methods for reaction optimisation. Chemistry-Methods, 1(2):116–122, 2021.DOI
  25. [25]Colin Doumont, Donney Fan, Natalie Maus, Jacob R Gardner, Henry Moss, and Geoff Pleiss. We still don’t understand high-dimensional bayesian optimization. arXiv preprint arXiv:2512.00170, 2025.arxiv.org/abs/2512.00170
  26. [26]Samuel Daulton, David Eriksson, Maximilian Balandat, and Eytan Bakshy. Multi-objective bayesian optimization over high-dimensional search spaces. In Uncertainty in Artificial Intelligence, pages 507–517. PMLR, 2022.arxiv.org/abs/2109.10964
  27. [27]Paul Kubelka and Franz Munk. Ein beitrag zur optik der farbanstriche. Z. tech. Phys, 12:593–601, 1931.
  28. [28]Chris Wyman, Peter-Pike Sloan, and Peter Shirley. Simple analytic approximations to the cie xyz color matching functions. J. Comput. Graph. Tech, 2(2):1–11, 2013.
  29. [29]Stephen R Carpenter, Donald Ludwig, and William A Brock. Management of eutrophication for lakes subject to potentially irreversible change. Ecological applications, 9(3):751–771, 1999.DOI
  30. [30]Maximilian Balandat, Brian Karrer, Daniel Jiang, Samuel Daulton, Ben Letham, Andrew G Wilson, and Eytan Bakshy. Botorch: A framework for efficient monte-carlo bayesian optimization. Advances in neural information processing systems, 33:21524–21538, 2020.arxiv.org/abs/1910.06403

Paper details

vlnc:W_pm28wu4d
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 8

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents