WorksComputer Science
LLM-Based Evaluation Planning for Heterogeneous Optimization
Abstract
Heterogeneous optimization problems require an optimizer to decide not only which candidate solutions to evaluate, but also which objectives and constraints should be evaluated under different costs and fidelities. Existing SOTA methods such as MFE-NSGA-III address this setting with manually designed selection metrics, but their behavior is largely determined by predefined heuristic rules and hyperparameters. In this work, we propose REP-HO, a reasoning-based evaluation planner that replaces the rule-based mixed-fidelity selection step with a large language model (LLM) planner. The LLM receives the same primitive optimization-state information as the heuristic planner, such as surrogate predictions, uncertainties, evaluation costs, and budget status. We first show that REP-HO achieves competitive performance compared to the human-designed metric-based method (MFE-NSGA-III). We then further enhance each decision with an adaptive nearest-neighbor summary, leading to better and more stable results. We evaluate REP-HO across five heterogeneous regimes covering cost imbalance, constraint difficulty, surrogate fidelity, budget level, and objective conflict. Across these regimes, one of the two variants obtains the best mean ranking score in every regime, REP-HO(Deepseek) is first or second throughout, and the planner shows adaptive allocation behavior, such as prioritizing constraint evaluations under tighter budgets and reallocating constraint evaluation budget when surrogate reliability changes. These results suggest that LLMs can serve as flexible evaluation planners for heterogeneous optimization, offering a general alternative to fixed hand-designed metrics.
Introduction
Many real-world optimization problems are not homogeneous. In scientific discovery, engineering design, simulation-based decision making, and machine learning system design, different quantities of interest may require substantially different evaluation procedures, runtimes, monetary costs, or reliability levels [2]. For example, in molecular and materials design, cheap predictors can rapidly estimate drug-likeness or synthetic accessibility, whereas more reliable binding or toxicity assessments may require expensive physics-based simulations or wet-lab experiments [9]. Similarly, in modern machine learning workflows, small-scale experiments, partial training runs, and low-fidelity approximations can provide useful but imperfect information before costly full-scale evaluations are conducted (Yen et al., 2025). Such heterogeneity is not merely a practical detail, but a central factor in budget-limited optimization, and can change which evaluations are most valuable under a fixed budget [2, 6, 5]. Classical multi-objective optimization methods typically assume that all objectives and constraints are evaluated together for each candidate solution. However, when evaluation costs or fidelities differ significantly, full evaluation may waste budget on quantities that are cheap but over-sampled, expensive but uninformative, or unnecessary for candidates that are already clearly infeasible or dominated. Recent work in evolutionary optimization and Bayesian optimization has therefore shown that explicitly exploiting heterogeneous evaluation structures can substantially improve optimization efficiency, especially when expensive objectives or constraints dominate the total computational cost [17, 2, 22] (Wang et al. 2021; Xue et al. 2021; Wang et al. 2023; Santoshkumar & Deb 2026).
Existing methods address heterogeneous evaluation through specialised modelling, acquisition, transfer, or selection mechanisms, including fidelity-aware kernels, correlation-aware surrogate models, cost-aware acquisition functions, heterogeneous selection operators, and cross-objective or cross-fidelity transfer [11, 14, 22] (Xing et al. 2025; Wang et al. 2023; Santoshkumar & Deb 2026; Wang et al. 2021; Tan et al. 2023). These methods provide valuable algorithmic insights, but they are usually tailored to a particular form of heterogeneity, such as asynchronous evaluation time, unequal evaluation cost, objective-wise landscape variation, or fidelity discrepancy. Some are further restricted to bi-objective, single-objective, or unconstrained settings, making them difficult to extend directly to general multi-objective constrained problems with multiple interacting sources of heterogeneity. A particularly relevant recent method is MFE-NSGA-III (Santoshkumar & Deb, 2026), which integrates heterogeneous fidelity and evaluation cost into a unified mixed-fidelity selection metric for multi-objective constrained optimization. However, its evaluation decisions are still governed by manually predefined metrics, selection rules, and hyperparameter choices. Although effective, their behaviour is largely determined by human-designed criteria, which is complex and time-consuming, and often limit their adaptability to other problems and heterogeneities.
Multi-fidelity Bayesian optimization provides another important perspective by treating fidelity as an additional decision dimension and balancing cheap low-fidelity queries against expensive high-fidelity evaluations [8, 9, 10] (Xing et al. 2025; Yen et al. 2025). However, current multi-fidelity methods are mostly developed for single-objective or unconstrained settings, or for task-specific scenarios such as engineering simulation, data-mixture optimization for LLMs, molecular evaluation, and low-cost tuning of LLM judge designs [9, 21] (Schuetz et al. 2025; Yen et al. 2025). This specialization limits their direct transfer to general heterogeneous multi-objective constrained optimization, where objectives, constraints, fidelities, and costs may all interact under a shared budget.
To address these limitations, this work investigates whether LLMs can serve as dynamic evaluation planners for heterogeneous optimization without additional training, hyperparameter tuning or prompt optimization. At each generation, the LLM receives the same optimization-state information available to the optimizer, without privileged information or problem-specific heuristics. Further, each prompt also lists a few nearby high-fidelity infill points chosen by an adaptive nearest-neighbor rule. Using a fixed prompt template and only in-context reasoning, it plans the evaluation allocation across candidates, objectives, constraints, fidelities, and costs. We compare the proposed LLM-based planner against MFE-NSGA-III, a strong predefined-metric-based state-of-the-art baseline for multi-objective constrained optimization with heterogeneous fidelity and evaluation cost. To systematically examine its behaviour and adaptability, we design five heterogeneous regimes that emphasize fidelity, imbalanced cost, objective difficulty, and constraint difficulty respectively. Across these regimes, our results show that the LLM planner adapts its evaluation strategy to different heterogeneous structures and obtains competitive results and often better results with small token costs, (below $0.11 per run Appendix § 12), suggesting that LLMs can provide a flexible planner for heterogeneous optimization, offering a promising alternative to manually predefined metrics and problem-specific heuristic designs.
Related Work
Heterogeneous Optimization The optimization community has studied heterogeneity mainly through objective-wise evaluation time, evaluation cost, constraint cost, and heterogeneous objective landscapes. For objectives with different evaluation times, Tr-SAEA (Wang et al., 2021), SBP-BO (Wang et al., 2023), and LCE-SAEA [11] use transfer learning, search-bias penalties, or local correlation estimation to exploit the information from cheaper or faster objectives. These methods improve the use of partially evaluated information, but they are usually designed for unconstrained or bi-objective problems and rely on specific assumptions about the relation between objectives. Other works focus on cheap and expensive objectives or constraints. CHVEI/CHVPOI [14] and HHCBQ [17] modify hypervolume-based acquisition functions for bi-objective problems where one objective is cheap and the other is expensive. IOC-SAMO-COBRA [7] trains surrogates only for expensive functions and directly evaluates inexpensive ones, while SPArEacost [18] orders expensive constraints according to predicted violation and evaluation cost. These methods are effective when the cheap–expensive structure is predefined, but they often assume a fixed cost pattern, limited objective number, or specific objective–constraint roles. Heterogeneous objective landscapes have also been studied through MNK landscapes [5, 6], where they provide a controlled benchmark and analytical lens to show that objective heterogeneity fundamentally alters the local search geometry of MOO even when global pareto fronts remain similar. ExLLM [20] and MCCE [19] further proves that LLM can also be used for heterogeneous problems in optimization. The most relevant line of work is HET-NSGA-III and MFE-NSGA-III by Santoshkumar & Deb [22] (2026). They use manually designed selection metrics to decide which candidate–objective or candidate–constraint pairs should receive expensive evaluations. MFE-NSGA-III further integrates heterogeneous fidelity and evaluation cost into a unified mixed-fidelity selection metric, making it a strong baseline for heterogeneous multi-objective constrained optimization.
Methodology
Problem Definition We consider a constrained multi-objective optimization problem with heterogeneous evaluation fidelity and heterogeneous evaluation cost. Let denote a candidate solution. The problem is formulated as
Here is the number of objectives and is the number of constraints. The goal is to approximate the feasible Pareto front under a fixed evaluation budget . The budget is measured in normalized time units: one full high-fidelity evaluation of all objectives and all constraints costs one time unit. Thus, if is the high-fidelity cost of objective , and is the high-fidelity cost of constraint , then . First, each objective and constraint can be evaluated at different fidelities. A high-fidelity evaluation uses the true evaluation function and returns the actual value of or , whereas a low-fidelity evaluation often uses a surrogate model prediction. For a solution , the surrogate provides the predicted objective value, predicted constraint value, and their associated uncertainties. Second, the high-fidelity costs are imbalanced across targets: some objectives or constraints may be much more expensive than others. The algorithm must therefore decide not only which candidate solutions to evaluate, but also which subset of objectives and constraints should receive high-fidelity evaluation under the remaining budget.
Performance Metrics We evaluate the quality of the final nondominated feasible approximation set using IGD+ and hypervolume (HV). Let denote a reference set sampled from the true Pareto front. The IGD+ metric is
A smaller IGD+ value indicates that the obtained approximation set is closer to and better covers the reference Pareto front. The HV metric measures the objective-space volume dominated by and bounded by a user-defined reference point :
where denotes the Lebesgue measure. A larger HV value indicates better convergence and diversity, assuming the same reference point is used across methods.
Baselines and Preliminaries
NSGA-III is a reference-vector-guided evolutionary multi-objective optimization algorithm. It uses nondominated sorting and a reference direction based niching mechanism to maintain a diverse approximation set in high-dimensional objective spaces. In our setting, NSGA-III serves as the high-fidelity baseline: every selected solution is evaluated by the true evaluation functions for all objectives and all constraints.
SA-NSGA-III is a surrogate-assisted version of NSGA-III. It uses surrogate models to search cheaply between high-fidelity evaluations, but the final infill solutions are still evaluated in a uniform manner. That is, when a candidate is selected for high-fidelity evaluation, all objectives and all constraints are evaluated together, without exploiting target-wise cost imbalance.
MFE-NSGA-III (Santoshkumar & Deb, 2026) is based on NSGA-III for heterogeneous constrained multi-objective optimization with an important metric: mixed-fidelity selection (MFS) which incorporates the likelihood of a solution dominating its neighbors, computational cost, surrogate model error, and the extent of constraint violation. MFE-NSGA-III first identifies near-boundary candidates by -scores of constraints, then computes their MFS scores to determine which objectives or constraints of which candidates should be evaluated at high fidelity. The remaining targets are then evaluated by surrogate model (low fidelity). For each solution and constraint , the surrogate prediction and uncertainty are denoted by and , respectively. The constraint -score and overall probability of satisfying all constraints are defined as
where is the standard normal cumulative distribution function. Because constraints are written as , larger values indicate higher predicted feasibility. Using a threshold , each population member is classified by its worst constraint score: solutions with are treated as feasible or near-boundary members, while the remaining solutions are treated as largely infeasible.
For every feasible or near-boundary solution , MFE-NSGA-III computes an MFS metric for each objective :
Here , defined by Santoshkumar & Deb [22], is the probability that solution is nondominated among its neighboring members with respect to the reference vectors (from NSGA-III), is the surrogate uncertainty for objective , and is the range of the th objective in the current population. The normalized evaluation time represents the relative high-fidelity cost of objective . The hyperparameter controls the strength of the uncertainty term, and changes over the run to balance cheap and expensive target evaluations as the budget is consumed.
The selection procedure uses these quantities as follows. If enough feasible or near-boundary members are available, MFE-NSGA-III selects candidates across reference-vector neighborhoods using , evaluates the objective with the largest selected , and evaluates only those constraints whose values lie near the boundary, . If feasible members are insufficient, the remaining slots are filled by largely infeasible solutions ranked by , usually without objective evaluation. If all members are largely infeasible, the method evaluates the least violated constraint, i.e., the constraint with the largest , to guide the search toward feasibility. In this way, MFE-NSGA-III spends high-fidelity budget. The hyperparameter and are set to be 20 and -0.5 respectively, following the best settings reported in MFE-NSGA-III.
REP-HO
We propose Reasoning-based Evaluation Planning for Heterogeneous Optimization (REP-HO) as shown in Figure 1. The key difference is the evaluation-planning step. In MFE-NSGA-III, the decision of which objectives and constraints receive high-fidelity evaluation is made by fixed rules based on the constraint -score , the constraint satisfaction probability , and the mixed-fidelity selection metric . REP-HO keeps the same surrogate-assisted optimization backbone, candidate generation process, reference-vector neighborhood information, and budget accounting, but replaces this rule-based target allocation with an LLM planner. At each generation, the optimizer first constructs an infill candidate set using the same surrogate-assisted evolutionary search as MFE-NSGA-III. Let be the set of candidate solutions selected for possible evaluation at generation . For each , the LLM chooses which objective and constraint targets should be evaluated with high fidelity.

Figure 1. The workflow of REP-HO.
LLM State For each LLM call, REP-HO builds a structured state containing only information available to the optimizer at generation . The state consists of the following components: where is the current budget state, is the target-cost table, is the displayed candidate batch, and is the decision rule and output schema. The same state and action template is fixed for all problems. Only the brief problem-specific textual introduction and the dimensions and change with the benchmark; the information fields, allowed actions, output format, and batch size are kept identical across all experiments. The budget state is
where is the current consumed time, is the initial design of experiments cost, and is the total time budget. Thus the LLM is informed that objectives and constraints have unequal high-fidelity costs, and that only requested high-fidelity targets consume budget. Each candidate is shown with a stable label and the following state:
Here and are surrogate predictions, and are surrogate uncertainties, is the surrogate-based probability that is nondominated in the corresponding objective view, and is the overall constraint satisfaction probability. The binary variables and indicate whether the corresponding target has already been evaluated with high fidelity, representing repeated high-fidelity requests for the same candidate-target pair. The constraint -score is the same statistic used by MFE-NSGA-III. The prompt also states the feasibility convention and the objective direction.
For fairness, the LLM planner is given the same amount of information that is available to the MFS calculation at the selection stage: surrogate predictions, surrogate uncertainties, nondomination probabilities, constraint -scores, overall constraint satisfaction probability, already-evaluated target flags, evaluation costs, and budget status. However, REP-HO does not approximate the MFS score, and the MFS score is not included in the prompt. The LLM must therefore form its own evaluation plan from the same primitive state variables rather than observing the hand-designed MFE score. In addition, high-fidelity constraint selection is also decided by the LLM, replacing the original rule-based constraint selection based on boundary Z-scores. This design enables a direct comparison between reasoning-based planning and the predefined MFE heuristic under matched information, candidate sets, evaluation costs, and budget accounting. Each LLM query processes a fixed batch of 10 candidate solutions for all problems. If the population size is , REP-HO therefore sends five LLM queries to complete the target-planning step for one generation. The LLM is told the batch index, the number of total batches, the number of shown candidates, and the number of candidates appearing in the other batches, so that each batch is planned as part of the same generation-level budget. In our experiments, the same planner is instantiated with the official hosted versions of Qwen3.5-35B-A3B and DeepSeek-V4-Flash, reported as REP-HO(Qwen) and REP-HO(Deepseek).
LLM Action Space For each candidate , the LLM returns an action
The set contains the objectives requested for high-fidelity evaluation, and contains the constraints requested for high-fidelity evaluation. Empty sets are allowed, meaning that the candidate is kept with surrogate predictions for all non-requested targets. The complete LLM decision for a batch is . The action is returned as structured JSON with one decision for every candidate in the batch. A concrete prompt and response example is provided in Appendix § 16. After parsing the response, the optimizer evaluates only the requested candidate-target pairs with high fidelity; all non-requested objective and constraint values remain low-fidelity surrogate values for that generation.
Design of regimes
We evaluate REP-HO under five controlled experimental regimes. Unless otherwise stated, all experiments use budget , uniform evaluation costs, a fixed surrogate family. We will briefly introduce each regime in this section, the full computing details and settings are explained in Appendix § 10.
Regime 1: cost imbalance. This regime tests whether the planner adapts to imbalanced evaluation costs across objectives and constraints. We use TNK, MW3, Welded Beam, and Carside. For TNK and MW3, we compare uniform, objective-heavy, and constraint-heavy settings. Regime 2: constraint difficulty. This regime tests whether the planner adapts when the feasible region is large or small. We use TNK and Welded Beam, and their constraint difficulty is controlled by a shared shift , where is calibrated by pilot sampling: easy settings target a feasible ratio of about 25%, and hard settings target about 0.1%. Regime 3: fidelity correlation. This regime tests how the planner changes its behaviors when the surrogate model is high or low fidelity. We also use TNK and Welded Beam. The high-correlation setting uses reliable surrogate predictions, while the low-correlation setting weakens the surrogate mild noise and structural bias. The settings are calibrated on an independent reference pool using objective-wise LF-HF Spearman correlation and constraint-wise correlation/MAE. Regime 4: budget. This regime tests whether REP-HO performs genuine budget-aware planning rather than relying on a single fixed budget. We evaluate low, mid, and high budgets, , using uniform costs and high-correlation surrogates. The main problems are TNK, MW3, and Carside. Regime 5: objective conflict. This regime tests whether the planner responds to weak versus strong objective conflict. We use ZDT1 as a low-conflict unconstrained case and Kursawe as a high-conflict unconstrained case. We also use BNH as a lower-conflict constrained case and OSY as a higher-conflict constrained case.
Adaptive nearest-neighbor context
We further add a simple but general component to REP-HO, a lists a few previously evaluated infill points near the candidate being planned is provided in prompt. Initial design-of-experiment points are excluded: they are fully high-fidelity evaluated, and their coverage is already reflected by the surrogate’s information. Distance is the root-mean-square distance in the bound-normalized design space. Let be the distance to the nearest remaining historical infill. A neighbor is kept only if its distance is at most , and at most three neighbors are shown. If fewer points falls inside this this radius the list is shorter, and if no later infill exists the list is empty. Only high-fidelity values of those neighbors are shown, low fidelity values are shown as not evaluated. The same planner is instantiated with Qwen3.5-35B-A3B and DeepSeek-V4-Flash.
Results
Baseline results We first evaluate REP-HO without Adaptive Nearest-Neighbor Context (ANNC) to isolate the performance of pure LLM-based planning under the same state information. As shown in Table 1, REP-HO (Qwen) already achieves competitive performance, ranking first across all regimes except Regime 5. We then incorporate ANNC into REP-HO and further investigate how it adapts to different forms of heterogeneity.
| Regime | NSGA-III | SA-NSGA-III | MFE-NSGA-III | REP-HO (Qwen) |
| 1 | 3.625 | 3.208 | 1.667 | 1.500 |
| 2 | 2.875 | 2.750 | 1.875 | 1.375 |
| 3 | – | 2.875 | 1.875 | 1.250 |
| 4 | 3.333 | 3.333 | 2.083 | 1.250 |
| 5 | 3.292 | 3.229 | 1.708 | 1.771 |
Table 1. Overall ranking scores without the adaptive nearest-neighbor context. Lower is better.
Regime 1 For each setting, we run each method with 20 random seeds and report the mean and standard deviation. For each task, the best-performing method receives a rank of 1, the second-best receives a rank of 2, and so on. The final ranking score is computed as the average rank across all experiments, where a lower score indicates better overall performance. Table 2 reports the cost-imbalance results with two planner backbones. REP-HO(Deepseek) obtains the best mean ranking (1.958), followed by REP-HO(Qwen) (2.125) and MFE-NSGA-III (2.188); SA-NSGA-III (4.167) and NSGA-III (4.562) remain far behind. The two planners split the wins: REP-HO(Deepseek) is strongest on MW3 and on constraint-heavy TNK, while REP-HO(Qwen) leads IGD+ on the skewed Carside settings and on uniform and objective-heavy TNK. MFE-NSGA-III still has the best HV on all three Carside settings and the best HV and IGD+ on constraint-heavy Welded Beam. Both REP-HO variants and MFE-NSGA-III clearly outperform NSGA-III and SA-NSGA-III, so the gain comes from target-wise heterogeneous planning.
| Problem | Setting | NSGA-III | SA-NSGA-III | MFE-NSGA-III | REP-HO(Qwen) | REP-HO(Deepseek) |
| Carside | uniform | |||||
| Carside | constraint-heavy | |||||
| Carside | objective-heavy | |||||
| MW3 | constraint-heavy | |||||
| MW3 | uniform | |||||
| MW3 | objective-heavy | |||||
| TNK | constraint-heavy | |||||
| TNK | uniform | |||||
| TNK | objective-heavy | |||||
| WeldedBeam | constraint-heavy | |||||
| WeldedBeam | uniform | |||||
| WeldedBeam | objective-heavy | |||||
| Mean ranking integrated with IGD+ in Table 7 | 4.562 | 4.167 | 2.188 | 2.125 | 1.958 |
Table 2. Regime 1: Cost Imbalance HV performance summary (HV ↑).
Regime 2 Table 3 summarizes the easy and hard constraint settings. REP-HO(Qwen) obtains the best mean ranking (2.000), just ahead of REP-HO(Deepseek) (2.062), then MFE-NSGA-III (2.375), NSGA-III (3.312), and SA-NSGA-III (3.938). On easy TNK every method reaches IGD+ 0, but REP-HO(Qwen) is far ahead on HV. On hard TNK, REP-HO(Qwen) has the best IGD+ while NSGA-III has the best HV. On Welded Beam, REP-HO(Deepseek) has the best IGD+ at both difficulties, and REP-HO(Qwen) has the best HV.
| Problem | Constraint Difficulty | NSGA-III | SA-NSGA-III | MFE-NSGA-III | REP-HO(Qwen) | REP-HO(Deepseek) |
| TNK | hard | |||||
| TNK | easy | |||||
| WeldedBeam | hard | |||||
| WeldedBeam | easy | |||||
| Mean ranking integrated with IGD+ in Table 9 | 3.312 | 3.938 | 2.375 | 2.000 | 2.062 |
Table 3. Regime 2: Constraint Difficulty HV performance summary (HV ↑).
We further analyse the allocation strategies learned by different methods. As shown in Figure 2, the methods exhibit similar allocation patterns over objectives, but differ substantially in constraint allocation. To quantify constraint difficulty, we report the near-boundary rate under different values in Table 8; a larger indicates that a constraint is more frequently close to the feasibility boundary and thus requires more evaluation budget to resolve. For TNK under high constraint difficulty, and have values of 0.0769 and 0.1247, respectively. For WeldedBeam, the corresponding values for – are 0.8425, 0.6648, 0.3616, and 0.0167. When the constraint difficulty increases, constraint allocation clearly increases in the early stage for both methods on both problems, indicating that both methods respond to constraint difficulty and allocate early budget to improve feasibility. While MFE-NSGA-III tends to ignore some constraints such as in TNK and in WeldedBeam, REP-HO allocates a non-negligible budget to it, which is more consistent with the estimated constraint difficulty. Figure 3 further shows that REP-HO achieves better convergence of the feasibility-rate than the baselines.

Figure 2. Cumulative allocations on objectives and constraints, the first row is TNK and second row is WeldedBeam problem; the left images are of low constraint difficulty and the right images are of high constraint difficulty.

Figure 3. Feasibility rate convergence, left: TNK with high constraint difficulty, right: WeldedBeam with high constraint difficulty
| Problem | Level | Feasible ratio | Near-boundary rate by constraint | |
| TNK | Easy | 1.80 | 0.2546 | : 0.0000, : 0.0624 |
| TNK | Normal | 0.00 | 0.0499 | : 0.0766, : 0.0860 |
| TNK | Hard | -0.45 | 0.0111 | : 0.0769, : 0.1247 |
| Welded Beam | Easy | 0.05 | 0.3560 | : 0.1340, : 0.1639, : 0.7463, : 0.0239 |
| Welded Beam | Normal | 0.00 | 0.3190 | : 0.1473, : 0.1793, : 0.7500, : 0.0224 |
| Welded Beam | Hard | -0.65 | 0.0011 | : 0.8425, : 0.6648, : 0.3616, : 0.0167 |
Table 8. Constraint-difficulty calibration for Regime 2. Feasible ratios and near-boundary rates are estimated from the 30,000-point LHS pilot scan.
| Problem | Surrogate Fidelity | SA-NSGA-III | MFE-NSGA-III | REP-HO(Qwen) | REP-HO(Deepseek) |
| TNK | high | ||||
| TNK | low | ||||
| WeldedBeam | high | ||||
| WeldedBeam | low | ||||
| Mean ranking integrated with IGD+ in Table 11 | 3.750 | 2.188 | 1.750 | 2.125 |
Table 4. Regime 3: Fidelity Correlation HV performance summary (HV ↑).
Regime 3 In Table 4, REP-HO(Qwen) has the best mean ranking (1.750), ahead of REP-HO(Deepseek) (2.125) and MFE-NSGA-III (2.188). It leads both metrics on high-fidelity TNK and low-fidelity Welded Beam. Under the low-fidelity surrogate setting, both MFE-NSGA-III and REP-HO allocate less budget to constraints than under the high-fidelity setting. This suggests that lower LF-HF correlation does not simply trigger more constraint checking. Instead, when the surrogate becomes less reliable, constraint-side ranking information becomes weaker, so the marginal value of repeatedly upgrading constraint predictions to high fidelity can decrease as shown in Figure 4. This effect is especially clear on Welded Beam. In Table 10, we further compute the Spearman correlation for each objective and constraint under both setting. In the high-fidelity setting, REP-HO allocates most constraint evaluations to and , while and receive little budget. This is consistent with the calibration results in Table 10: and have very high Spearman correlations under high fidelity, so their surrogate predictions are already reliable. Under low fidelity, however, and degrade sharply, while and retain relatively stronger correlation. As a result, REP-HO shifts part of the constraint budget away from and toward , producing a more distributed constraint-allocation pattern. Overall, this indicates that REP-HO is not merely increasing constraint evaluations when fidelity worsens, it also adapts allocation according to where the surrogate still provides useful planning signal.

Figure 4. Cumulative allocations on objectives and constraints, the first row is TNK and second row is WeldedBeam problem; the left images are of high surrogate fidelity (easy) and the right images are of low surrogate fidelity (hard).
Regime 4 In table 5, REP-HO(Deepseek) obtains the best mean ranking (1.889), followed by REP-HO(Qwen) (2.167) and MFE-NSGA-III (2.639). REP-HO(Deepseek) leads MW3 at 200 on both metrics and TNK at 400 on both metrics, and it has the best HV on Welded Beam at 600. REP-HO(Qwen) leads TNK at 200. NSGA-III is best only on Welded Beam at 200. Since the total budget differs across settings, we compare objective/constraint allocation percentages rather than raw counts. When the budget is small, both MFE-NSGA-III and REP-HO allocate a larger fraction of their early planning evaluations to constraints. For example, under , the early constraint share of REP-HO is 58.7% on MW3, 50.1% on TNK, and 57.8% on Welded Beam. Under , these shares decrease to 51.3%, 31.7%, and 56.5%, respectively. A similar trend appears for MFE-NSGA-III, although its constraint share is generally lower. This pattern indicates that, under tighter budgets, the planner prioritizes rapid feasibility identification. With fewer opportunities for later correction, early high-fidelity evaluations are used more aggressively to resolve constraint uncertainty and avoid spending the limited budget on infeasible regions. In contrast, when , the planner can afford a more gradual search: constraint allocation remains important in the early stage, but the later stages shift more budget toward objectives and refinement of the Pareto front. Thus, REP-HO appears to behave more selectively under low budget and more comprehensively under high budget, consistent with a budget-aware planning strategy rather than a fixed allocation rule.
Regime 5 Table 13 separates the test problems into lower-conflict cases (ZDT1 and BNH) and higher-conflict cases (Kursawe and OSY) using Pareto-front geometry. ZDT1 and BNH have nearly flat fronts, with very small mean second-difference and turning-angle values, whereas Kursawe and OSY show much stronger curvature and nonlinear trade-off structure. The results in Table 6 shows that REP-HO(Deepseek) has the best mean ranking (1.958), slightly ahead of MFE-NSGA-III (2.062), with REP-HO(Qwen) third (2.708). REP-HO(Deepseek) is best on every OSY setting. On the lower-conflict problems, MFE-NSGA-III and the two planners remain close. Overall Objective-conflict geometry is not where REP-HO gains its largest advantage, but REP-HO(Deepseek) is still the strongest method overall in this regime.
| Problem | Setting | NSGA-III | SA-NSGA-III | MFE-NSGA-III | REP-HO(Qwen) | REP-HO(Deepseek) |
| BNH | constraint-heavy | 3.7813 0.0121 | 3.8042 0.0002 | 3.8044 0.0001 | 3.8044 0.0001 | 3.8044 0.0001 |
| BNH | uniform | 3.7813 0.0121 | 3.8042 0.0002 | 3.8045 0.0000 | 3.8044 0.0001 | 3.8044 0.0001 |
| BNH | objective-heavy | 3.7813 0.0121 | 3.8042 0.0002 | 3.8044 0.0001 | 3.8043 0.0002 | 3.8044 0.0001 |
| Kursawe | -heavy | 2.6705 0.1693 | 2.3337 0.6176 | 2.4251 0.3712 | 2.6131 0.2896 | 2.6175 0.3333 |
| Kursawe | uniform | 2.6705 0.1693 | 2.3337 0.6176 | 2.5635 0.4923 | 2.4755 0.5329 | 2.4127 0.7118 |
| Kursawe | -heavy | 2.6705 0.1693 | 2.3337 0.6176 | 2.6025 0.3051 | 2.5619 0.3585 | 2.4714 0.2334 |
| OSY | constraint-heavy | 2.1089 0.3122 | 3.5004 0.2007 | 3.6209 0.2162 | 3.6073 0.0814 | 3.6257 0.0757 |
| OSY | uniform | 2.1089 0.3122 | 3.5004 0.2007 | 3.6098 0.2154 | 3.5980 0.0718 | 3.6239 0.0745 |
| OSY | objective-heavy | 2.1089 0.3122 | 3.5004 0.2007 | 3.5602 0.2110 | 3.5967 0.0703 | 3.6015 0.0730 |
| ZDT1 | -heavy | 0.0403 0.0999 | 3.4211 0.1563 | 3.5321 0.1595 | 3.5132 0.1515 | 3.5360 0.1676 |
| ZDT1 | uniform | 0.0403 0.0999 | 3.4211 0.1563 | 3.5181 0.1361 | 3.5466 0.1040 | 3.5377 0.1511 |
| ZDT1 | -heavy | 0.0403 0.0999 | 3.4211 0.1563 | 3.5829 0.0371 | 3.5196 0.1724 | 3.5660 0.1038 |
| Ranking score | 4.042 | 4.229 | 2.062 | 2.708 | 1.958 |
Table 6. Regime 5: Objective Conflict performance summary.
| Problem | Mean 2nd diff | Mean turning angle (deg) |
| ZDT1 | 0.000011 | 0.031107 |
| Kursawe | 0.007836 | 4.232093 |
| OSY | 0.005388 | 4.605905 |
| BNH | 0.000392 | 0.912159 |
Table 13. Objective-conflict diagnostics for Regime 5. Larger front curvature and turning angle indicate stronger conflict/nonlinearity in the trade-off structure.
We also provide an additional ablation study to test the LLM planner’s effect: a budget-matched random allocator, given the same candidate set and generation-level budget, ranks behind both REP-HO variants on Regime 1 (overall score 3.250, against 2.250 for REP-HO(Deepseek) and 2.417 for REP-HO(Qwen); Appendix § 11). The cost of REP-HO is less than 0.11 dollar per experiment, as reported in Appendix § 12, making it a cost-efficient alternative to human-designed metrics.
Conclusion
REP-HO uses an off-the-shelf LLM as a flexible planner for which objectives and constraints to evaluate under heterogeneous cost and fidelity. It keeps the surrogate-assisted backbone of MFE-NSGA-III, replaces the hand-designed mixed-fidelity rule with in-context planning, and conditions each decision on a short adaptive nearest-neighbor summary of nearby high-fidelity infill points. Across five regimes and two open-weight backbones, REP-HO obtains the best mean ranking in every regime, while the added API cost stays small. These results show that an LLM can serve as a flexible alternative to fixed, heterogeneity-specific selection metrics.
Acknowledgements
This work is supported by the Engineering and Physical Sciences Research Council [EP/Y030826/1]. The supervisory team of Nian Ran is: Richard Allmendinger, Mauricio Alvarez, Markus Olhofer, Sebastian Brulin.
AI Use Statement
In this work, we did not use generative AI tools for any tasks requiring mandatory disclosure. In particular, generative AI was not used to develop theoretical or conceptual frameworks, formulate mathematical claims, propose or refine hypotheses, design the research methodology or experiments, implement the proposed methods, analyze data, or interpret results.
We used generative AI tools only for tasks for which disclosure is recommended: editing the manuscript to improve grammar, clarity, and readability, and assisting with code organization and cleanup. The AI-assisted code changes were limited to restructuring, formatting, and improving code readability, and did not involve the design or implementation of the proposed methods. All AI-assisted text and code were reviewed by the authors.
We take full responsibility for the final content of this work, including all text, claims, code, and other artifacts produced with the aid of generative AI.
Reproducibility
We release our anonymous code at https://anonymous.4open.science/r/REP-HO-A870/README.md. The repository contains the implementation of REP-HO, baseline methods, experimental regimes, configuration files, and scripts for generating the main tables and figures. The LLM backbones used in our experiments are the official hosted versions of Qwen3.5-35B-A3B and DeepSeek-V4-Flash. All query parameters are kept at their official default values, except that we set reasoning=effort=none to avoid unnecessary overthinking and to keep the planner behavior consistent across runs. And for all problems the prompt template, parameters are all fixed and recorded in code, demonstrating the generalizability and flexibility of our method. Token use, estimated API cost, and wall-clock time are reported in Appendix § 12.
Additional Related work
LLMs for Optimization Planning. Recent work has begun to explore the use of large language models as decision-making components within optimization loops. In Bayesian optimization, LLMs have been used to improve warm-starting, surrogate modelling, candidate generation, and representation learning from textual or structured problem descriptions [13, 3] (Zeng et al. 2026). These methods show that pretrained language models can provide useful prior knowledge for expensive black-box optimization, but the LLM is usually used as a surrogate, representation model, initializer, or candidate proposer rather than as an online planner for evaluation decisions. More closely related to our motivation, recent work has considered LLMs for controlling parts of the optimization process. For example, LMABO [15] uses an LLM to select acquisition functions online from the current Bayesian optimization state, while FunBO [1] uses LLMs to discover acquisition functions that can replace manually designed rules. Similarly, in evolutionary and heuristic optimization, LLMs have been used to generate search operators, heuristics, or neighborhood-selection policies [12] (Yao et al. 2025; Ye et al. 2025). These studies suggest that LLMs can act not only as black-box optimizers, but also as reasoning modules that replace hand-crafted search or control rules.
However, existing LLM-based optimization methods are mostly studied in single-objective, unconstrained, or task-specific settings, and rarely address heterogeneous evaluation allocation. A closely related recent work is LECMO [4], which applies an LLM to expensive constrained multi-objective optimization. However, its LLM is mainly used to approximate objective and constraint values within a fixed two-stage evolutionary framework, rather than to decide which heterogeneous evaluations should be performed. In contrast, our work investigates whether an off-the-shelf LLM can serve as an online evaluation planner for heterogeneous multi-objective constrained optimization, reasoning over candidates, objectives, constraints, fidelities, costs, and remaining budget without additional training or problem-specific tuning.
Detail of designing five regimes
This appendix gives the full design of the five experimental regimes. Unless a regime explicitly changes a factor, the default configuration is:
total budget ;
uniform evaluation-cost setting;
fixed surrogate family across all compared methods;
All cost vectors are normalized so that evaluating all objectives and constraints once costs one time unit as shown in § 3.
| Problem | Setting | NSGA-III | SA-NSGA-III | MFE-NSGA-III | REP-HO(Qwen) | REP-HO(Deepseek) |
| Carside | uniform | 0.0532 ± 0.0167 | 0.0031 ± 0.0015 | 0.0010 ± 0.0008 | 0.0013 ± 0.0009 | 0.0015 ± 0.0009 |
| Carside | constraint-heavy | 0.0532 ± 0.0167 | 0.0031 ± 0.0015 | 0.0010 ± 0.0005 | 0.0010 ± 0.0008 | 0.0013 ± 0.0006 |
| Carside | objective-heavy | 0.0532 ± 0.0167 | 0.0026 ± 0.0012 | 0.0011 ± 0.0006 | 0.0010 ± 0.0005 | 0.0013 ± 0.0007 |
| MW3 | constraint-heavy | 0.4049 ± 0.1509 | 0.1058 ± 0.0857 | 0.0486 ± 0.0748 | 0.0584 ± 0.0718 | 0.0407 ± 0.0246 |
| MW3 | uniform | 0.4049 ± 0.1509 | 0.1058 ± 0.0857 | 0.0568 ± 0.0725 | 0.0516 ± 0.0736 | 0.0372 ± 0.0277 |
| MW3 | objective-heavy | 0.4049 ± 0.1509 | 0.1058 ± 0.0857 | 0.0550 ± 0.0745 | 0.0447 ± 0.0292 | 0.0295 ± 0.0278 |
| TNK | constraint-heavy | 0.0841 ± 0.0275 | 0.0333 ± 0.0408 | 0.0190 ± 0.0158 | 0.0100 ± 0.0096 | 0.0068 ± 0.0007 |
| TNK | uniform | 0.0841 ± 0.0275 | 0.0333 ± 0.0408 | 0.0224 ± 0.0166 | 0.0072 ± 0.0007 | 0.0100 ± 0.0091 |
| TNK | objective-heavy | 0.0841 ± 0.0275 | 0.0333 ± 0.0408 | 0.0236 ± 0.0169 | 0.0100 ± 0.0088 | 0.0133 ± 0.0136 |
| WeldedBeam | constraint-heavy | 0.0916 ± 0.0319 | 0.2065 ± 0.1348 | 0.0605 ± 0.0526 | 0.1049 ± 0.1320 | 0.1758 ± 0.3441 |
| WeldedBeam | uniform | 0.0916 ± 0.0319 | 0.4889 ± 0.4804 | 0.0743 ± 0.0638 | 0.0870 ± 0.0676 | 0.0822 ± 0.0486 |
| WeldedBeam | objective-heavy | 0.0916 ± 0.0319 | 0.4889 ± 0.4804 | 0.1196 ± 0.1783 | 0.2241 ± 0.4696 | 0.0885 ± 0.0645 |
| Mean ranking integrated with HV in Table 2 | 4.562 | 4.167 | 2.188 | 2.125 | 1.958 |
Table 7. Regime 1: Cost Imbalance IGD+ performance summary (IGD+ ↓).
Regime 1: Cost Imbalance
This regime varies the distribution of high-fidelity evaluation cost across objectives and constraints. The purpose is to test whether the planner adapts to objective-heavy, constraint-heavy, and uniform cost structures without depending on a fixed hand-designed metric preference.
TNK and MW3. TNK and MW3 have two objectives and two constraints in this regime. We use:
Here is uniform, is objective-heavy, and is constraint-heavy.
Welded Beam. The intended design pattern is two objectives and several constraints. Welded Beam has two objectives and four constraints, so the cost vector is normalized over six evaluation channels. The main-text settings are:
These are normalized versions of the desired uniform, objective-heavy, and constraint-heavy patterns.
Carside. Carside is evaluated as a three-objective, ten-constraint problem. The main settings are:
Additional Results
Regime 2: Constraint Difficulty
This regime varies the size of the feasible region while holding cost, surrogate fidelity, and budget fixed. The main problems are TNK and Welded Beam, with uniform costs:
Each original inequality constraint is shifted by a shared slack parameter :
Larger relaxes the constraints and increases the feasible region, whereas smaller tightens them. The value of is chosen by scanning. For each problem, we draw 30,000 Latin hypercube samples and scan a problem-specific grid of values. For each candidate , the pilot feasible ratio is
The easy setting selects the scanned whose feasible ratio lies in 25% and is closest to the midpoint of that band. The hard setting selects the scanned whose feasible ratio lies in 0.1% or 1% and is closest to the midpoint of that band. The reason we choose that small ratio for hard settings is that we found that these ratios can effectively increase the difficulty of increasing feasibility, while a larger value such as 5% or 3% is not effective enough. We additionally near-boundary rate of each constraint to classify the constraints into hard and easy constraints, here, “hard” means that a constraint is frequently near the boundary and therefore requires more frequent high-fidelity evaluation. The selected and its information are recorded in Table 8.
Additional Results
| Problem | Constraint Difficulty | NSGA-III | SA-NSGA-III | MFE-NSGA-III | REP-HO(Qwen) | REP-HO(Deepseek) |
| TNK | hard | 0.4224 ± 0.0548 | 0.3613 ± 0.0476 | 0.3437 ± 0.0295 | 0.3376 ± 0.0100 | 0.3457 ± 0.0372 |
| TNK | easy | 0.0000 ± 0.0000 | 0.0000 ± 0.0000 | 0.0000 ± 0.0000 | 0.0000 ± 0.0000 | 0.0000 ± 0.0000 |
| WeldedBeam | hard | 0.5541 ± 0.0572 | 0.4981 ± 0.0413 | 0.4789 ± 0.0582 | 0.4965 ± 0.0809 | 0.4756 ± 0.0390 |
| WeldedBeam | easy | 0.0943 ± 0.0445 | 0.3056 ± 0.6120 | 0.1770 ± 0.4074 | 0.1130 ± 0.1630 | 0.0666 ± 0.0529 |
| Mean ranking integrated with IGD+ in Table 3 | 3.312 | 3.938 | 2.375 | 2.000 | 2.062 |
Table 9. Regime 2: Constraint Difficulty IGD+ performance summary (IGD+ ↓).
Regime 3: Fidelity Correlation
This regime controls how reliable the low-fidelity surrogate is. The main problems are TNK and Welded Beam, with uniform costs and . The high-correlation setting uses a stronger low-fidelity model, while the low-correlation setting weakens the model through model choice and mild label degradation with Gaussian noise. Calibration is performed with a fixed surrogate family and an independent reference pool. Let denote true high-fidelity values for target on the reference pool, and let denote the corresponding surrogate predictions. For each objective, we compute
For each constraint, we compute the analogous Spearman correlation and the mean absolute error:
The calibration target is mean objective Spearman correlation above 0.8 for high-correlation settings and approximately 0.3 for low-correlation settings. Constraint Spearman and constraint MAE are reported to verify that the constraint-side low-fidelity quality changes consistently. In implementation, candidate fidelity configurations are generated by varying the model choice, the label-noise level, and a mild structural bias term. The selected settings and the resulting per-target correlations are shown together in Table 10.
| Problem | Fidelity | Model | Noise std. | Bias strength | Objective Spearman | Constraint Spearman |
| TNK | High | kriging-const | 0.2 | 1.0 | , (mean 0.7752) | , (mean 0.7868) |
| TNK | Low | kriging-const | 1.0 | 5.0 | , (mean 0.2637) | , (mean 0.3020) |
| Welded Beam | High | kriging-lin | 0.0 | 0.0 | , (mean 0.7466) | , , , (mean 0.8103) |
| Welded Beam | Low | kriging-const | 1.0 | 1.0 | , (mean 0.3025) | , , , (mean 0.3723) |
Table 10. Surrogate-fidelity calibration for Regime 3. Each setting uses . The per-target entries report LF-HF Spearman correlations for individual objectives and constraints, with the corresponding mean shown in parentheses.
In this regime, NSGA-III is removed from comparison because it is unfair to put it here because it is not affected by surrogate fidelity.
Additional analysis
| Problem | Surrogate Fidelity | SA-NSGA-III | MFE-NSGA-III | REP-HO(Qwen) | REP-HO(Deepseek) |
| TNK | high | 0.4083 ± 0.0937 | 0.3396 ± 0.0926 | 0.3044 ± 0.0806 | 0.3469 ± 0.1026 |
| TNK | low | 0.5767 ± 0.0292 | 0.5631 ± 0.0825 | 0.4813 ± 0.1105 | 0.5544 ± 0.1119 |
| WeldedBeam | high | 0.3335 ± 0.4246 | 0.0697 ± 0.0297 | 0.0919 ± 0.0621 | 0.0761 ± 0.0524 |
| WeldedBeam | low | 0.8589 ± 0.6880 | 0.7401 ± 0.5475 | 0.6253 ± 0.5352 | 0.7342 ± 0.8379 |
| Mean ranking integrated with IGD+ in Table 4 | 3.750 | 2.188 | 1.750 | 2.125 |
Table 11. Regime 3: Fidelity Correlation IGD+ performance summary (IGD+ ).
Regime 4: Budget
This regime varies the total time budget while holding cost and surrogate fidelity fixed. The main design uses TNK, MW3, and Carside with:
The mid-budget case is with the global default. The low-budget and high-budget cases test whether the planner becomes more selective when evaluations are scarce and more comprehensive when evaluations are plentiful. Costs are set to the uniform setting and fidelity correlation is set to high-correlation.
We report final IGD+, HV, feasible rate, objective/constraint budget share, and allocation entropy. For a generation-level allocation distribution over targets, entropy is computed as
Lower entropy indicates a more concentrated plan, while higher entropy indicates more even target coverage. We also report sparsity, defined as the fraction of candidate-target pairs not selected for high-fidelity evaluation.
Additional Analysis

Figure 5. Cumulative allocations on objectives and constraints, the first row is TNK, the second row is MW3, and third row is WeldedBeam problem; the left images are with 200 time unit budget and the right images are with 400 time unit.
Regime 5: Objective Conflict
This regime varies the conflict structure among objectives. The unconstrained main comparison uses ZDT1 as a low-conflict representative and Kursawe as a high-conflict representative. The constrained comparison uses BNH as a lower-conflict representative and OSY as a higher-conflict representative.
The default uniform-cost settings are:
ZDT1: ,
Kursawe: ,
BNH: ,
OSY: .
Additional cost-skewed experiments are:
ZDT1: , ,
Kursawe: , ,
OSY objective-heavy: ,
BNH objective-heavy: ,
OSY constraint-heavy: ,
BNH constraint-heavy: .
Objective conflict is characterized using the geometry of the Pareto front. This is the criterion used for this regime, rather than Spearman correlation or rank disagreement between objectives because they are often insufficient in identifying the complex conflicting nature of the objectives. The front is first sorted by and normalized objective-wise. Let the normalized front points be . We compute the mean second-difference norm,
and the mean turning angle between consecutive front segments,
Larger values indicate a more nonlinear and conflict-sensitive front, which requires finer discrimination between objective trade-offs.
Based on Table 13, ZDT1 and BNH are used as lower-conflict cases because their Pareto fronts are close to smooth and weakly curved. ZDT1 has an almost zero mean second-difference value (0.000011) and a very small mean turning angle (), indicating a nearly linear local front geometry. BNH is also relatively mild, with a mean second difference of 0.000392 and a mean turning angle of . In contrast, Kursawe and OSY show much stronger front nonlinearity. Kursawe has a mean second difference of 0.007836 and a mean turning angle of , while OSY has a mean second difference of 0.005388 and the largest mean turning angle, . We therefore treat Kursawe and OSY as higher-conflict cases.
Additional Results
Budget-matched random allocation
The random allocator uses the same candidate set and the same generation-level high-fidelity budget as REP-HO, but selects unevaluated candidate–objective and candidate–constraint pairs uniformly at random until that budget is exhausted. Table 15 reports its Regime 1 results under the same HV and IGD+ definitions as the main tables. Ranking scores below are recomputed on these 12 settings with the random allocator included, so they are not the five-method scores in Table 2. Lower is better. Both REP-HO variants and MFE-NSGA-III remain ahead of random allocation, while random allocation itself remains ahead of NSGA-III and SA-NSGA-III.
| Problem | Setting | HV | IGD+ |
| Carside | uniform | 8.5781 ± 0.0615 | 0.0018 ± 0.0016 |
| Carside | constraint-heavy | 8.6521 ± 0.0523 | 0.0012 ± 0.0008 |
| Carside | objective-heavy | 8.6366 ± 0.0483 | 0.0009 ± 0.0008 |
| MW3 | constraint-heavy | 3.1401 ± 0.2353 | 0.0579 ± 0.0516 |
| MW3 | uniform | 3.1514 ± 0.1915 | 0.0510 ± 0.0360 |
| MW3 | objective-heavy | 3.1327 ± 0.2020 | 0.0521 ± 0.0338 |
| TNK | constraint-heavy | 3.2528 ± 0.0882 | 0.0125 ± 0.0142 |
| TNK | uniform | 3.2370 ± 0.0792 | 0.0167 ± 0.0145 |
| TNK | objective-heavy | 3.1807 ± 0.2292 | 0.0288 ± 0.0529 |
| WeldedBeam | constraint-heavy | 3.2279 ± 1.1061 | 0.1199 ± 0.1100 |
| WeldedBeam | uniform | 3.4880 ± 0.3572 | 0.1514 ± 0.1210 |
| WeldedBeam | objective-heavy | 3.6152 ± 0.2149 | 0.0997 ± 0.0622 |
Table 15. Regime 1 budget-matched random allocation (HV , IGD+ ).
| Problem | Setting | NSGA-III | SA-NSGA-III | MFE-NSGA-III | REP-HO(Qwen) | REP-HO(Deepseek) |
| BNH | constraint-heavy | 0.0098 ± 0.0019 | 0.0059 ± 0.0001 | 0.0058 ± 0.0000 | 0.0058 ± 0.0001 | 0.0058 ± 0.0001 |
| BNH | uniform | 0.0098 ± 0.0019 | 0.0059 ± 0.0001 | 0.0057 ± 0.0000 | 0.0058 ± 0.0000 | 0.0058 ± 0.0001 |
| BNH | objective-heavy | 0.0098 ± 0.0019 | 0.0059 ± 0.0001 | 0.0058 ± 0.0001 | 0.0058 ± 0.0001 | 0.0058 ± 0.0000 |
| Kursawe | -heavy | 0.1804 ± 0.0453 | 0.3471 ± 0.2232 | 0.2792 ± 0.1203 | 0.2300 ± 0.0864 | 0.2204 ± 0.0883 |
| Kursawe | uniform | 0.1804 ± 0.0453 | 0.3471 ± 0.2232 | 0.2622 ± 0.1750 | 0.2975 ± 0.2353 | 0.3121 ± 0.3094 |
| Kursawe | -heavy | 0.1804 ± 0.0453 | 0.3471 ± 0.2232 | 0.2292 ± 0.0967 | 0.2631 ± 0.1310 | 0.2827 ± 0.0846 |
| OSY | constraint-heavy | 0.5655 ± 0.1651 | 0.0659 ± 0.0787 | 0.0305 ± 0.0849 | 0.0304 ± 0.0213 | 0.0246 ± 0.0168 |
| OSY | uniform | 0.5655 ± 0.1651 | 0.0659 ± 0.0787 | 0.0334 ± 0.0845 | 0.0322 ± 0.0193 | 0.0259 ± 0.0172 |
| OSY | objective-heavy | 0.5655 ± 0.1651 | 0.0659 ± 0.0787 | 0.0468 ± 0.0824 | 0.0332 ± 0.0179 | 0.0324 ± 0.0170 |
| ZDT1 | -heavy | 1.8220 ± 0.1522 | 0.0736 ± 0.0494 | 0.0387 ± 0.0506 | 0.0419 ± 0.0463 | 0.0390 ± 0.0538 |
| ZDT1 | uniform | 1.8220 ± 0.1522 | 0.0736 ± 0.0494 | 0.0355 ± 0.0438 | 0.0357 ± 0.0311 | 0.0360 ± 0.0538 |
| ZDT1 | -heavy | 1.8220 ± 0.1522 | 0.0736 ± 0.0494 | 0.0203 ± 0.0090 | 0.0419 ± 0.0588 | 0.0248 ± 0.0326 |
| Mean ranking integrated with IGD+ in Table 6 | 4.042 | 4.229 | 2.062 | 2.708 | 1.958 |
Table 14. Regime 5: Objective Conflict IGD+ performance summary (IGD+ ).
| Method | Mean ranking score |
| REP-HO(Deepseek) | 2.250 |
| REP-HO(Qwen) | 2.417 |
| MFE-NSGA-III | 2.521 |
| Random | 3.250 |
| SA-NSGA-III | 5.167 |
| NSGA-III | 5.396 |
Table 16. Regime 1 mean ranking score with the random allocator included. Lower is better.
LLM call cost
Table 17 summarizes every completed agg run of REP-HO(Qwen) and REP-HO(Deepseek) across the five regimes (41 settings, 20 seeds each). Dollar cost is not stored in the logs. It is estimated from the recorded token counts using published list prices: DeepSeek-V4-Flash at the official off-peak cache-miss rate of $0.15 / $0.60 per million input/output tokens, and Qwen3.5-35B-A3B at $0.14 / $1.00 per million input/output tokens.
| Backbone | Calls | Input tokens | Output tokens | Est. cost (USD) | Wall-clock (s) |
| REP-HO(Qwen) | |||||
| REP-HO(Deepseek) |
Table 17. Per-run LLM usage, mean standard deviation.
Pareto Fronts

Figure 6. Pareto front of the problems in regime 1.
Broader impacts
This work studies whether large language models can be used as flexible evaluation planners for heterogeneous optimization problems. A positive societal impact is that such planners may reduce the design complexity and increase the efficiency of expensive optimization tasks in engineering design, scientific discovery, and machine learning system development. By allocating high-fidelity evaluations more selectively, the method may help lower energy consumption but achieve better performance. The negative side is that LLM-based planners may introduce opaque decision behavior, and is hard to comprehensively evaluate all the behaviors of it.
Convergence

Figure 7. HV convergence of experiments in regime 1.
Prompt template example
This appendix provides one concrete prompt–response example used by REP-HO. The example is taken from Regime 1 on TNK with the constraint-heavy cost setting (0.15, 0.05 | 0.70, 0.10). The logged backbone is DeepSeek-V4-Flash; Qwen3.5-35B-A3B receives the same template. The problem name is not included. Repeated candidates are abbreviated. This first-generation log has an empty neighbor list for every candidate, so candidate_A below shows two illustrative neighbors rather than points from that log.
Prompt example
{
"task_intro": "You are assisting a constrained bi-objective optimizer. Your goal is to cho
"problem_intro": "This is a two-objective constrained design problem. Lower objective value
"task": {
"obj_dirs": {"f1": "minimize", "f2": "minimize"},
"constraint_rule": "For every constraint, predicted_value <= 0 means predicted feasible,
"metric_def": "p_nd means the current surrogate-based probability that a candidate is non
},
"notes": [
"Candidate labels are stable identifiers. Use them exactly as given.",
"Some targets may already been evaluated with high-fidelity. Do not request them again."
"Only targets evaluated with high-fidelity consume budget, low-fidelity targets do not co
"nearest_history_points lists at most 3 later infill points in normalized design space. I
"Overall feasibility is determined by satisfying all constraints at the same time. A cand
],
"budget": {
"current_cost_used": 100,
"cost_used_since_initial_doe": 0,
"total_budget": 250,
"remaining_budget": 150,
"fillup_cost_for_full_generation": 50,
"budget_pressure_ratio": 0
},
"targets": {
"objectives": [
{"name": "f1", "evaluation_cost": 0.15},
{"name": "f2", "evaluation_cost": 0.05}
],
"constraints": [
{"name": "g1", "evaluation_cost": 0.7},
{"name": "g2", "evaluation_cost": 0.1}
]
},
"candidates": [
{
"label": "candidate_A",
"x": [1.0404, 0.122],
"objectives": {
"f1": {"predicted_value": 1.0404, "uncertainty": 1.7211e-15, "p_nd": 1, "already_hfe
"f2": {"predicted_value": 0.122, "uncertainty": 1.9071e-15, "p_nd": 0, "already_hfe
},
"constraints": {
"g1": {"predicted_value": -0.03, "uncertainty": 0.0793, "satisfied_if_prediction_is_e
"g2": {"predicted_value": -0.1302, "uncertainty": 9.421e-15, "satisfied_if_prediction
},
"ocsp": 0.6475,
"nearest_history_points": [
{
"distance": 0.041,
"closer_than_fraction": 0.97,
"x": [1.01, 0.16],
"objectives": {
"f1": {"high_fidelity_evaluated": true, "value": 1.01},
"f2": {"high_fidelity_evaluated": true, "value": 0.16}
},
"constraints": {
"g1": {"high_fidelity_evaluated": true, "value": -0.021},
"g2": {"high_fidelity_evaluated": false, "value": null}
}
},
{
"distance": 0.058,
"closer_than_fraction": 0.91,
"x": [0.98, 0.19],
"objectives": {
"f1": {"high_fidelity_evaluated": true, "value": 0.98},
"f2": {"high_fidelity_evaluated": false, "value": null}
},
"constraints": {
"g1": {"high_fidelity_evaluated": true, "value": 0.014},
"g2": {"high_fidelity_evaluated": true, "value": -0.44}
}
}
]
},
...
{
"label": "candidate_J",
"x": [0.7852, 0.5972],
"objectives": { "...": "same fields as candidate_A" },
"constraints": { "...": "same fields as candidate_A" },
"ocsp": 0.9997,
"nearest_history_points": []
}
],
"batch_condition": "There are 5 prompts in total for a generation, each prompt contains ran
"request": {
"scope": "Every candidate shown here has already been selected by the original optimizer
"n_candidates_in_batch": 10,
"allowed_targets_per_candidate": "You may request zero or more objectives and zero or mor
},
"output": {
"format": "Return JSON only. Do not use markdown.",
"example schema": {
"selections": [
{"label": "candidate_A", "evaluate_objectives": ["f1", "f2"], "evaluate_constraints"
]
},
"rules": [
"The objectives and constraints you put in the brackets will be evaluated with high-fid
"Use labels and target names exactly as provided.",
"Do not output indices. Do not give any explanation.",
"Return a decision for every candidate shown in this batch.",
"Empty objective and constraint lists are allowed when no high-fidelity evaluation is n
"Return valid JSON that can be parsed directly."
]
}
}
LLM output example
{
"selections": [
{"label": "candidate_A", "evaluate_objectives": ["f2"], "evaluate_constraints": ["g1"]},
{"label": "candidate_B", "evaluate_objectives": ["f2"], "evaluate_constraints": ["g1"]},
...
{"label": "candidate_J", "evaluate_objectives": ["f1", "f2"], "evaluate_constraints": ["g
]
}
| Problem | NSGA-III | SA-NSGA-III | MFE-NSGA-III | REP-HO(Qwen) | REP-HO(Deepseek) | |
| MW3 | 200 | 0.2667 ± 0.6363 | 2.4960 ± 0.6122 | 2.9173 ± 0.2242 | 2.8853 ± 0.3005 | 2.9391 ± 0.1424 |
| MW3 | 400 | 2.4192 ± 0.2714 | 3.0687 ± 0.3497 | 3.3314 ± 0.1688 | 3.2635 ± 0.1909 | 3.3148 ± 0.1797 |
| MW3 | 600 | 2.6598 ± 0.2544 | 3.3510 ± 0.1495 | 3.0109 ± 0.0056 | 3.2995 ± 0.1927 | 3.3439 ± 0.1551 |
| TNK | 200 | 2.8096 ± 0.1397 | 2.4770 ± 1.2677 | 3.1943 ± 0.1330 | 3.2577 ± 0.0345 | 3.2443 ± 0.0647 |
| TNK | 400 | 2.9979 ± 0.1291 | 2.9807 ± 0.7573 | 3.2099 ± 0.0899 | 3.2387 ± 0.1555 | 3.2795 ± 0.0565 |
| TNK | 600 | 3.0899 ± 0.1094 | 3.2905 ± 0.0052 | 3.2573 ± 0.0673 | 3.2964 ± 0.0074 | 3.2842 ± 0.0282 |
| WeldedBeam | 200 | 3.5019 ± 0.1630 | 2.2884 ± 1.3171 | 3.0686 ± 1.1407 | 3.3188 ± 0.6547 | 3.4573 ± 0.3199 |
| WeldedBeam | 400 | 3.6412 ± 0.1139 | 3.0826 ± 1.0734 | 3.6368 ± 0.8365 | 3.7537 ± 0.1385 | 3.7410 ± 0.1804 |
| WeldedBeam | 600 | 3.7287 ± 0.1047 | 3.5583 ± 0.8216 | 3.8385 ± 0.0369 | 3.7461 ± 0.3370 | 3.8602 ± 0.0689 |
| Mean ranking integrated with IGD+ in Table 12 | 4.389 | 3.917 | 2.639 | 2.167 | 1.889 |
Table 5. Regime 4: Budget HV performance summary (HV ↑).
| Problem | NSGA-III | SA-NSGA-III | MFE-NSGA-III | REP-HO(Qwen) | REP-HO(Deepseek) | |
| MW3 | 200 | 0.4343 ± 0.0482 | 0.1943 ± 0.0901 | 0.1011 ± 0.0707 | 0.1090 ± 0.0826 | 0.0962 ± 0.0326 |
| MW3 | 400 | 0.2574 ± 0.0856 | 0.0798 ± 0.0990 | 0.0278 ± 0.0301 | 0.0329 ± 0.0292 | 0.0276 ± 0.0277 |
| MW3 | 600 | 0.1840 ± 0.0744 | 0.0216 ± 0.0233 | 0.0729 ± 0.0019 | 0.0295 ± 0.0294 | 0.0214 ± 0.0232 |
| TNK | 200 | 0.1135 ± 0.0271 | 0.0500 ± 0.0878 | 0.0238 ± 0.0276 | 0.0102 ± 0.0066 | 0.0151 ± 0.0134 |
| TNK | 400 | 0.0636 ± 0.0233 | 0.0466 ± 0.0991 | 0.0212 ± 0.0158 | 0.0168 ± 0.0278 | 0.0089 ± 0.0086 |
| TNK | 600 | 0.0399 ± 0.0174 | 0.0066 ± 0.0005 | 0.0121 ± 0.0107 | 0.0069 ± 0.0012 | 0.0086 ± 0.0033 |
| WeldedBeam | 200 | 0.1251 ± 0.0525 | 0.5967 ± 0.6835 | 0.2661 ± 0.4119 | 0.2056 ± 0.2596 | 0.1639 ± 0.1197 |
| WeldedBeam | 400 | 0.0750 ± 0.0294 | 0.1512 ± 0.1082 | 0.0292 ± 0.0233 | 0.0531 ± 0.0382 | 0.0557 ± 0.0524 |
| WeldedBeam | 600 | 0.0485 ± 0.0248 | 0.0591 ± 0.0430 | 0.0178 ± 0.0083 | 0.0583 ± 0.1192 | 0.0206 ± 0.0192 |
| Mean ranking integrated with IGD+ in Table 5 | 4.389 | 3.917 | 2.639 | 2.167 | 1.889 |
Table 12. Regime 4: Budget IGD+ performance summary (IGD+ ↓).
References
- [1]Virginia Aglietti, Ira Klena, Jessica Schrouff, Eleni Sgouritsa, Francisco JR Ruiz, Alan Malek, Alexis Bellot, and Silvia Chiappa. Fundo: Discovering acquisition functions for bayesian optimization with funsearch. arXiv preprint arXiv:2406.04824, 2024.arxiv.org/abs/2406.04824
- [2]Richard Allmendinger and Joshua Knowles. Heterogeneous objectives: state-of-the-art and future research. In Many-Criteria Optimization and Decision Analysis: State-of-the-Art, Present Challenges, and Future Perspectives, pp. 317–335. Springer, 2023.arxiv.org/abs/2103.15546
- [3]Mayk Caldas Ramos, Shane S Michtavy, Andrew D White, and Marc D Porosoff. Bayesian optimization of catalysis with in-context learning. ACS Central Science, 2023.arxiv.org/abs/2304.05341
- [4]Siyu Chen, Haoran Tang, Jinyuan Zhang, and Ke Tang. A large language model-assisted evolutionary algorithm for expensive constrained multi-objective optimization. In Proceedings of the Genetic and Evolutionary Computation Conference Companion, pp. 347–350, 2025.
- [5]Raphaël Cosson, Roberto Santana, Bilel Derbel, and Arnaud Liefooghe. Multi-objective nk landscapes with heterogeneous objectives. In Proceedings of the Genetic and Evolutionary Computation Conference, pp. 502–510, 2022.DOI
- [6]Raphaël Cosson, Roberto Santana, Bilel Derbel, and Arnaud Liefoghe. On bi-objective combinatorial optimization with heterogeneous objectives. European Journal of Operational Research, 319(1): 89–101, 2024.DOI
- [7]Roy De Winter, Bas Milatz, Julian Blank, Niki Van Stein, Thomas Bäck, and Kalyanmoy Deb. Parallel multi-objective optimization for expensive and inexpensive objectives and constraints. Swarm and Evolutionary Computation, 86:101508, 2024.DOI
- [8]Bach Do and Ruda Zhang. Multifidelity bayesian optimization: a review. AIAA Journal, 63(6): 2286–2322, 2025.
- [9]Peter Eckmann, Dongxia Wu, Germano Heinzellmann, Michael K Gilson, and Rose Yu. Mf-lal: Drug compound generation using multi-fidelity latent space active learning. In Forty-second International Conference on Machine Learning, 2025.arxiv.org/abs/2410.11226
- [10]Zahra Zanjani Foumani, Mehdi Shishehbor, Amin Yousefpour, and Ramin Bostanabad. Multi-fidelity cost-aware bayesian optimization. Computer Methods in Applied Mechanics and Engineering, 407:115937, 2023.
- [11]Chenyan Gu and Handing Wang. A local correlation estimation surrogate-assisted bi-objective evolutionary algorithm for heterogeneous objectives. Applied Soft Computing, 151:111175, 2024.
- [12]Yuxiao Huang, Shenghao Wu, Wenjie Zhang, Jibin Wu, Liang Feng, and Kay Chen Tan. Autonomous multi-objective optimization using large language model. IEEE Transactions on Evolutionary Computation, 2025.arxiv.org/abs/2406.08987
- [13]Tennison Liu, Nicolás Astorga, Nabeel Seedat, and Mihaela van der Schaar. Large language models to enhance bayesian optimization. In The Twelfth International Conference on Learning Representations, 2024.arxiv.org/abs/2402.03921
- [14]Nasrulloh Loka, Ivo Couckuyt, Federico Garbuglia, Domenico Spina, Inneke Van Nieuwenhuyse, and Tom Dhaene. Bi-objective bayesian optimization of engineering problems with cheap and expensive cost functions. Engineering with Computers, 39(3):1923–1933, 2023.DOI
- [15]Giang Ngo, Dat Phan Trong, Dang Nguyen, Sunil Gupta, and Svetha Venkatesh. Adaptive acquisition selection for bayesian optimization with large language models. In The Fourteenth International Conference on Learning Representations, 2026.arxiv.org/abs/2602.07904
- [17]Yong Pang, Xiaonan Lai, Yitang Wang, Xiwang He, Shuai Zhang, and Xueguan Song. Surrogate-assisted expensive constrained bi-objective optimization with highly heterogeneous evaluations. Swarm and Evolutionary Computation, 83:101401, 2023.
- [18]Kamrul Rahi, Hemant Singh, and Tapabrata Ray. Towards solving expensive optimization problems with heterogeneous constraint costs. In Proceedings of the Genetic and Evolutionary Computation Conference Companion, pp. 2032–2040, 2024.DOI
- [19]Nian Ran, Zhongzheng Li, Yue Wang, Qingsong Ran, Xiaoyuan Zhang, Shikun Feng, Richard Allmendinger, and Xiaoguang Zhao. Mcce: A framework for multi-llm collaborative co-evolution. arXiv preprint arXiv:2510.06270, 2025a.
- [20]Nian Ran, Yue Wang, Xiaoyuan Zhang, Zhongzheng Li, Qingsong Ran, Wenhao Li, and Richard Allmendinger. Exllm: Experience-enhanced llm optimization for molecular design and beyond. arXiv preprint arXiv:2502.12845, 2025b.arxiv.org/abs/2502.12845
- [21]David Salinas, Omar Swelam, and Frank Hutter. Tuning llm judge design decisions for 1/1000 of the cost. In Forty-second International Conference on Machine Learning, 2025.arxiv.org/abs/2501.17178
- [22]Balija Santoshkumar and Kalyanmoy Deb. Handling objectives with heterogeneous evaluation times in surrogate-assisted evolutionary multi-objective optimization. IEEE Transactions on Evolutionary Computation, 2025.