Multi-Objective Generative Optimization via Expected Hypervolume Fine-Tuning

Introduction

Scientific discovery often requires balancing competing objectives. A peptide with high binding affinity to a protein, for example, may have low cell permeability, and vice versa [56]. The most useful compromise may remain unknown until experimental evidence or downstream preferences become available. A useful generator should therefore produce candidate sets that collectively represent high-quality, complementary trade-offs.

Pretrained generative models provide a prior over plausible designs, but their sampling distributions are not optimized for the properties of interest. Generative optimization adapts these distributions toward improved objective values [20, 16, 29, 34]. In the multi-objective setting, however, improving candidates individually is not enough: samples must also complement one another across competing objectives. The key challenge is therefore to optimize this collective behavior, rather than a fixed score assigned to each candidate in isolation.

Existing methods provide several forms of multi-objective steering. Inference-time search and guidance can seek favorable trade-offs, but they require additional optimization whenever candidates are generated [56, 2] (Yao et al. 2024) and, without updating the model, may have limited ability to depart from the pretrained prior [35]. Fine-tuning amortizes this cost, yet it faces other limitations. Fixed scalarizations score candidates independently [57, 44, 49, 38]; preference-conditioned and composed models require specifying a preference distribution [24, 12]; and dominance-based signals neither distinguish mutually nondominated candidates nor measure how each contributes to the generated set [52]. No algorithm directly trains the sampling distribution for both candidate quality and collective trade-off coverage.

Motivated by this challenge, we address the following question:

How can we fine-tune a generative model to directly produce samples that jointly achieve high objective quality and broad trade-off coverage?

We introduce Hypervolume Fine-Tuning (HVFT), which fine-tunes a single generative model to maximize the expected hypervolume of its generated candidate sets while controlling departure from the pretrained distribution. Hypervolume measures the objective-space region dominated by a candidate set relative to a fixed reference point [28] (Zitzler & Thiele 1999). It therefore rewards both strong individual candidates and complementary trade-off coverage, without requiring a distribution over preferences.

Unlike standard fine-tuning objectives, expected hypervolume cannot be expressed as the expectation of a fixed per-candidate reward. A candidate’s value depends on the objective-space regions covered by the other samples, so its reward must adapt to the current sampling distribution [20, 50] (Zheng et al. 2026; Zhu et al. 2026). HVFT addresses this limitation using Flow Density Control [16], which optimizes a distributional objective through a sequence of KL-proximal subproblems driven by its first variation (see Figure 1). A key observation is that, for expected hypervolume, this first variation is the expected marginal hypervolume contribution of a candidate relative to samples from the current model. This identity gives the reward a clear interpretation and makes it tractable to estimate directly from model samples. At each outer iteration, HVFT estimates this adaptive reward under the current distribution, uses it with a model-specific scalar-reward solver, and recomputes it after the update. We further establish concavity and relative smoothness of the objective, yielding monotone improvement and convergence guarantees for this procedure.

Across therapeutic-peptide [56 b] and molecular design [52] tasks spanning discrete diffusion and continuous flow-matching models, HVFT achieves state-of-the-art expected hypervolume and trade-off coverage without additional inference-time search.

Our main contributions are as follows:

• We introduce expected hypervolume, a distributional training criterion for multi-objective generative optimization that jointly rewards quality and collective trade-off coverage.

• We express the first variation of expected hypervolume in terms of marginal hypervolume contributions and derive an unbiased sample-based estimator. This yields HVFT, a sequence of scalar-reward fine-tuning subproblems for which we establish convergence guarantees.

• We demonstrate that HVFT achieves state-of-the-art performance on therapeutic-peptide and molecular design tasks spanning discrete diffusion and continuous flow-matching models, improving the expected hypervolume of sampled sets without inference-time search.

Related work

We first discuss existing methods for multi-objective steering of pretrained generators and then review the methodological foundations of HVFT. We conclude by briefly discussing adjacent multi-objective optimization settings that target different problems.

Multi-objective steering of pretrained generators. Methods for steering pretrained generators toward multiple objectives differ primarily in the information carried by their steering signal.

Ordinal Pareto signals. Pareto dominance guides inference-time search in PepTune {(citep}(Tang et al., 2025a)}, replay-buffer construction and off-policy fine-tuning in TR2D2 and its extensions {(citep}(Tang et al., 2025b; Mahood et al., 2026; Patel et al., 2026)}, and the selection of preference pairs for direct preference optimization {(citep}(Schneuing et al., 2025)}. Dominance records which candidates are preferable but not by how much: it is invariant to strictly increasing transformations of each objective, cannot order mutually nondominated candidates, and does not quantify their contributions to a candidate set.

Scalarized per-candidate rewards. A second family maps each objective vector to a scalar utility and applies standard reward-based fine-tuning, sometimes incorporating thresholds or predictive uncertainty {(citep}(Lin et al., 2026; Chen et al., 2025a)}. TR2-D2-based methods, for example, sum the objectives of candidates in their nondominated replay buffer {(citep}(Tang et al., 2025b; Mahood et al., 2026; Patel et al., 2026)}. Because a fixed scalarization scores candidates independently and generally favors a particular trade-off, repeated samples need not cover the front.

Preference-conditioned generation. A third family makes the trade-off an input by conditioning on a preference vector during training {(citep}(Golan et al., 2026)} or inference {(citep}(Chen et al., 2025b)}, or by composing separately aligned models at sampling time {(citep}(Cheng et al., 2026)}. These methods provide precise control when the desired trade-off is known. Producing a set of alternatives instead requires choosing a distribution over preferences, such as a grid or a Das–Dennis construction. This choice becomes harder as the number of objectives grows and does not directly optimize the coverage of the resulting samples.

Positioning of HVFT. The preceding approaches rely on ordinal comparisons, fixed per-candidate utilities, or externally specified preferences. HVFT instead introduces a cardinal set-level training signal: each candidate is valued by the hypervolume it contributes relative to samples from the current model. This signal distinguishes mutually nondominated candidates, accounts for the trade-offs already covered by the generator, and requires no specified preference distribution.

Methodological foundations. We discuss generators and hypervolume as a set-level objective. We further describe the distributional optimization framework that connects them.

Reward-based generative fine-tuning. Pretrained diffusion, flow-matching, and discrete-diffusion models provide priors over plausible molecular structures and biological sequences {(citep}(Ho et al., 2020; Lipman et al., 2023; Lou et al., 2024; Tang et al., 2025a; Dunn & Koes, 2026)}. Reward-based methods adapt these models toward target properties {(citep}(Domingo-Enrich et al., 2025; Liu et al., 2025; Zheng et al., 2026)}. HVFT likewise adapts a pretrained model using a compatible reward-based method as the inner solver within a sequential fine-tuning procedure, with the marginal hypervolume reward recomputed as the sampling distribution changes.

Hypervolume as an optimization signal. Hypervolume measures the region of objective space dominated by a candidate set relative to a reference point {(citep}(Zitzler & Thiele, 1999; Guerreiro et al., 2022)}; its diminishing-returns structure relates it to submodular coverage {(citep}(Nemhauser et al., 1978; Krause & Golovin, 2014)}. Beyond evaluation, hypervolume has been used in Bayesian optimization, evolutionary selection, Pareto-set learning, and reinforcement-learning {(citep}(Dalton et al., 2020; Beume et al., 2007; Bader & Zitzler, 2011; Zhang et al., 2023; Lu et al., 2026; Steinberg et al., 2025)}. These methods optimize the hypervolume of a finite set selected or maintained by the algorithm. HVFT instead uses the expected hypervolume of samples from the generator as its steering signal.

Distributional optimization of non-additive objectives. Set-level objectives make each candidate’s value depend on the others. Related work in submodular reinforcement learning (RL) captures these interactions through marginal-gain estimators and successive additive approximations {(citep}(Prajapat et al., 2024; De Santi et al., 2024)}, while work in general utilities, or convex RL optimizes distributional functionals via iterative linearizations {(citep}(Hazan et al., 2019; Zahavy et al., 2021; Mutti et al., 2023)}.

Flow Density Control provides a general framework for optimizing nonlinear functionals of generative distributions via generative model adaptation: at each outer iteration, it evaluates the first variation at the current sampling distribution and uses it as a scalar reward in a KL-proximal fine-tuning subproblem (De Santi et al., [16]). While prior work employed distributional flow fine-tuning to improve design-space coverage (e.g., De Santi et al., [15]; 2026c;a), here we tackle the problem of multi-objective optimization, i.e., a specific form of objective-space coverage. HVFT uses this framework to optimize expected hypervolume through sequential scalar-reward fine-tuning. To make this possible, we derive the marginal hypervolume first variation, construct an unbiased sample-based estimator, and establish a relative-smoothness bound that controls the step size.

Adjacent multi-objective optimization settings. Several related formulations differ in their evaluation access, prior information, and outputs.

Multi-objective Bayesian optimization methods treat objective evaluations as scarce and repeatedly select batches that improve the hypervolume of an observed archive (Daulton et al., [13]; Stanton et al., [54]; Gruver et al., [27]; Jain et al., [33]; Steinberg et al., [55]). HVFT instead assumes objectives can be queried at training scale and returns a sampler that can subsequently generate candidates without re-solving the optimization problem. Pareto-set approximation methods return an explicit approximation of the front or a map from preferences to solutions, using, for example, hypervolume-based survival selection (Beume et al., [6]; Bader & Zitzler, [4]) or learned Pareto-set representations (Lin et al., [39]; Zhang et al., 2023). They do not use a pretrained distribution as a prior over plausible designs. Offline multi-objective optimization methods work from a fixed dataset without further objective queries and steer generation using learned predictors or dominance classifiers (Yuan et al., 2025; Annadani et al., [2]). HVFT instead queries the objectives during training and uses only their evaluated values.

Hypervolume Fine-Tuning

Let $\mathcal{X}$ denote a design space and let $\mathbf{r}(x) = (r_1(x), \ldots, r_M(x)) \in\mathbb{R}^M$ denote a vector of competing objectives, all taken to be maximized. We assume that these objectives can be evaluated on generated candidates, but they may be black-box and nondifferentiable. Let $p_0$ denote a pretrained generative distribution over $\mathcal{X}$. We seek to adapt this distribution so that its samples collectively achieve high objective quality and broad trade-off coverage while preserving useful behavior learned during pretraining.

Achieving this goal requires a distributional objective that values candidate sets and a way to optimize that objective via generative model fine-tuning. The remainder of this section develops these two ingredients. We first define expected hypervolume as a distributional objective (§ 3.1) and then derive its first variation, which is an expected marginal hypervolume contribution (§ 3.2). This local reward leads to the sequential fine-tuning procedure in § 3.3.

Expected hypervolume as a distributional objective

Evaluating samples collectively requires accounting for both objective quality and complementary coverage. Hypervolume provides such a measure: relative to a reference point $\mathbf{r}^{\mathrm{ref}} \in\mathbb{R}^M$, it quantifies the volume of objective space jointly dominated by a collection of candidates (Zitzler & Thiele, 1999; Guerreiro et al., [28]). Unlike a fixed scalarization, hypervolume does not reduce the objectives to a prescribed per-candidate score; unlike Pareto dominance alone, it quantifies how much each candidate contributes to collective coverage. The reference point and objective scaling determine the region and relative volumes being measured, and we report these choices for each task in Appendix § 11.§ 2.

Writing $u \succeq v$ when $u_m \ge v_m$ for every objective $m$, we define the hypervolume of a set $A \subseteq\mathcal{X}$ as

where $\operatorname{Leb}_{M}$ denotes $M$-dimensional Lebesgue measure and $\operatorname{HV}(\varnothing; \mathbf{r}^{\mathrm{ref}}) = 0$. A candidate contributes only the portion of its dominated region not already covered by the others. Hypervolume therefore rewards stronger objective values, which expand the dominated region, and complementary trade-offs, which extend that region in previously uncovered directions. See Figure 1b for an illustration of hypervolume.

We lift hypervolume to a distributional training objective by taking its expectation over $n$ independent samples:¹

The parameter $n$ determines how many samples interact within the fine-tuning objective rather than how many samples may subsequently be generated. Empirically, we find that training with modest values of $n$ improves hypervolume even when evaluating larger numbers of samples (§ 4).

To limit departure from the pretrained model, we penalize deviations from the pretrained distribution. Let $\mathcal{P}(\mathcal{X})$ denote the set of probability distributions on $\mathcal{X}$. Our optimization target is

where $\alpha\ge0$ balances hypervolume performance against fidelity to the pretrained distribution.

The optimization in Equation (3) is not a standard fixed-reward fine-tuning problem ([20, 41]). When $n>1$, the value of a candidate depends on the other $n-1$ samples and hence on the sampling distribution itself. Thus, $F_n$ is generally nonlinear in $p$ and cannot be written as the expectation of a fixed per-candidate reward.

HVFT handles this nonadditivity locally. At iteration $k$, we linearize the objective around the current distribution $p_k$. Its first variation assigns each candidate a scalar value equal to its first-order effect on the objective. We optimize this local reward within a KL-controlled neighborhood of $p_k$ to obtain $p_{k+1}$, recompute the reward at the new distribution, and repeat. The next subsection derives this reward; § 3.3 then uses it to define the sequence of fine-tuning subproblems.

First variation and the marginal hypervolume reward

We first derive the scalar reward that locally represents changes in $F_n$. Following the distributional optimization framework of De Santi et al. [16], a first variation of $F_n$ at $p$ is a function $f_{n,p}$ such that, for every admissible distribution $q$,

At the current iterate $p=p_k$, both $F_n(p_k)$ and $\mathbb{E}_{p_k}[f_{n,p_k}(X)]$ are constant with respect to $q$. Maximizing the linear approximation over $q$ therefore reduces to maximizing $\mathbb{E}_q[f_{n,p_k}(X)]$, which has the additive form required by a scalar-reward fine-tuning solver.

For expected hypervolume, this local reward has a direct interpretation. Define the marginal hypervolume contribution of a candidate $x\in\mathcal{X}$ relative to a background set $A\subseteq\mathcal{X}$ as

Equation (6) rewards a candidate for the objective space it covers beyond a background set drawn from the current model (see Figure 1b). The reward is large when the candidate has strong objective values and adds a trade-off that current samples are unlikely to cover. It is not a fixed scalarization of $r(x)$: the background distribution changes with $p$, so the reward changes as the model improves. Candidate interactions are therefore retained in the outer loop even though the inner solver receives a scalar reward. The proof of Proposition 1 is given in Appendix § 9.

Sequential fine-tuning via marginal hypervolume rewards

Flow Density Control (De Santi et al., [16]) turns the local reward into a distributional update. Starting from the pretrained distribution $p_0$, we linearize both terms of the penalized objective at $p_k$ and constrain the update with a KL-proximal penalty. The $k$-th subproblem is

Algorithm 1 Hypervolume Fine-Tuning (HVFT)

Require: Pretrained distribution $p_0$, objective map $r$, reference point $r^{\mathrm{ref}}$, set size $n \ge1$, background count $B \ge1$, penalty coefficient $\alpha\ge0$, maximum iterations $K$, step sizes $\eta_k > 0$

1. for $k = 0, \ldots, K - 1$ do

2. Draw independent background sets $A_b \sim p_k^{\otimes(n-1)}$, $b = 1, \ldots, B$

3. Evaluate and cache $r(a)$ for each background design $a \in A_b$

4. Construct $\widehat{f}_{n,p_k}$ using Equation (8)

5. Approximately solve Equation (7) with $f_{n,p_k}$ replaced by $\widehat{f}_{n,p_k}$

6. end for

7. return $p_K$

The first variation of $p \mapsto D_{\mathrm{KL}}(p\Vert p_0)$ at $p_k$ is $\log(p_k(x)/p_0(x)) + 1$. The additive constant does not affect the optimizer and is omitted in Equation (7). Therefore, $f_{n,p_k}$ is the expected-hypervolume component of the linearized reward, and the log-density ratio is the pretrained-model penalty component. The final term, $\eta_k^{-1}D_{\mathrm{KL}}(p\Vert p_k)$, is not part of the original objective but it is included to limit how far the solution of the local subproblem can move from $p_k$.

The linearized reward remains fixed during the $k$-th inner solve. Its solution defines $p_{k+1}$, at which point we recompute both reward components to form the next subproblem. Thus, the outer loop updates the local reward, while the inner loop solves a standard scalar-reward fine-tuning problem.

Theoretical guarantee. The key question is whether progress on successive local subproblems also improves the original penalized objective. As shown in Appendix § 9.§ 2, expected hypervolume is concave in the sampling distribution, and the pretrained-model KL penalty makes the objective strictly concave when $\alpha> 0$ (Appendix § 9.§ 3). More importantly, relative smoothness bounds the error of the linear approximation by a multiple of $D_{\mathrm{KL}}(p_{k+1}\Vert p_k)$ (Appendix § 9.§ 4). A sufficiently strong KL-proximal penalty dominates this error, linking progress on each local subproblem to progress on the original objective. The following theorem formalizes this guarantee.

Practical implementation. The expectation in Equation (6) is approximated by Monte Carlo. At iteration $k$, we draw independent background sets $A_b \sim p_k^{\otimes(n-1)}$, for $b = 1, \ldots, B$, and define

For every fixed $x$, this estimator is unbiased: $\mathbb{E}[\widehat{f}_{n,p_k}(x)] = f_{n,p_k}(x)$.

During the $k$-th inner solve, we hold the background sets and their objective values fixed, making $\widehat{f}_{n,p_k}$ a fixed empirical reward. A model-specific scalar-reward method approximately solves the subproblem over the generator parameters. After the outer iteration, we draw new background sets from $p_{k+1}$ and construct the next reward. Our experiments in § 4 instantiate the inner solver with methods for continuous flow-matching and discrete diffusion models (Zheng et al., 2026; Tang et al., [57]). Appendix § 10 specifies the corresponding training losses and treatment of local regularization. Algorithm 1 summarizes the procedure.

Experiments

We evaluate HVFT on therapeutic-peptide and small-molecule design tasks using discrete diffusion and continuous flow-matching models. Our experiments test whether HVFT improves objective-space quality and trade-off coverage while preserving validity and diversity in the design space.

Evaluation protocol. At each evaluation, we generate candidates until obtaining 1,000 valid designs and record the valid fraction over all generation attempts. From this pool, we repeatedly draw random subsets of 10 and 100 observations without replacement and report their mean hypervolume, while computing diversity and design-space coverage over the full pool. We evaluate 100-hypervolume at regular intervals and use it to select the best-performing epoch, applying the same rule to baselines wherever possible. Across all tasks, HVFT uses $n = 10$; we study the effect of $n$ in Appendix § 12. We repeat each experiment with three random seeds, corresponding to independent fine-tuning runs for training-based methods, and report means and standard errors across runs. This across-run aggregation is distinct from the repeated subset sampling used to estimate each run’s hypervolume. Complete metric definitions and evaluation details are provided in Appendix § 11.

Therapeutic peptides

Peptides are molecules composed of amino-acid residues linked by peptide bonds and are an established class of therapeutic agents (Wang et al., 2022). We fine-tune the pretrained peptide masked diffusion model of Tang et al. [56] for five objectives: binding affinity, solubility, non-hemolysis, non-fouling, and membrane permeability. All objectives are evaluated using the classifiers and regressors from the same work and are taken to be maximized.

We consider three binding targets: the transferrin receptor (TfR), a common target for organ-specific drug delivery [30]; the glucagon-like peptide-1 receptor (GLP-1R), which is relevant to type 2 diabetes and obesity [1]; and the glutamate-aspartate transporter (GLAST), which is abundant on astrocytes and implicated in neurological disorders [48]. Only peptides that pass the SMILES2PEPTIDE verifier [56] are scored.

Baselines. Our primary fine-tuning baseline is TR2D2 [57], a state-of-the-art method for multi-objective fine-tuning of discrete diffusion models. We also include TR2D2 ++, an enhanced variant in which rewards are normalized to [0, 1] to avoid scale bias and the best epoch is selected by hypervolume. For additional context, we evaluate the pretrained model and PEPTUNE [56], an inference-time method that guides generation using Monte Carlo tree search.

HVFT configuration. We use the single-objective variant of TR2D2 as HVFT’s inner solver. We leave the marginal hypervolume rewards unscaled because scaling gave only minor improvements and requires a problem-dependent factor. See Appendix § 11.§ 2 for details.

Results. As shown in Table 1, HVFT achieves the highest 10- and 100-hypervolume on all three targets. It also yields the highest design-space diversity and coverage. Figure 2 illustrates these gains on GLAST, where HVFT produces a Pareto set with broader objective-space coverage and stronger individual candidates than TR2D2.

Task	Method	100-Hypervolume	10-Hypervolume	Diversity	Coverage	Valid fraction
GLAST	Base	2.78 ± 0.04	1.61 ± 0.01	60.35 ± 0.14	401.50 ± 0.28	0.321 ± 0.007
	PEPTUNE	2.79 ± 0.07	1.59 ± 0.03	39.78 ± 1.05	134.95 ± 5.87	0.603 ± 0.028
	TR2D2	7.78 ± 1.06	6.79 ± 1.41	9.08 ± 6.24	136.64 ± 74.24	0.249 ± 0.031
	TR2D2 ++	_8.56 ± 0.86_	_7.68 ± 0.77_	4.13 ± 1.28	89.43 ± 28.69	0.233 ± 0.010
	HVFT	11.67 ± 0.27	10.56 ± 0.25	64.78 ± 7.34	465.45 ± 15.71	0.336 ± 0.024
GLP-1R	Base	3.01 ± 0.04	1.77 ± 0.01	_60.35 ± 0.14_	_401.50 ± 0.28_	0.321 ± 0.007
	PEPTUNE	3.02 ± 0.11	1.77 ± 0.06	40.46 ± 1.06	137.70 ± 6.58	0.614 ± 0.012
	TR2D2	7.07 ± 0.19	5.64 ± 0.12	13.24 ± 8.00	182.56 ± 77.61	0.167 ± 0.031
	TR2D2 ++	_8.93 ± 0.98_	_8.12 ± 0.83_	3.59 ± 0.82	75.66 ± 21.14	_0.349 ± 0.060_
	HVFT	11.23 ± 0.08	10.34 ± 0.05	93.41 ± 2.14	521.27 ± 5.74	0.243 ± 0.071
TfR	Base	2.93 ± 0.04	1.69 ± 0.01	_60.35 ± 0.14_	_401.50 ± 0.28_	0.321 ± 0.007
	PEPTUNE	2.82 ± 0.18	1.65 ± 0.09	39.97 ± 0.78	136.73 ± 4.67	0.551 ± 0.059
	TR2D2	4.41 ± 0.91	2.78 ± 1.14	6.46 ± 1.63	126.79 ± 34.38	0.200 ± 0.015
	TR2D2 ++	_7.63 ± 1.87_	_6.75 ± 1.90_	4.91 ± 1.45	99.92 ± 25.29	_0.331 ± 0.068_
	HVFT	11.57 ± 0.16	10.15 ± 0.18	75.89 ± 9.42	485.80 ± 17.27	0.250 ± 0.021

Small molecules

The vast space of drug-like molecules makes exhaustive screening infeasible, motivating generative models that explore chemical space and propose compounds with desirable properties ([26]). We fine-tune FlowMol ([21]), a continuous flow-matching model trained on GEOM-Drugs ([3]), and evaluate two nested tasks: Topology, which uses graph-level objectives and validity checks, and Geometry, which adds conformation-level objectives and 3D validity checks.

For Topology, we maximize Quantitative Estimate of Drug-likeness (QED) ([7]) and synthetic accessibility (SA) ([22]), as implemented in RDKit ([37]). We reverse and rescale the original SA score to [0, 1] so that higher values are better. Only molecules that pass the chemical validity checks of Buttenschoen et al. [9] are scored.

Geometry includes the Topology objectives and additionally maximizes the dipole moment computed at the GFN2-xTB level of theory ([5]). Scored molecules must also pass the PoseBusters checks for bond lengths, bond angles, planar aromatic rings, planar double bonds, and internal steric clashes ([8]).

Baselines. We compare HVFT against DiffusionNFT ((Zheng et al., 2026)), a sample-efficient method that we apply to a sum scalarization of the objectives, and DiffusionBlend ([12]), a state-of-the-art multi-objective method that combines one separately aligned model per objective during sampling. Although its original implementation uses low-rank adapters, we fully fine-tune each model with DiffusionNFT to avoid restricting model capacity. FlowMol is reported for context.

HVFT configuration. We use DiffusionNFT as HVFT’s scalar inner solver and keep its hyperparameters fixed across HVFT and all fine-tuning baselines. Details are provided in Appendix C.2.

Results. Table 2 shows that HVFT achieves the highest 10- and 100-hypervolume on both tasks. It also preserves substantially more design-space diversity than the other fine-tuning methods. Figure 3 compares representative four-candidate subsets on Topology.

Task	Method	Objective-space metrics: 100-Hypervolume	Objective-space metrics: 10-Hypervolume	Design-space metrics: Diversity	Design-space metrics: Coverage	Design-space metrics: Valid fraction
Topology	FlowMol	3.116 ± 0.014	2.665 ± 0.007	266.1 ± 4.9	507.1 ± 6.4	0.705 ± 0.008
	DiffusionNFT	3.140 ± 0.004	<u>2.754 ± 0.005</u>	128.8 ± 2.7	312.8 ± 8.7	0.952 ± 0.009
	DiffusionBlend	<u>3.337 ± 0.024</u>	2.750 ± 0.031	150.3 ± 18.8	345.8 ± 30.6	0.674 ± 0.132
	HVFT	3.416 ± 0.005	3.247 ± 0.004	154.9 ± 9.0	351.6 ± 11.1	<u>0.734 ± 0.007</u>
Geometry	FlowMol	4.279 ± 0.024	3.655 ± 0.010	91.4 ± 2.3	331.6 ± 7.0	0.483 ± 0.009
	DiffusionNFT	<u>4.335 ± 0.029</u>	<u>3.669 ± 0.018</u>	70.9 ± 1.6	<u>269.4 ± 6.8</u>	<u>0.663 ± 0.007</u>
	DiffusionBlend	4.332 ± 0.066	3.670 ± 0.034	51.6 ± 1.9	187.4 ± 2.2	0.707 ± 0.043
	HVFT	4.988 ± 0.049	4.151 ± 0.024	<u>87.5 ± 2.2</u>	249.8 ± 1.9	0.518 ± 0.017

Discussion

HVFT achieves the highest 10- and 100-hypervolume on every task, showing consistent gains in candidate quality and collective trade-off coverage across generative modalities and model architectures. Crucially, HVFT builds on established methods as inner solvers: single-objective TR2D2 for peptides and DiffusionNFT for molecules. Nevertheless, it outperforms TR2D2 and TR2D2 ++ on the peptide tasks and DiffusionNFT on the molecular tasks, supporting the benefit of the sequential marginal-hypervolume rewards. HVFT also outperforms DiffusionBlend and achieves these gains without manually specified scalarization weights or preference allocation across objectives.

Beyond objective-space performance, HVFT also performs strongly in the design space. Among steering methods, it attains the highest diversity on every task and the highest coverage on all but Geometry. In the peptide experiments, HVFT exceeds even the pretrained model in both metrics while maintaining a comparable valid fraction. One possible explanation is that distinct objective trade-offs may correspond to widely separated regions of the design space, so seeking objective-space coverage can encourage broader design-space exploration. Finally, the PepTune comparison places these fine-tuning gains in the context of inference-time steering. The inference-time baseline achieves substantially lower hypervolume than the training-based methods on the peptide tasks. This pattern is consistent with evidence that inference-time steering can have limited ability to move generation beyond regions favored by the pretrained model [35, 55, 18].

Despite strong performance, HVFT has limitations: it relies on a bi-level optimization structure and introduces hyperparameters (regularization $\alpha$, Monte-Carlo sample size $B$, set-size $n$; see Appendices § 10 and § 12). Additionally, hypervolume relies on the choice of reference point and is expensive to compute in many-objective settings. Monte-Carlo estimation can help reduce this cost, and extending HVFT to address many-objective scalability remains a direction for future work.

Conclusion

Multi-objective generative design requires samples that are individually valuable and collectively span useful trade-offs, yet standard fine-tuning relies on fixed per-candidate rewards that do not capture this collective behavior. We introduced Hypervolume Fine-Tuning (HVFT), which adapts a pretrained generator to maximize expected hypervolume while regularizing toward the pretrained distribution. Its key observation is that the first variation of expected hypervolume is given by the expected marginal contribution of a candidate and can be estimated directly from model samples, yielding scalar-reward subproblems for which we establish convergence guarantees.

Across therapeutic-peptide and small-molecule design tasks, HVFT achieves the highest 10- and 100-hypervolume with both discrete diffusion and continuous flow-matching models. It outperforms existing multi-objective methods, including standalone versions of its inner solvers, while preserving strong design-space diversity and coverage. Together, these results establish HVFT as a principled and tractable approach to fine-tuning generative models for high-quality, complementary designs without predefined scalarization weights or additional inference-time optimization.

Acknowledgments

This publication was made possible by the ETH AI Center doctoral fellowship to Riccardo De Santi. The project has received funding from the Swiss National Science Foundation under NCCR Catalysis (grant number 180544 and 225147) and NCCR Automation (grant agreement 51NF40 180545), National Centres of Competence in Research funded by the Swiss National Science Foundation.

Reproducibility Statement

The objective, first variation, estimator, and proofs are given in § 3 and Appendix § 9. § 4 and Appendix § 11 specify the evaluation rules and implementation choices.

AI Use Statement

In this work, we used generative AI tools to identify related work, polish the writing, implement our algorithms in code, and assist with mathematical proofs. All AI-assisted code was carefully checked and experimentally validated. All AI-assisted proofs were independently verified. All literature retrieved through AI was checked against the original sources. We take full responsibility for the final content of this work, including all text, claims, and artifacts.

Theory

The first two portions of these sections are dedicated to the expected-hypervolume objective $F_n(p)$ in Equation (2). Appendix § 9.§ 1 proves that the first variation of $F_n(p)$ is an expected marginal hypervolume contribution as defined in Proposition 1. This first variation, named $f_{n,p}$, drives the reward at the next iteration of HVFT. Appendix § 9.§ 2 establishes concavity of our objective $F_n(p)$ in the sampling distribution and provides further intuition about the structure it induces on $p$.

Appendix § 9.§ 3 studies the regularized objective $F_n(p)-\alpha D_{\mathrm{KL}}(p\Vert p_0)$, and proves the existence of a unique maximizer $p^*$. Appendix § 9.§ 4 then establishes curvature bounds of the regularized objective. Equipped with these bounds, we prove the convergence guarantees of HVFT in Appendix § 9.§ 5.

We use the notation of § 3 throughout. We assume the objective map $\mathbf{r}$ is measurable, the reference point $\mathbf{r}^{\mathrm{ref}}$ is fixed, and that $(r_m(\mathbf{x})-r_m^{\mathrm{ref}})_+ \le C$ for all $M$ objectives. This final assumption ensures that $0 \le\operatorname{HV}(A;\mathbf{r}^{\mathrm{ref}}) \le C^M$ for every finite candidate set $A$.

First variation of Expected Hypervolume

Proof. Fix an integer $n \ge1$ and a probability distribution $p$ on $\mathcal{X}$. Let $h$ be a finite signed measure with $h(\mathcal{X}) = 0$ such that $p_\epsilon= p+\epsilon h$ is a probability distribution for all sufficiently small $\epsilon\ge0$. Writing $x_{1:n}=(x_1,\ldots,x_n)$, the definition of expected hypervolume gives:

Let us recall the identity $(p_1+p_2)\otimes(q_1+q_2)=p_1\otimes q_1+p_1\otimes q_2+p_2\otimes q_1+p_2\otimes q_2$. Applying this identity repeatedly, we have:

Substituting Equation (11) into Equation (10) gives:

In the first equality, the boundedness of hypervolume preserves the $O(\epsilon^2)$ remainder under integration. In the $j$th integral of Equation (12), the coordinate $x_j$ is integrated against $h$, while every other coordinate is integrated against $p$. Since hypervolume is symmetrical, we can change the order of integration to have $x_j$ as the final integral. Thus every term inside the summand is equal and we can write the following expression:

Here $x$ denotes the candidate integrated against $h$, and $A$ is the set formed by the other $n-1$ independent draws from $p$. We are almost done. We just need to rewrite Equation (13) in terms of the marginal hypervolume contribution in Equation (5). To do this, we note that the background hypervolume is independent of $x$, and thus:

We may therefore subtract (14) inside the linear term in (13):

Consequently,

Finally, for any admissible probability distribution $q$, taking $h=q-p$ gives $p_\epsilon=(1-\epsilon)p+\epsilon q$ and

This yields the first-variation identity in Equation (4).

Concavity of Expected Hypervolume

Define $\phi_n(s) := 1 - (1-s)^n$ for $s \in[0,1]$. This is the probability of at least one success in $n$ independent trials, each with success probability $s$. In this section, we derive an alternative representation of $F_n(p)$ in terms of coverage, defined shortly. This representation will make concavity apparent.

For a fixed objective-space point $u \succeq r^{\mathrm{ref}}$ and design $X \in\mathcal{X}$, we say that $x$ covers $u$ if and only if $\mathbf{r}(x) \succeq u$. When $x$ is sampled from a distribution $p$, $\mathbf{r}(x) \succeq u$ is a probabilistic event. This leads us to the definition of the coverage probability $S_p(u)$:

A set $A$ covers $u$ if at least one of its elements covers $u$. For $A = \{X_1,\ldots,X_n\}$ with $X_1,\ldots,X_n \mathrel{\overset{\mathrm{i.i.d.}}{\sim}} p$, independence implies that all $n$ draws fail to cover $u$ with probability $(1-S_p(u))^n$, hence

We can rewrite hypervolume in term of coverage, as follows:

Taking expectations under the product measure $p^{\otimes n}$, applying Tonelli’s theorem, and using the coverage probability above gives

From the representation in Equation (17), we can proceed to show concavity of $F_n(p)$

Proof. For each fixed $u$, the quantity $S_p(u)$ is linear in the distribution $p$, since it is just the expectation $\mathbb{E}_p$ of the fixed indicator function $\mathbb{1}[\mathbf{r}(X) \succeq u]$. Meanwhile, $\phi_n$ is increasing and concave on $[0,1]$. Thus, $\phi_n(S_p(u))$ is concave in $p$, and integration preserves concavity.

The coverage representation in Equation (17) clarifies the preferences induced by $F_n$. The marginal value of increasing a point’s coverage probability is $\phi_n'(s)=n(1-s)^{n-1}$, which decreases in $s$ for $n>1$. Thus, the same increase in coverage probability yields a greater benefit at a point that is rarely covered than at one that is already reached reliably. This diminishing-returns structure favors allocating probability mass across complementary trade-offs, without requiring a fixed set of preference directions to be specified in advance.

KL regularization, existence, and uniqueness

HVFT optimizes expected hypervolume while staying close to a prior, as shown in the optimization problem (3). We show this regularized objective has a unique maximizer $p^*$. For $\alpha>0$, write the objective in Equation (3) as

KL divergence is strictly convex in its first argument on the convex domain $D$, so $-\alpha D_{\mathrm{KL}}(p\mathbin\Vert p_0)$ is strictly concave ([16]). Because summation preserves (strict) concavity and $F_n(p)$ is concave (see Theorem 2), $G_{n,\alpha}$ is also strictly concave.

A consequence of strict concavity is that if $G_{n,\alpha}$ attains its maximum on $D$, its maximizer $p^*$ is unique. To see this, assume for the sake of contradiction that there exist two maximizers $p$ and $q$. Strict concavity would give $G_{n,\alpha}((p+q)/2)>(G_{n,\alpha}(p)+G_{n,\alpha}(q))/2$, contradicting maximality.

The KL penalty also provides the control needed to prove existence. We show that distributions whose objective values approach the supremum must converge to a common limit, and that this limit attains the supremum.

Proof. Let $S:=\sup_{q\in D}G_{n,\alpha}(q)$. Since $p_0\in D$, $D_{\mathrm{KL}}(p_0\mathbin\Vert p_0)=0$, and $0\le F_n\le C^M$, we have $G_{n,\alpha}(p_0)\le S\le C^M$. Choose a maximizing sequence $(p_j)\subset D$ with $S-1/j<G_{n,\alpha}(p_j)\le S$. Its KL divergences satisfy the uniform bound

Finite KL divergence implies $p_j\ll p_0$. By the Radon–Nikodym theorem, each $p_j$ therefore has a density $v_j:=\frac{dp_j}{dp_0}$ relative to $p_0$: for every measurable $A\subseteq\mathcal{X}$,

In particular, these Radon–Nikodym derivatives belong to $L^1(p_0)$.

We next show that these densities form a Cauchy sequence. Since $(p_j+p_k)/2\in D$, the entropy midpoint identity gives

Combining this identity with concavity of $F_n$ gives

Pinsker’s inequality bounds each KL divergence below by half the squared $L^1(p_0)$ distance between the corresponding densities. The density of $(p_j+p_k)/2$ relative to $p_0$ is $(v_j+v_k)/2$, so

Substituting these bounds into the preceding inequality yields

Since $G_{n,\alpha}((p_j+p_k)/2)\leq S$, it follows that

Completeness of $L^1(p_0)$ therefore gives a limit $v_j\to v$ in $L^1(p_0)$. Nonnegativity and unit integral pass to this limit, so $p(A):=\int_A v\,dp_0$ defines a probability distribution.

We have shown that $p$ is a probability distribution. To establish $p\in\mathcal{D}$, it therefore remains only to show that $D_{\mathrm{KL}}(p\mathbin\Vert p_0)<\infty$. The bounded scalar sequence $(D_{\mathrm{KL}}(p_j\mathbin\Vert p_0))_{j\geq1}$ has a subsequence converging to its limit inferior. The corresponding densities still converge to $v$ in $L^1(p_0)$, so they admit a further subsequence converging $p_0$-almost everywhere. Denote the indices of this final subsequence of the original maximizing sequence by $j_k$. Thus both

With the convention $0\log0=0$, Fatou’s lemma applied to the nonnegative functions $v_{j_k}\log v_{j_k}+1/e$ gives

Thus $p\in\mathcal{D}$.

Finally, we show that $p$ attains $S$. For the full sequence $(p_j)$ and every objective-space point $u$,

The integrands in the coverage representation Equation (17) therefore converge pointwise. They lie in $[0,1]$ and vanish outside the finite coverage box, so dominated convergence gives $F_n(p_j)\to F_n(p)$. Combining this convergence with the lower semicontinuity of KL established above yields

Since $p\in\mathcal{D}$, the reverse inequality follows from the definition of $S$. Hence $G_{n,\alpha}(p)=S$.

Observation: first variation of the regularized objective. As in the main text, write $p(x)$ and $p_0(x)$ for densities with respect to a common dominating measure. When $p$ and $p_0$ are mutually absolutely continuous and $\log(p/p_0)$ is bounded $p_0$-almost everywhere, a first variation of $G_{n,\alpha}$ is

Indeed, the first variation of $D_{\mathrm{KL}}(p\mathbin\Vert p_0)$ is $\log(p(x)/p_0(x))+1$. The constant $-\alpha$ can be omitted because it integrates to zero against $q-p$. Thus, the full linearized reward combines the marginal hypervolume reward with the log-ratio penalty used in Equation (7).

Bounding Linear Approximation Error: Relative Smoothness

We bound the error of the linear approximations used by HVFT in terms of KL divergence. For a concave functional, this means bounding how far the functional can fall below its tangent approximation; equivalently, its negative is smooth relative to entropy.

Relative smoothness of $F_n(p)$

Define

Proof. For $n \ge2$, set $p_t=(1-t)p+tq$ and $\psi(t)=F_n(p_t)$. Since $S_{p_t}(u)=(1-t)S_p(u)+tS_q(u)$, differentiating the coverage representation in Equation (17) twice gives

The finite coverage box and bounded integrands justify differentiation under the integral; endpoint values are understood by continuity. Because $S_q(u)-S_p(u)$ is the difference of the probabilities of the same measurable event, $\lvert S_q(u)-S_p(u)\rvert\le\lVert q-p\rVert_{\mathrm{TV}}$. Thus

Taylor’s formula with integral remainder, together with $\psi'(0)=\int_{\mathcal{X}}f_{n,p}(x)\,\mathrm{d}(q-p)(x)$, implies

where the last inequality is Pinsker’s inequality. For $n=1$, $F_1$ is linear and the result holds with $L_1=0$.

Relative smoothness of $G_{n,\alpha}(p)$

The KL regularizer also contributes to the linearization error. Using the first variation in Equation (20), we account for the log-ratio term explicitly. Its contribution to the error is exactly $\alpha D_{\mathrm{KL}}(q\mathbin\Vert p)$, which adds to the relative smoothness constant and supplies a quantitative strong-concavity bound.

Thus $-G_{n,\alpha}$ is $(L_n+\alpha)$-smooth relative to entropy, and $G_{n,\alpha}$ is $\alpha$-strongly concave relative to entropy for these pairs of distributions.

Proof. The assumptions ensure that all terms in the entropy identity

are finite. Substituting $G_{n,\alpha}(p)=F_n(p)-\alpha D_{\mathrm{KL}}(p\|p_0)$ and $g_{\alpha,p}(x)=f_{n,p}(x)-\alpha\log(p(x)/p_0(x))$ into the linearization error gives

The last equality uses the KL identity above. By Proposition 2, the bracketed linearization error of $F_n$ lies between 0 and $L_nD_{\mathrm{KL}}(q\|p)$. Adding $\alpha D_{\mathrm{KL}}(q\|p)$ to these two bounds gives Equation (22): the lower bound establishes relative strong concavity, and the upper bound establishes relative smoothness.

Convergence of the linearized HVFT update

We now analyze the exact distributional update in Equation (7), which uses the full reward $g_{\alpha,p_k}$, including the linearized log-ratio penalty:

The pretrained-model penalty enters through $-\alpha\log(p_k(x)/p_0(x))$ in the reward. Its curvature is accounted for by the $L_n+\alpha$ step-size bound below.

Throughout the next three lemmas, assume the standing measurability and boundedness conditions, fix $n\ge1$ and $\alpha>0$, initialize at $p_0$, and choose $0<\eta\le(L_n+\alpha)^{-1}$. Write $\rho=1-\alpha\eta\in[0,1)$.

Proof. Initially $\log(p_0/p_0)=0$. If $\log(p_k/p_0)$ is bounded, then $g_{\alpha,p_k}$ is bounded because $0\le f_{n,p_k}\le nC^M$. Consequently $0<Z_k<\infty$, and the exponential update defines a probability distribution with the same null sets as $p_k$. Moreover,

is bounded. Induction therefore gives $p_{k+1}\in\mathcal{D}$ and the conditions required by Proposition 3 at every iteration.

For any $p\in\mathcal{D}$, substituting the exponential update into $D_{\mathrm{KL}}(p\|p_{k+1})$ and rearranging yields

All terms are finite by the bounded log ratios and $p\in\mathcal{D}$. Since KL divergence is nonnegative and vanishes exactly at $p=p_{k+1}$, this distribution uniquely solves the subproblem.

Proof. Taking logarithms of the update in Lemma 1 gives

Integrate against $q-p_{k+1}$. The constant $\log Z_k$ cancels because both distributions have mass one. Furthermore,

while integration of the same logarithm against $p_{k+1}$ gives $D_{\mathrm{KL}}(p_{k+1}\Vert p_k)$. Therefore

Next apply the two bounds in Equation (22) at the same base point $p_k$. Relative strong concavity bounds the comparator’s objective from above:

Relative smoothness bounds the next iterate’s objective from below:

Subtracting these inequalities cancels $G_{n,\alpha}(p_k)$ and combines the two integrals into an integral against $q-p_{k+1}$:

Substituting Equation (26) and collecting the coefficients of each KL divergence proves Equation (25). For $q=p_k$, it gives

since $\eta^{-1}\geq L_n+\alpha$.

Proof. The initial divergence $D_0$ is finite because $p^\star\in\mathcal{D}$. Every subsequent $D_k$ is finite as well, since

and the logarithm is bounded by Lemma 1. Taking $q=p^\star$ in Equation (25), dropping its final nonpositive term, and using $\eta^{-1}-\alpha=\eta^{-1}\rho$ yields

The left inequality follows from maximality of $p^\star$. It forces $D_{k+1}\leq\rho D_k$, and iteration gives $D_k\leq\rho^kD_0$. For $k\geq1$, nonnegativity of $D_k$ also gives

If $\rho=0$, the same inequalities imply $D_1=0$ and $\Delta_1=0$, so $p_1=p^\star$. Otherwise, both bounds decay geometrically because $0<\rho<1$.

The three lemmas establish the following convergence guarantee.

Proof. Lemma 1 establishes existence and uniqueness of each update. Lemma 2 proves monotonic improvement, and Lemma 3 gives Equations (27) and (28), including the case $\rho=0$. Finally, Pinsker’s inequality gives

which proves convergence in total variation. $\square$

At the largest admissible step size, $\eta=(L_n+\alpha)^{-1}$, the contraction factor is $\rho=L_n/(L_n+\alpha)$. These guarantees apply to exact first variations and exact optimization over distributions. Monte Carlo rewards, neural parameterization, and approximate inner optimization require additional error analysis.

Practical implementation aspects

Dealing with the log-ratio

A practical implementation must account for the log-density ratio $\log p(X)-\log p_0(X)$ in Equation (7), which involves marginal densities that are often intractable for diffusion models. We consider two approaches. The first, used in our experiments, sets $\alpha=0$ while retaining the local KL penalty relative to $p_k$. This avoids evaluating the density ratio and applies to both continuous and discrete generators. The local penalty constrains each update, while the cumulative departure from $p_0$ depends on the step sizes $\eta_k$ and the stopping point. This approach is empirically effective, as shown in § 4.

The second approach, applicable to continuous diffusion and flow-matching models, retains $\alpha>0$ and uses a solver driven by reward gradients, such as Adjoint Matching Domingo-Enrich et al. [20]. The adaptive reward has gradient

wherever the required derivatives exist. This approach requires estimates of the output-distribution scores, which are readily available for diffusion and flow-matching models Domingo-Enrich et al. [20]. The only remaining obstacle is the first term: a differentiable approximation of the marginal hypervolume reward $f_{n,p_k}$.

Random hypervolume scalarizations provide a convenient representation of this reward Golovin & Zhang [25]. For the objective vector $\mathbf{r}(x)\in\mathbb{R}^M$ and reference point $\mathbf{r}^{\mathrm{ref}}$, define

where $[a]_+=\max\{a,0\}$ and

The marginal hypervolume reward in Equation (6) can then be written as

This representation supports Monte Carlo estimation over directions $\lambda$ and background sets $A$: for each sample, draw a direction $\lambda$ and a set $A$ of $n-1$ points from $p_k$, and evaluate the bracketed expression.

A smooth surrogate of Equation (29) is obtained by replacing the minimum in $s_\lambda$ and each positive-part operation $[a]_+$ with softmin and softplus respectively. The maximum over the background set term $\max_{x'\in A}s_\lambda(\mathbf{r}(x')-\mathbf{r}^{\mathrm{ref}})$ need not be smoothed, since it is independent of $x$.

Automatic differentiation of this surrogate then yields an estimate of the reward gradient $\nabla_x f_{n,p_k}(x)$ for use in Adjoint Matching, provided that $\mathbf{r}$ is itself differentiable. Combined with the score terms above, this furnishes the full gradient of the entropy-regularized reward required by the solver.

Computational Cost

At first glance, Algorithm 1 appears to require a $K$-fold increase in training computation. In practice, however, the total training budget can be split across the $K$ fine-tuning rounds, so the overall number of gradient steps remains unchanged. HVFT’s only additional cost relative to running the inner solving algorithm for the same number of epochs therefore comes from generating and scoring the background sets—a small overhead relative to the full fine-tuning pipeline. Table 3 compares HVFT’s runtime to that of its inner solver counterpart in § 4. For the TR2-D2 comparison, statistics and standard errors are calculated on 9 runs of peptide tasks. Each run used one 40GB NVIDIA A-100 GPU. For the DiffusionNFT comparison, statistics are computed over 3 runs of the Geometry task. Each run used a 32GB NVIDIA RTX 5000 Ada Generation GPU.

Method	Execution time (hours)
TR2-D2	12.384 ± 1.114
HVFT	10.734 ± 0.310
DiffusionNFT	6.399 ± 0.069
HVFT	7.204 ± 0.082

Table 3: Average execution time in hours. The first block aggregates results across the three peptide tasks. The second block aggregates runs for the Geometry task. The performance of these runs is reported in § 4. HVFT’s execution time remains comparable to its inner solver counterpart.

Experimental details

Metrics

For each metric, we sample 1000 valid designs from the generator. Since generated designs may not always be valid, we allow repeated sampling until the 1000 quota is fulfilled or a maximum of 10000 trials is exceeded.

Our main objective-space metric is the mean hypervolume of 100 samples from the generator. We estimate the 100-hypervolume with sub-sampling: we draw $P=100000$ subsets of size 100 without replacement from the pool and average their hypervolumes. For each index $i\in[1,1000]$, we average the hypervolume values of the permutations that exclude $\mathbf{r}(X_i)$ to form a leave-one-out estimate $\widehat{\mathrm{HV}}_{-i}$. The final hypervolume is the mean of these leave-one-out estimates, $\widehat{\mathrm{HV}}=\frac{1}{n}\sum_i\widehat{\mathrm{HV}}_{-i}$. 10-hypervolume is computed the same way.

We used the full set of (up to) 1000 samples for the design-space metrics. Validity is computed as the number of valid samples divided by the number of attempts required to reach this quota, where validity is determined by problem-specific criteria. Following De Santi et al. [18], we measure generator diversity via the Vendi score of valid designs (Friedman & Dieng, [23]).

To assess design-space coverage, we adopt a metric similar to the fixed-threshold clustering metric proposed in the same work. Designs are embedded in a metric space and grouped into clusters such that any two designs within a cluster lie within a fixed radius of one another; the resulting number of clusters can be interpreted as the number of hyperspheres needed to cover an equal-sized set of designs from each method, with more clusters indicating broader coverage. Because this metric depends on the choice of radius — which encodes an arbitrary notion of how ”similar” two designs must be to count as the same — we avoid committing to a single threshold. Instead, we compute the number of clusters across 100 equally spaced thresholds and integrate numerically, yielding a single coverage score that is independent of any particular similarity cutoff.

Implementation Details

HVFT introduces four method-specific parameters: the background count $B$, the set size $n$, the regularization coefficient $\alpha$, and the number of outer loop steps $K$. The background count controls the precision of the Monte Carlo estimator of $f_{n,p_k}$. We found that $B = 85$ provides sufficient accuracy in practice and use this value throughout our experiments. We also keep $n = 10$ fixed across experiments and examine its effect in Appendix § 12. Following the discussion in Appendix § 10, we set the pretrained-model regularization coefficient $\alpha= 0$, while retaining the local KL penalty between successive iterates. The local trust region coefficient $\eta_k$ is fixed across $k$ so we drop the subscript in the remainder of this section. $K$ is task specific.

Therapeutic Peptides

To compute Vendi score and coverage, we use 1024-bit Morgan fingerprints (radius 2) (Morgan, [45]) and Tanimoto similarity. The reference point for hypervolume computation is written in Table 4 and is shared for all peptide tasks.

Property	Binding Affinity	Permeability	Nonfouling	Solubility	Hemolysis
Reference Point	4.5	-10.0	0.0	0.0	0.0

TR2-D2, TR2-D2++. Following Tang et al. [57], we finetune the base model for 1000 epochs total, evaluating every 100 epochs. Hyperparameters for TR2-D2 and TR2-D2++ are taken directly from Tang et al. [57].

HVFT. HVFT decomposes this same 1000-epoch budget into $K = 50$ linear finetuning subproblems, each run for 20 epochs. Its inner solver is the single-objective variant of TR2-D2, and it shares the same hyperparameters as the baselines except for $\eta^{-1}$. Table 5 summarizes these settings.

Hyperparameter	Name in Tang et al. [57]	Value
Num. of Replicates for Weighted Denoising Cross-Entropy	$R$	16
Buffer Size	$B$	20
Sequence Length	$L_{\mathrm{seq}}$	200
Num. of Children	$M$	50
Num. of Decoding Steps	$N_{\mathrm{steps}}$	128
Num. of MCTS Iterations	$N_{\mathrm{iter}}$	20
Resample Every $n$ Steps	$N_{\mathrm{resample}}$	20
Max Token Length	$L$	200
Learning rate	–	$10^{-4}$
$\eta^{-1}$	$\alpha$	0.1 (TR2-D2) or 1 (HVFT)

Peptune. To ensure a fair comparison, we depart from the evaluation protocol used for Peptune in Tang et al. [56]. Specifically, we set $N_{\mathrm{iter}} = 200$ and $M = 50$, so that at most 10000 samples are considered during inference. Every valid candidate encountered is retained in a running buffer until it reaches 1000 samples. Candidates that are unrolled during the tree search but turn out invalid still count toward the validity total.

Molecular Design

In all tasks, we fix the number of atoms to 10. We normalize Synthetic Accessibility via $(10 - SA)/9$ and additionally scale the dipole moment by a factor of 20 to avoid magnitude bias across baselines. Invalid candidates receive a reward of $-1$, giving the finetuning algorithms a signal when a design is invalid. The reference points for the Topology and Geometry tasks are given in Table 6.

Property	QED	SA	Dipole Moment
Reference Point	-1.0	-1.0	-1.0

All finetuning methods rely on DiffusionNFT, either as the main solver or as a component of their algorithm. Its hyperparameters are held fixed across methods and are summarized in Table 7.

Hyperparameter	Name in Zheng et al. (2026)	Value
Guidance Strength	$\beta$	50
KL Regularization Coefficient	Not in paper, but in official implementation	0.5
Learning Rate	–	1e-5

Topology. For the topology task, we allow 900 finetuning epochs, evaluating every 50. We compute Vendi score and coverage using 1024-bit Morgan fingerprints (radius 2) with Tanimoto similarity. Here, HVFT uses $K = 9$ to get 100 epochs per inner finetuning subproblem.

Geometry. For the geometry task, computational constraints limit us to 600 epochs, evaluating every 50. We compute Vendi score and coverage using USRCAT fingerprints and the USRCAT similarity score [53], as implemented in RDKit [37]. Here, HVFT uses $K = 6$ to get 100 epochs per inner finetuning subproblem.

DiffusionBlend. Cheng et al. [12] do not describe an automatic weight-selection procedure. We generate evenly spread reward-weighting tradeoffs via Das–Dennis sampling (for Geometry) or uniform sampling in $[0, 1]$ (for Topology).

Ablations

To assess HVFT’s sensitivity to $n$, we conduct additional experiments on representative therapeutic-peptide and small-molecule design tasks, holding all other parameters fixed as specified in Appendix C.2. Table 8 shows substantial hypervolume gains over the pretrained model across the tested values of $n$, which span up to two orders of magnitude. These results suggest that HVFT is robust to the choice of $n$.

Task	Model / $n$	100-Hypervolume	10-Hypervolume	Diversity	Coverage	Valid fraction
		Objective-space metrics			Design-space metrics
TfR	Base	2.93 $\pm$ 0.04	1.69 $\pm$ 0.01	60.35 $\pm$ 0.14	401.50 $\pm$ 0.28	0.321 $\pm$ 0.007
	10	11.57 $\pm$ 0.16	10.15 $\pm$ 0.18	75.89 $\pm$ 9.42	485.80 $\pm$ 17.27	0.250 $\pm$ 0.021
	50	12.231 $\pm$ 0.153	10.589 $\pm$ 0.226	93.2 $\pm$ 3.0	437.9 $\pm$ 62.9	0.210 $\pm$ 0.078
	150	13.757 $\pm$ 1.721	11.107 $\pm$ 0.891	107.3 $\pm$ 8.6	454.1 $\pm$ 43.6	0.235 $\pm$ 0.114
Topology	FlowMol	3.116 $\pm$ 0.014	2.665 $\pm$ 0.007	266.1 $\pm$ 4.9	507.1 $\pm$ 6.4	0.705 $\pm$ 0.008
	10	3.416 $\pm$ 0.005	3.247 $\pm$ 0.004	154.9 $\pm$ 9.0	351.6 $\pm$ 11.1	0.734 $\pm$ 0.007
	100	3.405 $\pm$ 0.004	3.200 $\pm$ 0.006	254.5 $\pm$ 3.5	473.1 $\pm$ 5.0	0.675 $\pm$ 0.008
	1000	3.362 $\pm$ 0.010	3.082 $\pm$ 0.017	310.6 $\pm$ 6.3	547.4 $\pm$ 6.5	0.708 $\pm$ 0.012

On the Topology task, which has two objectives, increasing $n$ does not improve hypervolume: $n = 10$ achieves the highest values. This suggests that relatively few samples may suffice to capture the main trade-offs on this task’s Pareto front.

Conversely, on the peptide task TfR, increasing $n$ consistently yields better hypervolume values. With five rewards, the Pareto front of this task is likely more complex and not easily covered by a small set of points. The larger variance at $n = 150$ has a simple explanation: Since $B = 85$, each point’s marginal contribution is estimated over 85 sets, each containing 149 other scores. Contributing additional hypervolume coverage against a set of that size is hard, so most of those 85 estimates are zero, and only a handful are non-zero. As $n$ grows, this sparsity makes the average marginal contribution noisier, producing higher variance across runs.