betaFeedback

WorksComputer Science

Binding Mode Matters: Hotspot-Aware Drug Discovery via Explorative Preferences

Abstract

Discovering hit molecules requires not just high binding affinity, but also the identification of diverse binding modes, which are critical for experimental assays. Existing generative approaches have predominantly relied on optimizing a scalar docking score, obscuring the distinct contributions of key binding determinants. To this end, we introduce a paradigm shift by formulating target-based drug design as a multi-objective exploration task, where each objective explicitly corresponds to enhancing interactions with a specific hotspot. Here, we introduce BindMol, a novel generative framework driven by a customized multi-objective reinforcement learning algorithm. By incorporating explorative preferences during training, our approach efficiently uncovers molecules with diverse and desirable binding profiles. Empirical results demonstrate that BindMol facilitates the discovery of high-affinity compounds characterized by both structural novelty and diverse binding modes. Validated across target-based drug discovery and multi-property optimization tasks, our approach provides a versatile paradigm for goal-oriented drug discovery.

Introduction

Target-based drug design aims to identify ligands that not only bind to a target protein with high affinity but also engage specific residues to trigger a desired pharmacological response [11]. The recent surge in deep generative models has accelerated this process, offering a powerful toolkit that spans de novo discovery and goal-oriented generation [8]. Reinforcement learning, in particular, has emerged as a dominant paradigm for conditional generation [41, 35] and optimization [62, 59]. It generates molecular structures by maximizing a reward reflecting drug-likeness and bioactivity [41].

Despite these advances, a misalignment persists between the complexity of biochemical interactions and the simplified objectives used in RL training. Most existing approaches reduce the ligand-protein interaction to a single scalar metric, such as the docking score. While computationally convenient, such scalar measures often overlook the complexity of interactions: compounds targeting the same protein can adopt different binding modes, leading to distinct functional and selectivity profiles. For instance,

two well-known GLP-1 agonists exhibit similar affinities but engage distinct sub-regions, resulting in either broad-spectrum or biased signaling [60], as illustrated in Figure 1.

Two agonists of GLP-1R showing distinct binding modes

Figure 1. Illustration of two agonists of GLP-1R, highlighting their distinct binding modes within the same protein pocket.

To address this limitation, we reformulate target-based drug discovery as a multi-objective optimization problem, where each objective corresponds to a distinct hotspot within the binding pocket. This conceptual shift naturally encourages the exploration of diverse binding modes by explicitly exposing trade-offs between interactions with different pocket regions. It also provides a more robust alternative to scalar objectives, which are known to be prone to reward hacking and suboptimal local solutions [13]. Specifically, we propose a novel reinforcement learning framework for drug discovery, termed BindMol. Its realization relies on three parts that bridge chemical validity, biological specificity, and algorithmic efficiency.

We explore the discrete chemical space using a fragment-based action space [59], where molecules are incrementally assembled in a Lego-like fashion. A compact yet informative fragment vocabulary is obtained via an energy-based model trained with noise contrastive estimation [21]. We theoretically ground fragment selection as a stability analysis problem on a thermodynamic potential surface, where high-scoring fragments act as stabilizing anchors for the ligand–protein complex. With a robust action space established, we next focus on the reward signal. Instead of flattening complex biological interactions into a docking score, we introduce a hotspot-aware vectorized reward by clustering the binding pocket into distinct hotspots based on reference structures, with each hotspot defining a separate objective. This design compels the agent to navigate trade-offs across sub-pockets, naturally leading to the discovery of diverse binding modes.

However, maximizing a vectorized reward is far more challenging than a scalar one [37, 1]. To enable precise control over multi-objective trade-offs, we establish Envelope SAC, a principled actor-critic framework that unifies the optimization of preference-aware critics and policies. This formulation significantly advances the theoretical scope of the envelope operator [58], moving beyond its value-based origins to the continuous domain. By constructing a utility envelope that aligns vectorized Bellman updates, our method efficiently approximates the convex hull of the Pareto frontier, ensuring robust performance across diverse preferences.

We empirically validate BindMol on binding affinity optimization tasks across five protein targets and the extensive GuacaMol benchmark. Our results demonstrate that BindMol not only generates valid, high-affinity compounds but also successfully discovers diverse binding modes that cover a broader range of the bioactive chemical space compared to state-of-the-art baselines. The main contributions of this work are three-fold:

❶ We propose the first molecular generation framework that treats binding-mode diversity as a first-class modeling objective, explicitly accounting for biologically grounded structural variations. This moves target-based design one step closer to treating molecular generation as an exploration of protein–ligand interaction patterns.

❷ We propose BindMol, a principled multi-objective reinforcement learning framework that integrates energy-based fragment scoring with hotspot-aware vectorized rewards, and develops Envelope SAC, an actor–critic algorithm capable of efficiently approximating the Pareto frontier under preference-conditioned optimization.

❸ We demonstrate the superiority of BindMol across diverse benchmarks. It outperforms existing methods in generating high-affinity ligands with novel binding modes across five protein targets and achieves state-of-the-art results on the comprehensive GuacaMol multi-objective benchmark.

Related Work

Fragment-based molecular generation Fragment-based molecular generation has emerged as an effective paradigm for drug discovery. Methods such as MARS [55], FREED [59], and MoLeR [38] assemble molecules from fragments to efficiently explore chemical space. Related approaches exploit informative substructures, including principal subgraph mining [30] and molecular rationales [27], to guide multi-objective generation. Incorporating protein structural context further improves generation quality, for example through transferable subpocket prototypes [61] or pocket-aware fragment assembly via diffusion [17]. Beyond generation, HN-GFN [63] enables sample-efficient multi-objective optimization, and recent antibiotic discovery results [31] demonstrate the practical impact of fragment-based strategies.

RL-driven molecular generation Deep reinforcement learning has been extensively applied to molecular design using both sequence- and graph-based formulations. Sequence-based policy optimization methods, including REINVENT [41] and its variants with curriculum learning [20], adapt SMILES generators toward target properties. Graph-based approaches [62] instead operate directly on molecular structures through chemistry-aware edit actions, often without pretraining. Fragment-based RL methods further address exploration efficiency and chemical validity by assembling molecules from learned vocabularies [59, 35]. More recently, structure-aware techniques such as AliDiff [18] explicitly align generative models with protein-binding preferences. Together, these works demonstrate the flexibility of RL for molecular design, while highlighting persistent challenges related to reward design and exploration.

Multi-objective reinforcement learning Multi-objective reinforcement learning (MORL) spans both tabular and deep settings, with single-policy and multi-policy formulations [10]. Tabular methods either optimize scalarized objectives [49] or maintain non-dominated value estimates, as in Pareto Q-learning [48]. In the deep setting, single-policy approaches such as EUPG [45] optimize user-defined utilities, while multi-policy methods dominate, including weight-conditioned models [58, 37], evolutionary approaches [57], return-conditioned models [44], and GPI frameworks [1]. These methods differ mainly in how preferences are incorporated and how the Pareto front is approximated.

Preliminaries

Markov decision process. We formalize a multi-objective reinforcement learning problem as a Multi-Objective Markov Decision Process (MOMDP), defined by

M=⟨S,A,P,r,Ω⟩\mathcal{M} = \langle\mathcal{S}, \mathcal{A}, \mathcal{P}, r, \Omega\rangle

with state space S\mathcal{S}, action space A\mathcal{A}, transition probability distribution P(s′∣s,a)\mathcal{P}(s' \mid s, a), vector reward function rr, and the space of preference Ω\Omega. If ω∈Ω\omega\in\Omega is fixed to a single value, this MOMDP collapses into a standard MDP. On the other hand, if we consider all possible returns from an MOMDP, we have a Pareto frontier F⋆:={r^∣r^≥r,r^≠r}\mathcal{F}^{\star} := \{\hat{r} \mid\hat{r} \geq r, \hat{r} \ne r\}, where the return r^:=∑tγtr(st,at)\hat{r} := \sum_t \gamma^t r(s_t, a_t).

Fragment-based molecular generation Following Yang et al. [59], each transition is driven by a triplet action at=(ut,ft,vt)a_t = (u_t, f_t, v_t), which attaches a chosen fragment ftf_t from the vocabulary to the current graph Gt\mathcal{G}_t at using the fragment’s site vtv_t. The generation policy πθ(at∣st)\pi_{\theta}(a_t \mid s_t) is factorized into three hierarchical sub-policies:

πθ(at∣Gt)=P(ut∣Gt)⋅P(ft∣ut,Gt)⋅P(vt∣ft,ut,Gt).(1)\pi_{\theta}(a_t \mid\mathcal{G}_t) = \mathbb{P}(u_t \mid\mathcal{G}_t) \cdot\mathbb{P}(f_t \mid u_t, \mathcal{G}_t) \cdot\mathbb{P}(v_t \mid f_t, u_t, \mathcal{G}_t). \tag*{(1)}

Detailed descriptions of the molecular generator are provided in Appendix § 10.

Method

The three components of BindMol address complementary stages of one design problem. Energy-based fragment scoring (§ 4.1) constructs a compact, target-adapted molecular search space; hotspot-aware vector rewards (§ 4.2) formulate preference-controllable exploration of binding modes; and Envelope SAC (§ 4.3) optimizes the resulting conflicting objectives in the combinatorial generation process. The overall framework is illustrated in Figure 2.

Figure 2. Overview of the BindMol consisting of three components: energy-based fragment scoring, Envelope SAC, and hotspot-level reward design. (Top left) Energy-based fragment scoring selects stabilizing fragments by an energy-based model trained via noise contrastive estimation. (Bottom left) Overview of the Envelope SAC workflow (double QθQ_{\theta} models omitted for clarity). (Top right) Hotspot-aware rewards are computed from the sum of interaction counts and docking scores, followed by normalization. (Bottom right) Illustration of envelope utility: unlike scalarized RL without maximization over preference, our method identifies the optimal utility for each preference.

Energy-Based Fragment Scoring

We formulate the identification of goal-aware fragments as a stability analysis problem within a learned thermodynamic potential. We hypothesize that high-activity molecules represent metastable states in a joint structure-activity landscape [7]. Consequently, critical substructures act as stabilizing anchors; their perturbation should yield a significant energy fluctuation, pushing the system towards an unstable manifold.

Let the joint state space be Z=G×Y\mathcal{Z}=\mathcal{G}\times\mathcal{Y}, where G=(Vg,Eg)\mathcal{G}=(\mathcal{V}_g,\mathcal{E}_g) denotes the molecular graph with node feature matrix H∈R∣Vg∣×d\mathbf{H}\in\mathbb{R}^{\lvert V_g\rvert\times d}, and y∈Y⊂Ry\in\mathcal{Y}\subset\mathbb{R} is the target property. We model the compatibility probability density as a Boltzmann distribution pθ(z)∝e−Eθ(z)p_\theta(z)\propto e^{-E_\theta(z)}, governed by a parameterized energy function Eθ:Z→RE_\theta:\mathcal{Z}\to\mathbb{R} [33]. The energy architecture employs a message-passing encoder ΦG:G→Rh\Phi_G:\mathcal{G}\to\mathbb{R}^h and a scalar projection Φy:Y→Rh\Phi_y:\mathcal{Y}\to\mathbb{R}^h. Their shared output width is a compact implementation choice, not a mathematical requirement: ΦG\Phi_G summarizes the graph, whereas Φy\Phi_y lifts the scalar property into a conditioning representation, and different widths could also be concatenated. The scalar potential is derived via a fused non-linear map Ψ\Psi:

Eθ(G,y):=Ψ(ΦG(H,Eg)∥Φy(y)),(2)E_\theta(\mathcal{G},y):=\Psi\left(\Phi_G(\mathbf{H},\mathcal{E}_g)\mathbin{\|}\Phi_y(y)\right), \tag*{(2)}

where ∥\mathbin{\|} denotes concatenation. To ensure the physical validity of the landscape, we train EθE_\theta using Noise Contrastive Estimation (NCE) [21]. We define the training objective J(θ)\mathcal{J}(\theta) to maximize the discriminative margin between the empirical distribution Pdata\mathbb{P}_{\mathrm{data}} and a generated noise distribution Pnoise\mathbb{P}_{\mathrm{noise}}:

J(θ)=E(G,y)∼Pdata,(G~,y~)∼Pnoise[log⁡(1+eEθ(G,y)−Eθ(G~,y~))].(3)\mathcal{J}(\theta)=\mathbb{E}_{(G,y)\sim\mathbb{P}_{\mathrm{data}},(\widetilde{G},\widetilde{y})\sim\mathbb{P}_{\mathrm{noise}}}\left[\log\left(1+e^{E_\theta(G,y)-E_\theta(\widetilde{G},\widetilde{y})}\right)\right]. \tag*{(3)}

Here, Pnoise\mathbb{P}_{\mathrm{noise}} is constructed by stochastically corrupting either the topological structure Eg\mathcal{E}_g or the property label yy, forcing the model to assign global minima only to chemically valid and property-matched pairs. The empirical distribution may be formed from experimentally annotated target ligands or, for an underexplored target, from a public library scored by virtual screening. Thus, more target-relevant data improves the learned landscape, but the intended prerequisite is a reliable target structure and pocket definition rather than a large experimental activity set; a dataset-size analysis is provided in Section 6.3.

Upon convergence, we utilize EθE_\theta to quantify the contribution of substructures. For a molecule decomposed into a fragment set {Fk}k=1K\{\mathcal{F}_k\}_{k=1}^{K}, let PFk⊥:R∣Vg∣×d→R∣Vg∣×d\mathcal{P}_{\mathcal{F}_k}^{\perp}:\mathbb{R}^{\lvert V_g\rvert\times d}\to\mathbb{R}^{\lvert V_g\rvert\times d} be an orthogonal projection operator that effectively filters out the signal from fragments Fk\mathcal{F}_k (i.e., masking). The structural importance I(Fk)\mathcal{I}(\mathcal{F}_k) is defined as the variation in potential energy under this projection:

I(Fk):=δEθ(G,y;Fk)=Eθ(PFk⊥(H),Eg,y)−Eθ(H,Eg,y).(4)\mathcal{I}(\mathcal{F}_k):=\delta E_\theta(\mathcal{G},y;\mathcal{F}_k)=E_\theta\left(\mathcal{P}_{\mathcal{F}_k}^{\perp}(\mathbf{H}),\mathcal{E}_g,y\right)-E_\theta(\mathbf{H},\mathcal{E}_g,y). \tag*{(4)}

A high value of I(Fk)\mathcal{I}(\mathcal{F}_k) implies that the fragment Fk\mathcal{F}_k is essential for maintaining the low-energy state associated with high bioactivity.

Theoretical Understanding. The following result connects the energy change induced by fragment masking to the path-integrated gradient of the learned energy function, providing a principled fragment-importance signal rather than a purely heuristic score.

Theorem 1 (Manifold Gradient Equivalence). Let γ:[0,1]→R∣Vg∣×d\gamma:[0,1]\to\mathbb{R}^{\lvert V_g\rvert\times d} be a linear path parameterizing the feature ablation from H\mathbf{H} to PFk⊥H\mathcal{P}_{\mathcal{F}_k}^{\perp}\mathbf{H}. Assuming EθE_\theta is continuously differentiable, the discrete energy variation I(Fk)\mathcal{I}(\mathcal{F}_k) is exactly the path integral of the directional derivatives along γ\gamma:

I(Fk)=−∫01⟨∇HEθ(γ(t),y),1Fk⊙H⟩F dt,(5)\mathcal{I}(\mathcal{F}_k)=-\int_0^1\left\langle\nabla_{\mathbf{H}}E_\theta(\gamma(t),y),\mathbf{1}_{\mathcal{F}_k}\odot\mathbf{H}\right\rangle_{\mathrm{F}}\,\mathrm{d}t, \tag*{(5)}

where ⟨⋅,⋅⟩F\langle\cdot,\cdot\rangle_{\mathrm{F}} denotes the Frobenius inner product, 1Fk\mathbf{1}_{\mathcal{F}_k} is the indicator mask for fragment Fk\mathcal{F}_k, and ⊙\odot is the Hadamard product.

In essence, Theorem 1 characterizes the learned energy-gap score as an integrated sensitivity measure along the masking path. Detailed proofs, error analysis, and conditions for ranking consistency are provided in Appendix § 11; pseudocode is given in Algorithm 1.

Hotspot-Based Multi-Objective Reward Design

To guide the agent in exploring diverse binding modes rather than converging to a single dominant interaction pattern, we formulate the molecular generation task as a multi-objective optimization problem. We decompose the binding pocket into distinct spatial subpockets based on structural information derived from reference ligand-protein complexes from Protein Data Bank (PDB) [2].

Let D={(Pk,Lk)}k=1K\mathcal{D} = \{(\mathcal{P}_k, \mathcal{L}_k)\}_{k=1}^{K} denote a dataset of KK reference complexes for the target protein, where each entry consists of a protein structure Pk\mathcal{P}_k and its corresponding ligand Lk\mathcal{L}_k. We represent the geometry of each residue vv by its side-chain centroid cv\mathbf{c}_v, and the ligand Lk\mathcal{L}_k by its set of heavy atom coordinates A(Lk)\mathcal{A}(\mathcal{L}_k). For each complex, the binding site residue set Rk\mathcal{R}_k is identified via a centroid-to-atom distance threshold δ\delta (e.g., 4Å):

Rk={v∈Pk | min⁡a∈A(Lk)∥cv−a∥2<δ}.(6)\mathcal{R}_k = \left\{v \in\mathcal{P}_k \,\middle|\, \min_{a \in\mathcal{A}(\mathcal{L}_k)} \lVert\mathbf{c}_v - a \rVert_2 < \delta\right\}. \tag*{(6)}

To maximize the distinctiveness of the objectives and reduce the bias toward common binding residues, we construct a non-overlapping residue set V\mathcal{V} by excluding the intersection of all binding sites from their union:

V=(⋃k=1KRk)∖(⋂k=1KRk).(7)\mathcal{V} = \left(\bigcup_{k=1}^{K} \mathcal{R}_k\right) \setminus\left(\bigcap_{k=1}^{K} \mathcal{R}_k\right). \tag*{(7)}

Excluding the intersection does not discard conserved interactions from optimization: the global docking term below is shared by every objective and maintains pressure toward overall affinity. Instead, the hotspot-specific terms use differentiating residues; otherwise, conserved contacts can correlate all reward dimensions and let the policy improve every objective through the same anchor. This set V\mathcal{V} serves as the basis for subpocket identification. We partition it into mm spatial subpockets {S1,…,Sm}\{S_1, \ldots, S_m\} using spectral clustering [50] on the Gaussian kernel of pairwise centroid distances. A full-union ablation is reported in Table 9, and clustering details are provided in Section C.

MethodPARP1FA75HT1BBRAFJAK2
Full union (including intersection)35.23 ±\pm 0.6318.33 ±\pm 0.8935.21 ±\pm 0.5324.21 ±\pm 0.9638.67 ±\pm 0.91
Non-overlapping residues42.62 ±\pm 0.7324.58 ±\pm 0.6542.15 ±\pm 0.1330.77 ±\pm 0.9744.86 ±\pm 0.98

Table 9. Effect of retaining conserved residues in every hotspot objective. Novel hit ratio (%) over three runs.

The reward function evaluates the generated molecule M\mathcal{M} based on both global binding affinity and local subpocket interactions. We define the reward vector r∈Rm\mathbf{r} \in\mathbb{R}^{m}, where the jj-th component rjr_j corresponds to the jj-th subpocket SjS_j and consists of two terms: rj(M)=ρdock(M)+ρhot(M,Sj)r_j(\mathcal{M}) = \rho_{\mathrm{dock}}(\mathcal{M}) + \rho_{\mathrm{hot}}(\mathcal{M}, S_j). Here, ρdock\rho_{\mathrm{dock}} represents the docking score normalized to a fixed range (e.g., [0,1][0, 1]). Docking is performed once for each completed molecule, rather than after every fragment action; PLIP [46] reuses that pose to compute ρhot\rho_{\mathrm{hot}} without another docking call.

Let η(v,M)\eta(v, \mathcal{M}) be the count of non-covalent interactions between molecule M\mathcal{M} and residue vv. We first compute the average interaction density within the subpocket, denoted as ηˉj=1∣Sj∣∑v∈Sjη(v,M)\bar{\eta}_j = \frac{1}{|S_j|}\sum_{v \in S_j} \eta(v, \mathcal{M}). This value is then clipped and normalized to encourage valid contacts without over-saturation: ρhot(M,Sj)=110min⁡(ηˉj,10)\rho_{\mathrm{hot}}(\mathcal{M}, S_j) = \frac{1}{10}\min(\bar{\eta}_j, 10). This formulation ensures that the agent optimizes for molecules that are not only energetically favorable globally but also form dense, specific interaction patterns with the target hotspots.

Envelope Soft Actor-Critic

The reward formulation in § 4.2 yields a vector r∈Rm\mathbf{r} \in\mathbb{R}^{m}, representing interactions across spatially disjoint subpockets. Reducing this vector to a scalar via fixed weights prevents the agent from dynamically exploring trade-offs between different hotspots. To address this, we propose Envelope SAC, which learns a universal policy capable of approximating the entire Pareto frontier by conditioning on a preference vector ω∈Ω\boldsymbol{\omega} \in\Omega. The training pipeline of Envelope SAC is described in Algorithm 2 in Section A.

Envelope target. We parameterize the critic as a vectorized function Qθ:S×A×Ω→RmQ_{\theta} : \mathcal{S} \times\mathcal{A} \times\Omega\to\mathbb{R}^{m}, which estimates the expected discounted return for each objective dimension independently. A key challenge in MORL is that the optimal policy for a specific preference query ω\boldsymbol{\omega} is initially unknown. Standard off-policy updates, which use a′∼πϕ(⋅∣s′,ω)a' \sim\pi_{\phi}(\cdot\mid s', \boldsymbol{\omega}) to update Qθ(⋅,⋅,ω)Q_{\theta}(\cdot, \cdot, \boldsymbol{\omega}), suffer from inefficiency as they ignore potentially superior actions discovered by the policy when conditioned on other preferences. Given a batch of sampled preferences {ωk}k=1K\{\omega_k\}_{k=1}^{K}, we generate a candidate action set Acand={ak′∼πϕ(⋅∣s′,ωk)}k=1K\mathcal{A}_{\mathrm{cand}} = \{a'_k \sim\pi_\phi(\cdot|s', \omega_k)\}_{k=1}^{K} for the next state s′s'. Instead of restricting the update to the action generated by ωi\omega_i, we allow the critic to "borrow" the best action from the entire candidate pool. The envelope target yiy_i for preference ωi\omega_i is computed as:

yi=r+γ(1−d)Qθˉ(s′,ak∗′,ωi),where k∗=arg⁡max⁡k∈{1,…,K} ⟨ωi,Qθˉ(s′,ak′,ωi)⟩.(8)y_i = r + \gamma(1-d)Q_{\bar{\theta}}(s', a'_{k^*}, \omega_i), \qquad \text{where } k^* = \underset{k\in\{1,\ldots,K\}}{\arg\max}\ \langle\omega_i, Q_{\bar{\theta}}(s', a'_k, \omega_i)\rangle. \tag*{(8)}

By selecting the index k∗k^* maximizing scalar utility for ωi\omega_i, this operator effectively approximates the convex hull of the Pareto front, accelerating convergence by leveraging preference diversity.

Homotopy-based critic update. The optimization of the critic presents a two-fold challenge: accurate estimation of the multi-dimensional return vector and maximization of the projected scalar utility. Training solely on scalarized utility leads to information loss regarding individual objectives, while training solely on vector outputs may fail to prioritize the specific preference query ω\omega.

To resolve this, we employ a homotopy-based optimization LQ(θ)\mathcal{L}_Q(\theta) [54] that linearly interpolates between vector-wise regression and scalar-wise alignment using a dynamic coefficient β\beta:

LQ(θ)=EB[β⋅(⟨ω,Qθ(s,a,ω)⟩−⟨ω,y⟩)2+(1−β)⋅∥Qθ(s,a,ω)−y∥22].(9)\mathcal{L}_Q(\theta) = \mathbb{E}_{\mathcal{B}}\left[\beta\cdot\left(\langle\omega, Q_\theta(s,a,\omega)\rangle- \langle\omega,y\rangle\right)^2 + (1-\beta) \cdot\left\|Q_\theta(s,a,\omega)-y\right\|_2^2\right]. \tag*{(9)}

The coefficient β∈[0,1]\beta\in[0,1] serves as a curriculum schedule. In the early training stage (β→0)(\beta\to0), the optimization prioritizes Lvec\mathcal{L}_{\mathrm{vec}} to ensure the critic accurately learns the structure of the objective space. As training progresses (β→1)(\beta\to1), the focus shifts to Lutil\mathcal{L}_{\mathrm{util}} to refine the policy alignment with the preferences, ensuring optimality on the Pareto frontier.

Preference-aware policy update. The actor πϕ(a∣s,ω)\pi_\phi(a|s,\omega) is trained to maximize the expected envelope utility subject to entropy regularization. By conditioning on ω\omega, the policy learns a continuous mapping from preference weights to the optimal action manifold. The policy loss is defined as:

Lπ(ϕ)=Es∼D,ω∼Ω[αlog⁡πϕ(a~∣s,ω)−min⁡j=1,2⟨ω,Qθj(s,a~,ω)⟩].(10)\mathcal{L}_\pi(\phi) = \mathbb{E}_{s\sim\mathcal{D},\omega\sim\Omega}\left[\alpha\log\pi_\phi(\tilde{a}|s,\omega)-\min_{j=1,2}\langle\omega,Q_{\theta_j}(s,\tilde{a},\omega)\rangle\right]. \tag*{(10)}

where a~\tilde{a} is sampled via the reparameterization trick, and the minimum over two critics (j=1,2)(j=1,2) is used to mitigate value overestimation [22]. This formulation ensures the agent not only adheres to the preference query but also maintains sufficient exploration to discover diverse molecular structures.

Experiments

To comprehensively validate the effectiveness of our proposed BindMol, we conduct comprehensive experiments on two important scenarios in drug discovery: (1) binding affinity optimization (§ 5.1), and (2) similarity-based rediscovery (§ 5.2), both of which can be formulated as multi-objective optimization tasks. We further demonstrate the effectiveness of our model design through ablation experiments and case studies in § 5.3.

Experimental Settings We adhere to the task settings proposed by [35] for fair comparison. The policy model performs random sampling for the first 4,000 steps to collect experience, and parameter updates after 3,000 steps to ensure a sufficient replay buffer. Every method uses the same evaluation budget of 3,000 completed molecules and docking calls. We report Novel hit ratio and Novel top 5% Docking Score for hit discovery [59], area under the curve (AUC) of top-KK value for rediscovery [16], and chemical-space coverage (#Circles) [56]. Please refer to Section B for experimental settings, evaluation protocols, baselines, and compute profiling.

Optimization of Binding Affinity

For binding affinity optimization, we aim to design novel, drug-like, and synthesizable binders to the specific target. Therefore, we validate BindMol on five protein targets, parp1, fa7, 5ht1b, braf, and jak2, under the quantitative estimate of drug-likeness (QED) [3], synthetic accessibility (SA) [9], and novelty constraints following [34].

Table 2: Novel top 5% docking score (kcal/mol) results. The results are the means and the standard deviations of 3 runs. The best results are highlighted in bold.

MethodTarget protein
parp1fa75ht1bbrafjak2
REINVENT [41]-8.702 (±0.523\pm0.523)-7.205 (±0.264\pm0.264)-8.770 (±0.316\pm0.316)-8.392 (±0.400\pm0.400)-8.165 (±0.277\pm0.277)
Graph GA [24]-10.949 (±0.532\pm0.532)-7.365 (±0.326\pm0.326)-10.422 (±0.670\pm0.670)-10.789 (±0.341\pm0.341)-10.167 (±0.576\pm0.576)
MORLD [25]-7.532 (±0.260\pm0.260)-6.263 (±0.165\pm0.165)-7.869 (±0.650\pm0.650)-8.040 (±0.337\pm0.337)-7.816 (±0.133\pm0.133)
HierVAE [26]-9.487 (±0.278\pm0.278)-6.812 (±0.274\pm0.274)-8.081 (±0.252\pm0.252)-8.978 (±0.525\pm0.525)-8.285 (±0.370\pm0.370)
RationaleRL [27]-10.663 (±0.086\pm0.086)-8.129 (±0.048\pm0.048)-9.005 (±0.155\pm0.155)No hit found-9.398 (±0.076\pm0.076)
FREED [59]-10.579 (±0.104\pm0.104)-8.378 (±0.044\pm0.044)-10.714 (±0.183\pm0.183)-10.561 (±0.080\pm0.080)-9.735 (±0.022\pm0.022)
PS-VAE [30]-9.978 (±0.091\pm0.091)-8.028 (±0.050\pm0.050)-9.887 (±0.115\pm0.115)-9.637 (±0.049\pm0.049)-9.464 (±0.129\pm0.129)
MOOD [34]-10.865 (±0.113\pm0.113)-8.160 (±0.071\pm0.071)-11.145 (±0.042\pm0.042)-11.063 (±0.034\pm0.034)-10.147 (±0.060\pm0.060)
GDSS [28]-9.967 (±0.028\pm0.028)-7.775 (±0.039\pm0.039)-9.459 (±0.101\pm0.101)-9.224 (±0.068\pm0.068)-8.926 (±0.089\pm0.089)
RetMol [53]-8.590 (±0.475\pm0.475)-5.448 (±0.688\pm0.688)-6.980 (±0.740\pm0.740)-8.811 (±0.574\pm0.574)-7.133 (±0.242\pm0.242)
GEAM [35]-12.536 (±0.115\pm0.115)-9.611 (±0.019\pm0.019)-12.385 (±0.059\pm0.059)-12.165 (±0.092\pm0.092)-11.822 (±0.035\pm0.035)
MOLLEO [51]-11.981 (±0.125\pm0.125)-8.921 (±0.008\pm0.008)-12.588 (±0.039\pm0.039)-11.365 (±0.117\pm0.117)-10.988 (±0.019\pm0.019)
BindMol (ours)-12.930 (±0.055\pm0.055)-10.189 (±0.023\pm0.023)-13.028 (±0.097\pm0.097)-12.497 (±0.087\pm0.087)-12.011 (±0.015\pm0.015)

Table 2.

Performance comparison. Empirical results are summarized in Table 1 to Table 3. Our systematic study suggests the following trends: (i) BindMol achieves state-of-the-art performance across all targets, highlighting its ability to generate novel, drug-like, synthesizable compounds with strong binding affinity. In particular, for challenging targets such as fa7, our method attains a novel hit ratio above 24%, surpassing GEAM (17%) and far outperforming most competing methods, many of which approach zero. (ii) Fragment-based reinforcement learning methods are especially effective at exploring high-reward regions. FREED, MOOD, GEAM, and our BindMol all adopt fragment-based RL strategies and consistently produce lower docking scores. This advantage arises from a well-defined action space: fragment-level operations are limited yet high-quality, enabling more efficient exploration than token-based SMILES generation. *(iii) BindMol achieves higher #Circle while maintaining a strong hit ratio, indicating that the generated molecules are not only effective but also diverse. This reflects the benefit of explorative preferences, which encourage RL to explore a wider range of action sequences and thereby cover chemical space more comprehensively. By contrast, RationaleRL achieves relatively high diversity on 5ht1b and jak2, but its overly aggressive exploration leads to very low hit discovery rates.

Table 1: Novel hit ratio (%) results. The results are the means and the standard deviations of 3 runs.

MethodTarget protein
parp1fa75ht1bbrafjak2
REINVENT [41]0.480 (±0.344\pm0.344)0.213 (±0.081\pm0.081)2.453 (±0.561\pm0.561)0.127 (±0.088\pm0.088)0.613 (±0.167\pm0.167)
Graph GA [24]4.811 (±1.661\pm1.661)0.422 (±0.193\pm0.193)7.011 (±2.732\pm2.732)3.767 (±1.498\pm1.498)5.311 (±1.667\pm1.667)
MORLD [25]0.047 (±0.050\pm0.050)0.007 (±0.013\pm0.013)0.880 (±0.735\pm0.735)0.047 (±0.040\pm0.040)0.227 (±0.118\pm0.118)
HierVAE [26]0.553 (±0.214\pm0.214)0.007 (±0.013\pm0.013)0.507 (±0.278\pm0.278)0.207 (±0.220\pm0.220)0.227 (±0.127\pm0.127)
RationaleRL [27]4.267 (±0.450\pm0.450)0.900 (±0.098\pm0.098)2.967 (±0.307\pm0.307)0.000 (±0.000\pm0.000)2.967 (±0.196\pm0.196)
FREED [59]4.627 (±0.727\pm0.727)1.332 (±0.113\pm0.113)16.767 (±0.897\pm0.897)2.940 (±0.359\pm0.359)5.800 (±0.295\pm0.295)
PS-VAE [30]1.644 (±0.389\pm0.389)0.478 (±0.140\pm0.140)12.622 (±1.437\pm1.437)0.367 (±0.047\pm0.047)4.178 (±0.933\pm0.933)
MOOD [34]7.017 (±0.428\pm0.428)0.733 (±0.141\pm0.141)18.673 (±0.423\pm0.423)5.240 (±0.285\pm0.285)9.200 (±0.524\pm0.524)
GDSS [28]1.933 (±0.208\pm0.208)0.368 (±0.103\pm0.103)4.667 (±0.306\pm0.306)0.167 (±0.134\pm0.134)1.167 (±0.281\pm0.281)
RetMol [53]0.011 (±0.016\pm0.016)0.000 (±0.000\pm0.000)0.033 (±0.027\pm0.027)0.000 (±0.000\pm0.000)0.011 (±0.016\pm0.016)
GEAM [35]40.635 (±0.763\pm0.763)17.903 (±1.220\pm1.220)36.667 (±1.929\pm1.929)27.887 (±1.584\pm1.584)42.025 (±1.187\pm1.187)
MOLLEO [51]35.725 (±0.825\pm0.825)20.651 (±1.573\pm1.573)32.489 (±0.533\pm0.533)25.112 (±1.837\pm1.837)39.120 (±1.025\pm1.025)
BindMol (ours)42.617 (±0.733\pm0.733)24.580 (±0.652\pm0.652)42.149 (±0.125\pm0.125)30.767 (±0.974\pm0.974)44.864 (±0.982\pm0.982)

Table 1.

Table 3: #Circles of generated hit molecules. The #Circles threshold is set to 0.75. The results are the means and the standard deviations of 3 runs. The best results are highlighted in bold.

MethodTarget protein
parp1fa75ht1bbrafjak2
REINVENT [41]44.2 (±15.5\pm15.5)23.2 (±6.6\pm6.6)138.8 (±19.4\pm19.4)18.0 (±2.1\pm2.1)59.6 (±8.1\pm8.1)
MORLD [25]1.4 (±1.5\pm1.5)0.2 (±0.4\pm0.4)22.2 (±16.1\pm16.1)1.4 (±1.2\pm1.2)6.6 (±3.7\pm3.7)
HierVAE [26]4.8 (±1.6\pm1.6)0.8 (±0.7\pm0.7)5.8 (±1.0\pm1.0)3.6 (±1.4\pm1.4)4.8 (±0.7\pm0.7)
RationaleRL [27]61.3 (±1.2\pm1.2)2.0 (±0.0\pm0.0)312.7 (±6.3\pm6.3)1.0 (±0.0\pm0.0)199.3 (±7.1\pm7.1)
FREED [59]34.8 (±4.9\pm4.9)21.2 (±4.0\pm4.0)88.2 (±13.4\pm13.4)34.4 (±8.2\pm8.2)59.6 (±8.2\pm8.2)
PS-VAE [30]38.0 (±6.4\pm6.4)18.0 (±5.9\pm5.9)180.7 (±11.6\pm11.6)16.0 (±0.8\pm0.8)83.7 (±11.9\pm11.9)
MOOD [34]86.4 (±11.2\pm11.2)19.2 (±4.0\pm4.0)144.4 (±15.1\pm15.1)50.8 (±3.8\pm3.8)81.8 (±5.7\pm5.7)
GEAM [35]123.0 (±7.8\pm7.8)79.0 (±9.2\pm9.2)144.3 (±8.6\pm8.6)84.7 (±8.6\pm8.6)118.3 (±0.9\pm0.9)
MOLLEO [51]74.2 (±2.5\pm2.5)56.1 (±3.2\pm3.2)125.3 (±7.9\pm7.9)68.1 (±5.4\pm5.4)95.2 (±3.3\pm3.3)
BindMol (ours)136.7 (±4.8\pm4.8)97.5 (±3.3\pm3.3)173.2 (±7.1\pm7.1)103.2 (±5.1\pm5.1)132.3 (±0.9\pm0.9)

Table 3.

Multi-residue exploration. We evaluate the ability of BindMol to explore diverse binding modes using the Hypervolume metric, which quantifies the dominated volume of the Pareto front with respect to a preference point. As multi-objective hotspot optimization typically involves three to five objectives, many existing methods become computationally prohibitive. We therefore compare against GEAM [35], a strong baseline for binding tasks, and GPI [1], a multi-objective reinforcement learning method scalable to 3–5 objectives. As shown in Figure 3, multi-objective RL methods achieve broader Pareto-front coverage than GEAM, reflecting improved balance across competing interaction hotspots. Our method consistently attains the most extensive coverage across all five objectives, spanning multiple trade-off directions. This uniform Pareto exploration indicates that BindMol avoids collapsing to a single optimization mode and instead generates molecules with more diverse binding modes.

Figure 3. The Hypervolume comparison of BindMol.

Direct binding-mode diversity. Unlike #Circles and Hypervolume, which measure ligand-space and objective-space coverage, respectively, the PLIP-IFP metrics in Table 5 directly compare docking-predicted interaction patterns. BindMol yields more clusters and more balanced mode populations than GEAM on every target. We additionally select BRAF and JAK2 because they provide well-established, within-target pairs of type-I/DFG-in and type-II/DFG-out crystal complexes; the other targets lack equally clear paired mode labels. Reference-pair IFP/residue Jaccard similarities are 0.375/0.462 for BRAF and 0.188/0.250 for JAK2, and BindMol recovers all four crystallographic modes more frequently than the baselines.

MetricMethodPARP1FA75HT1BBRAFJAK2
IFP clustersGEAM138.6761.678.3320.3347.33
IFP clustersBindMol192.0094.3311.6735.0077.33
Entropy HHGEAM2.931.850.240.681.52
Entropy HHBindMol3.452.370.321.112.31
Effective modesGEAM18.736.361.271.974.57
Effective modesBindMol31.4010.711.373.0310.10
ModeBindMolGEAMFREED
BRAF DFG-in / I48.920.311.7
BRAF DFG-out / II12.16.40.0
JAK2 type-I1.30.00.0
JAK2 type-II11.55.72.8
MethodHit (%)DS
GPI32.15−10.60
C-MORL32.35−10.63
PreCo32.87−10.73
BindMol37.01−12.13

Table 5. Direct binding-mode and matched MORL evaluation. IFP statistics are computed from full-pocket PLIP residue–interaction-type fingerprints; GEAM is the strongest baseline on all three statistics. Recovery denotes the percentage of generated hits with IFP Jaccard similarity ≥0.4\ge0.4 to the native crystal ligand. MORL results average the five targets; complete per-target results and protocols are in Section G.1. (a) Interaction-pattern diversity. (b) Crystallographic-mode recovery (%). (c) Matched MORL averages.

Matched MORL comparison. We adapt GPI, C-MORL, and PreCo to the same molecular action space, hotspot reward vector, training protocol, docking budget, and three seeds. Table 5(c) summarizes the gains, with full per-target results in Table 8. Our claim is specific to preference-conditioned molecular exploration with 3–5 hotspot objectives, rather than general-purpose MORL superiority.

MethodPARP1FA75HT1BBRAFJAK2
GPI37.38 ±\pm 1.1219.84 ±\pm 0.9137.67 ±\pm 0.7427.53 ±\pm 1.0838.31 ±\pm 1.16
C-MORL38.74 ±\pm 0.9520.92 ±\pm 0.8336.46 ±\pm 0.6227.38 ±\pm 0.8938.27 ±\pm 1.04
PreCo38.81 ±\pm 0.8821.67 ±\pm 0.7237.93 ±\pm 0.5126.84 ±\pm 1.0339.12 ±\pm 0.92
BindMol42.62 ±\pm 0.7324.58 ±\pm 0.6542.15 ±\pm 0.1330.77 ±\pm 0.9744.86 ±\pm 0.98
MethodPARP1FA75HT1BBRAFJAK2
GPI−10.98±0.14-10.98 \pm0.14−9.01±0.11-9.01 \pm0.11−11.02±0.16-11.02 \pm0.16−10.31±0.13-10.31 \pm0.13−10.67±0.12-10.67 \pm0.12
C-MORL−11.85±0.10-11.85 \pm0.10−8.92±0.08-8.92 \pm0.08−11.24±0.12-11.24 \pm0.12−10.63±0.11-10.63 \pm0.11−10.51±0.09-10.51 \pm0.09
PreCo−11.24±0.08-11.24 \pm0.08−9.17±0.05-9.17 \pm0.05−11.61±0.11-11.61 \pm0.11−10.88±0.10-10.88 \pm0.10−10.73±0.06-10.73 \pm0.06
BindMol−12.93±0.06-12.93 \pm0.06−10.19±0.02-10.19 \pm0.02−13.03±0.10-13.03 \pm0.10−12.50±0.09-12.50 \pm0.09−12.01±0.02-12.01 \pm0.02

Table 8. Matched MORL comparison. Means and standard deviations over three runs. Novel hit ratio (%) and novel top-5% docking score.

Optimization of Multi-property Objectives in GuacaMol

For similarity-based rediscovery, our goal is to generate molecules similar to the target compound and fulfill multiple required property profiles. Following [16], we select the multi-property optimization tasks from the GuacaMol benchmarks [6] for capability evaluation. Notably, unlike prior studies that scalarize multiple objectives into a single target, we treat them as a genuine multi-objective problem to better match our algorithm. To ensure fairness, we use each individual objective score prior to scalarization as the single entry of multi-objective reward.

Performance comparison. Empirical results for MPO tasks are presented in Table 4 and Table 12. It can be observed that BindMol significantly outperforms all baseline models across all MPO tasks, which further demonstrates the effectiveness of the proposed Envelope SAC algorithm. On the two rediscovery tasks of Fexofenadine and Sitagliptin, it achieves relative improvements of 7.1% and 17.8% over the second-best models, respectively. Moreover, on the most challenging Zaleplon rediscovery task, BindMol attains a record-breaking score of 0.418. It is worth noting that MOLLEO (MoleculeSTM), a framework that applies genetic algorithms over large-scale molecular corpora with language models, is still outperformed by our method on rediscovery tasks. We exclude other variants of MOLLEO (e.g., those using GPT-4 as the base LLM) from comparison due to their unfair advantage in this setting.

MethodAmlodipineFexofenadineOsimertinibPerindoprilRanolazineSitagliptinZaleplonAverage
REINVENT [41]0.6080.7520.8060.5110.7190.0060.3250.532
MolDQN [62]0.2300.4310.6360.1250.0180.0000.0020.206
Graph GA [24]0.6220.7310.7990.5030.6700.3300.3050.566
Graph MCTS [24]0.3850.5220.6550.2190.6940.1170.1650.394
DoG-Gen [4]0.4890.6400.7060.4220.6010.0150.0730.421
GPBO [47]0.5380.6850.7500.4600.6940.1170.1650.487
STONED [39]0.5930.7770.7990.4720.7380.3510.3070.577
SynNet [15]0.5330.7200.7590.5120.6900.0070.2230.492
DST [14]0.4690.6900.7420.4250.5790.0170.0890.430
GEAM [35]0.6270.7860.8170.5140.7140.4080.3860.607
Augmented Memory [19]0.6070.6570.8180.5180.7250.3980.3550.582
MOLLEO (MoleculeSTM) [51]0.6170.7650.7990.5160.6750.4150.3920.597
BindMol (Ours)0.6710.8420.8450.5330.7480.4890.4180.649

Table 4. Table 4: GuacaMol MPO AUC Top-100 results. The results are the means of 3 runs. We highlight the best- and the second-performing results in boldface and <u>underlined</u>, respectively.

MethodAmlodipineFexofenadineOsimertinibPerindoprilRanolazineSitagliptinZaleplon
REINVENT [41]303.7343.3452.3318.3253.3398.3275.3
Graph GA [24]258.7333.3270.3278.7<u>364.7</u>306.3272.7
STONED [39]303.7330.3301.3301.0316.7326.3280.3
GEAM-static [35]412.0397.7315.3318.0256.7233.0267.0
GEAM [35]424.0<u>502.0</u>435.0377.3295.3257.0<u>336.0</u>
BindMol (Ours)480.5542.1475.5<u>374.8</u>405.2<u>352.7</u>362.6

Table 12. GuacaMol MPO #Circles results. The results are the means of 3 runs. The best and second-best results are highlighted in bold and underlined, respectively.

Ablation Studies

We perform ablation studies to isolate the contributions of the key components in our framework. Specifically, we remove energy-based fragment scoring and construct the action space from a default fragment library (BindMol-r); ablate the hotspot-aware reward and use only target-wise docking scores for optimization (BindMol-h); and remove the envelope target, reducing Envelope SAC to standard SAC with vector-valued rewards (BindMol-v). As shown in Figure 4, removing any component leads to consistent performance degradation in both binding affinity and chemical-space coverage. The energy-based vocabulary most directly improves affinity-oriented search, explaining the larger decline of BindMol-r on docking metrics. The hotspot reward serves a different role: removing it reduces BRAF IFP clusters from 35.00±\pm1.41 to 25.67±\pm1.53 and effective modes from 3.03±\pm0.08 to 2.20±\pm0.08 (Table 10), directly supporting its contribution to interaction-pattern diversity. The envelope update further improves preference-conditioned exploration, as reflected by BindMol-v.

Performance of different ablation variants of BindMol

Figure 4. Performance of different ablation variants of BindMol. DS denotes Docking Score.

MethodIFP clustersIFP entropy HHEffective modes
BindMol-h25.67 ±\pm 1.530.89 ±\pm 0.032.20 ±\pm 0.08
BindMol35.00 ±\pm 1.411.11 ±\pm 0.083.03 ±\pm 0.08

Table 10. Interaction-level contribution of the hotspot reward on BRAF.

Case Studies

To provide an intuitive understanding of the binding modes exploration of BindMol, we cluster the generated molecules according to the number of protein–ligand interactions across different targets using k-means (k=2k = 2). For each cluster, we select and visualize representative samples to examine their interaction patterns. Specifically, we perform clustering analysis on the parp1 and jak2 targets, resulting in four protein–ligand complexes for visualization, as shown in Figure 5.

Visualization of representative binding modes on two protein complexes

Figure 5. Visualization of representative binding modes on two protein complexes: parp1 and jak2.

In the parp1 complexes, the two ligands exhibit distinct binding modes: PARP1-L1, with a polyaromatic scaffold, is dominated by aromatic stacking with Y228 and Y246, whereas PARP1-L2, featuring a heteroaromatic scaffold, relies more heavily on hydrogen bonding with diminished stacking. In the jak2 complexes, the differences are also evident. JAK2-L1 adopts a stacking- and cation–π\pi-driven mode, while JAK2-L2 introduces heteroatoms and halogen substitution that favor additional hydrogen and halogen bonds, reducing dependence on R92.

Conclusion

In this work, we revisit target-based drug discovery from the perspective of binding-mode diversity and argue that optimizing a single scalar objective is insufficient to capture the complexity of protein–ligand interactions. To this end, we propose BindMol, a multi-objective reinforcement learning framework that explicitly models hotspot-level interactions and their inherent trade-offs. By combining energy-based fragment scoring, biologically grounded vectorized rewards, and the Envelope SAC algorithm for preference-aware optimization, BindMol enables efficient exploration of diverse, high-affinity binding modes within the same target. Experiments across multiple targets and benchmarks demonstrate consistent improvements over existing approaches.

Broader impacts. Given that our application tasks fall within the molecular domain, improper use of methods for tasks such as molecular property prediction may result in significant deviations. This could impact subsequent applications of the molecules in drug development or materials design, especially in predicting properties like toxicity and stability. We recommend further experimental validation of key molecules after using the model to ensure the reliability of the results.

Acknowledgements. This work is jointly supported by National Science and Technology Major Project (2023ZD0120901) and Zhongguancun Academy (XTS0047, C20250208).

Algorithms

Algorithm 1: Energy-Based Fragment Scoring

Input: Molecular dataset DD, property labels Y\mathcal{Y}, batch size BB, learning rate η\eta

Initialize energy network EθE_{\theta} parametrized by GNN, and empty vocabulary V←∅\mathcal{V} \leftarrow\varnothing;

// Phase 1: Energy Landscape Learning

repeat

Sample batch of positive pairs Breal={(Gi,yi)}i=1BB_{\mathrm{real}} = \{(G_i, y_i)\}_{i=1}^{B} from DD;

for each sample (G,y)(G, y) in BrealB_{\mathrm{real}} do

Sample noisy property y~∼puniform(Y)\tilde{y} \sim p_{\mathrm{uniform}}(\mathcal{Y});

Generate corrupted graph G~\tilde{G} by masking random node features: HG~←HG⊙Mrand\mathbf{H}_{\tilde{G}} \leftarrow\mathbf{H}_{G} \odot\mathbf{M}_{\mathrm{rand}};

Compute energy for real and noisy pairs:;

Ereal←Eθ(G,y)E_{\mathrm{real}} \leftarrow E_{\theta}(G, y); Enoise←Eθ(G,y~)E_{\mathrm{noise}} \leftarrow E_{\theta}(G, \tilde{y});

Compute InfoNCE loss to shape the landscape:;

LNCE←log⁡(1+exp⁡(Ereal−Enoise))\mathcal{L}_{\mathrm{NCE}} \leftarrow\log(1+\exp(E_{\mathrm{real}}-E_{\mathrm{noise}}));

Update parameters θ\theta by minimizing the batch loss: θ←θ−η∇θ∑LNCE\theta\leftarrow\theta-\eta\nabla_{\theta}\sum\mathcal{L}_{\mathrm{NCE}};

until Convergence;

// Phase 2: Goal-Aware Fragment Extraction

Filter high-activity molecules Dhigh⊂DD_{\mathrm{high}} \subset D;

for each molecule G∈DhighG \in D_{\mathrm{high}} do

Decompose GG into candidate fragments {F1,…,Fk}\{F_1,\ldots,F_k\} (e.g., via BRICS);

for each fragment FjF_j do

Calculate Importance Score (Energy Gap).;

I(Fj)←Eθ(G∖Fj,yhigh)−Eθ(G,yhigh)\mathcal{I}(F_j) \leftarrow E_{\theta}(G\setminus F_j,y_{\mathrm{high}})-E_{\theta}(G,y_{\mathrm{high}});

if I(Fj)>τ\mathcal{I}(F_j)>\tau then

Add fragment to vocabulary: V←V∪{Fj}\mathcal{V} \leftarrow\mathcal{V}\cup\{F_j\};

Output: Goal-Aware Fragment Vocabulary V\mathcal{V}

Algorithm 2: Envelope Soft Actor-Critic

Input: Preference sampling distribution Ω\Omega, batch size BB, number of sampled preferences KK

Initialize policy πϕ\pi_{\phi}, critics Qθ1,θ2Q_{\theta_1,\theta_2}, target critics Qθˉ1,θˉ2Q_{\bar{\theta}_1,\bar{\theta}_2}, and replay buffer DD;

repeat

Interact with the environment using πϕ(⋅∣s,ωenv)\pi_{\phi}(\cdot\mid s,\omega_{\mathrm{env}}), and store transitions (s,a,r,s′,d)(s,a,r,s',d) in DD;

for each gradient step do

Sample minibatch B={(s,a,r,s′,d)}\mathcal{B}=\{(s,a,r,s',d)\} from DD and preferences {ωi}i=1K∼Ω\{\omega_i\}_{i=1}^{K}\sim\Omega;

// Critic Update

Generate candidate actions for next state: Acand={ak′∼πϕ(⋅∣s′,ωk)}k=1K\mathcal{A}_{\mathrm{cand}}=\{a'_k\sim\pi_{\phi}(\cdot\mid s',\omega_k)\}_{k=1}^{K};

for each preference ωi\omega_i do

Select envelope action index:

k∗←arg⁡max⁡k∈{1,…,K}⟨ωi,min⁡j=1,2Qθˉj(s′,ak′,ωi)⟩k^* \leftarrow\underset{k\in\{1,\ldots,K\}}{\arg\max}\left\langle\omega_i,\min_{j=1,2}Q_{\bar{\theta}_j}(s',a'_k,\omega_i)\right\rangle;

Compute envelope target vector:

yi←r+γ(1−d)min⁡j=1,2Qθˉj(s′,ak∗′,ωi)y_i \leftarrow r+\gamma(1-d)\min_{j=1,2}Q_{\bar{\theta}_j}(s',a'_{k^*},\omega_i);

Update critics θ1,2\theta_{1,2} by minimizing the homotopy loss:

LQ=EB,ω[β(⟨ω,Qθ(s,a,ω)⟩−⟨ω,y⟩)2+(1−β)∥Qθ(s,a,ω)−y∥22]\mathcal{L}_{Q}=\mathbb{E}_{\mathcal{B},\omega}\left[\beta\left(\langle\omega,Q_{\theta}(s,a,\omega)\rangle-\langle\omega,y\rangle\right)^2+(1-\beta)\lVert Q_{\theta}(s,a,\omega)-y\rVert_2^2\right]
// Policy Update

Sample actions a~∼πϕ(⋅∣s,ω)\tilde{a}\sim\pi_{\phi}(\cdot\mid s,\omega) using reparameterization;

Update policy ϕ\phi by minimizing:

Lπ=EB,ω[αlog⁡πϕ(a~∣s,ω)−min⁡j=1,2⟨ω,Qθj(s,a,ω)⟩]\mathcal{L}_{\pi}=\mathbb{E}_{\mathcal{B},\omega}\left[\alpha\log\pi_{\phi}(\tilde{a}\mid s,\omega)-\min_{j=1,2}\langle\omega,Q_{\theta_j}(s,a,\omega)\rangle\right]

Update target critics: θˉj←ρθˉj+(1−ρ)θj\bar{\theta}_j\leftarrow\rho\bar{\theta}_j+(1-\rho)\theta_j for j=1,2j=1,2;

until Convergence;

Experimental Settings

Computing Infrastructures

Software infrastructures. All of the experiments are implemented in Python 3.10, with the following supporting libraries: PyTorch 2.1.0 [43], PyG 2.0.4 [12], DGL 2.4.0 [52], Gym 0.21.0 [5], Open Babel 3.1.1 [40], PLIP 2.3.1 [46], and RDKit 2022.03.1 [32].

Hardware infrastructures. We conduct all experiments on a computer server with 8 NVIDIA A800 GPUs and 2 Intel Xeon Gold 6348 CPUs (with 56 cores and 112 threads).

Implementation Details

Following [59, 16, 35], we base our experimental setup on the ZINC250k dataset [23], using the standard split for training and testing. For our generative model, the training process begins with an initial fragment vocabulary of 300 fragments, which is dynamically expanded to a maximum size of 1000 by adding up to 50 new fragments per cycle. Any fragments that cause RDKit sanitization errors are filtered out.

The fragment-based generator employs Envelope SAC for training, where episode starts from a benzene ring scaffold with three attachment points (ortho-, meta-, para-). An episode terminates if the heavy atom count of molecule exceeds 40. To ensure sufficient exploration, the policy model samples molecules randomly for the first 4,000 steps to populate the replay buffer, while parameter updates commence after 3,000 steps. For fair evaluation, molecules generated during this initial exploration phase are included in the final results. The fragment modification module utilizes a genetic algorithm with a population size of 100 and a mutation rate of 0.1.

All neural networks use a hidden dimension of 64 and are trained with a batch size of 256. Both the value and policy networks are optimized using the Adam optimizer [29] with a learning rate of 1×10−41 \times10^{-4}. For exploration in our multi-objective setting, we sample 5 preference vectors per batch. Moreover, we observe that as the number of objectives increases, the training benefits from a higher ratio of off-policy updates (i.e., utilizing more samples from the replay buffer relative to on-policy samples). We hypothesize that in high-dimensional objective spaces, the model requires more frequent re-visitation of past action episodes to effectively learn the complex trade-offs and stabilize convergence.

Oracle budget and runtime. Each target-based method is evaluated with the same budget of 3,000 completed molecules and docking calls. Docking is invoked only after a molecule is completed; the resulting pose is reused by PLIP, so hotspot profiling requires no additional docking call. On the same server, representative training times are 2.3, 9.7, 11.6, 12.4, and 13.2 hours for REINVENT, FREED, MOOD, GEAM, and BindMol, respectively. REINVENT benefits from its lightweight RNN, while BindMol remains comparable to fragment-based baselines. Preference vectors are replicated across the batch for parallel actor/critic evaluation rather than processed in a Python loop. With fixed poses, multi-threaded PLIP processes 10410^{4} molecules within minutes, so its overhead is small relative to docking.

Evaluation Protocols

We adopt several well-defined metrics to evaluate our framework:

  • For binding affinity optimization, we evaluate the generated molecules following [59]. A molecule is considered a hit if it achieves a docking score lower than the median of experimentally known actives, while also satisfying QED >0.5> 0.5 and SA <5< 5. To assess affinity optimization, we report the Novel top 5% Docking Score (kcal/mol), which is the average docking score of the best-performing 5% unique and novel hits. We also measure the Novel hit ratio, defined as the fraction of hits that are novel.

  • For rediscovery tasks, we report the area under the curve (AUC) of top-K average property value versus the number of oracle calls (AUC top-K) following [16] for fair comparison.

  • We further report #Circles [56] for both of the tasks, which quantifies the coverage of chemical space by the generated hits.

Baselines

Binding affinity tasks. To comprehensively evaluate the effectiveness of our approach, we select 11 competitive molecular generation models as baselines: REINVENT [41], Graph GA [24], MORLD [25], HierVAE [26], RationaleRL [27], FREED [59], PS-VAE [30], MOOD [34], GDSS [28], RetMol [53], GEAM [35] and MOLLEO [51]. These baselines span both SMILES- and graph-based modalities, and encompass diverse training paradigms including distribution learning, genetic algorithms, and reinforcement learning. Note that our model is trained from scratch, rather than finetuned from a large pretrained molecular generative model. To ensure a fair and controlled comparison under this setting, we do not include approaches that rely heavily on large-scale pretraining [42, 36].

Multi-property tasks in GuacaMol. We select 12 molecular generation models with strong performance on the GuacaMol benchmark as baselines: REINVENT [41], MolDQN [62], Graph GA [24], Graph MCTS [24], DoG-Gen [4], GPBO [47], STONED [39], SynNet [15], DST [14], GEAM [35], Augmented Memory [19], and MOLLEO (MoleculeSTM) [51]. These methods also span various molecular modalities and training strategies.

Spatial Clustering of Hotspot Residues

Given the non-overlapping residue set V\mathcal{V} defined in § 4.2, our goal is to further partition these residues into a set of spatially coherent subpockets that capture distinct local interaction patterns within the binding site. The clustering is performed purely based on the three-dimensional geometry of residues, without relying on ligand-specific distance thresholds. Each residue v∈Vv \in\mathcal{V} is represented by a three-dimensional centroid cv\mathbf{c}_v, computed from its side-chain atoms to emphasize interaction-relevant geometry. Backbone atoms (N, Cα_\alpha, C, O) are excluded whenever side-chain atoms are available; for residues lacking side chains, the centroid is computed over all atoms of the residue. This representation yields a set of residue-level points embedded in R3\mathbb{R}^3.

We construct a pairwise distance matrix D∈R∣V∣×∣V∣\mathbf{D} \in\mathbb{R}^{|\mathcal{V}| \times|\mathcal{V}|}, where each entry

Duv=∥cu−cv∥2,u,v∈V,D_{uv} = \lVert\mathbf{c}_u - \mathbf{c}_v \rVert_2,\qquad u,v \in\mathcal{V},

quantifies the spatial proximity between residues. Based on this distance matrix, we define a Gaussian affinity graph with weights

Wuv=exp⁡(−Duv2σ2),W_{uv} = \exp\left(-\frac{D_{uv}^{2}}{\sigma^{2}}\right),

where σ\sigma controls the spatial scale at which residue interactions are considered local. The diagonal entries of W\mathbf{W} are set to zero. To identify spatially separated but internally compact residue groups, we apply spectral clustering to the affinity graph. Specifically, we construct the unnormalized graph Laplacian L=DW−W\mathbf{L} = \mathbf{D}_{\mathbf{W}} - \mathbf{W}, where DW\mathbf{D}_{\mathbf{W}} is the diagonal degree matrix of W\mathbf{W}. The eigenvectors corresponding to the mm smallest eigenvalues of L\mathbf{L} are extracted and row-normalized to form a low-dimensional spectral embedding of residues in V\mathcal{V}.

Finally, kk-means clustering is performed on this embedding space to partition V\mathcal{V} into mm disjoint subsets {S1,…,Sm}\{S_1,\ldots,S_m\}, each representing a spatial hotspot within the binding pocket. These subpockets serve as the structural basis for defining preference-conditioned reward components in § 4.2. The visualization of distance matrices and clustered hotspots are provided in Figure 7.

Figure 7. Visualization results for five protein targets. The first row shows residue distance matrices, and the second row shows clustered residues (hotspots) for each target.

Fragment-based Generator

Following [59], we frame the molecular generation as a sequential decision process. The vivid illustration of this fragment-based generator is provided in Figure 6. At time step tt, the state is the partially built graph GtG_t. Each transition is achieved by a triplet action

Figure 6. Fragment-based generation sequentially attaches fragments from a fragment vocabulary to a partial graph until termination.

at=(ut,ft,vt),a_t = (u_t, f_t, v_t),

where ut∈U(Gt)u_t \in\mathcal{U}(G_t) is an attachment site in the current graph, ft∈Sf_t \in\mathcal{S} is a fragment drawn from the vocabulary S\mathcal{S}, and vt∈V(ft)v_t \in\mathcal{V}(f_t) is an attachment site on the chosen fragment. Attaching ftf_t to GtG_t at (ut,vt)(u_t, v_t) yields Gt+1G_{t+1}.

Formally, we define three action components governed by sub-policies:

P1(ut∣Gt)=Π1(H(Gt),Hsite(Gt)),(11)\mathbb{P}_1(u_t \mid G_t) = \Pi_1\left(\mathcal{H}(G_t), \mathcal{H}_{\mathrm{site}}(G_t)\right), \tag*{(11)}
P2(ft∣ut,Gt)=Π2(ϕ(ut),Φ(S)),(12)\mathbb{P}_2(f_t \mid u_t, G_t) = \Pi_2\left(\phi(u_t), \Phi(\mathcal{S})\right), \tag*{(12)}
P3(vt∣ft,ut,Gt)=Π3(ψ(ft),ψsite(ft)),(13)\mathbb{P}_3(v_t \mid f_t, u_t, G_t) = \Pi_3\left(\psi(f_t), \psi_{\mathrm{site}}(f_t)\right), \tag*{(13)}

where H(Gt)∈Rd\mathcal{H}(G_t) \in\mathbb{R}^{d} is a global embedding of the partial graph, Hsite(Gt)={hu:u∈U(Gt)}\mathcal{H}_{\mathrm{site}}(G_t) = \{h_u : u \in\mathcal{U}(G_t)\} gives embeddings of candidate sites, and ϕ(ut):=hut\phi(u_t) := h_{u_t} denotes the embedding of the chosen site. The fragment vocabulary is represented by Φ(S)={ϕb:b∈S}\Phi(\mathcal{S}) = \{\phi_b : b \in\mathcal{S}\}, a collection of precomputed fragment descriptors. Each fragment ff is further encoded as a global embedding ψ(f)∈Rd\psi(f) \in\mathbb{R}^{d} and its attachment sites as ψsite(f)={gv(f):v∈V(f)}\psi_{\mathrm{site}}(f) = \{g_v^{(f)} : v \in\mathcal{V}(f)\}.

Here Π1,Π2,Π3\Pi_1, \Pi_2, \Pi_3 are neural scoring functions that map their inputs to logits; applying a softmax yields categorical distributions P1,P2,P3\mathbb{P}_1, \mathbb{P}_2, \mathbb{P}_3 over the respective action spaces. The overall policy can be factorized as

π(at∣Gt)=P1(ut∣Gt)P2(ft∣ut,Gt)P3(vt∣ft,ut,Gt).(14)\pi(a_t \mid G_t) = \mathbb{P}_1(u_t \mid G_t)\mathbb{P}_2(f_t \mid u_t, G_t)\mathbb{P}_3(v_t \mid f_t, u_t, G_t). \tag*{(14)}

Inspired by [35], we introduce new fragments by applying graph genetic operations (mutation and crossover) on generated molecules [24] and incorporating novel substructures back into S\mathcal{S}. When the vocabulary grows too large, only the top-ranked fragments are retained.

Theoretical Proofs and Analysis

This appendix provides a comprehensive theoretical treatment of the energy-based fragment scoring framework introduced in § 4.1. Our goal is to move beyond heuristic feature importance by grounding our scoring mechanism in the rigorous principles of differential geometry and calculus on manifolds.

Specifically, we address a fundamental question: How does the discrete removal of a chemical substructure relate to the continuous gradient information learned by the neural network? To answer this, we model the ablation process as a trajectory through the feature space and analyze the energy dynamics along this path. We rigorously establish that our discrete ablation score is mathematically equivalent to the path integral of gradients (Integrated Gradients), thereby satisfying the axiom of completeness in attribution theory. Furthermore, we provide error bounds to justify the stability of this metric against local perturbations.

Preliminaries: Geometry of the Feature Space

To formalize our analysis, we treat the molecular representation not merely as a matrix of numbers, but as a vector in a high-dimensional Hilbert space.

Let H=R∣V∣×d\mathcal{H} = \mathbb{R}^{|V| \times d} denote the Hilbert space of node features equipped with the Frobenius inner product. For any two feature matrices A,B∈H\mathbf{A}, \mathbf{B} \in\mathcal{H}, the inner product is defined as:

⟨A,B⟩F:=Tr⁡(ATB)=∑i,jAijBij.(15)\langle\mathbf{A}, \mathbf{B} \rangle_{F} := \operatorname{Tr}(\mathbf{A}^{\mathsf{T}}\mathbf{B}) = \sum_{i,j} A_{ij}B_{ij}. \tag*{(15)}

The induced norm ∥A∥F=⟨A,A⟩F\lVert\mathbf{A} \rVert_{F} = \sqrt{\langle\mathbf{A}, \mathbf{A} \rangle_{F}} provides a measure of the signal magnitude. The energy function Eθ(H,y)E_{\theta}(\mathbf{H}, y) maps a point in this feature space (conditioned on property yy) to a scalar potential. The gradient ∇HEθ(H,y)\nabla_{\mathbf{H}} E_{\theta}(\mathbf{H}, y) serves as a vector field over H\mathcal{H}, indicating the direction of steepest energy ascent.

Definition 1 (Fragment Subspace Projection). Given a fragment F⊆V\mathcal{F} \subseteq\mathcal{V}, we define the binary indicator mask 1F∈{0,1}∣V∣×d\mathbf{1}_{\mathcal{F}} \in\{0,1\}^{|V| \times d} such that entries corresponding to nodes in F\mathcal{F} are 1, and 0 otherwise. Geometrically, the operation 1F⊙H\mathbf{1}_{\mathcal{F}} \odot\mathbf{H} represents the orthogonal projection of the molecule’s features onto the subspace spanned by the fragment. Conversely, the ablation is a projection onto the orthogonal complement, realized by (1−1F)⊙H(\mathbf{1} - \mathbf{1}_{\mathcal{F}}) \odot\mathbf{H}.

Proof of Theorem 1: Manifold Gradient Equivalence

One might intuitively expect that the importance of a fragment is proportional to the gradients of its atoms. However, in deep non-linear networks, gradients can saturate or vanish. The "Energy Gap" method we propose (computing Eoriginal−EablatedE_{\mathrm{original}} - E_{\mathrm{ablated}}) captures the global effect of the fragment. Theorem 1 bridges the gap between this global discrete difference and the local continuous gradients.

Theorem 1. Let γ:[0,1]→R∣V∣×d\gamma: [0,1] \to\mathbb{R}^{|V| \times d} be a linear path parameterizing the feature ablation from H\mathbf{H} to PF⊥(H)\mathcal{P}_{\mathcal{F}^{\perp}}(\mathbf{H}). Assuming EθE_{\theta} is continuously differentiable, the discrete energy variation I(F)\mathcal{I}(\mathcal{F}) is exactly the path integral of the directional derivatives along γ\gamma:

I(F)=−∫01⟨∇HEθ(γ(t),y),1F⊙H⟩F dt.(16)\mathcal{I}(\mathcal{F}) = -\int_{0}^{1} \langle\nabla_{\mathbf{H}} E_{\theta}(\gamma(t), y), \mathbf{1}_{\mathcal{F}} \odot\mathbf{H} \rangle_{F}\,\mathrm{d}t. \tag*{(16)}

Proof Derivation

The proof relies on constructing a virtual trajectory connecting the intact molecule to the ablated version and applying the Fundamental Theorem of Calculus along this path.

Lemma 1 (Path Dynamics). We define the ablation path γ(t)\gamma(t) connecting the original molecule state H\mathbf{H} (at t=0t = 0) to the ablated state (at t=1t = 1) as:

γ(t)=H−t(1F⊙H),t∈[0,1].(17)\gamma(t) = \mathbf{H} - t(\mathbf{1}_{\mathcal{F}} \odot\mathbf{H}), \qquad t \in[0,1]. \tag*{(17)}

The velocity vector (tangent) of this path is constant and points in the negative direction of the fragment features:

ddtγ(t)=−(1F⊙H).(18)\frac{\mathrm{d}}{\mathrm{d}t}\gamma(t) = -(\mathbf{1}_{\mathcal{F}} \odot\mathbf{H}). \tag*{(18)}

Proof of Lemma 1. By definition, the ablated state is H−1F⊙H\mathbf{H} - \mathbf{1}_{\mathcal{F}} \odot\mathbf{H}. The linear interpolation is:

γ(t)=(1−t)H+t(H−1F⊙H)=H−t(1F⊙H).(19)\begin{aligned} \gamma(t) &= (1-t)\mathbf{H} + t(\mathbf{H} - \mathbf{1}_{\mathcal{F}} \odot\mathbf{H}) \\ &= \mathbf{H} - t(\mathbf{1}_{\mathcal{F}} \odot\mathbf{H}). \tag*{(19)} \end{aligned}
γ(t)=H−t(1F⊙H).(20)\gamma(t) = \mathbf{H} - t(\mathbf{1}_{\mathcal{F}} \odot\mathbf{H}). \tag*{(20)}

Differentiating strictly with respect to the scalar parameter tt yields the tangent vector γ′(t)=−(1F⊙H)\gamma'(t) = -(\mathbf{1}_{\mathcal{F}} \odot\mathbf{H}). This confirms the path is a straight line in H\mathcal{H}. □\square

Proof of Theorem 1. Let g(t)=Eθ(γ(t),y)g(t)=E_\theta(\gamma(t),y) represent the scalar energy evaluated along the ablation trajectory. Since EθE_\theta is a composition of smooth functions (neural network layers), it is differentiable. We apply the Chain Rule to analyze the rate of energy change. The derivative of the composite function g(t)g(t) is the inner product of the gradient of the outer function (the energy field) and the derivative of the inner function (the path velocity):

dgdt=⟨∇γ(t)Eθ(γ(t),y),dγdt⟩F.(21)\frac{dg}{dt}=\left\langle\nabla_{\gamma(t)}E_\theta(\gamma(t),y),\frac{d\gamma}{dt}\right\rangle_F . \tag*{(21)}

Substituting the result from Lemma 1, the rate of change becomes:

dgdt=⟨∇HEθ(γ(t),y),−(1F⊙H)⟩F.(22)\frac{dg}{dt}=\left\langle\nabla_{\mathbf{H}}E_\theta(\gamma(t),y),-\left(\mathbf{1}_{\mathcal{F}}\mathbin{\odot}\mathbf{H}\right)\right\rangle_F . \tag*{(22)}

By the linearity of the inner product, we pull the negative sign out:

dgdt=−⟨∇HEθ(γ(t),y),1F⊙H⟩F.(23)\frac{dg}{dt}=-\left\langle\nabla_{\mathbf{H}}E_\theta(\gamma(t),y),\mathbf{1}_{\mathcal{F}}\mathbin{\odot}\mathbf{H}\right\rangle_F . \tag*{(23)}

The total change in energy, which defines our importance score I(F)\mathcal{I}(\mathcal{F}), is obtained by accumulating these infinitesimal changes from the initial state to the final ablated state:

I(F)=Eθ(γ(1),y)−Eθ(γ(0),y)(24)\mathcal{I}(\mathcal{F})=E_\theta(\gamma(1),y)-E_\theta(\gamma(0),y) \tag*{(24)}
=∫01dgdt dt(25)=\int_0^1\frac{dg}{dt}\,dt \tag*{(25)}
=−∫01⟨∇HEθ(γ(t),y),1F⊙H⟩Fdt.(26)=-\int_0^1\left\langle\nabla_{\mathbf{H}}E_\theta(\gamma(t),y),\mathbf{1}_{\mathcal{F}}\mathbin{\odot}\mathbf{H}\right\rangle_Fdt. \tag*{(26)}

Remark 1 (Theoretical Implication). The integral form derived above corresponds to Integrated Gradients (IG) applied to the fragment subspace. Theorem 1 proves that our "Energy Gap" score is not a heuristic; it is an exact implementation of IG. This ensures that our method satisfies the Completeness Axiom: the sum of attributions equals the total function output difference. Consequently, our method correctly accounts for non-linear saturation effects that would otherwise mislead a simple gradient-based analysis.

Error Analysis and Bounds

While Theorem 1 provides an exact equality, computing the integral requires evaluating the neural network multiple times along the path, which can be computationally expensive. In practice, one might approximate this using only the gradient at the initial point (the original molecule). Here, we analyze the theoretical error introduced by such a first-order approximation. This analysis tells us how "trustworthy" the local gradients are for estimating global fragment importance.

Assumption 1 (Lipschitz Smoothness). We assume the energy landscape is smooth. Specifically, the gradient ∇HEθ\nabla_{\mathbf{H}}E_\theta is LL-Lipschitz continuous over the domain H\mathcal{H}. This means there exists a constant L>0L>0 representing the maximum curvature of the energy surface, such that for any A,B∈H\mathbf{A},\mathbf{B}\in\mathcal{H}:

∥∇HEθ(A,y)−∇HEθ(B,y)∥F≤L∥A−B∥F.(27)\left\|\nabla_{\mathbf{H}}E_\theta(\mathbf{A},y)-\nabla_{\mathbf{H}}E_\theta(\mathbf{B},y)\right\|_F\le L\left\|\mathbf{A}-\mathbf{B}\right\|_F . \tag*{(27)}

Theorem 2 (Lagrangian Error Bound). The deviation of the exact discrete importance score from its first-order Taylor approximation is quadratically bounded by the energy norm of the fragment features. Specifically:

∣I(F)−(−⟨∇HEθ(H,y),1F⊙H⟩F)∣≤L2∥1F⊙H∥F2.(28)\left|\mathcal{I}(\mathcal{F})-\left(-\left\langle\nabla_{\mathbf{H}}E_\theta(\mathbf{H},y),\mathbf{1}_{\mathcal{F}}\mathbin{\odot}\mathbf{H}\right\rangle_F\right)\right|\le\frac{L}{2}\left\|\mathbf{1}_{\mathcal{F}}\mathbin{\odot}\mathbf{H}\right\|_F^2. \tag*{(28)}

Proof. Let ΔH=−(1F⊙H)\Delta\mathbf{H}=-\left(\mathbf{1}_{\mathcal{F}}\mathbin{\odot}\mathbf{H}\right). By the fundamental theorem of calculus derived in the previous section:

I(F)=∫01⟨∇Eθ(H+tΔH,y),ΔH⟩Fdt.(29)\mathcal{I}(\mathcal{F})=\int_0^1\left\langle\nabla E_\theta(\mathbf{H}+t\Delta\mathbf{H},y),\Delta\mathbf{H}\right\rangle_Fdt. \tag*{(29)}

We aim to compare this to the linear approximation ⟨∇Eθ(H,y),ΔH⟩F\langle\nabla E_\theta(\mathbf{H}, y), \Delta\mathbf{H}\rangle_F. To do so, we add and subtract the gradient at the origin point inside the integral:

I(F)=∫01⟨∇Eθ(H,y),ΔH⟩F dt+∫01⟨∇Eθ(H+tΔH,y)−∇Eθ(H,y),ΔH⟩F dt.(30)\begin{aligned} \mathcal{I}(\mathcal{F}) &= \int_0^1 \langle\nabla E_\theta(\mathbf{H}, y), \Delta\mathbf{H}\rangle_F\,dt \\ &\quad+ \int_0^1 \langle\nabla E_\theta(\mathbf{H}+t\Delta\mathbf{H}, y)-\nabla E_\theta(\mathbf{H}, y), \Delta\mathbf{H}\rangle_F\,dt. \tag*{(30)} \end{aligned}

The first term is constant with respect to tt and simplifies to the linear approximation:

∫01⟨∇Eθ(H,y),−(1F⊙H)⟩F dt=−⟨∇Eθ(H,y),1F⊙H⟩F.(31)\int_0^1 \langle\nabla E_\theta(\mathbf{H}, y), -(1_{\mathcal{F}}\odot\mathbf{H})\rangle_F\,dt = -\langle\nabla E_\theta(\mathbf{H}, y), 1_{\mathcal{F}}\odot\mathbf{H}\rangle_F. \tag*{(31)}

Let R\mathcal{R} denote the second term (the remainder/error). We bound ∣R∣|\mathcal{R}| using the Cauchy-Schwarz inequality and Assumption 1:

∣R∣=∣∫01⟨∇Eθ(H+tΔH,y)−∇Eθ(H,y),ΔH⟩F dt∣(32)|\mathcal{R}| = \left|\int_0^1 \langle\nabla E_\theta(\mathbf{H}+t\Delta\mathbf{H}, y)-\nabla E_\theta(\mathbf{H}, y), \Delta\mathbf{H}\rangle_F\,dt\right| \tag*{(32)}
≤∫01∥∇Eθ(H+tΔH,y)−∇Eθ(H,y)∥F∥ΔH∥F dt(33)\le\int_0^1 \|\nabla E_\theta(\mathbf{H}+t\Delta\mathbf{H}, y)-\nabla E_\theta(\mathbf{H}, y)\|_F\|\Delta\mathbf{H}\|_F\,dt \tag*{(33)}
≤∫01L∥tΔH∥F∥ΔH∥F dt(34)\le\int_0^1 L\|t\Delta\mathbf{H}\|_F\|\Delta\mathbf{H}\|_F\,dt \tag*{(34)}
=L∥ΔH∥F2∫01t dt.(35)= L\|\Delta\mathbf{H}\|_F^2\int_0^1 t\,dt. \tag*{(35)}

Computing the standard scalar integral ∫01t dt=12\int_0^1 t\,dt=\frac{1}{2}, and substituting ΔH\Delta\mathbf{H}, we obtain the final bound:

∣R∣≤L2∥1F⊙H∥F2.(36)|\mathcal{R}|\le\frac{L}{2}\|1_{\mathcal{F}}\odot\mathbf{H}\|_F^2. \tag*{(36)}

Interpretation: The error bound L2∥1F⊙H∥F2\frac{L}{2}\|1_{\mathcal{F}}\odot\mathbf{H}\|_F^2 implies that the linear approximation is most accurate for (1) "flat" energy landscapes where LL is small, or (2) fragments with small feature norms. For large, complex fragments, the non-linearity (captured by R\mathcal{R}) becomes significant, further justifying our use of the exact Energy Gap method rather than simple gradient maps.

Consistency of Gradient-Based Ranking

Finally, we utilize the previous results to justify the consistency between the rigorous ablation score and simpler gradient-based metrics often used in saliency maps. In many practical scenarios, we are interested in the ranking of fragments rather than their exact energy contribution.

Corollary 1 (Ranking Alignment under Homogeneity). Suppose the fragment features satisfy a local homogeneity condition where the gradient is aligned with the feature vector. Then, ranking fragments by the magnitude of the energy gap ∣I(F)∣|\mathcal{I}(\mathcal{F})| is asymptotically equivalent to ranking by the summed gradient norms.

Proof. From Theorem 2, if the remainder term is small relative to the linear term, we have:

∣I(F)∣≈∣⟨∇HEθ(H,y),1F⊙H⟩F∣.(37)|\mathcal{I}(\mathcal{F})| \approx|\langle\nabla_{\mathbf{H}}E_\theta(\mathbf{H}, y), 1_{\mathcal{F}}\odot\mathbf{H}\rangle_F|. \tag*{(37)}

This term can be decomposed into individual node contributions:

∣∑i∈F∇xiEθ(H,y)Txi∣.(38)\left|\sum_{i\in\mathcal{F}}\nabla_{\mathbf{x}_i}E_\theta(\mathbf{H}, y)^\mathsf{T}\mathbf{x}_i\right|. \tag*{(38)}

Assuming alignment (∇Tx∝∥∇∥\nabla^\mathsf{T}\mathbf{x}\propto\|\nabla\|), this is proportional to ∑i∈F∥∇xiEθ∥2\sum_{i\in\mathcal{F}}\|\nabla_{\mathbf{x}_i}E_\theta\|_2. Thus, the computationally expensive ablation score I(F)\mathcal{I}(\mathcal{F}) and the efficient gradient norm score induce the same ordering of fragments. This result provides theoretical grounding for using gradient-based screening as a preliminary filter before precise energy-based scoring.

Limitations and future works

While BindMol demonstrates strong performance in exploring diverse binding modes, our current reward formulation relies on simplified interaction counts and static docking poses. First, treating all interaction types equally—without weighting them by energetic contribution—prioritizes interaction diversity over fine-grained affinity modeling. Assigning biochemically accurate weights is non-trivial and target-dependent, yet future iterations could leverage learnable or physics-based weighting schemes to better capture residue-specific importance. Second, like most structure-based generative models, our reliance on rapid static docking (e.g., QVina) may occasionally reward suboptimal or physically strained poses. The PLIP-IFP analyses therefore quantify docking-predicted interaction modes rather than experimentally confirmed binding poses, and IFP cluster counts may retain some sample-size dependence despite the complementary entropy statistics. Crystallographic-mode recovery is limited to BRAF and JAK2 because only these targets provide directly comparable paired mode labels. Future work should incorporate receptor ensembles, rigorous energy minimization, and experimental structural validation.

Additional Empirical Results

Interaction-level diversity and crystallographic-mode recovery

For every qualified docked complex, we construct a binary full-pocket PLIP fingerprint whose features are residue–interaction-type pairs. Pairwise Jaccard distances are clustered with the Butina algorithm at distance cutoff 0.5. We report the number of recurring clusters; entropy H=−∑cpclog⁡pcH = -\sum_c p_c \log p_c, where pcp_c is the fraction of hits in cluster cc; and the effective number of modes exp⁡(H)\exp(H). Cluster count measures breadth, whereas entropy and effective modes also account for population balance. Because cluster count can depend on the number of qualified hits, we interpret all three statistics jointly.

TargetMethodIFP clustersIFP entropy HHEffective modes exp⁡(H)\exp(H)
PARP1REINVENT52.33 ± 4.041.62 ± 0.095.05 ± 0.45
PARP1FREED102.67 ± 6.112.47 ± 0.0811.83 ± 0.95
PARP1RationaleRL96.33 ± 5.512.42 ± 0.1011.25 ± 1.12
PARP1GEAM138.67 ± 7.022.93 ± 0.0718.73 ± 1.26
PARP1BindMol192.00 ± 6.203.45 ± 0.0731.40 ± 0.96
FA7REINVENT21.33 ± 3.210.77 ± 0.092.16 ± 0.20
FA7FREED44.67 ± 4.511.43 ± 0.084.18 ± 0.34
FA7RationaleRL41.33 ± 4.041.38 ± 0.103.97 ± 0.40
FA7GEAM61.67 ± 5.511.85 ± 0.076.36 ± 0.45
FA7BindMol94.33 ± 1.532.37 ± 0.0610.71 ± 0.34
5HT1BREINVENT3.67 ± 0.580.07 ± 0.031.07 ± 0.03
5HT1BFREED6.33 ± 0.580.17 ± 0.041.19 ± 0.05
5HT1BRationaleRL6.00 ± 1.000.15 ± 0.051.16 ± 0.06
5HT1BGEAM8.33 ± 0.580.24 ± 0.041.27 ± 0.05
5HT1BBindMol11.67 ± 0.580.32 ± 0.021.37 ± 0.03
BRAFREINVENT7.33 ± 1.530.18 ± 0.061.20 ± 0.07
BRAFFREED13.67 ± 2.080.41 ± 0.081.51 ± 0.12
BRAFRationaleRL12.67 ± 1.530.38 ± 0.091.46 ± 0.13
BRAFGEAM20.33 ± 2.520.68 ± 0.071.97 ± 0.14
BRAFBindMol35.00 ± 1.411.11 ± 0.083.03 ± 0.08
JAK2REINVENT14.33 ± 2.080.39 ± 0.081.48 ± 0.12
JAK2FREED31.67 ± 3.510.97 ± 0.092.64 ± 0.24
JAK2RationaleRL28.67 ± 3.060.92 ± 0.102.51 ± 0.26
JAK2GEAM47.33 ± 4.041.52 ± 0.084.57 ± 0.37
JAK2BindMol77.33 ± 2.162.31 ± 0.0510.10 ± 0.35

Table 6. Full-pocket interaction-fingerprint diversity. Results are means and standard deviations over three runs. The best result for each target is highlighted in bold.

For crystallographic-mode recovery, we use BRAF and JAK2 because both provide well-established pairs of type-I/DFG-in and type-II/DFG-out structures within the same target. Specifically, the references are BRAF 4MNF (DFG-in/type-I) and 5HI2 (DFG-out/type-II), and JAK2 6VGL (type-I) and 7TEU (type-II/DFG-out). The remaining targets do not provide equally clear and directly comparable paired labels. For each reference conformation, we re-dock generated hits into that receptor and compare their PLIP IFPs with the native ligand IFP; similarity ≥0.4\ge0.4 defines recovery.

TargetCrystal modePDBBindMolGEAMFREED
BRAFDFG-in/type-I4MNF48.9%20.3%11.7%
BRAFDFG-out/type-II5HI212.1%6.4%0.0%
JAK2type-I6VGL1.3%0.0%0.0%
JAK2type-II/DFG-out7TEU11.5%5.7%2.8%
BRAFJAK2
IFP Jaccard0.3750.188
Shared IFP features6/163/16
Residue Jaccard0.4620.250
Shared residues6/133/12

Table 7. Crystallographic-mode recovery and reference separation. The left panel reports recovery rates; the right panel compares the two native references for each target using the same full-pocket representation.

Both native-pair IFP similarities are below the recovery threshold, so recovery does not merely match a conserved interaction core. Absolute rates remain conservative because training uses a fixed receptor while evaluation uses independent crystal structures with induced-fit, side-chain, DFG-loop, and α\alphaC-helix differences. These are docking-predicted interaction modes, not experimentally confirmed poses.

Matched MORL comparisons and hotspot ablations

All MORL baselines use the same fragment action space, vector reward, training protocol, docking-oracle budget, and three random seeds as BindMol.

The shared docking component protects affinity-driving core contacts in both variants. Removing the intersection only from the hotspot-specific term reduces correlation among objectives and encourages differentiating subpocket interactions, explaining the consistent advantage in Table 9.

Sensitivity to energy-model dataset size

We train the energy model on stratified 10%, 25%, 50%, 75%, and 100% subsets of the empirical BRAF and JAK2 distributions, preserving the property-label distribution in each subset. We then evaluate the downstream fragment vocabulary under the same generation protocol.

MetricTarget10%25%50%75%100%
Novel hit ratio (%)BRAF15.24 ± 0.7620.83 ± 0.8426.17 ± 0.9129.24 ± 0.9530.77 ± 0.97
JAK230.76 ± 1.0334.68 ± 1.0140.02 ± 0.9943.46 ± 0.9844.86 ± 0.98
Novel top-5% docking scoreBRAF−10.35 ± 0.04−11.14 ± 0.06−11.89 ± 0.07−12.28 ± 0.08−12.50 ± 0.09
JAK2−10.08 ± 0.04−10.76 ± 0.03−11.43 ± 0.03−11.81 ± 0.02−12.01 ± 0.02

Table 11. Dataset-size sensitivity of energy-based fragment scoring. Means and standard deviations over three runs.

Performance improves and begins to saturate with more target-relevant data. This supports the intended interpretation that such data are beneficial rather than an all-or-nothing prerequisite: for underexplored targets, a virtually screened public library can supply the scored empirical distribution before directed generation.

Sensitivity of preference sampling size

As mentioned in § 4.3, the maximization over ωi\omega_i requires approximation. It is sampled by first drawing i.i.d. standard normal vectors from N(0,I)\mathcal{N}(0, I), applying an element-wise absolute value to ensure non-negativity, and then performing ℓ1\ell_1 normalization. This ensures that the resulting vectors are uniformly distributed on the probability simplex. These weights are then replicated across the batch dimension to enable efficient preference-conditioned training. However, since replicating preference vectors for a batch of PyG graphs increases memory usage, very dense sampling would significantly increase the GPU footprint. In practice, we therefore sample ωi\omega_i five times, which provides a good balance between computational efficiency and performance. To further demonstrate the effect of sampling size kk, we have conducted a sensitivity analysis and the results are shown in Table 13.

Novel hit ratio (%)parp1fa75ht1bbrafjak2
k=1k = 135.02518.27736.27219.33339.047
k=3k = 338.90321.66739.62225.88742.887
k=5k = 5 (BindMol)42.61724.58042.14930.76744.964
k=10k = 1043.62224.89142.27731.04745.219
k=20k = 2045.66725.89144.23933.12746.975

Table 13. Sensitivity analysis of preference sampling size kk.

It can be observed that with sampling size increase, the performance of novel hit ratio on five targets steadily increase. However, due to the memory burden, we set sampling size to k=5k = 5 as the final configuration to achieve the trade-off between performance and efficiency.

Molecular weight distribution analysis

To investigate whether our per-residue interaction rewards unintentionally incentivize the generation of larger molecules, we conducted a comparative analysis of the molecular weight (MW) distributions produced by BindMol and several representative baselines. For each method, we sample an equal number of generated molecules for the target jak2 and compute their MW values using RDkit. As an external reference, we also included the MW of experimentally validated ligand extracted from the PDB (PDB id: 7F7W). The results are shown in Figure 8.

Molecular weight distribution comparison of generated molecules using BindMol and representative baseline methods

Figure 8. The comparison of molecular weight of generated molecules on jak2 target using different competitive generative approaches.

As shown in Figure 8, the MW distribution of BindMol closely aligns with those of the baseline methods and is broadly consistent with the distribution observed in PDB ligands. These results indicate that our per-hotspot interaction rewards do not systematically drive the model toward generating heavier molecules. The similarity between generated molecules and reference ligands further suggests that we operate within a chemically reasonable MW range that is typical for bioactive small molecules.

3D visualization of generated binders of five targets

Figure 9. 3D visualization of generated binders of five targets

Rediscovery molecules and generated top-5 molecules for the GuacaMol MPO tasks

Figure 10. The rediscovery molecules of the GuacaMol MPO tasks and the examples of the generated top-5 molecules. The scores are provided at the bottom of each generated molecule.

References

  1. [1]Alegre, L. N., Bazzan, A. L., Roijers, D. M., Nowé, A., and da Silva, B. C. Sample-efficient multi-objective learning via generalized policy improvement prioritization. arXiv preprint arXiv:2301.07784, 2023.
  2. [2]Berman, H. M., Westbrook, J., Feng, Z., Gilliland, G., Bhat, T. N., Weissig, H., Shindyalov, I. N., and Bourne, P. E. The protein data bank. Nucleic acids research, 28(1):235–242, 2000.
  3. [3]Bickerton, G. R., Paolini, G. V., Besnard, J., Muresan, S., and Hopkins, A. L. Quantifying the chemical beauty of drugs. Nature chemistry, 4(2):90–98, 2012.DOI
  4. [4]Bradshaw, J., Paige, B., Kusner, M. J., Segler, M., and Hernández-Lobato, J. M. Barking up the right tree: an approach to search over molecule synthesis dags. Advances in neural information processing systems, 33:6852–6866, 2020.
  5. [5]Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., and Zaremba, W. Openai gym. arXiv preprint arXiv:1606.01540, 2016.arxiv.org/abs/1606.01540
  6. [6]Brown, N., Fiscato, M., Segler, M. H., and Vaucher, A. C. Guacamol: benchmarking models for de novo molecular design. Journal of chemical information and modeling, 59(3):1096–1108, 2019.DOI
  7. [7]Bussi, G. and Laio, A. Using metadynamics to explore complex free-energy landscapes. Nature Reviews Physics, 2(4):200–212, 2020.DOI
  8. [8]Du, Y., Jamash, A. R., Guo, J., Fu, T., Harris, C., Wang, Y., Duan, C., Liò, P., Schwaller, P., and Blundell, T. L. Machine learning-aided generative molecular design. Nature Machine Intelligence, 6(6):589–604, 2024.DOI
  9. [9]Ertl, P. and Schuffenhauer, A. Estimation of synthetic accessibility score of drug-like molecules based on molecular complexity and fragment contributions. Journal of cheminformatics, 1:1–11, 2009.DOI
  10. [10]Felten, F., Alegre, L. N., Nowe, A., Bazzan, A., Talbi, E. G., Danoy, G., and C da Silva, B. A toolkit for reliable benchmarking and research in multi-objective reinforcement learning. Advances in Neural Information Processing Systems, 36:23671–23700, 2023.DOI
  11. [11]Ferreira, L. G., Dos Santos, R. N., Oliva, G., and Andricopulo, A. D. Molecular docking and structure-based drug design strategies. Molecules, 20(7):13384–13421, 2015.DOI
  12. [12]Fey, M. and Lenssen, J. E. Fast graph representation learning with pytorch geometric. arXiv preprint arXiv:1903.02428, 2019.arxiv.org/abs/1903.02428
  13. [13]Fromer, J. C. and Coley, C. W. Computer-aided multi-objective optimization in small molecule discovery. Patterns, 4(2), 2023.DOI
  14. [14]Fu, T., Gao, W., Xiao, C., Yasonik, J., Coley, C. W., and Sun, J. Differentiable scaffolding tree for molecular optimization. arXiv preprint arXiv:2109.10469, 2021.arxiv.org/abs/2109.10469
  15. [15]Gao, W., Mercado, R., and Coley, C. W. Amortized tree generation for bottom-up synthesis planning and synthesizable molecular design. arXiv preprint arXiv:2110.06389, 2021.arxiv.org/abs/2110.06389
  16. [16]Gao, W., Fu, T., Sun, J., and Coley, C. Sample efficiency matters: a benchmark for practical molecular optimization. Advances in neural information processing systems, 35:21342–21357, 2022.arxiv.org/abs/2206.12411
  17. [17]Ghorbani, M., Gendelev, L., Beroza, P., and Keiser, M. J. Autoregressive fragment-based diffusion for pocket-aware ligand design. arXiv preprint arXiv:2401.05370, 2023.arxiv.org/abs/2401.05370
  18. [18]Gu, S., Xu, M., Powers, A., Nie, W., Geffner, T., Kreis, K., Leskovec, J., Vahdat, A., and Ermon, S. Aligning target-aware molecule diffusion models with exact energy optimization. Advances in Neural Information Processing Systems, 37:44040–44063, 2024.arxiv.org/abs/2407.01648
  19. [19]Guo, J. and Schwaller, P. Augmented memory: sample-efficient generative molecular design with reinforcement learning. Jacs Au, 4(6):2160–2172, 2024.DOI
  20. [20]Guo, J., Fialková, V., Arango, J. D., Margreitter, C., Janet, J. P., Papadopoulos, K., Engkvist, O., and Patronov, A. Improving de novo molecular design with curriculum learning. Nature Machine Intelligence, 4(6):555–563, 2022.DOI
  21. [21]Guttmann, M. and Hyvärinen, A. Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pp. 297–304. JMLR Workshop and Conference Proceedings, 2010.
  22. [22]Haarnoja, T., Zhou, A., Abbeel, P., and Levine, S. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International conference on machine learning, pp. 1861–1870. Pmrl, 2018.arxiv.org/abs/1801.01290
  23. [23]Irwin, J. J., Sterling, T., Mysinger, M. M., Bolstad, E. S., and Coleman, R. G. Zinc: a free tool to discover chemistry for biology. Journal of chemical information and modeling, 52(7):1757–1768, 2012.DOI
  24. [24]Jensen, J. H. A graph-based genetic algorithm and generative model/monte carlo tree search for the exploration of chemical space. Chemical science, 10(12):3567–3572, 2019.DOI
  25. [25]Jeon, W. and Kim, D. Autonomous molecule generation using reinforcement learning and docking to develop potential novel inhibitors. Scientific Reports, 10, 12 2020.DOI
  26. [26]Jin, W., Barzilay, R., and Jaakkola, T. Hierarchical generation of molecular graphs using structural motifs. In International Conference on Machine Learning, pp. 4839–4848. PMLR, 2020.arxiv.org/abs/2002.03230
  27. [27]Jin, W., Barzilay, R., and Jaakkola, T. Multi-objective molecule generation using interpretable substructures. In International Conference on Machine Learning, pp. 4849–4859. PMLR, 2020.arxiv.org/abs/2002.03244
  28. [28]Jo, J., Lee, S., and Hwang, S. J. Score-based generative modeling of graphs via the system of stochastic differential equations. In Proceedings of the 39th International Conference on Machine Learning, 2022.arxiv.org/abs/2202.02514
  29. [29]Kingma, D. P. and Ba, J. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.arxiv.org/abs/1412.6980
  30. [30]Kong, X., Huang, W., Tan, Z., and Liu, Y. Molecule generation by principal subgraph mining and assembling. Advances in Neural Information Processing Systems, 35:2550–2563, 2022.arxiv.org/abs/2106.15098
  31. [31]Krishnan, A., Valeri, J. A., Jin, W., Donghia, N. M., Sieben, L., Luttens, A., Zhang, Y., Modaressi, S. M., Hennes, A., Fromer, J., et al. A generative deep learning approach to de novo antibiotic design. Cell, 2025.DOI
  32. [32]Landrum, G. et al. Rdkit: Open-source cheminformatics software. 2016. URL http://www.rdkit.org/, https://github.com/rdkit/rdkit, 149(150):650, 2016.
  33. [33]LeCun, Y., Chopra, S., Hadsell, R., Ranzato, M., Huang, F., et al. A tutorial on energy-based learning. Predicting structured data, 1(0), 2006.
  34. [34]Lee, S., Jo, J., and Hwang, S. J. Exploring chemical space with score-based out-of-distribution generation. Proceedings of the 40th International Conference on Machine Learning, 2023.arxiv.org/abs/2206.07632
  35. [35]Lee, S., Kreis, K., Veccham, S. P., Liu, M., Reidenbach, D., Paliwal, S., Vahdat, A., and Nie, W. Molecule generation with fragment retrieval augmentation. arXiv preprint arXiv:2411.12078, 2024.arxiv.org/abs/2411.12078
  36. [36]Lee, S., Kreis, K., Veccham, S. P., Liu, M., Reidenbach, D., Peng, Y., Paliwal, S., Nie, W., and Vahdat, A. Genmol: A drug discovery generalist with discrete diffusion. arXiv preprint arXiv:2501.06158, 2025.arxiv.org/abs/2501.06158
  37. [37]Lu, H., Herman, D., and Yu, Y. Multi-objective reinforcement learning: Convexity, stationarity and pareto optimality. In The Eleventh International Conference on Learning Representations, 2023.
  38. [38]Maziarz, K., Jackson-Flux, H., Cameron, P., Sirockin, F., Schneider, N., Stiefl, N., Segler, M., and Brockschmidt, M. Learning to extend molecular scaffolds with structural motifs. arXiv preprint arXiv:2103.03864, 2021.arxiv.org/abs/2103.03864
  39. [39]Nigam, A., Pollice, R., Krenn, M., dos Passos Gomes, G., and Aspuru-Guzik, A. Beyond generative models: superfast traversal, optimization, novelty, exploration and discovery (stoned) algorithm for molecules using selfies. Chemical science, 12(20):7079–7090, 2021.DOI
  40. [40]O’Boyle, N. M., Banck, M., James, C. A., Morley, C., Vandermeersch, T., and Hutchison, G. R. Open babel: An open chemical toolbox. Journal of cheminformatics, 3(1):33, 2011.DOI
  41. [41]Olivecrona, M., Blaschke, T., Engkvist, O., and Chen, H. Molecular de-novo design through deep reinforcement learning. Journal of cheminformatics, 9(1):1–14, 2017.
  42. [42]Pandey, M., Subbaraj, G., Cherkasov, A., Ester, M., and Bengio, E. Pretraining generative flow networks with inexpensive rewards for molecular graph generation. arXiv preprint arXiv:2503.06337, 2025.arxiv.org/abs/2503.06337
  43. [43]Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.arxiv.org/abs/1912.01703
  44. [44]Reymond, M., Bargiacchi, E., and Nowé, A. Pareto conditioned networks. arXiv preprint arXiv:2204.05036, 2022.arxiv.org/abs/2204.05036
  45. [45]Roijers, D., Steckelmacher, D., and Nowé, A. Multi-objective reinforcement learning for the expected utility of the return. In Adaptive Learning Agents Workshop 2018, 2018.
  46. [46]Salentin, S., Schreiber, S., Haupt, V. J., Adasme, M. F., and Schroeder, M. Flip: fully automated protein–ligand interaction profiler. Nucleic acids research, 43(W1):W443–W447, 2015.
  47. [47]Tripp, A., Simm, G. N., and Hernández-Lobato, J. M. A fresh look at de novo molecular design benchmarks. In NeurIPS 2021 AI for Science Workshop, 2021.
  48. [48]Van Moffaert, K. and Nowé, A. Multi-objective reinforcement learning using sets of pareto dominating policies. The Journal of Machine Learning Research, 15(1):3483–3512, 2014.DOI
  49. [49]Van Moffaert, K., Drugan, M. M., and Nowé, A. Scalarized multi-objective reinforcement learning: Novel design techniques. In 2013 IEEE symposium on adaptive dynamic programming and reinforcement learning (ADPRL), pp. 191–199. IEEE, 2013.
  50. [50]Von Luxburg, U. A tutorial on spectral clustering. Statistics and computing, 17(4):395–416, 2007.arxiv.org/abs/0711.0189
  51. [51]Wang, H., Skreta, M., Ser, C.-T., Gao, W., Kong, L., Strieth-Kalthoff, F., Duan, C., Zhuang, Y., Yu, Y., Zhu, Y., et al. Efficient evolutionary search over chemical space with large language models. arXiv preprint arXiv:2406.16976, 2025.arxiv.org/abs/2406.16976
  52. [52]Wang, M. Y. Deep graph library: Towards efficient and scalable deep learning on graphs. In ICLR workshop on representation learning on graphs and manifolds, 2019.
  53. [53]Wang, Z., Nie, W., Qiao, Z., Xiao, C., Baraniuk, R., and Anandkumar, A. Retrieval-based controllable molecule generation. In International Conference on Learning Representations, 2023.arxiv.org/abs/2208.11126
  54. [54]Watson, L. T. and Haftka, R. T. Modern homotopy methods in optimization. Computer Methods in Applied Mechanics and Engineering, 74(3):289–305, 1989.DOI
  55. [55]Xie, Y., Shi, C., Zhou, H., Yang, Y., Zhang, W., Yu, Y., and Li, L. Mars: Markov molecular sampling for multi-objective drug discovery. arXiv preprint arXiv:2103.10432, 2021.arxiv.org/abs/2103.10432
  56. [56]Xie, Y., Xu, Z., Ma, J., and Mei, Q. How much space has been explored? measuring the chemical space covered by databases and machine-generated molecules. International Conference on Learning Representations, 2023.
  57. [57]Xu, J., Tian, Y., Ma, P., Rus, D., Sueda, S., and Matusik, W. Prediction-guided multi-objective reinforcement learning for continuous robot control. In International conference on machine learning, pp. 10607–10616. PMLR, 2020.
  58. [58]Yang, R., Sun, X., and Narasimhan, K. A generalized algorithm for multi-objective reinforcement learning and policy adaptation. Advances in neural information processing systems, 32, 2019.arxiv.org/abs/1908.08342
  59. [59]Yang, S., Hwang, D., Lee, S., Ryu, S., and Hwang, S. J. Hit and lead discovery with explorative rl and fragment-based molecule generation. Advances in Neural Information Processing Systems, 34:7924–7936, 2021.arxiv.org/abs/2110.01219
  60. [60]Zhang, X., Belousoff, M. J., Zhao, P., Kooistra, A. J., Truong, T. T., Ang, S. Y., Underwood, C. R., Egebjerg, T., Šenel, P., Stewart, G. D., et al. Differential glp-1r binding and activation by peptide and non-peptide agonists. Molecular Cell, 80(3):485–500, 2020.DOI
  61. [61]Zhang, Z. and Liu, Q. Learning subpocket prototypes for generalizable structure-based drug design. In International Conference on Machine Learning, pp. 41382–41398. PMLR, 2023.arxiv.org/abs/2305.13997
  62. [62]Zhou, Z., Kearnes, S., Li, L., Zare, R. N., and Riley, P. Optimization of molecules via deep reinforcement learning. Scientific reports, 9(1):10752, 2019.DOI
  63. [63]Zhu, Y., Wu, J., Hu, C., Yan, J., Hou, T., Wu, J., et al. Sample-efficient multi-objective molecular optimization with gflownets. Advances in Neural Information Processing Systems, 36:79667–79684, 2023.arxiv.org/abs/2302.04040

Paper details

vlnc:W_94kphjww
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents