betaFeedback

WorksComputer Science

Dynamic Neural Optimal Transport for Subset Alignment

Abstract

We study optimal transport between distributions when only a subset of the target population is relevant to the source. The goal is to map the source distribution to a reweighted target distribution whose density ratio with respect to the full target lies in [0,c][0, c]. This constraint enables the transported distribution to select an appropriate target subpopulation while limiting how strongly any region of the target can be emphasized. We propose a dynamic neural optimal transport framework for this setting using two neural networks. The first parameterizes a time-dependent potential whose spatial gradient defines the velocity field and whose terminal value enforces the density-ratio constraint. The second parameterizes an interpolation between the source distribution and the learned terminal distribution. Unlike prior neural dynamic optimal transport methods, which typically assume a fixed terminal distribution, our formulation learns the terminal distribution within the support of the target. We demonstrate the method on positive-unlabeled learning, unpaired domain translation, and trajectory inference, where meaningful alignments exist between source and target distributions, but direct distribution matching is inappropriate because the target contains additional or irrelevant components

Introduction

Distribution alignment is a problem of immense importance across machine learning, statistics, and optimization. Wasserstein distance and ff-divergences are two widely used families of measures of disparity between distributions. Given source and target distributions μ\mu and ν\nu defined on subsets of a convex set Ω⊆Rd\Omega\subseteq\mathbb{R}^{d}, specifically on supp⁡(μ)\operatorname{supp}(\mu) and supp⁡(ν)\operatorname{supp}(\nu), respectively, the objective of optimal transport is to obtain a mapping T:supp⁡(μ)→supp⁡(ν)T:\operatorname{supp}(\mu) \to\operatorname{supp}(\nu) from the source distribution to the target distribution that minimizes the expected transport cost, thereby characterizing the Wasserstein distance. This can be relaxed by mapping the source to a distribution near the target, using an ff-divergence to quantify the disparity between the output of the map and the target distribution. In particular, we consider a dynamic deterministic formulation of TT that uses a neural network to parameterize the velocity field. We also use a particular choice of ff-divergence that constrains the mapped density to lie within a subset of supp⁡(ν)\operatorname{supp}(\nu) while satisfying a prescribed density-ratio bound. While previous work has studied relaxed neural optimal transport with static transport maps [40, 26, 27] and unbalanced dynamic optimal transport with mass growth [89], there is no existing neural dynamic optimal transport (OT) formulation that learns a map from the source distribution to a subset of the target distribution.

Specifically, our formulation of support subset alignment relies on a relaxation of the Benamou-Brenier problem, formulated in terms of a probability path ρt\rho_t and velocity field vt\mathbf{v}_t (Benamou & Brenier, 2000; Santambrogio, 2015; [36]. For probability measures μ,ν∈P(Ω)\mu, \nu\in\mathcal{P}(\Omega), the semi-relaxed dynamic OT problem we study is

inf⁡ρt,vt∫01Ex∼ρt[12∥vt(x)∥2] dt,s.t.∂tρt(x)+div⁡(ρt(x)vt(x))=0, ρ0=μ, ρ1≤cν,(1)\inf_{\rho_t,v_t} \int_0^1 \mathbb{E}_{x\sim\rho_t}\left[\frac{1}{2}\lVert v_t(x)\rVert^2\right]\,dt,\quad\text{s.t.}\quad\partial_t\rho_t(x)+\operatorname{div}(\rho_t(x)v_t(x))=0,\ \rho_0=\mu,\ \rho_1\leq c\nu, \tag*{(1)}

where the initial marginal constraint ρ0=μ\rho_0=\mu is preserved, whereas the terminal marginal ρ1\rho_1 is allowed to vary subject to the density-ratio constraint ρ1/ν∈[0,c]\rho_1/\nu\in[0,c]. This allows some regions of supp⁡(ν)\operatorname{supp}(\nu) to be up-weighted while others are down-weighted or removed. The relaxed terminal constraint is equivalent to a partial optimal transport relaxation of static optimal transport using ff-divergences, as introduced by Séjourné et al. [108]. The case c=1c=1 corresponds to prescribing the endpoint marginal as ρ1=ν\rho_1=\nu. Under suitable regularity, weak solutions of the continuity equation induce deterministic flows via mass conservation (Villani et al. (2009, p. 15)), while locally Lipschitz flows induce unique probability paths (Villani (2021, Theorem 5.34)).

In practice, when only samples are available, we use neural networks to parameterize both the velocity field vtv_t and the probability path ρt\rho_t. Dynamic neural OT is closely connected to flow matching, diffusion-type models, trajectory inference, and generative modeling ([74, 77, 1, 116, 88 b]).

To position our formulation, we introduce necessary notation and discuss unbalanced optimal transport (UOT) and the dynamic version. Let P(X)\mathcal{P}(\mathcal{X}) and M+(X)\mathcal{M}_+(\mathcal{X}) denote probability and finite nonnegative measures on X⊆Ω\mathcal{X}\subseteq\Omega. For a map SS, S#μS_\#\mu is the pushforward; for π∈M+(X×Y)\pi\in\mathcal{M}_+(\mathcal{X}\times\mathcal{Y}), (pr⁡X)#π(\operatorname{pr}_{\mathcal{X}})_\#\pi and (pr⁡Y)#π(\operatorname{pr}_{\mathcal{Y}})_\#\pi are its marginals. We write ν~≪ν\tilde{\nu}\ll\nu if ν(A)=0⇒ν~(A)=0\nu(A)=0\Rightarrow\tilde{\nu}(A)=0. When absolutely continuous, we use the same symbol for a measure and its density; thus, if dν~=ν~(x)dxd\tilde{\nu}=\tilde{\nu}(x)dx and dν=ν(x)dxd\nu=\nu(x)dx, then dν~/dν=ν~/νd\tilde{\nu}/d\nu=\tilde{\nu}/\nu on {ν>0}\{\nu>0\}. We use ρt\rho_t, vtv_t, and gtg_t for time-varying measures, velocity fields, and growth fields, respectively, and E\mathbb{E} for expectations under probability measures.

Unbalanced optimal transport (UOT) generalizes balanced OT (Santambrogio [104]) by allowing transport between finite nonnegative measures μ∈M+(X)\mu\in\mathcal{M}_+(\mathcal{X}) and ν∈M+(Y)\nu\in\mathcal{M}_+(\mathcal{Y}) for which μ(X)\mu(\mathcal{X}) and ν(Y)\nu(\mathcal{Y}) need not be equal. Let f1,f2:R+→R∪{+∞}f_1,f_2:\mathbb{R}_+\to\mathbb{R}\cup\{+\infty\} be proper, convex, lower semicontinuous functions satisfying f1(1)=f2(1)=0f_1(1)=f_2(1)=0. Given π∈M+(X×Y)\pi\in\mathcal{M}_+(\mathcal{X}\times\mathcal{Y}), define μ~=(pr⁡X)#π\tilde{\mu}=(\operatorname{pr}_{\mathcal{X}})_\#\pi and ν~=(pr⁡Y)#π\tilde{\nu}=(\operatorname{pr}_{\mathcal{Y}})_\#\pi. The UOT problem is then stated as ([38, 108])

WUOT{f1,f2}(μ,ν)=inf⁡π∈M+(X×Y)∫X×Yc(x,y) dπ(x,y)+γ1Df1(μ~∥μ)+γ2Df2(ν~∥ν),(2)\mathcal{W}^{\{f_1,f_2\}}_{\mathrm{UOT}}(\mu,\nu)=\inf_{\pi\in\mathcal{M}_+(\mathcal{X}\times\mathcal{Y})}\int_{\mathcal{X}\times\mathcal{Y}}c(x,y)\,d\pi(x,y)+\gamma_1D_{f_1}(\tilde{\mu}\Vert\mu)+\gamma_2D_{f_2}(\tilde{\nu}\Vert\nu), \tag*{(2)}

where γ1,γ2>0\gamma_1,\gamma_2>0, and Df1D_{f_1} and Df2D_{f_2} denote the ff-divergences corresponding to f1f_1 and f2f_2, respectively. Even when μ\mu and ν\nu are probability measures, the optimal solution of equation 2 is generally not a coupling of μ\mu and ν\nu. Thus, UOT relaxes the strict marginal equality constraints of OT by replacing them with ff-divergence penalties, making it applicable to general finite nonnegative measures. UOT has also been used to estimate transport maps TT using neural network parameterizations ([26, 27, 40, 41] (Yang & Uhler 2019)).

Dynamic UOT handles general positive measures by explicitly modeling mass creation and destruction through a scalar growth field gtg_t (Chizat et al. [24]), and has been applied to problems in generative modeling and trajectory inference ([88, 90]). Dynamic UOT between μ,ν∈M+\mu,\nu\in\mathcal{M}_+ is

inf⁡ρt,vt,gt∫01∫Ω[∥vt(x)∥2+gt2(x)2]dρt(x) dt,s.t.∂∂tρt(x)+div⁡(ρt(x)vt(x))=gt(x)ρt(x), ρ0=μ, ρ1=ν.(3)\inf_{\rho_t,v_t,g_t}\int_0^1\int_{\Omega}\left[\frac{\lVert v_t(x)\rVert^2+g_t^2(x)}{2}\right]d\rho_t(x)\,dt,\quad\text{s.t.}\quad\frac{\partial}{\partial t}\rho_t(x)+\operatorname{div}(\rho_t(x)v_t(x))=g_t(x)\rho_t(x),\ \rho_0=\mu,\ \rho_1=\nu. \tag*{(3)}

Thus, UOT penalizes deviations from prescribed marginals, whereas dynamic UOT modifies the continuity equation through a mass growth term. When μ\mu and ν\nu are probability measures and mass creation/destruction is not needed, one may set gt=0g_t=0, recovering balanced dynamic OT.

Positioning and contributions. Unlike UOT, which allows positive measures and relaxes marginal constraints, and dynamic UOT, which models mass creation/destruction but preserves marginals, our dynamic subset alignment preserves the source mass and transports it to a terminal distribution supported on a subset of the target support and within a density-ratio bound of the target. This is achieved by replacing the terminal marginal equality constraint with an ff-divergence-based penalty, which recovers balanced OT as a special case. We derive a dual dynamic formulation whose potential identifies the selected target subset and whose gradient defines the transport velocity. We use neural network-based parameterizations of the velocity field and path distribution to achieve the alignment and subset selection. Furthermore, we demonstrate the framework on unpaired domain translation, PU-learning, and trajectory inference problems.

Related Work

Our work on dynamic subset alignment is most directly related to Lagrangian neural optimal transport (Pooladian et al. [99]), action matching (Neklyudov et al. [88]), and Wasserstein Lagrangian flows (Neklyudov et al. [90]).

The neural optimal transport framework with Lagrangian costs [99] studies optimal transport with general Lagrangian potentials in Euclidean space. Wasserstein Lagrangian flows [90] are primarily developed for applications in cellular trajectory inference and quantum many-body problems and extend to more general settings such as the Wasserstein-Fisher-Rao metric (Chizat et al., 2018a,b; Séjourné et al., 2023), with the ability to model mass growth/destruction and different types of dynamics.

These approaches, as well as standard flow-based models, are developed for settings in which the endpoint marginals are prescribed, which prevents their direct application to the subset-alignment problem. A more extensive review of recent work on dynamic neural optimal transport is included in Appendix § 9.§ 2.

In contrast, our proposed dynamic support subset-selection framework preserves one marginal while dynamically transporting its mass to a subset of the support of the other marginal while minimizing the transport cost. Therefore, our approach can be viewed as a novel extension of prior work [90]. Although we focus on the ℓ2\ell_2 cost, our method is compatible with other Lagrangian costs [99], which could be useful for incorporating side information, as in semi-supervised domain adaptation.

Additionally, we review advances in static neural optimal transport in Appendix § 9.§ 1. In Appendix § 7.§ 2, we also discuss the static formulation of subset alignment. The static formulation of our framework can be related to incomplete optimal transport (IOT) [40] and unbalanced optimal transport mapping (UOTM) [26]. IOT [40] can be interpreted as a proximal form of static subset alignment. UOTM can also be reduced to static subset alignment between two probability measures by choosing, in equation (2), f1f_1 as the ff-divergence corresponding to the indicator function ι{1}(x)={0,x=1,+∞,x≠1,\iota_{\{1\}}(x) = \begin{cases} 0, & x = 1, \\ +\infty, & x \ne1, \end{cases} and f2=ι[0,c]f_2 = \iota_{[0,c]}.

Methodology

In our semi-relaxed dynamic OT formulation equation (1), we can replace the inequality ρ1≤cν\rho_1 \le c\nu constraint by adding the equivalent ff-divergence penalty term Dι[0,c]D_{\iota_{[0,c]}} to the objective, where Dι[0,c]D_{\iota_{[0,c]}} is a special case of the range divergence Dι[a,b]D_{\iota_{[a,b]}} corresponding to the convex indicator function ι[a,b]\iota_{[a,b]}, which, along with its Legendre–Fenchel conjugate ι[a,b]∗\iota^*_{[a,b]}, is given as

ι[a,b](r)={0,r∈[a,b]+∞,o.w.,ι[a,b]∗(t)=sup⁡u∈[a,b](u⋅t)=max⁡(−at,bt).(4)\iota_{[a,b]}(r) = \begin{cases} 0, & r \in[a,b] \\ +\infty, & \text{o.w.} \end{cases} ,\qquad \iota^*_{[a,b]}(t) = \sup_{u \in[a,b]} (u \cdot t) = \max(-at,bt). \tag*{(4)}

Setting a=0a=0 and b=cb=c, one obtains Dι[0,c](ρ1∥ν)D_{\iota_{[0,c]}}(\rho_1\|\nu) with ι[0,c]∗(u)=c⋅max⁡(0,u)\iota^*_{[0,c]}(u)=c\cdot\max(0,u). Since ι[0,c]\iota_{[0,c]} is convex and lower semicontinuous, ι[0,c]=ι[0,c]∗∗\iota_{[0,c]}=\iota^{**}_{[0,c]}. The variational form of ff-divergences [91, 92], [97 Theorem 7.26] Df(ν~∥ν)=sup⁡η:Ω→R∫Ωη(x)dν~(x)−∫Ωf∗(η(x))dν(x)D_f(\widetilde{\nu}\|\nu)=\sup_{\eta:\Omega\to\mathbb{R}}\int_\Omega\eta(x)d\widetilde{\nu}(x)-\int_\Omega f^*(\eta(x))d\nu(x), yields

Dι[0,c](ρ1∥ν)=sup⁡η:Ω→R×Ω∼ρ1E[η(x)]−Ex∼ν[c⋅max⁡(0,η(x))].(5)D_{\iota_{[0,c]}}(\rho_1\|\nu) = \sup_{\eta:\Omega\to\mathbb{R}\times\Omega\sim\rho_1} \mathbb{E}[\eta(x)]-\mathbb{E}_{x\sim\nu}[c\cdot\max(0,\eta(x))]. \tag*{(5)}

To solve the relaxed dynamic optimal transport problem in equation (1), we replace the terminal inequality constraint with the additive penalty Dι[0,c](ρ1∥ν)D_{\iota_{[0,c]}}(\rho_1\|\nu) in the objective. We introduce the Lagrange multiplier φt\varphi_t for the continuity equation and the Lagrange multiplier ψ0\psi_0 for the initial marginal constraint ρ0=μ\rho_0=\mu. After further simplification (see Appendix § 7.§ 3 for details), one obtains the conditions.

vt(x)=∇φt(x),ψ0(x)=φ0(x),η(x)=−φ1(x).(6)v_t(x) = \nabla\varphi_t(x),\quad\psi_0(x) = \varphi_0(x),\quad\eta(x) = -\varphi_1(x). \tag*{(6)}

These conditions can then be used to eliminate vt(x)v_t(x), ψ0(x)\psi_0(x), and η(x)\eta(x), yielding the optimization problem sup⁡ρtinf⁡φtL(ρt,φt)\sup_{\rho_t}\inf_{\varphi_t}\mathcal{L}(\rho_t,\varphi_t) with the Lagrangian

L(ρt,φt)=Ex∼μ[φ0(x)]+Ex∼ν[c⋅max⁡(0,−φ1(x))]⏟boundary part+∫01Ext∼ρt[∂∂tφt(xt)+∥∇φt(xt)∥22]dt⏟intermediate part.(7)\mathcal{L}(\rho_t,\varphi_t) = \underbrace{\mathbb{E}_{x\sim\mu}[\varphi_0(x)] + \mathbb{E}_{x\sim\nu}[c\cdot\max(0,-\varphi_1(x))]}_{\text{boundary part}} + \underbrace{\int_0^1 \mathbb{E}_{x_t\sim\rho_t}\left[\frac{\partial}{\partial t}\varphi_t(x_t) + \frac{\lVert\nabla\varphi_t(x_t)\rVert^2}{2}\right]dt}_{\text{intermediate part}}. \tag*{(7)}

To approximate the solution, we directly parameterize the dual potential φt\varphi_t with a neural network and parameterize the probability path ρt\rho_t using a neural network that acts on samples from the terminal marginals μ\mu and ν\nu. A similar parameterization is used in the computational framework for learning Wasserstein–Lagrangian flows (WLF) [90]. In contrast, flow-based models typically do not learn ρt\rho_t directly, but instead use analytically defined interpolants between two fixed marginals during training [74, 77, 1]. Our problem differs from both flow matching and WLF because we aim to map the source distribution μ\mu to an unknown distribution ρ1∗\rho^*_1 supported on a subset of supp⁡(ν)\operatorname{supp}(\nu). To this end, for t∈[0,1]t\in[0,1], we construct an interpolant xt∼ρtx_t\sim\rho_t from independently sampled points x0∼μx_0\sim\mu and x1∼νx_1\sim\nu as

xt=(1−t)x0+t(x1+Qt(x0,x1)).(8)x_t = (1-t)x_0 + t(x_1 + Q_t(x_0,x_1)). \tag*{(8)}

where QtQ_t is a time-dependent neural network. At t=1t=1, we have x1+Q1(x0,x1)∼ρ1\mathbf{x}_{1}+Q_{1}(\mathbf{x}_{0},\mathbf{x}_{1}) \sim\rho_{1}. Since in general, ρ1≠ν\rho_{1} \ne\nu, the network must learn a new terminal marginal. Let Φt:Ω→Ω\Phi_t:\Omega\to\Omega denote the flow induced by the velocity field vt\mathbf{v}_t, so that (Φt)#μ=ρt(\Phi_t)_{\#}\mu=\rho_t and Φt(x0)=x0+∫0tvτ(xτ) dτ\Phi_t(\mathbf{x}_0)=\mathbf{x}_0+\int_{0}^{t}\mathbf{v}_{\tau}(\mathbf{x}_{\tau})\,\mathrm{d}\tau, x0∈supp⁡(μ)\mathbf{x}_0 \in\operatorname{supp}(\mu). The optimal correction Qt⋆Q_{t}^{\star} corresponding to this flow is then given by Q1⋆(x0,x1)=Φ1(x0)−x1Q_{1}^{\star}(\mathbf{x}_0,\mathbf{x}_1)=\Phi_{1}(\mathbf{x}_0)-\mathbf{x}_1 and Qt⋆(x0,x1)=1t(Φt(x0)−(1−t)x0)−x1Q_{t}^{\star}(\mathbf{x}_0,\mathbf{x}_1)=\frac{1}{t}\left(\Phi_t(\mathbf{x}_0)-(1-t)\mathbf{x}_0\right)-\mathbf{x}_1. By the standard neural-network universal approximation theorem [53], bounds on the ϵ\epsilon-approximation error ∥Qt−Qt⋆∥\lVert Q_t-Q_{t}^{\star}\rVert translate into bounds on ∥Φt−Φt⋆∥\lVert\Phi_t-\Phi_{t}^{\star}\rVert. Consequently, for any admissible velocity field, one obtains W2((Φt)#μ,ρt)≤CϵW_{2}((\Phi_t)_{\#}\mu,\rho_t)\le C\epsilon for some constant C>0C>0.

Range-based penalty 5 also allows us to distinguish between data points sampled from the selected and rejected subsets of the target support supp⁡(ν)\operatorname{supp}(\nu). Let Φt⋆\Phi_{t}^{\star} denote the optimal flow operator corresponding to the optimal dual potential φt⋆\varphi_{t}^{\star}. Then points sampled from the selected subset supp⁡((Φ1⋆)#μ)\operatorname{supp}((\Phi_{1}^{\star})_{\#}\mu) and from the rejected subset supp⁡(ν)∖supp⁡((Φ1⋆)#μ)\operatorname{supp}(\nu)\setminus\operatorname{supp}((\Phi_{1}^{\star})_{\#}\mu) can be separated by the following test

φ1⋆(x)≤0,almost surely on supp⁡((Φ1⋆)#μ),φ1⋆(x)>0,almost surely on supp⁡(ν)∖supp⁡((Φ1⋆)#μ).(9)\begin{aligned} \varphi_{1}^{\star}(\mathbf{x}) &\le0,\quad&&\text{almost surely on }\operatorname{supp}((\Phi_{1}^{\star})_{\#}\mu),\\ \varphi_{1}^{\star}(\mathbf{x}) &> 0,\quad&&\text{almost surely on }\operatorname{supp}(\nu)\setminus\operatorname{supp}((\Phi_{1}^{\star})_{\#}\mu). \tag*{(9)} \end{aligned}

Test 9 follows directly from the optimality conditions for the range penalty and dual φt\varphi_t (see Appendix § 8) and is used in our PU-learning and melanoma experiments.

Algorithm 1: Dynamic-Neural-SS: Learning Algorithm for Dynamic Subset Selection

Require: Samples from distributions μ\mu and ν\nu, time-dependent neural network φt(⋅;θφ)\varphi_t(\cdot;\theta_{\varphi}), interpolant network Qt(⋅,⋅;θρ)Q_t(\cdot,\cdot;\theta_{\rho}), batch size NN, number of updates nφn_{\varphi} and nρn_{\rho}, gradient-based optimizers optim⁡φ\operatorname{optim}_{\varphi} and optim⁡ρ\operatorname{optim}_{\rho}.

Ensure: Sample-based neural estimate for φt(⋅;θφ)\varphi_t(\cdot;\theta_{\varphi}).

  1. for learning iteration k=0,1,…k=0,1,\ldots do

  2. for u=1,…,nφu=1,\ldots,n_{\varphi} do

  3. Sample {x0i}i=1N∼μ\{\mathbf{x}_{0}^{i}\}_{i=1}^{N}\sim\mu, {x1i}i=1N∼ν\{\mathbf{x}_{1}^{i}\}_{i=1}^{N}\sim\nu, and {ti}i=1N∼Uniform⁡([0,1])\{t^{i}\}_{i=1}^{N}\sim\operatorname{Uniform}([0,1]).

  4. {xti←(1−ti)x0i+ti(x1i+Qti(x0i,x1i;θρ))}i=1N\{\mathbf{x}_{t}^{i}\leftarrow(1-t^{i})\mathbf{x}_{0}^{i}+t^{i}(\mathbf{x}_{1}^{i}+Q_{t^{i}}(\mathbf{x}_{0}^{i},\mathbf{x}_{1}^{i};\theta_{\rho}))\}_{i=1}^{N}.

  5. grad⁡θφ←∇θφ1N∑i=1N[∂∂tφti(xti;θφ)+∥∇φti(xti;θφ)∥22+φ0(x0i;θφ)+cmax⁡(0,−φ1(x1i;θφ))]\operatorname{grad}_{\theta_{\varphi}}\leftarrow\nabla_{\theta_{\varphi}}\frac{1}{N}\sum_{i=1}^{N}\left[\frac{\partial}{\partial t}\varphi_{t^{i}}(\mathbf{x}_{t}^{i};\theta_{\varphi})+\frac{\lVert\nabla\varphi_{t^{i}}(\mathbf{x}_{t}^{i};\theta_{\varphi})\rVert^{2}}{2}+\varphi_{0}(\mathbf{x}_{0}^{i};\theta_{\varphi})+c\max(0,-\varphi_{1}(\mathbf{x}_{1}^{i};\theta_{\varphi}))\right].

  6. Using optim⁡φ\operatorname{optim}_{\varphi} and grad⁡θφ\operatorname{grad}_{\theta_{\varphi}} to update θφ\theta_{\varphi} via gradient descent.

  7. end for

  8. for u=1,…,nρu=1,\ldots,n_{\rho} do

  9. Sample {x0i}i=1N∼μ\{\mathbf{x}_{0}^{i}\}_{i=1}^{N}\sim\mu, {x1i}i=1N∼ν\{\mathbf{x}_{1}^{i}\}_{i=1}^{N}\sim\nu, and {ti}i=1N∼Uniform⁡([0,1])\{t^{i}\}_{i=1}^{N}\sim\operatorname{Uniform}([0,1]).

  10. {xti←(1−ti)x0i+ti(x1i+Qti(x0i,x1i;θρ))}i=1N\{\mathbf{x}_{t}^{i}\leftarrow(1-t^{i})\mathbf{x}_{0}^{i}+t^{i}(\mathbf{x}_{1}^{i}+Q_{t^{i}}(\mathbf{x}_{0}^{i},\mathbf{x}_{1}^{i};\theta_{\rho}))\}_{i=1}^{N}.

  11. grad⁡θρ←∇θρ1N∑i=1N[∂∂tφti(xti(θρ);θφ)+∥∇φti(xti(θρ);θφ)∥22]\operatorname{grad}_{\theta_{\rho}}\leftarrow\nabla_{\theta_{\rho}}\frac{1}{N}\sum_{i=1}^{N}\left[\frac{\partial}{\partial t}\varphi_{t^{i}}(\mathbf{x}_{t}^{i}(\theta_{\rho});\theta_{\varphi})+\frac{\lVert\nabla\varphi_{t^{i}}(\mathbf{x}_{t}^{i}(\theta_{\rho});\theta_{\varphi})\rVert^{2}}{2}\right].

  12. Update θρ\theta_{\rho} using optim⁡ρ\operatorname{optim}_{\rho} and grad⁡θρ\operatorname{grad}_{\theta_{\rho}} via gradient ascent.

  13. end for

  14. end for

Experiments and Results

We present experimental results for subset alignment on an easily interpretable image-to-image task, where the MNIST dataset [30] is the source and the EMNIST dataset [29] is the target. In this setting, digit images form a subset of the characters in EMNIST. We also applied dynamic subset alignment to the PU-Learning task. We then apply our proposed approach to domain translation on the FFHQ dataset (Karras et al., [60]) in the 512-dimensional latent space of ALAE (Pidhorskyi et al., [96]). We use Algorithm 1 to solve all the subset alignment problems discussed above. We also extended dynamic subset alignment to a multi-marginal setting and applied it to the melanoma dataset from Bunne et al. [18].

Toy Examples

First, we apply subset alignment to 1D and 2D point distributions. For the 1D toy example, we align a uniform distribution with a 1D semicircular distribution. Results for c∈{1,2,4,8}c \in\{1, 2, 4, 8\} are shown in Figure 2. As the parameter cc increases, the source distribution μ\mu is transported to a subset of the support of the target distribution ν\nu, for which the potential satisfies φ1≤0\varphi_1 \le0. The transport between two 2D uniform distributions for c∈{1,2,4,8}c \in\{1, 2, 4, 8\} is shown in Figure 1, where one can observe that φ1(x)\varphi_1(x) is non-positive in selected subsets. Additional toy examples are provided in Appendix § 10, including subset alignment between two 1D uniform distributions and between two bimodal 2D distributions.

Subset alignment between two uniform distributions

Figure 1. Subset alignment between two uniform distributions in R2\mathbb{R}^{2} at t=1t=1, obtained using Algorithm 1 with c∈{1,2,4,8}c \in\{1,2,4,8\}. The black contour line indicates the zero-level set of φ1(x)\varphi_{1}(\mathbf{x}).

Source, target, and transported source histograms with sampling densities for 1D subset alignment

Figure 2. Source, target, and transported source histograms with sampling densities for 1D subset alignment between a uniform distribution and a semicircular density. φ1\varphi_1 is shown in red.

MNIST to EMNIST Domain Translation

The MNIST dataset contains 60,000 images of digits from 0 to 9 in the training set and 10,000 images in the test set. The MNIST dataset is roughly balanced, in the sense that the proportions of the classes are approximately the same. The EMNIST ByClass (EMNIST) dataset contains English letters and digits. EMNIST consists of 62 imbalanced classes, of which 10 classes correspond to digits between 0–9, while the remaining 52 classes correspond to uppercase and lowercase letters of the English alphabet. Roughly, 16% of EMNIST consists of digits, while the remaining 84% consists of letters.

Since our goal is to transfer MNIST images to EMNIST images in such a way that MNIST digits are mapped to EMNIST digits while ignoring letters, we trained a neural network classifier to distinguish between digits and letters and used it to evaluate the learned mapping; see Appendix § 11.§ 2 for implementation details. After training the classifier, we used both static and dynamic subset-selection approaches for domain translation between MNIST and EMNIST. Implementation details of the underlying models and their training are provided in Appendix § 11.§ 3. We evaluated

classification accuracy on translations of the entire MNIST dataset; the corresponding confusion matrices are provided in Appendix § 12. Accuracies of the translated outputs are in Table 1.

Methodc=1c = 1c=2c = 2c=4c = 4c=8c = 8
static (Gazdieva et al., [40])46.9375.3382.4487.32
dynamic (one step)64.8575.1693.5795.84
dynamic (ODE)58.8070.4792.6895.00

Table 1. Classification accuracies of translated images for MNIST→\toEMNIST.

In our experiments, we trained and evaluated both static and dynamic models using the corresponding subset-selection frameworks for c∈{1,2,4,8}c \in\{1, 2, 4, 8\}. Similar to flow-based generative models, dynamic subset alignment requires numerical integration; in our experiments, we use ODE integration with an Euler-type numerical integrator with 100 integration steps. However, one-step integration can also be used (Liu et al., [77, 78]). Figure 3 shows that, perceptually, one-step integration performs worse than both static and ODE-based generation, but classification accuracies are better.

Image translation outputs for MNIST to EMNIST

Figure 3. Image translation outputs for MNIST → EMNIST.

The classifier-based evaluation should be interpreted together with the qualitative results. As discussed above, the classifier is trained on the target data, where the digit classes are assigned separate labels (0-9), while all alphabet classes are grouped into a single additional label. Therefore, this evaluation primarily measures whether translated MNIST samples are classified as digit-like EMNIST samples rather than alphabet-like samples. For small cc, one-step translation can achieve higher accuracy because it only slightly perturbs the source image and often stays within the same classifier decision region, even if the resulting image is visually less aligned with EMNIST. In contrast, the ODE solution follows the learned transport more faithfully, which can cause more label changes for small cc and lower classifier accuracy. As cc increases, the relevant target subset becomes easier to identify, and the gap between one-step and ODE results becomes small both quantitatively and qualitatively.

Positive-Unlabeled Learning

Positive-unlabeled (PU) learning is a binary classification problem in which only a subset of positive samples is labeled. This labeled subset is then used to train a model that distinguishes positive samples from negative samples within an unlabeled dataset containing both positive and negative examples ([12, 61, 20, 100]). Since the sign of the optimal potential function φt\varphi_t in our framework differs between the selected and unselected subsets, it can be used to distinguish positive samples from unlabeled samples; see Appendix § 8.

We applied both static and dynamic optimal transport methods to PU learning on the 20 UCI datasets (Kelly et al.), following the setting of Teisseyre et al. [115]. We used the same 75/25 train-test split for each dataset and the sampled-completely-at-random (SCAR) mechanism to select and label positive points. Dynamic subset alignment employs Algorithm 1, while static subset alignment employs Algorithm 2, described in Appendix § 7.§ 1.

Neural networks consisting of five MLP layers with appropriate input and output dimensions and Swish activation functions are used to train both the static and dynamic models. The learning rates are held constant during training for both models. Architecture and parameter details for each model are provided in Appendix § 11.§ 4. We trained 20 different models for each dataset using different train-test splits. In total, we trained 400 models for static subset alignment and 400 models for dynamic subset alignment. We adopted alternative sign-based and value-based label-assignment strategies for the unlabeled dataset. The performance of our approaches, measured in terms of balanced accuracy, along with the top-performing baselines PUSB (Kato et al., [61]) and NTC-MI (Teisseyre et al., [115]), is reported in Table 2. In all PU learning experiments, cc is fixed as c=1πc = \frac{1}{\pi}, where π\pi denotes the known prior proportion of positively labeled points in the dataset. Details are discussed in Appendix § 11.§ 4.

DatasetPUSBNTC-MIstatic sortedstatic signdynamic sorteddynamic sign
Abalone0.544 ± 0.0600.575 ± 0.0250.561 ± 0.0290.503 ± 0.0080.578 ± 0.0250.540 ± 0.054
Banknote0.829 ± 0.0500.922 ± 0.0190.883 ± 0.0370.882 ± 0.0390.952 ± 0.0220.939 ± 0.018
Breast-w0.766 ± 0.1450.870 ± 0.0280.930 ± 0.0280.941 ± 0.0270.927 ± 0.0240.931 ± 0.026
Diabetes0.546 ± 0.0420.700 ± 0.0390.635 ± 0.0440.629 ± 0.0440.629 ± 0.0430.630 ± 0.046
Haberman0.513 ± 0.0230.532 ± 0.0660.539 ± 0.0660.540 ± 0.0670.544 ± 0.0740.540 ± 0.075
Heart0.527 ± 0.0330.757 ± 0.0530.637 ± 0.0930.623 ± 0.0890.656 ± 0.0800.653 ± 0.094
Ionosphere0.440 ± 0.0850.755 ± 0.0590.773 ± 0.0910.762 ± 0.0880.827 ± 0.0660.807 ± 0.070
Isolet0.793 ± 0.0720.725 ± 0.0060.881 ± 0.0280.923 ± 0.0300.731 ± 0.1400.707 ± 0.182
Jm10.628 ± 0.0160.628 ± 0.0130.576 ± 0.0150.575 ± 0.0100.593 ± 0.0230.558 ± 0.029
Kc10.645 ± 0.0750.679 ± 0.0300.604 ± 0.0360.607 ± 0.0350.649 ± 0.0300.573 ± 0.032
Madelon0.496 ± 0.0300.519 ± 0.0280.533 ± 0.0250.523 ± 0.0150.535 ± 0.0180.517 ± 0.019
Musk0.712 ± 0.0360.767 ± 0.0120.841 ± 0.0180.847 ± 0.0180.857 ± 0.0210.847 ± 0.046
Segment0.848 ± 0.0740.803 ± 0.0140.898 ± 0.0380.927 ± 0.0310.911 ± 0.0380.936 ± 0.024
Semeion0.569 ± 0.0550.755 ± 0.0220.824 ± 0.0440.850 ± 0.0670.603 ± 0.0960.544 ± 0.067
Sonar0.497 ± 0.0410.573 ± 0.0570.561 ± 0.0910.524 ± 0.0740.602 ± 0.0890.590 ± 0.101
Spambase0.821 ± 0.0310.887 ± 0.0140.786 ± 0.0110.775 ± 0.0100.799 ± 0.0170.776 ± 0.018
Vehicle0.549 ± 0.0670.804 ± 0.0420.806 ± 0.0370.823 ± 0.0320.622 ± 0.0900.556 ± 0.077
Waveform0.860 ± 0.0120.829 ± 0.0120.795 ± 0.0150.743 ± 0.0130.772 ± 0.01290.729 ± 0.016
Wdbc0.798 ± 0.1550.801 ± 0.0430.861 ± 0.0680.845 ± 0.0630.884 ± 0.0500.872 ± 0.039
Yeast0.517 ± 0.0510.657 ± 0.0240.630 ± 0.0400.612 ± 0.0490.627 ± 0.0290.533 ± 0.046

Table 2. Comparison of average balanced accuracies of 20 models trained using dynamic subset alignment methods with PUSB and NTC-MI reported by Teisseyre et al. [115], and our formulation of incomplete extremal optimal transport from Gazdieva et al. [40] (labeled static). The best methods are colored red and second best blue.

Since OT methods rely on the assumption that data points in a dataset can be geometrically related through a metric, we used t-SNE (van der Maaten & Hinton, 2008) embeddings with perplexity 30 for all datasets in Table 2 to evaluate the spatial separability of the positive and negative classes. We observed that our methods perform better or comparably to competing methods when the positive class (1) and negative class (0) have spatially separable t-SNE embeddings as shown by the t-SNE embeddings are provided in Appendix § 13.

FFHQ Image Translation

We also apply our proposed approaches to the unpaired image translation problem. We followed the experimental setup of Gazdieva et al. [41], where the FFHQ dataset, embedded in the latent space of the Adversarial Latent Autoencoder (ALAE) (Pidhorskyi et al., [96]), is divided either by gender (man or woman) or by age, as two orthogonal labels.

ClassManWoman
Young15K23K
Old7K3.5K

Table 3. Division of FFHQ train images.

Table 3, adapted from Gazdieva et al. [41], shows the number of images for each class; images with ages less than 16 are ignored, ages between 16 and 43 are labeled young, and the remainder are labeled old. Given these classes, the task is to learn to map a source distribution to a target distribution. There are four cases: young→old, old→young, man→woman, and woman→man. To evaluate the translation process, two classifiers pretrained in the ALAE latent space are used; one classifier is trained to classify young versus old, and the other to distinguish man versus woman. The target accuracy quantifies the proportion of translated images that lie within the target-class boundary. The source accuracy quantifies whether the translated images retain the orthogonal label. For example, with young→old, the source accuracy is whether the “aged” image of a young source image retains the same gender.

Implementation details for both static and dynamic subset selection to the FFHQ dataset are given in Appendix § 11.§ 5. Between young→old, old→young, man→woman and woman→man, it was observed that larger values of cc tend to preserve the source accuracy but often have lower target accuracy. This can be related to the fact that for larger values of cc, it takes more training steps to achieve the optimal subset selection. Example translated images for old→young

translation with dynamic subset alignment are shown in Figure 4, with other cases provided in Appendix § 14.

FFHQ old-to-young translated images

Figure 4. FFHQ old→young using dynamic subset alignment with Euler integration (100 steps).

We compare with LOT Gazdieva et al. [41], UOT-GAN Yang & Uhler (2019), UOTM, and UOT-SD Choi et al. [26, 27]. Results in Table 4 and Figure 5, using accuracy values for UOT-SD, UOT-GAN, and U-LOT reported by Gazdieva et al. [41], whereas the implementation for UOTM is discussed in Appendix § 7.§ 1. Methods with better target accuracy are worse for source/class accuracy.

TaskAccuracyc=1c=1 staticc=1c=1 dynamicc=2c=2 staticc=2c=2 dynamicc=4c=4 staticc=4c=4 dynamicc=8c=8 staticc=8c=8 dynamicUOT-SDUOT-GANU-LOTUOTM KL-KLUOTM χ2\chi^2-KLUOTM lin.-χ2\chi^2
Young→OldTarget84.0983.4570.4779.2347.6374.5127.0773.9387.3384.2581.7871.0672.6880.78
Young→OldClass70.4371.5591.4184.3195.4790.0396.0690.3045.7173.8584.4990.1788.6878.77
Old→YoungTarget96.0693.3692.5586.6585.0485.0372.9383.3397.3995.8887.7988.6789.7094.19
Old→YoungClass71.7771.9289.4687.6994.4389.8495.4590.8849.3074.7489.4889.6488.1980.68
Man→WomanTarget96.1194.5392.1891.7485.6690.9677.5590.2998.1697.3890.2390.6391.1994.51
Man→WomanClass83.6883.6487.4587.4390.0588.5291.2789.1175.5084.0490.3089.5288.8385.31
Woman→ManTarget93.3492.2687.3288.0978.3284.2864.8681.8994.9692.9188.5983.8584.1091.41
Woman→ManClass82.3982.6892.3391.5193.8992.5994.3593.2272.0384.5689.6691.7990.8587.17

Table 4. Target and source accuracy (%) for different domain translations on the FFHQ dataset. Dynamic subset selection is evaluated using Euler integration with 100 steps.

Accuracy curves for Young→Old, Old→Young, Man→Woman, and Woman→Man domain translations

Figure 5. Accuracy curves for c∈{1,2,4,8}c \in\{1, 2, 4, 8\}, in comparison to results from LOT (Gazdieva et al., [41]), UOT-GAN (Yang & Uhler, 2019), UOT-SD and UOTM (Choi et al., [26, 27]).

Melanoma Dataset

The subset-alignment framework is primarily developed for aligning a source distribution μ\mu to an unknown distribution ν~\tilde{\nu} whose support is contained within the support of a target distribution ν\nu. Following Neklyudov et al. [90] and Pariset et al. [93], we adapt our dynamic subset-alignment method to the melanoma treatment data of Bunne et al. [18], using the preprocessed 50-dimensional representation. Unlike our other experiments, the melanoma setting involves three observed marginals at t∈{8hr,24hr,48hr}t \in\{8\mathrm{hr}, 24\mathrm{hr}, 48\mathrm{hr}\} and we must match the intermediate marginal exactly. We therefore order the populations in reverse chronological order, μ0\mu_0 (48h), μ1\mu_1 (24h), and μ2\mu_2 (8h), and train a multi-marginal interpolant that preserves μ0\mu_0 and μ1\mu_1 while leaving the terminal marginal (here, the 8h population) unconstrained so that sub-setting acts on μ2\mu_2. After training, we select cells x∼μ2x \sim\mu_2 with φt2(x)≤0\varphi_{t_2}(x) \le0 and transport the selected subset back to t0t_0 using the learned velocity field ∇φt\nabla\varphi_t. (We use normalized times t0=0t_0 = 0, t1=0.6t_1 = 0.6, t2=1t_2 = 1.) For evaluation, we integrate the ODE with explicit Euler (step size 10−310^{-3}) and report the best checkpoint based on validation MMD. The training procedure, neural parameterizations and experimental parameters for the Melanoma experiment are given in the Appendix E.6. As shown in Table 5, subset alignment improves over previous neural Schrödinger-bridge baselines (Chen et al., [22]; Pariset et al., [93]), while remaining competitive with WLF (Neklyudov et al., [90]).

ModelMMDW2W_2ModelMMDW2W_2
SB-FBSDE (Chen et al., [22])1.86e−21.86\mathrm{e}{-2}6.23Subset Alignment (c=1c = 1, 476/490)5.21e−35.21\mathrm{e}{-3}4.83
uDSB (no growth) (Pariset et al., [93])1.86e−21.86\mathrm{e}{-2}6.27Subset Alignment (c=2.0c = 2.0, 247/490)6.1e−36.1\mathrm{e}{-3}4.68
uDSB (with growth) (Pariset et al., [93])1.75e−21.75\mathrm{e}{-2}6.11Subset Alignment (c=4.0c = 4.0, 148/490)9.24e−39.24\mathrm{e}{-3}5.40
WLF-OT no growth (Neklyudov et al., [90])5.04e−35.04\mathrm{e}{-3}5.20Subset Alignment (c=8.0c = 8.0, 103/490)1.48e−21.48\mathrm{e}{-2}6.60
WLF-UOT λ=0.1\lambda= 0.1 (Neklyudov et al., [90])9.16e−39.16\mathrm{e}{-3}5.01

Table 5. Results for the Melanoma dataset (Pariset et al., [93], Table 1) with three marginals and train/test splits. MMD and W2W_2 are measured between νt=48\nu_{t=48} and ρt=48\rho_{t=48} obtained by running dynamics from μt=8\mu_{t=8}. For subset alignment, dynamics are run from the selected subset of μt=8\mu_{t=8} (cells with φt=8≤0\varphi_{t=8} \le0; number of selected test cells in parentheses), and results are reported at the iteration with the lowest MMD on a validation set; note that the validation cells are a subset of the training cells.

Celeb-A to Anime Domain Translation

For domain translation between the CelebA (Liu et al., [79]) and Anime (Chao, [19]) datasets, we use highly trimmed variants of the DDPM++ architecture (Song et al., [114]) to parameterize both QtQ_t and φt\varphi_t so that the models fit within GPU memory. Because our subset-alignment framework maps the source distribution to a subset of the target distribution, FID is not an appropriate evaluation metric for this setting. We therefore rely on qualitative visual inspection. In our experiments, dynamic subset alignment successfully translates the source distribution toward the target domain; however, larger values of cc generally require longer training to achieve comparable visual quality and stability. A sample translation is shown in Figure 6.

CelebA to Anime sample translation

Figure 6. CelebA→\toAnime.

Limitations

Practically, one point of concern is that the sample estimators of Wasserstein distances suffer from the curse of dimensionality (Weed & Bach, 2019; Fournier & Guillin, [37]). This issue can be alleviated to some extent by employing entropic regularization (Genevay et al., [44]; Feydy et al., [35]), which, in the dynamic setting, is intimately connected to Schrödinger bridges (Peluchetti, [94]; Bortoli et al., [14]).

Recently, entropically regularized UOT has been studied for modeling birth or death processes in population dynamics (Pariset et al., [93]; Neklyudov et al., [88]). Our approach can also be applied to model death processes in settings where there is a canonical relationship between temporally ordered

populations. In such cases, we treat μ\mu as the final population of survivors and ν\nu as the initial population. The learned mapping then relates the survivors to a subset of the initial population. Moreover, the sign of the potential can be used to distinguish members of the initial population that are likely to survive from those that are not. We note that, in this and other applications, the choice of cc is critical. While cc has a clear interpretation as a constraint on the density ratio, automatic selection of cc based on the resulting transport cost may be possible, as previously studied for partial optimal transport in the discrete setting [95]. One consolidated approach would be to sample cc from a prescribed range and use multi-task learning to optimize networks across varying values of cc. From an implementation perspective, this can be achieved by using scalar embeddings of cc, analogous to the time-variable embeddings used in dynamic networks.

Another key limitation of dynamic models, including frameworks similar to ours, is that their iterative generation process requires more computational resources and longer inference time [77]. For example, Table 6 reports the average per-sample generation time for FFHQ image translation. These results show that, in this setting, one-step generation is at least an order of magnitude faster than ODE integration. Considerable recent effort has been devoted to improving the generation efficiency of dynamic models [78, 45].

MetricOne-StepODE
Time/sample (s)1.44×10−41.44 \times10^{-4}2.84×10−32.84 \times10^{-3}

Table 6. Average evaluation time per sample for FFHQ domain translation tasks from Table 4. Evaluation is performed with batch size 128.

We note that although this work focuses on relaxing the target distribution, the range-divergence framework could also be used to relax the source distribution. A fully relaxed version, in which both source and target marginals are relaxed, may be applicable to other classes of problems.

Conclusion

Our neural optimal transport approach with subset selection achieves subset alignment by translating between two distributions while reweighting and selecting a subset of the target support. Our results on image-translation tasks across two datasets and 20 tabular PU-learning tasks show that a meaningful subset can be learned simultaneously with a Monge map. Unlike previous work, our dynamic formulation allows the terminal distribution to differ from the original target marginal, thereby inducing flows toward the nearest target subset while maintaining the prescribed density-ratio constraint.

AI use statement

We used generative AI tools primarily for copy-editing (improving grammar, clarity, and flow). We did not use generative AI for code development, except in the Melanoma dataset section, where an AI tool helped identify a bug. All AI-assisted outputs were reviewed and verified by the authors. We take full responsibility for the final content of this work, including all text, claims, and artifacts.

Reproducibility statement

To facilitate full reproducibility, we provide detailed descriptions of all experiments in the appendix. The main body summarizes the experimental setup and procedures for each experiment, while the appendix includes the complete specifications needed to replicate our results.

Preliminaries and Problem Formulation

Gaspard Monge proposed the original idea of optimal transport as mathematical model for the problem of minimum-cost transportation of dirt from a source location to a destination [87]. In more modern parlance, given probability measures, μ\mu defined on compact set X⊆Rd\mathcal{X} \subseteq\mathbb{R}^{d}, ν\nu defined on compact set Y⊆Rd\mathcal{Y} \subseteq\mathbb{R}^{d}, and the bounded uniformly continuous cost c(⋅,⋅):X×Y→Rc(\cdot,\cdot):\mathcal{X} \times\mathcal{Y} \to\mathbb{R}, the Monge formulation of optimal transport is stated as

DMonge(μ,ν)=inf⁡T∈T(X,Y)∫Xc(x,T(x)) μ(x)dxs.t.T#μ=ν(10)\mathcal{D}_{\mathrm{Monge}}(\mu,\nu) = \inf_{T \in\mathcal{T}(\mathcal{X},\mathcal{Y})} \int_{\mathcal{X}} c(\mathbf{x},T(\mathbf{x}))\,\mu(\mathbf{x})d\mathbf{x} \qquad\text{s.t.} \qquad T_{\#}\mu= \nu \tag*{(10)}

where the set T(X,Y)\mathcal{T}(\mathcal{X},\mathcal{Y}) denotes the set of measurable maps between X\mathcal{X} and Y\mathcal{Y}. Monge formulation of the optimal transport problem requires that the transport map TT be a deterministic function. To satisfy the constraint in the Monge problem 10, the transport map TT must cover ν\nu up to some ν\nu-null sets. Usually, the cost cc is non-linearly dependent on the transportation map TT, making the problem 10 very cumbersome and very difficult to solve [104] (Villani et al. 2009). Kantorovich reformulated the Monge problem by relaxing the constraint that supports of μ\mu and ν\nu should be related to each other by a functional relation TT [59]. Instead, he allowed the marginals μ\mu and ν\nu to be related to each other by a joint measure. Kantorovich’s reformulation of the problem is a linear program, and its solution exists for all convex lower-semi-continuous costs [104, 36]. The Kantorovich problem is

W(μ,ν)=inf⁡π∫X×Yc(x,y) π(x,y)dxdys.t.∫Yπ(x,y)dy=μ(x),∫Xπ(x,y)dx=ν(y),(11)\mathcal{W}(\mu,\nu) = \inf_{\pi} \int_{\mathcal{X} \times\mathcal{Y}} c(\mathbf{x},\mathbf{y})\,\pi(\mathbf{x},\mathbf{y})d\mathbf{x}d\mathbf{y} \qquad\text{s.t.} \qquad\int_{\mathcal{Y}} \pi(\mathbf{x},\mathbf{y})d\mathbf{y} = \mu(\mathbf{x}), \qquad\int_{\mathcal{X}} \pi(\mathbf{x},\mathbf{y})d\mathbf{x} = \nu(\mathbf{y}), \tag*{(11)}

where it can be observed that integrals ∫X\int_{\mathcal{X}} and ∫Y\int_{\mathcal{Y}} marginalize with respect to spaces Y\mathcal{Y} and X\mathcal{X} respectively. Therefore, π\pi must be the joint measure between μ\mu and ν\nu defined on the product space X×Y\mathcal{X} \times\mathcal{Y}, i.e. π∈P(X×Y)\pi\in\mathcal{P}(\mathcal{X} \times\mathcal{Y}). In other words, constraints in the Kantorovich problem ensure that every feasible solution π\pi must be a joint distribution of μ\mu and ν\nu. While in general, the Kantorovich problem is much easier to solve in comparison to the Monge problem, there are conditions of practical importance where one can employ the solutions of the Kantorovich problem to obtain the solution of the Monge problem. Those conditions are more clearly discussed in terms of the dual form of the Kantorovich problem Santambrogio, [104] given as,

W(μ,ν)=sup⁡f,g∫Xf(x)μ(x) dx+∫Yg(y)ν(y) dys.t.f(x)+g(y)≤c(x,y).(12)W(\mu,\nu)=\sup_{f,g}\int_{\mathcal{X}}f(x)\mu(x)\,\mathrm{d}x+\int_{\mathcal{Y}}g(y)\nu(y)\,\mathrm{d}y\quad\text{s.t.}\quad f(x)+g(y)\leq c(x,y). \tag*{(12)}

The functions f(x)f(x) and g(y)g(y) are called Kantorovich potentials. By defining c-conjugate (also called c-transform) of f(x)f(x), and cˉ\bar{c}-conjugate (also called cˉ\bar{c}-transform) of g(y)g(y) as

fc(y)=inf⁡x∈Xc(x,y)−f(x),(13)f^{c}(y)=\inf_{x\in\mathcal{X}}c(x,y)-f(x), \tag*{(13)}
gˉc(x)=inf⁡y∈Yc(x,y)−g(y),(14)\bar{g}^{c}(x)=\inf_{y\in\mathcal{Y}}c(x,y)-g(y), \tag*{(14)}

Using c and cˉ\bar{c} conjugates, Kantorovich problem is expressed as

W(μ,ν)=sup⁡f∫Xf(x)μ(x) dx+∫Yfc(y)ν(y) dy(15)W(\mu,\nu)=\sup_{f}\int_{\mathcal{X}}f(x)\mu(x)\,\mathrm{d}x+\int_{\mathcal{Y}}f^{c}(y)\nu(y)\,\mathrm{d}y \tag*{(15)}
=sup⁡g∫Xgˉc(x)μ(x) dx+∫Yg(y)ν(y) dy(16)=\sup_{g}\int_{\mathcal{X}}\bar{g}^{c}(x)\mu(x)\,\mathrm{d}x+\int_{\mathcal{Y}}g(y)\nu(y)\,\mathrm{d}y \tag*{(16)}

Under very general conditions, one can relate the cost cc with the support of optimal coupling solution π⋆\pi^{\star} and optimal Kantorovich potentials f⋆(x)f^{\star}(x) and fc⋆(y)f^{c\star}(y) Santambrogio, [104] by

supp⁡(π⋆)⊂{(x,y)∈X×Y:f⋆(x)+fc⋆(y)=c(x,y)}(17)\operatorname{supp}(\pi^{\star})\subset\{(x,y)\in\mathcal{X}\times\mathcal{Y}:f^{\star}(x)+f^{c\star}(y)=c(x,y)\} \tag*{(17)}

In discrete domains, above result is equivalent to Karush-Kuhn-Tucker (KKT) conditions for optimality. One can further relate the optimal solutions of Monge and Kantorovich problems using a landmark result by Gangbo & McCann [39] Figalli & Glaudo, [36], which states that there exists an optimal Kantorovich coupling of the form π⋆=(Id×T⋆)#μ\pi^{\star}=({\rm Id}\times T^{\star})_{\#}\mu, where T⋆T^{\star} is Monge map satisfying

∇xc(x,T⋆(x))+∇f⋆(x)=0,(18)\nabla_{x}c(x,T^{\star}(x))+\nabla f^{\star}(x)=0, \tag*{(18)}

if the following conditions are satisfied

  • μ\mu is absolutely continuous,

  • ∀y∈Y\forall y\in\mathcal{Y} map x↦c(x,y)x\mapsto c(x,y) is differentiable ∀x∈X\forall x\in\mathcal{X},

  • ∀x∈X\forall x\in\mathcal{X} the gradient map y↦∇xc(x,y)y\mapsto\nabla_{x}c(x,y) is injective ∀y∈Y\forall y\in\mathcal{Y},

  • and, the gradient ∇xc(x,y)\nabla_{x}c(x,y) satisfies the local Lipschitz condition ∥∇xc(x,y)∥≤Cr\lVert\nabla_{x}c(x,y)\rVert\leq C_{r} for all x∈Brx\in B_{r} where BrB_{r} is a ball of radius rr around xx.

When the cost can be written as c(x,y)=h(x−y)c(x,y)=h(x-y), where hh is strictly convex and translation-invariant function, one can further relate the Monge mapping with optimal dual potential by Santambrogio, [104]

T⋆(x)=x−∇h⋆∘∇f⋆(x),(19)T^{\star}(x)=x-\nabla h^{\star}\circ\nabla f^{\star}(x), \tag*{(19)}

where h⋆h^{\star} is the Legendre-Fenchel conjugate of hh given by h⋆(y)=sup⁡x∈X{⟨y,x⟩−h(x)}h^{\star}(y)=\sup_{x\in\mathcal{X}}\{\langle y,x\rangle-h(x)\}. In the result above, when h(x−y)=12∥x−y∥2h(x-y)=\frac{1}{2}\lVert x-y\rVert^{2}, one obtains the result of the celebrated Brenier theorem of optimal transport for squared-Euclidean costs with T⋆(x)=∇f⋆(x)T^{\star}(x)=\nabla f^{\star}(x), where f⋆(x)f^{\star}(x) is convex Brenier, [15] Figalli & Glaudo, [36]. The Brenier theorem on optimal transport differs from another important theorem on polar factorization Brenier, [15] stating that under very general conditions a square-integrable vector field vv can be decomposed into the composition of the gradient of a unique convex function ξ\xi and a unique measure-preserving map uu, i.e. v(x)=∇ξ∘u(x)v(x)=\nabla\xi\circ u(x). Before the discussion on dynamic formulation of the problem, we would like to point out that much of the recent work on static neural optimal transport relies on the above results.

Benamou and Brenier formulated the Wasserstein distance with squared Euclidean cost as the expected kinetic energy minimization problem under the assumption that both the source μ\mu and target ν\nu distributions have finite second moments Benamou & Brenier, [13]; Santambrogio [104]; Figalli & Glaudo [36]. Assuming that supports of both source and target distributions lie in a convex set Ω⊂Rd\Omega\subset\mathbb{R}^{d}, whose normal at the boundary is given by n:∂Ω→Rd\mathbf{n}:\partial\Omega\to\mathbb{R}^{d}. For a bounded and smooth velocity-field vt(x):[0,1]×Ω→Rd\mathbf{v}_{t}(x):[0,1]\times\Omega\to\mathbb{R}^{d} such that ⟨vt(x),n⟩∣∂Ω=0\langle\mathbf{v}_{t}(x),\mathbf{n}\rangle|_{\partial\Omega}=0, the flow corresponding to the velocity-field vt\mathbf{v}_{t} is given by

ddtΦt(xt)=vt(Φt(xt)),Φ0(x0)=x0.(20)\frac{\mathrm{d}}{\mathrm{d}t}\Phi_{t}(x_{t})=\mathbf{v}_{t}(\Phi_{t}(x_{t})),\quad\Phi_{0}(x_{0})=x_{0}. \tag*{(20)}

Considering that there also exists a probability path ρt(x):[0,1]×Ω→R+\rho_t(\mathbf{x}): [0,1] \times\Omega\to\mathbb{R}_{+} corresponding to the flow Φt(x)\Phi_t(\mathbf{x}) such that ρt(x)=Φt#ρ0(x)\rho_t(\mathbf{x}) = \Phi_{t\#}\rho_0(\mathbf{x}), Benamou-Brenier formulation of optimal transport is

inf⁡ρt,vt∫01∫Ω∥vt(x)∥22ρt(x)dx dt(21)\inf_{\rho_t,v_t} \int_0^1 \int_{\Omega} \frac{\lVert v_t(\mathbf{x})\rVert^2}{2}\rho_t(\mathbf{x})d\mathbf{x}\,dt \tag*{(21)}

s.t.

∂∂tρt(x)+div⁡(ρt(x)vt(x))=0,ρ0(x)=μ(x),ρ1(x)=ν(x),\frac{\partial}{\partial t}\rho_t(\mathbf{x})+\operatorname{div}(\rho_t(\mathbf{x})v_t(\mathbf{x}))=0,\quad\rho_0(\mathbf{x})=\mu(\mathbf{x}),\quad\rho_1(\mathbf{x})=\nu(\mathbf{x}),

where, div⁡(⋅)\operatorname{div}(\cdot) denotes the divergence operator mapping scalar or vector fields to scalar for the field zt(x)z_t(\mathbf{x}) by div⁡(zt(x))=∑i∂∂xizt(x)\operatorname{div}(z_t(\mathbf{x})) = \sum_i \frac{\partial}{\partial x_i}z_t(\mathbf{x}). The optimal flow Φt⋆\Phi_t^{\star} is related to Monge mapping T⋆T^{\star} by displacement interpolation [84],

Φt⋆=(1−t)Id⁡+tT⋆.(22)\Phi_t^{\star}=(1-t)\operatorname{Id}+tT^{\star}. \tag*{(22)}

It is important to mention that the Benamou-Brenier formulation can be extended to Wasserstein-pp distances, for p>1p>1, under the assumption that both source and target distributions have finite ppth moments [104 Chapters 5 & 6].

Static support subset alignment

Our formulation of static support subset alignment with optimal transport is derived from relaxed optimal transport problem where the constraint on the first marginal of the joint density π\pi is maintained, while the second marginal ∫Xπ(x,y)dx=ν~(y)\int_{\mathcal{X}}\pi(\mathbf{x},\mathbf{y})d\mathbf{x}=\widetilde{\nu}(\mathbf{y}) is allowed to vary from within a range [0,c][0,c] of the target density ν\nu, such that 0≤ν~(y)ν(y)≤c0\leq\frac{\widetilde{\nu}(\mathbf{y})}{\nu(\mathbf{y})}\leq c. The density ν~\widetilde{\nu} can be interpreted as a reweighted target density ν~(y)=ω(y)ν(y)\widetilde{\nu}(\mathbf{y})=\omega(\mathbf{y})\nu(\mathbf{y}), 0≤ω(y)≤c0\leq\omega(\mathbf{y})\leq c, where portions of the support can be up-weighted while others are down-weighted or removed. The relaxed constraint is equivalent to a case of the partial optimal transport relaxations using ff-divergences introduced by Séjourné et al. [108],

inf⁡π∫X×Yc(x,y)π(x,y)dxdy+Dι[a,b](ν~∥ν)s.t.∫Yπ(x,y)dy=μ(x),(23)\inf_{\pi}\int_{\mathcal{X}\times\mathcal{Y}}c(\mathbf{x},\mathbf{y})\pi(\mathbf{x},\mathbf{y})d\mathbf{x}d\mathbf{y}+D_{\iota_{[a,b]}}(\widetilde{\nu}\Vert\nu)\quad\text{s.t.}\quad\int_{\mathcal{Y}}\pi(\mathbf{x},\mathbf{y})d\mathbf{y}=\mu(\mathbf{x}), \tag*{(23)}

where Dι[a,b]D_{\iota_{[a,b]}} is the range divergence with ι[a,b]\iota_{[a,b]} being the convex indicator function,

ι[a,b](r)={0,r∈[a,b]+∞,o.w.,ι[a,b]∗(t)=sup⁡u∈[a,b](u⋅t)=max⁡(−at,bt),(24)\iota_{[a,b]}(r)= \begin{cases} 0, & r\in[a,b]\\ +\infty, & \text{o.w.} \end{cases} ,\qquad \iota_{[a,b]}^{\ast}(t)=\sup_{u\in[a,b]}(u\cdot t)=\max(-at,bt), \tag*{(24)}

where ι[a,b]∗\iota_{[a,b]}^{\ast} denotes its Legendre-Fenchel conjugate. Since the function ι[a,b]\iota_{[a,b]} is convex lower semi-continuous, therefore ι[a,b]=ι[a,b]∗∗\iota_{[a,b]}=\iota_{[a,b]}^{\ast\ast}, we can apply the variational form of the ff-divergence [91], Polyanskiy & Wu, [97], exploited by ff-GAN [92], leading to a form requiring only expected values

Dι[a,b](ν~∥ν)=∫Yι[a,b](ν~(y)ν(y))ν(y)dy=sup⁡η∫Yη(y)ν~(y)dy−∫Yι[a,b]∗(η(y))ν(y)dy.(25)D_{\iota_{[a,b]}}(\widetilde{\nu}\Vert\nu)=\int_{\mathcal{Y}}\iota_{[a,b]}\left(\frac{\widetilde{\nu}(y)}{\nu(y)}\right)\nu(y)dy=\sup_{\eta}\int_{\mathcal{Y}}\eta(y)\widetilde{\nu}(y)dy-\int_{\mathcal{Y}}\iota_{[a,b]}^{\ast}(\eta(y))\nu(y)dy. \tag*{(25)}

To match 23, we focus on a=0a=0 and b=c≥1b=c\geq1, such that ι[0,c]∗(t)=c⋅max⁡(0,t)\iota_{[0,c]}^{\ast}(t)=c\cdot\max(0,t) and for notational compactness denote η+(y)=max⁡(0,η(y))\eta_{+}(y)=\max(0,\eta(y)). Introducing ψ\psi as a measurable function to act as a Lagrange multiplier to enforce the constraint in 23 and combining with 25 yields the problem

inf⁡πsup⁡ψ,η∫X×Y(c(x,y)+η(y)−ψ(x))π(x,y)dxdy+∫Xψ(x)μ(x)dx−c∫Yη+(y)ν(y)dy.(26)\inf_{\pi}\sup_{\psi,\eta}\int_{\mathcal{X}\times\mathcal{Y}}\left(c(\mathbf{x},\mathbf{y})+\eta(\mathbf{y})-\psi(\mathbf{x})\right)\pi(\mathbf{x},\mathbf{y})d\mathbf{x}d\mathbf{y}+\int_{\mathcal{X}}\psi(\mathbf{x})\mu(\mathbf{x})dx-c\int_{\mathcal{Y}}\eta_{+}(\mathbf{y})\nu(\mathbf{y})dy. \tag*{(26)}

As described in Appendix A.2, since cc is convex and lower semi-continuous, we interchange the inf⁡π\inf_{\pi} and sup⁡η\sup_{\eta} and apply what is known as the cc-transform of −η(y)-\eta(\mathbf{y}) [104] (Villani et al. 2009) to obtain the dual problem with measurable map T:X→YT:\mathcal{X}\to\mathcal{Y}

sup⁡ηinf⁡T∫X(c(x,T(x))+η(T(x)))μ(x)dx−c∫Yη+(y)ν(y)dy.(27)\sup_{\eta}\inf_T\int_{\mathcal{X}}\left(c(\mathbf{x},T(\mathbf{x}))+\eta(T(\mathbf{x}))\right)\mu(\mathbf{x})dx-c\int_{\mathcal{Y}}\eta_{+}(\mathbf{y})\nu(\mathbf{y})dy. \tag*{(27)}

For computational implementation TT and η\eta are parameterized using neural networks with associated parameters θT\theta_T and θη\theta_{\eta} and expectations are estimated using samples from μ\mu and ν\nu as described in the Algorithm 2.

Algorithm 2: Static-Neural-SS: Learning Algorithm for Static Subset Alignment

Require: Source distribution μ\mu and target distribution ν\nu, cost function c(⋅,⋅)c(\cdot,\cdot), reweighting bound cc, neural networks T(⋅;θT)T(\cdot;\theta_T) and η(⋅;θη)\eta(\cdot;\theta_\eta), batch size NN, number of updates nTn_T and nηn_\eta, optimizers optim⁡T\operatorname{optim}_T and optim⁡η\operatorname{optim}_\eta.

Ensure: Trained neural estimator for transport map TT.

  1. for ℓ=1,2,…,L\ell=1,2,\ldots,L do

  2. for k=1,2,…,nTk=1,2,\ldots,n_T do

  3. Sample {xi}i=1N∼μ\{x_i\}_{i=1}^{N}\sim\mu and {yj}j=1N∼ν\{y_j\}_{j=1}^{N}\sim\nu

  4. grad⁡θT←∇θT1N∑i=1N[c(xi,T(xi;θT))+η(T(xi;θT);θη)]\operatorname{grad}_{\theta_T}\leftarrow\nabla_{\theta_T}\frac{1}{N}\sum_{i=1}^{N}\left[c(x_i,T(x_i;\theta_T))+\eta(T(x_i;\theta_T);\theta_\eta)\right]

  5. Using optim⁡T\operatorname{optim}_T update θT\theta_T via gradient descent

  6. end for

  7. for k=1,2,…,nηk=1,2,\ldots,n_\eta do

  8. Sample {xi}i=1N∼μ\{x_i\}_{i=1}^{N}\sim\mu and {yj}j=1N∼ν\{y_j\}_{j=1}^{N}\sim\nu

  9. grad⁡θη←∇θη[cN∑j=1Nmax⁡(0,η(yj;θη))−1N∑i=1Nη(T(xi;θT);θη)]\operatorname{grad}_{\theta_\eta}\leftarrow\nabla_{\theta_\eta}\left[\frac{c}{N}\sum_{j=1}^{N}\max(0,\eta(y_j;\theta_\eta))-\frac{1}{N}\sum_{i=1}^{N}\eta(T(x_i;\theta_T);\theta_\eta)\right]

  10. Using optim⁡η\operatorname{optim}_\eta update θη\theta_\eta via gradient ascent

  11. end for

  12. end for

Derivation of Static Neural Subset Selection

We denote the problem expressed in 26 as inf⁡πsup⁡ψsup⁡ηL(π,ψ,η)\inf_{\pi}\sup_{\psi}\sup_{\eta}\mathcal{L}(\pi,\psi,\eta), where the Lagrangian is

L(π,ψ,η)=∫X×Y(c(x,y)+η(y)+ψ(x))π(x,y) dxdy+∫Xψ(x) μ(x) dx−c∫Yη+(y)ν(y) dy.(28)\mathcal{L}(\pi,\psi,\eta)=\int_{\mathcal{X}\times\mathcal{Y}}\left(c(x,y)+\eta(y)+\psi(x)\right)\pi(x,y)\,dxdy+\int_{\mathcal{X}}\psi(x)\,\mu(x)\,dx-c\int_{\mathcal{Y}}\eta_{+}(y)\nu(y)\,dy. \tag*{(28)}

We proceed to interchange the sup and inf,4 which is allowed due to the strong duality property associated with optimal transport when the cost cc is convex and lower semi-continuous,

inf⁡πsup⁡ψ,ηL(π,ψ,η)=sup⁡η,ψinf⁡πL(π,ψ,η).(29)\inf_{\pi}\sup_{\psi,\eta}\mathcal{L}(\pi,\psi,\eta)=\sup_{\eta,\psi}\inf_{\pi}\mathcal{L}(\pi,\psi,\eta). \tag*{(29)}

optimizing with respect to π\pi for given η\eta and ψ\psi, the integrand in the first term of 28 is unbounded from below at any point c(x,y)+η(y)−ψ(x)<0c(x,y)+\eta(y)-\psi(x)<0. Thus, η\eta and ψ\psi need to ensure that c(x,y)+η(y)−ψ(x)≥0c(x,y)+\eta(y)-\psi(x)\geq0. This constraint requires that, for any x∈supp⁡(μ)⊆Xx\in\operatorname{supp}(\mu)\subseteq\mathcal{X}, ψ(x)=inf⁡y∈Y(c(x,y)+η(y))\psi(x)=\inf_{y\in\mathcal{Y}}(c(x,y)+\eta(y)) (Villani et al., 2009, Theorem 5.10 and Remark 5.13). This definition of ψ\psi corresponds to the cc-transform of −η(y)-\eta(y) in the optimal transport literature (Santambrogio, 2015; Villani et al., 2009). Then, the inner infimum with respect to π\pi is attained with zero value if ∀ (x,y)∈(X×Y):π(x,y)>0⟹c(x,y)+η(y)−ψ(x)=0\forall\,(x,y)\in(\mathcal{X}\times\mathcal{Y}):\pi(x,y)>0\Longrightarrow c(x,y)+\eta(y)-\psi(x)=0. Therefore, the dual problem becomes

sup⁡η∫X(inf⁡y∈Y(c(x,y)+η(y)))μ(x) dx−c∫Yη+(y)ν(y) dy.(30)\sup_{\eta}\int_{\mathcal{X}}\left(\inf_{y\in\mathcal{Y}}(c(x,y)+\eta(y))\right)\mu(x)\,dx-c\int_{\mathcal{Y}}\eta_{+}(y)\nu(y)\,dy. \tag*{(30)}
=sup⁡ηinf⁡T∫X(c(x,T(x))+η(T(x)))μ(x) dx−c∫Yη+(y)ν(y) dy(31)=\sup_{\eta}\inf_T\int_{\mathcal{X}}(c(x,T(x))+\eta(T(x)))\mu(x)\,dx-c\int_{\mathcal{Y}}\eta_{+}(y)\nu(y)\,dy \tag*{(31)}
=sup⁡ηinf⁡TEx∼μ[c(x,T(x))+η(T(x))]−cEy∼ν[η+(y)].(32)=\sup_{\eta}\inf_T\mathbb{E}_{x\sim\mu}[c(x,T(x))+\eta(T(x))]-c\mathbb{E}_{y\sim\nu}[\eta_{+}(y)]. \tag*{(32)}
=inf⁡Tsup⁡ηEx∼μ[c(x,T(x))]+Ex∼μ[η(T(x))]−cEy∼ν[η+(y)].(33)=\inf_T\sup_\eta\mathbb{E}_{x\sim\mu}[c(x,T(x))]+\mathbb{E}_{x\sim\mu}[\eta(T(x))]-c\mathbb{E}_{y\sim\nu}[\eta_{+}(y)]. \tag*{(33)}
=inf⁡TEx∼μ[c(x,T(x))]+D[0,c](T♯μ∥ν)=inf⁡T♯μ≤cνEx∼μ[c(x,T(x))].(34)=\inf_T\mathbb{E}_{x\sim\mu}[c(x,T(x))]+\mathcal{D}_{[0,c]}(T_{\sharp}\mu\Vert\nu) =\inf_{T_{\sharp}\mu\leq c\nu}\mathbb{E}_{x\sim\mu}[c(x,T(x))]. \tag*{(34)}

Equation 30 and equation 31 are equal due to a theorem by Rockafellar (1976, Theorem 3A); equation 31 and equation 32 are equivalent by definition; equation 32 and equation 33 are equivalent since the function is convex with respect to TT and concave with respect to η\eta; and equation 33 and equation 34 are equivalent by the variational formula for the ff-divergence. Thus, we obtain a relaxed Monge formulation.

Derivation of Dynamic Neural Subset Selection

The modified Benamou-Brenier problem takes the following minimax form

inf⁡ρt,vt∫01Ex∼ρt[∥vt(x)∥22]dt+sup⁡η:Ω→REx∼ρ1[η(x)]−Ex∼ν[ι[0,c]∗(η(x))]\inf_{\rho_t,v_t}\int_0^1 \mathbb{E}_{x\sim\rho_t}\left[\frac{\lVert v_t(x)\rVert^2}{2}\right]dt+\sup_{\eta:\Omega\to\mathbb{R}}\mathbb{E}_{x\sim\rho_1}[\eta(x)]-\mathbb{E}_{x\sim\nu}\left[\iota^*_{[0,c]}(\eta(x))\right]
s.t.∂∂tρt(x)+div⁡(ρt(x)vt(x))=0,ρ0(x)=μ(x)(35)\text{s.t.}\qquad\frac{\partial}{\partial t}\rho_t(x)+\operatorname{div}(\rho_t(x)v_t(x))=0,\qquad\rho_0(x)=\mu(x) \tag*{(35)}

Combining the objective and constraints in equation 35 to obtain the Lagrangian

L(vt,ρt,ψ0,φt,η)=∫01∫Ω∥vt(x)∥22ρt(x)dxdt⏟(I)+∫Ωψ0(x)(ρ0(x)−μ(x))dx⏟(II)+∫01∫Ωφt(x)∂∂tρt(x)dxdt⏟(III)+∫01∫Ωφt(x)div⁡(ρt(x)vt(x))dxdt⏟(IV)+sup⁡η(∫Ωη(x)ρ1(x)dx−c∫Ωmax⁡(0,η(x))ν(x)dx)⏟(V).(36)\begin{aligned} \mathcal{L}(v_t,\rho_t,\psi_0,\varphi_t,\eta) ={}&\underbrace{\int_0^1\int_\Omega\frac{\lVert v_t(x)\rVert^2}{2}\rho_t(x)dxdt}_{\text{(I)}}+\underbrace{\int_\Omega\psi_0(x)(\rho_0(x)-\mu(x))dx}_{\text{(II)}}\\ &+\underbrace{\int_0^1\int_\Omega\varphi_t(x)\frac{\partial}{\partial t}\rho_t(x)dxdt}_{\text{(III)}}+\underbrace{\int_0^1\int_\Omega\varphi_t(x)\operatorname{div}(\rho_t(x)v_t(x))dxdt}_{\text{(IV)}}\\ &+\underbrace{\sup_\eta\left(\int_\Omega\eta(x)\rho_1(x)dx-c\int_\Omega\max(0,\eta(x))\nu(x)dx\right)}_{\text{(V)}}. \tag*{(36)} \end{aligned}

Since ρt(x)\rho_t(x) is supported on a bounded subset Ω⊂Rd\Omega\subset\mathbb{R}^d, one can change the order of integration. Therefore, for term (III), by changing the order of integration and then computing integration by parts one obtains,

(III)=∫Ω∫01φt(x)∂∂tρt(x)dxdt=∫Ωφ1(x)ρ1(x)dx−∫Ωφ0(x)ρ0(x)dx−∫Ω∫01∂∂tφt(x)ρt(x)dxdt.(37)\begin{aligned} \text{(III)} &=\int_\Omega\int_0^1\varphi_t(x)\frac{\partial}{\partial t}\rho_t(x)dxdt\\ &=\int_\Omega\varphi_1(x)\rho_1(x)dx-\int_\Omega\varphi_0(x)\rho_0(x)dx-\int_\Omega\int_0^1\frac{\partial}{\partial t}\varphi_t(x)\rho_t(x)dxdt. \tag*{(37)} \end{aligned}

To simplify (IV), we can use product rule of derivatives to write

φt(x)div⁡(ρt(x)vt(x))=div⁡(φt(x)ρt(x)vt(x))−ρt(x)⟨∇φt(x),vt(x)⟩.\varphi_t(x)\operatorname{div}(\rho_t(x)v_t(x))=\operatorname{div}\left(\varphi_t(x)\rho_t(x)v_t(x)\right)-\rho_t(x)\langle\nabla\varphi_t(x),v_t(x)\rangle.

Therefore by combining the above identity with Gauss’s theorem, one obtains

(IV)=∫01∫Ωdiv⁡(φt(x)ρt(x)vt(x))dxdt−∫01∫Ωρt(x)⟨∇φt(x),vt(x)⟩dxdt=∫01∮∂Ωφt(xt)ρt(xt)⟨vt(x),dn⟩dt−∫01∫Ωρt(x)⟨∇φt(x),vt(x)⟩dxdt.(38)\begin{aligned} \text{(IV)} &=\int_0^1\int_\Omega\operatorname{div}\left(\varphi_t(x)\rho_t(x)v_t(x)\right)dxdt-\int_0^1\int_\Omega\rho_t(x)\langle\nabla\varphi_t(x),v_t(x)\rangle dxdt\\ &=\int_0^1\oint_{\partial\Omega}\varphi_t(x_t)\rho_t(x_t)\langle v_t(x),dn\rangle dt-\int_0^1\int_\Omega\rho_t(x)\langle\nabla\varphi_t(x),v_t(x)\rangle dxdt. \tag*{(38)} \end{aligned}

From the boundary condition on optimal transport (see the discussion above equation (20), and also [36 section 4.1]), the first part of the right-hand side of equation 38 is zero; therefore,

(IV)=∫01∫Ωφt(x)⋅div⁡(ρt(x)vt(x))dxdt=−∫01∫Ω⟨∇φt(x),vt(x)⟩ρt(x)dxdt.(39)\text{(IV)}=\int_0^1\int_\Omega\varphi_t(x)\cdot\operatorname{div}(\rho_t(x)v_t(x))dxdt=-\int_0^1\int_\Omega\langle\nabla\varphi_t(x),v_t(x)\rangle\rho_t(x)dxdt. \tag*{(39)}

To eliminate the primal variable vt(x)v_t(x), substitute equation 37 and equation 39 into equation 36 and compute the variational derivative to obtain the stationary condition. For that one can write the terms of Lagrangian depending on vt(x)v_t(x) as

L~(vt)=∫01∫Ω(∥vt(x)∥22−⟨∇φt(x),vt(x)⟩)ρt(x)dxdt.(40)\widetilde{\mathcal{L}}(v_t)=\int_0^1\int_\Omega\left(\frac{\lVert v_t(x)\rVert^2}{2}-\langle\nabla\varphi_t(x),v_t(x)\rangle\right)\rho_t(x)dxdt. \tag*{(40)}

With the additive perturbation function τt\tau_t vanishing at t=0t=0 and t=1t=1 and a scalar ε\varepsilon, the Lagrangian L~(vt+ετt)\widetilde{\mathcal{L}}(v_t+\varepsilon\tau_t) is

L~(vt+ετt)=∫01∫Ω(∥vt(x)+ετt(x)∥22−⟨∇φt(x),vt(x)+ετt(x)⟩)ρt(x)dxdt=∫01∫Ω(∥vt(x)∥22−⟨∇φt(x),vt(x)⟩)ρt(x)dxdt+∫01∫Ω(ε2∥τt(x)∥22+ε⟨vt(x)−∇φt(x),τt(x)⟩)ρt(x)dxdt,\begin{aligned} \widetilde{\mathcal{L}}(v_t+\varepsilon\tau_t) ={}&\int_0^1\int_\Omega\left(\frac{\lVert v_t(x)+\varepsilon\tau_t(x)\rVert^2}{2}-\langle\nabla\varphi_t(x),v_t(x)+\varepsilon\tau_t(x)\rangle\right)\rho_t(x)dxdt\\ ={}&\int_0^1\int_\Omega\left(\frac{\lVert v_t(x)\rVert^2}{2}-\langle\nabla\varphi_t(x),v_t(x)\rangle\right)\rho_t(x)dxdt\\ &+\int_0^1\int_\Omega\left(\varepsilon^2\frac{\lVert\tau_t(x)\rVert^2}{2}+\varepsilon\langle v_t(x)-\nabla\varphi_t(x),\tau_t(x)\rangle\right)\rho_t(x)dxdt, \end{aligned}

and variational derivative is

δL~(vt(x))δvt∣vt=ddεL~(vt+ετt)∣ε=0=∫01∫Ω⟨vt(x)−∇φt(x),τt(x)⟩ρt(x) dxdt.(41)\left.\frac{\delta\widetilde{\mathcal{L}}(\mathbf{v}_t(\mathbf{x}))}{\delta\mathbf{v}_t}\right|_{\mathbf{v}_t} = \left.\frac{d}{d\varepsilon}\widetilde{\mathcal{L}}(\mathbf{v}_t+\varepsilon\boldsymbol{\tau}_t)\right|_{\varepsilon=0} = \int_0^1\int_{\Omega}\langle\mathbf{v}_t(\mathbf{x})-\nabla\varphi_t(\mathbf{x}),\boldsymbol{\tau}_t(\mathbf{x})\rangle\rho_t(\mathbf{x})\,d\mathbf{x}dt. \tag*{(41)}

The stationarity condition requires δvtL~(vt(x))=0\delta_{\mathbf{v}_t}\widetilde{\mathcal{L}}(\mathbf{v}_t(\mathbf{x}))=0. For arbitrary perturbation τt(x)\boldsymbol{\tau}_t(\mathbf{x}), the variational-derivative δvtL~(vt(x))=0\delta_{\mathbf{v}_t}\widetilde{\mathcal{L}}(\mathbf{v}_t(\mathbf{x}))=0 if and only if

vt(x)=∇φt(x).(42)\mathbf{v}_t(\mathbf{x})=\nabla\varphi_t(\mathbf{x}). \tag*{(42)}

Therefore, one can write the Lagrangian as

L(ρt,ψ0,φt,η)=∫Ωψ0(x)⋅(ρ0(x)−μ(x)) dx+∫Ωφ1(x)ρ1(x) dx−∫Ωφ0(x)ρ0(x) dx+∫Ωη(x)ρ1(x) dx−c⋅∫Ωmax⁡(0,η(x))ν(x) dx−∫01∫Ω(∂∂tφt(x)+∥∇φt(x)∥22)ρt(x) dxdt.\begin{aligned} \mathcal{L}(\rho_t,\psi_0,\varphi_t,\eta) ={}&\int_{\Omega}\psi_0(\mathbf{x})\cdot(\rho_0(\mathbf{x})-\mu(\mathbf{x}))\,d\mathbf{x} +\int_{\Omega}\varphi_1(\mathbf{x})\rho_1(\mathbf{x})\,d\mathbf{x} -\int_{\Omega}\varphi_0(\mathbf{x})\rho_0(\mathbf{x})\,d\mathbf{x}\\ &+\int_{\Omega}\eta(\mathbf{x})\rho_1(\mathbf{x})\,d\mathbf{x} -c\cdot\int_{\Omega}\max(0,\eta(\mathbf{x}))\nu(\mathbf{x})\,d\mathbf{x}\\ &-\int_0^1\int_{\Omega}\left(\frac{\partial}{\partial t}\varphi_t(\mathbf{x})+\frac{\|\nabla\varphi_t(\mathbf{x})\|^2}{2}\right)\rho_t(\mathbf{x})\,d\mathbf{x}dt. \end{aligned}

Similarly, by computing δψ0L\delta_{\psi_0}\mathcal{L}, δρ1L\delta_{\rho_1}\mathcal{L}, and using stationary conditions, one obtains the condition,

ψ0(x)=φ0(x),(43)\psi_0(\mathbf{x})=\varphi_0(\mathbf{x}), \tag*{(43)}
η(x)=−φ1(x).(44)\eta(\mathbf{x})=-\varphi_1(\mathbf{x}). \tag*{(44)}

Therefore, Lagrangian is simplified to

L(ρt,φt,η)=−∫Ωφ0(x)μ(x) dx−c∫Ωmax⁡(0,−φ1(x))ν(x) dx−∫01∫Ω(∂∂tφt(x)+∥∇φt(x)∥22)ρt(x) dxdt.(45)\begin{aligned} \mathcal{L}(\rho_t,\varphi_t,\eta) ={}&-\int_{\Omega}\varphi_0(\mathbf{x})\mu(\mathbf{x})\,d\mathbf{x} -c\int_{\Omega}\max(0,-\varphi_1(\mathbf{x}))\nu(\mathbf{x})\,d\mathbf{x}\\ &-\int_0^1\int_{\Omega}\left(\frac{\partial}{\partial t}\varphi_t(\mathbf{x})+\frac{\|\nabla\varphi_t(\mathbf{x})\|^2}{2}\right)\rho_t(\mathbf{x})\,d\mathbf{x}dt. \tag*{(45)} \end{aligned}

The simplified problem (equation 7 in the main body) is

sup⁡ρtinf⁡φtEx∼μ[φ0(x)]+c⋅Ex∼ν[max⁡(0,−φ1(x))]+∫01Ext∼ρt[∂∂tφt(xt)+∥∇φt(xt)∥22]dt.\sup_{\rho_t}\inf_{\varphi_t}\mathbb{E}_{\mathbf{x}\sim\mu}[\varphi_0(\mathbf{x})] +c\cdot\mathbb{E}_{\mathbf{x}\sim\nu}[\max(0,-\varphi_1(\mathbf{x}))] +\int_0^1\mathbb{E}_{\mathbf{x}_t\sim\rho_t}\left[\frac{\partial}{\partial t}\varphi_t(\mathbf{x}_t)+\frac{\|\nabla\varphi_t(\mathbf{x}_t)\|^2}{2}\right]dt.

Thresholding for PU-Learning and Rejection Sampling

Our idea of rejection sampling and thresholding for PU-learning relies on the fact that the dual form of range divergence is zero when the supremum in the dual is attained by the function η∗(x)\eta^*(\mathbf{x}) with ν~(x)=ρ1∗(x)\widetilde{\nu}(\mathbf{x})=\rho_1^*(\mathbf{x}) i.e.

Ex∼ν~[η∗(x)]−c Ex∼ν[ReLU⁡(η∗(x))]=0(46)\mathbb{E}_{\mathbf{x}\sim\widetilde{\nu}}[\eta^*(\mathbf{x})]-c\ \mathbb{E}_{\mathbf{x}\sim\nu}[\operatorname{ReLU}(\eta^*(\mathbf{x}))]=0 \tag*{(46)}

By defining A=supp⁡(ν)\mathcal{A}=\operatorname{supp}(\nu), A~=supp⁡(ν~)\widetilde{\mathcal{A}}=\operatorname{supp}(\widetilde{\nu}), and A‾=A/A~\overline{\mathcal{A}}=\mathcal{A}/\widetilde{\mathcal{A}}, one can also see that A~∩A‾=∅\widetilde{\mathcal{A}}\cap\overline{\mathcal{A}}=\varnothing, therefore one can write equation 46 as

∫A~η∗(x)ν~(x) dx−c∫A~ReLU⁡(η∗(x))ν(x) dx−c∫A‾ReLU⁡(η∗(x))ν(x) dx=0.(47)\int_{\widetilde{\mathcal{A}}}\eta^*(\mathbf{x})\widetilde{\nu}(\mathbf{x})\,d\mathbf{x} -c\int_{\widetilde{\mathcal{A}}}\operatorname{ReLU}(\eta^*(\mathbf{x}))\nu(\mathbf{x})\,d\mathbf{x} -c\int_{\overline{\mathcal{A}}}\operatorname{ReLU}(\eta^*(\mathbf{x}))\nu(\mathbf{x})\,d\mathbf{x}=0. \tag*{(47)}

One can further write

η∗(x)=ReLU⁡(η∗(x))−ReLU⁡(−η∗(x)).(48)\eta^*(\mathbf{x})=\operatorname{ReLU}(\eta^*(\mathbf{x}))-\operatorname{ReLU}(-\eta^*(\mathbf{x})). \tag*{(48)}

After substituting equation 48 into equation 47 one obtains

∫A~ReLU⁡(η∗(x))(ν~(x)−cν(x)) dx⏟LHS=∫A~ReLU⁡(−η∗(x))ν~(x) dx+c∫A‾ReLU⁡(η∗(x))ν(x) dx⏟RHS(49)\underbrace{\int_{\widetilde{\mathcal{A}}}\operatorname{ReLU}(\eta^*(\mathbf{x}))(\widetilde{\nu}(\mathbf{x})-c\nu(\mathbf{x}))\,d\mathbf{x}}_{\mathrm{LHS}} = \underbrace{\int_{\widetilde{\mathcal{A}}}\operatorname{ReLU}(-\eta^*(\mathbf{x}))\widetilde{\nu}(\mathbf{x})\,d\mathbf{x} +c\int_{\overline{\mathcal{A}}}\operatorname{ReLU}(\eta^*(\mathbf{x}))\nu(\mathbf{x})\,d\mathbf{x}}_{\mathrm{RHS}} \tag*{(49)}

The dual form equation 47 is optimal with zero duality gap, if the primal form satisfies ∀x∈A\forall\mathbf{x}\in\mathcal{A}, ι[0,c](ν~(x)ν(x))=0\iota_{[0,c]}\left(\frac{\widetilde{\nu}(\mathbf{x})}{\nu(\mathbf{x})}\right)=0, which can also be restricted to ∀x∈A~\forall\mathbf{x}\in\widetilde{\mathcal{A}} ι[0,c](ν~(x)ν(x))=0\iota_{[0,c]}\left(\frac{\widetilde{\nu}(\mathbf{x})}{\nu(\mathbf{x})}\right)=0. This is equivalent to ν~(x)≤cν(x)\widetilde{\nu}(\mathbf{x})\le c\nu(\mathbf{x}) almost-everywhere in A~\widetilde{\mathcal{A}}. Therefore, one can say that 0≥LHS0\ge\mathrm{LHS} and also

0≥∫A‾ReLU⁡(−η∗(x))ν(x) dx+c∫A‾ReLU⁡(η∗(x))ν(x) dx⏟RHS(50)0\ge \underbrace{\int_{\overline{\mathcal{A}}}\operatorname{ReLU}(-\eta^*(\mathbf{x}))\nu(\mathbf{x})\,d\mathbf{x} +c\int_{\overline{\mathcal{A}}}\operatorname{ReLU}(\eta^*(\mathbf{x}))\nu(\mathbf{x})\,d\mathbf{x}}_{\mathrm{RHS}} \tag*{(50)}

We can now see that both integrands in equation (49) are nonnegative and sum to a value less than or equal to zero, which is only possible if both are equal to zero. Therefore, one can write

0=∫A~ReLU⁡(−η⋆(x))ν(x) dx+c∫AˉReLU⁡(η⋆(x))ν(x) dx(51)0 = \int_{\tilde{A}} \operatorname{ReLU}(-\eta^{\star}(x))\nu(x)\,\mathrm{d}x + c\int_{\bar{A}} \operatorname{ReLU}(\eta^{\star}(x))\nu(x)\,\mathrm{d}x \tag*{(51)}

Further, two non-negative integrals are evaluated on two mutually exclusive sets; therefore, to have a sum equal to zero value, we can conclude that each integral is zero individually. Therefore, we can write

0=∫A~ReLU⁡(−η⋆(x))ν(x) dx=c∫AˉReLU⁡(η⋆(x))ν(x) dx(52)0 = \int_{\tilde{A}} \operatorname{ReLU}(-\eta^{\star}(x))\nu(x)\,\mathrm{d}x = c\int_{\bar{A}} \operatorname{ReLU}(\eta^{\star}(x))\nu(x)\,\mathrm{d}x \tag*{(52)}

Equation 52 is therefore equivalent to the following element-wise test:

η⋆(x)≥0, almost surely in A~\eta^{\star}(x) \ge0,\ \text{almost surely in }\tilde{A}
η⋆(x)<0, almost surely in Aˉ(53)\eta^{\star}(x) < 0,\ \text{almost surely in }\bar{A} \tag*{(53)}

Additionally, from the equation 50, one can also conclude that

∫A~ReLU⁡(η⋆(x))(ν~(x)−cν(x)) dx=0,(54)\int_{\tilde{A}} \operatorname{ReLU}(\eta^{\star}(x))(\tilde{\nu}(x)-c\nu(x))\,\mathrm{d}x = 0, \tag*{(54)}

which is a complementary slackness condition in the sense that ν~(x)<cν(x)⟹η⋆(x)=0\tilde{\nu}(x) < c\nu(x) \Longrightarrow\eta^{\star}(x) = 0 almost everywhere in AA. During the neural network training with finite data-points, the potential function η\eta is usually sub-optimal, and its sign cannot be relied on; therefore, instead of directly using the sign, one can sort values of potential at data points and select a predetermined proportion (prior) of data-points. Therefore, for training PU-learning models, we applied both sign-based and sorting-based filtration of data. From Figures 7, Figure 8 and Figure 9, one can observe that the optimal potential for static problem exactly follows equation (53), whereas in the dynamic case the sign of φ1\varphi_1 is inverted, which is due to the relation obtained in equation (44), which ensures that for the at optimal φ1⋆\varphi_1^{\star} following relation holds

Temporal snapshots of the evolution between bimodal and uniform distributions and the potential $\varphi_t$

Figure 8. Temporal snapshots at t∈{0.0,0.25,0.50,0.75,1.0}t \in\{0.0, 0.25, 0.50, 0.75, 1.0\} for c∈1,2,4,8c \in1, 2, 4, 8. (a) Evolution between two 2D bimodal distributions; the black contour line indicates the zero-level set of φt(x)\varphi_t(x). (b) Evolution from 1D uniform to another 1D uniform density, with dynamic subset subset alignment. φt(x)\varphi_t(x) is denoted by red curve.

Static subset alignment results for $c \in \{1,2,4,8\}$

Figure 9. Static subset alignment results for c∈{1,2,4,8}c \in\{1, 2, 4, 8\}. (a) Translation between two 2D uniform distributions shown in Figure 1. (b) 2D bimodal distributions shown in Figure 8a. Red curves indicate the zero-level sets of η(x)\eta(x).

φ1⋆(x)≤0, almost surely in A~\varphi_1^{\star}(x) \le0,\ \text{almost surely in }\tilde{A}
φ1⋆(x)>0, almost surely in Aˉ.(55)\varphi_1^{\star}(x) > 0,\ \text{almost surely in }\bar{A}. \tag*{(55)}

Survey of Recent Work on Neural Optimal Transport

In this section, we discuss the recent related work on computational optimal transport and its applications. More specifically, we consider the works that are related to neural estimation of optimal transport maps with occasional reference to theoretical developments.

Static Neural Optimal Transport

Seguy et al. [107] employed stochastic gradient-based approaches in one of the earliest works to estimate the optimal transport map using neural networks. Notably, the work by Seguy et al. [107] differed from early work by Genevay et al. [42] in the sense that the later work employed stochastic gradient-based methods to estimate the transport plan for large-scale data, whereas earlier work by Genevay et al. [42] only minimized the optimal transport loss using stochastic-gradient-based methods. This is also in contrast to the well-known Wasserstein-GAN (Arjovsky et al., [8]; Gulrajani et al., [48]) that employs the Kantorovich-Rubinstein duality to minimize the Wasserstein-1 loss function for generative modeling, where neural networks are employed as parameterizations for both dual-potential and data generator, but do not provide transport plans. Finally, the Sinkhorn-GAN employs an approximation of the discrete Wasserstein distance between latent representations of data and that of samples from non-informative prior (Genevay et al., [43]) for generative modeling. Now, we can see clear distinction between two different classes of approaches employing Wasserstein distances in generative modeling, the first class of works concerns with employing Wasserstein distance as a loss for generative modeling, without any explicit concern for obtaining the underlying transport plan across the distributions (Arjovsky et al., [8]; Gulrajani et al., [48]). The second class seeks to learn a transport plan to realize the generative model.

Efforts to learn Monge maps were motivated by a theorem by Brenier [15], which essentially states that, for continuous distributions with squared-Euclidean transportation cost, the optimal solution of the Monge problem is the gradient of a convex function (Figalli & Glaudo, [36], Theorem 2.5.10). Therefore, initially, gradients of input-convex neural networks (ICNN) (Amos et al., [6]) were employed to estimate the transport plan for the Wasserstein-2 distance [83, 68 b]. This approach has also been employed to supervised conditional neural Monge maps [16] and unbalanced optimal transport [81]. The study by Amos et al. [7] focuses on the development of an efficient neural optimal solution that could be implemented quickly in more practical scenarios. This approach to solve Wasserstein-2 distances employing convex potentials involves computationally challenging evaluation of the Fenchel conjugate of an ICNN-parameterized convex function. More recent work in this direction focuses on improved optimization strategies and better ICNN architectures to bypass problems related to Fenchel conjugate evaluations and ICNN training (Amos, 2023; Vesseron & Cuturi, 2024). Recent work also focuses on some batch-based methods, which have been devised to improve the regularity of learned neural Monge maps [31] (Uscidda & Cuturi 2023).

Another recent direction of work is based on the idea that ICNNs can be overly restrictive; therefore, more general neural network architectures should be employed to directly parameterize the transport maps [103, 71]. The work by Fan et al. [32, 34] focuses on employing neural networks to approximate the solution for Monge’s transport problem and also draws inspiration from the recent developments in neural-network-based parametric realizations for approximating Kantorovich plans. Recently, neural optimal transport has also been extended to unbalanced transportation settings [26] (Yang & Uhler 2019). Another work directly related to the static subset selection problem is the inexact optimal transport Gazdieva et al. [40].

Unless there is a corresponding Monge mapping [27, 86, 46], optimal transport requires a stochastic transport plan. A recent body of work (Korotin et al., 2023b,a; Asadulaev et al., 2024) deals with learning transportation plans using a weaker formulation of optimal transport [47, 11] along with noise outsourcing techniques, which have also been extended to more general costs. Apart from the applications in image translation [71], neural optimal transport has been applied for biomedical image registration [65] and to study single-cell perturbations [18]. Neural optimal transport has also been employed for metric learning [55, 106].

Dynamic Neural Optimal Transport

The potential applications of dynamic optimal transport in the cellular trajectory inference [116] and its connections with flow-based models for generative modeling [56, 57] has been instrumental in the recent research developments in this direction. Jordan-Kinderlehrer-Otto flow (JKO) is a time discretization scheme to solve Wasserstein gradient flows for different energy functionals [58] (Santambrogio 2017). Therefore, a lot of effort done in that regard is focused on neural network-parameterized schemes to solve the JKO-flow problem for both cellular trajectory inference and generative modeling [82, 33, 73, 17, 26, 28, 3, 85, 4] (Xu et al. 2023). The JKO scheme has also been studied for the applications related to molecular discovery [4]. A recent study deals with convergence properties of JKO-based generative models [23].

Recent developments in flow-matching models based on flow matching [74, 1, 77] for generative modeling led to even more interest in the development of algorithms to solve dynamic optimal transportation problems. The action-matching-based framework led to the development of a more general framework to solve both trajectory inference and generative modeling problems [88] for the cases where one could also sample from the trajectory between two terminal marginals. Rectified flow matching [77, 78] uses the neural optimal transport in an additional rectification step to improve the linearity of flows so that after training the model, images could be generated efficiently with only a single-step integration along straight-line paths. For generative modeling, in contrast to target-conditional flow matching [74], where during training, flows are conditioned on target samples, discrete optimal transport-conditioned flow matching employs the mini-batch optimal transport to create the conditionals [98, 99, 117]. Another recent work [67] attempts to alleviate the error accumulation problems associated with mini-batch optimal transport by learning straight paths between source and target distributions in a single step. Flow-matching [1, 2], diffusion models [110, 113, 112], and Schrödinger bridges (Wang et al., 2021; Liu et al., 2022, 2024a; Shi et al., 2023; Gushchin et al., 2023b,a; Somnath et al., 2023) are deeply interconnected under the framework of generalized bridge matching [117, 2, 117, 109]. Recently, there have also been attempts to understand diffusion models as approaches to minimize the dynamic Wasserstein distances [72, 64]. Recent works generalize flow matching from different perspectives, Chen & Lipman [21] generalize the flow matching to the flows on Riemannian manifolds, Atanackovic et al. [10] attempt to extend the flow models to return meaningful flows for the data beyond training distributions, and Haviv et al. [51] generalize the flow matching to the cases where data can be treated as distributions of distributions.

Temporal snapshots of the evolution between distributions and the potential $\varphi_t$

Figure 7. Temporal snapshots at t∈{0.0,0.25,0.50,0.75,1.0}t \in\{0.0, 0.25, 0.50, 0.75, 1.0\} for c∈1,2,4,8c \in1, 2, 4, 8. (a) Evolution between two 2D uniform distributions shown in Figure 1; the black contour line indicates the zero-level set. (b) Evolution from 1D uniform to semicircular density is shown in Figure 2 with support subset alignment. φt(x)\varphi_t(x) is denoted by red curve.

Additionally, there has been a recent dynamic extension to the conditional neural optimal transport ([54, 63]). There have also been efforts to study neural network-based scalable approaches to solve high-dimensional partial differential equations (Wan et al., 2023).

Additional Toy Results

For the toy results given in Figure 1 and Figure 2 the temporal snapshots of the evolution of points sampled from source along with the potential φt\varphi_t are given in Figures 7a and 7b, respectively. Similar temporal snapshots between two 2D bimodal uniform squares and two 1D uniform distributions are shown in Figure 8a and Figure 8b respectively.

The static framework for subset alignment is equivalent to the relaxations of the inexact optimal transport problem discussed by Gazdieva et al. [40]. Our approach for static subset alignment is discussed in Algorithm 2 and Appendix § 7.§ 1.

Implementation Details

Toy Results

For dynamic subset alignment, we use the model architectures in Listings 1 and 2, where SmodelS_{\mathrm{model}} and QmodelQ_{\mathrm{model}} parameterize θφ\theta_{\varphi} and θρ\theta_{\rho}, respectively. Listing 1 is used for the 1D toy experiments, while Listing 2 is used for the 2D toy experiments. We set num_hid=512 for all dynamic architectures. For both settings, we use Algorithm 1 with nφ=nρ=1n_{\varphi}=n_{\rho}=1. We train for 50,000 iterations in 1D and 100,000 iterations in 2D. In all dynamic toy experiments, we use the Adam optimizer (Kingma & Ba, [66]) with a learning rate of 1×10−41 \times10^{-4} and a batch size of 512.

For static subset alignment, we use the model architecture in Listing 3 for both the 1D and 2D toy experiments. The TmodelT_{\mathrm{model}} parameterizes the transport map TT via a residual connection, while DualNET parameterizes the dual function η\eta. The same architecture is used in both settings, with the input and output dimensions adjusted accordingly. We set num_hid=512 for all static architectures. For both settings, we use Algorithm 2 with nT=5n_T=5 and nη=1n_{\eta}=1. We train for 20,000 iterations in 1D and 50,000 iterations in 2D. In all static toy experiments, we use the Adam optimizer (Kingma & Ba, [66]) with a learning rate of 1×10−41 \times10^{-4} and a batch size of 512.

EMNIST Classifier

We merge all letters into a single class and treat each digit as its own class (digits 0–9 keep their numeric label, and all letters are labeled 10). To mitigate class imbalance during training, we use a class-reweighted SoftMax loss. For kk-class classification, let z∈Rkz \in\mathbb{R}^k denote the per-class counts in the training set; we define reweighting coefficients ω∈Rk\omega\in\mathbb{R}^k by

ωi=(∑j=1Nzizj)−1,∀i∈[k].(56)\omega_i = \left(\sum_{j=1}^{N}\frac{z_i}{z_j}\right)^{-1},\quad\forall i \in[k]. \tag*{(56)}

Given a one-hot label vector yy and the network SoftMax output y^\hat{y}, the reweighted loss (risk) is

ℓ(y,y^)=1k(ω⊙y⊙y^)(57)\ell(y,\hat{y}) = \mathbf{1}_k(\omega\odot y \odot\hat{y}) \tag*{(57)}

We train the EMNIST classifier using the train/validation split provided with the EMNIST dataset (Cohen et al., [29]). The model is ResNet-18 (He et al., [52]) and is optimized with the class-reweighted SoftMax loss in equation 57. We

import jax
from jax import numpy as jnp
from flax import linen as nn
import math
from ml_collections import config_dict

class Smodel(nn.Module):
    num_hid: int
    num_out: int
    use_layernorm: bool = True
    boundary_zero: bool = False

    @nn.compact
    def __call__(self, t, x):
        if jnp.ndim(t) == 0:
            t = jnp.broadcast_to(t, x.shape[:-1] + (1,))
        elif t.shape[-1] != 1:
            t = t.reshape(x.shape[:-1] + (1,))

        h = jnp.concatenate([t, x], axis=-1)
        for _ in range(4):
            h = nn.Dense(self.num_hid)(h)
            if self.use_layernorm:
                h = nn.LayerNorm()(h)
            h = nn.swish(h)

        v = nn.Dense(self.num_out)(h)
        return t * (1.0 - t) * v if self.boundary_zero else v

class Qmodel(nn.Module):
    num_hid: int
    num_out: int

    @nn.compact
    def __call__(self, t, x_0, x_1):
        h = jnp.concatenate([t, x_0, x_1, t < 0.5], axis=-1)
        for _ in range(3):
            h = nn.swish(nn.Dense(self.num_hid)(h))

        h = nn.Dense(self.num_out)(h)
        return (1.0 - t) * x_0 + t * (x_1 + h)

Listing 1: Model architectures used for toy 1D dynamic subset alignment experiments.

import jax
from jax import numpy as jnp
from flax import linen as nn
import math
from ml_collections import config_dict

class Smodel(nn.Module):
    num_hid: int
    num_out: int

    @nn.compact
    def __call__(self, t, x):
        if jnp.ndim(t) == 0:
            t = jnp.broadcast_to(t, x.shape[:-1] + (1,))
        h = jnp.concatenate([t, x], axis=-1)
        for _ in range(3):
            h = nn.swish(nn.Dense(self.num_hid)(h))
        return nn.Dense(self.num_out)(h)

class Qmodel(nn.Module):
    num_hid: int
    num_out: int

    @nn.compact
    def __call__(self, t, x_0, x_1):
        h = jnp.concatenate([t, x_0, x_1, t < 0.5], axis=-1)
        for _ in range(3):
            h = nn.swish(nn.Dense(self.num_hid)(h))
        h = nn.Dense(self.num_out)(h)
        return (1.0 - t) * x_0 + t * (x_1 + h)

Listing 2: Model architectures used for toy 2D dynamic subset alignment experiments.

Static subset alignment results for $c \in \{1, 2, 4, 8\}$

Figure 10. Static subset alignment results for c∈{1,2,4,8}c \in\{1, 2, 4, 8\}. (a) Transport from 1D uniform distribution to semicircular density shown in Figure 7b. (b) Transport from 1D uniform distribution to a uniform distribution shown in Figure 8b. Red curves indicate the η(x)\eta(x).

1 import jax
2 from jax import numpy as jnp
3 from flax import linen as nn
4 import math
5 from ml_collections import config_dict

7 class Tmodel(nn.Module):
8     num_hid: int
9     num_out: int

11     @nn.compact
12     def __call__(self, inputs):
13         h = inputs
14         for _ in range(4):
15             h = nn.swish(nn.Dense(self.num_hid)(h))
16         h = nn.Dense(self.num_out)(h)
17         return inputs + h

19 class DualNET(nn.Module):
20     num_hid: int
21     num_out: int

23     @nn.compact
24     def __call__(self, x):
25         h = x
26         for _ in range(4):
27             h = nn.swish(nn.Dense(self.num_hid)(h))
28         return nn.Dense(self.num_out)(h)

Listing 3: Model architectures used for 1D and 2D static subset alignment toy experiments.

use Adam [66] with a warmup–cosine learning-rate schedule [80], a peak learning rate of 1×10−31 \times10^{-3}, and 500 warmup steps. The cosine schedule decays over 20,000 steps to an end learning rate of 1×10−51 \times10^{-5}. Training is stopped after 20,000 steps, once the classifier exceeds 90% overall validation accuracy and 99% accuracy on digits. Confusion matrices are reported in Appendix § 12.

MNIST-EMNIST Translation Models

For dynamic subset selection, following Neklyudov et al. [88], we parameterize the vector field φt\varphi_t with a U-Net equipped with DDPM-style time embeddings (Song & Ermon [113]). As in action matching (Neklyudov et al. [88]), φt\varphi_t returns a scalar via

φt(x)=⟨UNet⁡(x),x⟩.(58)\varphi_t(x) = \langle\operatorname{UNet}(x), x\rangle. \tag*{(58)}

Likewise, QtQ_t, which parameterizes ρt\rho_t, is implemented as a U-Net with time embeddings. For both U-Nets, we use the same architectural settings: nonlinearity = swish, nf = 64, ch_mult = (1, 2, 2, 2), num_res_blocks = 2, and attn_resolutions = (16, 8). We use the AdamW optimizer with learning-rate scheduling for 50,000 training iterations. The optimizer parameters are β=(0,0.999)\beta= (0, 0.999), weight decay = 0.1, and dropout = 0.1. We additionally employ exponential moving averages (EMA) during training with an EMA rate of 0.999. These settings are very similar to those used in rectified flow matching and action matching [77, 88]. All input images in neural subset alignment are centered at [0.5, 0.5] and resized to 32×32×132 \times32 \times1. The learning rate is linearly increased from zero to its maximum value during the first 5,000 iterations and is then kept constant. The maximum learning rates are 2×10−42 \times10^{-4} and 1×10−41 \times10^{-4} for φt\varphi_t and QtQ_t, respectively. Additionally, gradients are clipped element-wise to the interval [−1,1][-1, 1]. Algorithm 1 is employed for 50,000 training iterations with nφ=2n_\varphi= 2 and nρ=1n_\rho= 1. A batch size of 128 is used to train the dynamic models for MNIST-EMNIST translation.

For static domain translation, the transport network TT is a U-Net Ronneberger et al. [102] with base factor 64, and the critic network η\eta is a ResNet; we use the same choices as Gazdieva et al. [40] for inexact extremal optimal transport. All inputs to the static models are centered at [0.5, 0.5] and resized to 32×32×132 \times32 \times1. We train TT and η\eta with Adam Kingma & Ba [66] using an initial learning rate of 1×10−41 \times10^{-4}, which is halved after 10,000+5,000c10{,}000 + 5{,}000c, 20,000+5,000c20{,}000 + 5{,}000c, 30,000+5,000c30{,}000 + 5{,}000c, 40,000+5,000c40{,}000 + 5{,}000c, and 70,000+5,000c70{,}000 + 5{,}000c steps. We run Algorithm 2 for 50,000 learning iterations with nT=10n_T = 10 and nη=1n_\eta= 1. A batch size of 128 is used for static MNIST–EMNIST translation.

Models for PU-Learning Using Subset Alignment

The architectures used for PU learning with dynamic subset alignment are given in Listing 4, where Smodel and Qmodel parameterize φt\varphi_t and ρt\rho_t, respectively. For both models, we use num_hid = 1024 and embedding_dim = 64. The output dimension is num_out = 1 for Smodel, while for Qmodel it equals the dataset dimension (Table 7). We run Algorithm 1 for 50,000 training iterations with nφ=2n_\varphi= 2 and nρ=1n_\rho= 1. Both models are optimized with AdamW using learning rate 5×10−65 \times10^{-6}, β1=0.9\beta_1 = 0.9, and ϵ=10−8\epsilon= 10^{-8}. We use 10,000 warmup steps and gradient clipping with maximum norm 0.1, and we maintain an exponential moving average (EMA) with rate 0.999. Batch sizes are dataset-dependent and are reported in Table 7.

The architectures used for PU learning with static subset alignment are given in Listing 5, where Tmodel and DualNET parameterize TT and η\eta, respectively. For both models, we use num_hid = 1024 and embedding_dim = 128. The output dimension for Tmodel matches the dataset dimension, while DualNET uses num_out = 1. We run Algorithm 2 for 20,000 training iterations with nT=10n_T = 10 and nη=1n_\eta= 1. Both models are optimized with learning rate 10−410^{-4} and use EMA with rate 0.999. Batch sizes are dataset-dependent and are reported in Table 7. For all PU-learning experiments, we fix c=1π+c = \frac{1}{\pi_+}.

Datasetdata-set sizedimensionπ\pibatch size
Abalone417780.1620
Banknote137240.4410
Breast-w69990.3410
Diabetes76880.356
Haberman30630.266
Heart270130.446
Ionosphere351340.646
Isolet77976170.044
Jm110885210.1920
Kc12109210.1520
Madelon26005000.520
Musk65981660.1520
Segment2310190.1420
Semeion15932560.14
Sonar208600.534
Spambase4601570.3920
Vehicle846180.266
Waveform5000400.3420
Wdbc569300.376
Yeast148480.3110

Table 7. UCI datasets for PU learning, along with the total number of data points (n), dimension (dim), positive prior (π\pi) and batch sizes employed in training the corresponding dynamic models.

Image-to-Image Translation on FFHQ

For dynamic subset alignment, the architectures for the networks parameterizing φt\varphi_t and ρt\rho_t are given in Listing 6, where Smodel and Qmodel parameterize φt\varphi_t and ρt\rho_t respectively. Both networks use t_embed_dim = 128 and num_hid = 1024, with num_out = 1 for the φt\varphi_t network and num_out = 512 for the ρt\rho_t network (matching the FFHQ latent dimensionality). We use AdamW with a fixed learning rate of 1×10−51 \times10^{-5}, gradient clipping with maximum norm 1.0, EMA rate 0.999, and batch size 128. We run Algorithm 1 for 500,000 learning iterations with nφ=2n_\varphi= 2 and nρ=1n_\rho= 1.

For static subset alignment, the neural network architectures TransportNET and DualNET given in Listing 7 parameterize TT and η\eta, respectively. Both networks use hidden_dim = 1024. TransportNET maps 512-dimensional FFHQ latents to 512-dimensional outputs, while DualNET returns a scalar. We use AdamW with a fixed learning rate of 1×10−51 \times10^{-5}, weight decay 10−810^{-8}, EMA rate 0.999, and batch size 1,024. We run Algorithm 2 for 50,000 learning iterations with nT=5n_T = 5 and nη=1n_\eta= 1. The same architectures are used for the UOTM experiments in Table 4.

Melanoma Experiment

Unlike the other experiments in this paper, the melanoma setting involves multiple intermediate marginals that must be matched. Because we want to preserve all marginals except the terminal one, we generalize equation (8) to the multi-marginal setting. Assuming we can sample from the marginals at times tkt_k and tk+1t_{k+1}, for τ∈[tk,tk+1]\tau\in[t_k, t_{k+1}] we define τ=t−tktk+1−tk\tau= \frac{t-t_k}{t_{k+1}-t_k} and use the interpolant

xt=(1−τ)xtk+τxtk+1+wk(τ)Q(t,xtk,xtk+1;θρ),(59)x_t = (1-\tau)x_{t_k} + \tau x_{t_{k+1}} + w_k(\tau)Q(t,x_{t_k},x_{t_{k+1}};\theta_\rho), \tag*{(59)}
import jax
from jax import numpy as jnp
from flax import linen as nn
import math
from ml_collections import config_dict


def get_timestep_embedding(timesteps, embedding_dim: int, max_positions: float = 10_000.0):
    t = jnp.asarray(timesteps)
    t = t.reshape((-1,)) if t.ndim == 1 else t.reshape((t.shape[0], -1))[:, 0]
    t = t.astype(jnp.float32)


    half = embedding_dim // 2
    if half < 1:
        return jnp.zeros((t.shape[0], embedding_dim), dtype=t.dtype)


    freqs = jnp.exp(-jnp.log(jnp.array(max_positions,dtype=t.dtype))
                    * jnp.arange(half, dtype=t.dtype) / (half - 1 + 1e-8))


    args = t[:, None] * freqs[None, :]          # (B, half)
    emb = jnp.concatenate([jnp.cos(args), jnp.sin(args)], axis=-1)  # (B, 2*half)


    if embedding_dim % 2 == 1:
        emb = jnp.pad(emb, ((0, 0), (0, 1)))


    return emb[:, :embedding_dim]


class Smodel(nn.Module):
    config: config_dict.ConfigDict

    @nn.compact
    def __call__(self, t, x, *, train: bool = True):
        config = self.config
        time_emb_dim = config.embedding_dim
        num_hid = config.num_hid
        num_out = config.num_out
        x = jnp.asarray(x)
        if x.ndim == 1:                    # (D,) -> (1,D)
            x = x[None, :]

        # make t broadcastable to x batch: (B,1)
        t = jnp.asarray(t)
        t = jnp.broadcast_to(t, x.shape[:-1] + (1,))

        temb = get_timestep_embedding(t, time_emb_dim)  # should return (B, time_emb_dim)
        temb = nn.swish(nn.Dense(num_hid)(temb))
        temb = nn.Dense(num_hid)(temb)                   # (B, num_hid)

        h = jnp.concatenate([t, x], axis=-1)
        h = nn.swish(nn.Dense(num_hid)(h))
        h = h + temb

        for _ in range(3):
            h = nn.swish(nn.Dense(num_hid)(h))

        return nn.Dense(num_out)(h)


class Qmodel(nn.Module):
    config: config_dict.ConfigDict

    @nn.compact
    def __call__(self, t, x_0, x_1, *, train: bool = True):
        config = self.config
        time_emb_dim = config.embedding_dim
        num_hid = config.num_hid
        num_out = config.num_out

        x_0 = jnp.asarray(x_0)
        x_1 = jnp.asarray(x_1)
        if x_0.ndim == 1:                    # (D,) -> (1,D)
            x_0 = x_0[None, :]
            x_1 = x_1[None, :]

        # broadcast t to (B,1) to match x batch
        t = jnp.asarray(t)
        t = jnp.broadcast_to(t, x_0.shape[:-1] + (1,))  # (B,1)

        # time embedding -> project to hidden width
        temb = get_timestep_embedding(t, time_emb_dim)  # (B, time_emb_dim)
        temb = nn.swish(nn.Dense(num_hid)(temb))
        temb = nn.Dense(num_hid)(temb)                  # (B, num_hid)

        heaviside_step = (t < 0.5).astype(x_0.dtype)    # (B,1)
        h = jnp.concatenate([t, x_0, x_1, heaviside_step], axis=-1)

        h = nn.swish(nn.Dense(num_hid)(h))
        h = h + temb

        for _ in range(3):
            h = nn.swish(nn.Dense(num_hid)(h))

        h = nn.Dense(num_out)(h)

        x_t = (1.0 - t) * x_0 + t * (x_1 + h)
        return x_t

Listing 4: Model architectures for PU-Learning for dynamic subset alignment

import jax
from jax import numpy as jnp
from flax import linen as nn
import math
from ml_collections import config_dict

class etamodel(nn.Module):
    num_hid: int
    num_out: int
    @nn.compact
    def __call__(self, x):
        h = x
        for _ in range(3):
            h = nn.swish(nn.Dense(self.num_hid)(h))
        return nn.Dense(self.num_out)(h)

class Tmodel(nn.Module):
    num_hid: int
    num_out: int
    @nn.compact
    def __call__(self, x):
        h = x
        for _ in range(4):
            h = nn.swish(nn.Dense(self.num_hid)(h))
        h = nn.Dense(self.num_out)(h)
        residual = nn.Dense(self.num_out)(x)
        return h + residual

Listing 5: Model architectures for PU learning with static subset alignment

import jax
from jax import numpy as jnp
from flax import linen as nn
import math

def get_timestep_embedding(t, dim, max_positions=10000):
    t = jnp.reshape(t, (-1,))
    half = dim // 2
    scale = jnp.exp(
        jnp.arange(half, dtype=jnp.float32)
        * (-math.log(max_positions) / (half - 1))
    )
    emb = t[:, None] * scale[None, :]
    emb = jnp.concatenate([jnp.sin(emb), jnp.cos(emb)], axis=1)
    return jnp.pad(emb, [(0, 0), (0, dim % 2)])

class Smodel(nn.Module):
    num_hid: int
    num_out: int
    t_embed_dim: int

    @nn.compact
    def __call__(self, t, x):
        if t.ndim == 0:
            t = jnp.broadcast_to(t, (x.shape[0], 1, 1))

        e = get_timestep_embedding(t, self.t_embed_dim)
        e = jnp.reshape(e, x.shape[:-1] + (self.t_embed_dim,))

        h = jnp.concatenate([e, x], axis=-1)
        for i in range(4):
            h = nn.swish(nn.Dense(self.num_hid)(h))
            if i < 3:
                h = jnp.concatenate([e, h], axis=-1)

        return nn.Dense(self.num_out)(h)

class Qmodel(nn.Module):
    num_hid: int
    num_out: int
    t_embed_dim: int

    @nn.compact
    def __call__(self, t, x_0, x_1):
        e = get_timestep_embedding(t.squeeze(), self.t_embed_dim)
        h = jnp.concatenate([e, x_0, x_1, t < 0.5], axis=-1)

        for i in range(4):
            h = nn.swish(nn.Dense(self.num_hid)(h))
            if i < 3:
                h = jnp.concatenate([e, h], axis=-1)

        h = nn.Dense(self.num_out)(h)
        return (1.0 - t) * x_0 + t * (x_1 + h)

Listing 6: Model architectures for FFHQ dynamic subset alignment.

Model architectures for FFHQ static subset alignment

Figure 7 (PDF p. 30). Model architectures for FFHQ static subset alignment.

where

wk(τ)={1−τ2−(1−τ)2,k<K−1,1−(1−τ)2,k=K−1,(60)w_k(\tau) = \begin{cases} 1-\tau^2-(1-\tau)^2, & k < K-1, \\ 1-(1-\tau)^2, & k = K-1, \end{cases} \tag*{(60)}

where QQ is a neural network shared across segments. Since wkw_k vanishes at τ=0\tau= 0 for all kk and at τ=1\tau= 1 for k<K−1k < K-1, the interpolant matches μ0,…,μK−1\mu_0,\ldots,\mu_{K-1} exactly, while wK−1(1)=1w_{K-1}(1) = 1 leaves the terminal marginal unconstrained: xtK↦xtK+Q(tK,xtK−1,xtK;θρ)x_{t_K} \mapsto x_{t_K} + Q(t_K,x_{t_{K-1}},x_{t_K};\theta_\rho). For K=1K = 1, this reduces to (1−t)x0+tx1+t(2−t)Q(1-t)x_0 + tx_1 + t(2-t)Q, which, like (1−t)x0+t(x1+Q)(1-t)x_0 + t(x_1+Q), fixes μ0\mu_0 while relaxing the terminal marginal. In addition, we use the Wasserstein gradient-flow updates of Neklyudov et al. [90] to improve stability and convergence.

The Algorithm 3 summarizes the training procedure for the melanoma experiment. The Neural Network Architectures employed in Melanoma experiments are provided in Listing 8. We should emphasize that the interpolation equation 59 has been implemented as part of interpolant neural network architecture, but for clear description of learning procedure, we included interpolation equation a part of learning algorithm.

We use the first 50 principal components of the melanoma data [18], centered and divided by the largest per-dimension standard deviation of the training set. The three observed populations are ordered in reverse time, μ0\mu_0 (48h), μ1\mu_1 (24h) and μ2\mu_2 (8h), at normalised times t0=0t_0 = 0, t1=0.6t_1 = 0.6 and t2=1t_2 = 1 (K=2K = 2), so that the sub-setting constraint acts on the μK=2\mu_{K=2}. The potential φt(x;θφ)=⟨f(t,x;θφ),x⟩\varphi_t(x;\theta_\varphi) = \langle f(t,x;\theta_\varphi),x\rangle is parameterized by a Smodel in Listing 8 with $n_f = 512 and activations act⁡=Swish\operatorname{act}=\mathrm{Swish}; the time tt is mapped by a two-layer embedding network of width 512 and added to the input of each of the three inner hidden layers. The interpolant network Qt(xtk,xtk+1;θρ)Q_t(x_{t_k},x_{t_{k+1}};\theta_\rho) is parameterized by Qmodel in Listing 8, which takes concatenation (t,xtk,xtk+1)(t,x_{t_k},x_{t_{k+1}}) as input and has a single hidden layer of width 512 with swish activation and a linear output in R50\mathbb{R}^{50}. Both networks are trained with AdamW (β1=0.9, β2=0.999(\beta_1=0.9,\ \beta_2=0.999, weight decay 10−4)10^{-4}) at learning rate 2×10−42\times10^{-4} with a linear warm-up over the first 5,000 iterations and element-wise gradient clipping at 1, using batch size N=512N=512, nϕ=nρ=1n_\phi=n_\rho=1 update per iteration, M=10M=10 Wasserstein gradient steps with step size η=10−2\eta=10^{-2}, and 200,000 iterations; the times tit^i are drawn from a low-discrepancy sequence on [0,1][0,1]. We report results for sub-setting constants c∈{1.0,2.0,4.0,8.0}c\in\{1.0,2.0,4.0,8.0\} in table 5. For evaluation we use an exponential moving average of the parameters with rate 0.999, and transport cells with the learned velocity field ∇φt\nabla\varphi_t by integrating the ODE from tKt_K back to t0t_0 with the explicit Euler scheme and step size 1×10−31\times10^{-3}.

Algorithm 3: Modified-Dynamic-Neural-SS: Modified Learning Algorithm for Dynamic Subset Alignment on Melanoma

Require: Samples from distributions {μk}k=0K\{\mu_k\}_{k=0}^{K} observed at times {tk}k=0K\{t_k\}_{k=0}^{K}, time-dependent neural network φt(⋅;θφ)\varphi_t(\cdot;\theta_\varphi), interpolant network Qt(⋅,⋅;θρ)Q_t(\cdot,\cdot;\theta_\rho), sub-setting constant c≥1c\geq1, batch size NN, number of updates nφn_\varphi and nρn_\rho, gradient clipping parameter ϵ\epsilon, number of Wasserstein gradient steps MM with step size η\eta, gradient-based optimizers optim⁡φ\operatorname{optim}_\varphi and optim⁡ρ\operatorname{optim}_\rho.

Ensure: Sample-based neural estimate for φt(⋅;θφ)\varphi_t(\cdot;\theta_\varphi)

  1. for learning iteration ℓ=0,1,…\ell=0,1,\ldots do

  2. for u=1,…,nφu=1,\ldots,n_\varphi do

  3. Sample {xtki}i=1N∼μk\{x_{t_k}^i\}_{i=1}^{N}\sim\mu_k for k=0,…,Kk=0,\ldots,K, and {ti}i=1N∼Uniform⁡([0,1])\{t^i\}_{i=1}^{N}\sim\operatorname{Uniform}([0,1]).

    {xtii←(1−τi)xtki+τixtk+1i+wk(τi)Qti(xtki,xtk+1i;θρ)}i=1N,\left\{x_{t^i}^i\leftarrow(1-\tau^i)x_{t_k}^i+\tau^i x_{t_{k+1}}^i+w_{k}(\tau^i)Q_{t^i}(x_{t_k}^i,x_{t_{k+1}}^i;\theta_\rho)\right\}_{i=1}^{N},

  1. where ti∈[tk,tk+1]t^i\in[t_k,t_{k+1}], τi=ti−tktk+1−tk\tau^i=\frac{t^i-t_k}{t_{k+1}-t_k},

    wk(τ)=1−τ2−(1−τ)2w_k(\tau)=1-\tau^2-(1-\tau)^2 for k<K−1k<K-1, and wK−1(τ)=1−(1−τ)2w_{K-1}(\tau)=1-(1-\tau)^2

  1. for j=1,…,Mj=1,\ldots,M do

  2. {xtii←xtii+ηwk(τi)clip⁡(∇x[∂∂tφti(xtii;θφ)+∥∇φti(xtii;θφ)∥22],−ϵ,ϵ)}i=1N\left\{x_{t^i}^i\leftarrow x_{t^i}^i+\eta w_k(\tau^i)\operatorname{clip}\left(\nabla_x\left[\frac{\partial}{\partial t}\varphi_{t^i}(x_{t^i}^i;\theta_\varphi)+\frac{\lVert\nabla\varphi_{t^i}(x_{t^i}^i;\theta_\varphi)\rVert^2}{2}\right],-\epsilon,\epsilon\right)\right\}_{i=1}^{N}

  3. end for

  4. grad⁡θφ←∇θφ1N∑i=1N[∂∂tφti(xtii;θφ)+∥∇φti(xtii;θφ)∥22+φt0(xt0i;θφ)+c⋅max⁡(0,−φtK(xtKi;θφ))]\operatorname{grad}_{\theta_\varphi}\leftarrow\nabla_{\theta_\varphi}\frac{1}{N}\sum_{i=1}^{N}\left[\frac{\partial}{\partial t}\varphi_{t^i}(x_{t^i}^i;\theta_\varphi)+\frac{\lVert\nabla\varphi_{t^i}(x_{t^i}^i;\theta_\varphi)\rVert^2}{2}+\varphi_{t_0}(x_{t_0}^i;\theta_\varphi)+c\cdot\max\left(0,-\varphi_{t_K}(x_{t_K}^i;\theta_\varphi)\right)\right]

  5. Using optim⁡φ\operatorname{optim}_\varphi and grad⁡θφ\operatorname{grad}_{\theta_\varphi} to update θφ\theta_\varphi via gradient descent.

  6. end for

  7. for u=1,…,nρu=1,\ldots,n_\rho do

  8. Sample {xtki}i=1N∼μk\{x_{t_k}^i\}_{i=1}^{N}\sim\mu_k for k=0,…,Kk=0,\ldots,K, and {ti}i=1N∼Uniform⁡([0,1])\{t^i\}_{i=1}^{N}\sim\operatorname{Uniform}([0,1]).

  9. {xtii(θρ)←(1−τi)xtki+τixtk+1i+wk(τi)Qti(xtki,xtk+1i;θρ)}i=1N\left\{x_{t^i}^i(\theta_\rho)\leftarrow(1-\tau^i)x_{t_k}^i+\tau^i x_{t_{k+1}}^i+w_k(\tau^i)Q_{t^i}(x_{t_k}^i,x_{t_{k+1}}^i;\theta_\rho)\right\}_{i=1}^{N}

  10. grad⁡θρ←∇θρ1N∑i=1N[∂∂tφti(xtii(θρ);θφ)+∥∇φti(xtii(θρ);θφ)∥22]\operatorname{grad}_{\theta_\rho}\leftarrow\nabla_{\theta_\rho}\frac{1}{N}\sum_{i=1}^{N}\left[\frac{\partial}{\partial t}\varphi_{t^i}(x_{t^i}^i(\theta_\rho);\theta_\varphi)+\frac{\lVert\nabla\varphi_{t^i}(x_{t^i}^i(\theta_\rho);\theta_\varphi)\rVert^2}{2}\right]

  11. Update θρ\theta_\rho using optim⁡ρ\operatorname{optim}_\rho and grad⁡θρ\operatorname{grad}_{\theta_\rho} via gradient ascent.

  12. end for

  13. end for

Confusion matrices for MNIST to EMNIST domain translation

import jax
from jax import numpy as jnp
from flax import linen as nn
import math

class Smodel(nn.Module):
    config: ml_collections.ConfigDict

    @nn.compact
    def __call__(self, t: jnp.ndarray, x: jnp.ndarray, train: bool):
        config = self.config
        act = get_act(config)
        nf = config.nf

        if config.embed_time:
            temb = t
            temb = nn.Dense(nf)(temb)
            temb = nn.Dense(nf)(act(temb))
            h = x
        else:
            h = jnp.hstack([x, t])

        h = act(nn.Dense(nf)(h))
        for _ in range(config.n_layers):
            if config.embed_time:
                h += temb
            h = nn.Dropout(config.dropout)(h, deterministic=not train)
            if config.skip:
                h = act(nn.Dense(nf)(h)) + h
            else:
                h = act(nn.Dense(nf)(h))
        h = act(nn.Dense(nf)(h))
        h = nn.Dense(config.input_dim)(h)
        return (h*x).sum(1, keepdims=True)

class Qmodel(nn.Module):
    config: ml_collections.ConfigDict

    @nn.compact
    def __call__(self,t:jnp.ndarray,batch:jnp.ndarray,train:bool):
        config=self.config
        act=get_act(config)
        nf=config.nf

        timesteps,x=batch
        B=x.shape[0]
        n_marginals=timesteps.shape[1]

        t_reshaped=t.reshape(-1,1,1)
        t_right=(timesteps<t_reshaped).sum(1)
        t_right=jnp.fmax(t_right,jnp.ones_like(t_right).astype(int))
        t_left=t_right-1

        batch_idx=jnp.arange(B)
        x_left=x[batch_idx,t_left.ravel(),:]
        x_right=x[batch_idx,t_right.ravel(),:]
        t_0=timesteps[batch_idx,t_left.ravel(),:]
        t_1=timesteps[batch_idx,t_right.ravel(),:]

        temb=layers.get_timestep_embedding(t.ravel(),nf)
        temb=nn.Dense(nf)(temb)
        temb=nn.Dense(nf)(act(temb))

        h=jnp.concatenate([x_left,x_right],axis=1)
        h=jnp.hstack([t,h])

        if config.indicator:
            h=jnp.hstack([h,t<0.5*(t_0+t_1)])

        h=act(nn.Dense(nf)(h))

        for _ in range(config.n_layers):
            h=nn.Dropout(config.dropout)(h,deterministic=not train)
            h=act(nn.Dense(nf)(h))+h if config.skip else act(nn.Dense(nf)(h))

        h=nn.Dense(config.input_dim)(h)

        alpha=(t-t_0)/(t_1-t_0)
        beta=(t_1-t)/(t_1-t_0)

        w_inner=1.0-alpha**2-beta**2
        w_last=1.0-beta**2

        w=jnp.where(t_right==(n_marginals-1),w_last,w_inner)

        if config.get('pin_last',False):
            w=w_inner

        return beta*x_left+alpha*x_right+w*h

Listing 8: Model architectures for Melanoma Experiment.

Normalized and unnormalized confusion matrices for the EMNIST classifier

Figure 11. (a) Normalized and (b) unnormalized confusion matrices for the EMNIST classifier discussed in § 4.2.

Confusion matrices for MNIST to EMNIST domain translation using static subset selection

Figure 12. Confusion matrices for MNIST→\toEMNIST domain translation using static subset selection. Accuracy is computed as the ratio between the trace and the sum of all entries in the confusion matrices.

Confusion matrices for MNIST→EMNIST domain translation using dynamic subset selection

Figure 13. Confusion matrices for MNIST→EMNIST domain translation using dynamic subset selection.

t-SNE embeddings of UCI datasets

t-SNE embeddings for UCI datasets

Figure 14. The t-SNE embeddings for (a) Abalone, (b) Banknote, (c) Breast-w, (d) Diabetes, (e) Haberman, (f) Heart, (g) Ionosphere, (h) Isolet, (i) Kc1, (j) Madelon, (k) Musk, (l) Semeion, (m) Sonar, (n) Spambase, (o) Vehicle, (p) Waveform, (q) Wdbc, (r) Yeast, (s) Jm1, and (t) Segment datasets.

Results from FFHQ

Static subset selection

(a)

Dynamic subset selection

(b)

Figure 15. FFHQ young→\toold translation using (a) static and (b) dynamic subset selection. Dynamic subset selection is evaluated using Euler integration with 100 steps.

Static subset selection

(a)

Dynamic subset selection

(b)

Figure 16. FFHQ woman→\toman translation using (a) static and (b) dynamic subset selection. Dynamic subset selection is evaluated using Euler integration with 100 steps.

Static subset selection

(a)

Dynamic subset selection

(b)

Figure 17. FFHQ man→\towoman translation using (a) static and (b) dynamic subset selection. Dynamic subset selection is evaluated using Euler integration with 100 steps.

References

  1. [1]Michael S. Albergo and Eric Vanden-Eijnden. Building Normalizing Flows with Stochastic Interpolants. In The Eleventh International Conference on Learning Representations, 2023. [paper].arxiv.org/abs/2209.15571
  2. [2]Michael S. Albergo, Nicholas M. Boffi, and Eric Vanden-Eijnden. Stochastic Interpolants: A Unifying Framework for Flows and Diffusions. Journal of Machine Learning Research, 26(209):1–80, 2025. [paper].
  3. [3]Fabian Altekrüger, Johannes Hertrich, and Gabriele Steidl. Neural Wasserstein Gradient Flows for Discrepancies with Riesz Kernels. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 664–690. PMLR, 23–29 Jul 2023. [paper].arxiv.org/abs/2301.11624
  4. [4]David Alvarez-Melis, Yair Schiff, and Youssef Mroueh. Optimizing Functionals on the Space of Probabilities with Input Convex Neural Networks. Transactions on Machine Learning Research, 2022. ISSN 2835-8856. [paper].
  5. [6]Brandon Amos, Lei Xu, and J Zico Kolter. Input Convex Neural Networks. In International Conference on Machine Learning, pp. 146–155. PMLR, PMLR, 2017. [paper].arxiv.org/abs/1609.07152
  6. [7]Brandon Amos, Giulia Luise, Samuel Cohen, and Ievgen Redko. Meta Optimal Transport. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 791–813. PMLR, 23–29 Jul 2023. [paper].
  7. [8]Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein Generative Adversarial Networks. In International Conference on Machine Learning, pp. 214–223. PMLR, 2017. [paper].
  8. [10]Lazar Atanackovic, Xi Zhang, Brandon Amos, Mathieu Blanchette, Leo J Lee, Yoshua Bengio, Alexander Tong, and Kirill Neklyudov. Meta Flow Matching: Integrating Vector Fields on the Wasserstein Manifold. In The Thirteenth International Conference on Learning Representations, 2025. [paper].arxiv.org/abs/2408.14608
  9. [11]Julio Backhoff-Veraguas, Mathias Beiglböck, and Gudmun Pammer. Existence, duality, and cyclical monotonicity for weak transport costs. Calculus of Variations and Partial Differential Equations, 58(6):203, 2019. [paper].
  10. [12]Jessa Bekker and Jesse Davis. Learning from positive and unlabeled data: a survey. Machine Learning, 109:719–760, 2020. [paper].
  11. [13]Jean-David Benamou and Yann Brenier. A computational fluid mechanics solution to the Monge-Kantorovich mass transfer problem. Numerische Mathematik, 84(3):375–393, 2000. [paper].DOI
  12. [14]Valentin De Bortoli, Iryna Korshunova, Andriy Mnih, and Arnaud Doucet. Schrödinger Bridge Flow for Unpaired Data Translation. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. [paper].arxiv.org/abs/2409.09347
  13. [15]Yann Brenier. Polar factorization and monotone rearrangement of vector-valued functions. Communications on Pure and Applied Mathematics, 44(4):375–417, 1991. [paper].DOI
  14. [16]Charlotte Bunne, Andreas Krause, and marco cuturi. Supervised Training of Conditional Monge Maps. In Advances in Neural Information Processing Systems, 2022a. [paper].arxiv.org/abs/2206.14262
  15. [17]Charlotte Bunne, Laetitia Papaxanthos, Andreas Krause, and Marco Cuturi. Proximal Optimal Transport Modeling of Population Dynamics. In Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, volume 151 of Proceedings of Machine Learning Research, pp. 6511–6528. PMLR, 28–30 Mar 2022. [paper].arxiv.org/abs/2106.06345
  16. [18]Charlotte Bunne, Stefan G Stark, Gabriele Gut, Jacobo Sarabia Del Castillo, Mitch Levesque, Kjong-Van Lehmann, Lucas Pelkmans, Andreas Krause, and Gunnar Rätsch. Learning single-cell perturbation responses using neural optimal transport. Nature Methods, 20(11):1759–1768, 2023. [paper].
  17. [19]Brian Chao. Anime face dataset: a collection of high-quality anime faces., 2019. [dataset].
  18. [20]Laetitia Chapel, Mokhtar Z Alaya, and Gilles Gasso. Partial Optimal Transport with applications on Positive-Unlabeled Learning. Advances in Neural Information Processing Systems, 33:2903–2913, 2020. [paper].arxiv.org/abs/2002.08276
  19. [21]Ricky TQ Chen and Yaron Lipman. Flow Matching on General Geometries. In The Twelfth International Conference on Learning Representations, 2024. [paper].
  20. [22]Tianrong Chen, Guan-Hорng Liu, and Evangelos Theodorou. Likelihood training of schrödinger bridge using forward-backward SDEs theory. In International Conference on Learning Representations, 2022. [paper].arxiv.org/abs/2110.11291
  21. [23]Xiuyuan Cheng, Jianfeng Lu, Yixin Tan, and Yao Xie. Convergence of Flow-Based Generative Models via Proximal Gradient Descent in Wasserstein Space. IEEE Transactions on Information Theory, 70(11):8087–8106, 2024. [paper].DOI
  22. [24]Lenaic Chizat, Gabriel Peyré, Bernhard Schmitzer, and François-Xavier Vialard. An Interpolating Distance Between Optimal Transport and Fisher–Rao Metrics. Foundations of Computational Mathematics, 18:1–44, 2018a. [paper].
  23. [26]Jaemoo Choi, Jaewoong Choi, and Myungjoo Kang. Generative Modeling through the Semi-dual Formulation of Unbalanced Optimal Transport. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. [paper].arxiv.org/abs/2305.14777
  24. [27]Jaemoo Choi, Jaewoong Choi, and Myungjoo Kang. Analyzing and Improving Optimal-Transport-based Adversarial Networks. In The Twelfth International Conference on Learning Representations, 2024a. [paper].arxiv.org/abs/2310.02611
  25. [28]Jaemoo Choi, Jaewoong Choi, and Myungjoo Kang. Scalable Wasserstein Gradient Flow for Generative Modeling through Unbalanced Optimal Transport. In Forty-first International Conference on Machine Learning, 2024b. [paper].arxiv.org/abs/2402.05443
  26. [29]Gregory Cohen, Saeed Afshar, Jonathan Tapson, and André van Schaik. EMNIST: Extending MNIST to handwritten letters. In 2017 International Joint Conference on Neural Networks (IJCNN), pp. 2921–2926, 2017. [paper].
  27. [30]Li Deng. The MNIST Database of Handwritten Digit Images for Machine Learning Research. IEEE Signal Processing Magazine, 29(6):141–142, 2012. [paper].
  28. [31]Luca Eyring, Dominik Klein, Théo Uscidda, Giovanni Palla, Niki Kilbertus, Zeynep Akata, and Fabian J Theis. Unbalancedness in Neural Monge Maps Improves Unpaired Domain Translation. In The Twelfth International Conference on Learning Representations, 2024. [paper].arxiv.org/abs/2311.15100
  29. [32]Jiaojiao Fan, Shu Liu, Shaojun Ma, Yongxin Chen, and Hao-Min Zhou. Scalable Computation of Monge Maps with General Costs. In ICLR Workshop on Deep Generative Models for Highly Structured Data, 2022a. [paper].
  30. [33]Jiaojiao Fan, Qinsheng Zhang, Amirhossein Taghvaei, and Yongxin Chen. Variational Wasserstein gradient flow. In proceedings of International Conference on Machine Learning, 2022b. [paper].
  31. [34]Jiaojiao Fan, Shu Liu, Shaojun Ma, Hao-Min Zhou, and Yongxin Chen. Neural Monge Map estimation and its applications. Transactions on Machine Learning Research, 2023. [paper].
  32. [35]Jean Feydy, Thibault Séjourné, François-Xavier Vialard, Shun-ichi Amari, Alain Trouvé, and Gabriel Peyré. Interpolating between Optimal Transport and MMD using Sinkhorn Divergences. In The 22nd International Conference on Artificial Intelligence and Statistics, pp. 2681–2690. PMLR, 2019. [paper].arxiv.org/abs/1810.08278
  33. [36]Alessio Figalli and Federico Glaudo. An invitation to optimal transport, Wasserstein distances, and gradient flows: Second Edition. European Mathematical Society, 2023. [book].DOI
  34. [37]Nicolas Fournier and Arnaud Guillin. On the rate of convergence in Wasserstein distance of the empirical measure. Probability Theory and Related Fields, 162(3):707–738, 2015. [paper].
  35. [38]Charlie Frogner, Chiyuan Zhang, Hossein Mobahi, Mauricio Araya, and Tomaso Poggio. Learning with a wasserstein loss. In C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 28. Curran Associates, Inc., 2015. [paper].arxiv.org/abs/1506.05439
  36. [39]Wilfrid Gangbo and Robert J McCann. The geometry of optimal transportation. Acta Mathematica, 177:113–161, 1996. [paper].
  37. [40]Milena Gazdieva, Alexander Korotin, Daniil Selikhanovych, and Evgeny Burnaev. Extremal Domain Translation with Neural Optimal Transport. Advances in Neural Information Processing Systems, 36:40381–40413, 2023. [paper].arxiv.org/abs/2301.12874
  38. [41]Milena Gazdieva, Arip Asadulaev, Evgeny Burnaev, and Alexander Korotin. Light Unbalanced Optimal Transport. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. [paper].
  39. [42]Aude Genevay, Marco Cuturi, Gabriel Peyré, and Francis Bach. Stochastic optimization for large-scale optimal transport. In Advances in Neural Information Processing Systems, volume 29, 2016. [paper].arxiv.org/abs/1605.08527
  40. [43]Aude Genevay, Gabriel Peyré, and Marco Cuturi. Learning Generative Models with Sinkhorn Divergences. In International Conference on Artificial Intelligence and Statistics, pp. 1608–1617. PMLR, 2018. [paper].arxiv.org/abs/1706.00292
  41. [44]Aude Genevay, Lénaïc Chizat, Francis Bach, Marco Cuturi, and Gabriel Peyré. Sample Complexity of Sinkhorn Divergences. In The 22nd International Conference on Artificial Intelligence and Statistics, pp. 1574–1583. PMLR, 2019. [paper].
  42. [45]Zhengyang Geng, Mingyang Deng, Xingjian Bai, J Zico Kolter, and Kaiming He. Mean flows for one-step generative modeling. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2026. [paper].
  43. [46]Jonathan Geuter, Gregor Kornhardt, Ingimar Tomasson, and Vaios Laschos. Universal Neural Optimal Transport. In Forty-second International Conference on Machine Learning, 2025. [paper].
  44. [47]Nathael Gozlan, Cyril Roberto, Paul-Marie Samson, and Prasad Tetali. Kantorovich duality for general transport costs and applications. Journal of Functional Analysis, 273(11):3327–3405, 2017. [paper].
  45. [48]Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville. Improved Training of Wasserstein GANs. Advances in Neural Information Processing Systems, 30, 2017. [paper].
  46. [51]Doron Haviv, Aram-Alexandre Pooladian, Dana Pe’er, and Brandon Amos. Wasserstein Flow Matching: Generative Modeling Over Families of Distributions. In Forty-second International Conference on Machine Learning, 2025. [paper].arxiv.org/abs/2411.00698
  47. [52]Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep Residual Learning for Image Recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778, 2016. [paper].
  48. [53]Kurt Hornik. Approximation capabilities of multilayer feedforward networks. Neural Networks, 4(2):251–257, 1991. ISSN 0893-6080. doi: https://doi.org/10.1016/0893-6080(91)90009-T. [paper].
  49. [54]Bamd​​ad Hosseini, Alexander W. Hsu, and Amirhossein Taghvae​​i. Conditional Optimal Transport on Function Spaces. SIAM/ASA Journal on Uncertainty Quantification, 13(1):304–338, 2025. [paper].
  50. [55]Samuel Howard, George Deligiannidis, Patrick Rebeschini, and James Thornton. Differentiable Cost-Parameterized Monge Map Estimators. In ICML 2024 Workshop on Differentiable Almost Everything: Differentiable Relaxations, Algorithms, Operators, and Simulators, 2024. [paper].arxiv.org/abs/2406.08399
  51. [56]Chin-Wei Huang, Ricky T. Q. Chen, Christos Tsirigotis, and Aaron Courville. Convex Potential Flows: Universal Probability Distributions with Optimal Transport and Convex Optimization. In International Conference on Learning Representations, 2021. [paper].arxiv.org/abs/2012.05942
  52. [57]Guillaume Huguet, Daniel Sumner Magruder, Alexander Tong, Oluwadamilola Fasina, Manik Kuchroo, Guy Wolf, and Smita Krishnaswamy. Manifold Interpolating Optimal-Transport Flows for Trajectory Inference. In Advances in Neural Information Processing Systems, 2022. [paper].
  53. [58]Richard Jordan, David Kinderlehrer, and Felix Otto. The Variational Formulation of the Fokker–Planck Equation. SIAM Journal on Mathematical Analysis, 29(1):1–17, 1998. [paper].
  54. [59]Leonid V Kantorovich. On the translocation of masses. Doklady Akademii Nauk SSSR (New Series), 37:199–201, 1942. [paper: english translation].
  55. [60]Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition, pp. 4401–4410, 2019. [paper].
  56. [61]Masahiro Kato, Takeshi Teshima, and Junya Honda. Learning from Positive and Unlabeled Data with a Selection Bias. In International Conference on Learning Representations, 2019. [paper].
  57. [63]Gavin Kerrigan, Giosue Migliorini, and Pad​​hraic Smyth. Dynamic Conditional Optimal Transport through Simulation-Free Flows. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. [paper].arxiv.org/abs/2404.04240
  58. [64]Valentin Khrulkov, Gleb Ryzhakov, Andrei Chertkov, and Ivan Oseledets. Understanding DDPM Latent Codes Through Optimal Transport. In The Eleventh International Conference on Learning Representations, 2023. [paper].arxiv.org/abs/2202.07477
  59. [65]Boah Kim, Yan Zhuang, Tejas Sudharshan Mathai, and Ronald M Summers. OTMorph: Unsupervised Multi-domain Abdominal Medical Image Registration Using Neural Optimal Transport. IEEE Transactions on Medical Imaging, 2024. [paper].DOI
  60. [66]Diederik P. Kingma and Jimmy Ba. Adam: A Method for Stochastic Optimization, 2017. [paper].
  61. [67]Nikita Maksimovich Kornilov, Petr Mokrov, Alexander Gasnikov, and Alexander Korotin. Optimal Flow Matching: Learning Straight Trajectories in Just One Step. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. [paper].arxiv.org/abs/2403.13117
  62. [68]Alexander Korotin, Vage Egiazarian, Arip Asadulaev, Alexander Safin, and Evgeny Burnaev. Wasserstein-2 Generative Networks. In International Conference on Learning Representations, 2021a. [paper].
  63. [71]Alexander Korotin, Daniil Selikhanovych, and Evgeny Burnaev. Neural Optimal Transport. In The Eleventh International Conference on Learning Representations, 2023b. [paper].
  64. [72]Dohyun Kwon, Ying Fan, and Kangwook Lee. Score-based generative modeling secretly minimizes the wasserstein distance. In Advances in Neural Information Processing Systems, 2022. [paper].arxiv.org/abs/2212.06359
  65. [73]Marc Lambert, Sinho Chewi, Francis Bach, Silvère Bonnabel, and Philippe Rigollet. Variational inference via wasserstein gradient flows. In Advances in Neural Information Processing Systems, 2022. [paper].arxiv.org/abs/2205.15902
  66. [74]Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow Matching for Generative Modeling. In The Eleventh International Conference on Learning Representations, 2023. [paper].
  67. [77]Xingchao Liu, Chengyu Gong, and Qiang Liu. Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow. In The Eleventh International Conference on Learning Representations, 2023. [paper].arxiv.org/abs/2209.03003
  68. [78]Xingchao Liu, Xiwen Zhang, Jianzhu Ma, Jian Peng, and qiang liu. InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generation. In The Twelfth International Conference on Learning Representations, 2024b. [paper].arxiv.org/abs/2309.06380
  69. [79]Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaogang Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015. [dataset].
  70. [80]Ilya Loshchilov and Frank Hutter. SGDR: Stochastic Gradient Descent with Warm Restarts. In International Conference on Learning Representations, 2017. [paper].arxiv.org/abs/1608.03983
  71. [81]Frederike Lübeck, Charlotte Bunne, Gabriele Gut, Jacobo Sarabia del Castillo, Lucas Pelkmans, and David Alvarez-Melis. Neural Unbalanced Optimal Transport via Cycle-Consistent Semi-Couplings. In NeurIPS 2022 AI for Science: Progress and Promises, 2022. [paper].arxiv.org/abs/2209.15621
  72. [82]Shaojun Ma, Shu Liu, Hongyuan Zha, and Haomin Zhou. Learning Stochastic Behaviour from Aggregate Data. In International Conference on Machine Learning, pp. 7258–7267. PMLR, 2021. [paper].
  73. [83]Ashok Makkuva, Amirhossein Taghvaei, Sewoong Oh, and Jason Lee. Optimal transport mapping via input convex neural networks. In International Conference on Machine Learning, pp. 6672–6681. PMLR, 2020. [paper].
  74. [84]Robert J McCann. A Convexity Principle for Interacting Gases. Advances in Mathematics, 128(1):153–179, 1997. [paper].
  75. [85]Petr Mokrov, Alexander Korotin, Lingxiao Li, Aude Genevay, Justin Solomon, and Evgeny Burnaev. Large-Scale Wasserstein Gradient Flows. In Advances in Neural Information Processing Systems, 2021. [paper].arxiv.org/abs/2106.00736
  76. [86]Petr Mokrov, Alexander Korotin, Alexander Kolesov, Nikita Gushchin, and Evgeny Burnaev. Energy-guided Entropic Neural Optimal Transport. In The Twelfth International Conference on Learning Representations, 2024. [paper].arxiv.org/abs/2304.06094
  77. [87]Gaspard Monge. Mémoire sur la théorie des déblais et des remblais. Mémoires de Mathématique et de Physique, présentés à l’Académie Royale des Sciences, pp. 666–704, 1781. [paper].
  78. [88]Kirill Neklyudov, Rob Brekelmans, Daniel Severo, and Alireza Makhzani. Action Matching: Learning Stochastic Dynamics from Samples. In International Conference on Machine Learning, pp. 25858–25889. PMLR, 2023a. [paper].arxiv.org/abs/2210.06662
  79. [89]Kirill Neklyudov, Jannes Nys, Luca Thiede, Juan Carrasquilla, Qiang Liu, Max Welling, and Alireza Makhzani. Wasserstein Quantum Monte Carlo: A Novel Approach for Solving the Quantum Many-Body Schrödinger Equation. Advances in Neural Information Processing Systems, 36, 2023b. [paper].arxiv.org/abs/2307.07050
  80. [90]Kirill Neklyudov, Rob Brekelmans, Alexander Tong, Lazar Atanackovic, Qiang Liu, and Alireza Makhzani. A Computational Framework for Solving Wasserstein Lagrangian Flows. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 37461–37485. PMLR, 21–27 Jul 2024. [paper].arxiv.org/abs/2310.10649
  81. [91]XuanLong Nguyen, Martin J Wainwright, and Michael I Jordan. Estimating divergence functionals and the likelihood ratio by convex risk minimization. IEEE Transactions on Information Theory, 56(11):5847–5861, 2010. [paper].DOI
  82. [92]Sebastian Nowozin, Botond Cs e k e, and Ryota Tomioka. f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization. Advances in Neural Information Processing Systems, 29, 2016. [paper].arxiv.org/abs/1606.00709
  83. [93]Matteo Pariset, Ya-Ping Hsieh, Charlotte Bunne, Andreas Krause, and Valentin De Bortoli. Unbalanced diffusion schrodinger bridge. In ICML Workshop on New Frontiers in Learning, Control, and Dynamical Systems, 2023. [paper].
  84. [94]Stefano Peluchetti. Diffusion Bridge Mixture Transports, Schrödinger Bridge Problems and Generative Modeling. Journal of Machine Learning Research, 24(374):1–51, 2023. [paper].arxiv.org/abs/2304.00917
  85. [95]Abhijeet Phatak, Sharath Raghvendra, Chittaranjan Tripathy, and Kaiyi Zhang. Computing all Optimal Partial Transports. In The Eleventh International Conference on Learning Representations, 2023. [paper].
  86. [96]Stanislav Pidhorskyi, Donald A Adjeroh, and Gianfranco Doretto. Adversarial Latent Autoencoders. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14104–14113, 2020. [paper].
  87. [97]Yury Polynskiy and Yihong Wu. Information Theory: From Coding to Learning. Cambridge University Press, 2024. [book].
  88. [98]Aram-Alexandre Pooladian, Heli Ben-Hamu, Carles Domingo-Enrich, Brandon Amos, Yaron Lipman, and Ricky T Chen. Multisample Flow Matching: Straightening Flows with Minibatch Couplings. In International Conference on Machine Learning, 2023. [paper].arxiv.org/abs/2304.14772
  89. [99]Aram-Alexandre Pooladian, Carles Domingo-Enrich, Ricky T. Q. Chen, and Brandon Amos. Neural Optimal Transport with Lagrangian Costs. In The 40th Conference on Uncertainty in Artificial Intelligence, 2024. [paper].
  90. [100]Bilal Riaz, Yuksel Karahan, and Austin J. Brockmeier. Partial Optimal Transport for Support Subset Selection. Transactions on Machine Learning Research, 2023. [paper].
  91. [102]Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional Networks for Biomedical Image Segmentation. In Medical Image Computing and Computer-assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18, pp. 234–241. Springer, 2015. [paper].
  92. [103]Litu Rout, Alexander Korotin, and Evgeny Burnaev. Generative Modeling with Optimal Transport Maps. In International Conference on Learning Representations, 2022. [paper].
  93. [104]Filippo Santambrogio. Optimal Transport For Applied Mathematians. Birkäuser, NY, 55(58-63):94, 2015. [book].
  94. [106]Christopher Scarvelis and Justin Solomon. Riemannian Metric Learning via Optimal Transport. In The Eleventh International Conference on Learning Representations, 2023. [paper].
  95. [107]Vivien Seguy, Bharath Bhushan Damodaran, Remi Flamary, Nicolas Courty, Antoine Rolet, and Mathieu Blondel. Large Scale Optimal Transport and Mapping Estimation. In International Conference on Learning Representations, 2018. [paper].
  96. [108]Thibault Séjourné, Gabriel Peyréa, and François-Xavier Vialard. Unbalanced Optimal Transport, from theory to numerics. Numerical Control: Part B, pp. 407, 2023. paper.
  97. [109]Yuyang Shi, Valentin De Bortoli, Andrew Campbell, and Arnaud Doucet. Diffusion Schrödinger Bridge Matching. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. [paper].
  98. [110]Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep Unsupervised Learning using Nonequilibrium Thermodynamics. In Proceedings of the 32nd International Conference on Machine Learning, volume 37 of Proceedings of Machine Learning Research, pp. 2256–2265, Lille, France, 07–09 Jul 2015. PMLR. [paper].
  99. [112]Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising Diffusion Implicit Models. In International Conference on Learning Representations, 2021a. [paper].
  100. [113]Yang Song and Stefano Ermon. Improved Techniques for Training Score-Based Generative Models. Advances in Neural Information Processing Systems, 33:12438–12448, 2020. [paper].
  101. [114]Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-Based Generative Modeling through Stochastic Differential Equations. In International Conference on Learning Representations, 2021b. [paper].
  102. [115]Paweł Teisseyre, Timo Martens, Jessa Bekker, and Jesse Davis. Learning from biased positive-unlabeled data via threshold calibration. In The 28th International Conference on Artificial Intelligence and Statistics, 2025. [paper].
  103. [116]Alexander Tong, Jessie Huang, Guy Wolf, David Van Dijk, and Smita Krishnaswamy. TrajectoryNet: A Dynamic Optimal Transport Network for Modeling Cellular Dynamics. In International conference on machine learning, pp. 9526–9536. PMLR, 2020. [paper].
  104. [117]Alexander Tong, Kilian Fatras, Nikolay Malkin, Guillaume Huguet, Yanlei Zhang, Jarrid Rector-Brooks, Guy Wolf, and Yoshua Bengio. Improving and generalizing flow-based generative models with minibatch optimal transport. Transactions on Machine Learning Research, 2024a. ISSN 2835-8856. [paper].

Paper details

vlnc:W_r7fr4vx3
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 1

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

generative AI tools

View statement
Looking for related papers…
Contents