WorksComputer Science
Dynamic Neural Optimal Transport for Subset Alignment
Abstract
We study optimal transport between distributions when only a subset of the target population is relevant to the source. The goal is to map the source distribution to a reweighted target distribution whose density ratio with respect to the full target lies in . This constraint enables the transported distribution to select an appropriate target subpopulation while limiting how strongly any region of the target can be emphasized. We propose a dynamic neural optimal transport framework for this setting using two neural networks. The first parameterizes a time-dependent potential whose spatial gradient defines the velocity field and whose terminal value enforces the density-ratio constraint. The second parameterizes an interpolation between the source distribution and the learned terminal distribution. Unlike prior neural dynamic optimal transport methods, which typically assume a fixed terminal distribution, our formulation learns the terminal distribution within the support of the target. We demonstrate the method on positive-unlabeled learning, unpaired domain translation, and trajectory inference, where meaningful alignments exist between source and target distributions, but direct distribution matching is inappropriate because the target contains additional or irrelevant components
Introduction
Distribution alignment is a problem of immense importance across machine learning, statistics, and optimization. Wasserstein distance and -divergences are two widely used families of measures of disparity between distributions. Given source and target distributions and defined on subsets of a convex set , specifically on and , respectively, the objective of optimal transport is to obtain a mapping from the source distribution to the target distribution that minimizes the expected transport cost, thereby characterizing the Wasserstein distance. This can be relaxed by mapping the source to a distribution near the target, using an -divergence to quantify the disparity between the output of the map and the target distribution. In particular, we consider a dynamic deterministic formulation of that uses a neural network to parameterize the velocity field. We also use a particular choice of -divergence that constrains the mapped density to lie within a subset of while satisfying a prescribed density-ratio bound. While previous work has studied relaxed neural optimal transport with static transport maps [40, 26, 27] and unbalanced dynamic optimal transport with mass growth [89], there is no existing neural dynamic optimal transport (OT) formulation that learns a map from the source distribution to a subset of the target distribution.
Specifically, our formulation of support subset alignment relies on a relaxation of the Benamou-Brenier problem, formulated in terms of a probability path and velocity field (Benamou & Brenier, 2000; Santambrogio, 2015; [36]. For probability measures , the semi-relaxed dynamic OT problem we study is
where the initial marginal constraint is preserved, whereas the terminal marginal is allowed to vary subject to the density-ratio constraint . This allows some regions of to be up-weighted while others are down-weighted or removed. The relaxed terminal constraint is equivalent to a partial optimal transport relaxation of static optimal transport using -divergences, as introduced by Séjourné et al. [108]. The case corresponds to prescribing the endpoint marginal as . Under suitable regularity, weak solutions of the continuity equation induce deterministic flows via mass conservation (Villani et al. (2009, p. 15)), while locally Lipschitz flows induce unique probability paths (Villani (2021, Theorem 5.34)).
In practice, when only samples are available, we use neural networks to parameterize both the velocity field and the probability path . Dynamic neural OT is closely connected to flow matching, diffusion-type models, trajectory inference, and generative modeling ([74, 77, 1, 116, 88 b]).
To position our formulation, we introduce necessary notation and discuss unbalanced optimal transport (UOT) and the dynamic version. Let and denote probability and finite nonnegative measures on . For a map , is the pushforward; for , and are its marginals. We write if . When absolutely continuous, we use the same symbol for a measure and its density; thus, if and , then on . We use , , and for time-varying measures, velocity fields, and growth fields, respectively, and for expectations under probability measures.
Unbalanced optimal transport (UOT) generalizes balanced OT (Santambrogio [104]) by allowing transport between finite nonnegative measures and for which and need not be equal. Let be proper, convex, lower semicontinuous functions satisfying . Given , define and . The UOT problem is then stated as ([38, 108])
where , and and denote the -divergences corresponding to and , respectively. Even when and are probability measures, the optimal solution of equation 2 is generally not a coupling of and . Thus, UOT relaxes the strict marginal equality constraints of OT by replacing them with -divergence penalties, making it applicable to general finite nonnegative measures. UOT has also been used to estimate transport maps using neural network parameterizations ([26, 27, 40, 41] (Yang & Uhler 2019)).
Dynamic UOT handles general positive measures by explicitly modeling mass creation and destruction through a scalar growth field (Chizat et al. [24]), and has been applied to problems in generative modeling and trajectory inference ([88, 90]). Dynamic UOT between is
Thus, UOT penalizes deviations from prescribed marginals, whereas dynamic UOT modifies the continuity equation through a mass growth term. When and are probability measures and mass creation/destruction is not needed, one may set , recovering balanced dynamic OT.
Positioning and contributions. Unlike UOT, which allows positive measures and relaxes marginal constraints, and dynamic UOT, which models mass creation/destruction but preserves marginals, our dynamic subset alignment preserves the source mass and transports it to a terminal distribution supported on a subset of the target support and within a density-ratio bound of the target. This is achieved by replacing the terminal marginal equality constraint with an -divergence-based penalty, which recovers balanced OT as a special case. We derive a dual dynamic formulation whose potential identifies the selected target subset and whose gradient defines the transport velocity. We use neural network-based parameterizations of the velocity field and path distribution to achieve the alignment and subset selection. Furthermore, we demonstrate the framework on unpaired domain translation, PU-learning, and trajectory inference problems.
Related Work
Our work on dynamic subset alignment is most directly related to Lagrangian neural optimal transport (Pooladian et al. [99]), action matching (Neklyudov et al. [88]), and Wasserstein Lagrangian flows (Neklyudov et al. [90]).
The neural optimal transport framework with Lagrangian costs [99] studies optimal transport with general Lagrangian potentials in Euclidean space. Wasserstein Lagrangian flows [90] are primarily developed for applications in cellular trajectory inference and quantum many-body problems and extend to more general settings such as the Wasserstein-Fisher-Rao metric (Chizat et al., 2018a,b; Séjourné et al., 2023), with the ability to model mass growth/destruction and different types of dynamics.
These approaches, as well as standard flow-based models, are developed for settings in which the endpoint marginals are prescribed, which prevents their direct application to the subset-alignment problem. A more extensive review of recent work on dynamic neural optimal transport is included in Appendix § 9.§ 2.
In contrast, our proposed dynamic support subset-selection framework preserves one marginal while dynamically transporting its mass to a subset of the support of the other marginal while minimizing the transport cost. Therefore, our approach can be viewed as a novel extension of prior work [90]. Although we focus on the cost, our method is compatible with other Lagrangian costs [99], which could be useful for incorporating side information, as in semi-supervised domain adaptation.
Additionally, we review advances in static neural optimal transport in Appendix § 9.§ 1. In Appendix § 7.§ 2, we also discuss the static formulation of subset alignment. The static formulation of our framework can be related to incomplete optimal transport (IOT) [40] and unbalanced optimal transport mapping (UOTM) [26]. IOT [40] can be interpreted as a proximal form of static subset alignment. UOTM can also be reduced to static subset alignment between two probability measures by choosing, in equation (2), as the -divergence corresponding to the indicator function and .
Methodology
In our semi-relaxed dynamic OT formulation equation (1), we can replace the inequality constraint by adding the equivalent -divergence penalty term to the objective, where is a special case of the range divergence corresponding to the convex indicator function , which, along with its Legendre–Fenchel conjugate , is given as
Setting and , one obtains with . Since is convex and lower semicontinuous, . The variational form of -divergences [91, 92], [97 Theorem 7.26] , yields
To solve the relaxed dynamic optimal transport problem in equation (1), we replace the terminal inequality constraint with the additive penalty in the objective. We introduce the Lagrange multiplier for the continuity equation and the Lagrange multiplier for the initial marginal constraint . After further simplification (see Appendix § 7.§ 3 for details), one obtains the conditions.
These conditions can then be used to eliminate , , and , yielding the optimization problem with the Lagrangian
To approximate the solution, we directly parameterize the dual potential with a neural network and parameterize the probability path using a neural network that acts on samples from the terminal marginals and . A similar parameterization is used in the computational framework for learning Wasserstein–Lagrangian flows (WLF) [90]. In contrast, flow-based models typically do not learn directly, but instead use analytically defined interpolants between two fixed marginals during training [74, 77, 1]. Our problem differs from both flow matching and WLF because we aim to map the source distribution to an unknown distribution supported on a subset of . To this end, for , we construct an interpolant from independently sampled points and as
where is a time-dependent neural network. At , we have . Since in general, , the network must learn a new terminal marginal. Let denote the flow induced by the velocity field , so that and , . The optimal correction corresponding to this flow is then given by and . By the standard neural-network universal approximation theorem [53], bounds on the -approximation error translate into bounds on . Consequently, for any admissible velocity field, one obtains for some constant .
Range-based penalty 5 also allows us to distinguish between data points sampled from the selected and rejected subsets of the target support . Let denote the optimal flow operator corresponding to the optimal dual potential . Then points sampled from the selected subset and from the rejected subset can be separated by the following test
Test 9 follows directly from the optimality conditions for the range penalty and dual (see Appendix § 8) and is used in our PU-learning and melanoma experiments.
Algorithm 1: Dynamic-Neural-SS: Learning Algorithm for Dynamic Subset Selection
Require: Samples from distributions and , time-dependent neural network , interpolant network , batch size , number of updates and , gradient-based optimizers and .
Ensure: Sample-based neural estimate for .
for learning iteration do
for do
Sample , , and .
.
.
Using and to update via gradient descent.
end for
for do
Sample , , and .
.
.
Update using and via gradient ascent.
end for
end for
Experiments and Results
We present experimental results for subset alignment on an easily interpretable image-to-image task, where the MNIST dataset [30] is the source and the EMNIST dataset [29] is the target. In this setting, digit images form a subset of the characters in EMNIST. We also applied dynamic subset alignment to the PU-Learning task. We then apply our proposed approach to domain translation on the FFHQ dataset (Karras et al., [60]) in the 512-dimensional latent space of ALAE (Pidhorskyi et al., [96]). We use Algorithm 1 to solve all the subset alignment problems discussed above. We also extended dynamic subset alignment to a multi-marginal setting and applied it to the melanoma dataset from Bunne et al. [18].
Toy Examples
First, we apply subset alignment to 1D and 2D point distributions. For the 1D toy example, we align a uniform distribution with a 1D semicircular distribution. Results for are shown in Figure 2. As the parameter increases, the source distribution is transported to a subset of the support of the target distribution , for which the potential satisfies . The transport between two 2D uniform distributions for is shown in Figure 1, where one can observe that is non-positive in selected subsets. Additional toy examples are provided in Appendix § 10, including subset alignment between two 1D uniform distributions and between two bimodal 2D distributions.

Figure 1. Subset alignment between two uniform distributions in at , obtained using Algorithm 1 with . The black contour line indicates the zero-level set of .

Figure 2. Source, target, and transported source histograms with sampling densities for 1D subset alignment between a uniform distribution and a semicircular density. is shown in red.
MNIST to EMNIST Domain Translation
The MNIST dataset contains 60,000 images of digits from 0 to 9 in the training set and 10,000 images in the test set. The MNIST dataset is roughly balanced, in the sense that the proportions of the classes are approximately the same. The EMNIST ByClass (EMNIST) dataset contains English letters and digits. EMNIST consists of 62 imbalanced classes, of which 10 classes correspond to digits between 0–9, while the remaining 52 classes correspond to uppercase and lowercase letters of the English alphabet. Roughly, 16% of EMNIST consists of digits, while the remaining 84% consists of letters.
Since our goal is to transfer MNIST images to EMNIST images in such a way that MNIST digits are mapped to EMNIST digits while ignoring letters, we trained a neural network classifier to distinguish between digits and letters and used it to evaluate the learned mapping; see Appendix § 11.§ 2 for implementation details. After training the classifier, we used both static and dynamic subset-selection approaches for domain translation between MNIST and EMNIST. Implementation details of the underlying models and their training are provided in Appendix § 11.§ 3. We evaluated
classification accuracy on translations of the entire MNIST dataset; the corresponding confusion matrices are provided in Appendix § 12. Accuracies of the translated outputs are in Table 1.
| Method | ||||
| static (Gazdieva et al., [40]) | 46.93 | 75.33 | 82.44 | 87.32 |
| dynamic (one step) | 64.85 | 75.16 | 93.57 | 95.84 |
| dynamic (ODE) | 58.80 | 70.47 | 92.68 | 95.00 |
Table 1. Classification accuracies of translated images for MNISTEMNIST.
In our experiments, we trained and evaluated both static and dynamic models using the corresponding subset-selection frameworks for . Similar to flow-based generative models, dynamic subset alignment requires numerical integration; in our experiments, we use ODE integration with an Euler-type numerical integrator with 100 integration steps. However, one-step integration can also be used (Liu et al., [77, 78]). Figure 3 shows that, perceptually, one-step integration performs worse than both static and ODE-based generation, but classification accuracies are better.

Figure 3. Image translation outputs for MNIST → EMNIST.
The classifier-based evaluation should be interpreted together with the qualitative results. As discussed above, the classifier is trained on the target data, where the digit classes are assigned separate labels (0-9), while all alphabet classes are grouped into a single additional label. Therefore, this evaluation primarily measures whether translated MNIST samples are classified as digit-like EMNIST samples rather than alphabet-like samples. For small , one-step translation can achieve higher accuracy because it only slightly perturbs the source image and often stays within the same classifier decision region, even if the resulting image is visually less aligned with EMNIST. In contrast, the ODE solution follows the learned transport more faithfully, which can cause more label changes for small and lower classifier accuracy. As increases, the relevant target subset becomes easier to identify, and the gap between one-step and ODE results becomes small both quantitatively and qualitatively.
Positive-Unlabeled Learning
Positive-unlabeled (PU) learning is a binary classification problem in which only a subset of positive samples is labeled. This labeled subset is then used to train a model that distinguishes positive samples from negative samples within an unlabeled dataset containing both positive and negative examples ([12, 61, 20, 100]). Since the sign of the optimal potential function in our framework differs between the selected and unselected subsets, it can be used to distinguish positive samples from unlabeled samples; see Appendix § 8.
We applied both static and dynamic optimal transport methods to PU learning on the 20 UCI datasets (Kelly et al.), following the setting of Teisseyre et al. [115]. We used the same 75/25 train-test split for each dataset and the sampled-completely-at-random (SCAR) mechanism to select and label positive points. Dynamic subset alignment employs Algorithm 1, while static subset alignment employs Algorithm 2, described in Appendix § 7.§ 1.
Neural networks consisting of five MLP layers with appropriate input and output dimensions and Swish activation functions are used to train both the static and dynamic models. The learning rates are held constant during training for both models. Architecture and parameter details for each model are provided in Appendix § 11.§ 4. We trained 20 different models for each dataset using different train-test splits. In total, we trained 400 models for static subset alignment and 400 models for dynamic subset alignment. We adopted alternative sign-based and value-based label-assignment strategies for the unlabeled dataset. The performance of our approaches, measured in terms of balanced accuracy, along with the top-performing baselines PUSB (Kato et al., [61]) and NTC-MI (Teisseyre et al., [115]), is reported in Table 2. In all PU learning experiments, is fixed as , where denotes the known prior proportion of positively labeled points in the dataset. Details are discussed in Appendix § 11.§ 4.
| Dataset | PUSB | NTC-MI | static sorted | static sign | dynamic sorted | dynamic sign |
| Abalone | 0.544 ± 0.060 | 0.575 ± 0.025 | 0.561 ± 0.029 | 0.503 ± 0.008 | 0.578 ± 0.025 | 0.540 ± 0.054 |
| Banknote | 0.829 ± 0.050 | 0.922 ± 0.019 | 0.883 ± 0.037 | 0.882 ± 0.039 | 0.952 ± 0.022 | 0.939 ± 0.018 |
| Breast-w | 0.766 ± 0.145 | 0.870 ± 0.028 | 0.930 ± 0.028 | 0.941 ± 0.027 | 0.927 ± 0.024 | 0.931 ± 0.026 |
| Diabetes | 0.546 ± 0.042 | 0.700 ± 0.039 | 0.635 ± 0.044 | 0.629 ± 0.044 | 0.629 ± 0.043 | 0.630 ± 0.046 |
| Haberman | 0.513 ± 0.023 | 0.532 ± 0.066 | 0.539 ± 0.066 | 0.540 ± 0.067 | 0.544 ± 0.074 | 0.540 ± 0.075 |
| Heart | 0.527 ± 0.033 | 0.757 ± 0.053 | 0.637 ± 0.093 | 0.623 ± 0.089 | 0.656 ± 0.080 | 0.653 ± 0.094 |
| Ionosphere | 0.440 ± 0.085 | 0.755 ± 0.059 | 0.773 ± 0.091 | 0.762 ± 0.088 | 0.827 ± 0.066 | 0.807 ± 0.070 |
| Isolet | 0.793 ± 0.072 | 0.725 ± 0.006 | 0.881 ± 0.028 | 0.923 ± 0.030 | 0.731 ± 0.140 | 0.707 ± 0.182 |
| Jm1 | 0.628 ± 0.016 | 0.628 ± 0.013 | 0.576 ± 0.015 | 0.575 ± 0.010 | 0.593 ± 0.023 | 0.558 ± 0.029 |
| Kc1 | 0.645 ± 0.075 | 0.679 ± 0.030 | 0.604 ± 0.036 | 0.607 ± 0.035 | 0.649 ± 0.030 | 0.573 ± 0.032 |
| Madelon | 0.496 ± 0.030 | 0.519 ± 0.028 | 0.533 ± 0.025 | 0.523 ± 0.015 | 0.535 ± 0.018 | 0.517 ± 0.019 |
| Musk | 0.712 ± 0.036 | 0.767 ± 0.012 | 0.841 ± 0.018 | 0.847 ± 0.018 | 0.857 ± 0.021 | 0.847 ± 0.046 |
| Segment | 0.848 ± 0.074 | 0.803 ± 0.014 | 0.898 ± 0.038 | 0.927 ± 0.031 | 0.911 ± 0.038 | 0.936 ± 0.024 |
| Semeion | 0.569 ± 0.055 | 0.755 ± 0.022 | 0.824 ± 0.044 | 0.850 ± 0.067 | 0.603 ± 0.096 | 0.544 ± 0.067 |
| Sonar | 0.497 ± 0.041 | 0.573 ± 0.057 | 0.561 ± 0.091 | 0.524 ± 0.074 | 0.602 ± 0.089 | 0.590 ± 0.101 |
| Spambase | 0.821 ± 0.031 | 0.887 ± 0.014 | 0.786 ± 0.011 | 0.775 ± 0.010 | 0.799 ± 0.017 | 0.776 ± 0.018 |
| Vehicle | 0.549 ± 0.067 | 0.804 ± 0.042 | 0.806 ± 0.037 | 0.823 ± 0.032 | 0.622 ± 0.090 | 0.556 ± 0.077 |
| Waveform | 0.860 ± 0.012 | 0.829 ± 0.012 | 0.795 ± 0.015 | 0.743 ± 0.013 | 0.772 ± 0.0129 | 0.729 ± 0.016 |
| Wdbc | 0.798 ± 0.155 | 0.801 ± 0.043 | 0.861 ± 0.068 | 0.845 ± 0.063 | 0.884 ± 0.050 | 0.872 ± 0.039 |
| Yeast | 0.517 ± 0.051 | 0.657 ± 0.024 | 0.630 ± 0.040 | 0.612 ± 0.049 | 0.627 ± 0.029 | 0.533 ± 0.046 |
Table 2. Comparison of average balanced accuracies of 20 models trained using dynamic subset alignment methods with PUSB and NTC-MI reported by Teisseyre et al. [115], and our formulation of incomplete extremal optimal transport from Gazdieva et al. [40] (labeled static). The best methods are colored red and second best blue.
Since OT methods rely on the assumption that data points in a dataset can be geometrically related through a metric, we used t-SNE (van der Maaten & Hinton, 2008) embeddings with perplexity 30 for all datasets in Table 2 to evaluate the spatial separability of the positive and negative classes. We observed that our methods perform better or comparably to competing methods when the positive class (1) and negative class (0) have spatially separable t-SNE embeddings as shown by the t-SNE embeddings are provided in Appendix § 13.
FFHQ Image Translation
We also apply our proposed approaches to the unpaired image translation problem. We followed the experimental setup of Gazdieva et al. [41], where the FFHQ dataset, embedded in the latent space of the Adversarial Latent Autoencoder (ALAE) (Pidhorskyi et al., [96]), is divided either by gender (man or woman) or by age, as two orthogonal labels.
| Class | Man | Woman |
| Young | 15K | 23K |
| Old | 7K | 3.5K |
Table 3. Division of FFHQ train images.
Table 3, adapted from Gazdieva et al. [41], shows the number of images for each class; images with ages less than 16 are ignored, ages between 16 and 43 are labeled young, and the remainder are labeled old. Given these classes, the task is to learn to map a source distribution to a target distribution. There are four cases: young→old, old→young, man→woman, and woman→man. To evaluate the translation process, two classifiers pretrained in the ALAE latent space are used; one classifier is trained to classify young versus old, and the other to distinguish man versus woman. The target accuracy quantifies the proportion of translated images that lie within the target-class boundary. The source accuracy quantifies whether the translated images retain the orthogonal label. For example, with young→old, the source accuracy is whether the “aged” image of a young source image retains the same gender.
Implementation details for both static and dynamic subset selection to the FFHQ dataset are given in Appendix § 11.§ 5. Between young→old, old→young, man→woman and woman→man, it was observed that larger values of tend to preserve the source accuracy but often have lower target accuracy. This can be related to the fact that for larger values of , it takes more training steps to achieve the optimal subset selection. Example translated images for old→young
translation with dynamic subset alignment are shown in Figure 4, with other cases provided in Appendix § 14.

Figure 4. FFHQ old→young using dynamic subset alignment with Euler integration (100 steps).
We compare with LOT Gazdieva et al. [41], UOT-GAN Yang & Uhler (2019), UOTM, and UOT-SD Choi et al. [26, 27]. Results in Table 4 and Figure 5, using accuracy values for UOT-SD, UOT-GAN, and U-LOT reported by Gazdieva et al. [41], whereas the implementation for UOTM is discussed in Appendix § 7.§ 1. Methods with better target accuracy are worse for source/class accuracy.
| Task | Accuracy | static | dynamic | static | dynamic | static | dynamic | static | dynamic | UOT-SD | UOT-GAN | U-LOT | UOTM KL-KL | UOTM -KL | UOTM lin.- |
| Young→Old | Target | 84.09 | 83.45 | 70.47 | 79.23 | 47.63 | 74.51 | 27.07 | 73.93 | 87.33 | 84.25 | 81.78 | 71.06 | 72.68 | 80.78 |
| Young→Old | Class | 70.43 | 71.55 | 91.41 | 84.31 | 95.47 | 90.03 | 96.06 | 90.30 | 45.71 | 73.85 | 84.49 | 90.17 | 88.68 | 78.77 |
| Old→Young | Target | 96.06 | 93.36 | 92.55 | 86.65 | 85.04 | 85.03 | 72.93 | 83.33 | 97.39 | 95.88 | 87.79 | 88.67 | 89.70 | 94.19 |
| Old→Young | Class | 71.77 | 71.92 | 89.46 | 87.69 | 94.43 | 89.84 | 95.45 | 90.88 | 49.30 | 74.74 | 89.48 | 89.64 | 88.19 | 80.68 |
| Man→Woman | Target | 96.11 | 94.53 | 92.18 | 91.74 | 85.66 | 90.96 | 77.55 | 90.29 | 98.16 | 97.38 | 90.23 | 90.63 | 91.19 | 94.51 |
| Man→Woman | Class | 83.68 | 83.64 | 87.45 | 87.43 | 90.05 | 88.52 | 91.27 | 89.11 | 75.50 | 84.04 | 90.30 | 89.52 | 88.83 | 85.31 |
| Woman→Man | Target | 93.34 | 92.26 | 87.32 | 88.09 | 78.32 | 84.28 | 64.86 | 81.89 | 94.96 | 92.91 | 88.59 | 83.85 | 84.10 | 91.41 |
| Woman→Man | Class | 82.39 | 82.68 | 92.33 | 91.51 | 93.89 | 92.59 | 94.35 | 93.22 | 72.03 | 84.56 | 89.66 | 91.79 | 90.85 | 87.17 |
Table 4. Target and source accuracy (%) for different domain translations on the FFHQ dataset. Dynamic subset selection is evaluated using Euler integration with 100 steps.

Figure 5. Accuracy curves for , in comparison to results from LOT (Gazdieva et al., [41]), UOT-GAN (Yang & Uhler, 2019), UOT-SD and UOTM (Choi et al., [26, 27]).
Melanoma Dataset
The subset-alignment framework is primarily developed for aligning a source distribution to an unknown distribution whose support is contained within the support of a target distribution . Following Neklyudov et al. [90] and Pariset et al. [93], we adapt our dynamic subset-alignment method to the melanoma treatment data of Bunne et al. [18], using the preprocessed 50-dimensional representation. Unlike our other experiments, the melanoma setting involves three observed marginals at and we must match the intermediate marginal exactly. We therefore order the populations in reverse chronological order, (48h), (24h), and (8h), and train a multi-marginal interpolant that preserves and while leaving the terminal marginal (here, the 8h population) unconstrained so that sub-setting acts on . After training, we select cells with and transport the selected subset back to using the learned velocity field . (We use normalized times , , .) For evaluation, we integrate the ODE with explicit Euler (step size ) and report the best checkpoint based on validation MMD. The training procedure, neural parameterizations and experimental parameters for the Melanoma experiment are given in the Appendix E.6. As shown in Table 5, subset alignment improves over previous neural Schrödinger-bridge baselines (Chen et al., [22]; Pariset et al., [93]), while remaining competitive with WLF (Neklyudov et al., [90]).
| Model | MMD | Model | MMD | ||
| SB-FBSDE (Chen et al., [22]) | 6.23 | Subset Alignment (, 476/490) | 4.83 | ||
| uDSB (no growth) (Pariset et al., [93]) | 6.27 | Subset Alignment (, 247/490) | 4.68 | ||
| uDSB (with growth) (Pariset et al., [93]) | 6.11 | Subset Alignment (, 148/490) | 5.40 | ||
| WLF-OT no growth (Neklyudov et al., [90]) | 5.20 | Subset Alignment (, 103/490) | 6.60 | ||
| WLF-UOT (Neklyudov et al., [90]) | 5.01 |
Table 5. Results for the Melanoma dataset (Pariset et al., [93], Table 1) with three marginals and train/test splits. MMD and are measured between and obtained by running dynamics from . For subset alignment, dynamics are run from the selected subset of (cells with ; number of selected test cells in parentheses), and results are reported at the iteration with the lowest MMD on a validation set; note that the validation cells are a subset of the training cells.
Celeb-A to Anime Domain Translation
For domain translation between the CelebA (Liu et al., [79]) and Anime (Chao, [19]) datasets, we use highly trimmed variants of the DDPM++ architecture (Song et al., [114]) to parameterize both and so that the models fit within GPU memory. Because our subset-alignment framework maps the source distribution to a subset of the target distribution, FID is not an appropriate evaluation metric for this setting. We therefore rely on qualitative visual inspection. In our experiments, dynamic subset alignment successfully translates the source distribution toward the target domain; however, larger values of generally require longer training to achieve comparable visual quality and stability. A sample translation is shown in Figure 6.

Figure 6. CelebAAnime.
Limitations
Practically, one point of concern is that the sample estimators of Wasserstein distances suffer from the curse of dimensionality (Weed & Bach, 2019; Fournier & Guillin, [37]). This issue can be alleviated to some extent by employing entropic regularization (Genevay et al., [44]; Feydy et al., [35]), which, in the dynamic setting, is intimately connected to Schrödinger bridges (Peluchetti, [94]; Bortoli et al., [14]).
Recently, entropically regularized UOT has been studied for modeling birth or death processes in population dynamics (Pariset et al., [93]; Neklyudov et al., [88]). Our approach can also be applied to model death processes in settings where there is a canonical relationship between temporally ordered
populations. In such cases, we treat as the final population of survivors and as the initial population. The learned mapping then relates the survivors to a subset of the initial population. Moreover, the sign of the potential can be used to distinguish members of the initial population that are likely to survive from those that are not. We note that, in this and other applications, the choice of is critical. While has a clear interpretation as a constraint on the density ratio, automatic selection of based on the resulting transport cost may be possible, as previously studied for partial optimal transport in the discrete setting [95]. One consolidated approach would be to sample from a prescribed range and use multi-task learning to optimize networks across varying values of . From an implementation perspective, this can be achieved by using scalar embeddings of , analogous to the time-variable embeddings used in dynamic networks.
Another key limitation of dynamic models, including frameworks similar to ours, is that their iterative generation process requires more computational resources and longer inference time [77]. For example, Table 6 reports the average per-sample generation time for FFHQ image translation. These results show that, in this setting, one-step generation is at least an order of magnitude faster than ODE integration. Considerable recent effort has been devoted to improving the generation efficiency of dynamic models [78, 45].
| Metric | One-Step | ODE |
| Time/sample (s) |
Table 6. Average evaluation time per sample for FFHQ domain translation tasks from Table 4. Evaluation is performed with batch size 128.
We note that although this work focuses on relaxing the target distribution, the range-divergence framework could also be used to relax the source distribution. A fully relaxed version, in which both source and target marginals are relaxed, may be applicable to other classes of problems.
Conclusion
Our neural optimal transport approach with subset selection achieves subset alignment by translating between two distributions while reweighting and selecting a subset of the target support. Our results on image-translation tasks across two datasets and 20 tabular PU-learning tasks show that a meaningful subset can be learned simultaneously with a Monge map. Unlike previous work, our dynamic formulation allows the terminal distribution to differ from the original target marginal, thereby inducing flows toward the nearest target subset while maintaining the prescribed density-ratio constraint.
AI use statement
We used generative AI tools primarily for copy-editing (improving grammar, clarity, and flow). We did not use generative AI for code development, except in the Melanoma dataset section, where an AI tool helped identify a bug. All AI-assisted outputs were reviewed and verified by the authors. We take full responsibility for the final content of this work, including all text, claims, and artifacts.
Reproducibility statement
To facilitate full reproducibility, we provide detailed descriptions of all experiments in the appendix. The main body summarizes the experimental setup and procedures for each experiment, while the appendix includes the complete specifications needed to replicate our results.
Preliminaries and Problem Formulation
Gaspard Monge proposed the original idea of optimal transport as mathematical model for the problem of minimum-cost transportation of dirt from a source location to a destination [87]. In more modern parlance, given probability measures, defined on compact set , defined on compact set , and the bounded uniformly continuous cost , the Monge formulation of optimal transport is stated as
where the set denotes the set of measurable maps between and . Monge formulation of the optimal transport problem requires that the transport map be a deterministic function. To satisfy the constraint in the Monge problem 10, the transport map must cover up to some -null sets. Usually, the cost is non-linearly dependent on the transportation map , making the problem 10 very cumbersome and very difficult to solve [104] (Villani et al. 2009). Kantorovich reformulated the Monge problem by relaxing the constraint that supports of and should be related to each other by a functional relation [59]. Instead, he allowed the marginals and to be related to each other by a joint measure. Kantorovich’s reformulation of the problem is a linear program, and its solution exists for all convex lower-semi-continuous costs [104, 36]. The Kantorovich problem is
where it can be observed that integrals and marginalize with respect to spaces and respectively. Therefore, must be the joint measure between and defined on the product space , i.e. . In other words, constraints in the Kantorovich problem ensure that every feasible solution must be a joint distribution of and . While in general, the Kantorovich problem is much easier to solve in comparison to the Monge problem, there are conditions of practical importance where one can employ the solutions of the Kantorovich problem to obtain the solution of the Monge problem. Those conditions are more clearly discussed in terms of the dual form of the Kantorovich problem Santambrogio, [104] given as,
The functions and are called Kantorovich potentials. By defining c-conjugate (also called c-transform) of , and -conjugate (also called -transform) of as
Using c and conjugates, Kantorovich problem is expressed as
Under very general conditions, one can relate the cost with the support of optimal coupling solution and optimal Kantorovich potentials and Santambrogio, [104] by
In discrete domains, above result is equivalent to Karush-Kuhn-Tucker (KKT) conditions for optimality. One can further relate the optimal solutions of Monge and Kantorovich problems using a landmark result by Gangbo & McCann [39] Figalli & Glaudo, [36], which states that there exists an optimal Kantorovich coupling of the form , where is Monge map satisfying
if the following conditions are satisfied
is absolutely continuous,
map is differentiable ,
the gradient map is injective ,
and, the gradient satisfies the local Lipschitz condition for all where is a ball of radius around .
When the cost can be written as , where is strictly convex and translation-invariant function, one can further relate the Monge mapping with optimal dual potential by Santambrogio, [104]
where is the Legendre-Fenchel conjugate of given by . In the result above, when , one obtains the result of the celebrated Brenier theorem of optimal transport for squared-Euclidean costs with , where is convex Brenier, [15] Figalli & Glaudo, [36]. The Brenier theorem on optimal transport differs from another important theorem on polar factorization Brenier, [15] stating that under very general conditions a square-integrable vector field can be decomposed into the composition of the gradient of a unique convex function and a unique measure-preserving map , i.e. . Before the discussion on dynamic formulation of the problem, we would like to point out that much of the recent work on static neural optimal transport relies on the above results.
Benamou and Brenier formulated the Wasserstein distance with squared Euclidean cost as the expected kinetic energy minimization problem under the assumption that both the source and target distributions have finite second moments Benamou & Brenier, [13]; Santambrogio [104]; Figalli & Glaudo [36]. Assuming that supports of both source and target distributions lie in a convex set , whose normal at the boundary is given by . For a bounded and smooth velocity-field such that , the flow corresponding to the velocity-field is given by
Considering that there also exists a probability path corresponding to the flow such that , Benamou-Brenier formulation of optimal transport is
s.t.
where, denotes the divergence operator mapping scalar or vector fields to scalar for the field by . The optimal flow is related to Monge mapping by displacement interpolation [84],
It is important to mention that the Benamou-Brenier formulation can be extended to Wasserstein- distances, for , under the assumption that both source and target distributions have finite th moments [104 Chapters 5 & 6].
Static support subset alignment
Our formulation of static support subset alignment with optimal transport is derived from relaxed optimal transport problem where the constraint on the first marginal of the joint density is maintained, while the second marginal is allowed to vary from within a range of the target density , such that . The density can be interpreted as a reweighted target density , , where portions of the support can be up-weighted while others are down-weighted or removed. The relaxed constraint is equivalent to a case of the partial optimal transport relaxations using -divergences introduced by Séjourné et al. [108],
where is the range divergence with being the convex indicator function,
where denotes its Legendre-Fenchel conjugate. Since the function is convex lower semi-continuous, therefore , we can apply the variational form of the -divergence [91], Polyanskiy & Wu, [97], exploited by -GAN [92], leading to a form requiring only expected values
To match 23, we focus on and , such that and for notational compactness denote . Introducing as a measurable function to act as a Lagrange multiplier to enforce the constraint in 23 and combining with 25 yields the problem
As described in Appendix A.2, since is convex and lower semi-continuous, we interchange the and and apply what is known as the -transform of [104] (Villani et al. 2009) to obtain the dual problem with measurable map
For computational implementation and are parameterized using neural networks with associated parameters and and expectations are estimated using samples from and as described in the Algorithm 2.
Algorithm 2: Static-Neural-SS: Learning Algorithm for Static Subset Alignment
Require: Source distribution and target distribution , cost function , reweighting bound , neural networks and , batch size , number of updates and , optimizers and .
Ensure: Trained neural estimator for transport map .
for do
for do
Sample and
Using update via gradient descent
end for
for do
Sample and
Using update via gradient ascent
end for
end for
Derivation of Static Neural Subset Selection
We denote the problem expressed in 26 as , where the Lagrangian is
We proceed to interchange the sup and inf,4 which is allowed due to the strong duality property associated with optimal transport when the cost is convex and lower semi-continuous,
optimizing with respect to for given and , the integrand in the first term of 28 is unbounded from below at any point . Thus, and need to ensure that . This constraint requires that, for any , (Villani et al., 2009, Theorem 5.10 and Remark 5.13). This definition of corresponds to the -transform of in the optimal transport literature (Santambrogio, 2015; Villani et al., 2009). Then, the inner infimum with respect to is attained with zero value if . Therefore, the dual problem becomes
Equation 30 and equation 31 are equal due to a theorem by Rockafellar (1976, Theorem 3A); equation 31 and equation 32 are equivalent by definition; equation 32 and equation 33 are equivalent since the function is convex with respect to and concave with respect to ; and equation 33 and equation 34 are equivalent by the variational formula for the -divergence. Thus, we obtain a relaxed Monge formulation.
Derivation of Dynamic Neural Subset Selection
The modified Benamou-Brenier problem takes the following minimax form
Combining the objective and constraints in equation 35 to obtain the Lagrangian
Since is supported on a bounded subset , one can change the order of integration. Therefore, for term (III), by changing the order of integration and then computing integration by parts one obtains,
To simplify (IV), we can use product rule of derivatives to write
Therefore by combining the above identity with Gauss’s theorem, one obtains
From the boundary condition on optimal transport (see the discussion above equation (20), and also [36 section 4.1]), the first part of the right-hand side of equation 38 is zero; therefore,
To eliminate the primal variable , substitute equation 37 and equation 39 into equation 36 and compute the variational derivative to obtain the stationary condition. For that one can write the terms of Lagrangian depending on as
With the additive perturbation function vanishing at and and a scalar , the Lagrangian is
and variational derivative is
The stationarity condition requires . For arbitrary perturbation , the variational-derivative if and only if
Therefore, one can write the Lagrangian as
Similarly, by computing , , and using stationary conditions, one obtains the condition,
Therefore, Lagrangian is simplified to
The simplified problem (equation 7 in the main body) is
Thresholding for PU-Learning and Rejection Sampling
Our idea of rejection sampling and thresholding for PU-learning relies on the fact that the dual form of range divergence is zero when the supremum in the dual is attained by the function with i.e.
By defining , , and , one can also see that , therefore one can write equation 46 as
One can further write
After substituting equation 48 into equation 47 one obtains
The dual form equation 47 is optimal with zero duality gap, if the primal form satisfies , , which can also be restricted to . This is equivalent to almost-everywhere in . Therefore, one can say that and also
We can now see that both integrands in equation (49) are nonnegative and sum to a value less than or equal to zero, which is only possible if both are equal to zero. Therefore, one can write
Further, two non-negative integrals are evaluated on two mutually exclusive sets; therefore, to have a sum equal to zero value, we can conclude that each integral is zero individually. Therefore, we can write
Equation 52 is therefore equivalent to the following element-wise test:
Additionally, from the equation 50, one can also conclude that
which is a complementary slackness condition in the sense that almost everywhere in . During the neural network training with finite data-points, the potential function is usually sub-optimal, and its sign cannot be relied on; therefore, instead of directly using the sign, one can sort values of potential at data points and select a predetermined proportion (prior) of data-points. Therefore, for training PU-learning models, we applied both sign-based and sorting-based filtration of data. From Figures 7, Figure 8 and Figure 9, one can observe that the optimal potential for static problem exactly follows equation (53), whereas in the dynamic case the sign of is inverted, which is due to the relation obtained in equation (44), which ensures that for the at optimal following relation holds

Figure 8. Temporal snapshots at for . (a) Evolution between two 2D bimodal distributions; the black contour line indicates the zero-level set of . (b) Evolution from 1D uniform to another 1D uniform density, with dynamic subset subset alignment. is denoted by red curve.

Figure 9. Static subset alignment results for . (a) Translation between two 2D uniform distributions shown in Figure 1. (b) 2D bimodal distributions shown in Figure 8a. Red curves indicate the zero-level sets of .
Survey of Recent Work on Neural Optimal Transport
In this section, we discuss the recent related work on computational optimal transport and its applications. More specifically, we consider the works that are related to neural estimation of optimal transport maps with occasional reference to theoretical developments.
Static Neural Optimal Transport
Seguy et al. [107] employed stochastic gradient-based approaches in one of the earliest works to estimate the optimal transport map using neural networks. Notably, the work by Seguy et al. [107] differed from early work by Genevay et al. [42] in the sense that the later work employed stochastic gradient-based methods to estimate the transport plan for large-scale data, whereas earlier work by Genevay et al. [42] only minimized the optimal transport loss using stochastic-gradient-based methods. This is also in contrast to the well-known Wasserstein-GAN (Arjovsky et al., [8]; Gulrajani et al., [48]) that employs the Kantorovich-Rubinstein duality to minimize the Wasserstein-1 loss function for generative modeling, where neural networks are employed as parameterizations for both dual-potential and data generator, but do not provide transport plans. Finally, the Sinkhorn-GAN employs an approximation of the discrete Wasserstein distance between latent representations of data and that of samples from non-informative prior (Genevay et al., [43]) for generative modeling. Now, we can see clear distinction between two different classes of approaches employing Wasserstein distances in generative modeling, the first class of works concerns with employing Wasserstein distance as a loss for generative modeling, without any explicit concern for obtaining the underlying transport plan across the distributions (Arjovsky et al., [8]; Gulrajani et al., [48]). The second class seeks to learn a transport plan to realize the generative model.
Efforts to learn Monge maps were motivated by a theorem by Brenier [15], which essentially states that, for continuous distributions with squared-Euclidean transportation cost, the optimal solution of the Monge problem is the gradient of a convex function (Figalli & Glaudo, [36], Theorem 2.5.10). Therefore, initially, gradients of input-convex neural networks (ICNN) (Amos et al., [6]) were employed to estimate the transport plan for the Wasserstein-2 distance [83, 68 b]. This approach has also been employed to supervised conditional neural Monge maps [16] and unbalanced optimal transport [81]. The study by Amos et al. [7] focuses on the development of an efficient neural optimal solution that could be implemented quickly in more practical scenarios. This approach to solve Wasserstein-2 distances employing convex potentials involves computationally challenging evaluation of the Fenchel conjugate of an ICNN-parameterized convex function. More recent work in this direction focuses on improved optimization strategies and better ICNN architectures to bypass problems related to Fenchel conjugate evaluations and ICNN training (Amos, 2023; Vesseron & Cuturi, 2024). Recent work also focuses on some batch-based methods, which have been devised to improve the regularity of learned neural Monge maps [31] (Uscidda & Cuturi 2023).
Another recent direction of work is based on the idea that ICNNs can be overly restrictive; therefore, more general neural network architectures should be employed to directly parameterize the transport maps [103, 71]. The work by Fan et al. [32, 34] focuses on employing neural networks to approximate the solution for Monge’s transport problem and also draws inspiration from the recent developments in neural-network-based parametric realizations for approximating Kantorovich plans. Recently, neural optimal transport has also been extended to unbalanced transportation settings [26] (Yang & Uhler 2019). Another work directly related to the static subset selection problem is the inexact optimal transport Gazdieva et al. [40].
Unless there is a corresponding Monge mapping [27, 86, 46], optimal transport requires a stochastic transport plan. A recent body of work (Korotin et al., 2023b,a; Asadulaev et al., 2024) deals with learning transportation plans using a weaker formulation of optimal transport [47, 11] along with noise outsourcing techniques, which have also been extended to more general costs. Apart from the applications in image translation [71], neural optimal transport has been applied for biomedical image registration [65] and to study single-cell perturbations [18]. Neural optimal transport has also been employed for metric learning [55, 106].
Dynamic Neural Optimal Transport
The potential applications of dynamic optimal transport in the cellular trajectory inference [116] and its connections with flow-based models for generative modeling [56, 57] has been instrumental in the recent research developments in this direction. Jordan-Kinderlehrer-Otto flow (JKO) is a time discretization scheme to solve Wasserstein gradient flows for different energy functionals [58] (Santambrogio 2017). Therefore, a lot of effort done in that regard is focused on neural network-parameterized schemes to solve the JKO-flow problem for both cellular trajectory inference and generative modeling [82, 33, 73, 17, 26, 28, 3, 85, 4] (Xu et al. 2023). The JKO scheme has also been studied for the applications related to molecular discovery [4]. A recent study deals with convergence properties of JKO-based generative models [23].
Recent developments in flow-matching models based on flow matching [74, 1, 77] for generative modeling led to even more interest in the development of algorithms to solve dynamic optimal transportation problems. The action-matching-based framework led to the development of a more general framework to solve both trajectory inference and generative modeling problems [88] for the cases where one could also sample from the trajectory between two terminal marginals. Rectified flow matching [77, 78] uses the neural optimal transport in an additional rectification step to improve the linearity of flows so that after training the model, images could be generated efficiently with only a single-step integration along straight-line paths. For generative modeling, in contrast to target-conditional flow matching [74], where during training, flows are conditioned on target samples, discrete optimal transport-conditioned flow matching employs the mini-batch optimal transport to create the conditionals [98, 99, 117]. Another recent work [67] attempts to alleviate the error accumulation problems associated with mini-batch optimal transport by learning straight paths between source and target distributions in a single step. Flow-matching [1, 2], diffusion models [110, 113, 112], and Schrödinger bridges (Wang et al., 2021; Liu et al., 2022, 2024a; Shi et al., 2023; Gushchin et al., 2023b,a; Somnath et al., 2023) are deeply interconnected under the framework of generalized bridge matching [117, 2, 117, 109]. Recently, there have also been attempts to understand diffusion models as approaches to minimize the dynamic Wasserstein distances [72, 64]. Recent works generalize flow matching from different perspectives, Chen & Lipman [21] generalize the flow matching to the flows on Riemannian manifolds, Atanackovic et al. [10] attempt to extend the flow models to return meaningful flows for the data beyond training distributions, and Haviv et al. [51] generalize the flow matching to the cases where data can be treated as distributions of distributions.

Figure 7. Temporal snapshots at for . (a) Evolution between two 2D uniform distributions shown in Figure 1; the black contour line indicates the zero-level set. (b) Evolution from 1D uniform to semicircular density is shown in Figure 2 with support subset alignment. is denoted by red curve.
Additionally, there has been a recent dynamic extension to the conditional neural optimal transport ([54, 63]). There have also been efforts to study neural network-based scalable approaches to solve high-dimensional partial differential equations (Wan et al., 2023).
Additional Toy Results
For the toy results given in Figure 1 and Figure 2 the temporal snapshots of the evolution of points sampled from source along with the potential are given in Figures 7a and 7b, respectively. Similar temporal snapshots between two 2D bimodal uniform squares and two 1D uniform distributions are shown in Figure 8a and Figure 8b respectively.
The static framework for subset alignment is equivalent to the relaxations of the inexact optimal transport problem discussed by Gazdieva et al. [40]. Our approach for static subset alignment is discussed in Algorithm 2 and Appendix § 7.§ 1.
Implementation Details
Toy Results
For dynamic subset alignment, we use the model architectures in Listings 1 and 2, where and parameterize and , respectively. Listing 1 is used for the 1D toy experiments, while Listing 2 is used for the 2D toy experiments. We set num_hid=512 for all dynamic architectures. For both settings, we use Algorithm 1 with . We train for 50,000 iterations in 1D and 100,000 iterations in 2D. In all dynamic toy experiments, we use the Adam optimizer (Kingma & Ba, [66]) with a learning rate of and a batch size of 512.
For static subset alignment, we use the model architecture in Listing 3 for both the 1D and 2D toy experiments. The parameterizes the transport map via a residual connection, while DualNET parameterizes the dual function . The same architecture is used in both settings, with the input and output dimensions adjusted accordingly. We set num_hid=512 for all static architectures. For both settings, we use Algorithm 2 with and . We train for 20,000 iterations in 1D and 50,000 iterations in 2D. In all static toy experiments, we use the Adam optimizer (Kingma & Ba, [66]) with a learning rate of and a batch size of 512.
EMNIST Classifier
We merge all letters into a single class and treat each digit as its own class (digits 0–9 keep their numeric label, and all letters are labeled 10). To mitigate class imbalance during training, we use a class-reweighted SoftMax loss. For -class classification, let denote the per-class counts in the training set; we define reweighting coefficients by
Given a one-hot label vector and the network SoftMax output , the reweighted loss (risk) is
We train the EMNIST classifier using the train/validation split provided with the EMNIST dataset (Cohen et al., [29]). The model is ResNet-18 (He et al., [52]) and is optimized with the class-reweighted SoftMax loss in equation 57. We
import jax
from jax import numpy as jnp
from flax import linen as nn
import math
from ml_collections import config_dict
class Smodel(nn.Module):
num_hid: int
num_out: int
use_layernorm: bool = True
boundary_zero: bool = False
@nn.compact
def __call__(self, t, x):
if jnp.ndim(t) == 0:
t = jnp.broadcast_to(t, x.shape[:-1] + (1,))
elif t.shape[-1] != 1:
t = t.reshape(x.shape[:-1] + (1,))
h = jnp.concatenate([t, x], axis=-1)
for _ in range(4):
h = nn.Dense(self.num_hid)(h)
if self.use_layernorm:
h = nn.LayerNorm()(h)
h = nn.swish(h)
v = nn.Dense(self.num_out)(h)
return t * (1.0 - t) * v if self.boundary_zero else v
class Qmodel(nn.Module):
num_hid: int
num_out: int
@nn.compact
def __call__(self, t, x_0, x_1):
h = jnp.concatenate([t, x_0, x_1, t < 0.5], axis=-1)
for _ in range(3):
h = nn.swish(nn.Dense(self.num_hid)(h))
h = nn.Dense(self.num_out)(h)
return (1.0 - t) * x_0 + t * (x_1 + h)
Listing 1: Model architectures used for toy 1D dynamic subset alignment experiments.
import jax
from jax import numpy as jnp
from flax import linen as nn
import math
from ml_collections import config_dict
class Smodel(nn.Module):
num_hid: int
num_out: int
@nn.compact
def __call__(self, t, x):
if jnp.ndim(t) == 0:
t = jnp.broadcast_to(t, x.shape[:-1] + (1,))
h = jnp.concatenate([t, x], axis=-1)
for _ in range(3):
h = nn.swish(nn.Dense(self.num_hid)(h))
return nn.Dense(self.num_out)(h)
class Qmodel(nn.Module):
num_hid: int
num_out: int
@nn.compact
def __call__(self, t, x_0, x_1):
h = jnp.concatenate([t, x_0, x_1, t < 0.5], axis=-1)
for _ in range(3):
h = nn.swish(nn.Dense(self.num_hid)(h))
h = nn.Dense(self.num_out)(h)
return (1.0 - t) * x_0 + t * (x_1 + h)
Listing 2: Model architectures used for toy 2D dynamic subset alignment experiments.

Figure 10. Static subset alignment results for . (a) Transport from 1D uniform distribution to semicircular density shown in Figure 7b. (b) Transport from 1D uniform distribution to a uniform distribution shown in Figure 8b. Red curves indicate the .
1 import jax
2 from jax import numpy as jnp
3 from flax import linen as nn
4 import math
5 from ml_collections import config_dict
7 class Tmodel(nn.Module):
8 num_hid: int
9 num_out: int
11 @nn.compact
12 def __call__(self, inputs):
13 h = inputs
14 for _ in range(4):
15 h = nn.swish(nn.Dense(self.num_hid)(h))
16 h = nn.Dense(self.num_out)(h)
17 return inputs + h
19 class DualNET(nn.Module):
20 num_hid: int
21 num_out: int
23 @nn.compact
24 def __call__(self, x):
25 h = x
26 for _ in range(4):
27 h = nn.swish(nn.Dense(self.num_hid)(h))
28 return nn.Dense(self.num_out)(h)
Listing 3: Model architectures used for 1D and 2D static subset alignment toy experiments.
use Adam [66] with a warmup–cosine learning-rate schedule [80], a peak learning rate of , and 500 warmup steps. The cosine schedule decays over 20,000 steps to an end learning rate of . Training is stopped after 20,000 steps, once the classifier exceeds 90% overall validation accuracy and 99% accuracy on digits. Confusion matrices are reported in Appendix § 12.
MNIST-EMNIST Translation Models
For dynamic subset selection, following Neklyudov et al. [88], we parameterize the vector field with a U-Net equipped with DDPM-style time embeddings (Song & Ermon [113]). As in action matching (Neklyudov et al. [88]), returns a scalar via
Likewise, , which parameterizes , is implemented as a U-Net with time embeddings. For both U-Nets, we use the same architectural settings: nonlinearity = swish, nf = 64, ch_mult = (1, 2, 2, 2), num_res_blocks = 2, and attn_resolutions = (16, 8). We use the AdamW optimizer with learning-rate scheduling for 50,000 training iterations. The optimizer parameters are , weight decay = 0.1, and dropout = 0.1. We additionally employ exponential moving averages (EMA) during training with an EMA rate of 0.999. These settings are very similar to those used in rectified flow matching and action matching [77, 88]. All input images in neural subset alignment are centered at [0.5, 0.5] and resized to . The learning rate is linearly increased from zero to its maximum value during the first 5,000 iterations and is then kept constant. The maximum learning rates are and for and , respectively. Additionally, gradients are clipped element-wise to the interval . Algorithm 1 is employed for 50,000 training iterations with and . A batch size of 128 is used to train the dynamic models for MNIST-EMNIST translation.
For static domain translation, the transport network is a U-Net Ronneberger et al. [102] with base factor 64, and the critic network is a ResNet; we use the same choices as Gazdieva et al. [40] for inexact extremal optimal transport. All inputs to the static models are centered at [0.5, 0.5] and resized to . We train and with Adam Kingma & Ba [66] using an initial learning rate of , which is halved after , , , , and steps. We run Algorithm 2 for 50,000 learning iterations with and . A batch size of 128 is used for static MNIST–EMNIST translation.
Models for PU-Learning Using Subset Alignment
The architectures used for PU learning with dynamic subset alignment are given in Listing 4, where Smodel and Qmodel parameterize and , respectively. For both models, we use num_hid = 1024 and embedding_dim = 64. The output dimension is num_out = 1 for Smodel, while for Qmodel it equals the dataset dimension (Table 7). We run Algorithm 1 for 50,000 training iterations with and . Both models are optimized with AdamW using learning rate , , and . We use 10,000 warmup steps and gradient clipping with maximum norm 0.1, and we maintain an exponential moving average (EMA) with rate 0.999. Batch sizes are dataset-dependent and are reported in Table 7.
The architectures used for PU learning with static subset alignment are given in Listing 5, where Tmodel and DualNET parameterize and , respectively. For both models, we use num_hid = 1024 and embedding_dim = 128. The output dimension for Tmodel matches the dataset dimension, while DualNET uses num_out = 1. We run Algorithm 2 for 20,000 training iterations with and . Both models are optimized with learning rate and use EMA with rate 0.999. Batch sizes are dataset-dependent and are reported in Table 7. For all PU-learning experiments, we fix .
| Dataset | data-set size | dimension | batch size | |
| Abalone | 4177 | 8 | 0.16 | 20 |
| Banknote | 1372 | 4 | 0.44 | 10 |
| Breast-w | 699 | 9 | 0.34 | 10 |
| Diabetes | 768 | 8 | 0.35 | 6 |
| Haberman | 306 | 3 | 0.26 | 6 |
| Heart | 270 | 13 | 0.44 | 6 |
| Ionosphere | 351 | 34 | 0.64 | 6 |
| Isolet | 7797 | 617 | 0.04 | 4 |
| Jm1 | 10885 | 21 | 0.19 | 20 |
| Kc1 | 2109 | 21 | 0.15 | 20 |
| Madelon | 2600 | 500 | 0.5 | 20 |
| Musk | 6598 | 166 | 0.15 | 20 |
| Segment | 2310 | 19 | 0.14 | 20 |
| Semeion | 1593 | 256 | 0.1 | 4 |
| Sonar | 208 | 60 | 0.53 | 4 |
| Spambase | 4601 | 57 | 0.39 | 20 |
| Vehicle | 846 | 18 | 0.26 | 6 |
| Waveform | 5000 | 40 | 0.34 | 20 |
| Wdbc | 569 | 30 | 0.37 | 6 |
| Yeast | 1484 | 8 | 0.31 | 10 |
Table 7. UCI datasets for PU learning, along with the total number of data points (n), dimension (dim), positive prior () and batch sizes employed in training the corresponding dynamic models.
Image-to-Image Translation on FFHQ
For dynamic subset alignment, the architectures for the networks parameterizing and are given in Listing 6, where Smodel and Qmodel parameterize and respectively. Both networks use t_embed_dim = 128 and num_hid = 1024, with num_out = 1 for the network and num_out = 512 for the network (matching the FFHQ latent dimensionality). We use AdamW with a fixed learning rate of , gradient clipping with maximum norm 1.0, EMA rate 0.999, and batch size 128. We run Algorithm 1 for 500,000 learning iterations with and .
For static subset alignment, the neural network architectures TransportNET and DualNET given in Listing 7 parameterize and , respectively. Both networks use hidden_dim = 1024. TransportNET maps 512-dimensional FFHQ latents to 512-dimensional outputs, while DualNET returns a scalar. We use AdamW with a fixed learning rate of , weight decay , EMA rate 0.999, and batch size 1,024. We run Algorithm 2 for 50,000 learning iterations with and . The same architectures are used for the UOTM experiments in Table 4.
Melanoma Experiment
Unlike the other experiments in this paper, the melanoma setting involves multiple intermediate marginals that must be matched. Because we want to preserve all marginals except the terminal one, we generalize equation (8) to the multi-marginal setting. Assuming we can sample from the marginals at times and , for we define and use the interpolant
import jax
from jax import numpy as jnp
from flax import linen as nn
import math
from ml_collections import config_dict
def get_timestep_embedding(timesteps, embedding_dim: int, max_positions: float = 10_000.0):
t = jnp.asarray(timesteps)
t = t.reshape((-1,)) if t.ndim == 1 else t.reshape((t.shape[0], -1))[:, 0]
t = t.astype(jnp.float32)
half = embedding_dim // 2
if half < 1:
return jnp.zeros((t.shape[0], embedding_dim), dtype=t.dtype)
freqs = jnp.exp(-jnp.log(jnp.array(max_positions,dtype=t.dtype))
* jnp.arange(half, dtype=t.dtype) / (half - 1 + 1e-8))
args = t[:, None] * freqs[None, :] # (B, half)
emb = jnp.concatenate([jnp.cos(args), jnp.sin(args)], axis=-1) # (B, 2*half)
if embedding_dim % 2 == 1:
emb = jnp.pad(emb, ((0, 0), (0, 1)))
return emb[:, :embedding_dim]
class Smodel(nn.Module):
config: config_dict.ConfigDict
@nn.compact
def __call__(self, t, x, *, train: bool = True):
config = self.config
time_emb_dim = config.embedding_dim
num_hid = config.num_hid
num_out = config.num_out
x = jnp.asarray(x)
if x.ndim == 1: # (D,) -> (1,D)
x = x[None, :]
# make t broadcastable to x batch: (B,1)
t = jnp.asarray(t)
t = jnp.broadcast_to(t, x.shape[:-1] + (1,))
temb = get_timestep_embedding(t, time_emb_dim) # should return (B, time_emb_dim)
temb = nn.swish(nn.Dense(num_hid)(temb))
temb = nn.Dense(num_hid)(temb) # (B, num_hid)
h = jnp.concatenate([t, x], axis=-1)
h = nn.swish(nn.Dense(num_hid)(h))
h = h + temb
for _ in range(3):
h = nn.swish(nn.Dense(num_hid)(h))
return nn.Dense(num_out)(h)
class Qmodel(nn.Module):
config: config_dict.ConfigDict
@nn.compact
def __call__(self, t, x_0, x_1, *, train: bool = True):
config = self.config
time_emb_dim = config.embedding_dim
num_hid = config.num_hid
num_out = config.num_out
x_0 = jnp.asarray(x_0)
x_1 = jnp.asarray(x_1)
if x_0.ndim == 1: # (D,) -> (1,D)
x_0 = x_0[None, :]
x_1 = x_1[None, :]
# broadcast t to (B,1) to match x batch
t = jnp.asarray(t)
t = jnp.broadcast_to(t, x_0.shape[:-1] + (1,)) # (B,1)
# time embedding -> project to hidden width
temb = get_timestep_embedding(t, time_emb_dim) # (B, time_emb_dim)
temb = nn.swish(nn.Dense(num_hid)(temb))
temb = nn.Dense(num_hid)(temb) # (B, num_hid)
heaviside_step = (t < 0.5).astype(x_0.dtype) # (B,1)
h = jnp.concatenate([t, x_0, x_1, heaviside_step], axis=-1)
h = nn.swish(nn.Dense(num_hid)(h))
h = h + temb
for _ in range(3):
h = nn.swish(nn.Dense(num_hid)(h))
h = nn.Dense(num_out)(h)
x_t = (1.0 - t) * x_0 + t * (x_1 + h)
return x_t
Listing 4: Model architectures for PU-Learning for dynamic subset alignment
import jax
from jax import numpy as jnp
from flax import linen as nn
import math
from ml_collections import config_dict
class etamodel(nn.Module):
num_hid: int
num_out: int
@nn.compact
def __call__(self, x):
h = x
for _ in range(3):
h = nn.swish(nn.Dense(self.num_hid)(h))
return nn.Dense(self.num_out)(h)
class Tmodel(nn.Module):
num_hid: int
num_out: int
@nn.compact
def __call__(self, x):
h = x
for _ in range(4):
h = nn.swish(nn.Dense(self.num_hid)(h))
h = nn.Dense(self.num_out)(h)
residual = nn.Dense(self.num_out)(x)
return h + residual
Listing 5: Model architectures for PU learning with static subset alignment
import jax
from jax import numpy as jnp
from flax import linen as nn
import math
def get_timestep_embedding(t, dim, max_positions=10000):
t = jnp.reshape(t, (-1,))
half = dim // 2
scale = jnp.exp(
jnp.arange(half, dtype=jnp.float32)
* (-math.log(max_positions) / (half - 1))
)
emb = t[:, None] * scale[None, :]
emb = jnp.concatenate([jnp.sin(emb), jnp.cos(emb)], axis=1)
return jnp.pad(emb, [(0, 0), (0, dim % 2)])
class Smodel(nn.Module):
num_hid: int
num_out: int
t_embed_dim: int
@nn.compact
def __call__(self, t, x):
if t.ndim == 0:
t = jnp.broadcast_to(t, (x.shape[0], 1, 1))
e = get_timestep_embedding(t, self.t_embed_dim)
e = jnp.reshape(e, x.shape[:-1] + (self.t_embed_dim,))
h = jnp.concatenate([e, x], axis=-1)
for i in range(4):
h = nn.swish(nn.Dense(self.num_hid)(h))
if i < 3:
h = jnp.concatenate([e, h], axis=-1)
return nn.Dense(self.num_out)(h)
class Qmodel(nn.Module):
num_hid: int
num_out: int
t_embed_dim: int
@nn.compact
def __call__(self, t, x_0, x_1):
e = get_timestep_embedding(t.squeeze(), self.t_embed_dim)
h = jnp.concatenate([e, x_0, x_1, t < 0.5], axis=-1)
for i in range(4):
h = nn.swish(nn.Dense(self.num_hid)(h))
if i < 3:
h = jnp.concatenate([e, h], axis=-1)
h = nn.Dense(self.num_out)(h)
return (1.0 - t) * x_0 + t * (x_1 + h)
Listing 6: Model architectures for FFHQ dynamic subset alignment.

Figure 7 (PDF p. 30). Model architectures for FFHQ static subset alignment.
where
where is a neural network shared across segments. Since vanishes at for all and at for , the interpolant matches exactly, while leaves the terminal marginal unconstrained: . For , this reduces to , which, like , fixes while relaxing the terminal marginal. In addition, we use the Wasserstein gradient-flow updates of Neklyudov et al. [90] to improve stability and convergence.
The Algorithm 3 summarizes the training procedure for the melanoma experiment. The Neural Network Architectures employed in Melanoma experiments are provided in Listing 8. We should emphasize that the interpolation equation 59 has been implemented as part of interpolant neural network architecture, but for clear description of learning procedure, we included interpolation equation a part of learning algorithm.
We use the first 50 principal components of the melanoma data [18], centered and divided by the largest per-dimension standard deviation of the training set. The three observed populations are ordered in reverse time, (48h), (24h) and (8h), at normalised times , and (), so that the sub-setting constraint acts on the . The potential is parameterized by a Smodel in Listing 8 with $n_f = 512 and activations ; the time is mapped by a two-layer embedding network of width 512 and added to the input of each of the three inner hidden layers. The interpolant network is parameterized by Qmodel in Listing 8, which takes concatenation as input and has a single hidden layer of width 512 with swish activation and a linear output in . Both networks are trained with AdamW , weight decay at learning rate with a linear warm-up over the first 5,000 iterations and element-wise gradient clipping at 1, using batch size , update per iteration, Wasserstein gradient steps with step size , and 200,000 iterations; the times are drawn from a low-discrepancy sequence on . We report results for sub-setting constants in table 5. For evaluation we use an exponential moving average of the parameters with rate 0.999, and transport cells with the learned velocity field by integrating the ODE from back to with the explicit Euler scheme and step size .
Algorithm 3: Modified-Dynamic-Neural-SS: Modified Learning Algorithm for Dynamic Subset Alignment on Melanoma
Require: Samples from distributions observed at times , time-dependent neural network , interpolant network , sub-setting constant , batch size , number of updates and , gradient clipping parameter , number of Wasserstein gradient steps with step size , gradient-based optimizers and .
Ensure: Sample-based neural estimate for
for learning iteration do
for do
Sample for , and .
where , ,
for , and
for do
end for
Using and to update via gradient descent.
end for
for do
Sample for , and .
Update using and via gradient ascent.
end for
end for
Confusion matrices for MNIST to EMNIST domain translation
import jax
from jax import numpy as jnp
from flax import linen as nn
import math
class Smodel(nn.Module):
config: ml_collections.ConfigDict
@nn.compact
def __call__(self, t: jnp.ndarray, x: jnp.ndarray, train: bool):
config = self.config
act = get_act(config)
nf = config.nf
if config.embed_time:
temb = t
temb = nn.Dense(nf)(temb)
temb = nn.Dense(nf)(act(temb))
h = x
else:
h = jnp.hstack([x, t])
h = act(nn.Dense(nf)(h))
for _ in range(config.n_layers):
if config.embed_time:
h += temb
h = nn.Dropout(config.dropout)(h, deterministic=not train)
if config.skip:
h = act(nn.Dense(nf)(h)) + h
else:
h = act(nn.Dense(nf)(h))
h = act(nn.Dense(nf)(h))
h = nn.Dense(config.input_dim)(h)
return (h*x).sum(1, keepdims=True)
class Qmodel(nn.Module):
config: ml_collections.ConfigDict
@nn.compact
def __call__(self,t:jnp.ndarray,batch:jnp.ndarray,train:bool):
config=self.config
act=get_act(config)
nf=config.nf
timesteps,x=batch
B=x.shape[0]
n_marginals=timesteps.shape[1]
t_reshaped=t.reshape(-1,1,1)
t_right=(timesteps<t_reshaped).sum(1)
t_right=jnp.fmax(t_right,jnp.ones_like(t_right).astype(int))
t_left=t_right-1
batch_idx=jnp.arange(B)
x_left=x[batch_idx,t_left.ravel(),:]
x_right=x[batch_idx,t_right.ravel(),:]
t_0=timesteps[batch_idx,t_left.ravel(),:]
t_1=timesteps[batch_idx,t_right.ravel(),:]
temb=layers.get_timestep_embedding(t.ravel(),nf)
temb=nn.Dense(nf)(temb)
temb=nn.Dense(nf)(act(temb))
h=jnp.concatenate([x_left,x_right],axis=1)
h=jnp.hstack([t,h])
if config.indicator:
h=jnp.hstack([h,t<0.5*(t_0+t_1)])
h=act(nn.Dense(nf)(h))
for _ in range(config.n_layers):
h=nn.Dropout(config.dropout)(h,deterministic=not train)
h=act(nn.Dense(nf)(h))+h if config.skip else act(nn.Dense(nf)(h))
h=nn.Dense(config.input_dim)(h)
alpha=(t-t_0)/(t_1-t_0)
beta=(t_1-t)/(t_1-t_0)
w_inner=1.0-alpha**2-beta**2
w_last=1.0-beta**2
w=jnp.where(t_right==(n_marginals-1),w_last,w_inner)
if config.get('pin_last',False):
w=w_inner
return beta*x_left+alpha*x_right+w*h
Listing 8: Model architectures for Melanoma Experiment.

Figure 11. (a) Normalized and (b) unnormalized confusion matrices for the EMNIST classifier discussed in § 4.2.

Figure 12. Confusion matrices for MNISTEMNIST domain translation using static subset selection. Accuracy is computed as the ratio between the trace and the sum of all entries in the confusion matrices.

Figure 13. Confusion matrices for MNIST→EMNIST domain translation using dynamic subset selection.
t-SNE embeddings of UCI datasets

Figure 14. The t-SNE embeddings for (a) Abalone, (b) Banknote, (c) Breast-w, (d) Diabetes, (e) Haberman, (f) Heart, (g) Ionosphere, (h) Isolet, (i) Kc1, (j) Madelon, (k) Musk, (l) Semeion, (m) Sonar, (n) Spambase, (o) Vehicle, (p) Waveform, (q) Wdbc, (r) Yeast, (s) Jm1, and (t) Segment datasets.
Results from FFHQ

(a)

(b)
Figure 15. FFHQ youngold translation using (a) static and (b) dynamic subset selection. Dynamic subset selection is evaluated using Euler integration with 100 steps.

(a)

(b)
Figure 16. FFHQ womanman translation using (a) static and (b) dynamic subset selection. Dynamic subset selection is evaluated using Euler integration with 100 steps.

(a)

(b)
Figure 17. FFHQ manwoman translation using (a) static and (b) dynamic subset selection. Dynamic subset selection is evaluated using Euler integration with 100 steps.
References
- [1]Michael S. Albergo and Eric Vanden-Eijnden. Building Normalizing Flows with Stochastic Interpolants. In The Eleventh International Conference on Learning Representations, 2023. [paper].arxiv.org/abs/2209.15571
- [2]Michael S. Albergo, Nicholas M. Boffi, and Eric Vanden-Eijnden. Stochastic Interpolants: A Unifying Framework for Flows and Diffusions. Journal of Machine Learning Research, 26(209):1–80, 2025. [paper].
- [3]Fabian Altekrüger, Johannes Hertrich, and Gabriele Steidl. Neural Wasserstein Gradient Flows for Discrepancies with Riesz Kernels. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 664–690. PMLR, 23–29 Jul 2023. [paper].arxiv.org/abs/2301.11624
- [4]David Alvarez-Melis, Yair Schiff, and Youssef Mroueh. Optimizing Functionals on the Space of Probabilities with Input Convex Neural Networks. Transactions on Machine Learning Research, 2022. ISSN 2835-8856. [paper].
- [6]Brandon Amos, Lei Xu, and J Zico Kolter. Input Convex Neural Networks. In International Conference on Machine Learning, pp. 146–155. PMLR, PMLR, 2017. [paper].arxiv.org/abs/1609.07152
- [7]Brandon Amos, Giulia Luise, Samuel Cohen, and Ievgen Redko. Meta Optimal Transport. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 791–813. PMLR, 23–29 Jul 2023. [paper].
- [8]Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein Generative Adversarial Networks. In International Conference on Machine Learning, pp. 214–223. PMLR, 2017. [paper].
- [10]Lazar Atanackovic, Xi Zhang, Brandon Amos, Mathieu Blanchette, Leo J Lee, Yoshua Bengio, Alexander Tong, and Kirill Neklyudov. Meta Flow Matching: Integrating Vector Fields on the Wasserstein Manifold. In The Thirteenth International Conference on Learning Representations, 2025. [paper].arxiv.org/abs/2408.14608
- [11]Julio Backhoff-Veraguas, Mathias Beiglböck, and Gudmun Pammer. Existence, duality, and cyclical monotonicity for weak transport costs. Calculus of Variations and Partial Differential Equations, 58(6):203, 2019. [paper].
- [12]Jessa Bekker and Jesse Davis. Learning from positive and unlabeled data: a survey. Machine Learning, 109:719–760, 2020. [paper].
- [13]Jean-David Benamou and Yann Brenier. A computational fluid mechanics solution to the Monge-Kantorovich mass transfer problem. Numerische Mathematik, 84(3):375–393, 2000. [paper].DOI
- [14]Valentin De Bortoli, Iryna Korshunova, Andriy Mnih, and Arnaud Doucet. Schrödinger Bridge Flow for Unpaired Data Translation. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. [paper].arxiv.org/abs/2409.09347
- [15]Yann Brenier. Polar factorization and monotone rearrangement of vector-valued functions. Communications on Pure and Applied Mathematics, 44(4):375–417, 1991. [paper].DOI
- [16]Charlotte Bunne, Andreas Krause, and marco cuturi. Supervised Training of Conditional Monge Maps. In Advances in Neural Information Processing Systems, 2022a. [paper].arxiv.org/abs/2206.14262
- [17]Charlotte Bunne, Laetitia Papaxanthos, Andreas Krause, and Marco Cuturi. Proximal Optimal Transport Modeling of Population Dynamics. In Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, volume 151 of Proceedings of Machine Learning Research, pp. 6511–6528. PMLR, 28–30 Mar 2022. [paper].arxiv.org/abs/2106.06345
- [18]Charlotte Bunne, Stefan G Stark, Gabriele Gut, Jacobo Sarabia Del Castillo, Mitch Levesque, Kjong-Van Lehmann, Lucas Pelkmans, Andreas Krause, and Gunnar Rätsch. Learning single-cell perturbation responses using neural optimal transport. Nature Methods, 20(11):1759–1768, 2023. [paper].
- [19]Brian Chao. Anime face dataset: a collection of high-quality anime faces., 2019. [dataset].
- [20]Laetitia Chapel, Mokhtar Z Alaya, and Gilles Gasso. Partial Optimal Transport with applications on Positive-Unlabeled Learning. Advances in Neural Information Processing Systems, 33:2903–2913, 2020. [paper].arxiv.org/abs/2002.08276
- [21]Ricky TQ Chen and Yaron Lipman. Flow Matching on General Geometries. In The Twelfth International Conference on Learning Representations, 2024. [paper].
- [22]Tianrong Chen, Guan-Hорng Liu, and Evangelos Theodorou. Likelihood training of schrödinger bridge using forward-backward SDEs theory. In International Conference on Learning Representations, 2022. [paper].arxiv.org/abs/2110.11291
- [23]Xiuyuan Cheng, Jianfeng Lu, Yixin Tan, and Yao Xie. Convergence of Flow-Based Generative Models via Proximal Gradient Descent in Wasserstein Space. IEEE Transactions on Information Theory, 70(11):8087–8106, 2024. [paper].DOI
- [24]Lenaic Chizat, Gabriel Peyré, Bernhard Schmitzer, and François-Xavier Vialard. An Interpolating Distance Between Optimal Transport and Fisher–Rao Metrics. Foundations of Computational Mathematics, 18:1–44, 2018a. [paper].
- [26]Jaemoo Choi, Jaewoong Choi, and Myungjoo Kang. Generative Modeling through the Semi-dual Formulation of Unbalanced Optimal Transport. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. [paper].arxiv.org/abs/2305.14777
- [27]Jaemoo Choi, Jaewoong Choi, and Myungjoo Kang. Analyzing and Improving Optimal-Transport-based Adversarial Networks. In The Twelfth International Conference on Learning Representations, 2024a. [paper].arxiv.org/abs/2310.02611
- [28]Jaemoo Choi, Jaewoong Choi, and Myungjoo Kang. Scalable Wasserstein Gradient Flow for Generative Modeling through Unbalanced Optimal Transport. In Forty-first International Conference on Machine Learning, 2024b. [paper].arxiv.org/abs/2402.05443
- [29]Gregory Cohen, Saeed Afshar, Jonathan Tapson, and André van Schaik. EMNIST: Extending MNIST to handwritten letters. In 2017 International Joint Conference on Neural Networks (IJCNN), pp. 2921–2926, 2017. [paper].
- [30]Li Deng. The MNIST Database of Handwritten Digit Images for Machine Learning Research. IEEE Signal Processing Magazine, 29(6):141–142, 2012. [paper].
- [31]Luca Eyring, Dominik Klein, Théo Uscidda, Giovanni Palla, Niki Kilbertus, Zeynep Akata, and Fabian J Theis. Unbalancedness in Neural Monge Maps Improves Unpaired Domain Translation. In The Twelfth International Conference on Learning Representations, 2024. [paper].arxiv.org/abs/2311.15100
- [32]Jiaojiao Fan, Shu Liu, Shaojun Ma, Yongxin Chen, and Hao-Min Zhou. Scalable Computation of Monge Maps with General Costs. In ICLR Workshop on Deep Generative Models for Highly Structured Data, 2022a. [paper].
- [33]Jiaojiao Fan, Qinsheng Zhang, Amirhossein Taghvaei, and Yongxin Chen. Variational Wasserstein gradient flow. In proceedings of International Conference on Machine Learning, 2022b. [paper].
- [34]Jiaojiao Fan, Shu Liu, Shaojun Ma, Hao-Min Zhou, and Yongxin Chen. Neural Monge Map estimation and its applications. Transactions on Machine Learning Research, 2023. [paper].
- [35]Jean Feydy, Thibault Séjourné, François-Xavier Vialard, Shun-ichi Amari, Alain Trouvé, and Gabriel Peyré. Interpolating between Optimal Transport and MMD using Sinkhorn Divergences. In The 22nd International Conference on Artificial Intelligence and Statistics, pp. 2681–2690. PMLR, 2019. [paper].arxiv.org/abs/1810.08278
- [36]Alessio Figalli and Federico Glaudo. An invitation to optimal transport, Wasserstein distances, and gradient flows: Second Edition. European Mathematical Society, 2023. [book].DOI
- [37]Nicolas Fournier and Arnaud Guillin. On the rate of convergence in Wasserstein distance of the empirical measure. Probability Theory and Related Fields, 162(3):707–738, 2015. [paper].
- [38]Charlie Frogner, Chiyuan Zhang, Hossein Mobahi, Mauricio Araya, and Tomaso Poggio. Learning with a wasserstein loss. In C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 28. Curran Associates, Inc., 2015. [paper].arxiv.org/abs/1506.05439
- [39]Wilfrid Gangbo and Robert J McCann. The geometry of optimal transportation. Acta Mathematica, 177:113–161, 1996. [paper].
- [40]Milena Gazdieva, Alexander Korotin, Daniil Selikhanovych, and Evgeny Burnaev. Extremal Domain Translation with Neural Optimal Transport. Advances in Neural Information Processing Systems, 36:40381–40413, 2023. [paper].arxiv.org/abs/2301.12874
- [41]Milena Gazdieva, Arip Asadulaev, Evgeny Burnaev, and Alexander Korotin. Light Unbalanced Optimal Transport. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. [paper].
- [42]Aude Genevay, Marco Cuturi, Gabriel Peyré, and Francis Bach. Stochastic optimization for large-scale optimal transport. In Advances in Neural Information Processing Systems, volume 29, 2016. [paper].arxiv.org/abs/1605.08527
- [43]Aude Genevay, Gabriel Peyré, and Marco Cuturi. Learning Generative Models with Sinkhorn Divergences. In International Conference on Artificial Intelligence and Statistics, pp. 1608–1617. PMLR, 2018. [paper].arxiv.org/abs/1706.00292
- [44]Aude Genevay, Lénaïc Chizat, Francis Bach, Marco Cuturi, and Gabriel Peyré. Sample Complexity of Sinkhorn Divergences. In The 22nd International Conference on Artificial Intelligence and Statistics, pp. 1574–1583. PMLR, 2019. [paper].
- [45]Zhengyang Geng, Mingyang Deng, Xingjian Bai, J Zico Kolter, and Kaiming He. Mean flows for one-step generative modeling. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2026. [paper].
- [46]Jonathan Geuter, Gregor Kornhardt, Ingimar Tomasson, and Vaios Laschos. Universal Neural Optimal Transport. In Forty-second International Conference on Machine Learning, 2025. [paper].
- [47]Nathael Gozlan, Cyril Roberto, Paul-Marie Samson, and Prasad Tetali. Kantorovich duality for general transport costs and applications. Journal of Functional Analysis, 273(11):3327–3405, 2017. [paper].
- [48]Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville. Improved Training of Wasserstein GANs. Advances in Neural Information Processing Systems, 30, 2017. [paper].
- [51]Doron Haviv, Aram-Alexandre Pooladian, Dana Pe’er, and Brandon Amos. Wasserstein Flow Matching: Generative Modeling Over Families of Distributions. In Forty-second International Conference on Machine Learning, 2025. [paper].arxiv.org/abs/2411.00698
- [52]Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep Residual Learning for Image Recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778, 2016. [paper].
- [53]Kurt Hornik. Approximation capabilities of multilayer feedforward networks. Neural Networks, 4(2):251–257, 1991. ISSN 0893-6080. doi: https://doi.org/10.1016/0893-6080(91)90009-T. [paper].
- [54]Bamdad Hosseini, Alexander W. Hsu, and Amirhossein Taghvaei. Conditional Optimal Transport on Function Spaces. SIAM/ASA Journal on Uncertainty Quantification, 13(1):304–338, 2025. [paper].
- [55]Samuel Howard, George Deligiannidis, Patrick Rebeschini, and James Thornton. Differentiable Cost-Parameterized Monge Map Estimators. In ICML 2024 Workshop on Differentiable Almost Everything: Differentiable Relaxations, Algorithms, Operators, and Simulators, 2024. [paper].arxiv.org/abs/2406.08399
- [56]Chin-Wei Huang, Ricky T. Q. Chen, Christos Tsirigotis, and Aaron Courville. Convex Potential Flows: Universal Probability Distributions with Optimal Transport and Convex Optimization. In International Conference on Learning Representations, 2021. [paper].arxiv.org/abs/2012.05942
- [57]Guillaume Huguet, Daniel Sumner Magruder, Alexander Tong, Oluwadamilola Fasina, Manik Kuchroo, Guy Wolf, and Smita Krishnaswamy. Manifold Interpolating Optimal-Transport Flows for Trajectory Inference. In Advances in Neural Information Processing Systems, 2022. [paper].
- [58]Richard Jordan, David Kinderlehrer, and Felix Otto. The Variational Formulation of the Fokker–Planck Equation. SIAM Journal on Mathematical Analysis, 29(1):1–17, 1998. [paper].
- [59]Leonid V Kantorovich. On the translocation of masses. Doklady Akademii Nauk SSSR (New Series), 37:199–201, 1942. [paper: english translation].
- [60]Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition, pp. 4401–4410, 2019. [paper].
- [61]Masahiro Kato, Takeshi Teshima, and Junya Honda. Learning from Positive and Unlabeled Data with a Selection Bias. In International Conference on Learning Representations, 2019. [paper].
- [63]Gavin Kerrigan, Giosue Migliorini, and Padhraic Smyth. Dynamic Conditional Optimal Transport through Simulation-Free Flows. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. [paper].arxiv.org/abs/2404.04240
- [64]Valentin Khrulkov, Gleb Ryzhakov, Andrei Chertkov, and Ivan Oseledets. Understanding DDPM Latent Codes Through Optimal Transport. In The Eleventh International Conference on Learning Representations, 2023. [paper].arxiv.org/abs/2202.07477
- [65]Boah Kim, Yan Zhuang, Tejas Sudharshan Mathai, and Ronald M Summers. OTMorph: Unsupervised Multi-domain Abdominal Medical Image Registration Using Neural Optimal Transport. IEEE Transactions on Medical Imaging, 2024. [paper].DOI
- [66]Diederik P. Kingma and Jimmy Ba. Adam: A Method for Stochastic Optimization, 2017. [paper].
- [67]Nikita Maksimovich Kornilov, Petr Mokrov, Alexander Gasnikov, and Alexander Korotin. Optimal Flow Matching: Learning Straight Trajectories in Just One Step. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. [paper].arxiv.org/abs/2403.13117
- [68]Alexander Korotin, Vage Egiazarian, Arip Asadulaev, Alexander Safin, and Evgeny Burnaev. Wasserstein-2 Generative Networks. In International Conference on Learning Representations, 2021a. [paper].
- [71]Alexander Korotin, Daniil Selikhanovych, and Evgeny Burnaev. Neural Optimal Transport. In The Eleventh International Conference on Learning Representations, 2023b. [paper].
- [72]Dohyun Kwon, Ying Fan, and Kangwook Lee. Score-based generative modeling secretly minimizes the wasserstein distance. In Advances in Neural Information Processing Systems, 2022. [paper].arxiv.org/abs/2212.06359
- [73]Marc Lambert, Sinho Chewi, Francis Bach, Silvère Bonnabel, and Philippe Rigollet. Variational inference via wasserstein gradient flows. In Advances in Neural Information Processing Systems, 2022. [paper].arxiv.org/abs/2205.15902
- [74]Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow Matching for Generative Modeling. In The Eleventh International Conference on Learning Representations, 2023. [paper].
- [77]Xingchao Liu, Chengyu Gong, and Qiang Liu. Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow. In The Eleventh International Conference on Learning Representations, 2023. [paper].arxiv.org/abs/2209.03003
- [78]Xingchao Liu, Xiwen Zhang, Jianzhu Ma, Jian Peng, and qiang liu. InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generation. In The Twelfth International Conference on Learning Representations, 2024b. [paper].arxiv.org/abs/2309.06380
- [79]Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaogang Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015. [dataset].
- [80]Ilya Loshchilov and Frank Hutter. SGDR: Stochastic Gradient Descent with Warm Restarts. In International Conference on Learning Representations, 2017. [paper].arxiv.org/abs/1608.03983
- [81]Frederike Lübeck, Charlotte Bunne, Gabriele Gut, Jacobo Sarabia del Castillo, Lucas Pelkmans, and David Alvarez-Melis. Neural Unbalanced Optimal Transport via Cycle-Consistent Semi-Couplings. In NeurIPS 2022 AI for Science: Progress and Promises, 2022. [paper].arxiv.org/abs/2209.15621
- [82]Shaojun Ma, Shu Liu, Hongyuan Zha, and Haomin Zhou. Learning Stochastic Behaviour from Aggregate Data. In International Conference on Machine Learning, pp. 7258–7267. PMLR, 2021. [paper].
- [83]Ashok Makkuva, Amirhossein Taghvaei, Sewoong Oh, and Jason Lee. Optimal transport mapping via input convex neural networks. In International Conference on Machine Learning, pp. 6672–6681. PMLR, 2020. [paper].
- [84]Robert J McCann. A Convexity Principle for Interacting Gases. Advances in Mathematics, 128(1):153–179, 1997. [paper].
- [85]Petr Mokrov, Alexander Korotin, Lingxiao Li, Aude Genevay, Justin Solomon, and Evgeny Burnaev. Large-Scale Wasserstein Gradient Flows. In Advances in Neural Information Processing Systems, 2021. [paper].arxiv.org/abs/2106.00736
- [86]Petr Mokrov, Alexander Korotin, Alexander Kolesov, Nikita Gushchin, and Evgeny Burnaev. Energy-guided Entropic Neural Optimal Transport. In The Twelfth International Conference on Learning Representations, 2024. [paper].arxiv.org/abs/2304.06094
- [87]Gaspard Monge. Mémoire sur la théorie des déblais et des remblais. Mémoires de Mathématique et de Physique, présentés à l’Académie Royale des Sciences, pp. 666–704, 1781. [paper].
- [88]Kirill Neklyudov, Rob Brekelmans, Daniel Severo, and Alireza Makhzani. Action Matching: Learning Stochastic Dynamics from Samples. In International Conference on Machine Learning, pp. 25858–25889. PMLR, 2023a. [paper].arxiv.org/abs/2210.06662
- [89]Kirill Neklyudov, Jannes Nys, Luca Thiede, Juan Carrasquilla, Qiang Liu, Max Welling, and Alireza Makhzani. Wasserstein Quantum Monte Carlo: A Novel Approach for Solving the Quantum Many-Body Schrödinger Equation. Advances in Neural Information Processing Systems, 36, 2023b. [paper].arxiv.org/abs/2307.07050
- [90]Kirill Neklyudov, Rob Brekelmans, Alexander Tong, Lazar Atanackovic, Qiang Liu, and Alireza Makhzani. A Computational Framework for Solving Wasserstein Lagrangian Flows. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 37461–37485. PMLR, 21–27 Jul 2024. [paper].arxiv.org/abs/2310.10649
- [91]XuanLong Nguyen, Martin J Wainwright, and Michael I Jordan. Estimating divergence functionals and the likelihood ratio by convex risk minimization. IEEE Transactions on Information Theory, 56(11):5847–5861, 2010. [paper].DOI
- [92]Sebastian Nowozin, Botond Cs e k e, and Ryota Tomioka. f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization. Advances in Neural Information Processing Systems, 29, 2016. [paper].arxiv.org/abs/1606.00709
- [93]Matteo Pariset, Ya-Ping Hsieh, Charlotte Bunne, Andreas Krause, and Valentin De Bortoli. Unbalanced diffusion schrodinger bridge. In ICML Workshop on New Frontiers in Learning, Control, and Dynamical Systems, 2023. [paper].
- [94]Stefano Peluchetti. Diffusion Bridge Mixture Transports, Schrödinger Bridge Problems and Generative Modeling. Journal of Machine Learning Research, 24(374):1–51, 2023. [paper].arxiv.org/abs/2304.00917
- [95]Abhijeet Phatak, Sharath Raghvendra, Chittaranjan Tripathy, and Kaiyi Zhang. Computing all Optimal Partial Transports. In The Eleventh International Conference on Learning Representations, 2023. [paper].
- [96]Stanislav Pidhorskyi, Donald A Adjeroh, and Gianfranco Doretto. Adversarial Latent Autoencoders. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14104–14113, 2020. [paper].
- [97]Yury Polynskiy and Yihong Wu. Information Theory: From Coding to Learning. Cambridge University Press, 2024. [book].
- [98]Aram-Alexandre Pooladian, Heli Ben-Hamu, Carles Domingo-Enrich, Brandon Amos, Yaron Lipman, and Ricky T Chen. Multisample Flow Matching: Straightening Flows with Minibatch Couplings. In International Conference on Machine Learning, 2023. [paper].arxiv.org/abs/2304.14772
- [99]Aram-Alexandre Pooladian, Carles Domingo-Enrich, Ricky T. Q. Chen, and Brandon Amos. Neural Optimal Transport with Lagrangian Costs. In The 40th Conference on Uncertainty in Artificial Intelligence, 2024. [paper].
- [100]Bilal Riaz, Yuksel Karahan, and Austin J. Brockmeier. Partial Optimal Transport for Support Subset Selection. Transactions on Machine Learning Research, 2023. [paper].
- [102]Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional Networks for Biomedical Image Segmentation. In Medical Image Computing and Computer-assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18, pp. 234–241. Springer, 2015. [paper].
- [103]Litu Rout, Alexander Korotin, and Evgeny Burnaev. Generative Modeling with Optimal Transport Maps. In International Conference on Learning Representations, 2022. [paper].
- [104]Filippo Santambrogio. Optimal Transport For Applied Mathematians. Birkäuser, NY, 55(58-63):94, 2015. [book].
- [106]Christopher Scarvelis and Justin Solomon. Riemannian Metric Learning via Optimal Transport. In The Eleventh International Conference on Learning Representations, 2023. [paper].
- [107]Vivien Seguy, Bharath Bhushan Damodaran, Remi Flamary, Nicolas Courty, Antoine Rolet, and Mathieu Blondel. Large Scale Optimal Transport and Mapping Estimation. In International Conference on Learning Representations, 2018. [paper].
- [108]Thibault Séjourné, Gabriel Peyréa, and François-Xavier Vialard. Unbalanced Optimal Transport, from theory to numerics. Numerical Control: Part B, pp. 407, 2023. paper.
- [109]Yuyang Shi, Valentin De Bortoli, Andrew Campbell, and Arnaud Doucet. Diffusion Schrödinger Bridge Matching. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. [paper].
- [110]Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep Unsupervised Learning using Nonequilibrium Thermodynamics. In Proceedings of the 32nd International Conference on Machine Learning, volume 37 of Proceedings of Machine Learning Research, pp. 2256–2265, Lille, France, 07–09 Jul 2015. PMLR. [paper].
- [112]Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising Diffusion Implicit Models. In International Conference on Learning Representations, 2021a. [paper].
- [113]Yang Song and Stefano Ermon. Improved Techniques for Training Score-Based Generative Models. Advances in Neural Information Processing Systems, 33:12438–12448, 2020. [paper].
- [114]Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-Based Generative Modeling through Stochastic Differential Equations. In International Conference on Learning Representations, 2021b. [paper].
- [115]Paweł Teisseyre, Timo Martens, Jessa Bekker, and Jesse Davis. Learning from biased positive-unlabeled data via threshold calibration. In The 28th International Conference on Artificial Intelligence and Statistics, 2025. [paper].
- [116]Alexander Tong, Jessie Huang, Guy Wolf, David Van Dijk, and Smita Krishnaswamy. TrajectoryNet: A Dynamic Optimal Transport Network for Modeling Cellular Dynamics. In International conference on machine learning, pp. 9526–9536. PMLR, 2020. [paper].
- [117]Alexander Tong, Kilian Fatras, Nikolay Malkin, Guillaume Huguet, Yanlei Zhang, Jarrid Rector-Brooks, Guy Wolf, and Yoshua Bengio. Improving and generalizing flow-based generative models with minibatch optimal transport. Transactions on Machine Learning Research, 2024a. ISSN 2835-8856. [paper].