betaFeedback

WorksMathematics

ENFR: Evidence Networks with Flow-Matched References for Amortized Simulation-Based Model Inference

Abstract

The evidence, Z=p(x∣M)\mathcal{Z} = p(x | \mathcal{M}), of a data-generating model M\mathcal{M} is a powerful tool in Bayesian frameworks: it enables model inference, model misspecification tests, and goodness-of-fit assessment as an alternative to standard discrepancy statistics, e.g. χ2χ^2. While versatile, the evidence is notoriously difficult to compute, even when the likelihood is available. In both likelihood-based and simulation-based inference, estimating Z\mathcal{Z} provides an immediate solution to high-dimensional goodness-of-fit or model misspecification tests: using Z(x)\mathcal{Z}(x) as the test statistic induces a minimum-volume level-αα test. Such tests are essential to assess model fidelity, which is necessary to trust simulation-based parameter inference. Existing learned approaches largely target Bayes factors, which only measure evidence ratios. Instead, we provide accurate, fast, and amortized estimates of the absolute evidence. We present ENFR: Evidence Networks with Flow-matched References. Building on the Evidence Network framework of Jeffrey & Wandelt (2024), the proposed method first learns Bayes factors with respect to a flow-matched reference, from which the absolute evidence is recovered. We show improvement in estimation of Z\mathcal{Z} over the flow-based baseline in all applications considered and the best performance of all density estimation methods considered in our benchmarks. We furthermore introduce and provide the proof for an architecture that jointly learns the individual evidences of a set of models {Mi}\{\mathcal{M}_i\}. We demonstrate the performance and scaling behavior of these approaches on data from a variety of scientific applications.

Introduction

Science is fundamentally tasked with modeling nature from first principles, and increasingly this modeling is done with high-fidelity simulators that produce samples from a likelihood that cannot itself be evaluated (e.g. [2] (Mesinger et al. 2011; Villaescusa-Navarro et al. 2021)). Simulation-based inference (SBI) has changed how such models are confronted with data: instead of requiring a tractable likelihood, it enables likelihood-free inference directly from simulator samples [4, 6, 11] (Papamakarios & Murray 2016). These methods have already unlocked information in the data that was previously inaccessible (e.g. von Wietersheim-Kramsta et al., 2025; Thomsen et al., 2026, for recent scientific applications), and are being readily adopted across scientific disciplines.

The focus of SBI so far has been parameter estimation. What we ultimately care about in science, however, is assessing the relative fits of models when confronted to data. In a Bayesian framework this is measured by the evidence p(x∣M)p(x \mid\mathcal{M}), which is the probability that the observed data xx has been produced by a physical or statistical model M\mathcal{M}. In parameterized settings it is common to express the evidence as the marginal likelihood using the model parameters θ\theta:

p(x∣M)=∫p(x∣θ,M)p(θ∣M)dθ,(1)p(x \mid\mathcal{M}) = \int p(x \mid\theta,\mathcal{M})p(\theta\mid\mathcal{M})d\theta, \tag*{(1)}

where p(θ∣M)p(\theta\mid\mathcal{M}) denotes the prior on the parameters and p(x∣θ,M)p(x \mid\theta,\mathcal{M}) is the model likelihood. The evidence serves this purpose in two ways. For model comparison, we are interested in the Bayes factor log⁡K=log⁡p(x∣M)p(x∣M0)\log K = \log\frac{p(x \mid\mathcal{M})}{p(x \mid\mathcal{M}_0)} with respect to some reference model M0\mathcal{M}_0. Beyond model comparison, the absolute evidence enables powerful tests of the data such as model misspecification tests, and an alternative to χ2\chi^2 goodness-of-fit tests in high dimensions that does not require summary statistics.

Estimating the evidence has not been the focus of SBI, and the methods to do so are correspondingly less developed. Traditionally, computing the evidence has been a daunting task as the naive estimation of the integral is quickly intractable in high dimensions. Computationally heavy methods such as nested sampling (e.g. [13] (Handley et al. 2015; Speagle 2020)) have been developed for these cases, but they require the ability to evaluate the likelihood, and even then have seen limited use in scientific applications due to computational infeasibility. How to evaluate the model evidence and Bayes factors when the likelihood is intractable but a simulator is available has remained a largely underexplored topic.

Jeffrey & Wandelt (2024) presented the application of neural network based density ratio estimators to learn Bayes factors in this regime. We build on this previous work and introduce ENFR, Evidence Networks with Flow-matched References, which combine flow-based models and evidence networks: using a generative flow as the reference model, we obtain not only Bayes factors but precise estimates of the absolute evidence. We furthermore introduce a new loss function to jointly estimate the Bayesian evidence and Bayes factors of multiple models in § 3.

Alongside the method, we introduce an evaluation framework built on scientifically relevant applications from a range of disciplines in § 4. It includes cases with a known likelihood, where the evidence can be estimated with nested sampling and our estimates can therefore be tested accurately against a ground truth, as well as simulation-based cases where only the calibration of the Bayes factors can be assessed. Across these applications, ENFR performs best of all learned estimators considered.

Related work

Generative models for evidence estimation Flow based generative models can both obtain samples from the target distribution and evaluate its density. In recent years, generative models such as flow matching Lipman et al., 2023 or score-based diffusion Song & Ermon, 2019; Song et al., 2021 have been successfully pushed into very high-dimensional regimes. However, their primary success criterion is often sample quality. An exact evaluation of the density is often not an explicit goal and its quality is not a main concern. In cases where the estimated density is evaluated as a metric, it is usually maximized, without a “true” value to aim for or any interpretable measure for its quality. For evidence estimation, however, the evaluation of the density is the final goal without any concerns of sample quality. Given the optimization objective in those two generative models, the velocity field and the score, it seems clear that these models do not prioritize the evaluation of the likelihood.

Flow matching In this work we consider flow matching Lipman et al. (2023), learning the velocity field vϕ=dxdtv_{\phi} = \frac{dx}{dt} when mapping from samples of a known noise distribution z∼p(z)z \sim p(z) to data samples xx with learnable parameters ϕ\phi and tt parameterizing the path. Throughout this work we consider linear interpolants for the flow, i.e. xt=tx+(1−t)zx_t = tx + (1-t)z. The density can then be obtained via integration from the continuity equation (see Appendix § 8.§ 1).

Evidence Networks Given a reference model M0\mathcal{M}_0, one can use classifiers to learn a density ratio, or equivalently, the Bayes factor log⁡K\log K Jeffrey & Wandelt (2024). This can be achieved through a variety of loss functions Masnadi-shirazi & Vasconcelos (2008). Jeffrey & Wandelt (2024) recommend an exponential loss function which is given by:

L(x,m)=e(12−m)Jα(fψ(x))(2)\mathcal{L}(x,m) = e^{(\frac{1}{2}-m)J_{\alpha}(f_{\psi}(x))} \tag*{(2)}

where m∈{0,1}m \in\{0,1\} denotes the model label (e.g. m=1m = 1 if x∼p(x∣M)x \sim p(x|\mathcal{M}) and m=0m = 0 if x∼p(x∣M0)x \sim p(x|\mathcal{M}_0)) and fψf_{\psi} is a neural network with trainable parameters ψ\psi. JαJ_{\alpha} denotes the l-POP transform defined as Jα(x)=x+sgn⁡(x)∣x∣αJ_{\alpha}(x) = x + \operatorname{sgn}(x)|x|^{\alpha} which provides for more accurate computation of extreme evidence ratios than cross-entropy loss. We explicitly show the minimizer of this loss to be the Bayes factor in Appendix § 11. The hyperparameter α\alpha can be chosen freely but is recommended by Jeffrey & Wandelt (2024) to be set to α=2\alpha= 2. We follow this recommendation and will refer to this architecture as an evidence network (EN). As we are fundamentally interested in evidences as opposed to Bayes factors, we need to choose a reference model with known density to reconstruct an evidence:

log⁡qψ(x∣M)=log⁡Kψ+log⁡p(x∣M0)\log q_{\psi}(x|\mathcal{M}) = \log K_{\psi} + \log p(x|\mathcal{M}_0).

The choice of the reference model can be critical for good evidence estimation: evidence networks suffer from a well-known failure mode in the case where the distance between the two distributions p(x∣M)p(x|\mathcal{M}) and p(x∣M0)p(x|\mathcal{M}_0) is very large Rhodes et al. (2020). This is known in the literature as the density chasm problem. In the following we describe approaches to mitigate this issue.

Telescoping ratio estimation As a solution to the density chasm problem, an iterative procedure has been proposed by Rhodes et al. (2020), in which intermediate waymark distributions are constructed, to produce an array of reference distributions and to learn the Bayes factors (or equivalently, density ratios) between consecutive waymark distributions. Choosing a known noise density p(z)p(z), enables the reconstruction of the initial density. While Rhodes et al. (2020) have applied this also to energy based modelling with a flow-based noise distribution they do not consider controlled density-estimation benchmarks in which the reconstructed target density can be compared directly to its known normalized pointwise density. Furthermore our work here differs in architecture.

Time Score Matching (CTSM) A related continuous formulation is given by Choi et al. [10]; Yu et al. (2025); Chen et al. [8], in which neural networks learn the time score s(x,t)=∂tlog⁡p(xt)s(x,t) = \partial_t \log p(x_t). These models are in training similar to flow-matching, i.e. the target and noise distribution are connected with some interpolant. The learned time score can then be integrated to obtain a density estimate: log⁡q(x)=log⁡p(z)+∫01s(x,t) dt\log q(x) = \log p(z) + \int_0^1 s(x,t)\,dt. We train these models using the conditional time score matching (CTSM) function (for more details see Yu et al. (2025)) with either the variance preserving interpolant (VP) or the Schrödinger bridge interpolant (SB) (we show the interpolants in Appendix § 8.§ 3). We note that while these methods have been primarily formulated for density ratio estimation, we evaluate them here as density estimators since we are fundamentally interested in evidences rather than Bayes factors. In our experiments, we have found the variance of the target to be very high close to the data distribution and to be limiting for the performance of this method. We also note, that under imperfect training, the resulting probability density function is not guaranteed to be normalized as opposed to flow-based generative models. In our numerical experiments we compare with CTSM-VP, and CTSM-SB with a learned flow matching reference distribution, which we call CTSM-SB + FlowRef.

Featurized Density Ratio Estimation Another related approach is featurized density ratio estimation Choi et al. [9] which uses an invertible flow to learn a common feature space of samples from two densities and to approximate the density ratio in this feature space. Again our work differs as we are not only interested in density ratios, but also actual densities.

Flow contrastive estimation Gao et al. [14] propose flow contrastive estimation, which jointly trains an energy-based model and a normalizing flow through a noise-contrastive binary-classification objective, with the flow serving as an adaptive noise distribution. The flow is brought iteratively closer to the data distribution. While they do present density estimation results, they do not fully benchmark their density estimation for pointwise ground-truth examples beyond two dimensions. They furthermore show that maximizing the likelihood and improving sample quality do not necessarily go together. Their method differs from ours both in architectural setup and training method.

ENFR: Evidence Networks with Flow-Matched References

Evidence Networks with Flow-matched References: ENFR Given that ENs are a powerful method for the estimation of the evidence if a fitting reference model is present, we propose to learn such a reference using a generative model (see Figure 1). We choose flow matching for this task. The EN with learnable parameters ψ\psi then only has to learn a correction log⁡Kψ\log K_{\psi} from the flow estimate to the true underlying density by discriminating simulations from flow samples:

Figure 1. A visualization of our proposed SingleENFR and MultiENFR architectures. Given simulated data from one or multiple models, we train flow matching once over all models. In a second step, we use the simulated data and samples from the flow to train an evidence network classifier, which learns all Bayes factors. Those can be combined with the evaluated flow density to obtain the evidence under a given model.

log⁡p(x∣M)=log⁡qϕ(x∣M)+log⁡p(x∣M)qϕ(x∣M)≈log⁡qϕ(x∣M)+log⁡Kψ.(3)\log p(x|M) = \log q_{\phi}(x|M) + \log\frac{p(x|M)}{q_{\phi}(x|M)} \approx\log q_{\phi}(x|M) + \log K_{\psi}. \tag*{(3)}

In this setup, we can benefit from the fact that we only need accurate sampling from the flow to generate half of the training data for the evidence network. The flow density has to be evaluated only at inference time. We call this setup SingleENFR.

We show that when the flow is already a very good fit to the data and no informative density-ratio correction is available, the population-optimal constant correction is zero under balanced training in Appendix F. While we could use any evidence network loss function for the estimation of log⁡K\log K, we follow the recommendation of Jeffrey & Wandelt (2024) and choose l-POP. We note that in a similar fashion, one can use a pre-trained flow as the base distribution for CTSM using the SB bridge which has been used in [8]. We also compare to that case in our experiments, calling it CTSM-SB + FlowRef.

ENFR with multiple models (MultiENFR) We further introduce a more streamlined way to obtain an array of Bayesian evidences for a set of multiple models based on the l-POP loss function which we call MultiENFR (see Figure 1). Given NN such models, we introduce the pairwise loss:

L(x,m)=1N−1∑i≠me−12(Jα(fψ,m(x))−Jα(fψ,i(x))).(4)\mathcal{L}(x,\mathbf{m}) = \frac{1}{N-1}\sum_{i\ne m}e^{-\frac{1}{2}\left(\mathcal{J}_{\alpha}(f_{\psi,m}(x))-\mathcal{J}_{\alpha}(f_{\psi,i}(x))\right)}. \tag*{(4)}

Theorem 1 (Asymptotic recovery of Bayes factors). Consider N≥2N \ge2 models {Mi}i=1N\{\mathcal{M}_i\}_{i=1}^{N} with evidences pi(x)≡p(x∣Mi)p_i(x) \equiv p(x \mid\mathcal{M}_i), sampled with equal probability p(Mi)=1/Np(\mathcal{M}_i)=1/N. Let fψ,i(x)f_{\psi,i}(x) denote the ii-th output of the multiclass evidence network and let Jα\mathcal{J}_{\alpha} be a fixed scalar link function, with transformed outputs uψ,i(x)≡Jα(fψ,i(x))u_{\psi,i}(x) \equiv\mathcal{J}_{\alpha}(f_{\psi,i}(x)).

Suppose the network is trained with the pairwise exponential loss L(x,m)=1N−1∑j≠mexp⁡[−12(uψ,m(x)−uψ,j(x))]\mathcal{L}(x,\mathbf{m}) = \frac{1}{N-1}\sum_{j\ne m}\exp[-\frac{1}{2}(u_{\psi,m}(x)-u_{\psi,j}(x))] (Equation (4)). Assume that the densities pi(x)p_i(x) are positive almost everywhere on the common support, that the population risk is finite, that the transformed network class is sufficiently expressive to represent ui(x)=log⁡pi(x)+c(x)u_i(x)=\log p_i(x)+c(x) for some model-independent function c(x)c(x), and that asymptotic training reaches a global minimizer of the population risk.

Then every global population-risk minimizer satisfies ui∗(x)=log⁡p(x∣Mi)+c(x)u_i^*(x)=\log p(x\mid\mathcal{M}_i)+c(x) almost everywhere, where c(x)c(x) is independent of the model index. Consequently, for any pair of models Mi\mathcal{M}_i and Mj\mathcal{M}_j,

Jα(fψ∗,i(x))−Jα(fψ∗,j(x))=log⁡p(x∣Mi)p(x∣Mj)=log⁡Kij(x).(5)\mathcal{J}_{\alpha}(f_{\psi^*,i}(x))-\mathcal{J}_{\alpha}(f_{\psi^*,j}(x))=\log\frac{p(x\mid\mathcal{M}_i)}{p(x\mid\mathcal{M}_j)}=\log K_{ij}(x). \tag*{(5)}

Proof. See Appendix G.

In order to handle the density chasm problem, we introduce one more class, which is given by synthetic data samples from a flow qϕmultiq_{\phi}^{\mathrm{multi}} with trainable parameters ϕ\phi. The generating flow is pretrained on simulated data from all the other models: x∼1N∑n=1Np(x∣Mn)x \sim\frac{1}{N}\sum_{n=1}^{N}p(x\mid\mathcal{M}_n). We note that this is a fundamentally different flow density target than in SingleENFR, as the result is no longer an evidence estimate in itself. It is rather an estimate of the mixture of the evidences of many models. We use this flow reference as an additional model in our multiclass evidence network: there are therefore N+1N+1 classes to discriminate. We generate an equal share of training data from this pretrained flow. The learned Bayes factor with respect to the flow class along with the evaluated flow density then delivers the evidence log⁡p(x∣Mi)=log⁡Ki,N+1+log⁡qϕ(x)\log p(x\mid\mathcal{M}_i)=\log K_{i,N+1}+\log q_{\phi}(x).

Results

In this section we benchmark the performance of our proposed ENFR architecture and compare them to other learned evidence estimates on a variety of different scientific applications.

Analytical evidence: Gaussian mixture model

We begin with a Gaussian mixture model (GMM) where the true evidence is analytically known and we compare root mean square errors (RMSE) of our simulation-based estimates with respect to the ground truth. In the known ground truth case we generate data from a dd-dimensional correlated Gaussian mixture model (GMM) with three components. The details of the experimental setup are given in Appendix A.1.

Figure 2 shows that in our scaling tests the SingleENFR setup consistently outperforms the other tested models. We also find that CTSM is highly outperformed by all other methods. Using CTSM with a flow matched reference model, does not improve the flow significantly in all combinations tested here. A comparison of the RMSE values of the evidence estimates with respect to the ground truth is given in Table 11. We also find that our model learns only minor improvements starting at 512 dimensions.

Figure 2. Scaling of density-estimation accuracy with (a) data dimensionality and (b) number of training simulations on the Gaussian mixture model of Appendix § 7.§ 1, for which the true density is known analytically. We report the RMSE between estimated and true log-density over 200 held-out test points. In (a) the simulation budget is fixed at 5×1065 \times10^{6}; in (b) the dimensionality is fixed at 64. The line shows the mean across three trained models and the errorbars show the estimated standard deviation. SingleENFR (orange) improves on the flow it is built on (blue) at every point and is the most accurate of all estimators considered, while CTSM-VP (yellow) is consistently the least accurate.

Table 11: RMSE values of the estimated density log q(x)q(x) with respect to the true density log p(x)p(x) over 200 held out test draws (lower is better) averaged over the three seeds. The SingleENFR combination consistently outperforms the other models. The flow is shared at each seed.

ExampleFlowCTSM-VPSingleENFRCTSM-SB + FlowRef
GMM 64D0.762.140.451.06
GMM 128D2.307.701.322.46
GMM 256D6.6127.873.046.57
GMM 512D15.7965.3614.4415.87
GMM 1024D34.99163.2133.4435.01

Table 11.

Analytical likelihood applications

In this section we present examples where the true underlying likelihoods are known, such that the evidences can be estimated with established likelihood-based frameworks. We benchmark the models by comparing the simulation-based evidences to the likelihood-based estimates via the root mean square error (RMSE) of the log probabilities. For our likelihood-based estimates we run nested sampling using the dynesty library Speagle (2020).

Baryonic acoustic oscillations

We further test on a scientific example from cosmology. In the early Universe, sound waves propagated through the tightly coupled photon baryon plasma, driven by the competition between gravity and radiation pressure. When photons decoupled from baryons, these waves froze in and imprinted a characteristic length scale, the sound horizon rdr_{d}, on the correlations between cosmic structures. Baryonic acoustic oscillations (BAO) thus provide a standard ruler: measuring its apparent size along and across the line of sight at different redshifts constrains the expansion history H(z)H(z), and therefore the nature of the dark energy that drives the late-time accelerated expansion.

In the standard cosmological model, Λ\LambdaCDM, dark energy is a cosmological constant with a fixed equation of state w=−1w=-1. Whether this description is sufficient is currently an open question, and different extensions have been proposed: a constant but free equation of state (wwCDM), a time-evolving one parameterized as w(a)=w0+wa(1−a)w(a)=w_{0}+w_{a}(1-a) (w0waw_{0}w_{a}CDM), as well as one-parameter extensions that free the neutrino mass sum (ν\nuCDM) or the spatial curvature (Ωk\Omega_{k}CDM). Recent

BAO measurements from DESI Abdul Karim et al. [1] have renewed interest in dynamical dark energy, making a principled comparison of these models a timely application. We compare the evidences of these five models for BAO data vectors (more details in Appendix § 7.§ 2).

Table 1 shows the resulting evidences and Bayes factors on 100 held out test simulations for the different cosmological models. We find that our models SingleENFR and MultiENFR are consistently outperforming both the flow, and CTSM-SB with the flow reference. Table 2 shows the evidence estimates of our best model, MultiENFR, on the state-of-the-art measurement DESI DR2 (Abdul Karim et al., [1]). The estimates on the data agree on the order of the nested sampling error bars.

SimulationflowSingleENFR (ours)MultiENFR (ours)CTSM-SB + FlowRef
Λ\LambdaCDM1.340.530.522.74
wwCDM1.400.760.672.46
w0waw_{0}w_{a}CDM1.160.830.813.79
ν\nuCDM1.410.470.461.77
Ωk\Omega_{k}CDM1.520.540.782.29

Table 1. RMSE of our learned log-evidence estimates against the likelihood-based log-evidence estimate over 100 held out BAO test simulations (lower is better). Our Models SingleENFR and MultiENFR consistently outperform both the flow and CTSM-SB with a flow reference.

ModelLikelihood-based analysisMultiENFRΔ\DeltaMultiENFR log KK
Λ\LambdaCDM−13.99±0.16-13.99\pm0.16−14.07±0.34-14.07\pm0.34−0.08-0.080.000.00
wwCDM−16.41±0.18-16.41\pm0.18−16.00±0.35-16.00\pm0.350.400.40−1.94-1.94
w0waw_0w_aCDM−15.63±0.18-15.63\pm0.18−15.46±0.40-15.46\pm0.400.170.17−1.40-1.40
ν\nuCDM−14.06±0.16-14.06\pm0.16−14.01±0.30-14.01\pm0.300.040.040.060.06
Ωk\Omega_kCDM−15.88±0.17-15.88\pm0.17−15.72±0.23-15.72\pm0.230.160.16−1.65-1.65

Table 2. Evidence estimates using MultiENFR on the DESI DR2 observed data. For MultiENFR we also quote the estimate of the standard deviation over the ensemble of 5 members. We can reproduce the likelihood-based evidence estimates of the DESI DR2 data with our simulation-based framework within the ensemble scatter.

Time series from an ODE

In this example we apply our method to a 64 dimensional time series from an ordinary differential equation (ODE). The data consists of noisy measurements of the value x(t)x(t) at the 64 evenly spaced time steps. We consider the harmonic (β=0\beta= 0, γ=0\gamma= 0), damped(β=0\beta= 0, γ≠0\gamma\ne0), and Duffing (β≠0\beta\ne0, γ≠0\gamma\ne0) oscillators:

d2xdt2=−2γdxdt−ω2x−βx3.(6)\frac{\mathrm{d}^2x}{\mathrm{d}t^2} = -2\gamma\frac{\mathrm{d}x}{\mathrm{d}t} - \omega^2x - \beta x^3. \tag*{(6)}

At each time step we add noise ϵ∼N(0,(0.1)2)\epsilon\sim\mathcal{N}(0,(0.1)^2). The priors are given in Table 4.

ParameterPriorHarmonicDampedDuffing
x0x_{0}U(−2,2)U(-2, 2)✓✓✓
v0v_{0}U(−2,2)U(-2, 2)✓✓✓
ω\omegaU(0.4,2.5)U(0.4, 2.5)✓✓✓
γ\gammaU(0,0.4)U(0, 0.4)✓✓
β\betaU(−0.8,0.8)U(-0.8, 0.8)✓

Table 4. Prior distributions for the ODE oscillator models. All priors are independent uniforms; parameters shared between models have identical priors.

Figure 3a shows that our MultiENFR model outperforms all other models tested here. Both MultiENFR and SingleENFR evidence estimates are significantly more precise than those of the flow. We present our full results in Table 5, where we find again that the combination of CTSM-SB with a flow reference has the highest RMSE, i.e. the worst performance of the models tested.

Figure 3. (a) Evidence estimation for noisy observations of an ODE driven time series using flow matching and our ENFR models. MultiENFR outperforms the other models. (b) Diffusion drift application of standard flow matching and our ENFR networks. MultiENFR shows the best performance and good evidence estimation over a large dynamical range.

MethodharmonicdampedDuffingAll
flow6.133.693.914.71
SingleENFR2.542.442.602.53
MultiENFR2.360.942.171.93
flow + CTSM-SB (ens.)135.4726.237.9679.80

Table 5. RMSE of the learned evidence against nested sampling (dynesty) on the ODE oscillator benchmark, over 10 held-out simulations per model.

Diffusion drift SDE

We present an application from neuroscience, modeling the decision process of a brain. We integrate an SDE that collects information about a particular state of the world:

dx=(v+λx)dt+dW,(7)\mathrm{d}x = (v + \lambda x)\mathrm{d}t + \mathrm{d}W, \tag*{(7)}

where we differentiate between the constant drift (λ=0\lambda= 0) and leaky drift (λ<0\lambda< 0) models. The prior is chosen such that the modes are not nested and are shown in Table 6. We integrate until we hit a collapsing symmetric boundary B(t)=±Be−κtB(t) = \pm B e^{-\kappa t}. We record the reaction time of hitting this boundary as the result of the run, with ±\pm indexing whether the positive or negative boundary was hit. We also add the non-decision time as an additional parameter to the recorded decision time, which accounts for delays in the decision making process. For each parameter combination, we simulate 400 times and record the reaction times. The data vector is the set of those 400 times. In order to respect exchangeability, we use permutation equivariant / invariant neural networks for the flow and the EN / CTSM.

ParameterDescriptionPriorM0\mathcal{M}_0M1\mathcal{M}_1
vvdrift rateU(0,5)\mathcal{U}(0, 5)✓✓
λ\lambdaleak rateU(−20,−5)\mathcal{U}(-20, -5)✓
BBinitial boundary heightU(0.3,2)\mathcal{U}(0.3, 2)✓✓
κ\kappaboundary collapse rateU(0.5,1.5)\mathcal{U}(0.5, 1.5)✓✓
tndt_{\mathrm{nd}}non-decision time [s]U(0.1,0.3)\mathcal{U}(0.1, 0.3)✓✓

Table 6. Prior distributions for the drift-diffusion models.

We show our results for the flow, SingleENFR and MultiENFR in Figure 3b. We find that the flow alone can already produce precise estimates of the evidence for both the constant drift and the leaky drift model. Adding the SingleENFR correction however again improves the resulting estimates in RMSE. Our best model is MultiENFR, which produces the highest precision in evidence estimates.

Simulation-based applications

In this section we present examples where the true likelihood is not available, and therefore no likelihood-based estimate of the evidence can be compared to. As we have no way of assessing the evidence estimates in this case, we instead perform blind coverage testing to test the resulting Bayes factors (Zadrozny & Elkan, 2002; Niculescu-Mizil & Caruana, 2005; Jeffrey & Wandelt, 2024). Given the evidence estimate, we compute the model posteriors pi=p(Mi∣x)p_i = p(\mathcal{M}_i|x). For a given model Mi\mathcal{M}_i, we can then bin the model posterior predictions for a sample of test simulations in probability bins. Among simulations assigned posterior probability pip_i to model Mi\mathcal{M}_i, approximately a fraction pip_i should actually have been generated from that model. As we know the underlying ground truth model to each simulation we can perform such a test.

The Hodgkin-Huxley model

The Hodgkin-Huxley model is a biophysical model for the behavior of biological neurons. The neuron membrane is modeled as an electrical circuit and the observed quantity is the membrane voltage V(t)V(t), whose dynamics are given by an SDE:

CmdV=(Iext(t)−INa−IK−IL−z0IM−z1ICaL)dt+σdW,(8)C_m dV = \left(I_{\mathrm{ext}}(t) - I_{\mathrm{Na}} - I_{\mathrm{K}} - I_{\mathrm{L}} - z_0 I_{\mathrm{M}} - z_1 I_{\mathrm{CaL}}\right)dt + \sigma dW, \tag*{(8)}

where CmC_m is a capacitance, and II are currents induced by present elements. Each II except for ILI_{\mathrm{L}} and IextI_{\mathrm{ext}} are described by so-called gating variables which are defined by a system of coupled ODEs driven by the voltage, forming one coupled system. We consider four different models Mz0z1\mathcal{M}_{z_0z_1}, the first of which only contains the sodium (Na), potassium (K) and leak (L) contributions (M00\mathcal{M}_{00}). The other three models are extensions adding the slow potassium (M) (M10\mathcal{M}_{10}), and L-type calcium (CaL) (M01\mathcal{M}_{01}) contributions, or both (M11\mathcal{M}_{11}). We give more details in Appendix A.6. Figure 4 shows the blind calibration test for our MultiENFR model. We find that the resulting Bayes factors are well calibrated.

Figure 4. Blind coverage testing for the simulation-based applications for (a) the Hodgkin-Huxley application, and (b) the Galaxy assembly bias application. As we do not have access to a likelihood-based evidence estimate for our simulation based models we test whether the resulting Bayes factors are well calibrated. We find that our MultiENFR architecture is well calibrated in both cases. The error bars show 2σ2\sigma binomial errors.

Galaxy assembly bias

In cosmology, we observe the 3D distribution of galaxies in the Universe as a biased tracer of the underlying dark matter density field. However, theoretical predictions can only accurately model the dark matter distribution. It is therefore of major interest to model the connection between dark matter and galaxies, the so-called bias, and understand what are its major drivers. We compare two bias models using 362-dimensional summary statistics from simulated galaxy catalogs: a mass-only model and an extension with assembly and phase-space bias. Details and parameter priors are given in Appendix § 7.7. Figure 4b shows that the MultiENFR model is well calibrated and could be used to determine the underlying bias model from observations in a likelihood-free way.

Fast evidences in action: hypothesis testing

As all of the models presented here can estimate evidences extremely fast once they are trained, we can use the evidence in traditionally intractable ways. For example, we can turn the evidence into a test statistic and run a hypothesis test [5]. We conduct such an analysis by drawing 1000 samples from the prior predictive of our baryonic acoustic oscillations application, and use our trained MultiENFR model to estimate the evidence of each simulation under the Λ\LambdaCDM model. We consider this our null hypothesis. Comparing this to the evidence estimate of the data, allows us to quantify a prior predictive p-value:

plowerpp=P[qϕ,ψ(xpp∣M)≤qϕ,ψ(xobs∣M)],(9)p_{\mathrm{lower}}^{\mathrm{pp}} = P[q_{\phi,\psi}(x_{\mathrm{pp}} \mid M) \leq q_{\phi,\psi}(x_{\mathrm{obs}} \mid M)], \tag*{(9)}

i.e. we can estimate how many simulations have lower evidence than the data under a given model. We use the finite sample estimator p^lowerpp=R+1n+1\hat{p}_{\mathrm{lower}}^{\mathrm{pp}} = \frac{R+1}{n+1}, where RR is the number of simulations with lower or equal evidence.

Figure 5 shows the evidence estimates of 1000 test simulations from the prior predictive as well as the evidence estimate of the DESI DR2 data for the Λ\LambdaCDM model. We infer plowerpp=0.461p_{\mathrm{lower}}^{\mathrm{pp}} = 0.461 finding no indication of model misspecification for our dataset. We also show a simulation with a very non-standard dark energy parameterization (see Appendix § 7.§ 3) and could reject the null hypothesis at plowerpp=0.001p_{\mathrm{lower}}^{\mathrm{pp}} = 0.001.

Figure 5. Evidence estimates of the MultiENFR model on prior predictive simulations and the DESI DR2 data under the Λ\LambdaCDM model. While 46.1% of simulations have lower evidence than the data, no simulation has lower evidence than the toy w(z)w(z) model, and we can clearly pick up on the model misspecification.

Limitations and Conclusions

Two main limitations remain. First, in our GMM scaling test the flow reference degrades beyond ∼500\sim500 dimensions and the evidence network recovers only a small correction. High-dimensional density estimation remains a bottleneck. Second, our benchmarks use O(105)O(10^5) simulations per model, which many scientific simulators cannot afford. Encouragingly, the evidence-network correction is most effective precisely in the small data regime (Figure 2b), so the method is well placed for the simulation-limited settings where it is needed most.

Within these limits, ENFR provides a practical framework for simulation-based estimation of the Bayesian evidence by combining a flow-matched reference with an evidence network. The single-model version (SingleENFR) improves on the flow it is built on in every benchmark, and the multi-model version (MultiENFR), based on a new pairwise l-POP loss for which we prove asymptotic recovery of all Bayes factors, is the most accurate estimator across our scientific applications. Because inference is amortized, the evidence becomes cheap enough to use as a test statistic, which we demonstrated with a prior predictive misspecification test on DESI DR2 data. Our work pushes the simulation-based paradigm beyond parameter estimation, unlocking more of the standard Bayesian toolkit for the simulation-based case: principled model misspecification testing, goodness-of-fit assessment, and model comparison through the Bayesian evidence.

AI Use Statement

Generative AI tools were used throughout the preparation of this manuscript. We used Claude Opus 5 and GPT-5.4 and GPT-5.5 for setting up and running numerical experiments and analyses, and discussing the methodology of this project. This includes generation of code which produced the training datasets used in this work. Generated code for our core methods has been checked by the authors. We have also used generative AI to assist in formatting the mathematical proofs presented in this work. GPT-5.5 and GPT-5.6 Sol were used for reviewing this manuscript and stylistic editing. We have not used generative AI tools to draft entire sections of this manuscript. We have not used generative AI tools for the direct generation of any plot. The responsibility stays with the authors.

Ethics Statement

Our work is focused on the evaluation of simulation-based models for scientific reasons.

Reproducibility Statement

We provide the details necessary for reproducing the results presented here. We present in great detail the setups of our experiments along with preprocessing steps in Appendix § 7. The details about the architectures used are in Appendix § 8.

Acknowledgments

We would like to thank Adrian Bayer, Konstantin Leyde, Henrique Rubira, and Nils Schöneberg for stimulating and insightful discussions. KL acknowledges support via the KISS consortium (05D23WM1) funded by the German Federal Ministry of Education and Research BMBF in the ErUM-Data action plan. We are also grateful for support from the Cambridge-LMU strategic partnership. CCL, FVN, and SK would like to thank the Aspen Center for Physics, which is supported by National Science Foundation grant PHY-2210452, where part of this work was discussed. S.K. has been partially supported by STFC consolidated grants ST/T000694/1 and ST/X000664/1. The authors acknowledge support from the Simons Foundation as part of the Simons Collaboration on Learning the Universe. Part of this work was done at the Flatiron Institute as part of the CCA predoctoral program. The Center for Computational Astrophysics is supported by the Simons Foundation. This research used data obtained with the Dark Energy Spectroscopic Instrument (DESI). DESI construction and operations is managed by the Lawrence Berkeley National Laboratory. This material is based upon work supported by the U.S. Department of Energy, Office of Science, Office of High-Energy Physics, under Contract No. DE–AC02–05CH11231, and by the National Energy Research Scientific Computing Center, a DOE Office of Science User Facility under the same contract. Additional support for DESI was provided by the U.S. National Science Foundation (NSF), Division of Astronomical Sciences under Contract No. AST-0950945 to the NSF’s National Optical-Infrared Astronomy Research Laboratory; the Science and Technology Facilities Council of the United Kingdom; the Gordon and Betty Moore Foundation; the Heising-Simons Foundation; the French Alternative Energies and Atomic Energy Commission (CEA); the National Council of Humanities, Science and Technology of Mexico (CONAHCYT); the Ministry of Science and Innovation of Spain (MICINN), and by the DESI Member Institutions. The DESI collaboration is honored to be permitted to conduct scientific research on I’oligam Du’ag (Kitt Peak), a mountain with particular significance to the Tohono O’odham Nation. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the U.S. National Science Foundation, the U.S. Department of Energy, or any of the listed funding agencies.

Details about the experiments

Here we provide more details about the numerical experiments we ran.

GMM

Our GMM is defined as follows:

p(x)=∑k=13wkN(x∣μk,Σk)w=(0.45,0.35,0.20).(10)p(x) = \sum_{k=1}^{3} w_k \mathcal{N}(x \mid\mu_k, \Sigma_k) \qquad w = (0.45, 0.35, 0.20). \tag*{(10)}

The components are placed on a sphere of radius r=3.8r = 3.8, i.e. μk=ruk\mu_k = r u_k with uku_k sampled uniformly from the unit sphere. Each covariance is constructed as

Σk=Dk+βLkLkT+ϵI,(11)\Sigma_k = D_k + \beta L_k L_k^{\mathsf{T}} + \epsilon I, \tag*{(11)}

where DkD_k is a diagonal matrix with entries dk,i∼U[0.7,1.8]d_{k,i} \sim\mathcal{U}[0.7, 1.8], Lk∼N(0,1/3)∈Rd×3L_k \sim\mathcal{N}(0, 1/\sqrt{3}) \in\mathbb{R}^{d \times3} is a random factor, β=0.55\beta= 0.55 controls correlation strength and is fixed in our analysis, and ϵ=0.15\epsilon= 0.15 is a jitter term ensuring positive definiteness and stable inverses.

For the dimensionality scaling we draw 5M samples. 4.5M are used for training and 500k for the evaluation set for the flow. EN and CTSM use 4M for training and 1M for validation for the dimensionality scaling. We test on 200 more fresh draws from the GMM, which are not used in training or evaluation. We preprocess the data by standardizing it with parameters estimated from the training set:

x=xraw−μtrainσtrain,(12)x = \frac{x_{\mathrm{raw}} - \mu_{\mathrm{train}}}{\sigma_{\mathrm{train}}}, \tag*{(12)}

where μtrain\mu_{\mathrm{train}} is the estimated mean of the training set and σtrain\sigma_{\mathrm{train}} is the estimated standard deviation of the training set. As we end up estimating densities, we have to take into account the Jacobian JJ of this transformation:

log⁡p(xraw)=log⁡p(x)+log⁡∣det⁡J∣.(13)\log p(x_{\mathrm{raw}}) = \log p(x) + \log\left|\det J\right|. \tag*{(13)}

The entire runtime for the dimensionality scaling of all models for one of the seeds presented here was ∼55\sim55 hours in wall clock, where we trained on a single NVIDIA A40 48GB VRAM.

BAO and priors

Baryonic acoustic oscillations are a characteristic scale in the Universe that is set by traveling sound waves in the early Universe. At some point, due to temperature changes, these sound waves can no longer propagate and are frozen in. This imprints the sound horizon scale which is the distance of how far a sound wave could have traveled until it is frozen. Today, we can still observe this scale by measuring the statistics of the galaxies that have formed.

The observed BAO scale depends on the expansion history of the Universe. Along the line of sight, it is related to the Hubble distance,

DH(z)=cH(z),(14)D_H(z) = \frac{c}{H(z)}, \tag*{(14)}

while perpendicular to the line of sight it is related to the transverse comoving distance,

DM(z)={cH0Ωksinh⁡[ΩkH0DC(z)c],Ωk>0,DC(z),Ωk=0,cH0∣Ωk∣sin⁡[∣Ωk∣H0DC(z)c],Ωk<0,(15)D_M(z) = \begin{cases} \frac{c}{H_0\sqrt{\Omega_k}} \sinh\left[\sqrt{\Omega_k}\frac{H_0D_C(z)}{c}\right], & \Omega_k > 0, \\ D_C(z), & \Omega_k = 0, \\ \frac{c}{H_0\sqrt{|\Omega_k|}} \sin\left[\sqrt{|\Omega_k|}\frac{H_0D_C(z)}{c}\right], & \Omega_k < 0, \end{cases} \tag*{(15)}
ParameterΛ\LambdaCDMwwCDMw0waw_0w_aCDMν\nuCDMΩk\Omega_kCDM
Base parameters (shared)
H0H_0[20,100][20, 100]
Ωbh2\Omega_b h^2[0.005,0.1][0.005, 0.1]
Ωch2\Omega_c h^2[0.001,0.99][0.001, 0.99]
τ\tau[0.01,0.8][0.01, 0.8]
log⁡(1010As)\log(10^{10}A_s)[1.61,3.91][1.61, 3.91]
nsn_s[0.8,1.2][0.8, 1.2]
Extension parameters
ww (or w0w_0)—[−3,−0.333][-3, -0.333][−3,1][-3, 1]——
waw_a——[−3,2][-3, 2]——
∑mν\sum m_\nu [eV]———[0.06,2][0.06, 2]—
Ωk\Omega_k————[−0.3,0.3][-0.3, 0.3]
Number of Parameters67877

Table 3. Boundaries of the uniform priors for the parameters of the five cosmological models used in our BAO example. The six base parameters of the Λ\LambdaCDM model are shared by every model. Each extension introduces either one or two more parameters.

where

DC(z)=c∫0zdz′H(z′).(16)D_C(z) = c \int_0^z \frac{\mathrm{d}z'}{H(z')}. \tag*{(16)}

Here, cc is the speed of light, zz is the redshift, and H(z)H(z) is the Hubble expansion rate at redshift zz. When the BAO signal is measured isotropically, these two distances are combined into the volume-averaged distance,

DV(z)=[zDH(z)DM2(z)]1/3.(17)D_V(z) = \left[z D_H(z) D_M^2(z)\right]^{1/3}. \tag*{(17)}

BAO measurements therefore constrain the dimensionless combinations DH(z)/rdD_H(z)/r_d, DM(z)/rdD_M(z)/r_d, or DV(z)/rdD_V(z)/r_d, depending on the analysis, where rdr_d is the sound horizon at the baryon drag epoch. The data vector consists of 13 of these quantities measured at 7 different redshifts.

Table 3 shows the boundaries of our uniform priors on the parameters of the cosmological models. We follow the choices by Ong et al. (2026). We run simulations using the Gaussian likelihood and the DESI DR2 covariance Abdul Karim et al. [1] using the cobaya (Torrado & Lewis, 2021) code. We run 500k simulations of each model for training and 2k for evaluation. For each model we run nested sampling for an additional 100 simulations per model. Due to the presence of outliers in the data, before applying Equation 12, we apply the transform:

x=asinh⁡(xraw−median⁡(xraw)IQR),(18)x = \operatorname{asinh}\left(\frac{x_{\mathrm{raw}}-\operatorname{median}(x_{\mathrm{raw}})}{\mathrm{IQR}}\right), \tag*{(18)}

where IQR is the range between the 25th and 75th percentile. We subsequently standardize according to (12). For density estimation, the Jacobian of this transformation is taken into account in our experiments.

We ran the BAO experiments on a single NVIDIA A40 with 48GB VRAM.

Toy dark energy model

As a toy model for late-time model misspecification, we consider a smooth transition in the dark-energy equation of state,

w(z)=−1+Δw2[1−tanh⁡(z−ztΔz)].(19)w(z) = -1 + \frac{\Delta w}{2}\left[1-\tanh\left(\frac{z-z_t}{\Delta z}\right)\right]. \tag*{(19)}

Table 4: Prior distributions for the ODE oscillator models. All priors are independent uniforms; parameters shared between models have identical priors.

Table 5: RMSE of the learned evidence against nested sampling (dynesty) on the ODE oscillator benchmark, over 10 held-out simulations per model.

We choose

Δw=0.4,zt=0.6,Δz=0.15.(20)\Delta w = 0.4,\qquad z_{t} = 0.6,\qquad\Delta z = 0.15. \tag*{(20)}

The model therefore approaches w=−1w=-1 at high redshift, while at low redshift it transitions to w≃−0.6w\simeq-0.6.

ODE application details and priors

Table 4 shows the priors on the parameters. We generate a training set of 1M simulations per model, 10% of which is used as validation data. We apply the preprocessing transformation from (12). We quote our results in Table 5.

DDM

We show the priors of our DDM experiment in Table 6, adopting the parameter prior ranges of Schröder & Macke (2024). The likelihood is available by solving the corresponding Fokker-Planck equation. We use the pyDDM implementation for this and refer to its documentation for further details (Shinn et al., 2020). We generate 500k simulations for each model for training out of which we split 10% for validation. We apply the preprocessing transformation from Equation 18. We show our results in Table 7.

Hodgkin-Huxley

The currents are defined as follows:

INa=gNam3h(V−ENa)IK=gKn4(V−EK)IL=gL(V−EL)IM=gMp(V−EK)ICaL=gCaq2r(V−ECa),(21)\begin{aligned} I_{\mathrm{Na}} &= g_{\mathrm{Na}}m^{3}h(V-E_{\mathrm{Na}})\\ I_{\mathrm{K}} &= g_{\mathrm{K}}n^{4}(V-E_{\mathrm{K}})\\ I_{\mathrm{L}} &= g_{\mathrm{L}}(V-E_{\mathrm{L}})\\ I_{\mathrm{M}} &= g_{\mathrm{M}}p(V-E_{\mathrm{K}})\\ I_{\mathrm{CaL}} &= g_{\mathrm{Ca}}q^{2}r(V-E_{\mathrm{Ca}}), \tag*{(21)} \end{aligned}

where gg are the maximal conductances, EE are reversal potentials and m,h,n,p,q,r∈[0,1]m,h,n,p,q,r \in[0,1] are gating variables. The gating variables xgate,a∈{m,h,n}x_{\mathrm{gate},a} \in\{m,h,n\} are obtained by a first order differential equation: x˙gate,a=αx(V−Vt)(1−xgate,a)−βx(V−Vt)xgate,a\dot{x}_{\mathrm{gate},a}=\alpha_{x}(V-V_{t})(1-x_{\mathrm{gate},a})-\beta_{x}(V-V_{t})x_{\mathrm{gate},a}, where α\alpha and β\beta are voltage-dependent rate functions as defined in Pospischil et al. (2008). The gating variables xgate,b∈{q,r}x_{\mathrm{gate},b} \in\{q,r\} evolve according to x˙gate,b=αx(V)(1−xgate,b)−βx(V)xgate,b\dot{x}_{\mathrm{gate},b}=\alpha_{x}(V)(1-x_{\mathrm{gate},b})-\beta_{x}(V)x_{\mathrm{gate},b}. pp is obtained through p˙=(p∞(V)−p)/τp(V)\dot{p}=(p_{\infty}(V)-p)/\tau_{p}(V) where p∞p_{\infty} and τp\tau_{p} are defined by:

ModelRMSE under M0\mathcal{M}_0RMSE under M1\mathcal{M}_1
flow4.05.5
SingleENFR2.43.5
MultiENFR1.81.7
flow + CTSM-SB20.3115.4

Table 7. Estimated evidence against nested sampling (dynesty) on the DDM benchmark, over 50 held-out datasets from each model.

p∞=1/(1+e−(V+35)/10)p_{\infty} = 1/(1 + e^{-(V+35)/10})
τp=τM/(3.3e(V+35)/20+e−(V+35)/20).(22)\tau_{p} = \tau_{M}/(3.3e^{(V+35)/20} + e^{-(V+35)/20}). \tag*{(22)}

All gates are initialized at V(0)=−70V(0) = -70 mV. The injected current is a step of 2 μA cm−22\,\mu\mathrm{A}\,\mathrm{cm}^{-2} between 225 and 1225 ms with added noise, Iinj(tk)=Istep(tk)+σξk/ΔtI_{\mathrm{inj}}(t_k) = I_{\mathrm{step}}(t_k) + \sigma\xi_k/\sqrt{\Delta t} where ξk∼N(0,1)\xi_k \sim\mathcal{N}(0, 1). We integrate the voltage using a Crank–Nicolson integrator at Δt=0.025\Delta t = 0.025 ms.

From each simulation we extract 13 hand-picked summaries: resting potential, spike latency, inverse first/mean/median inter-spike interval, two ISI-adaptation indices, two AP-amplitude adaptation indices, AP amplitude, inverse AP width, and the coefficient of variation and Fano factor of the AP amplitudes. Some summaries are undefined in some of the simulations, e.g. due to the absence of spikes. In that case they are set to their physical limit (latency 1000 ms, inverse ISI and inverse width 0, adaptation indices 1, amplitude, CV and Fano factor 0).

We use the jaxley Deistler et al. [12] library for implementation. We generate 400k simulations of each model for training and use 10% of that for validation. We adopt the parameter prior ranges of Schröder & Macke (2024), which were originally adapted from Gonçalves et al. (2020). We present those prior choices in Table 8 and list fixed parameters in Table 9. We preprocess using Equation (12).

ParameterPriorUnit
gLg_{\mathrm{L}}U(10−6, 3×10−4)\mathcal{U}(10^{-6},\,3 \times10^{-4})S cm−2^{-2}
ELE_{\mathrm{L}}U(−80, −60)\mathcal{U}(-80,\,-60)mV
gKg_{\mathrm{K}}U(1.5×10−3, 1.5×10−2)\mathcal{U}(1.5 \times10^{-3},\,1.5 \times10^{-2})S cm−2^{-2}
VtV_{\mathrm{t}}U(−70, −50)\mathcal{U}(-70,\,-50)mV
gNag_{\mathrm{Na}}U(8×10−3, 8×10−2)\mathcal{U}(8 \times10^{-3},\,8 \times10^{-2})S cm−2^{-2}
σ\sigmaU(10−4, 0.15)\mathcal{U}(10^{-4},\,0.15)μ\muA cm−2^{-2} ms1/2^{1/2}
gMg_{\mathrm{M}}U(10−5, 6×10−4)\mathcal{U}(10^{-5},\,6 \times10^{-4})S cm−2^{-2}
τM\tau_{\mathrm{M}}U(200, 2000)\mathcal{U}(200,\,2000)ms
gCag_{\mathrm{Ca}}U(5×10−5, 10−3)\mathcal{U}(5 \times10^{-5},\,10^{-3})S cm−2^{-2}

Table 8. Prior distributions for the Hodgkin–Huxley models.

QuantityValueUnit
ENaE_{\mathrm{Na}}50mV
EKE_{\mathrm{K}}−90-90mV
ECaE_{\mathrm{Ca}}120mV
CmC_{\mathrm{m}}1μ\muF cm−2^{-2}
Compartment length / radius10 / 1μ\mum
Step current density (225–1225 ms)2μ\muA cm−2^{-2}
Trace duration / integration step1450 / 0.025ms

Table 9. Fixed parameters for the Hodgkin–Huxley simulations.

Galaxy assembly bias

We populate halo catalogs from 2000 high-resolution Quijote Latin-hypercube simulations (Villaescusa-Navarro et al., 2020) at z=0.5z = 0.5 in periodic cubes of side length 1000 h−1Mpc1000\,h^{-1}\mathrm{Mpc}. Model M0\mathcal{M}_0 uses a mass-only halo occupation distribution (HOD) (Zheng et al., 2007), implemented with Halotools (Hearin et al., 2017), and baseline mean occupations

μcen(M)=12[1+erf⁡(log⁡10M−log⁡10Mmin⁡σlog⁡M)],\mu_{\mathrm{cen}}(M) = \frac{1}{2}\left[1 + \operatorname{erf}\left(\frac{\log_{10} M - \log_{10} M_{\min}}{\sigma_{\log M}}\right)\right],
μsat(M)=μcen(M)(M−M0M1)α,M>M0,(23)\mu_{\mathrm{sat}}(M) = \mu_{\mathrm{cen}}(M)\left(\frac{M - M_0}{M_1}\right)^{\alpha}, \qquad M > M_0, \tag*{(23)}

with μsat=0\mu_{\mathrm{sat}} = 0 otherwise. Central and satellite counts are drawn separately from Bernoulli and Poisson distributions. Centrals follow the halo centers and bulk velocities while satellites have Navarro–Frenk–White radial profiles and Jeans-equilibrium velocities. Model M1\mathcal{M}_1 adds concentration-dependent occupation at fixed mass using a decorated HOD (Hearin et al., 2016), with strengths AcenA_{\mathrm{cen}} and AsatA_{\mathrm{sat}}. It also includes velocity bias (e.g. Guo et al. (2015)) and a rescaled satellite concentration: central velocity dispersion αcVvir\alpha_{\mathrm{c}}V_{\mathrm{vir}}, satellite velocity multiplier αs\alpha_{\mathrm{s}} relative to the halo, and satellite concentration cgal=Fsatchaloc_{\mathrm{gal}} = F_{\mathrm{sat}}c_{\mathrm{halo}}. For more details on the HOD models see Zheng et al. (2007); Hearin et al. (2016).

Table 10 lists the priors in our analysis. Shared parameters have identical priors and the models have equal prior probability. For log⁡10Mmin\log_{10} M_{\mathrm{min}}, we uniformly sample the z=0.5z = 0.5 entries of 10610^{6} stored draws from an empirical prior separately constrained using the redshift distribution of the BOSS DR12 CMASS North Galactic Cap sample (Alam et al. [3]). Other HOD parameters are drawn independently as tabulated. No HOD redshift evolution is used.

ParameterM0\mathcal{M}_0M1\mathcal{M}_1
Ωm\Omega_{\mathrm{m}}U(0.1,0.5)\mathcal{U}(0.1, 0.5)U(0.1,0.5)\mathcal{U}(0.1, 0.5)
Ωb\Omega_{\mathrm{b}}U(0.03,0.07)\mathcal{U}(0.03, 0.07)U(0.03,0.07)\mathcal{U}(0.03, 0.07)
hhU(0.5,0.9)\mathcal{U}(0.5, 0.9)U(0.5,0.9)\mathcal{U}(0.5, 0.9)
nsn_{\mathrm{s}}U(0.8,1.2)\mathcal{U}(0.8, 1.2)U(0.8,1.2)\mathcal{U}(0.8, 1.2)
σ8\sigma_8U(0.6,1.0)\mathcal{U}(0.6, 1.0)U(0.6,1.0)\mathcal{U}(0.6, 1.0)
log⁡10Mmin⁡\log_{10} M_{\min}NGC empirical priorNGC empirical prior
σlog⁡M\sigma_{\log M}U(0.1,0.6)\mathcal{U}(0.1, 0.6)U(0.1,0.6)\mathcal{U}(0.1, 0.6)
log⁡10M0\log_{10} M_0U(13,15)\mathcal{U}(13, 15)U(13,15)\mathcal{U}(13, 15)
log⁡10M1\log_{10} M_1U(13,15)\mathcal{U}(13, 15)U(13,15)\mathcal{U}(13, 15)
α\alphaU(0,1.5)\mathcal{U}(0, 1.5)U(0,1.5)\mathcal{U}(0, 1.5)
AcenA_{\mathrm{cen}}0TN(0,0.22;[−1,1])\mathcal{T}\mathcal{N}(0, 0.2^2;[-1,1])
AsatA_{\mathrm{sat}}0TN(0,0.22;[−1,1])\mathcal{T}\mathcal{N}(0, 0.2^2;[-1,1])
αc\alpha_{\mathrm{c}}0U(0,0.7)\mathcal{U}(0, 0.7)
αs\alpha_{\mathrm{s}}1U(0.2,2.0)\mathcal{U}(0.2, 2.0)
FsatF_{\mathrm{sat}}1U(0.2,2.0)\mathcal{U}(0.2, 2.0)

Table 10. Priors for the Quijote galaxy-bias comparison. U(a,b)\mathcal{U}(a,b) denotes a uniform distribution and TN(0,0.22;[−1,1])\mathcal{T}\mathcal{N}(0, 0.2^2;[-1,1]) a zero-mean Gaussian of standard deviation 0.2, truncated to [−1,1][-1,1]. Mass logarithms are base ten with masses in h−1M⊙h^{-1}M_{\odot}. Numerical constants denote fixed values. The HOD mass M1M_1 is distinct from the model label M1\mathcal{M}_1.

The LHID-disjoint dataset split contains 64,000 training simulations (32,000 per model), 8,000 validation simulations and 8,000 frozen test simulations; the estimators also reserve internal validation subsets from the training partition. A single flow-matching model is fitted to the pooled training set, and 32,000 generated samples form a third class alongside the two equally sized real-model classes. We train a five-member ensemble of three-output multiclass Evidence Networks with the pairwise loss, using standardization from Equation (12) for both the flow and the networks.

Architectures

We present the architectures of the neural networks used in this work. All networks are implemented in PyTorch (Paszke et al. (2019)). We have also made use of the numpy (Harris et al. (2020)), scipy (Virtanen et al. (2020)), sklearn (Pedregosa et al. (2011)), torchdiffeq (Chen [7]), and matplotlib (Hunter (2007)) libraries.

Software and data licenses PyTorch, NumPy, SciPy, and scikit-learn are used under their BSD-3-Clause licenses; dynesty, pyddm, and torchdiffeq under the MIT license; Cobaya under LGPL-3.0-or-later with its stated additional conditions; jaxley under the Apache 2.0 licence; and Matplotlib under the Matplotlib license. The DESI data products are used under CC BY 4.0.

Flow matching

We parameterize the velocity field of the flow matching algorithm with a 256 unit, 4 hidden layer fully connected multilayer perceptron using SiLU activation functions. We train with uniform time sampling, using the Adam Kingma & Ba (2014) optimizer with a batch size of 256 for the Hodgkin and DDM applications and 128 for all other applications. For the GMM and ODE models we use a learning rate of 10−410^{-4}, for the BAO, DDM and Hodgkin models 10−310^{-3}. For density estimation via integration of the flow, we use the dopri5 integrator implemented in torchdiffeq to solve

log⁡qϕ(x∣M)=log⁡p(z)−∫01∇⋅vϕ(xt,t) dt,(24)\log q_{\phi}(x|\mathcal{M}) = \log p(z) - \int_0^1 \nabla\cdot v_{\phi}(x_t,t)\,\mathrm{d}t, \tag*{(24)}

where xtx_t follows the learned ODE trajectory. We leave the impact of a thorough hyperparameter search and of a different optimizer to further study.

For the DDM application we use permutation equivariant DeepSets (Zaheer et al., 2017). First we apply a neural network to each dimension:

hi=ϕ(xi,t),(25)h_i = \phi(x_i,t), \tag*{(25)}

to the outputs of which we apply permutation invariant aggregation functions:

A=[hˉ,max⁡ihi,min⁡ihi,std⁡i(hi)].(26)A = [\bar{h},\max_i h_i,\min_i h_i,\operatorname{std}_i(h_i)]. \tag*{(26)}

The velocity field is then learned from a concatenation of these aggregated features with the individual observation:

vi=ρ(xi,hi,A,t).(27)v_i = \rho(x_i,h_i,A,t). \tag*{(27)}

The neural networks ϕ\phi and ρ\rho are parameterized as MLPs with 256 neurons in each of the 3 layers. The output dimension of ϕ\phi is also 256.

ENFR

The evidence networks are fully connected neural networks. For the GMM example we use a rather large network with 1024 x 4 neurons with the SiLU activation function. For the BAO example we use a substantially smaller network with 128x2 neurons for MultiENFR and 128x3 neurons for SingleENFR with the ReLU activation function. For the ODE application we use 256x4 neurons with ReLU activation. For the Hodgkin application we use 128x2 neurons with ReLU activation. In all examples except for the GMM we train an ensemble of 5 evidence networks with equal weight, while the GMM test only shows single networks. Batch sizes are 128 (GMM,BAO), 256 (Hodgkin, DDM), and 1024 (ODE).

We train using the Adam Kingma & Ba (2014) optimizer with a learning rate of 10−410^{-4}.

For the DDM experiment MultiENFR is a permutation invariant DeepSet (Zaheer et al., 2017). A first neural network delivers

hi=ϕ(xi),(28)h_i = \phi(x_i), \tag*{(28)}

where xix_i are the simulated signed decision times. The output is then aggregated:

A=[hˉ, max⁡ihi, min⁡ihi, std⁡i(hi)],(29)A = [\bar{h},\ \max_i h_i,\ \min_i h_i,\ \operatorname{std}_i(h_i)], \tag*{(29)}

and fed into a second network

yj=ρ(A).(30)y_j = \rho(A). \tag*{(30)}

ρ\rho and ϕ\phi are parameterized with MLPs with two and three layers respectively, with 128 neurons in each layer, using a ReLU activation function. We train an equally weighted 5 member ensemble with learning rate 10−410^{-4} batch size 256 and patience 10 in early stopping.

CTSM

The CTSM models are also fully connected multilayer perceptrons. There are two different interpolants we use, the variance preserving bridge (VP):

xt=1−t2z+tx1with z∼N(0,1),(31)x_t = \sqrt{1-t^2}z + tx_1 \quad\text{with } z \sim\mathcal{N}(0,1), \tag*{(31)}

and the Schrödinger bridge (SB):

xt=(1−t)x0+tx1+σt(1−t)ϵwith ϵ∼N(0,1),(32)x_t = (1-t)x_0 + tx_1 + \sigma\sqrt{t(1-t)}\epsilon\quad\text{with } \epsilon\sim\mathcal{N}(0,1), \tag*{(32)}

where x0∼qϕ(x)x_0 \sim q_\phi(x) is drawn from the flow reference and x1∼p(x∣M)x_1 \sim p(x|\mathcal{M}) are data samples. We choose the noise scale σ\sigma to be 1.

In all CTSM runs, the architectures have always been identical to the corresponding SingleENFR runs.

Results of the GMM scaling experiment

We show the results of the GMM scaling in Table 11. We further show an example plot of the improvement that comes from adding the evidence network in 128 dimensions in Figure 6.

Figure 6. The performance on test points in 128 dimensions for the flow and SingleENFR.

Bayes factor analysis for DESI DR2 data

Table 12 shows the explicit comparison of our MultiENFR Bayes factors compared to the likelihood-based analysis.

Table 12: Bayes factors w.r.t. Λ\LambdaCDM on the DESI DR2 data. The error of our simulation-based estimate is largely comparable to the error of the likelihood-based estimate in three of the four models.

ModelLikelihood Based AnalysisMultiENFRΔ\Delta
wwCDM−2.42±0.24-2.42\pm0.24−1.94-1.94+0.48+0.48
w0waw_0w_aCDM−1.64±0.24-1.64\pm0.24−1.40-1.40+0.25+0.25
ν\nuCDM−0.07±0.22-0.07\pm0.220.060.06+0.13+0.13
Ωk\Omega_kCDM−1.89±0.23-1.89\pm0.23−1.65-1.65+0.24+0.24

Table 12.

Proof of minimizer of evidence networks

We present the proof that the minimizer of the risk of the l-POP loss is indeed the Bayes factor if the models are weighted equally a priori.

Proposition. For a sample (x,m)(x,m) with m∈{0,1}m \in\{0,1\}, the l-POP exponential loss is

L(x,m)=exp⁡[(12−m)u(x)].\mathcal{L}(x,m)=\exp\left[\left(\frac{1}{2}-m\right)u(x)\right].

using the shorthand notation u(x)=Jα(fψ(x))u(x)=\mathcal{J}_{\alpha}(f_{\psi}(x)). Assuming equal model sampling probabilities, the population optimum satisfies

u∗(x)=log⁡p(x∣M)p(x∣M0)=log⁡K,u^{*}(x)=\log\frac{p(x\mid\mathcal{M})}{p(x\mid\mathcal{M}_{0})}=\log K,

with the ∗* superscript indicating at minimizing parameters ψ=ψ^\psi=\hat{\psi}.

Proof. Consider the population risk as a functional of u(⋅)u(\cdot). Under equal sampling probabilities for the two models,

P(M0)=P(M)=12,P(\mathcal{M}_{0})=P(\mathcal{M})=\frac{1}{2},

the expected loss is

R[u]=12∫[p(x∣M0)eu(x)/2+p(x∣M)e−u(x)/2] dx.\mathcal{R}[u]=\frac{1}{2}\int\left[p(x\mid\mathcal{M}_{0})e^{u(x)/2}+p(x\mid\mathcal{M})e^{-u(x)/2}\right]\,dx.

We proceed with standard variational calculus, differentiating with respect to some perturbation ε\varepsilon at ε=0\varepsilon=0:

δR[u;η]=ddεR[u+εη]∣ε=0.\delta\mathcal{R}[u;\eta]=\left.\frac{d}{d\varepsilon}\mathcal{R}[u+\varepsilon\eta]\right|_{\varepsilon=0}.

Thus,

δR[u;η]=14∫[p(x∣M0)eu(x)/2−p(x∣M)e−u(x)/2]η(x) dx.\delta\mathcal{R}[u;\eta]=\frac{1}{4}\int\left[p(x\mid\mathcal{M}_{0})e^{u(x)/2}-p(x\mid\mathcal{M})e^{-u(x)/2}\right]\eta(x)\,dx.

At a minimizer u∗u^{*}, the variation must vanish so we find

p(x∣M0)eu∗(x)/2−p(x∣M)e−u∗(x)/2=0.p(x\mid\mathcal{M}_{0})e^{u^{*}(x)/2}-p(x\mid\mathcal{M})e^{-u^{*}(x)/2}=0.

Hence,

p(x∣M0)eu∗(x)/2=p(x∣M)e−u∗(x)/2.p(x\mid\mathcal{M}_{0})e^{u^{*}(x)/2}=p(x\mid\mathcal{M})e^{-u^{*}(x)/2}.

Multiplying by eu∗(x)/2e^{u^{*}(x)/2} gives

p(x∣M0)eu∗(x)=p(x∣M),p(x\mid\mathcal{M}_{0})e^{u^{*}(x)}=p(x\mid\mathcal{M}),

and therefore

eu∗(x)=p(x∣M)p(x∣M0).e^{u^{*}(x)}=\frac{p(x\mid\mathcal{M})}{p(x\mid\mathcal{M}_{0})}.

Taking logarithms,

u∗(x)=log⁡p(x∣M)p(x∣M0)=log⁡K.u^*(x) = \log\frac{p(x \mid\mathcal{M})}{p(x \mid\mathcal{M}_0)} = \log K.

Thus,

u∗(x)=log⁡K.u^*(x) = \log K.

Without the shorthand this delivers

Jα(f∗(x))=log⁡K.\mathcal{J}_{\alpha}(f^*(x)) = \log K.

EN correction to the flow under constant prediction

We use the l-POP loss EN to correct the density estimates of a generative flow. It is interesting to examine the case where the EN cannot learn anything from the data and just returns a minimizing constant, i.e. f(x)=constf(x) = \mathrm{const}. In this case we can show, that the EN correction asymptotically does not degrade the flow’s evidence estimate under equal model weights.

Consider again the population risk of the l-POP loss of a constant predictor u(x)=Jα(fψ(x))=cu(x) = \mathcal{J}_{\alpha}(f_{\psi}(x)) = c under model priors which are reflected by the training class probabilities p(M)p(\mathcal{M}) and p(M0)p(\mathcal{M}_0):

R[c]=∫p(M0)p(x∣M0)ec/2+p(M)p(x∣M)e−c/2 dx=p(M0)ec/2+p(M)e−c/2.R[c] = \int p(\mathcal{M}_0)p(x \mid\mathcal{M}_0)e^{c/2} + p(\mathcal{M})p(x \mid\mathcal{M})e^{-c/2}\,\mathrm{d}x = p(\mathcal{M}_0)e^{c/2} + p(\mathcal{M})e^{-c/2}.

We consider the variation

δR[c;a]=ddϵR[c+ϵa]∣ϵ=0,\delta R[c;a] = \left.\frac{\mathrm{d}}{\mathrm{d}\epsilon}R[c+\epsilon a]\right|_{\epsilon=0},

which delivers

δR[c;a]=a2[p(M0)ec/2−p(M)e−c/2].\delta R[c;a] = \frac{a}{2}\left[p(\mathcal{M}_0)e^{c/2} - p(\mathcal{M})e^{-c/2}\right].

At the variational optimum c∗c^*, the variation must vanish so

p(M0)ec∗/2−p(M)e−c∗/2=0,p(\mathcal{M}_0)e^{c^*/2} - p(\mathcal{M})e^{-c^*/2} = 0,

which implies

ec∗=p(M)p(M0).e^{c^*} = \frac{p(\mathcal{M})}{p(\mathcal{M}_0)}.

Hence the optimal constant correction is

c∗=log⁡p(M)p(M0).c^* = \log\frac{p(\mathcal{M})}{p(\mathcal{M}_0)}.

For equal model weights,

p(M0)=p(M),p(\mathcal{M}_0) = p(\mathcal{M}),

and therefore

c∗=0.c^* = 0.

The asymptotically optimal EN therefore does not degrade a well-trained flow in this case.

Proof of minima of the Multiclass Evidence Network

We present the proof that the multiclass evidence network loss function is minimized by the Bayes factors.

Proposition. Consider NN models

M1,…,MN\mathcal{M}_1,\ldots,\mathcal{M}_N

and define

ui(x)=Jα(fi(x)).u_i(x) = \mathcal{J}_{\alpha}(f_i(x)).

For a sample xx generated from model Mm\mathcal{M}_m, consider the pairwise loss

L(x,m)=1N−1∑j≠mexp⁡[−12(um(x)−uj(x))].\mathcal{L}(x,m) = \frac{1}{N-1}\sum_{j\ne m}\exp\left[-\frac{1}{2}\left(u_m(x)-u_j(x)\right)\right].

Assume that the models are sampled with equal probability,

p(Mi)=1N.p(\mathcal{M}_i) = \frac{1}{N}.

Then the population-risk functional, up to the positive constant factor 1/[N(N−1)]1/[N(N-1)], is

R[u]=∫∑i=1Np(x∣Mi)∑j≠iexp⁡[−12(ui(x)−uj(x))] dx.\mathcal{R}[\mathbf{u}] = \int\sum_{i=1}^{N} p(x\mid\mathcal{M}_i)\sum_{j\ne i}\exp\left[-\frac{1}{2}\left(u_i(x)-u_j(x)\right)\right]\,\mathrm{d}x.

Define

pi(x)≡p(x∣Mi).p_i(x) \equiv p(x\mid\mathcal{M}_i).

Adding the diagonal terms i=ji=j, which contribute only

∑ipi(x)\sum_i p_i(x)

and are independent of u\mathbf{u}, we may equivalently minimize

R~[u]=∫(∑ipi(x)e−ui(x)/2)(∑jeuj(x)/2) dx.\widetilde{\mathcal{R}}[\mathbf{u}] = \int\left(\sum_i p_i(x)e^{-u_i(x)/2}\right)\left(\sum_j e^{u_j(x)/2}\right)\,\mathrm{d}x.

For compactness, define

A(x)=∑ipi(x)e−ui(x)/2,B(x)=∑ieui(x)/2.A(x) = \sum_i p_i(x)e^{-u_i(x)/2}, \qquad B(x) = \sum_i e^{u_i(x)/2}.

Then

R~[u]=∫A(x)B(x) dx.\widetilde{\mathcal{R}}[\mathbf{u}] = \int A(x)B(x)\,dx.

Again we follow variation of the functional:

δR~[u;η]=ddεR~[u+εη]∣ε=0.\delta\widetilde{\mathcal{R}}[\mathbf{u};\eta] = \left.\frac{d}{d\varepsilon}\widetilde{\mathcal{R}}[\mathbf{u}+\varepsilon\eta]\right|_{\varepsilon=0}.

Using the product rule, we first apply this to each term separately:

Aε(x)=∑ipi(x)e−[ui(x)+εηi(x)]/2,A_{\varepsilon}(x)=\sum_i p_i(x)e^{-[u_i(x)+\varepsilon\eta_i(x)]/2},

and therefore

dAεdε∣ε=0=−12∑ipi(x)e−ui(x)/2ηi(x).\left.\frac{dA_{\varepsilon}}{d\varepsilon}\right|_{\varepsilon=0} =-\frac{1}{2}\sum_i p_i(x)e^{-u_i(x)/2}\eta_i(x).

Similarly for the second term,

Bε(x)=∑ie[ui(x)+εηi(x)]/2,B_{\varepsilon}(x)=\sum_i e^{[u_i(x)+\varepsilon\eta_i(x)]/2},

so that

dBεdε∣ε=0=12∑ieui(x)/2ηi(x).\left.\frac{dB_{\varepsilon}}{d\varepsilon}\right|_{\varepsilon=0} =\frac{1}{2}\sum_i e^{u_i(x)/2}\eta_i(x).

Now according to the product rule we can write

δR~=∫[δA(x)B(x)+A(x)δB(x)]dx=12∫[∑k=1N[A(x)euk(x)/2−B(x)pk(x)e−uk(x)/2]ηk(x)]dx.\begin{aligned} \delta\widetilde{\mathcal{R}}=\int\left[\delta A(x)B(x)+A(x)\delta B(x)\right]dx \\ &=\frac{1}{2}\int\left[\sum_{k=1}^{N}\left[A(x)e^{u_k(x)/2}-B(x)p_k(x)e^{-u_k(x)/2}\right]\eta_k(x)\right]dx. \end{aligned}

To find the extremum we set the variation to 0:

δR~[u∗;η]=0.\delta\widetilde{\mathcal{R}}[\mathbf{u}^{*};\eta]=0.

By the fundamental lemma of the calculus of variations, this implies, for each kk,

A(x)euk∗(x)/2−B(x)pk(x)e−uk∗(x)/2=0.A(x)e^{u_k^{*}(x)/2}-B(x)p_k(x)e^{-u_k^{*}(x)/2}=0.

Thus

A(x)euk∗(x)/2=B(x)pk(x)e−uk∗(x)/2.A(x)e^{u_k^{*}(x)/2}=B(x)p_k(x)e^{-u_k^{*}(x)/2}.

Multiplying by euk∗(x)/2e^{u_k^{*}(x)/2} gives

A(x)euk∗(x)=B(x)pk(x),A(x)e^{u_k^{*}(x)}=B(x)p_k(x),

and therefore

euk∗(x)=pk(x)B(x)A(x).e^{u_k^*(x)} = p_k(x)\frac{B(x)}{A(x)}.

Taking logarithms,

uk∗(x)=log⁡pk(x)+log⁡B(x)A(x).u_k^*(x) = \log p_k(x) + \log\frac{B(x)}{A(x)}.

The second term is independent of the model index kk. Hence the stationary solutions have the form

uk∗(x)=log⁡p(x∣Mk)+c(x),u_k^*(x) = \log p(x \mid\mathcal{M}_k) + c(x),

where c(x)c(x) is a common, non-identifiable offset.

Taking the difference between any two outputs eliminates this common offset:

ui∗(x)−uj∗(x)=log⁡p(x∣Mi)−log⁡p(x∣Mj).u_i^*(x) - u_j^*(x) = \log p(x \mid\mathcal{M}_i) - \log p(x \mid\mathcal{M}_j).

Therefore,

ui∗(x)−uj∗(x)=log⁡p(x∣Mi)p(x∣Mj)=log⁡Kij(x).u_i^*(x) - u_j^*(x) = \log\frac{p(x \mid\mathcal{M}_i)}{p(x \mid\mathcal{M}_j)} = \log K_{ij}(x).

and removing the shorthand notation

Jα(fi∗(x))−Jα(fj∗(x))=log⁡Kij(x).\mathcal{J}_{\alpha}(f_i^*(x)) - \mathcal{J}_{\alpha}(f_j^*(x)) = \log K_{ij}(x).

To verify that this stationary family gives the global minimum, note that at each xx,

A(x)B(x)=(∑ipi(x)e−ui(x)/2)(∑ieui(x)/2).A(x)B(x) = \left(\sum_i p_i(x)e^{-u_i(x)/2}\right)\left(\sum_i e^{u_i(x)/2}\right).

By the Cauchy–Schwarz inequality,

A(x)B(x)≥(∑ipi(x))2.A(x)B(x) \ge\left(\sum_i \sqrt{p_i(x)}\right)^2.

Equality holds if and only if

pi(x)e−ui(x)/4=λ(x)eui(x)/4\sqrt{p_i(x)}e^{-u_i(x)/4} = \lambda(x)e^{u_i(x)/4}

for every ii, which is equivalent to

ui(x)=log⁡pi(x)−2log⁡λ(x)=log⁡pi(x)+c(x).u_i(x) = \log p_i(x) - 2\log\lambda(x) = \log p_i(x) + c(x).

Thus the variational stationary family obtained above attains the pointwise lower bound of the population-risk integrand and is therefore a global minimizer. The common function c(x)c(x) remains undetermined because the loss depends only on pairwise differences ui−uju_i-u_j.

References

  1. [1]M. Abdul Karim, J. Aguilar, S. Ahlen, S. Alam, L. Allen, C. Allende Prieto, O. Alves, A. Anand, U. Andrade, E. Armengaud, A. Aviles, S. Bailey, C. Baltay, P. Bansal, A. Bault, J. Behera, S. Ben-Zvi, D. Bianchi, C. Blake, S. Brieden, A. Brodzeller, D. Brooks, E. Buckley-Geer, E. Burtin, R. Calderon, R. Canning, A. Carnero Rosell, P. Carrilho, L. Casas, F. J. Castander, M. Charles, E. Chaussidon, J. Chaves-Montero, D. Chebat, X. Chen, T. Claybaugh, S. Cole, A. P. Cooper, A. Cuceu, K. S. Dawson, A. de la Macorra, A. de Mattia, N. Deiosso, J. Della Costa, R. Demina, A. Dey, B. Dey, Z. Ding, P. Doel, J. Edelstein, D. J. Eisenstein, W. Elbers, P. Fagrelius, K. Fanning, E. Fernández-García, S. Ferraro, A. Font-Ribera, J. E. Forero-Romero, C. S. Frenk, C. Garcia-Quintero, L. H. Garrison, E. Gaztañaga, H. Gil-Marín, S. Gontcho A Gontcho, D. Gonzalez, A. X. Gonzalez-Morales, C. Gordon, D. Green, G. Gutierrez, J. Guy, B. Hadzhiyska, C. Hahn, S. He, M. Herbold, H. K. Herrera-Alcantar, M.-F. Ho, K. Honscheid, C. Howlett, D. Huterer, M. Ishak, S. Juneau, N. V. Kambale, N. G. Karaçaylı, R. Kehoe, S. Kent, A. G. Kim, D. Kirkby, T. Kisner, S. E. Koposov, A. Kremin, A. Krolewski, O. Lahav, C. Lammann, M. Landriau, D. Lang, J. Lasker, J. M. Le Goff, L. Le Guillou, A. Leauthaud, M. E. Levi, Q. Li, T. S. Li, K. Lodha, M. Lokken, F. Lozano-Rodríguez, C. Magneville, M. Manera, P. Martini, W. L. Matthewson, A. Meisner, J. Mena-Fernández, A. Menegas, T. Mergulhão, R. Miquel, J. Moustakas, A. Muñoz-Gutiérrez, D. Muñoz-Santos, A. D. Myers, S. Nadathur, K. Naidoo, L. Napolitano, J. A. Newman, G. Niz, H. E. Noriega, E. Paillas, N. Palanque-Delabrouille, J. Pan, J. A. Peacock, M. Pellejero Ibanez, W. J. Percival, A. Pérez-Fernández, I. Pérez-Ràfols, M. M. Pieri, C. Poppett, F. Prada, D. Rabinowitz, A. Raichoor, C. Ramírez-Pérez, M. Rashkovetskyi, C. Ravoux, J. Rich, A. Rocher, C. Rockosi, J. Rohlf, J. O. Román-Herrera, A. J. Ross, G. Rossi, R. Ruggeri, V. Ruhlmann-Kleider, L. Samushia, E. Sanchez, N. Sanders, D. Schlegel, M. Schubnell, H. Seo, A. Shafieloo, R. Sharples, J. Silber, F. Sinigaglia, D. Sprayberry, T. Tan, G. Tarle, P. Taylor, W. Turner, L. A. Ureña-López, R. Vaisakh, F. Valdes, G. Valogiannis, M. Vargas-Magaña, L. Verde, M. Walther, B. A. Weaver, D. H. Weinberg, M. White, M. Wolfson, C. Yèche, J. Yu, E. A. Zaborowski, P. Zarrouk, Z. Zhai, H. Zhang, C. Zhao, G. B. Zhao, R. Zhou, H. Zou, and DESI Collaboration. DESI DR2 results II. Measurements of baryon acoustic oscillations and cosmological constraints. Phys. Rev. D, 112(8):083515, October 2025. doi: 10.1103/tr6y-kpc6.
  2. [2]S. Agostinelli, J. Allison, K. Amako, J. Apostolakis, H. Araujo, P. Arce, M. Asai, D. Axen, S. Banerjee, G. Barrand, F. Behnar, L. Bellagamba, J. Boudreau, L. Broglia, A. Brunengo, H. Burkhardt, S. Chauvie, J. Chuma, R. Chytracek, G. Cooper­man, G. Cosmo, P. Degtyarenko, A. Dell’Acqua, G. Depaola, D. Dietrich, R. Enami, A. Felicello, C. Ferguson, H. Hesefeldt, G. Folger, F. Foppiano, A. Forti, S. Garelli, S. Giani, R. Giannitrapani, D. Gibin, J. J. Gómez Cadenas, I. González, G. Gracia Abril, G. Greeniaus, W. Greiner, V. Grichine, A. Grossheim, S. Guatelli, P. Gumplinger, R. Hamatsu, K. Hashimoto, H. Hasui, A. Heikkinen, A. Howard, V. Ivanchenko, A. Johnson, F. W. Jones, J. Kallenbach, N. Kanaya, M. Kawabata, Y. Kawabata, M. Kawabati, M. Kelenr, P. Kent, A. Kimura, T. Kodama, R. Kokolulin, M. Kossov, H. Kurashige, E. Lamanna, T. Lampén, V. Lara, V. Lefebure, F. Lei, M. Liendl, W. Lockman, F. Longo, S. Magni, M. Maire, E. Medernach, K. Minamimoto, P. Mora de Freitas, Y. Morita, K. Murakami, M. Nagamatu, R. Nartallo, P. Nieminen, T. Nishimura, K. Ohtsubo, M. Okumura, S. O’Neale, Y. Oohata, K. Paech, J. Perl, A. Pfeiffer, M. G. Pia, F. Ranjard, A. Rybin, S. Sadilov, E. Di Salvo, G. Santin, T. Sasaki, N. Savvas, Y. Sawada, S. Scherer, S. Sei, V. Sirotenko, D. Smith, N. Starkov, H. Stoecker, J. Sulkimo, M. Takahata, S. Tanaka, E. Tcherniaev, E. Safai Tehrani, M. Tropeano, P. Truscott, H. Uno, L. Urban, P. Urban, M. Verderi, A. Walkden, W. Wander, H. Weber, J. P. Wellisch, T. Wenaus, D. C. Williams, D. Wright, T. Yamada, H. Yoshida, D. Zschiesche, and G EANT4 Collaboration. GEANT4—a simulation toolkit. Nuclear Instruments and Methods in Physics Research A, 506 (3):250–303, July 2003. doi: 10.1016/S0168-9002(03)01368-8.DOI
  3. [3]Shadab Alam et al. The clustering of galaxies in the completed SDSS-III Baryon Oscillation Spectroscopic Survey: cosmological analysis of the DR12 galaxy sample. Monthly Notices of the Royal Astronomical Society, 470(3):2617–2652, 2017. doi: 10.1093/mnras/stx721. URL https://arxiv.org/abs/1607.03155.
  4. [4]Justin Alsing, Benjamin Wandelt, and Stephen Feeney. Massive optimal data compression and density estimation for scalable, likelihood-free inference in cosmology. MNRAS, 477(3):2874–2885, July 2018. doi: 10.1093/mnras/sty819.DOI
  5. [5]George E. P. Box. Sampling and bayes’ inference in scientific modelling and robustness. Royal Statistical Society. Journal. Series A: General, 143(4):383–404, 07 1980. ISSN 0035-9238. doi: 10.2307/2982063. URL https://doi.org/10.2307/2982063.
  6. [6]Johann Brehmer, Gilles Louppe, Juan Pavez, and Kyle Cranmer. Mining gold from implicit models to improve likelihood-free inference. Proceedings of the National Academy of Sciences, 117(10):5242–5249, 2020. ISSN 0027-8424. doi: 10.1073/pnas.1915980117. URL https://www.pnas.org/content/117/10/5242.DOI
  7. [7]Ricky T. Q. Chen. torchdiffseq, 2018. URL https://github.com/rtqichen/torchdiffseq.
  8. [8]Wei Chen, Shigui Li, Jiacheng Li, Junmei Yang, John Paisley, and Delu Zeng. Dequantified diffusion-schrödinger bridge for density ratio estimation. In Parti Singh, Maryam Fazel, Daniel Hsu, Simon Lacoste-Julien, Felix Berkenkamp, Tegan Maharaj, Kiri Wagstaff, and Jerry Zhu (eds.), Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pp. 8427–8452. PMLR, 13–19 Jul 2025. URL https://proceedings.mlr.press/v267/chen25aj.html.arxiv.org/abs/2505.05034
  9. [9]Kristy Choi, Madeline Liao, and Stefano Ermon. Featurized density ratio estimation. In Cassio de Campos and Marcos L. M. Matthuis (eds.), Proceedings of the Thirty-Seventh Conference on Uncertainty in Artificial Intelligence, volume 161 of Proceedings of Machine Learning Research, pp. 172–182. PMLR, 27–30 Jul 2021. URL https://proceedings.mlr.press/v161/choi21a.html.arxiv.org/abs/2107.02212
  10. [10]Kristy Choi, Chenlin Meng, Yang Song, and Stefano Ermon. Density ratio estimation via infinitesimal classification. In Gustau Camps-Valls, Francisco J. R. Ruiz, and Isabel Valera (eds.), Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, volume 151 of Proceedings of Machine Learning Research, pp. 2552–2573. PMLR, 28–30 Mar 2022. URL https://proceedings.mlr.press/v151/choi22a.html.arxiv.org/abs/2111.11010
  11. [11]Kyle Cranmer, Johann Brehmer, and Gilles Louppe. The frontier of simulation-based inference. Proceedings of the National Academy of Science, 117(48):30055–30062, December 2020. doi: 10.1073/pnas.1912789117.DOI
  12. [12]Michael Deistler, Kyra L Kadhim, Matthijs Pals, Jonas Beck, Ziwei Huang, Manuel Gloeckler, Janne K Lappalainen, Cornelius Schröder, Philipp Berens, Pedro J Gonçalves, et al. Jaxley: differentiable simulation enables large-scale training of detailed biophysical models of neural dynamics. Nature Methods, pp. 1–9, 2025.DOI
  13. [13]F. Feroz, M. P. Hobson, and M. Bridges. MULTINEST: an efficient and robust Bayesian inference tool for cosmology and particle physics. MNRAS, 398(4):1601–1614, October 2009. doi: 10.1111/j.1365-2966.2009.14548.x.DOI
  14. [14]Ruiqi Gao, Erik Nijkamp, Diederik P. Kingma, Zhen Xu, Andrew M. Dai, and Ying Nian Wu. Flow contrastive estimation of energy-based models. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7515–7525, 2019. URL https://api.semanticscholar.org/CorpusID:208527348.arxiv.org/abs/1912.00589

Paper details

vlnc:W_re47vx63
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Mathematics

Claude Opus 5 (bare), GPT-5.4, GPT-5.5 (bare), GPT-5.6 Sol (bare)

View statement
Looking for related papers…
Contents