Introduction
Many domains in science and engineering rely on stochastic simulators whose sampling distribution p ( x ∣ θ ) p(x \mid\theta) p ( x ∣ θ ) is intractable but from which data samples x x x can be drawn given parameters θ \theta θ . Bayesian parameter inference in such settings is the subject of Simulation-Based Inference (SBI) [1 ] . Neural SBI methods have matured rapidly over the past five years. Sequential Neural Posterior Estimation [SNPE; [2 , 3 ] ] targets the posterior, Sequential Neural Likelihood Estimation [SNLE; [4 ] ] the likelihood, and Sequential Neural Ratio Estimation [SNRE; [5 , 6 ] ] the likelihood-to-evidence ratio. Sequential Neural Variational Inference [SNVI; [7 ] ] combines these with variational inference (VI).
A common difficulty arises when the prior-to-posterior volume ratio (R p \mathcal{R}_{p} R p ) is large: when p ( θ ) p(\theta) p ( θ ) is broad compared to the posterior, most simulator calls fall in regions of negligible posterior density, and efficiency vanishes. Sequential variants can reduce this cost by retraining the proposal across rounds, but this comes at a price. SNPE with atomic proposals suffers from posterior leakage outside prior support [3 ] , and naive implementations of variational approaches collapse onto a single mode of the posterior [7 ] . Truncation schemes [8 , 9 , 10 ] instead restrict the effective prior, so their result is exact only if the truncated region contains the posterior, or else must be corrected at inference time, for example by importance sampling [11 ] .
A growing class of scientific simulators are differentiable : GPU-native models written within the JAX [12 ] /PyTorch [13 ] frameworks, learnable physical simulators, and neural emulators trained to mimic expensive codes all expose the gradient ∇ θ s ( θ ) \nabla_{\theta} s(\theta) ∇ θ s ( θ ) of the parameters θ \theta θ under the simulator output, s ( θ ) s(\theta) s ( θ ) . This represents an opportunity for SBI: algorithms can exploit gradients directly rather than treating the simulator as a black box.
Recent advances in neural samplers, in particular the Adjoint Schrödinger Bridge Sampler (ASBS; [14 ] ), provide an opportunity to exploit such gradients. ASBS recasts sampling from an unnormalised target distribution as a Schrödinger bridge problem solved via stochastic optimal control: a controlled SDE transports a tractable source distribution to the target, with the control trained by adjoint matching and a learned corrector [15 ] . The recent Well-Tempered variant (WT-ASBS; [16 ] ) augments ASBS with a metadynamics-style bias [17 ] , overcoming free-energy barriers between modes during training and enabling broad mode coverage at fixed simulation cost. Two properties make this family particularly well-suited to SBI. First, ASBS queries the target only through gradients of its energy (the negative log of the unnormalised target density), ∇ θ E \nabla_{\theta} E ∇ θ E , which a differentiable simulator supplies. Second, the well-tempered bias addresses the mode-seeking pathology that limits standard variational SBI [7 ] on multi-modal posteriors. To our knowledge, neither ASBS nor its well-tempered extension have previously been deployed for likelihood-free Bayesian inference.
We introduce RSBI , a two-phase SBI framework that leverages differentiable simulators while keeping the target posterior fixed as p ( θ ∣ x o ) p(\theta\mid x_o) p ( θ ∣ x o ) , where x o x_o x o denotes the observed data. Phase 1 places posterior mass rapidly using a cheap surrogate likelihood, parameterized by bandwidth ϵ \epsilon ϵ and built from distance metric D ϵ ( s , x o ) \mathcal{D}_{\epsilon}(s,x_o) D ϵ ( s , x o ) between observed and simulated data, plugged into a Well-Tempered Adjoint Schrödinger Bridge Sampler [14 , 16 ] (WT-ASBS). Phase 2 refines the resulting Phase-1 proposal using WT-ASBS, augmented by an NRE trained on simulator samples drawn from the ‘zoomed in’ proposal. Figure 1 outlines the process.
Figure 1. Actual diffusion trajectories from the RSBI pipeline on the Two Moons task detailed in Sec.5. RSBI is able to produce a high quality posterior despite the large prior size.
Our contributions are threefold:
Benchmarking robustness at large prior-to-posterior volume ratio R p \mathcal{R}_p R p . We benchmark RSBI and other methods on Two Moons [18 ] , SLCP [18 , 19 ] , Lotka–Volterra [18 ] , Bernoulli GLM [18 ] , and a new differentiable gravitational-wave ringdown task (dim θ = 6 \theta= 6 θ = 6 , dim x = 820 x = 820 x = 820 ), sweeping prior-to-posterior volume ratios up to log 10 R p ≈ 12 \log_{10} \mathcal{R}_p \approx12 log 10 R p ≈ 12 .
Related Work
RSBI sits at the intersection of three research threads. The first is neural simulation-based inference , in which a density or ratio estimator is trained on simulated ( θ , x ) (\theta, x) ( θ , x ) pairs to target the posterior, likelihood, or likelihood-to-evidence ratio [2 , 3 , 4 , 5 , 6 , 20 ] . Sequential variants iteratively refine their proposal to concentrate simulations in informative regions of parameter space. SNVI+fKL [7 ] pairs likelihood(-ratio) estimation with VI and is our closest methodological sibling. Like RSBI, SNVI combines a learned neural likelihood ratio estimated via Neural Ratio Estimation (NRE) with a variational sampler, and, like RSBI, it explicitly addresses the mode-seeking pathology of standard reverse-KL VI, in its case by training under the mass-covering forward KL with sampling–importance-resampling for proposal updates. RSBI instead handles mode coverage through the well-tempered metadynamics bias of WT-ASBS [16 ] , which flattens free-energy barriers between modes. The two methods also differ in how the initial proposal is obtained: SNVI draws its first round of simulations from the original prior, whereas RSBI is initialised by a one-shot surrogate-guided Schrödinger bridge.
The second thread is the surrogate-likelihood lineage that connects classical ABC [21 ] to modern VI. Any differentiable distance metric D ϵ ( s , x o ) D_{\epsilon}(s, x_o) D ϵ ( s , x o ) induces a pseudo-likelihood whose gradients propagate through a simulator, an idea explored by ABC-VI [22 ] , as well as score-based approaches [23 , 24 ] . RSBI borrows from this for Phase 1, but rather than amortising over x x x or running an MCMC against the surrogate, we use it to construct the energy on which WT-ASBS is trained. This gives a non-mode-seeking proposal that Phase 2 then transports to the true posterior, with a likelihood specified by an NRE learned only where the proposal places mass. This use of a surrogate kernel as a warm-start for a downstream refinement, rather than as the inference target itself, can alleviate the bandwidth-versus-bias trade-off that limits standalone ABC-VI.
The third thread is the rapidly maturing ecosystem of GPU-native, auto-differentiable simulators across many scientific fields and neural emulators [25 , 26 , 27 , 28 ] . Direct access to simulation gradients ∇ θ s ( θ ) \nabla_{\theta}s(\theta) ∇ θ s ( θ ) enables gradient-based inference for models that previously admitted only analytical-likelihood treatments: ASBS [14 ] queries its target exclusively through energy gradients, which the differentiable simulator supplies via a distance metric D ϵ ( s , x o ) D_{\epsilon}(s, x_o) D ϵ ( s , x o ) , sidestepping the density-estimation step of standard SNP/ESNLE/SNRE. A growing number of works that exploit gradients have been used to improve neural SBI. Brehmer et al. Brehmer et al. [29 ] augment the training losses of ratio and likelihood estimators with the joint score and likelihood ratio, and relaxations such as the Gumbel-Softmax [30 ] extend such gradients to simulators with discrete randomness. To our knowledge, the only gradient-informed NPE method adds a score-matching term to the NPE loss [31 ] , leaving the proposal, the simulation allocation and the sampler unchanged. In all of these, gradients regularise an estimator fit to prior-wide simulations. In RSBI they instead drive the Phase 1 sampler directly, through an auto-encoder-like comparison of simulations with the observation.
Background
Simulation Based Inference
We consider a stochastic simulator s : Θ × Ω → X s : \Theta\times\Omega\to\mathcal{X} s : Θ × Ω → X with parameters θ ∈ Θ ⊆ R d \theta\in\Theta\subseteq\mathbb{R}^{d} θ ∈ Θ ⊆ R d and outputs x = s ( θ , ω ) ∈ X x = s(\theta,\omega) \in\mathcal{X} x = s ( θ , ω ) ∈ X , where ω ∈ Ω \omega\in\Omega ω ∈ Ω collects all internal randomness of the simulator (noise draws, random seeds) and is drawn from a distribution p ( ω ) p(\omega) p ( ω ) that does not depend on θ \theta θ . The simulator implicitly defines the conditional density p ( x ∣ θ ) p(x \mid\theta) p ( x ∣ θ ) and the distribution of s ( θ , ω ) s(\theta,\omega) s ( θ , ω ) under ω ∼ p ( ω ) \omega\sim p(\omega) ω ∼ p ( ω ) . The likelihood θ ↦ p ( x o ∣ θ ) \theta\mapsto p(x_o \mid\theta) θ ↦ p ( x o ∣ θ ) is this density evaluated at the observed data x o x_o x o . It can be sampled by running the simulator but not evaluated. We call the simulator differentiable if θ ↦ s ( θ , ω ) \theta\mapsto s(\theta,\omega) θ ↦ s ( θ , ω ) is differentiable for every ω \omega ω , so that pathwise gradients ∇ θ s ( θ , ω ) \nabla_{\theta}s(\theta,\omega) ∇ θ s ( θ , ω ) are available. We write s ( θ ) s(\theta) s ( θ ) for s ( θ , ω ) s(\theta,\omega) s ( θ , ω ) with ω ∼ p ( ω ) \omega\sim p(\omega) ω ∼ p ( ω ) , and s ( θ , ω 1 : K ) = { s ( θ , ω k ) } k = 1 K s(\theta,\omega_{1:K}) = \{s(\theta,\omega_k)\}_{k=1}^{K} s ( θ , ω 1 : K ) = { s ( θ , ω k ) } k = 1 K with ω k ∼ i i d p ( ω ) \omega_k \stackrel{\mathrm{iid}}{\sim} p(\omega) ω k ∼ iid p ( ω ) for K K K independent replicates at the same θ \theta θ . Given a prior p ( θ ) p(\theta) p ( θ ) and observation x o x_o x o , simulation-based inference (SBI) targets the posterior p ( θ ∣ x o ) ∝ p ( x o ∣ θ ) p ( θ ) p(\theta\mid x_o) \propto p(x_o \mid\theta)p(\theta) p ( θ ∣ x o ) ∝ p ( x o ∣ θ ) p ( θ ) using only forward simulations [1 ] .
Classical approximate Bayesian computation (ABC) accepts prior draws θ ∼ p ( θ ) \theta\sim p(\theta) θ ∼ p ( θ ) whose simulations fall within a tolerance ϵ \epsilon ϵ of the observation, D ϵ ( s ( θ ) , x o ) < ϵ D_{\epsilon}(s(\theta),x_o) < \epsilon D ϵ ( s ( θ ) , x o ) < ϵ , usually after reducing the data to summary statistics [32 ] . It scales poorly with dim( x ) (x) ( x ) and is sensitive to the choice of summaries and of ϵ \epsilon ϵ . Neural SBI instead trains a density estimator q ϕ q_{\phi} q ϕ on a corpus of simulations ( θ i , x i ) (\theta_i,x_i) ( θ i , x i ) with θ i ∼ p ( θ ) \theta_i \sim p(\theta) θ i ∼ p ( θ ) and x i ∼ p ( x ∣ θ i ) x_i \sim p(x \mid\theta_i) x i ∼ p ( x ∣ θ i ) , targeting the posterior (NPE) [2 , 3 ] , the likelihood (NLE) [4 ] or the likelihood-to-evidence ratio (NRE) [5 , 6 , 20 ] . Amortised variants train once to learn p ( θ ) p ( x ∣ θ ) p(\theta)p(x \mid\theta) p ( θ ) p ( x ∣ θ ) and apply to any x x x , whereas sequential variants concentrate simulations around x o x_o x o , giving up amortisation for simulation efficiency.
Diffusion Sampling
Neural diffusion samplers learn a Stochastic Differential Equation (SDE) which transports samples X 0 ∼ μ ( x ) X_0 \sim\mu(x) X 0 ∼ μ ( x ) from source distribution μ ( x ) \mu(x) μ ( x ) ; at time t = 0 t = 0 t = 0 to the Boltzmann distribution v ( x ) v(x) v ( x ) at time t = 1 t = 1 t = 1 given by:
v ( x ) = e − E ( x ) / T Z , Z = ∫ X e − E ( x ) / T d x < ∞ (1) v(x) = \frac{e^{-E(x)/T}}{Z}, \qquad Z = \int_{\mathcal{X}} e^{-E(x)/T}\,dx < \infty \tag*{(1)} v ( x ) = Z e − E ( x ) / T , Z = ∫ X e − E ( x ) / T d x < ∞ (1)
The distribution is characterized by an unnormalized energy E : X ⊆ R d → R E : \mathcal{X} \subseteq\mathbb{R}^{d} \to\mathbb{R} E : X ⊆ R d → R and temperature T T T , with the normalizing constant Z Z Z being intractable. Diffusion samplers aim to provide faster convergence using fewer evaluations of the energy function than Markov Chain Monte Carlo (MCMC) [33 ] methods by learning better proposal distributions. Especially when v ( x ) v(x) v ( x ) is defined through an energy function, diffusion samplers provide a scalable alternative to sampling methods such as score matching [34 , 35 ] or normalizing flows [2 , 4 ] whose naive implementations rely on samples from v ( x ) v(x) v ( x ) .
Adjoint Schrödinger Bridge Sampling: ASBS[14 ] is a state-of-the-art diffusion sampler building on Adjoint Sampling (AS)[15 ] that solves the Schrödinger Bridge (SB) problem between source distribution μ ( x ) \mu(x) μ ( x ) and target v ( x ) v(x) v ( x ) through stochastic optimal control. Within the context of RSBI, we consider the following drift-less SDE:
d X t = σ t u t ( X t ) d t + σ t d W t , X 0 ∼ μ ( X 0 ) , (2) dX_t = \sigma_t u_t(X_t)dt + \sigma_t dW_t,\qquad X_0 \sim\mu(X_0), \tag*{(2)} d X t = σ t u t ( X t ) d t + σ t d W t , X 0 ∼ μ ( X 0 ) , (2)
where σ t \sigma_t σ t is the diffusion coefficient, u t u_t u t is a learned control network, and ( W t ) t (W_t)_t ( W t ) t is a d d d -dimensional Brownian motion. The optimal control u t ∗ u_t^* u t ∗ is then given by:
u ∗ = arg min u D K L ( p u ∥ p b a s e ) = arg min u E X ∼ p u [ ∫ 0 1 1 2 ∥ u t ( X t ) ∥ 2 d t ] , (3) u^* = \underset{u}{\arg\min}\ D_{\mathrm{KL}}(p^u\|p^{\mathrm{base}}) = \underset{u}{\arg\min}\ \mathbb{E}_{X\sim p^u}\left[\int_0^1 \frac{1}{2}\|u_t(X_t)\|^2dt\right], \tag*{(3)} u ∗ = u arg min D KL ( p u ∥ p base ) = u arg min E X ∼ p u [ ∫ 0 1 2 1 ∥ u t ( X t ) ∥ 2 d t ] , (3)
s.t. d X t = σ t u t ( X t ) d t + σ t d W t , X 0 ∼ μ ( X 0 ) , X 1 ∼ v ( X 1 ) . (4) \text{s.t. } dX_t = \sigma_tu_t(X_t)dt + \sigma_tdW_t,\qquad X_0 \sim\mu(X_0),\qquad X_1 \sim v(X_1). \tag*{(4)} s.t. d X t = σ t u t ( X t ) d t + σ t d W t , X 0 ∼ μ ( X 0 ) , X 1 ∼ v ( X 1 ) . (4)
Therefore, u ∗ u^* u ∗ minimizes the relative entropy between the path probability p u p^u p u characterized by (2) and the base path probability p b a s e p^{\mathrm{base}} p base given when u = 0 u = 0 u = 0 . To solve (3), ASBS defines an additional corrector network h h h and iterates over two objectives; Adjoint Matching (AM):
u ( k ) = arg min u E p t ∣ 0 , 1 b a s e p 0 , 1 u [ ∥ u t ( X t ) + σ t ( ∇ X 1 E ( X 1 ) + h ( k − 1 ) ( X 1 ) ) ∥ 2 ] , u ˉ = stopgrad ( u ) , (5) u^{(k)} = \underset{u}{\arg\min}\ \mathbb{E}_{p^{\mathrm{base}}_{t|0,1}p^u_{0,1}}\left[\left\|u_t(X_t) + \sigma_t\left(\nabla_{X_1}E(X_1) + h^{(k-1)}(X_1)\right)\right\|^2\right],\qquad\bar{u} = \operatorname{stopgrad}(u), \tag*{(5)} u ( k ) = u arg min E p t ∣0 , 1 base p 0 , 1 u [ u t ( X t ) + σ t ( ∇ X 1 E ( X 1 ) + h ( k − 1 ) ( X 1 ) ) 2 ] , u ˉ = stopgrad ( u ) , (5)
and Corrector Matching (CM):
h ( k ) = arg min h E p 0 , 1 u [ ∥ h ( X 1 ) − ∇ X 1 log p b a s e ( X 1 ∣ X 0 ) ∥ 2 ] . (6) h^{(k)} = \underset{h}{\arg\min}\ \mathbb{E}_{p^u_{0,1}}\left[\left\|h(X_1)-\nabla_{X_1}\log p^{\mathrm{base}}(X_1|X_0)\right\|^2\right]. \tag*{(6)} h ( k ) = h arg min E p 0 , 1 u [ h ( X 1 ) − ∇ X 1 log p base ( X 1 ∣ X 0 ) 2 ] . (6)
Iterating through AM and CM stages constitutes the well-known iterative proportional fitting[36 ] algorithm.
Theorem 3.1 (Convergence of ASBS). As k → ∞ k \to\infty k → ∞ and both AM(5) and CM(6) stages reach their critical points, u ( k ) = u ∗ u^{(k)} = u^* u ( k ) = u ∗ , the SB solution of (3).
For a proof of Theorem 3.1 we refer to Liu et. al. [14 ] . Note that ASBS relies only on the energy gradient ∇ X 1 E ( X 1 ) \nabla_{X_1}E(X_1) ∇ X 1 E ( X 1 ) during the AM stage, and does not require any energy evaluations during the CM stage. This makes the algorithm efficient in terms of energy evaluations, and is what makes it highly suitable for use in RSBI, where we only have access to the likelihood and prior’s gradient. A consequence is that ASBS distinguishes between two kinds of low-density region. Because the energy is queried only at the terminal point X 1 X_1 X 1 , trajectories can pass through regions of low or zero target density in transit without obstruction, unlike MCMC, whose chains must traverse every intermediate state. Regions that separate components of the terminal distribution are different. Adjoint Matching trains on trajectories drawn from the current controller, so a component that no trajectory reaches provides no training signal and can remain uncovered. The well-tempered bias of Appendix § 11 .§ 1 addresses this by pushing samples towards regions not yet visited.
Method
We adopt the SBI setting of § 3.1 : a simulator s ( θ , ω ) s(\theta, \omega) s ( θ , ω ) , prior p ( θ ) p(\theta) p ( θ ) , and observation x o x_o x o , with the unnormalized posterior p ( θ ∣ x o ) p(\theta\mid x_o) p ( θ ∣ x o ) as the inference target. We additionally assume that s s s is differentiable in θ \theta θ , so that ∇ θ s ( θ , ω ) \nabla_\theta s(\theta, \omega) ∇ θ s ( θ , ω ) is available for any ω ∼ p ( ω ) \omega\sim p(\omega) ω ∼ p ( ω ) . A detailed illustration of RSBI can be found in Figure 2 . We also provide simplified pseudocode for ASBS in Algorithm 1 and for RSBI in Algorithm 2.
Figure 2. Left: Schematic representation of the two-phase procedure of RSBI. Right: Comparison of posteriors from RSBI (our method) and SNPE on a Two Moons task with 10 4 10^4 1 0 4 simulations and R p ∼ 10 4.3 \mathcal{R}_p \sim10^{4.3} R p ∼ 1 0 4.3 , with two sequential steps for both SNPE and Phase 2 of RSBI.
Phase 1: Surrogate-Guided Zoom
When the prior-to-posterior volume ratio R p \mathcal{R}_p R p is large, learning a neural likelihood or posterior over the full prior support is impractical: most simulator calls fall in regions of negligible posterior density. Phase 1 sidesteps this bottleneck with an analytic surrogate likelihood, in the spirit of ABC. Given a differentiable distance D ϵ ( s ( θ ) , x o ) \mathcal{D}_{\epsilon}(s(\theta), x_o) D ϵ ( s ( θ ) , x o ) between simulated and observed data, we define the surrogate likelihood
p ~ ϵ ( x o ∣ θ ) ∝ exp ( − E D ϵ ( s ( θ ) , x o ) ) , (7) \tilde{p}_{\epsilon}(x_o \mid\theta) \propto\exp\left(-\mathbb{E}\mathcal{D}_{\epsilon}(s(\theta), x_o)\right), \tag*{(7)} p ~ ϵ ( x o ∣ θ ) ∝ exp ( − E D ϵ ( s ( θ ) , x o ) ) , (7)
where s ( θ ) s(\theta) s ( θ ) inside D ϵ \mathcal{D}_{\epsilon} D ϵ denotes K K K replicates s ( θ , ω 1 : K ) s(\theta, \omega_{1:K}) s ( θ , ω 1 : K ) and the expectation is over ω 1 : K \omega_{1:K} ω 1 : K . The bandwidth ϵ > 0 \epsilon> 0 ϵ > 0 is the scale on which D ϵ \mathcal{D}_{\epsilon} D ϵ compares simulated and observed data, and so controls how sharply p ~ ϵ \tilde{p}_{\epsilon} p ~ ϵ concentrates around x o x_o x o . I.e, the noise variance of a Gaussian negative log-likelihood, or the length scale of an energy distance or MMD kernel. ϵ \epsilon ϵ is either fixed or set adaptively from the replicates. We detail how the adaptive bandwidth is set for different surrogate likelihoods D ϵ \mathcal{D}_{\epsilon} D ϵ and conduct ablations on it’s choice in Appendix § 13 . We also discuss a heuristic method that exploits a cheap survey of the gradient landscape in order to select ϵ \epsilon ϵ in Appendix § 13 .§ 7 .
The surrogate posterior p ~ ϵ ( θ ∣ x o ) ∝ p ~ ϵ ( x o ∣ θ ) p ( θ ) \tilde{p}_{\epsilon}(\theta\mid x_o) \propto\tilde{p}_{\epsilon}(x_o \mid\theta)p(\theta) p ~ ϵ ( θ ∣ x o ) ∝ p ~ ϵ ( x o ∣ θ ) p ( θ ) has the Boltzmann form of (1). ASBS only queries the gradient of its energy, so D ϵ \mathcal{D}_{\epsilon} D ϵ need only be known up to an additive constant:
Algorithm 1: Well-Tempered ASBS - One Round
1 def wt_asbs(source μ, energy gradient ∇E, control u, corrector h, bias V(ξ(θ)), stages S):
2 for stage k in 1, 2, . . . , S and each epoch :
3 Obtain X₁ by drawing X₀ ∼ μ and integrating (2)
4 if k is odd (AM stage) :
5 Add (X₀, X₁, ∇E(X₁) + h(X₁) + ∇(V∘ξ)(X₁)) to the replay buffer
6 if k < S : V ← V + WT update (9) at ξ(X₁)
7 Update u on the buffer through WT-AM (10)
8 else: add (X₀, X₁) to the replay buffer and update h by solving CM (5)
9 return u, h, V, saved simulations 𝓑
Algorithm 2: Rapid Simulation-Based Inference
def rsbi(prior p ( θ ) p(\theta) p ( θ ) , simulator s ( θ , ω ) s(\theta,\omega) s ( θ , ω ) , observation x o x_o x o , surrogate distance D ϵ ( s , x o ) D_\epsilon(s,x_o) D ϵ ( s , x o ) , temperature T T T , ratio network r ψ ( θ , x ) r_\psi(\theta,x) r ψ ( θ , x ) , rounds R R R ):
Phase 1: Surrogate-guided zoom E s ( θ ) ← [ E D ϵ ( s ( θ ) , x o ) − log p ( θ ) ] / T E_s(\theta) \leftarrow[\mathbb{E}\,D_\epsilon(s(\theta),x_o)-\log p(\theta)]/T E s ( θ ) ← [ E D ϵ ( s ( θ ) , x o ) − log p ( θ )] / T u ˉ , V ˉ , B ← wt_asbs ( μ = p ( θ ) , ∇ E = ∇ E s , new u , new h , V = 0 ) \bar{u},\bar{V},\mathcal{B} \leftarrow\operatorname{wt\_asbs}(\mu=p(\theta),\nabla E=\nabla E_s,\text{new }u,\text{new }h,V=0) u ˉ , V ˉ , B ← wt_asbs ( μ = p ( θ ) , ∇ E = ∇ E s , new u , new h , V = 0 ) Draw θ i ∼ resample V ˉ ( p 1 u ˉ ) \theta_i \sim\operatorname{resample}_{\bar{V}}(p_1^{\bar{u}}) θ i ∼ resample V ˉ ( p 1 u ˉ ) , top up B \mathcal{B} B with new ( θ i , s ( θ i ) ) (\theta_i,s(\theta_i)) ( θ i , s ( θ i )) μ ← p 1 u ˉ \mu\leftarrow p_1^{\bar{u}} μ ← p 1 u ˉ , optionally resample V ˉ ( p 1 u ˉ ) \bar{V}(p_1^{\bar{u}}) V ˉ ( p 1 u ˉ ) ; u 0 , h 0 ← u_0,h_0 \leftarrow u 0 , h 0 ← fresh networks
Phase 2: Sequential NRE-ASBS for r r r in 1 , 2 , … , R 1,2,\ldots,R 1 , 2 , … , R :
Optimise r ψ ( θ , x ) r_\psi(\theta,x) r ψ ( θ , x ) on B \mathcal{B} B via the NRE-C loss (11)
E r ( θ ) ← − log r ψ ( θ , x o ) − log p ( θ ) E_r(\theta)\leftarrow-\log r_\psi(\theta,x_o)-\log p(\theta) E r ( θ ) ← − log r ψ ( θ , x o ) − log p ( θ ) (8)
u r , h r , V r , _ ← wt_asbs ( μ , ∇ E = ∇ E r , u r − 1 , h r − 1 , V = 0 ) u_r,h_r,V_r,\_ \leftarrow\operatorname{wt\_asbs}(\mu,\nabla E=\nabla E_r,u_{r-1},h_{r-1},V=0) u r , h r , V r , _ ← wt_asbs ( μ , ∇ E = ∇ E r , u r − 1 , h r − 1 , V = 0 )
if r < R r<R r < R : draw θ i ∼ p 1 u r \theta_i\sim p_1^{u_r} θ i ∼ p 1 u r , top up B \mathcal{B} B
return μ , u R , V R \mu,u_R,V_R μ , u R , V R
Theorem 4.1 (Equivalence under differentiation). Let D ~ ϵ : X K × X → R \widetilde{D}_\epsilon:\mathcal{X}^K\times\mathcal{X}\to\mathbb{R} D ϵ : X K × X → R be differentiable and equal to a positive-semi-definite (PSD) distance D ϵ D_\epsilon D ϵ up to a constant, D ϵ ( x , x ′ ) = D ~ ϵ ( x , x ′ ) + C D_\epsilon(x,x')=\widetilde{D}_\epsilon(x,x')+C D ϵ ( x , x ′ ) = D ϵ ( x , x ′ ) + C for some C ∈ R C\in\mathbb{R} C ∈ R and all x , x ′ x,x' x , x ′ . Then ∇ D ~ ϵ = ∇ D ϵ \nabla\widetilde{D}_\epsilon=\nabla D_\epsilon ∇ D ϵ = ∇ D ϵ .
The energy gradient can therefore be computed from any such D ~ ϵ \widetilde{D}_\epsilon D ϵ ,
∇ θ E s ( θ ) = E ∇ θ D ~ ϵ ( s ( θ ) , x o ) − ∇ θ log p ( θ ) , (8) \nabla_\theta E_s(\theta)=\mathbb{E}\,\nabla_\theta\widetilde{D}_\epsilon(s(\theta),x_o)-\nabla_\theta\log p(\theta), \tag*{(8)} ∇ θ E s ( θ ) = E ∇ θ D ϵ ( s ( θ ) , x o ) − ∇ θ log p ( θ ) , (8)
where the gradient passes inside the expectation because p ( ω ) p(\omega) p ( ω ) does not depend on θ \theta θ . We hand (7) to ASBS with the prior p ( θ ) p(\theta) p ( θ ) as source (full derivation in Appendix § 11 .§ 3 ). By Theorem 3.1 , this converges to the Schrödinger-bridge solution that transports p ( θ ) p(\theta) p ( θ ) to the surrogate posterior. Although it may seem trivial, Theorem 4.1 lets standard losses be used without their constants, such as the Gaussian NLL without its normalising constant and MSE, alongside PSD discrepancies such as the energy distance [37 ] and MMD [38 ] . A Gaussian NLL with adaptive variance is not of this form but still gives a proper surrogate posterior (Appendix § 13 .§ 1 ).
Because p ~ ϵ ( θ ∣ x o ) \widetilde{p}_\epsilon(\theta\mid x_o) p ϵ ( θ ∣ x o ) is defined directly through the simulator rather than learned, Phase 1 needs no density estimator. Each simulator call moves the proposal towards x o x_o x o instead of providing a training example for an estimator fitted from prior draws, which is what limits sequential neural methods at large R p R_p R p . The distance gradient points toward x o x_o x o in data space, which the differentiable simulator pulls back into parameter space via ∇ θ s ( θ ) \nabla_\theta s(\theta) ∇ θ s ( θ ) . Phase 1 therefore efficiently allocates posterior mass to high-density regions under a constrained simulator budget, producing the focused proposal p ~ ϵ ( θ ∣ x o ) \widetilde{p}_\epsilon(\theta\mid x_o) p ϵ ( θ ∣ x o ) that Phase 2 refines. Appendix § 11 .§ 3 gives the admissibility class for D ϵ D_\epsilon D ϵ , whilst Appendix § 13 .§ 4 discusses the bandwidth–metric trade-offs inherited from ABC, and the surrogates used in § 7 .
Phase 2: Sequential NRE-ASBS Refinement
In Phase 2 we start from the surrogate posterior p ~ ϵ ( θ ∣ x o ) \widetilde{p}_\epsilon(\theta\mid x_o) p ϵ ( θ ∣ x o ) produced by Phase 1 and refine it to recover the simulator posterior p ( θ ∣ x o ) p(\theta\mid x_o) p ( θ ∣ x o ) (§ 3.1 ). Refinement proceeds over R R R rounds. Each round trains a neural ratio estimator [5 , 6 ] r ψ ( θ , x ) r_\psi(\theta,x) r ψ ( θ , x ) on ( θ , x ) (\theta,x) ( θ , x ) pairs simulated so far, which are drawn from the round’s proposal q r ( θ ) q_r(\theta) q r ( θ ) as well as optionally earlier ones used during phase 1 training. It then solves the ASBS objective (3) against the resulting target energy, with p ~ ϵ ( θ ∣ x o ) \widetilde{p}_\epsilon(\theta\mid x_o) p ϵ ( θ ∣ x o ) as the source distribution in every round. The SB therefore always transports samples from the Phase-1 posterior, and the sampler of round r r r provides the proposal q r + 1 q_{r+1} q r + 1 for the next round’s simulations. We use the contrastive (NRE-C) loss [20 ] throughout. To improve mode-discovery over both phases we also optionally use a simplified version of the recent Well-Tempered metadynamics (WT-ASBS) [16 ] extension that augments the energy landscape, for more details on WT-ASBS we refer to Appendix § 11 .§ 1 .
At optimum the classifier learns r ψ ∗ ( θ , x ) = p ( x ∣ θ ) / q r ( x ) r_\psi^*(\theta,x)=p(x\mid\theta)/q_r(x) r ψ ∗ ( θ , x ) = p ( x ∣ θ ) / q r ( x ) , where q r ( x ) q_r(x) q r ( x ) is the local marginal. Because Phase 2 conducts multiple rounds of refinement, the proposal q r ( θ ) q_r(\theta) q r ( θ ) changes between rounds as simulations concentrate in regions of high posterior probability. Following the same construction as the Phase 1 energy (7), the round-r r r energy is:
− ∇ θ log p ( θ ∣ x o ) = − ∇ θ log p ( x o ∣ θ ) q r ( x o ) − ∇ θ log p ( θ ) = ∇ θ E r ( θ ) , (9) -\nabla_\theta\log p(\theta\mid x_o) = -\nabla_\theta\log\frac{p(x_o \mid\theta)}{q_r(x_o)} - \nabla_\theta\log p(\theta) = \nabla_\theta E_r(\theta), \tag*{(9)} − ∇ θ log p ( θ ∣ x o ) = − ∇ θ log q r ( x o ) p ( x o ∣ θ ) − ∇ θ log p ( θ ) = ∇ θ E r ( θ ) , (9)
where the proposal-induced marginal q r ( x o ) q_r(x_o) q r ( x o ) has zero gradient in θ \theta θ and so drops out - this is a core appeal of using a diffusion sampler against an NRE-derived energy. Solving (3) via ASBS with energy E r ( θ ) E_r(\theta) E r ( θ ) and source p ~ ϵ ( θ ∣ x o ) \tilde{p}_{\epsilon}(\theta\mid x_o) p ~ ϵ ( θ ∣ x o ) therefore recovers the Schrödinger-bridge solution connecting the surrogate posterior and the simulator posterior. To avoid retraining from scratch each round we warm-start the controller u u u and corrector h h h from the previous round’s parameters. Because r ψ r_\psi r ψ is only identified where the classifier has seen training parameters, a user who trusts the Phase 1 posterior to cover the simulator posterior can optionally truncate the ratio outside the region spanned by the Phase 1 samples (Appendix § 11 .§ 4 ). Because E r E_r E r in (9) explicitly retains log p ( θ ) \log p(\theta) log p ( θ ) as an additive term, the round output requires no post-hoc correction for leakage outside prior support. Natural posterior leakage mitigation is an additional benefit of RSBI which we demonstrate with an additional experiment in Appendix § 12 .§ 2 .
Because Phase 2 transports the Phase 1 sampler rather than refitting a global density over the prior, it remains accurate where SNPE and SNVI, warm-started from the same Phase 1 samples and given the same simulation budget, still degrade or collapse onto spurious regions of a wide prior. We demonstrate this in an ablation in Appendix § 12 .§ 3 .
The full Phase 2 procedure including NRE-C training, the well-tempered bias update (9), and the augmented Adjoint Matching objective (10) is described in Appendices § 11 .§ 1 and § 11 .§ 2 , alongside the convergence guarantee (Proposition A.1) for RSBI.
Preconditioning the Phase-2 sampler for unimodal posteriors.
The Phase 2 sampling process is isotropic, so on ill-conditioned posteriors (e.g strongly correlated parameters, narrow ridges) no single noise scale σ t \sigma_t σ t suits every direction. We therefore provide an option to run Phase 2 in coordinates whitened by the mean θ ˉ \bar{\theta} θ ˉ and covariance Σ = L L ⊤ \Sigma= LL^\top Σ = L L ⊤ of the Phase 1 samples, z = L − 1 ( θ − θ ˉ ) z = L^{-1}(\theta- \bar{\theta}) z = L − 1 ( θ − θ ˉ ) , mapping only the energy gradient, ∇ z E r = L ⊤ ∇ θ E r \nabla_z E_r = L^\top\nabla_\theta E_r ∇ z E r = L ⊤ ∇ θ E r . The affine map leaves the target unchanged and costs no simulations. It is the analogue of mass-matrix adaptation in HMC [39 , 40 ] , with the metric supplied by Phase 1. As one covariance cannot describe several modes, Phase-2 preconditioning is useful for unimodal targets. For more discussion see Appendix. § 11 .§ 7 .
Experiments
We benchmark RSBI against two established neural-SBI methods: SNPE-C [3 ] and SNVI+fKL [7 ] (see Appendix § 15 for further details), on Two Moons [3 ] , SLCP [4 ] , Bernoulli GLM [18 ] and Lotka–Volterra [18 ] , as implemented in the sbibm [18 ] benchmarking suite, as well as a new differentiable gravitational-wave ringdown task (gw_ringdown). To make them differentiable, we re-implement the benchmarks from sbibm in PyTorch. For full details on the re-implementation we refer to Appendix § 14 . We make public our full code and checkpoints1 .
Posterior quality is measured by the classifier two-sample test (C2ST, [41 ] ) against task-appropriate reference posteriors, obtained via rejection sampling for Two Moons, sbibm’s reference samplers for SLCP, Bernoulli GLM, resampling the sbibm default for Lotka–Volterra, and MCMC with emcee [42 ] for gw_ringdown (see Appendix § 14 .§ 9 for details). For each task we sweep several prior sizes (uniform boxes, or Gaussian prior scales for Bernoulli GLM and Lotka–Volterra) and three simulation budgets N s i m ∈ { 10 3 , 10 4 , 10 5 } N_{\mathrm{sim}} \in\{10^3, 10^4, 10^5\} N sim ∈ { 1 0 3 , 1 0 4 , 1 0 5 } , using the log prior-to-posterior volume ratio log 10 R p \log_{10} \mathcal{R}_p log 10 R p , averaged over the observations, to quantify the volume reduction (see App. § 16 for the definition and App. § 14 .§ 10 for the values adopted here). The smallest priors are the ones implemented in the sbibm package. All baselines use their released sbibm or SNVI defaults at matched total simulator-call budget, with the number of SNVI rounds set to 5 and SNPE rounds set to 2-3 (instead of 10) to match RSBI closer and reduce computational cost. Lastly, the surrogate bandwidths were taken to be adaptive where possible. The Bernoulli-GLM experiment instead implemented the heuristic criteria of Appendix § 13 .§ 7 to choose ϵ g w _ r i n g d o w n ∈ = 1 \epsilon_{\mathrm{gw\_ringdown}} \in= 1 ϵ gw_ringdown ∈= 1 . Full task definitions, prior scenarios, hyperparameters, reference-sampler configurations and the RSBI training schedule are in Appendix § 14 .
Compute-Matched Experiment: Because doing the full forward + backward pass through a simulator is more expensive than only a forward pass, we also investigate RSBI’s performance under a compute-matched budget. These results are shown in Appendix B.1.
Main results: gravitational-wave ringdown
We first evaluate RSBI on a real-world inference task: estimating six ringdown parameters ( M , χ , A 220 , φ 220 , A 221 , φ 221 ) (M, \chi, A_{220}, \varphi_{220}, A_{221}, \varphi_{221}) ( M , χ , A 220 , φ 220 , A 221 , φ 221 ) from a black hole post-merger gravitational wave signal, with extrinsic dimensions fixed to truth (the natural single-detector setup with known sky position and distance). Simulator construction and reference posterior generation are described in Appendix D.2. Figure 3 shows the d θ = 6 d_\theta= 6 d θ = 6 2-d and 1-d marginal posteriors on observation 10 at the default and wide priors from N s i m = 10 5 N_{\mathrm{sim}} = 10^5 N sim = 1 0 5 : RSBI tracks the emcee reference closely across all six parameters, with a more constrained posterior in comparison to that of SNPE, as also evident in their respective C2ST scores (see Table 1 ) for the default prior (0.891 and 0.935, respectively) and wide prior (0.903 and 0.985, respectively). The strain inset shows the 2 σ 2\sigma 2 σ posterior-predictive band of RSBI tightly envelopes the injected ringdown signal, in comparison to the larger uncertainties of SNPE. We highlight the difference in the MCMC and RSBI posteriors, especially visible in the 1-dimensional marginal plot of A 220 A_{220} A 220 (see top row of the A 220 A_{220} A 220 corner column).
Figure 3. 6D gw_ringdown posteriors of RSBI, SNPE and the MCMC reference for observation 10 at N s i m = 10 5 N_{\mathrm{sim}} = 10^5 N sim = 1 0 5 , under the default prior (left, log 10 R p ≈ 2.62 \log_{10} R_p \approx2.62 log 10 R p ≈ 2.62 ) and the wide prior (right, ≈ 2.84 \approx2.84 ≈ 2.84 ; Table 21 ). Insets: whitened observation (grey), noise-free injection (red) and the 1 σ 1\sigma 1 σ posterior-predictive bands of SNPE and RSBI from 200 re-simulated posterior samples each.
Table 1. Mean C2ST on gw_ringdown against the emcee reference at N s i m = 10 5 N_{\mathrm{sim}} = 10^5 N sim = 1 0 5 , averaged over 10 simulated observations. For every method the phases φ 220 , φ 221 \varphi_{220}, \varphi_{221} φ 220 , φ 221 are wrapped to [ 0 , 2 π ) [0, 2\pi) [ 0 , 2 π ) before scoring. Prior (see Table 21 ).
Main results: robustness under R p R_p R p increase
Figure 4 reports C2ST against the simulation budget ( N s i m ) (N_{\mathrm{sim}}) ( N sim ) on the Two Moons, Lotka-Volterra, Bernoulli GLM, and SLCP benchmarks [18 ] . On Two Moons and Bernoulli GLM, all three methods start to converge towards 0.5 C2ST at N s i m = 10 5 N_{\mathrm{sim}} = 10^5 N sim = 1 0 5 . On the Lotka-Volterra and SLCP benchmarks this is not the case, with only RSBI converging on Lotka-Volterra. We also find that the C2ST gap widens as R p R_p R p increases. On Two Moons at log 10 R p ≈ 4.3 \log_{10} R_p \approx4.3 log 10 R p ≈ 4.3 , the sequential sbi baselines saturate at C2ST ≈ 1 \approx1 ≈ 1 for every budget, while RSBI maintains C2ST ≲ 0.7 \lesssim0.7 ≲ 0.7 throughout. On SLCP, all baselines degrade by 0.1–0.2 C2ST between the sbibm default and log 10 R p ≈ 2.69 \log_{10} R_p \approx2.69 log 10 R p ≈ 2.69 at 10 5 10^5 1 0 5 simulations. With the smallest R p R_p R p difference between priors, this experiment details a regime for which the gains of RSBI are not expected to be as pronounced. On Bernoulli GLM, all methods agree at the default prior, but at log 10 R p ≈ 9.9 \log_{10} R_p \approx9.9 log 10 R p ≈ 9.9 RSBI reaches a C2ST of 0.66 at 10 5 10^5 1 0 5 simulations, against 0.74 for SNVI+fKL and 0.80 for SNPE, and at log 10 R p ≈ 12.2 \log_{10} R_p \approx12.2 log 10 R p ≈ 12.2 0.89 against 0.91 and 0.95. With Lotka-Volterra we find only RSBI is able to achieve strong results within the simulation budget, reaching a C2ST of 0.55 at 10 5 10^5 1 0 5 simulations against 0.88 for SNVI+fKL and 0.94 for SNPE. On Lotka-Volterra we also implement the Phase 2 preconditioning of § 6.1 which considerably enhanced performance.
Figure 4. Robustness to prior-to-posterior volume ratio on Two Moons, Lotka–Volterra, Bernoulli GLM and SLCP benchmarks. Mean C2ST (± 1 σ \pm1\sigma ± 1 σ ) calculated against the reference posterior vs. simulation budget N s i m N_{\mathrm{sim}} N sim over 10 observations. Legends report log 10 R p \log_{10} R_p log 10 R p , full line is the sbibm default. C2ST = 0.5 matches the reference, 1.0 is fully distinguishable.
To investigate the convergence of RSBI under large simulation budgets, we also conduct an additional run on Two Moons, extending the number of sims to 10 6 10^6 1 0 6 on the default prior. For this run we achieve a mean C2ST (± 1 σ \pm1\sigma ± 1 σ ) of 0.536 (± 0.014 \pm0.014 ± 0.014 ), a slight improvement over the 10 5 10^5 1 0 5 run, which achieves 0.543 (± 0.018 \pm0.018 ± 0.018 ).
Discussion
The RSBI methodology introduces two novel components that can each be adopted independently of ASBS. The first is a surrogate proposal from a differentiable simulator. (6) is an unnormalised log density that can be differentiated with respect to θ \theta θ by backpropagating through the simulator, so any gradient-based method can target it. For example, VI can be run as in SNVI [7 ] with the learned likelihood replaced by the surrogate distance. Alternatively, ASBS can be swapped for annealed Langevin dynamics or another diffusion sampler designed for multimodal targets, which avoids the sampler-specific hyperparameters that ASBS requires.
The second component is diffusion-based NRE refinement via transport. A ratio estimator remains valid under any proposal, so every simulation is can be reused, and log p ( θ ) + log r ψ ( θ , x o ) \log p(\theta) + \log r_{\psi}(\theta, x_o) log p ( θ ) + log r ψ ( θ , x o ) is sampled starting from the Phase 1 proposal rather than from the prior or a truncated version of it. The proposal is not sufficient on its own. Our ablation (Appendix § 12 .§ 3 ) shows that seeding SNPE and SNVI with the Phase 1 proposal improves them, but a substantial further gain comes only when Phase 2 refines the Phase 1 proposal via NRE-ASBS transport. Additionally, preconditioning Phase 2 by whitening the Phase 1 proposal (§ 6.1 ) can alleviate issues arising from ill-conditioned unimodal posteriors, potentially improving the performance of the ASBS. On gw_ringdown (§ 7.1 ) RSBI is the only method substantially below a C2ST of 1, but the remaining discrepancy with the reference is concentrated in the periodic phases ϕ 220 , ϕ 221 \phi_{220}, \phi_{221} ϕ 220 , ϕ 221 , which no method resolves well. The cost of backpropagating through the simulator is incurred only in Phase 1 (Appendix § 12 .§ 1 ).
A further consequence of driving the proposal with a surrogate is that it can be engineered in ways a learned estimator cannot. Since the surrogate only has to point towards the posterior, it can be relaxed for discrete simulators, as with the straight-through Bernoulli relaxation (Appendix § 14 .§ 7 ). The prior also enters both Phase 1 and Phase 2 energies exactly, mitigating posterior mass leakage outside the support (Appendix § 12 .§ 2 ), whereas truncation-based methods must correct for this after the fact.
Limitations. RSBI requires a differentiable simulator. For black-box simulators, classical sequential SBI remains the appropriate choice. A backward pass through the simulator costs 1.7 to 4.0 times a forward pass (Appendix § 12 .§ 1 ), although this applies only to the Phase 1 share of the budget. Phase 2 costs more per iteration than amortised NPE, so RSBI pays off when simulation rather than network training is the bottleneck. It is also not amortised across x o x_o x o , a new x o x_o x o requires re-running both phases. Phase 1 adds three inference-level hyperparameters, the surrogate bandwidth ϵ \epsilon ϵ , the distance D ϵ D_\epsilon D ϵ and the temperature T T T (Appendices § 13 .§ 8 and § 13 .§ 9 ). Finally, MCMC is exact in the limit of infinite samples, whereas RSBI, like other neural SBI methods, is bounded by network capacity and optimization. On two moons, for example, accuracy saturates even at 10 6 10^6 1 0 6 simulations (§ 7.2 ).
A further set of limitations stems from our choice of ASBS as the sampler rather than from RSBI itself. ASBS is sensitive to its own hyperparameters, notably the reference noise σ max \sigma_{\max} σ m a x , the temperature and the training schedule. We set these per benchmark, keeping them as agnostic as possible to the surrogate bandwidth by using an adaptive variant or, for the ablations in Appendix § 13 , by inspecting the gradient landscape. The guarantees of the RSBI target against prior leakage and mode collapse only carry over to the samples if the ASBS sampler converges (Appendix § 12 .§ 2 ). When the reference noise σ max \sigma_{\max} σ m a x is comparable to the separation between modes, the mode-seeking objective and the self-refilled buffer can amplify small imbalances until modes empty. An unconverged controller can also scatter samples outside the support. These are failures of the sampler’s optimisation rather than of the target, but in practice they require σ max \sigma_{\max} σ m a x to be set relative to the posterior geometry. We stress that the novelty of RSBI, a surrogate-driven proposal refined against a target that retains the prior and the simulator, holds for any choice of sampler, and ASBS can be replaced by any method that requires only ∇ θ E \nabla_\theta\mathcal{E} ∇ θ E .
Conclusion
We introduced RSBI, a sequential SBI method that uses gradients of a differentiable simulator to zoom from a broad prior to the posterior before any estimator is trained, subsequently refining the prediction by transporting the proposal via diffusion under an NRE target. Each RSBI component is compatible with existing sequential pipelines and with any gradient-based sampler or VI. We showed RSBI is competitive across simulation budgets on broad priors where prior-initialized methods spend most of their simulations far from x o x_o x o .
Future work. A systematic study of the gradient landscape induced by differentiable simulators would show when this signal is reliable, and could yield curricula that adapt automatically instead of being set per task. Furthermore, the Phase 2 preconditioning we utilize is a single global affine whitening estimated from Phase 1 samples (Appendix § 11 .§ 7 ). We will quantify how much preconditioning ASBS needs as the posterior becomes more anisotropic or curved, and replace the global map with adaptive or nonlinear alternatives, such as running estimates of the posterior covariance or learned transport maps.
Two high-impact scientific applications stand out. One is extreme-mass-ratio inspirals (EMRIs), where the broad-prior regime is even more pronounced than for ringdown. The other is inference with neural emulators: any trained emulator is a differentiable simulator usable within our framework, and it reduces the per-call simulator cost to a network forward pass. The RSBI target is independent of the sampler, so in future work we will pair it with other samplers such as Langevin dynamics, flow-based proposals, or ASBS with geometry-adapted reference processes.
Acknowledgments
The authors thank the International Max Planck Research School for Intelligent Systems (IMPRS-IS) for supporting Uros Zivanovic. This work was supported by the Max Planck Artificial Intelligence Network (MP-AIX). RT and AS acknowledge funding from Next Generation EU, in the context of the National Recovery and Resilience Plan, Investment PE1 Project FAIR “Future Artificial Intelligence Research”. This resource was co-financed by the Next Generation EU [DM 1555 del 11.10.22]. RT is partially supported by the Fondazione ICSC, Spoke 3 “Astrophysics and Cosmos Observations”, Piano Nazionale di Ripresa e Resilienza Project ID CN00000013 “Italian Research Center on High-Performance Computing, Big Data and Quantum Computing” funded by MUR Missione 4 Componente 2 Investimento 1.4: Potenziamento strutture di ricerca e creazione di “campioni nazionali di R&S (M4C2-19)” - Next Generation EU (NGEU).
Additional methodological details
Countering Mode-Seeking
RSBI is variational, so training the controlled SDE against E r E_r E r amounts to minimising a reverse KL between the controlled-SDE marginal and the round-r r r target. Standard reverse-KL VI is mode-seeking [7 ] : when the target has multiple disconnected modes, the optimal proposal concentrates on whichever mode is cheapest to cover and abandons the rest. ASBS partially counters this at no extra cost: its replay buffer maintains trajectories from earlier in training, preventing the controller from collapsing onto a single mode within an ASBS round, and its small base diffusion coefficient σ \sigma σ preserves separation between modes that have already been discovered. Neither mechanism, however, drives the controller toward modes it has not yet visited; on multimodal posteriors with high inter-mode barriers, the ASBS optimum can still be a strict subset of the true target’s support.
We address this directly with the well-tempered ASBS extension of Nam et al. [16 ] : a low-dimensional collective variable ξ : Θ → S ⊆ R m \xi: \Theta\to S \subseteq\mathbb{R}^{m} ξ : Θ → S ⊆ R m (m ≤ d θ m \le d_\theta m ≤ d θ , chosen per-task) is introduced, and a bias V k ( ξ ( θ ) ) V_k(\xi(\theta)) V k ( ξ ( θ )) is deposited online during training. The bias accumulates in regions of CV space the sampler has already visited and pushes future samples toward novel ones. With kernel height h w t h_{\mathrm{wt}} h wt , kernel width σ c v \sigma_{\mathrm{cv}} σ cv , and bias factor γ > 1 \gamma> 1 γ > 1 , the update rule is
V k + 1 ( s ) = V k ( s ) + h w t ∑ i = 1 N exp ( − β γ − 1 V k ( ξ ( θ ( i ) ) ) ) K σ c v ( s , ξ ( θ ( i ) ) ) , (10) V_{k+1}(s) = V_k(s) + h_{\mathrm{wt}} \sum_{i=1}^{N} \exp\left(-\frac{\beta}{\gamma- 1} V_k\left(\xi\left(\theta^{(i)}\right)\right)\right) K_{\sigma_{\mathrm{cv}}}\left(s,\xi\left(\theta^{(i)}\right)\right), \tag*{(10)} V k + 1 ( s ) = V k ( s ) + h wt i = 1 ∑ N exp ( − γ − 1 β V k ( ξ ( θ ( i ) ) ) ) K σ cv ( s , ξ ( θ ( i ) ) ) , (10)
where K σ c v K_{\sigma_{\mathrm{cv}}} K σ cv is a Gaussian kernel and the θ ( i ) \theta^{(i)} θ ( i ) are the i.i.d. samples produced by the current controlled SDE. The well-tempering factor exp ( − β V k / ( γ − 1 ) ) \exp(-\beta V_k/(\gamma-1)) exp ( − β V k / ( γ − 1 )) shrinks the deposition rate as bias accumulates, so V k V_k V k converges to a finite limit rather than diverging. The Adjoint Matching objective (5) is augmented with the bias gradient through the chain rule:
u ( k ) = arg min u E p t ∣ 0 , 1 b a s e p 0 , 1 μ ~ [ ∥ u t ( X t ) + σ t ( ∇ E ( X 1 ) + ∇ ( V k ∘ ξ ) ( X 1 ) + h ( k − 1 ) ( X 1 ) ) ∥ 2 ] . (11) u^{(k)} = \underset{u}{\arg\min}\ \mathbb{E}_{p_{t|0,1}^{\mathrm{base}} p_{0,1}^{\tilde{\mu}}}\left[\left\|u_t(X_t) + \sigma_t\left(\nabla E(X_1) + \nabla(V_k \circ\xi)(X_1) + h^{(k-1)}(X_1)\right)\right\|^2\right]. \tag*{(11)} u ( k ) = u arg min E p t ∣0 , 1 base p 0 , 1 μ ~ [ u t ( X t ) + σ t ( ∇ E ( X 1 ) + ∇ ( V k ∘ ξ ) ( X 1 ) + h ( k − 1 ) ( X 1 ) ) 2 ] . (11)
After training, the bias is removed via importance reweighting by w ( θ ) ∝ exp ( β V ∗ ( ξ ( θ ) ) ) w(\theta) \propto\exp(\beta V^*(\xi(\theta))) w ( θ ) ∝ exp ( β V ∗ ( ξ ( θ ))) , recovering weighted samples from the original Phase-2 target.
Proposition A.1 (Convergence of RSBI). Within each Phase-2 round, under the regularity conditions of Proposition 3.1 of Nam et al. [16 ] , compact CV domain, positive-definite Gaussian kernel, and Robbins-Monro step schedule, the bias V k V_k V k in (10) converges almost surely to V ∗ ( s ) = − ( 1 − 1 / γ ) F r ( s ) + c o n s t V^*(s) = -(1 - 1/\gamma)F_r(s) + \mathrm{const} V ∗ ( s ) = − ( 1 − 1/ γ ) F r ( s ) + const , where F r ( s ) F_r(s) F r ( s ) is the round-r r r PMF along ξ \xi ξ . Combined with Theorem 3.1 and the consistency of the NRE-C estimator r ψ r → p ( x ∣ θ ) / q r ( x ) r_{\psi_r} \to p(x \mid\theta)/q_r(x) r ψ r → p ( x ∣ θ ) / q r ( x ) (Appendix § 11 .§ 2 ), when Algorithm 2 is run to convergence the round-r r r marginal of the controlled SDE, reweighted by w w w , converges to the target posterior p ( θ ∣ x o ) p(\theta\mid x_o) p ( θ ∣ x o ) .
Algorithm 2 details the full RSBI procedure. The structure mirrors that of Glöckler et al. [7 ] : a proposal is refined across rounds, but the target p ( θ ∣ x o ) p(\theta\mid x_o) p ( θ ∣ x o ) is held fixed throughout and reweighting is analytic rather than learned.
Training the ratio estimator
Phase 2’s energy E ψ r E_{\psi_r} E ψ r requires a ratio estimator r ψ r ( θ , x o ) r_{\psi_r}(\theta,x_o) r ψ r ( θ , x o ) that tracks the log-ratio log p ( x o ∣ θ ) / q r ( x o ) \log p(x_o \mid\theta)/q_r(x_o) log p ( x o ∣ θ ) / q r ( x o ) , whose dependence on θ \theta θ is proposal-invariant. We train r ψ r r_{\psi_r} r ψ r with the contrastive NRE-C loss of Miller et al. [20 ] , which generalises NRE-A [5 ] and NRE-B [6 ] . Given a minibatch of B B B paired samples ( θ b , x b ) b = 1 B (\theta_b,x_b)_{b=1}^{B} ( θ b , x b ) b = 1 B with θ b ∼ q r ( θ ) \theta_b \sim q_r(\theta) θ b ∼ q r ( θ ) and x b = s ( θ b , ω b ) x_b = s(\theta_b,\omega_b) x b = s ( θ b , ω b ) from the round-r r r training set B B B of Algorithm 2, where q r q_r q r is the mixture of all proposals so far, plus in-batch contrastive parameters θ k ∼ q r \theta_k \sim q_r θ k ∼ q r drawn for each anchor from the other pairs, the NRE-C objective with K c l a s s K_{\mathrm{class}} K class classes is
L N R E − C ( ψ ) = − 1 1 + η E q r ( x ) q r ⊗ K c l a s s ( θ 1 : K c l a s s ) [ log K c l a s s K c l a s s + η ∑ k = 1 K c l a s s r ψ ( θ k , x ) ] (12) \mathcal{L}_{\mathrm{NRE-C}}(\psi) = -\frac{1}{1+\eta}\mathbb{E}_{q_r(x)q_r^{\otimes K_{\mathrm{class}}}(\theta_1:K_{\mathrm{class}})}\left[\log\frac{K_{\mathrm{class}}}{K_{\mathrm{class}}+\eta\sum_{k=1}^{K_{\mathrm{class}}}r_\psi(\theta_k,x)}\right] \tag*{(12)} L NRE − C ( ψ ) = − 1 + η 1 E q r ( x ) q r ⊗ K class ( θ 1 : K class ) [ log K class + η ∑ k = 1 K class r ψ ( θ k , x ) K class ] (12)
− η 1 + η E q r ( θ 0 ) p ( x ∣ θ 0 ) q r ⊗ ( K c l a s s − 1 ) ( θ 1 : K c l a s s − 1 ) [ log η r ψ ( θ 0 , x ) K c l a s s + η ∑ k = 0 K c l a s s − 1 r ψ ( θ k , x ) ] , -\frac{\eta}{1+\eta}\mathbb{E}_{q_r(\theta_0)p(x\mid\theta_0)q_r^{\otimes(K_{\mathrm{class}}-1)}(\theta_{1:K_{\mathrm{class}}-1})}\left[\log\frac{\eta r_\psi(\theta_0,x)}{K_{\mathrm{class}}+\eta\sum_{k=0}^{K_{\mathrm{class}}-1}r_\psi(\theta_k,x)}\right], − 1 + η η E q r ( θ 0 ) p ( x ∣ θ 0 ) q r ⊗ ( K class − 1 ) ( θ 1 : K class − 1 ) [ log K class + η ∑ k = 0 K class − 1 r ψ ( θ k , x ) η r ψ ( θ 0 , x ) ] ,
@page 19/53 starts_mid=0 ends_mid=1 where η \eta η weights the joint against the marginal term. We use η = 1 \eta= 1 η = 1 throughout (NRE-B is recovered as η → ∞ \eta\rightarrow\infty η → ∞ , and NRE-A at K c l a s s = 1 K_{\mathrm{class}} = 1 K class = 1 , η = 1 \eta= 1 η = 1 ). The minimiser is the ratio r ψ ∗ ( θ , x ) = p ( x ∣ θ ) / q r ( x ) r_{\psi}^{*}(\theta, x) = p(x \mid\theta)/q_r(x) r ψ ∗ ( θ , x ) = p ( x ∣ θ ) / q r ( x ) , whose gradient in θ \theta θ does not depend on the proposal q r q_r q r used to generate training data, the property our Phase-2 derivation in § 6.1 explicitly relies on, and the property that distinguishes the NRE-C target from the proposal-conditioned target of SNPE-C/APT [3 ] .
Derivation of the Surrogate Energy
Starting from the surrogate posterior p ~ ϵ ( θ ∣ x o ) \widetilde{p}_{\epsilon}(\theta\mid x_o) p ϵ ( θ ∣ x o ) specified through the surrogate likelihood (6):
log p ~ ϵ ( θ ∣ x o ) = − E D ϵ ( s ( θ ) , x o ) + log p ( θ ) . (13) \log\widetilde{p}_{\epsilon}(\theta\mid x_o) = - \mathbb{E}D_{\epsilon}(s(\theta),x_o) + \log p(\theta). \tag*{(13)} log p ϵ ( θ ∣ x o ) = − E D ϵ ( s ( θ ) , x o ) + log p ( θ ) . (13)
This has the same exponential structure as the Boltzmann distribution (1), with the energy occupying the entire exponent. Sign-reversing and differentiating gives the surrogate energy gradient under a sufficient distance measure D ϵ D_{\epsilon} D ϵ :
∇ θ E s ( θ ) = E ∇ θ D ϵ ( s ( θ ) , x o ) − ∇ θ log p ( θ ) . \nabla_{\theta}E_s(\theta) = \mathbb{E}\nabla_{\theta}D_{\epsilon}(s(\theta),x_o) - \nabla_{\theta}\log p(\theta). ∇ θ E s ( θ ) = E ∇ θ D ϵ ( s ( θ ) , x o ) − ∇ θ log p ( θ ) .
For D ϵ D_{\epsilon} D ϵ to define a proper probability density p ~ ϵ ( θ ∣ x o ) \widetilde{p}_{\epsilon}(\theta\mid x_o) p ϵ ( θ ∣ x o ) , the requirement is that of a loss in a general Bayesian update [43 ] :
Definition A.1 (Sufficient distance measure). A function D ϵ : X K × X → R D_{\epsilon} : \mathcal{X}^{K} \times\mathcal{X} \to\mathbb{R} D ϵ : X K × X → R is a sufficient distance measure for a prior p ( θ ) p(\theta) p ( θ ) and observation x o x_o x o if it is differentiable in its first argument and the surrogate posterior has a finite, non-zero normalising constant:
0 < ∫ Θ exp ( − E D ϵ ( s ( θ ) , x o ) ) p ( θ ) d θ < ∞ . 0 < \int_{\Theta} \exp\left(-\mathbb{E}D_{\epsilon}(s(\theta),x_o)\right)p(\theta)\,d\theta< \infty. 0 < ∫ Θ exp ( − E D ϵ ( s ( θ ) , x o ) ) p ( θ ) d θ < ∞.
Since by Theorem 4.1, any differentiable D ~ ϵ \widetilde{D}_{\epsilon} D ϵ that equals D ϵ D_{\epsilon} D ϵ up to an additive constant satisfies ∇ D ~ ϵ = ∇ D ϵ \nabla\widetilde{D}_{\epsilon} = \nabla D_{\epsilon} ∇ D ϵ = ∇ D ϵ , as far as ASBS is concerned (7) we can replace D ϵ D_{\epsilon} D ϵ with D ~ ϵ \widetilde{D}_{\epsilon} D ϵ , equivalent under differentiation. In practice this relaxation matters because it admits standard losses whose constants are conventionally dropped, such as the d x 2 log 2 π \frac{d_x}{2}\log2\pi 2 d x log 2 π of the Gaussian NLL.
Optional truncation of the ratio estimator
A ratio estimator is only identified where its training data have support. Trained on ( θ i , x i ) (\theta_i, x_i) ( θ i , x i ) with θ i ∼ q r ( θ ) \theta_i \sim q_r(\theta) θ i ∼ q r ( θ ) , the classifier r ψ ( θ , x o ) r_{\psi}(\theta, x_o) r ψ ( θ , x o ) is constrained near the bulk of q r q_r q r , but in regions the proposal never visited its value is an extrapolation of the network. Because the Phase 2 sampler minimises E r = − log r ψ − log p ( θ ) E_r = -\log r_{\psi} - \log p(\theta) E r = − log r ψ − log p ( θ ) over the whole prior, a spuriously large extrapolated ratio can pull samples into regions with no posterior mass. We optionally allow RSBI to guard against this by restricting where the ratio is trusted. This may be the case, for example, when the Phase 1 zoom is assumed to be locally concentrated around the true posterior mass. The choice firmly remains with the user.
Let { θ j } \{\theta_j\} { θ j } be the parameters the region is fitted on (the Phase 1 sampler draws, or all NRE training parameters of the round). We compute their mean θ ˉ \bar{\theta} θ ˉ and covariance Σ \Sigma Σ , discard the points whose Mahalanobis distance exceeds the q q q -quantile of the distances, and recompute θ ˉ \bar{\theta} θ ˉ and Σ \Sigma Σ on the remaining core, so that outlying draws do not inflate the region. The distance of a parameter to the core is d ( θ ) = ( θ − θ ˉ ) T Σ − 1 ( θ − θ ˉ ) d(\theta) = \sqrt{(\theta-\bar{\theta})^{\mathsf{T}}\Sigma^{-1}(\theta-\bar{\theta})} d ( θ ) = ( θ − θ ˉ ) T Σ − 1 ( θ − θ ˉ ) . For multimodal posteriors a single ellipsoid around separated modes would cover most of the prior, so we first split { θ j } \{\theta_j\} { θ j } into J J J clusters with k-means, fit ( θ ˉ c , Σ c ) (\bar{\theta}_c,\Sigma_c) ( θ ˉ c , Σ c ) per cluster as above and set d ( θ ) = min c d c ( θ ) d(\theta) = \min_c d_c(\theta) d ( θ ) = min c d c ( θ ) . The truncated ratio is
log r ˉ ψ ( θ , x o ) = log r ψ ( θ , x o ) − λ relu ( d ( θ ) − κ ) 2 , (14) \log\bar{r}_{\psi}(\theta,x_o) = \log r_{\psi}(\theta,x_o) - \lambda\operatorname{relu}(d(\theta)-\kappa)^2, \tag*{(14)} log r ˉ ψ ( θ , x o ) = log r ψ ( θ , x o ) − λ relu ( d ( θ ) − κ ) 2 , (14)
which leaves the ratio unchanged where d ( θ ) ≤ κ d(\theta) \le\kappa d ( θ ) ≤ κ and adds a soft quadratic wall of strength λ \lambda λ beyond. Phase 2 then uses E r = − log r ˉ ψ − log p ( θ ) E_r = -\log\bar{r}_{\psi} - \log p(\theta) E r = − log r ˉ ψ − log p ( θ ) . The wall is refitted after every NRE update, acts only on the ratio, leaves the prior term untouched and needs no simulations beyond those already in the budget.
Restricting inference to a region that probably contains the posterior is a common device in sequential SBI. Truncated sequential NPE (TSNPE) [9 ] draws each round’s simulations from the prior restricted to the highest-probability region of the current posterior estimate. Truncated marginal NRE (TMNRE) [8 ] shrinks the prior to a box around the region where the ratio estimate is non-negligible and retrains within it. All of these methods, and ours, rest on the same observation: a learned density or ratio is only reliable where it has data, so inference is confined to that region rather than trusted everywhere. Our variant differs in three respects. (i) The region is fitted on samples that are already available (the Phase 1 output and the round’s training parameters), so it costs no additional simulations. (ii) The truncation is soft and acts on the energy rather than on the prior or the proposal, so the sampler is steered back into the region rather than forbidden from leaving it, and the prior in E r E_r E r stays the original one. (iii) The region is an ellipsoid (or a union of ellipsoids), not an axis-aligned box, and so follows correlated posteriors.
Inside { d ( θ ) ≤ k } \{d(\theta) \le k\} { d ( θ ) ≤ k } the truncated energy equals the untruncated one, so the Phase 2 target is unchanged wherever the region covers the posterior. As for TSNPE and TMNRE, the truncation biases the result only if non-negligible posterior mass lies outside the region, which k k k and the trim quantile q q q control. A larger k k k is safer but restricts less, and a smaller q q q fits a tighter core. The wall is a safeguard against extrapolation and is not needed when the ratio is well behaved, so it is optional and off unless stated otherwise.
Among the benchmark tasks the truncation is used only for Lotka–Volterra (App. § 14 .§ 8 ), where it is enabled for all prior widths and simulation budgets. Its Phase 1 posterior reliably covers the reference posterior there, while the oscillatory likelihood makes the ratio extrapolate poorly away from the training parameters. We use a single cluster (J = 1 J = 1 J = 1 ) fitted on the NRE training parameters of the current round and refitted every round, with λ = 50 \lambda= 50 λ = 50 , k = 3 k = 3 k = 3 and q = 0.99 q = 0.99 q = 0.99 . The only exception is the large prior at N s i m = 10 5 N_{\mathrm{sim}} = 10^5 N sim = 1 0 5 , where k = 2 k = 2 k = 2 and q = 0.9 q = 0.9 q = 0.9 give a tighter core. For Lotka–Volterra, log r ψ \log r_\psi log r ψ is in addition softly capped at the 0.999-quantile of its values over the training parameters plus a margin of 2, which guards against the same far-field blow-up of the classifier logits. Two Moons, SLCP, Bernoulli GLM and gw_ringdown use the untruncated ratio.
Preconditioning the Phase 2 sampler
The Phase 2 sampler is an adjoint Schrödinger bridge sampler (ASBS; [14 ] ) whose reference process is isotropic with the same noise scale σ t \sigma_t σ t is applied in every direction of θ \theta θ -space. This choice is natural when the target has a comparable width in all directions, but it is poorly suited to ill-conditioned targets, whose posterior is very narrow along some directions and broad along others. Such geometry arises whenever parameters are strongly correlated, when the posterior concentrates on a curved ridge, or when the posterior widths of different parameters differ by orders of magnitude. A noise scale small enough to resolve the stiff directions moves far too slowly along the shallow ones, while a noise scale matched to the shallow directions overshoots the stiff ones; in either case the sampler concentrates poorly within a fixed training budget. The same difficulty is well known for gradient-based MCMC: the efficiency of the Metropolis-adjusted and unadjusted Langevin algorithms degrades with the condition number of the target [44 , 45 ] , and Hamiltonian Monte Carlo with an identity mass matrix must choose a step size limited by the narrowest direction [39 ] . The standard remedy is to precondition the dynamics with a metric that approximates the posterior covariance, either fixed [39 ] , adapted from earlier samples [40 , 46 , 47 ] , position-dependent [48 ] , or given by a learned transport map [49 ] . For diffusion samplers the difficulty lies in learning rather than in sampling: on Gaussian targets the sampling error of a diffusion sampler with optimised hyperparameters does not depend on the condition number [50 ] , but regression-trained drifts fit high-variance directions quickly and low-variance ones slowly when the target is ill-conditioned, which preconditioning into an isotropic space removes [51 ] .
In our method, Phase 1 already provides the information needed for such a metric, at no extra simulation cost. Its samples are concentrated around the posterior: we use the round-0 NRE training parameters { θ j } \{\theta_j\} { θ j } for Lotka–Volterra, and 10 4 10^4 1 0 4 fresh draws from the trained Phase-1 sampler for gw_ringdown. Their mean θ ˉ \bar{\theta} θ ˉ and covariance Σ = L L T \Sigma= LL^{\mathsf{T}} Σ = L L T , with L L L the Cholesky factor, are estimated from the samples whose Mahalanobis distance to the sample mean lies below its q q q -quantile, where q q q is the whitening quantile . We use q = 0.99 q = 0.99 q = 0.99 , so that a few stray Phase-1 samples do not inflate the covariance (at 10 5 10^5 1 0 5 simulation budget, q = 0.9 q = 0.9 q = 0.9 for the large Lotka–Volterra prior, whose Phase-1 proposal has heavier tails). Together, θ ˉ \bar{\theta} θ ˉ and Σ \Sigma Σ describe the location, scale and orientation of the posterior. We then run Phase 2 in the whitened coordinates
z = L − 1 ( θ − θ ˉ ) , θ = θ ˉ + L z , (15) z = L^{-1}(\theta- \bar{\theta}), \qquad\theta= \bar{\theta} + Lz, \tag*{(15)} z = L − 1 ( θ − θ ˉ ) , θ = θ ˉ + L z , (15)
in which the Phase-1 sample cloud has approximately zero mean and unit covariance. The energy is still evaluated in the original coordinates, and only its gradient is transformed:
E r ( z ) = E r ( θ ˉ + L z ) , ∇ z E r = L ⊤ ∇ θ E r , (16) E_r(z) = E_r(\bar{\theta} + Lz), \qquad\nabla_z E_r = L^\top\nabla_\theta E_r, \tag*{(16)} E r ( z ) = E r ( θ ˉ + L z ) , ∇ z E r = L ⊤ ∇ θ E r , (16)
where E r ( θ ) = − log p ( θ ) − log r ψ r ( θ , x o ) E_r(\theta) = -\log p(\theta) - \log r_{\psi_r}(\theta, x_o) E r ( θ ) = − log p ( θ ) − log r ψ r ( θ , x o ) is the round-r r r energy of (8).
Because the map is an affine change of variables with constant Jacobian, it leaves the target posterior unchanged: samples drawn in z z z are mapped back by θ = θ ˉ + L z \theta= \bar{\theta} + Lz θ = θ ˉ + L z , and all simulations, NRE training and C2ST evaluation take place in θ \theta θ . The transform requires no additional simulator calls, since it is computed only from Phase-1 samples. It is fixed after round 0, so the Phase 2 networks, which are warm-started from one round to the next, remain valid across rounds. The construction is the analogue of mass-matrix adaptation in HMC [39 , 40 , 47 ] , and of the affine invariance of ensemble samplers [52 ] , with the metric supplied by Phase 1 rather than by a separate warm-up or tuning run. One consequence is that the reference noise scale σ t \sigma_t σ t is specified in z z z -units, that is, relative to the spread of the Phase 1 proposal rather than in the units of θ \theta θ .
Preconditioning is most useful for unimodal posteriors with an ill-conditioned energy landscape, such as strongly correlated parameters, long narrow ridges, or widths that differ by orders of magnitude between directions. For Lotka–Volterra, for instance, the posterior is about 40× narrower across its ridge than along it. There, a single affine map removes most of the anisotropy, after which the isotropic reference process is appropriate. For multimodal posteriors a global affine map is of little use, the mean and covariance of a multimodal Phase 1 proposal span all modes and match the local geometry of none of them. Handling such targets would require a position-dependent metric [48 ] or a nonlinear transport map [49 ] ; here we leave the sampler unpreconditioned. Developing such geometry-adapted reference processes for diffusion samplers, including learned transport maps and position-dependent metrics built from the Jacobian of the differentiable simulator, is the subject of our future work. For posteriors that are already well conditioned, preconditioning reduces to a near-identity rescaling and has little effect. In our experiments we therefore precondition Phase 2 for Lotka–Volterra and gw_ringdown, and not for the multimodal Two Moons and SLCP posteriors.
Important to note is that preconditioning does not give RSBI an advantage over the baselines, since SNPE and SNVI represent the posterior with normalising flows, whose learned bijections adapt to the posterior’s scale and orientation by construction. Preconditioning simply gives the Phase 2 sampler the same affine information, estimated from RSBI’s Phase 1 samples without additional simulator calls.
Additional Experiments
Compute-Matched Performance
For all three simulators we time 20 forward passes and 20 forward + backward passes using a batch size of 32 and 5 warmup batches. Gradients are calculated with regards to the sum of the outputs of the simulator (e.g., x.sum()). The timings are done on an NVIDIA GTX 1660 Ti laptop GPU. The per-batch timings are reported in Table 2 , C2ST under a compute-matched budget is reported in Figure 5 , and the number of simulator calls used is reported in Table 3 . Across all simulators we see an increased cost when doing the additional backward pass, ranging from ∼ 1.7 × \sim1.7\times ∼ 1.7 × to ∼ 4.0 × \sim4.0\times ∼ 4.0 × . We note that although the computational costs of doing the backward pass are significant, RSBI only requires this extra compute during Phase 1, which can take up only a small portion of the total simulation budget. We find across all benchmarks that RSBI maintains a strong performance across all simulation budgets even when compute-constrained. Especially in Two Moons and Bernoulli GLM, RSBI maintains it’s strong C2ST scores. We stress that the measured timings in Table 2 are hardware-dependent.
Table 2. Effect of backwards pass on per-batch mean compute time across 3 sbibm benchmarks.
Figure 5. Left: C2ST of RSBI against number of simulations. Right: C2ST of RSBI against the equivalent compute-matched simulation budget (reported in Table 3 ). Calculated for Two Moons, Bernoulli GLM and SLCP benchmarks. Y-axis labels include the compute ratio CR charged per Phase-1 simulation (Table 2 ). Lines are the prior scenarios of Figure 4 with error bars being the standard deviation over 10 observations.
Table 3. Total simulator calls made by the compute-matched runs of Figure 5 .
Posterior mass leakage mitigation
Leakage arises when an algorithm learns a density over θ \theta θ that is not tied to the prior support. Atomic SNPE-C fits a conditional density estimator q ψ ( θ ∣ x ) q_{\psi}(\theta\mid x) q ψ ( θ ∣ x ) , an unbounded flow or Gaussian mixture on R d \mathbb{R}^{d} R d , to simulated pairs, and its proposal correction normalises the estimator only over the finite set of contrasting atoms. Nothing in the loss evaluates the estimator outside the support, so when the posterior modes sit at the boundary, the smooth estimator spills mass across it and the leaked mass must be removed afterwards by rejection. Truncated SNPE (TSNPE; [9 ] ) avoids the atomic correction by drawing simulations from the prior truncated to a high-probability region and training with the standard NPE loss, which removes the optimisation pathology behind leakage; its estimator is nevertheless still a free density on R d \mathbb{R}^{d} R d , so any residual out-of-support mass is again removed by rejection. RSBI never learns such a density. Its Phase 2 sampler is trained against the energy E r ( θ ) = − log p ( θ ) − log r ψ r ( θ , x o ) E_{r}(\theta) = -\log p(\theta) - \log r_{\psi_{r}}(\theta, x_{o}) E r ( θ ) = − log p ( θ ) − log r ψ r ( θ , x o ) , which contains the prior exactly. Outside the support log p ( θ ) = − ∞ \log p(\theta) = -\infty log p ( θ ) = − ∞ , so the target assigns zero mass there whatever the ratio estimator outputs: an unconstrained classifier can be wrong about r ψ r r_{\psi_{r}} r ψ r off the support, but the product p ( θ ) r ψ r ( θ , x o ) p(\theta)r_{\psi_{r}}(\theta, x_{o}) p ( θ ) r ψ r ( θ , x o ) cannot be. Because the sampler is fitted to this target, mass placed outside the support is penalised by the energy, and its gradient pushes trajectories back inside. In practice we implement the box prior as a stiff quadratic wall rather than an infinite one, to keep gradients finite, so a small residual fraction of the raw samples falls just outside the boundary (Figure 6 ). This fraction is set by the wall stiffness and the sampler’s training error rather than by the estimator, and it vanishes from the final posterior, which is restricted to the support. The same argument applies to SNVI [7 ] , whose variational posterior is likewise fitted to the prior-weighted ratio p ( θ ) r ψ ( θ , x o ) p(\theta) r_{\psi}(\theta, x_o) p ( θ ) r ψ ( θ , x o ) rather than learned as a free density, and whose final importance-resampling step assigns zero weight to samples outside the support. This guarantee holds only to the extent that the ASBS sampler fits its target. When the controller fails to converge, for instance by collapsing onto a subset of modes or scattering trajectories across the box, its samples need not respect the energy, and out-of-support mass can reappear. This may not be in general true for other samplers/models used in Phase 1 of RSBI, which we discuss as a limitation of choosing WT-ASBS in § 8 .
Figure 6. Demonstration of posterior leakage and it’s mitigation with RSBI: Posterior samples on the tractable squared-location task (d = 2 d = 2 d = 2 , N s i m = 10 4 N_{\mathrm{sim}} = 10^{4} N sim = 1 0 4 , one observation). Dashed square: the support of the uniform prior, [ − b , b ] d [-b, b]^{d} [ − b , b ] d ; crosses: the four true modes at ( ± c , ± c ) (\pm c, \pm c) ( ± c , ± c ) , which lie close to the corners of the support. Each panel shows the raw sampler output of one method (before rejection for the flows, before removing the metadynamics bias for RSBI), with the fraction of its mass inside the support. The C2ST against the analytic reference (0.5 = indistinguishable) is computed on each method’s final posterior restricted to the support: the in-support samples of the flows and NPE, and the in-support reweighted samples of RSBI. The atomic SNPE-C flows and amortised NPE place much of their mass outside the support; RSBI keeps it inside while covering all four modes, like the analytic reference (top left).
To isolate how bounded prior support affects inference (leakage), we use a tractable benchmark with a closed-form posterior. The parameters are drawn from a uniform box, θ ∼ U ( [ − b , b ] d ) \theta\sim U([-b, b]^{d}) θ ∼ U ([ − b , b ] d ) , and the data are x = θ 2 + σ ϵ x = \theta^{2} + \sigma\epsilon x = θ 2 + σ ϵ , ϵ ∼ N ( 0 , I ) \epsilon\sim\mathcal{N}(0, I) ϵ ∼ N ( 0 , I ) , so the posterior for x o = ( c 2 , … , c 2 ) x_o = (c^{2}, \ldots, c^{2}) x o = ( c 2 , … , c 2 ) has 2 d 2^{d} 2 d symmetric modes at ( ± c , … , ± c ) (\pm c, \ldots, \pm c) ( ± c , … , ± c ) . Taking c c c close to b b b places every mode near a corner of the prior support (in d = 2 d = 2 d = 2 , a square), the regime in which density-based posteriors are known to “leak” probit mass outside the support. Because the reference posterior is available analytically, we can measure leakage exactly. We use d = 2 d = 2 d = 2 , b = 1 b = 1 b = 1 , c = 0.9 c = 0.9 c = 0.9 and σ = 0.1 \sigma= 0.1 σ = 0.1 .
At a fixed simulation budget (N s i m = 10 4 N_{\mathrm{sim}} = 10^4 N sim = 1 0 4 ) we compare atomic SNPE-C [3 ] , using the standard sbi implementation [53 ] , with masked autoregressive flow [54 ] and neural spline flow [55 ] density estimators, the non-atomic mixture-of-Gaussians variant (whose posterior estimator is a Gaussian mixture), and amortised NPE (a single round with a masked autoregressive flow), against RSBI. We report, on the raw sampler output, the fraction of mass inside the prior support and the fraction of the 2 d 2^d 2 d modes recovered (mode recall), and the classifier two-sample test (C2ST) against the analytic reference on each method’s final posterior restricted to the support.
For RSBI, Phase 1 uses the Gaussian negative log-likelihood surrogate with the exact noise variance σ 2 \sigma^2 σ 2 and reference noise scale σ max = 0.3 \sigma_{\max} = 0.3 σ m a x = 0.3 , and is stopped after 5 × 10 3 5 \times10^3 5 × 1 0 3 simulations. Phase 2 spends the remaining 5 × 10 3 5 \times10^3 5 × 1 0 3 simulations in a single round: a contrastive NRE is trained on fresh draws from the Phase 1 sampler, its ratio is evaluated only within the truncation region of Appendix § 11 .§ 4 , and the Phase 2 sampler, again with σ max = 0.3 \sigma_{\max} = 0.3 σ m a x = 0.3 , targets the posterior under the uniform box prior.
The flow-based amortised and sequential methods place a substantial fraction of their mass outside the prior support, and the mixture-of-Gaussians variant mitigates but does not eliminate this (Figure 6 ). In contrast, RSBI samples under a constrained energy and keeps almost all of its mass inside the support while recovering all four modes. This controlled experiment shows that the leakage pathology is structural to unconstrained density estimation on boundary-mode targets, and that the constrained Phase 1 and 2 samplers remove it by construction.
Ablation: replacing Phase 2 by standard sequential methods
RSBI has two components that could each explain its advantage on wide priors. Phase 1 localises the posterior by training ASBS on a surrogate, so it zooms from the prior onto the region that carries posterior mass. Phase 2 then refines this estimate with an NRE and an ASBS sampler that transports the Phase 1 sampler locally. To separate the two, we keep Phase 1 fixed and replace Phase 2 with established sequential methods that use Phase 1 as their first proposal, at the same simulation budget. The obvious candidate, atomic SNPE-C [3 ] , is poorly suited to this setting. Its contrastive loss normalises the posterior only over a few atoms drawn from the proposal, so posterior mass leaks into the large region of the prior that the localised proposal never visits. With the Phase 1 sampler as its first proposal it reaches a C2ST of only 0.96 and 0.93 at 10 3 10^3 1 0 3 and 10 4 10^4 1 0 4 simulations, far from RSBI. We therefore use truncated SNPE (TSNPE) [9 ] , which restricts the prior to a highest-density region (HPR) of the current posterior estimate and trains with the maximum-likelihood loss. It avoids leakage by construction and is the natural way to hand a localised proposal to a neural posterior estimator. As a second, variational baseline we use SNVI [7 ] , which, like Phase 2 of RSBI, fits a sampler to the prior times a learned ratio and draws the next round of simulations from it. Unlike RSBI, it fits a normalising flow over the whole prior by forward KL, so the flow is free to move mass anywhere in the prior box, including regions where the ratio estimator has never seen data. On a wide prior the classifier extrapolates there without control. In SNVI’s self-normalised importance weights a single proposal in such a region can dominate all others, so the flow is pulled onto spurious high-ratio regions even when initialised at the Phase 1 sampler, and the next round of simulations is spent there. RSBI’s Phase-2 instead, transports particles locally from the Phase-1 proposal due to ASBS using Phase 1’s p ~ ε ( θ ∣ x o ) \tilde{p}_{\varepsilon}(\theta\mid x_o) p ~ ε ( θ ∣ x o ) as the source distribution.
We use Two Moons with the widest prior, [ − 50 , 50 ] 2 [-50, 50]^2 [ − 50 , 50 ] 2 (log 10 R p ≈ 4.3 \log_{10} \mathcal{R}_p \approx4.3 log 10 R p ≈ 4.3 ), at N s i m ∈ { 10 3 , 10 4 , 10 5 } N_{\mathrm{sim}} \in\{10^3, 10^4, 10^5\} N sim ∈ { 1 0 3 , 1 0 4 , 1 0 5 } and observations 1–10. For each cell we reload the trained Phase 1 sampler of the corresponding RSBI run, without retraining. TSNPE then runs two rounds with the same per-round allocation as RSBI’s ratio estimator, 600/1000/5000 simulations in round 1 and 400/9000/95000 in round 2. Round 1 draws from the prior truncated to the 1 − ϵ 1-\epsilon 1 − ϵ HPR of a 10-component Gaussian mixture fitted to the Phase 1 samples, and round 2 from the HPR of the round-1 posterior at x o x_o x o , with ϵ = 10 − 4 \epsilon= 10^{-4} ϵ = 1 0 − 4 . TSNPE cannot be trained on the simulations RSBI’s ratio estimator uses. Maximum-likelihood NPE trained on draws from a proposal p ~ ( θ ) \tilde{p}(\theta) p ~ ( θ ) targets p ~ ( θ ∣ x ) ∝ p ~ ( θ ) p ( x ∣ θ ) \tilde{p}(\theta\mid x) \propto\tilde{p}(\theta) p(x \mid\theta) p ~ ( θ ∣ x ) ∝ p ~ ( θ ) p ( x ∣ θ ) , which equals the posterior inside the truncation region only when p ~ \tilde{p} p ~ is the truncated prior. The ratio estimator instead learns p ( x ∣ θ ) / q ( x ) p(x \mid\theta)/q(x) p ( x ∣ θ ) / q ( x ) (Eqn. 8) for any proposal whose support covers the posterior, so RSBI can train on the Phase 1 pairs and on draws from its Phase 2 sampler directly. Using the same pairs for TSNPE would require atomic SNPE-C, which suffers from leakage on wide priors, or importance weights p ( θ ) / p ~ ( θ ) p(\theta)/\tilde{p}(\theta) p ( θ ) / p ~ ( θ ) , which need a sampler density that ASBS does not provide. We therefore use the Phase 1 samples only to define the first truncation region and simulate afresh in both rounds, so TSNPE uses n 1 n_1 n 1 more simulations than RSBI, which favours TSNPE. This flexibility in the choice of proposal is a structural advantage of the ratio-based Phase 2. The density estimator (NSF, 50 hidden units, independent z-scoring) and the C2ST protocol are those of the main SNPE baseline. In every cell both truncation regions contain all reference samples, so the truncation never removes posterior mass.
Phase 1 + SNVI uses the configuration of our SNVI+fKL baseline (SNRE-B with a ResNet classifier, a neural spline flow fitted by forward KL, and sampling-importance resampling) and is matched to RSBI in both simulations and rounds. Its ratio estimator is trained in round 1 on the same 600/1 000/5 000 Phase 1 pairs as RSBI’s, and round 2 simulates the remaining budget from the round-1 variational posterior. Like the ASBS sampler of RSBI, the flow starts at the Phase 1 sampler, fitted to its samples by maximum likelihood. We compute the importance weights of the forward KL in log space, which avoids a numerical overflow of the released implementation that is detailed in Appendix E.
Table 4 and Figure 7 show that both phases are needed. On its own, Phase 1 is a poor posterior estimate (C2ST 0.995) because the surrogate is coarse, but it localises the posterior reliably. Using it as the first proposal improves TSNPE by 0.07, 0.12 and 0.25 in C2ST at 10 3 10^3 1 0 3 , 10 4 10^4 1 0 4 and 10 5 10^5 1 0 5 simulations. SNPE from the prior barely improves at any budget, so without this zoom a standard sequential method spends almost its whole budget in regions of negligible posterior mass.
Table 4. Phase-2 ablation on Two Moons with the widest prior ([ − 50 , 50 ] 2 [-50, 50]^2 [ − 50 , 50 ] 2 ): mean (sd) C2ST over ten observations at matched simulation budget. Phase 1 + TSNPE uses the trained Phase 1 sampler of each RSBI run as the first truncation of TSNPE [9 ] , with the same per-round simulation allocation as RSBI’s ratio estimator (n 1 n_1 n 1 more simulations, since the Phase 1 pairs are not truncated-prior draws). Phase 1 + SNVI replaces RSBI’s NRE and ASBS sampler by SNVI [7 ] (SNRE-B and a normalising flow trained by forward KL), trained on exactly the same simulations in round 1 and the same number in round 2. SNPE and SNVI+fKL from the prior are the main-benchmark configurations. Phase 1 alone (524 simulations) reaches 0.995 (0.001).
Figure 7. C2ST against the reference posterior on Two Moons with the widest prior (mean ± \pm ± sd over ten observations, lower is better). Values as in Table 4 .
The zoom alone is not sufficient, however. Given the same Phase 1 and the same per-round budget, replacing TSNPE by RSBI’s variational Phase 2 lowers the C2ST by a further 0.23, 0.26 and 0.11, and RSBI is better than Phase 1 + TSNPE on 10, 10 and 7 of the ten observations. Phase 2 accounts for most of the gain at 10 3 10^3 1 0 3 and 10 4 10^4 1 0 4 simulations, and RSBI reaches at 10 3 10^3 1 0 3 simulations (0.693) an accuracy that Phase 1 + TSNPE does not reach at 10 5 10^5 1 0 5 (0.738). Two properties of the variational Phase 2 explain this. First, the ratio estimator is valid under any proposal, so every simulation is used, including the Phase 1 pairs, whereas TSNPE must discard them and draw prior-shaped samples from the truncation region. Second, ASBS samples directly from the current posterior estimate, the prior times the learned ratio, so the round-2 simulations concentrate where the posterior has mass. TSNPE can only spread them over the prior restricted to an HPR of its round-1 estimate, and after a coarse zoom on the wide prior this region is still large compared with the posterior. At 10 5 10^5 1 0 5 simulations TSNPE closes part of the gap, as expected from a consistent estimator trained on a well-localised proposal, but it remains less accurate and much less stable across observations (sd 0.107 against 0.026). Phase 1 + SNVI confirms that substantial gain comes from how Phase 2 refines the posterior and not from the ratio estimator alone. With the same Phase 1 pairs and the same rounds as RSBI, it improves on SNVI+fKL from the prior at every budget, but RSBI is better on all ten observations at every budget. At 10 3 10^3 1 0 3 simulations the variational posterior moved onto a spurious high-ratio region at the boundary of the prior in 4 of the 10 observations (C2ST 1.000), the failure described above. Both SNVI variants also share a non-monotonic dependence on the budget, with C2ST rising from 0.943 to 0.993 (prior, R = 5 R=5 R = 5 ) and from 0.770 to 0.848 (Phase 1, R = 2 R=2 R = 2 ) between 10 4 10^4 1 0 4 and 10 5 10^5 1 0 5 simulations. The two use different implementations, numbers of rounds and initialisations, so this degradation reflects SNVI’s variational refinement on a wide prior. RSBI improves monotonically with the budget up to 10 4 10^4 1 0 4 simulations and remains stable at 10 5 10^5 1 0 5 . The advantage of RSBI on wide priors in this ablation study is therefore derivative of both the Phase 1 zoom and the variational ASBS refinement that follows it.
Surrogate Likelihoods: Details and ablations
This appendix expands on the surrogate likelihood used in Phase 1 (§ 4.1 ). We stress at the outset that the choice of distance, and hence of surrogate likelihood, is task dependent and deserves care from the analyst. The ABC literature offers a wide range of choices, each motivated by a particular class of problems. These include hand-crafted or learned summary statistics [56 ] , Gaussian synthetic likelihoods [57 , 58 ] , and full-data discrepancies such as the Wasserstein distance [59 ] , the maximum mean discrepancy [60 ] , the Kullback-Leibler divergence [61 ] and the energy statistic [62 ] . Comparative studies find that no single choice is best across problems [63 ] , and the ABC kernel itself can be read as a model of the error between simulator and data [64 ] , so the bandwidth ϵ \epsilon ϵ is a modelling choice as much as a tuning parameter.
We first describe the surrogate likelihoods used in this work and then discuss the bandwidth and metric trade-offs known from the ABC literature. Next, we show how inspecting the energy and gradient landscape of a surrogate can inform the choice of the bandwidth ϵ \epsilon ϵ . We close with ablations over the three surrogate likelihoods and over ϵ \epsilon ϵ . Figure 8 shows the Phase-1 posteriors obtained with the surrogates used for the main results. Phase 1 already concentrates on the posterior support and recovers its modes before any NRE is trained.
Figure 8. Phase-1 posteriors used for the main results in Figure 4 for Two Moons, SLCP, gw_ringdown and the Bernoulli GLM (top left to bottom right), each at its default prior with N s i m = 10 5 N_{\mathrm{sim}} = 10^{5} N sim = 1 0 5 at observation 1 (see Sec. G for the Phase-1 configuration). We show the 1- and 2-σ \sigma σ contours of the Phase-1 (blue) and reference (grey) posteriors over the prior (dashed red: the prior box for the uniform priors, and the 1- and 2-σ \sigma σ contours of the Gaussian prior for the Bernoulli GLM). For SLCP and gw_ringdown we show the 2-D marginals of the full 5-D and 6-D posteriors for the two parameters with the most pronounced structure. These are ( θ 3 , θ 4 ) (\theta_{3}, \theta_{4}) ( θ 3 , θ 4 ) , which separate the four SLCP modes, and the mass and spin pair ( M , χ ) (M, \chi) ( M , χ ) , whose posterior is a single curved ridge (Phase 1 also places small islands of mass off the ridge). For the unimodal Bernoulli GLM we show the most strongly correlated pair of its 10 parameters.
Gaussian negative log-likelihood with adaptive variance
For a simulator with output x = s ( θ , ω ) ∈ R d x x=s(\theta,\omega)\in\mathbb{R}^{d_x} x = s ( θ , ω ) ∈ R d x , we model per-dimension residuals as zero-mean Gaussian with diagonal covariance diag ( ϵ 1 ( θ ) , … , ϵ d x ( θ ) ) \operatorname{diag}(\epsilon_1(\theta),\ldots,\epsilon_{d_x}(\theta)) diag ( ϵ 1 ( θ ) , … , ϵ d x ( θ )) , which plays the role of the bandwidth ϵ \epsilon ϵ . The surrogate distance is
D ϵ N L L ( s ( θ ) , x o ) = 1 2 ∑ d = 1 d x ( x o ( d ) − x ˉ ( d ) ( θ ) ) 2 ϵ d ( θ ) + 1 2 ∑ d = 1 d x log ϵ d ( θ ) , (17) \mathcal{D}^{\mathrm{NLL}}_{\epsilon}(s(\theta),x_o)=\frac{1}{2}\sum_{d=1}^{d_x}\frac{\left(x_o^{(d)}-\bar{x}^{(d)}(\theta)\right)^2}{\epsilon_d(\theta)}+\frac{1}{2}\sum_{d=1}^{d_x}\log\epsilon_d(\theta), \tag*{(17)} D ϵ NLL ( s ( θ ) , x o ) = 2 1 d = 1 ∑ d x ϵ d ( θ ) ( x o ( d ) − x ˉ ( d ) ( θ ) ) 2 + 2 1 d = 1 ∑ d x log ϵ d ( θ ) , (17)
where, as in Sec. 4.1, s ( θ ) s(\theta) s ( θ ) denotes the K K K replicates x k = s ( θ , ω k ) x_k=s(\theta,\omega_k) x k = s ( θ , ω k ) , ω k ∼ i i d p ( ω ) \omega_k\mathrel{\overset{\mathrm{iid}}{\sim}}p(\omega) ω k ∼ iid p ( ω ) , sharing the parameter θ \theta θ but differing in the simulator’s internal randomness, and x ˉ ( d ) ( θ ) \bar{x}^{(d)}(\theta) x ˉ ( d ) ( θ ) and ϵ d ( θ ) \epsilon_d(\theta) ϵ d ( θ ) are their per-dimension mean and unbiased variance. Heteroscedastic Gaussian losses of this form have a long history in neural regression [65 ] and are now standard for deep predictive uncertainty quantification [66 ] . We use PyTorch’s gaussian_nll_loss with full=False, which omits the constant d x 2 log 2 π \frac{d_x}{2}\log2\pi 2 d x log 2 π (irrelevant for gradients by Theorem 4.1 ). The variance term in (16) acts as an entropy-like penalty that prevents the bridge from concentrating in regions where the simulator is highly stochastic.
We support two options for the variance ϵ d ( θ ) \epsilon_{d}(\theta) ϵ d ( θ ) . (i) Fixed: ϵ d ( θ ) = ϵ d ⋆ \epsilon_{d}(\theta) = \epsilon_{d}^{\star} ϵ d ( θ ) = ϵ d ⋆ for a user-supplied constant. A single simulator call per θ \theta θ suffices (K = 1 K = 1 K = 1 ), appropriate when the noise model is exact. (ii) Adaptive: ϵ d ( θ ) \epsilon_{d}(\theta) ϵ d ( θ ) is recomputed at every θ \theta θ from the K K K replicates. It varies smoothly with θ \theta θ but is noisy at small K K K .
With fixed variance, (17) is a weighted squared distance plus the constant 1 2 ∑ d log ϵ d ⋆ \frac{1}{2}\sum_{d}\log\epsilon_{d}^{\star} 2 1 ∑ d log ϵ d ⋆ , hence a sufficient distance measure (Definition A.1) up to a constant, covered by Theorem 4.1 . In the adaptive case, ϵ d = ϵ d ( θ , ω 1 : K ) \epsilon_{d} = \epsilon_{d}(\theta, \omega_{1:K}) ϵ d = ϵ d ( θ , ω 1 : K ) is the sample variance of the replicates drawn at θ \theta θ , recomputed at every evaluation and differentiated through. The term 1 2 ∑ d log ϵ d \frac{1}{2}\sum_{d}\log\epsilon_{d} 2 1 ∑ d log ϵ d is then not a constant, since its expectation varies with θ \theta θ wherever the simulator noise is heteroscedastic, so (16) is not a PSD distance up to a constant. It nonetheless defines a proper surrogate: the variance is floored, so D ϵ N L L D^{\mathrm{NLL}}_{\epsilon} D ϵ NLL is bounded below, exp ( − E D ϵ N L L ) p ( θ ) \exp(-\mathbb{E}D^{\mathrm{NLL}}_{\epsilon})p(\theta) exp ( − E D ϵ NLL ) p ( θ ) is integrable for any proper prior as per Def. A.1, and (8) remains the gradient of a well-defined energy that ASBS targets as before. Any mismatch with the true likelihood is removed in Phase 2.
Energy distance
We also consider the energy statistic of Székely and Rizzo [67 ] , a characteristic-kernel discrepancy known to be equivalent to MMD with a distance-induced kernel [38 , 68 ] , whose sample estimator differentiates only through the (smooth) simulator output and never through a non-differentiable comparison. We instantiate it as D ϵ D_{\epsilon} D ϵ by identifying P P P with the simulator distribution p ( x ∣ θ ) p(x \mid\theta) p ( x ∣ θ ) and Q Q Q with the (point-mass) data distribution at x o x_o x o . For two distributions P , Q P,Q P , Q on R d x \mathbb{R}^{d_x} R d x and the Euclidean kernel k ( u , v ) = − ∥ u − v ∥ 2 k(u,v)=-\lVert u-v\rVert_2 k ( u , v ) = − ∥ u − v ∥ 2 , the squared energy distance is
E 2 ( P , Q ) = 2 E X ∼ P , Y ∼ Q ∥ X − Y ∥ − E X , X ′ ∼ P ∥ X − X ′ ∥ − E Y , Y ′ ∼ Q ∥ Y − Y ′ ∥ , (18) \mathcal{E}^2(P,Q)=2\mathbb{E}_{X\sim P,Y\sim Q}\lVert X-Y\rVert-\mathbb{E}_{X,X'\sim P}\lVert X-X'\rVert-\mathbb{E}_{Y,Y'\sim Q}\lVert Y-Y'\rVert, \tag*{(18)} E 2 ( P , Q ) = 2 E X ∼ P , Y ∼ Q ∥ X − Y ∥ − E X , X ′ ∼ P ∥ X − X ′ ∥ − E Y , Y ′ ∼ Q ∥ Y − Y ′ ∥ , (18)
with E 2 ( P , Q ) = 0 \mathcal{E}^2(P,Q)=0 E 2 ( P , Q ) = 0 iff P = Q P=Q P = Q [67 ] . The bracket in Eq. (19) is its minimum-variance unbiased U U U -statistic. For each θ \theta θ in the training minibatch we draw K K K i.i.d. simulator replicates { x k } k = 1 K = { s ( θ ; ω k ) } k = 1 K \{x_k\}_{k=1}^{K}=\{s(\theta;\omega_k)\}_{k=1}^{K} { x k } k = 1 K = { s ( θ ; ω k ) } k = 1 K as samples from P P P , and treat the N N N available observations { y n } n = 1 N \{y_n\}_{n=1}^{N} { y n } n = 1 N as samples from Q Q Q (in our setting N = 1 N=1 N = 1 and y 1 = x o y_1=x_o y 1 = x o ). The unbiased U U U -statistic estimator, divided by a length scale ϵ > 0 \epsilon>0 ϵ > 0 , is then
D ϵ E D ( s ( θ ) , x o ) = 1 ϵ [ 2 K N ∑ k = 1 K ∑ n = 1 N ∥ x k − y n ∥ − 1 K ( K − 1 ) ∑ k ≠ k ′ ∥ x k − x k ′ ∥ − 1 N ( N − 1 ) ∑ n ≠ n ′ ∥ y n − y n ′ ∥ ] , (19) D_{\epsilon}^{\mathrm{ED}}(s(\theta),x_o)=\frac{1}{\epsilon}\left[\frac{2}{KN}\sum_{k=1}^{K}\sum_{n=1}^{N}\lVert x_k-y_n\rVert-\frac{1}{K(K-1)}\sum_{k\ne k'}\lVert x_k-x_{k'}\rVert-\frac{1}{N(N-1)}\sum_{n\ne n'}\lVert y_n-y_{n'}\rVert\right], \tag*{(19)} D ϵ ED ( s ( θ ) , x o ) = ϵ 1 K N 2 k = 1 ∑ K n = 1 ∑ N ∥ x k − y n ∥ − K ( K − 1 ) 1 k = k ′ ∑ ∥ x k − x k ′ ∥ − N ( N − 1 ) 1 n = n ′ ∑ ∥ y n − y n ′ ∥ , (19)
with the diagonals k = k ′ k=k' k = k ′ and n = n ′ n=n' n = n ′ excluded; the third term vanishes when N = 1 N=1 N = 1 . For a fixed ϵ \epsilon ϵ the estimator is unbiased, and its expectation over the replicates is exactly the population quantity, E D ϵ E D ( s ( θ ) , x o ) = E 2 ( p ( x ∣ θ ) , δ x o ) / ϵ \mathbb{E}D_{\epsilon}^{\mathrm{ED}}(s(\theta),x_o)=\mathcal{E}^2(p(x\mid\theta),\delta_{x_o})/\epsilon E D ϵ ED ( s ( θ ) , x o ) = E 2 ( p ( x ∣ θ ) , δ x o ) / ϵ , so the surrogate of § 4.1 is the energy distance itself. For N = 1 N=1 N = 1 the estimator is also non-negative, since ∥ x k − x k ′ ∥ ≤ ∥ x k − x o ∥ + ∥ x k ′ − x o ∥ \lVert x_k-x_{k'}\rVert\le\lVert x_k-x_o\rVert+\lVert x_{k'}-x_o\rVert ∥ x k − x k ′ ∥ ≤ ∥ x k − x o ∥ + ∥ x k ′ − x o ∥ , so it is a sufficient distance measure (Definition A.1). Since E \mathcal{E} E carries the units of x x x , 1 ϵ \frac{1}{\epsilon} ϵ 1 makes D ϵ E D D_{\epsilon}^{\mathrm{ED}} D ϵ ED unit-less. It does not change the minimiser and only weights the discrepancy against the prior. By default ϵ \epsilon ϵ is adaptive. It is the median pairwise distance among simulations at 256 prior draws (counted toward the simulation budget) and updated at every energy evaluation by an exponential moving average (decay 0.95) of the minibatch-averaged median pairwise distance among { x k } k = 1 K ∪ { x o } \{x_k\}_{k=1}^{K}\cup\{x_o\} { x k } k = 1 K ∪ { x o } , with no gradient flowing through ϵ \epsilon ϵ . Because ϵ \epsilon ϵ is a single detached scale, updated with weight 0.05 from a minibatch average, it depends only weakly on the replicates of any one θ \theta θ . Within each evaluation we treat it as fixed, so unbiasedness holds conditionally on ϵ \epsilon ϵ , while across training ϵ \epsilon ϵ drifts and the Phase-1 target with it. With a single replicate (K = 1 K=1 K = 1 , used on Two Moons) the second term in (19) is absent and D ϵ E D D_{\epsilon}^{\mathrm{ED}} D ϵ ED reduces to 2 ∥ x 1 − x o ∥ / ϵ 2\lVert x_1-x_o\rVert/\epsilon 2 ∥ x 1 − x o ∥ / ϵ , whose expectation exceeds E 2 / ϵ \mathcal{E}^2/\epsilon E 2 / ϵ by E ∥ X − X ′ ∥ / ϵ \mathbb{E}\lVert X-X'\rVert/\epsilon E ∥ X − X ′ ∥ / ϵ . For Two Moons the simulator noise is additive, so this offset does not depend on θ \theta θ , and by Theorem 4.1 , the gradient is that of the energy distance itself. The median over { x 1 , x o } \{x_1,x_o\} { x 1 , x o } is then simply ∥ x 1 − x o ∥ \lVert x_1-x_o\rVert ∥ x 1 − x o ∥ , so ϵ \epsilon ϵ tracks a moving average of the minibatch-mean distance between the simulations and x o x_o x o . It starts at the scale of the prior predictive and shrinks as Phase 1 concentrates on the posterior, which anneals the surrogate from smooth to sharp during training (Appendix § 13 .§ 9 compares this with fixed length scales).
Maximum mean discrepancy (MMD)
We also consider the squared maximum mean discrepancy (MMD) [38 ] with the Gaussian (RBF) kernel k ϵ ( u , v ) = exp ( − ∥ u − v ∥ 2 2 / ( 2 ϵ 2 ) ) k_{\epsilon}(u,v)=\exp(-\lVert u-v\rVert_2^2/(2\epsilon^2)) k ϵ ( u , v ) = exp ( − ∥ u − v ∥ 2 2 / ( 2 ϵ 2 )) of bandwidth ϵ > 0 \epsilon>0 ϵ > 0 , which is characteristic, so that MMD ϵ 2 ( P , Q ) = 0 \operatorname{MMD}_{\epsilon}^2(P,Q)=0 MMD ϵ 2 ( P , Q ) = 0 iff P = Q P=Q P = Q . As for the energy distance, we identify P P P with the simulator distribution p ( x ∣ θ ) p(x\mid\theta) p ( x ∣ θ ) and Q Q Q with the (point-mass) data distribution at x o x_o x o , and
MMD ϵ 2 ( P , Q ) = E X , X ′ ∼ P k ϵ ( X , X ′ ) + E Y , Y ′ ∼ Q k ϵ ( Y , Y ′ ) − 2 E X ∼ P , Y ∼ Q k ϵ ( Y , Y ′ ) . (20) \operatorname{MMD}_{\epsilon}^{2}(P,Q)=\mathbb{E}_{X,X'\sim P}k_{\epsilon}(X,X')+\mathbb{E}_{Y,Y'\sim Q}k_{\epsilon}(Y,Y')-2\mathbb{E}_{X\sim P,Y\sim Q}k_{\epsilon}(Y,Y'). \tag*{(20)} MMD ϵ 2 ( P , Q ) = E X , X ′ ∼ P k ϵ ( X , X ′ ) + E Y , Y ′ ∼ Q k ϵ ( Y , Y ′ ) − 2 E X ∼ P , Y ∼ Q k ϵ ( Y , Y ′ ) . (20)
For each θ \theta θ in the training minibatch we draw K K K i.i.d. simulator replicates { x k } k = 1 K = { s ( θ ; ω k ) } k = 1 K \{x_k\}_{k=1}^{K}=\{s(\theta;\omega_k)\}_{k=1}^{K} { x k } k = 1 K = { s ( θ ; ω k ) } k = 1 K and treat the N N N observations { y n } n = 1 N \{y_n\}_{n=1}^{N} { y n } n = 1 N as samples from Q Q Q (here N = 1 N=1 N = 1 , y 1 = x o y_1=x_o y 1 = x o ). Since the unbiased U U U -statistic of the Q Q Q –Q Q Q term is undefined for N = 1 N = 1 N = 1 , we use the plug-in (V V V -statistic) estimator
D ϵ M M D ( s ( θ ) , x o ) = 1 K 2 ∑ k = 1 K ∑ k ′ = 1 K k ϵ ( x k , x k ′ ) + 1 N 2 ∑ n = 1 N ∑ n ′ = 1 N k ϵ ( y n , y n ′ ) − 2 K N ∑ k = 1 K ∑ n = 1 N k ϵ ( x k , y n ) , (21) \mathcal{D}^{\mathrm{MMD}}_{\epsilon}(s(\theta),x_o) = \frac{1}{K^2}\sum_{k=1}^{K}\sum_{k'=1}^{K} k_{\epsilon}(x_k,x_{k'}) + \frac{1}{N^2}\sum_{n=1}^{N}\sum_{n'=1}^{N} k_{\epsilon}(y_n,y_{n'}) - \frac{2}{KN}\sum_{k=1}^{K}\sum_{n=1}^{N} k_{\epsilon}(x_k,y_n), \tag*{(21)} D ϵ MMD ( s ( θ ) , x o ) = K 2 1 k = 1 ∑ K k ′ = 1 ∑ K k ϵ ( x k , x k ′ ) + N 2 1 n = 1 ∑ N n ′ = 1 ∑ N k ϵ ( y n , y n ′ ) − K N 2 k = 1 ∑ K n = 1 ∑ N k ϵ ( x k , y n ) , (21)
diagonals included, so that for N = 1 N = 1 N = 1 the second term equals 1. As D ϵ M M D ∈ [ 0 , 2 ] \mathcal{D}^{\mathrm{MMD}}_{\epsilon} \in[0, 2] D ϵ MMD ∈ [ 0 , 2 ] is unit-less, no length-scale normalisation is applied. The bandwidth ϵ \epsilon ϵ is either fixed or, by default, set by the median heuristic separately for each θ \theta θ : ϵ ( θ ) \epsilon(\theta) ϵ ( θ ) is the median pairwise Euclidean distance among { x k } k = 1 K ∪ { x o } \{x_k\}_{k=1}^{K} \cup\{x_o\} { x k } k = 1 K ∪ { x o } , recomputed at every energy evaluation from the current replicates (no pilot simulations, no smoothing), and gradients are propagated through ϵ ( θ ) \epsilon(\theta) ϵ ( θ ) . The V V V -statistic lies in [ 0 , 2 ] [0, 2] [ 0 , 2 ] , so it is a sufficient distance measure (Definition A.1), but unlike the energy distance its expectation is biased, E D ϵ M M D = M M D ϵ 2 + 1 K ( 1 − E k ϵ ( X , X ′ ) ) \mathbb{E}\,\mathcal{D}^{\mathrm{MMD}}_{\epsilon} = \mathrm{MMD}_{\epsilon}^{2} + \frac{1}{K}(1-\mathbb{E}\,k_{\epsilon}(X,X')) E D ϵ MMD = MMD ϵ 2 + K 1 ( 1 − E k ϵ ( X , X ′ )) for fixed ϵ \epsilon ϵ , an additional penalty on simulator spread that vanishes as K → ∞ K \to\infty K → ∞ . With the median heuristic, ϵ ( θ ) \epsilon(\theta) ϵ ( θ ) depends on the replicates and is differentiated through, as for the adaptive variance of Appendix § 13 .§ 1 . MMD is used only in the surrogate ablations (Appendices § 13 .§ 8 and § 13 .§ 9 ).
Bandwidth and Metric Trade-offs
The impact of D ~ ϵ \widetilde{D}_{\epsilon} D ϵ and ϵ \epsilon ϵ on p ~ ϵ ( x o ∣ θ ) \widetilde{p}_{\epsilon}(x_o \mid\theta) p ϵ ( x o ∣ θ ) has been studied extensively in the ABC literature [21 , 32 , 56 , 60 ] . Under mild regularity, p ~ ϵ → p \widetilde{p}_{\epsilon} \to p p ϵ → p as ϵ → 0 \epsilon\to0 ϵ → 0 , but small ϵ \epsilon ϵ degrades acceptance and inflates Monte Carlo variance, while large ϵ \epsilon ϵ over-smooths the likelihood and biases the posterior [69 ] . The choice of D ~ ϵ \widetilde{D}_{\epsilon} D ϵ involves similar considerations, and kernel-based discrepancies such as MMD and the energy distance can improve the bias-variance trade-off relative to a fixed summary statistic.
In RSBI the trade-off takes a different form, because Phase 1 samples the surrogate with gradients rather than by acceptance. A small ϵ \epsilon ϵ gives a sharp energy whose gradient is dominated by Monte Carlo noise near x o x_o x o and vanishes far from it. A large ϵ \epsilon ϵ flattens the data term until the drift is set by the prior alone. Since Phase 2 NRE-ASBS refinement targets the true posterior, the surrogate need not approximate p ( θ ∣ x o ) p(\theta\mid x_o) p ( θ ∣ x o ) precisely. It only needs to place mass in the region that Phase 2 then refines, so larger ϵ \epsilon ϵ values can be tolerated than in standalone ABC. The surrogates we use are not guaranteed to reproduce the mode, shape or variance of the true likelihood, and Phase 2 corrects the resulting bias.
Our ablations show that both choices depend on the simulator and the data, and that no single surrogate or bandwidth rule is best across tasks. Among surrogates (Appendix § 13 .§ 8 ), the energy distance gives the best Phase-1 proposal on Two Moons and the Bernoulli GLM, where MMD fails to concentrate at all. On SLCP all three surrogates are within one standard deviation of each other in C2ST, and MMD places much of its mass on the edges of the prior box. Whether the bandwidth should be adaptive or fixed is equally task dependent (Appendix § 13 .§ 9 ). On SLCP an adaptive per-θ \theta θ variance is needed for the Gaussian NLL, because θ 3 \theta_3 θ 3 , θ 4 \theta_4 θ 4 and θ 5 \theta_5 θ 5 enter the simulator only through the noise covariance, and no fixed variance discovers the modes better than prior samples do. On the Bernoulli GLM the opposite holds. Per-θ \theta θ variances estimated from a few binary replicates are too noisy, adaptive bandwidths fail (C2ST ≥ 0.995 \geq0.995 ≥ 0.995 ) and a fixed bandwidth is required. Sensitivity to ϵ \epsilon ϵ also varies. Misspecifying the best bandwidth by an order of magnitude costs 0.13 to 0.18 in Phase-1 C2ST on Two Moons and raises it to 0.98 or more on the Bernoulli GLM, whereas on SLCP the C2ST changes by less than 0.03 over four orders of magnitude. The pilot landscape heuristic of Sec. C.5 chooses bandwidths without the reference posterior that lie within half a decade of the best value on Two Moons and coincide with it on the Bernoulli GLM. It remains a heuristic, however, and it does not decide between surrogates or between adaptive and fixed bandwidths. Making these choices in a principled way requires a better understanding of the gradient landscape that a differentiable simulator and a distance induce over the prior, which we leave to future work.
Bandwidth and temperature hyperparameter guesses via energy and gradient landscape
We suggest a cheap heuristic to aid in choosing the surrogate bandwidth ϵ \epsilon ϵ and the Phase-1 temperature T T T . This method leverages a cheap pilot run of O ( 10 2 ) \mathcal{O}(10^2) O ( 1 0 2 ) draws from the prior in order to observe the landscape of gradients and energy throughout the prior volume. The assumption is that bandwidth ϵ \epsilon ϵ and temperature T T T configurations that provide informative gradients for the Phase-1 zoom should produce a structured landscape with gradients that cohesively point to one or more energy wells, no matter where they are within the prior. We detail this process below, noting that exploring a fully fledged hyperparameter optimization regime for WT-ASBS is generally interesting and could constitute future work.
Before any training, we evaluate the surrogate energy E ( θ ) = D ϵ ( s ( θ ) , x o ) − log p ( θ ) E(\theta) = D_{\epsilon}(s(\theta), x_o) - \log p(\theta) E ( θ ) = D ϵ ( s ( θ ) , x o ) − log p ( θ ) on a small pilot batch of prior draws with a few simulator replicates ω 1 : K \omega_{1:K} ω 1 : K each, look at its landscape, and score candidate bandwidths on the same simulations. The heuristic needs no training and no reference posterior, and its only cost is the pilot simulations.
At each pilot draw we record the energy and the drift − ∇ θ E -\nabla_\theta E − ∇ θ E , obtained by back-propagating through the differentiable simulator. Splitting the replicates into two halves gives two independent estimates E 1 E_1 E 1 , E 2 E_2 E 2 , and the Monte Carlo noise of the energy is σ E = 1 2 median ∣ E 1 − E 2 ∣ \sigma_E = \frac{1}{2}\operatorname{median}|E_1-E_2| σ E = 2 1 median ∣ E 1 − E 2 ∣ . All quantities below are computed on the k k k -nearest-neighbour graph of the pilot draws, so every candidate ϵ \epsilon ϵ , T T T is evaluated on the same simulations without new simulator calls. In more than two dimensions a projection of the draws mixes points that differ in the hidden parameters, so the maps are drawn on 2-D slices of two nominal parameters only.
Each draw steps to the neighbour that is lower in energy and best aligned with its drift. Following these steps downhill ends in a sink, and the draws that end in the same sink form one basin, or well; its share of the draws is the share of prior mass it attracts. Two neighbouring draws in different basins whose drifts point away from each other, each drift having a negative projection onto the direction to the other draw, lie on a ridge . The barrier Δ E \Delta E Δ E of a basin is the lowest energy on its boundary (the higher endpoint of a cross-basin edge) minus the basin’s minimum. Adjacent basins are merged unless the barrier between them is at least 3 σ E 3\sigma_E 3 σ E deep and at least 30% of their boundary pairs diverge, so that only boundaries where the drift genuinely splits between two wells remain.
We introduce three metrics that can be used to score a given surrogate bandwidth ϵ \epsilon ϵ . First, the cohesion : the median cosine between a draw’s drift and the direction to its basin’s sink, which measures whether the drift points into its own well. Second, the ridge fraction : the share of cross-basin neighbour pairs whose drifts diverge, which measures whether the boundaries between wells are genuine ridges. Third, the depth : the mean over basins of min ( 1 , Δ E / 10 σ E ) \min(1,\Delta E/10\sigma_E) min ( 1 , Δ E /10 σ E ) , which measures whether the barriers stand above the Monte Carlo noise. For a single well the ridge term is 1 and the depth uses the energy contrast ( E 95 − E min ) / σ E (E_{95}-E_{\min})/\sigma_E ( E 95 − E m i n ) / σ E in place of a barrier. They combine into
structure = cohesion × ridge × depth , separation = { ridge × depth , ≥ 2 wells , 0 , one well . (22) \operatorname{structure} = \operatorname{cohesion} \times\operatorname{ridge} \times\operatorname{depth}, \qquad
\operatorname{separation} =
\begin{cases}
\operatorname{ridge} \times\operatorname{depth}, & \ge2\ \text{wells},\\
0, & \text{one well}.
\end{cases} \tag*{(22)} structure = cohesion × ridge × depth , separation = { ridge × depth , 0 , ≥ 2 wells , one well . (22)
Structure rewards a drift that points coherently into wells that are deep and cleanly separated, whereas separation rewards only the separation of several wells, regardless of the drift inside each. For a multimodal energy landscape we take the ϵ \epsilon ϵ , or the adaptive variant, with the highest separation. For a unimodal energy landscape, we take the highest structure. For certain priors, a large ϵ \epsilon ϵ leaves only the prior’s own well, which every score rates as perfect, so only settings whose median data gradient is at least three times the prior gradient are considered.
The landscape can inform the temperature T T T , but only when WT is on. The target exp ( − E / T ) \exp(-E/T) exp ( − E / T ) crosses a barrier of height Δ E \Delta E Δ E with a probability that scales as exp ( − Δ E / T ) \exp(-\Delta E/T) exp ( − Δ E / T ) (Sec. A.1), so what matters is the barrier in units of the temperature, Δ E / T \Delta E/T Δ E / T . Well-tempered metadynamics (WT) with bias factor γ \gamma γ fills the wells and lowers the effective barrier to roughly Δ E / γ \Delta E/\gamma Δ E / γ , leaving Δ E / ( γ T ) \Delta E/(\gamma T) Δ E / ( γ T ) . Let κ \kappa κ denote the barrier height, in units of T T T , that we want to remain after the bias. Setting Δ E / ( γ T ) = κ \Delta E/(\gamma T)=\kappa Δ E / ( γ T ) = κ , with Δ E \Delta E Δ E the median barrier of the significant wells, gives
T = { 1 , one well (no WT) , Δ E γ κ , several wells . (23) T =
\begin{cases}
1, & \text{one well (no WT)},\\
\frac{\Delta E}{\gamma\kappa}, & \text{several wells}.
\end{cases} \tag*{(23)} T = { 1 , γ κ Δ E , one well (no WT) , several wells . (23)
The value of κ \kappa κ trades mode separation against mixing. A remaining barrier of κ ≈ 3 \kappa\approx3 κ ≈ 3 gives a crossing factor e − 3 ≈ 5 % e^{-3} \approx5\% e − 3 ≈ 5% . Much smaller values flatten the wells and blur the modes, and much larger ones let the sampler stay trapped in the first well it finds. We use κ = 3 \kappa=3 κ = 3 as a heuristic default rather than a tuned value. A single well needs neither tempering nor a bias, so T = 1 T=1 T = 1 .
Figure 9 shows such landscapes for the Two moons, SLCP and Bernoulli GLM tasks. The colour shows the energy, with the white arrows depicting the drift/gradients from the average pilot. The stars depict the location of the basin sinks and the red edges show ridges such that the wells and barriers between energy basins can be inspected directly. Two Moons shows two wells separated by a ridge along θ 1 = − θ 2 \theta_1=-\theta_2 θ 1 = − θ 2 whilst SLCP shows four wells, one for each sign combination of ( θ 3 , θ 4 ) (\theta_3,\theta_4) ( θ 3 , θ 4 ) , separated by ridges along θ 3 = 0 \theta_3=0 θ 3 = 0 . We note that this energy landscape was only seen for the Gaussian surrogate on SLCP. The Bernoulli GLM shows a single well with the drift pointing into it from everywhere. Table 5 lists the scores of these landscapes.
Figure 9. Pilot energy landscapes for Two Moons (left), SLCP (centre) and the Bernoulli GLM (right), computed with the Phase-1 surrogate, bandwidth and temperature used for the benchmark main results at the default prior on observation 1 (see Sec. G for Phase-1 hyperparameters and surrogates used). Colour is log ( 1 + E / T − min E / T ) \log(1 + E/T - \min E/T) log ( 1 + E / T − min E / T ) ; white arrows show the direction of − ∇ θ E -\nabla_\theta E − ∇ θ E ; stars are basin sinks (gold: significant); red edges are ridges. SLCP and the Bernoulli GLM are shown on 2-D slices through ( θ 3 , θ 4 ) (\theta_3,\theta_4) ( θ 3 , θ 4 ) and ( θ 1 , θ 2 ) (\theta_1,\theta_2) ( θ 1 , θ 2 ) . The number in each panel title is the score used to rank bandwidths for that posterior: separation for the multimodal Two Moons and SLCP, structure for the unimodal Bernoulli GLM. Scores for these settings: Two Moons: separation 0.79, structure 0.79; SLCP: separation 0.97, structure 0.21; Bernoulli GLM: separation 0.00, structure 0.74.
Table 5. Landscape scores of the pilot batches in Figure 9 . Wells : significant basins. Cohesion : median cosine between a draw’s drift and the direction to its basin’s sink. Ridge : share of cross-basin neighbour pairs whose drifts point apart (– for a single well, where the ridge term is 1). Depth : mean over basins of min ( 1 , Δ E / 10 σ E ) \min(1,\Delta E/10\sigma_E) min ( 1 , Δ E /10 σ E ) , or min ( 1 , ( E 95 − E min ) / 10 σ E ) \min(1,(E_{95}-E_{\min})/10\sigma_E) min ( 1 , ( E 95 − E m i n ) /10 σ E ) for a single well. Structure = cohesion × ridge × depth. Separation = ridge × depth for two or more wells and 0 otherwise.
Surrogate likelihood ablations
We compare three surrogate distances D ϵ ( s ( θ , ω ) , x o ) \mathcal{D}_{\epsilon}(s(\theta,\omega),x_o) D ϵ ( s ( θ , ω ) , x o ) as the Phase-1 energy: the Gaussian negative log-likelihood (Gaussian-NLL), the energy distance and the maximum mean discrepancy (MMD). We run the comparison on Two Moons, SLCP and the Bernoulli GLM, each at its baseline prior. Within a task all surrogates share the same Phase-1 backbone, and training is capped at the simulation budget N s i m ∈ { 10 3 , 10 4 , 10 5 } N_{\mathrm{sim}}\in\{10^3,10^4,10^5\} N sim ∈ { 1 0 3 , 1 0 4 , 1 0 5 } . All entries are the mean ± \pm ± standard deviation over 10 observations. Tables 6–13 report the results; for the multimodal tasks, Figs. 10 and 11 show the Phase-1 samples of each surrogate on one observation.
These ablations are experiments in their own right, separate from the benchmark runs: rather than reusing the benchmark settings, each surrogate is given its own bandwidth ϵ \epsilon ϵ and temperature T T T , so that we compare reasonably tuned surrogates rather than one surrogate’s defaults. These settings are chosen using the heuristic method of Sec. C.5 from a pilot batch of prior-predictive simulations, which uses no reference posterior. On Two Moons, ϵ \epsilon ϵ is fixed at the scale of the pilot simulations (a Gaussian-NLL variance of 1.0 × 10 − 3 1.0\times10^{-3} 1.0 × 1 0 − 3 ; an energy-distance length scale and MMD bandwidth of 0.78, the median pairwise distance), which is also the value that best separates the two energy wells of the pilot landscape. T T T is then set from the same landscape so that, after well-tempered metadynamics with bias factor 10, about 3 k T kT k T of the median barrier between the wells remains: T = 12.8 T = 12.8 T = 12.8 , 0.068 and 0.032, with one simulator replicate per θ \theta θ . On SLCP the bandwidths are adaptive (as in the benchmark), and T T T is set from the energy landscape in the same way: T = 11.1 T = 11.1 T = 11.1 for the Gaussian-NLL and T = 0.011 T = 0.011 T = 0.011 for MMD, while the energy distance keeps T = 10 − 3 T = 10^{-3} T = 1 0 − 3 from short pilot runs on observation 1, with three replicates per θ \theta θ . On the Bernoulli GLM we keep T = 1 T = 1 T = 1 for all surrogates, because the prior is Gaussian and T T T would temper it as well; sharpness is instead set through a fixed ϵ \epsilon ϵ from the pilot: a Gaussian-NLL variance of 0.05, an energy-distance length scale of 0.914 (as in the benchmark runs) and an MMD bandwidth of 54.2 (about the median pairwise distance of the pilot simulations), with one, two and two replicates per θ \theta θ .
Phase 1 only has to seed Phase 2 with a proposal that places mass on every posterior mode; it does not have to match the posterior. We therefore report two metrics, and a third, mode discovery, for the multimodal Two Moons and SLCP. The first is the C2ST of the Phase-1 samples against the reference posterior (lower is better, 0.5 means indistinguishable). The second is a mode fidelity , which measures how mass is distributed across modes, something C2ST does not explicitly test. Let τ \tau τ be the 95th percentile of the nearest-neighbour distances within the reference sample. The precision P P P is the fraction of Phase-1 samples that lie within τ \tau τ of a reference sample. The recall R m R_m R m of mode m m m is the fraction of reference samples in mode m m m that have a Phase-1 sample within τ \tau τ . The mode fidelity is
F = P ⋅ min m R m ∈ [ 0 , 1 ] , (24) F = P \cdot\min_m R_m \in[0,1], \tag*{(24)} F = P ⋅ m min R m ∈ [ 0 , 1 ] , (24)
with 1 ideal. F → 0 F \to0 F → 0 if Phase 1 misses a mode (min m R m → 0 \min_m R_m \to0 min m R m → 0 ) or spreads its mass too thinly over the prior (P → 0 P \to0 P → 0 ); F > 0 F > 0 F > 0 means every mode is covered. For the unimodal Bernoulli GLM, F F F reduces to precision times recall. On SLCP we compute F F F on ( θ 3 , θ 4 ) (\theta_3, \theta_4) ( θ 3 , θ 4 ) only: these coordinates separate the four modes, whereas the posterior is so narrow in the full five dimensions that almost no Phase-1 sample falls within τ \tau τ of a reference sample, even when Phase 1 has found all four modes. Because F F F rewards samples that land on the reference support rather than near a mode, on Two Moons and SLCP we also report a coarser mode discovery : each Phase-1 sample is assigned to the nearest reference-mode centre (in θ \theta θ for Two Moons, in ( θ 3 , θ 4 ) (\theta_3, \theta_4) ( θ 3 , θ 4 ) for SLCP) if it lies within 1.5 times that mode’s 95th-percentile reference radius (at most half the distance between mode centres), and discovery is K K K times the smallest per-mode share (K K K modes; 1 means all mass split equally over the modes, 0 a missed mode). As a reference point, samples drawn from the prior score 0.04 on Two Moons and 0.23 on SLCP. Because the Phase-1 proposal is much broader than the posterior, F F F is small, and we report it in units of 10 − 3 10^{-3} 1 0 − 3 .
Two Moons. Table 6 shows that with larger budgets the C2ST improves monotonically, to 0.875 for the energy distance, 0.922 for MMD and 0.941 for the Gaussian-NLL at 10 5 10^5 1 0 5 . It never reaches 0.5, as expected, because Phase 1 learns the posterior under the surrogate likelihood rather than the simulator likelihood. Mode fidelity (Table 7 ) is positive for every surrogate and budget, so both moons are always seeded. The energy distance gives by far the best coverage at 10 4 10^4 1 0 4 and 10 5 10^5 1 0 5 simulations, albeit with large variation across observations. Mode discovery (Table 8 , Figure 10 ) ranks the surrogates as C2ST does: at 10 3 10^3 1 0 3 simulations all three are barely above the prior (0.09–0.11 against 0.04), while at 10 5 10^5 1 0 5 the energy distance reaches 0.64, MMD 0.53 and the Gaussian-NLL 0.32.
Table 6. Phase-1 C2ST on Two Moons (baseline prior) vs. the simulation budget (capped), mean ± \pm ± std over 10 observations. Lower is better (0.5 = indistinguishable from the reference).
Table 7. Phase-1 mode fidelity on Two Moons (baseline prior) vs. the simulation budget (capped), mean ± std over 10 observations. Values ×1000.
Table 8. Phase-1 mode discovery on Two Moons (baseline prior) vs. the simulation budget (capped), mean ± std over 10 observations. Higher is better. Samples drawn from the prior score 0.04.
Figure 10. Phase-1 samples on Two Moons for each surrogate’s profile (observation 1, 10 4 10^{4} 1 0 4 simulations; colour) over the reference posterior (grey). Dashed circles: the regions used for mode discovery (1.5 × 1.5\times 1.5 × each moon’s 95th-percentile reference radius). All three surrogates place a cluster on each moon; discovery is limited by how unevenly the mass is split between the moons and by clusters wider than the moons themselves.
SLCP. All three surrogates give a similar Phase-1 C2ST on SLCP, between 0.97 and 0.98 at every budget (Table 9 ); the differences are within one standard deviation. Mode fidelity on ( θ 3 , θ 4 ) (\theta_3, \theta_4) ( θ 3 , θ 4 ) (Table 10 ) is positive for every surrogate and budget, so all three seed all four modes. It is highest for the Gaussian-NLL (at 10 3 10^3 1 0 3 and 10 5 10^5 1 0 5 simulations) and the energy distance (at 10 4 10^4 1 0 4 ); at 10 4 10^4 1 0 4 and 10 5 10^5 1 0 5 it is three to four times lower for MMD, which places much of its mass on the edges of the prior box. Mode discovery (Table 11 , Figure 11 ) shows how weakly Phase 1 concentrates on SLCP at these budgets: the Gaussian-NLL and the energy distance are only moderately above the prior (0.34–0.39 at 10 4 10^4 1 0 4 and 10 5 10^5 1 0 5 simulations, against 0.23), with the Gaussian-NLL ahead at 10 3 10^3 1 0 3 (0.29), and MMD stays at the level of the prior (0.16–0.24); the variation across observations is large. The posterior itself is resolved in Phase 2.
Table 9. Phase-1 C2ST on SLCP (baseline prior) vs. the simulation budget (capped), mean ± \pm ± std over 10 observations. Lower is better (0.5 = indistinguishable from the reference).
Table 10. Phase-1 mode fidelity on SLCP (baseline prior) vs. the simulation budget (capped), mean ± \pm ± std over 10 observations. Values × 1000 \times1000 × 1000 . Mode fidelity is computed on ( θ 3 , θ 4 ) (\theta_3, \theta_4) ( θ 3 , θ 4 ) only, the coordinates that separate the four SLCP modes (and carry the metadynamics bias).
Table 11. Phase-1 mode discovery on SLCP (baseline prior) vs. the simulation budget (capped), mean ± \pm ± std over 10 observations. Higher is better. Samples drawn from the prior score 0.23.
Figure 11. Phase-1 samples on SLCP in the mode-separating coordinates ( θ 3 , θ 4 ) (\theta_3, \theta_4) ( θ 3 , θ 4 ) for each surrogate’s profile (observation 1, 10 4 10^4 1 0 4 simulations; colour) over the reference posterior (grey). Dashed circles: the regions used for mode discovery (1.5 × 1.5\times 1.5 × each mode’s 95th-percentile reference radius). The Gaussian-NLL puts its mass beside all four modes (∣ θ 3 ∣ ≈ 2 |\theta_3| \approx2 ∣ θ 3 ∣ ≈ 2 ), spread along θ 4 \theta_4 θ 4 ; the energy distance spreads its mass in a ring around the prior box that passes through them; MMD concentrates on the box edges.
Bernoulli GLM. The surrogates differ most on the Bernoulli GLM (Table 12 ). The energy distance is the only surrogate whose C2ST improves steadily with the budget, reaching 0.687 at 10 5 10^5 1 0 5 simulations, and it also attains the highest mode fidelity, F ≈ 0.77 F \approx0.77 F ≈ 0.77 (Table 13 ). The Gaussian-NLL stays at a C2ST of 0.97–0.99 with lower fidelity. MMD fails to concentrate on the posterior at all (C2ST = 1.000, F = 0 F = 0 F = 0 ).
Table 12. Phase-1 C2ST on Bernoulli GLM (baseline prior) vs. the simulation budget (capped), mean ± \pm ± std over 10 observations. Lower is better (0.5 = indistinguishable from the reference).
Table 13. Phase-1 mode fidelity on Bernoulli GLM (baseline prior) vs. the simulation budget (capped), mean ± \pm ± std over 10 observations. Values × 1000 \times1000 × 1000 .
Surrogate bandwidth ablations
Each surrogate has a bandwidth ϵ \epsilon ϵ , namely the per-dimension variance of the Gaussian-NLL (Appendix C.1), the length scale of the energy distance (Appendix C.2) and the kernel bandwidth of the MMD (Appendix C.3). Each can be fixed or set adaptively from the simulations, as defined in those appendices. The benchmark runs use the adaptive variance for the Gaussian-NLL on SLCP and Lotka–Volterra and the adaptive length scale for the energy distance on Two Moons. They use a fixed energy-distance length scale of 0.914 on the Bernoulli-GLM, selected from the heuristic procedure of Appendix C.5, and ϵ = 1 \epsilon= 1 ϵ = 1 for the energy distance on gw_ringdown. MMD is not used in the benchmark, and its adaptive variant is the median heuristic.
To measure how sensitive Phase 1 is to ϵ \epsilon ϵ , we rerun Phase 1 on observation 1 with each surrogate’s ablation settings of Appendix § 13 .§ 8 (temperature, replicates and metadynamics) and change only ϵ \epsilon ϵ , which we sweep over a log-spaced grid of fixed values. The temperature stays at the value chosen for the ablation bandwidth, so the sweep isolates ϵ \epsilon ϵ . The budget is N s i m = 10 4 N_{\mathrm{sim}} = 10^{4} N sim = 1 0 4 on SLCP and the Bernoulli GLM, while on Two Moons Phase 1 runs its total benchmark schedule (4,512 simulations). The grid contains the fixed ablation bandwidth of Appendix § 13 .§ 8 and, where the benchmark fixes one, the benchmark value (0.914 for the Bernoulli energy distance). On Two Moons it also contains 0.095, the value at which the benchmark’s adaptive length scale settles once Phase 1 is near the posterior. For the energy distance and a fixed-variance Gaussian-NLL, ϵ \epsilon ϵ only rescales the surrogate, so changing it is equivalent to changing the temperature of the likelihood term. For MMD it also changes the shape of the energy.
For each task we plot C2ST and mode fidelity F F F (Appendix § 13 .§ 8 ) against ϵ \epsilon ϵ and summarise the sweep in two tables. The C2ST table compares the profile with the best fixed bandwidth ϵ ⋆ \epsilon^{\star} ϵ ⋆ , and shows how much C2ST degrades when ϵ ⋆ \epsilon^{\star} ϵ ⋆ is off by a factor of ten in either direction. The fidelity table gives the band of ϵ \epsilon ϵ over which F > 0.01 F > 0.01 F > 0.01 , the bandwidth ϵ \epsilon ϵ peak that maximises F F F , with the precision P P P and worst-mode recall min m R m \min_m R_m min m R m there, and F F F for the profile. Outside this band fidelity collapses in one of two ways. A bandwidth that is too small loses modes (min m R m → 0 \min_m R_m \to0 min m R m → 0 ), and one that is too large spreads mass over the prior (P → 0 P \to0 P → 0 ). On SLCP a third table (Table 18 ) reports mode discovery. We do not consider mode discovery for two moons since all surrogates recover the modes significantly.
Table 18: Phase-1 mode discovery versus the surrogate bandwidth ϵ \epsilon ϵ on SLCP (10,000 simulations, one observation). profile : discovery at the surrogate’s profile setting (adaptive or fixed bandwidth); best fixed ϵ \epsilon ϵ : the swept bandwidth with the highest discovery; range: lowest and highest discovery over the fixed bandwidths. Mode discovery: each Phase-1 sample is assigned to the nearest reference-mode centre in ( θ 3 , θ 4 ) (\theta_3, \theta_4) ( θ 3 , θ 4 ) if it lies within 1.5 × 1.5\times 1.5 × that mode’s 95th-percentile reference radius (at most half the distance between mode centres); discovery = 4 × 4\times 4 × the smallest per-mode share (1 = all mass split equally over the 4 modes, 0 = a mode missed). Samples drawn from the prior score 0.16.
Table 18.
Two Moons. Misspecifying ϵ ⋆ \epsilon^{\star} ϵ ⋆ by an order of magnitude degrades the Phase-1 C2ST by 0.13–0.18, depending on the surrogate (Figure 12 , Table 14 ). The profile bandwidths, chosen without the reference, lie within half a decade of ϵ ⋆ \epsilon^{\star} ϵ ⋆ and are within 0.09 of its C2ST (0.894 vs. 0.809 for the Gaussian-NLL, 0.812 vs. 0.788 for the energy distance and 0.843 vs. 0.810 for MMD). Mode fidelity is non-trivial only in a narrow band of ϵ \epsilon ϵ (Table 15 ), yet every profile lies inside it (F = 0.050 F = 0.050 F = 0.050 , 0.167 and 0.079, against peaks of 0.19–0.25).
Figure 12. Phase-1 sensitivity to the surrogate bandwidth ϵ \epsilon ϵ on Two Moons (4,512 simulations, one observation). Left: C2ST against the reference (lower is better; dotted: 0.5). Right: mode fidelity (= precision × \times × worst-mode recall, log scale; zero drawn at 10 − 5 10^{-5} 1 0 − 5 ). Solid: each surrogate swept over its own bandwidth parameter (Gaussian-NLL variance, energy-distance length scale, MMD kernel bandwidth); dashed: the surrogate’s profile setting.
Table 14. Phase-1 sensitivity to the surrogate bandwidth ϵ \epsilon ϵ on Two Moons (4,512 simulations, one observation). profile : C2ST at each surrogate’s bandwidth and temperature as chosen, without the reference, from the energy-landscape diagnostics. ϵ ⋆ \epsilon^\star ϵ ⋆ : the best fixed bandwidth on the sweep (identified using the reference), with its C2ST, and the C2ST when ϵ ⋆ \epsilon^\star ϵ ⋆ is mis-set by one order of magnitude in either direction (change relative to ϵ ⋆ \epsilon^\star ϵ ⋆ in brackets). Lower is better (0.5 = indistinguishable from the reference).
Table 15. Phase-1 mode fidelity (= precision × worst-mode recall; higher is better) versus the surrogate bandwidth ϵ \epsilon ϵ on Two Moons (4,512 simulations, one observation). Non-trivial band: the range of ϵ \epsilon ϵ with mode fidelity above 0.01 (– if none); ϵ p e a k \epsilon^{\mathrm{peak}} ϵ peak : the bandwidth with the highest mode fidelity, with its precision and worst-mode recall; profile : mode fidelity at the surrogate’s profile setting.
SLCP. Phase-1 C2ST on SLCP is almost insensitive to ϵ \epsilon ϵ : all values lie between 0.978 and 1.000 over four orders of magnitude, and ϵ ⋆ \epsilon^\star ϵ ⋆ falls at the edge of the grid for the Gaussian-NLL and the energy distance (Figure 13 , Table 16 ). The profiles, which use adaptive bandwidths on SLCP, are within 0.005 of the best fixed value, and better than it for the Gaussian-NLL. For the Gaussian-NLL a fixed variance discards information: θ 3 \theta_3 θ 3 –θ 5 \theta_5 θ 5 enter SLCP only through the noise covariance, which the adaptive per-θ \theta θ variance captures and a fixed one does not. Accordingly, its mode fidelity on ( θ 3 , θ 4 ) (\theta_3, \theta_4) ( θ 3 , θ 4 ) stays at or below 0.003 for every fixed variance, against 0.007 for the adaptive profile (Table 17 ). The energy distance keeps F > 0.01 F > 0.01 F > 0.01 over the whole grid (peak 0.038) and MMD for ϵ ≥ 3 \epsilon\ge3 ϵ ≥ 3 (peak 0.032). Mode discovery is shown in Table 18 and shows the cost of fixing the Gaussian-NLL variance: the adaptive profile reaches 0.23, whereas no fixed variance exceeds the prior (≤ 0.14 \le0.14 ≤ 0.14 ), and variances up to 0.1 miss modes entirely. For the energy distance the adaptive profile (0.38) matches the best fixed length scale (0.37); for MMD the adaptive bandwidth (0.13) is below every fixed one (0.14–0.26). We note from the results of Figure 11 that the mode discovery and fidelity metrics do not consider distinct isolation of the Phase 1 modes present with the GNLL.
Table 16. Phase-1 sensitivity to the surrogate bandwidth ϵ \epsilon ϵ on SLCP (10,000 simulations, one observation). profile : C2ST at each surrogate’s bandwidth and temperature as chosen, without the reference, from the energy-landscape diagnostics. ϵ ⋆ \epsilon^\star ϵ ⋆ : the best fixed bandwidth on the sweep (identified using the reference), with its C2ST, and the C2ST when ϵ ⋆ \epsilon^\star ϵ ⋆ is mis-set by one order of magnitude in either direction (change relative to ϵ ⋆ \epsilon^\star ϵ ⋆ in brackets). Lower is better (0.5 = indistinguishable from the reference).
Figure 13. Phase-1 sensitivity to the surrogate bandwidth ϵ \epsilon ϵ on SLCP (10,000 simulations, one observation). Left: C2ST against the reference (lower is better; dotted: 0.5). Right: mode fidelity (= precision × \times × worst-mode recall, log scale; zero drawn at 10 − 5 10^{-5} 1 0 − 5 ). Solid: each surrogate swept over its own bandwidth parameter (Gaussian-NLL variance, energy-distance length scale, MMD kernel bandwidth); dashed: the surrogate’s profile setting. Mode fidelity is computed on ( θ 3 , θ 4 ) (\theta_3, \theta_4) ( θ 3 , θ 4 ) only, the coordinates that separate the four SLCP modes (and carry the metadynamics bias).
Table 17: Phase-1 mode fidelity (= precision × \times × worst-mode recall; higher is better) versus the surrogate bandwidth ϵ \epsilon ϵ on SLCP (10,000 simulations, one observation). Non-trivial band: the range of ϵ \epsilon ϵ with mode fidelity above 0.01 (– if none); ϵ p e a k \epsilon^{\mathrm{peak}} ϵ peak : the bandwidth with the highest mode fidelity, with its precision and worst-mode recall; profile : mode fidelity at the surrogate’s profile setting. Mode fidelity is computed on ( θ 3 , θ 4 ) (\theta_3, \theta_4) ( θ 3 , θ 4 ) only, the coordinates that separate the four SLCP modes (and carry the metadynamics bias).
Table 17.
Bernoulli GLM. On the Bernoulli GLM adaptive bandwidths fail (C2ST ≥ 0.995 \ge0.995 ≥ 0.995 , F = 0 F = 0 F = 0 on observation 1): for the Gaussian-NLL the per-θ \theta θ variance estimated from few replicates of the Bernoulli observations is too noisy. A fixed bandwidth is therefore required, and it matters (Figure 14 , Table 19 ): the profile bandwidths coincide with ϵ ⋆ \epsilon^\star ϵ ⋆ for both the energy distance (C2ST 0.770) and the Gaussian-NLL (0.957), but the C2ST rises to 0.98–1.00 when they are off by a factor of 10. Both have a broad band of bandwidths with non-trivial fidelity, peaking at F = 0.81 F = 0.81 F = 0.81 for the energy distance (Table 20 ). MMD fails at every bandwidth in the grid. The fixed bandwidth that was used for the benchmarking in the main results was derived from the profile value and corresponds to the best performing bandwidth in this ablation.
Figure 14. Phase-1 sensitivity to the surrogate bandwidth ϵ \epsilon ϵ on Bernoulli GLM (10,000 simulations, one observation). Left: C2ST against the reference (lower is better; dotted: 0.5). Right: mode fidelity (= precision × \times × worst-mode recall, log scale; zero drawn at 10 − 5 10^{-5} 1 0 − 5 ). Solid: each surrogate swept over its own bandwidth parameter (Gaussian-NLL variance, energy-distance length scale, MMD kernel bandwidth); dashed: the surrogate’s profile setting.
Table 19: Phase-1 sensitivity to the surrogate bandwidth ϵ \epsilon ϵ on Bernoulli GLM (10,000 simulations, one observation). profile : C2ST at each surrogate’s bandwidth and temperature as chosen, without the reference, from the energy-landscape diagnostics. ϵ ⋆ \epsilon^\star ϵ ⋆ : the best fixed bandwidth on the sweep (identified using the reference), with its C2ST, and the C2ST when ϵ ⋆ \epsilon^\star ϵ ⋆ is mis-set by one order of magnitude in either direction (change relative to ϵ ⋆ \epsilon^\star ϵ ⋆ in brackets). Lower is better (0.5 = indistinguishable from the reference).
Table 19.
Table 20: Phase-1 mode fidelity (= precision × \times × worst-mode recall; higher is better) versus the surrogate bandwidth ϵ \epsilon ϵ on Bernoulli GLM (10,000 simulations, one observation). Non-trivial band: the range of ϵ \epsilon ϵ with mode fidelity above 0.01 (– if none); ϵ p e a k \epsilon^{\mathrm{peak}} ϵ peak : the bandwidth with the highest mode fidelity, with its precision and worst-mode recall; profile : mode fidelity at the surrogate’s profile setting.
Table 20.
Task details
This appendix comprises details on computational expense as well as descriptive summaries of each experiment conducted in this paper. For the experiments, RSBI required differentiable versions of each simulator taken from the sbibm package. At the true parameters of observations 1 and 5, two-sample Kolmogorov–Smirnov tests on every output dimension, at the 5% level with Bonferroni correction, find no difference between the original sbibm simulators and our re-implementations for any task. Means agree to within 2.3 standard errors and standard deviations to within 6%, consistent with sampling noise at the sample sizes used (1.5 × 10 3 1.5 \times10^{3} 1.5 × 1 0 3 to 4 × 10 4 4 \times10^{4} 4 × 1 0 4 draws).
Compute Usage
Nearly all experiments in Section 5 are run on a single NVIDIA A100 40G GPU, with some run instead on an NVIDIA GTX 1660 Ti laptop GPU. The time taken for each RSBI run varies depending on the number of simulations and specific hyperparameters. In general, each RSBI run on the Two Moons simulator using the NVIDIA A100 40G GPU in the 10 5 10^{5} 1 0 5 simulation budget regime takes ∼ 25 \sim25 ∼ 25 minutes to run. Total compute time on this specific experiment therefore approaches ∼ 4 \sim4 ∼ 4 hours.
GW ringdown
Gravitational-wave ringdown is the final stage of a binary black-hole merger [70 ] , occurring after the two black holes merge and the newly formed remnant settles into a single spinning black hole. During this phase the signal is dominated by a superposition of damped sinusoids, the quasi-normal modes, whose frequencies and decay times are determined entirely by the remnant’s mass and spin according to general relativity. This makes ringdown a uniquely clean probe of strong-field gravity: measuring these modes constrains the remnant properties and tests general relativity in the high-curvature regime [71 , 72 ] .
Our differentiable ringdown simulator combines two quasi-normal modes (ℓ = m = 2 \ell= m = 2 ℓ = m = 2 , n ∈ { 0 , 1 } n \in\{0,1\} n ∈ { 0 , 1 } ) with the H1 antenna pattern at the LIGO Hanford detector and additive Gaussian noise drawn from the H1 power spectral density estimated from LIGO first-observing-run data around GW150914. The strain is whitened via the Cholesky factor of the Toeplitz noise covariance built from the autocorrelation implied by this PSD, giving a unit-variance representation that sets the natural surrogate length scale ϵ = 1 \epsilon= 1 ϵ = 1 . The analysis window is T = 0.2 T = 0.2 T = 0.2 s at a sampling rate of 4096 Hz, giving d x = 820 d_x = 820 d x = 820 samples per detector. The full forward map, from waveform generation through antenna projection, noise injection and whitening, is implemented in PyTorch so that ∇ θ s ( θ , ω ) \nabla_\theta s(\theta,\omega) ∇ θ s ( θ , ω ) is available end-to-end, which is what makes Phase 1’s surrogate gradient and Phase 2’s adjoint matching applicable. The noise enters as L z Lz L z with z ∼ N ( 0 , I ) z \sim\mathcal{N}(0,I) z ∼ N ( 0 , I ) , and L L L the Cholesky factor, so the simulator is reparameterised exactly. Unlike the other tasks, this simulator is not a re-implementation. The same class generates the observations, the reference posteriors, the simulations of the baselines and those of RSBI, with the same detector, window and sampling settings.
We infer θ = ( M , χ , A 220 , φ 220 , A 221 , φ 221 , cos ι , α , cos δ , ψ ) \theta= (M, \chi, A_{220}, \varphi_{220}, A_{221}, \varphi_{221}, \cos\iota, \alpha, \cos\delta, \psi) θ = ( M , χ , A 220 , φ 220 , A 221 , φ 221 , cos ι , α , cos δ , ψ ) with d θ = 10 d_\theta= 10 d θ = 10 in total. For the experiments reported in the main text the four extrinsic dimensions ( cos ι , α , cos δ , ψ ) (\cos\iota, \alpha, \cos\delta, \psi) ( cos ι , α , cos δ , ψ ) are pinned to truth, the natural conditional setup for a single-detector analysis with known sky position and distance, reducing the inferred dimension to d θ = 6 d_\theta= 6 d θ = 6 . Two prior scenarios are considered. The default scenario with M ∈ [ 40 , 100 ) M ⊙ M \in[40,100)\,M_\odot M ∈ [ 40 , 100 ) M ⊙ , χ ∈ [ 0 , 0.99 ] \chi\in[0,0.99] χ ∈ [ 0 , 0.99 ] , A 220 , A 221 ∈ [ 0 , 5 ] A_{220}, A_{221} \in[0,5] A 220 , A 221 ∈ [ 0 , 5 ] , φ 220 , φ 221 ∈ [ 0 , 2 π ] \varphi_{220}, \varphi_{221} \in[0,2\pi] φ 220 , φ 221 ∈ [ 0 , 2 π ] , cos ι , cos δ ∈ [ − 1 , 1 ] \cos\iota, \cos\delta\in[-1,1] cos ι , cos δ ∈ [ − 1 , 1 ] , α ∈ [ 0 , 2 π ] \alpha\in[0,2\pi] α ∈ [ 0 , 2 π ] , ψ ∈ [ 0 , π ] \psi\in[0,\pi] ψ ∈ [ 0 , π ] covers astrophysically plausible stellar-mass remnants. The wide scenario with M ∈ [ 10 , 200 ] M ⊙ M \in[10,200]\,M_\odot M ∈ [ 10 , 200 ] M ⊙ and A 220 , A 221 ∈ [ 0 , 15 ] A_{220}, A_{221} \in[0,15] A 220 , A 221 ∈ [ 0 , 15 ] , with the remaining ranges unchanged, relaxes the astrophysical priors to those a fundamental-physics analysis might use when remaining agnostic about the source population. The two scenarios give log 10 R p ≈ 2.6 \log_{10} R_p \approx2.6 log 10 R p ≈ 2.6 and 2.8 respectively (Table 21 ).
Table 21: Prior scenarios and mean log prior-to-posterior volume ratios, averaged over the ten observations. Posterior volumes are estimated from the reference samples as the product of per-dimension 95% marginal widths and as the volume of the sample-covariance ellipsoid. For Gaussian priors, the prior volume is measured in the same way from its covariance (marginal widths 2 × 1.96 σ d 2 \times1.96 \ \sigma_d 2 × 1.96 σ d ). Lotka–Volterra volumes are computed in log θ \log\theta log θ . Σ 0 \Sigma_0 Σ 0 denotes the default sbibm Bernoulli GLM prior covariance and μ 0 = ( − 0.125 , − 3 , − 0.125 , − 3 ) \mu_0 = (-0.125, -3, -0.125, -3) μ 0 = ( − 0.125 , − 3 , − 0.125 , − 3 ) .
Table 21.
Because the noise model is exact in whitened coordinates, the likelihood is analytic given θ \theta θ and we can sample a reference posterior directly. Ten injections are drawn uniformly from a sub-box of the default prior, with M ∈ [ 60 , 80 ] M ⊙ M \in[60,80]\,M_\odot M ∈ [ 60 , 80 ] M ⊙ , A 220 ∈ [ 3 , 5 ] A_{220} \in[3,5] A 220 ∈ [ 3 , 5 ] and A 221 ∈ [ 2 , 4 ] A_{221} \in[2,4] A 221 ∈ [ 2 , 4 ] and the remaining parameters over their full default ranges, and one noise realisation is added per injection to produce x o x_o x o . The same ten observations are used across both prior scenarios, since widening the prior does not regenerate the data, only the reference posterior. Figure 3 shows observation 10, whose injected source is M = 70.18 M ⊙ M = 70.18\,M_\odot M = 70.18 M ⊙ , χ = 0.558 \chi= 0.558 χ = 0.558 , A 220 = 4.249 A_{220} = 4.249 A 220 = 4.249 , φ 220 = 5.873 \varphi_{220} = 5.873 φ 220 = 5.873 rad, A 221 = 2.724 A_{221} = 2.724 A 221 = 2.724 and φ 221 = 1.099 \varphi_{221} = 1.099 φ 221 = 1.099 rad. We draw N r e f = 10 4 N_{\mathrm{ref}} = 10^{4} N ref = 1 0 4 reference samples per observation. The emcee runs are computed once per (prior, observation) pair and reused across method runs.
We run RSBI at a single simulation budget of N s i m = 10 5 N_{\mathrm{sim}} = 10^{5} N sim = 1 0 5 with R = 2 R = 2 R = 2 Phase-2 rounds. Phase 1 uses 5.99 × 10 4 5.99 \times10^{4} 5.99 × 1 0 4 simulations, which seed the first NRE (topped up to 6 × 10 4 6 \times10^{4} 6 × 1 0 4 ), and the second adds 4 × 10 4 4 \times10^{4} 4 × 1 0 4 simulations drawn from the round-0 Phase 2 sampler. Phase 1’s energy is the unbiased energy distance of Appendix § 13 .§ 2 with a fixed length scale ϵ = 1 \epsilon= 1 ϵ = 1 and K = 4 K = 4 K = 4 replicates. The source is the prior box with a soft boundary penalty (λ s o f t = 10 \lambda_{\mathrm{soft}} = 10 λ soft = 10 ), which keeps the bridge from spending capacity on the prior corners. Well-tempered metadynamics acts in Phase 1 along ξ ( θ ) = ( M , χ ) \xi(\theta) = (M, \chi) ξ ( θ ) = ( M , χ ) to cover the mass and spin basins, and Phase 2 starts from the Phase 1 samples reweighted by the final bias. Phase 2 runs without a bias, in coordinates whitened by the Phase 1 samples (Appendix § 11 .§ 7 ). Both prior scenarios share every setting, and the sampler operates on the prior box rescaled to [ 0 , 10 ] 6 [0, 10]^6 [ 0 , 10 ] 6 . Inference is on the 6D free subspace, with all MLP input dimensions remapped from the on-disk d θ = 10 d_{\theta} = 10 d θ = 10 to 6. The full configuration is given in Sec. G.
SLCP
We evaluate RSBI on the SLCP task of Lueckmann et al. [18 ] (θ ∈ R 5 \theta\in\mathbb{R}^{5} θ ∈ R 5 , x ∈ R 8 x \in\mathbb{R}^{8} x ∈ R 8 ) across the three prior boxes DEFAULT = [ − 3 , 3 ] 5 = [-3, 3]^5 = [ − 3 , 3 ] 5 , WIDE5X = [ − 5 , 5 ] 5 = [-5, 5]^5 = [ − 5 , 5 ] 5 , and WIDE10X = [ − 10 , 10 ] 5 = [-10, 10]^5 = [ − 10 , 10 ] 5 (see Table 21 for the corresponding prior-to-posterior volume ratios), each at simulator-call budgets N s i m ∈ { 10 3 , 10 4 , 10 5 } N_{\mathrm{sim}} \in\{10^3, 10^4, 10^5\} N sim ∈ { 1 0 3 , 1 0 4 , 1 0 5 } , on observations 1–10 from the sbibm reference set. Posterior quality is reported via the C2ST against the reference posterior of each prior, evaluated on 10 4 10^4 1 0 4 posterior samples.
The simulator draws four i.i.d. points x j ∼ N ( m θ , S θ ) x_j \sim\mathcal{N}(m_{\theta}, S_{\theta}) x j ∼ N ( m θ , S θ ) , with m θ = ( θ 1 , θ 2 ) m_{\theta} = (\theta_1, \theta_2) m θ = ( θ 1 , θ 2 ) and S θ S_{\theta} S θ built from s 1 = θ 3 2 s_1 = \theta_3^2 s 1 = θ 3 2 , s 2 = θ 4 2 s_2 = \theta_4^2 s 2 = θ 4 2 and ρ = tanh θ 5 \rho= \tanh\theta_5 ρ = tanh θ 5 as in sbibm. We re-implement it in PyTorch through the reparameterisation x j = m θ + L θ z j x_j = m_{\theta} + L_{\theta}z_j x j = m θ + L θ z j , z j ∼ N ( 0 , I 2 ) z_j \sim\mathcal{N}(0, I_2) z j ∼ N ( 0 , I 2 ) , where L θ L_{\theta} L θ is the Cholesky factor of S θ S_{\theta} S θ , computed in closed form for the 2 × 2 2 \times2 2 × 2 case with a diagonal jitter of 10 − 6 10^{-6} 1 0 − 6 , so that ∇ θ x \nabla_{\theta}x ∇ θ x is available by automatic differentiation. The baselines use the original sbibm simulator. The two implementations agree everywhere except where tanh θ 5 \tanh\theta_5 tanh θ 5 saturates in single precision (∣ θ 5 ∣ ≳ 8.3 |\theta_5| \gtrsim8.3 ∣ θ 5 ∣ ≳ 8.3 , reachable only under WIDE10X). There sbibm clamps ρ \rho ρ to ± 0.9999 \pm0.9999 ± 0.9999 , whereas we floor the diagonal of L θ L_{\theta} L θ at 10 − 3 10^{-3} 1 0 − 3 , and both only serve to keep the covariance positive definite.
The controlled-SDE drift u ϕ ( θ , t ) u_{\phi}(\theta, t) u ϕ ( θ , t ) and the corrector head are Fourier-feature MLPs with sinusoidal time embedding and GELU activations, 6 × 64 6 \times64 6 × 64 (depth × \times × width) in Phase 1 and 4 × 128 4 \times128 4 × 128 in Phase 2. The NRE classifier projects θ \theta θ and x x x separately to 32 features each, concatenates them and applies a 6 × 64 6 \times64 6 × 64 MLP with SiLU activations. Both phases integrate the controlled SDE on [ 0 , 1 ] [0, 1] [ 0 , 1 ] with an EDM-style noise schedule [73 ] (ρ = 7 \rho= 7 ρ = 7 ), using 64 Euler–Maruyama steps in Phase 1 and 256 in Phase 2.
Phase 1 fits a controlled SDE that approximately samples the surrogate posterior under the Gaussian-NLL surrogate distance D e n s N L L ( s ( θ ) , x o ) D^{\mathrm{NLL}}_{\mathrm{ens}}(s(\theta), x_o) D ens NLL ( s ( θ ) , x o ) of Section C.1, with the per-dimension variance estimated adaptively from K K K replicates of the differentiable simulator at each evaluation.
Phase 2 runs R R R rounds, R = 1 R = 1 R = 1 at N s i m = 10 3 N_{\mathrm{sim}} = 10^3 N sim = 1 0 3 and R = 2 R = 2 R = 2 at 10 4 10^4 1 0 4 and 10 5 10^5 1 0 5 . The first NRE is trained on the Phase 1 simulations, topped up with draws from the Phase 1 sampler (10 3 10^3 1 0 3 , 5 × 10 3 5 \times10^3 5 × 1 0 3 and 1.4 × 10 4 1.4 \times10^4 1.4 × 1 0 4 simulations in total), and the second round adds the remaining budget (5 × 10 3 5 \times10^3 5 × 1 0 3 and 8.6 × 10 4 8.6 \times10^4 8.6 × 1 0 4 ) from the previous round’s sampler. Each round refines the controlled SDE, warm-started from the previous round and started from the Phase 1 sampler, against E r ( θ ) = − log p ( θ ) − log ψ r ( θ , x o ) E_r(\theta) = -\log p(\theta) - \log\psi_{r}(\theta, x_o) E r ( θ ) = − log p ( θ ) − log ψ r ( θ , x o ) . Well-tempered metadynamics [16 , 17 ] biases the SDE along the collective variable ξ ( θ ) = ( θ 3 , θ 4 ) \xi(\theta) = (\theta_3, \theta_4) ξ ( θ ) = ( θ 3 , θ 4 ) to encourage exploration of the four SLCP modes. The bias is reset to zero on the final round, so that the sampled distribution is unbiased up to importance reweighting (§ 6.1 ). Hyperparameters are listed in Sec. G.
Two Moons
We evaluate RSBI on the Two Moons task of Lueckmann et al. [18 ] (θ ∈ R 2 \theta\in\mathbb{R}^{2} θ ∈ R 2 , x ∈ R 2 x \in\mathbb{R}^{2} x ∈ R 2 ) across the three prior boxes DEFAULT = [ − 1 , 1 ] 2 = [-1, 1]^2 = [ − 1 , 1 ] 2 , WIDE10X = [ − 10 , 10 ] 2 = [-10, 10]^2 = [ − 10 , 10 ] 2 , and WIDE50X = [ − 50 , 50 ] 2 = [-50, 50]^2 = [ − 50 , 50 ] 2 (see Table 21 for the corresponding prior-to-posterior volume ratios), each at simulator-call budgets N s i m ∈ { 10 3 , 10 4 , 10 5 } N_{\mathrm{sim}} \in\{10^3, 10^4, 10^5\} N sim ∈ { 1 0 3 , 1 0 4 , 1 0 5 } , on observations 1–10 from the sbibm reference set. Posterior quality is reported via the C2ST against the closed-form sbibm reference posterior, sampled by rejection and evaluated on 10 4 10^4 1 0 4 posterior samples.
The controlled-SDE drift u ϕ ( θ , t ) u_{\phi}(\theta, t) u ϕ ( θ , t ) and the corrector head are Fourier-feature MLPs with sinusoidal time embedding and GELU activations, 4 × 128 4 \times128 4 × 128 (depth × \times × width) in Phase 1 and 6 × 64 6 \times64 6 × 64 in Phase 2. The NRE classifier projects θ \theta θ and x x x separately to half its width each, concatenates them and applies a 6-layer MLP with SiLU activations, of width 64 to 256 depending on the cell. Phase 1 integrates its controlled SDE on [ 0 , 1 ] [0,1] [ 0 , 1 ] with a Brownian schedule on the DEFAULT prior and an EDM-style schedule [73 ] on the wide priors, over 128 Euler–Maruyama steps (256 for WIDE50X). Phase 2 uses an EDM-style schedule over 256 steps throughout.
The Two Moons simulator
s ( θ ; a , r ) = ( r cos a + 0.25 − ∣ θ 1 + θ 2 ∣ 2 , r sin a + θ 1 + θ 2 2 ) s(\theta; a,r)=\left(r\cos a+0.25-\frac{|\theta_{1}+\theta_{2}|}{\sqrt{2}},\ r\sin a+\frac{\theta_{1}+\theta_{2}}{\sqrt{2}}\right) s ( θ ; a , r ) = ( r cos a + 0.25 − 2 ∣ θ 1 + θ 2 ∣ , r sin a + 2 θ 1 + θ 2 )
is driven by the random angle a a a and radius r r r . We re-implement it in PyTorch with the noise drawn independently of θ \theta θ , a = π ( u − 1 2 ) a=\pi(u-\frac{1}{2}) a = π ( u − 2 1 ) with u ∼ U ( 0 , 1 ) u\sim U(0,1) u ∼ U ( 0 , 1 ) and r = 0.1 + 0.01 z r=0.1+0.01z r = 0.1 + 0.01 z with z ∼ N ( 0 , 1 ) z\sim\mathcal{N}(0,1) z ∼ N ( 0 , 1 ) , as in sbibm, so that s s s is differentiable in θ \theta θ away from the line θ 1 = − θ 2 \theta_{1}=-\theta_{2} θ 1 = − θ 2 , where automatic differentiation returns a subgradient of ∣ θ 1 + θ 2 ∣ |\theta_{1}+\theta_{2}| ∣ θ 1 + θ 2 ∣ . This implementation replaces the sbibm simulator for all methods, so RSBI and the baselines use identical code. Phase 1 uses the energy-distance surrogate D ϵ E D D_{\epsilon}^{\mathrm{ED}} D ϵ ED of Section C.2 with an adaptive length scale and K = 1 K=1 K = 1 replicate per parameter.
Phase 2 runs R = 2 R=2 R = 2 rounds in every cell. The first NRE is trained on the Phase 1 simulations, topped up with draws from the Phase 1 sampler, and the second round simulates the rest of the budget from the round-1 sampler, with 600 + 400 simulations at N s i m = 10 3 N_{\mathrm{sim}}=10^{3} N sim = 1 0 3 , and 5,000 + 5,000 and 55,000 + 45,000 on the DEFAULT prior at 10 4 10^{4} 1 0 4 and 10 5 10^{5} 1 0 5 (1,000 + 9,000 and 5,000 + 95,000 on the wide priors). Each round refines the warm-started controlled SDE against E r ( θ ) = − log p ( θ ) − log r ψ r ( θ , x o ) E_r(\theta)=-\log p(\theta)-\log r_{\psi_r}(\theta,x_o) E r ( θ ) = − log p ( θ ) − log r ψ r ( θ , x o ) . Well-tempered metadynamics is used in Phase 2 on the wide priors and on the DEFAULT prior at 10 3 10^{3} 1 0 3 simulations, and in Phase 1 on the DEFAULT prior at 10 4 10^{4} 1 0 4 and 10 5 10^{5} 1 0 5 simulations. It is off elsewhere. All Phase 1, Phase 2 and NRE hyperparameters, which differ between prior boxes and budgets, are listed in Sec. G.
Bernoulli GLM
We evaluate RSBI on the Bernoulli GLM task of Lueckmann et al. [18 ] (θ ∈ R 10 \theta\in\mathbb{R}^{10} θ ∈ R 10 , x ∈ R 10 x\in\mathbb{R}^{10} x ∈ R 10 ), a generalised linear model in which the spike train of a neuron, driven by a white-noise stimulus over 100 time bins, is reduced to ten sufficient statistics. The prior is Gaussian, θ ∼ N ( 0 , B − 1 ) \theta\sim\mathcal{N}(0,B^{-1}) θ ∼ N ( 0 , B − 1 ) , with the smoothness-inducing precision B B B of sbibm. We widen it by scaling the precision by 1, 0.04 and 0.01, which multiplies the prior standard deviations by 1, 5 and 10 (DEFAULT, MEDIUM and LARGE; see Table 21 for the corresponding prior-to-posterior volume ratios). Each prior is run at simulator-call budgets N s i m ∈ { 10 3 , 10 4 , 10 5 } N_{\mathrm{sim}}\in\{10^{3},10^{4},10^{5}\} N sim ∈ { 1 0 3 , 1 0 4 , 1 0 5 } on observations 1–10 from the sbibm reference set, and posterior quality is reported via the C2ST against the reference posterior of each prior on 10 4 10^{4} 1 0 4 posterior samples.
We re-implement the sbibm simulator in PyTorch, batched over θ \theta θ , with the same design matrix X ∈ R 100 × 10 X\in\mathbb{R}^{100\times10} X ∈ R 100 × 10 and stimulus. Each spike is drawn as y t ∼ Bernoulli ( σ ( ψ t ) ) y_t\sim\operatorname{Bernoulli}(\sigma(\psi_t)) y t ∼ Bernoulli ( σ ( ψ t )) with ψ = X θ \psi=X\theta ψ = X θ , and the summary statistics, the spike count and a nine-bin spike-triggered average, are computed exactly as in sbibm, which the baselines use. Binary spikes have no reparameterisation gradient. For Phase 1 we therefore differentiate through a straight-through Gumbel-softmax relaxation [30 ] of the spike draws (temperature 0.5). The forward pass returns the hard Gumbel-max sample with logits ( 0 , ψ t ) (0,\psi_t) ( 0 , ψ t ) , which is an exact Bernoulli draw, and the backward pass uses the gradient of the continuous relaxation, which is biased. All simulations, including those reused from Phase 1, are therefore exact, and the NRE is trained on the exact simulator.
The controlled-SDE drift u ϕ ( θ , t ) u_{\phi}(\theta,t) u ϕ ( θ , t ) and the corrector head are Fourier-feature MLPs with sinusoidal time embedding and GELU activations, 6 × 128 6\times128 6 × 128 (depth × \times × width) in both phases. The NRE classifier projects θ \theta θ and x x x separately to 64 features each, concatenates them and applies a 6 × 128 6\times128 6 × 128 MLP with SiLU activations. Both phases integrate the controlled SDE on [ 0 , 1 ] [0,1] [ 0 , 1 ] with 128 Euler–Maruyama steps and an EDM-style noise schedule [73 ] , with ρ = 5 \rho=5 ρ = 5 in Phase 1 and ρ = 7 \rho=7 ρ = 7 in Phase 2.
Phase 1 fits a controlled SDE that approximately samples the surrogate posterior under the energy-distance surrogate of Section C.2, with a fixed length scale ϵ = 0.914 \epsilon=0.914 ϵ = 0.914 and K = 2 K=2 K = 2 replicates of the differentiable simulator per parameter. Because the posterior is unimodal, no well-tempered bias is used in either phase. Phase 2 runs R = 2 R=2 R = 2 rounds. The first NRE is trained on 25% of the budget, the Phase 1 simulations (topped up with draws from the Phase 1 sampler at N s i m = 10 5 N_{\mathrm{sim}}=10^{5} N sim = 1 0 5 ), and the remaining 75% is simulated in the second round from the round-1 sampler. Each round refines the warm-started controlled SDE against E r ( θ ) = − log p ( θ ) − log r ψ r ( θ , x o ) E_r(\theta)=-\log p(\theta)-\log r_{\psi_r}(\theta,x_o) E r ( θ ) = − log p ( θ ) − log r ψ r ( θ , x o ) . The reference noise scale σ max \sigma_{\max} σ m a x grows with the prior width. All hyperparameters are listed in Sec. G.
Lotka–Volterra
Lotka–Volterra is the classic predator–prey model and a standard SBI benchmark task [18 ] . Four rate parameters θ = ( α , β , γ , δ ) \theta= (\alpha, \beta, \gamma, \delta) θ = ( α , β , γ , δ ) govern the oscillating populations of a prey and a predator species, and the goal is to infer them from a short, noisy record of both populations.
The simulator integrates u ˙ = f ( u , θ ) \dot{u} = f(u, \theta) u ˙ = f ( u , θ ) , f ( u , θ ) = ( α u 1 − β u 1 u 2 , − γ u 2 + δ u 1 u 2 ) f(u, \theta) = (\alpha u_1 - \beta u_1u_2, -\gamma u_2 + \delta u_1u_2) f ( u , θ ) = ( α u 1 − β u 1 u 2 , − γ u 2 + δ u 1 u 2 ) , from u ( 0 ) = ( 30 , 1 ) u(0) = (30, 1) u ( 0 ) = ( 30 , 1 ) and records both species at K = 10 K = 10 K = 10 times t k = k Δ t_k = k\Delta t k = k Δ , Δ = 2.1 \Delta= 2.1 Δ = 2.1 , with multiplicative noise x k = u ( t k ) ⊙ exp ( σ ξ k ) x_k = u(t_k) \odot\exp(\sigma\xi_k) x k = u ( t k ) ⊙ exp ( σ ξ k ) , ξ k ∼ N ( 0 , I 2 ) \xi_k \sim\mathcal{N}(0, I_2) ξ k ∼ N ( 0 , I 2 ) , σ = 0.1 \sigma= 0.1 σ = 0.1 , giving x ∈ R 2 K x \in\mathbb{R}^{2K} x ∈ R 2 K . A differentiable fourth-order Runge–Kutta (RK4) scheme (step 0.05) provides ∇ θ x \nabla_\theta x ∇ θ x by backpropagating through the unrolled RK4 steps, rather than by solving an adjoint ODE. sbibm instead solves the same ODE with Julia’s adaptive Tsit5 solver, which the baselines use and which generated the observations for the references. Our PyTorch re-implementation keeps the same initial state, observation times, output clipping to [ 10 − 10 , 10 4 ] [10^{-10}, 10^4] [ 1 0 − 10 , 1 0 4 ] and noise model. To check the fixed step, we compared noise-free outputs with an RK4 solve at step 0.001, at the true parameters and 200 reference posterior draws of every observation. The largest deviation is 2 × 10 − 4 2 \times10^{-4} 2 × 1 0 − 4 in log x \log x log x , three orders of magnitude below the observation noise σ = 0.1 \sigma= 0.1 σ = 0.1 . For rates far in the prior tails the solve can overflow, so populations are clamped to [ 10 − 10 , 10 6 ] [10^{-10}, 10^6] [ 1 0 − 10 , 1 0 6 ] after every step, simulator gradients that are non-finite for a sample are set to zero and the rest are capped at norm 10 12 10^{12} 1 0 12 . The prior term of the energy is differentiated separately, so such draws are still pulled back towards the support. RSBI samples ϕ = log θ \phi= \log\theta ϕ = log θ , under which the LogNormal prior becomes N ( μ 0 , s 2 I ) \mathcal{N}(\mu_0, s^2 I) N ( μ 0 , s 2 I ) , with s = 0.5 s = 0.5 s = 0.5 , 1 and 2 for the DEFAULT, MEDIUM and LARGE priors. This is the same reparameterisation SNVI applies through its automatic support transform. Samples are mapped back with θ = e ϕ \theta= e^\phi θ = e ϕ and evaluated in θ \theta θ -space against the same reference posteriors as all baselines.
The Phase 1 surrogate is the Gaussian negative log-likelihood with adaptive variance (Sec. C.1) at temperature T = 1 T = 1 T = 1 . Because each feature is scaled by its own variance, no additional feature standardisation (such as the per-feature z-scoring of the sbi toolkit, 53) is used.
The controlled-SDE drift u ϕ ( ϕ , t ) u_\phi(\phi, t) u ϕ ( ϕ , t ) and the corrector head are Fourier-feature MLPs with sinusoidal time embedding, 6 × 128 6 \times128 6 × 128 (depth × \times × width) with GELU activations in both phases. The NRE classifier projects ϕ \phi ϕ and x x x separately to 64 features each, concatenates them and applies a 6 × 128 6 \times128 6 × 128 MLP with SiLU activations, on inputs standardised by the mean and standard deviation of the first round’s training set. Both phases integrate the controlled SDE on [ 0 , 1 ] [0, 1] [ 0 , 1 ] with an EDM-style noise schedule [73 ] , using 64 Euler–Maruyama steps in Phase 1 and 128 in Phase 2, with ρ = 5 \rho= 5 ρ = 5 in Phase 1 (ρ = 5 \rho= 5 ρ = 5 in Phase 2 at N s i m = 10 5 N_{\mathrm{sim}} = 10^5 N sim = 1 0 5 and 7 otherwise). The reference noise scale is σ max = 0.3 \sigma_{\max} = 0.3 σ m a x = 0.3 in Phase 1 (0.5 for the wider priors at 10 5 10^5 1 0 5 ) and σ max = 1 \sigma_{\max} = 1 σ m a x = 1 in Phase 2, where it acts in the whitened coordinates of Appendix § 11 .§ 7 . Phase 1 uses two simulator replicates per parameter (three at 10 5 10^5 1 0 5 ) and energy gradients clipped at norm 256.
At N s i m = 10 3 N_{\mathrm{sim}} = 10^3 N sim = 1 0 3 and 10 4 10^4 1 0 4 , Phase 1 receives 70% of the budget and Phase 2 runs a single round (R = 1 R = 1 R = 1 ). Its NRE is trained on the Phase 1 simulations drawn after the horizon and shooting curriculum has ended, so it never sees masked or shooting-based data, topped up to the full budget with draws from the Phase 1 sampler. At 10 5 10^5 1 0 5 , Phase 1 is capped at 5 × 10 4 5 \times10^4 5 × 1 0 4 simulations, the first NRE is seeded with its post-curriculum simulations, and two sequential rounds of 2.5 × 10 4 2.5 \times10^4 2.5 × 1 0 4 simulations follow (R = 3 R = 3 R = 3 ). For the LARGE prior at 10 5 10^5 1 0 5 , Phase 1 instead receives 8 × 10 4 8 \times10^4 8 × 1 0 4 simulations and a single round (R = 1 R = 1 R = 1 ) is trained on its post-curriculum simulations topped up to the full budget. At every budget, Phase 2 is refined against the truncated ratio with k = 3 k = 3 k = 3 and q = 0.9 q = 0.9 q = 0.9 , and k = 2 k = 2 k = 2 and q = 0.9 q = 0.9 q = 0.9 for the LARGE prior at 10 5 10^5 1 0 5 , without a well-tempered bias since the posterior is unimodal. All hyperparameters are listed in Sec. G.
Surrogate likelihood choice Away from the posterior, a parameter error changes the period of the predator–prey cycle, so the phase error accumulates and late observations fall at essentially arbitrary points of a mismatched cycle. Any discrepancy on the full trajectory then becomes a rippled plateau in ϕ \phi ϕ whose gradient carries little information about the direction of the posterior. We therefore use two modifications of the Phase 1 surrogate. Neither changes the data, the prior, the reference posterior or the evaluation metric. (i) Horizon schedule. For a horizon H ≤ K H \le K H ≤ K , the later features of each simulation are replaced by those of the observation, [ Π H ( x ) ] k = x k [\Pi_H(x)]_k = x_k [ Π H ( x ) ] k = x k for k < H k < H k < H and x k o x_k^o x k o otherwise, so they contribute exactly zero to the discrepancy while the data dimension is unchanged. Over a short prefix the phase error cannot wrap around the cycle, so E E E is close to convex over a much larger region. With ν \nu ν the fraction of the Phase 1 budget consumed, H = 3 H = 3 H = 3 for ν < 0.3 \nu< 0.3 ν < 0.3 , H = 6 H = 6 H = 6 for 0.3 ≤ ν < 0.6 0.3 \le\nu< 0.6 0.3 ≤ ν < 0.6 and H = K H = K H = K afterwards (at N s i m = 10 5 N_{\mathrm{sim}} = 10^5 N sim = 1 0 5 the switches occur at fixed counts of 1.5 × 10 4 1.5 \times10^4 1.5 × 1 0 4 and 3 × 10 4 3 \times10^{4} 3 × 1 0 4 simulations), so the end of Phase 1 targets the full trajectory. (ii) Multiple shooting. With Φ Δ ( u , θ ) \Phi_{\Delta}(u,\theta) Φ Δ ( u , θ ) the RK4 flow over one observation interval, each observation is predicted from the observed state one interval earlier, x k s h = Φ Δ ( x k − 1 o , θ ) ⊙ e σ ξ k x_k^{\mathrm{sh}}=\Phi_{\Delta}(x_{k-1}^{o},\theta)\odot e^{\sigma\xi_k} x k sh = Φ Δ ( x k − 1 o , θ ) ⊙ e σ ξ k for k ≥ 1 k\ge1 k ≥ 1 (x 0 s h = u ( 0 ) ⊙ e σ ξ 0 x_0^{\mathrm{sh}}=u(0)\odot e^{\sigma\xi_0} x 0 sh = u ( 0 ) ⊙ e σ ξ 0 ). Phase errors cannot accumulate beyond one interval, and each segment needs 42 RK4 steps instead of up to 378. Conditioning on noisy states shifts the minimiser slightly (an errors-in-variables effect), so shooting is used only for ν < 0.5 \nu<0.5 ν < 0.5 , and the remaining bias is removed by the ratio-corrected Phase 2.
Both of the above are classical remedies for long-horizon gradients in differential-equation models, whose sensitivity to θ \theta θ grows polynomially for oscillatory and exponentially for chaotic dynamics [74 ] . Growing the fitted window is a continuation (curriculum) strategy [75 ] . Multiple shooting splits the trajectory into short segments started from estimated or observed states [76 , 77 ] and was recently shown to rescue neural ODE fitting on oscillatory data [78 ] . Smoothing-based estimators avoid long-horizon integration altogether [79 ] . Both modifications apply only to the Phase 1 surrogate, the energy that steers the variational sampler towards the posterior region before Phase 2 corrects it, and they cost no simulations beyond those Phase 1 already uses. They cannot be applied in the same way to SNPE or SNVI. These methods train a density or ratio estimator on a corpus of simulated pairs ( θ , x ) (\theta,x) ( θ , x ) , and that estimator is their final posterior. Masking the late observations of the corpus, or replacing its simulations with shooting segments, would change the conditional they estimate to p ( θ ∣ x < H ) p(\theta\mid x_{<H}) p ( θ ∣ x < H ) or to that of a different simulator, with no later phase to correct it. A horizon curriculum could at most supply an intermediate proposal, at the cost of training additional estimators on part of the budget, which neither method does. In RSBI, by contrast, the NRE and the Phase 2 target only ever see full, unmodified simulations. The horizon schedule needs only black-box access to θ ↦ x \theta\mapsto x θ ↦ x . Multiple shooting additionally assumes that the simulator can be restarted from an arbitrary state and that the full state is observed, both of which hold for Lotka–Volterra. It uses the simulator’s latent process in the spirit of Brehmer et al. [29 ] , the same white-box access RSBI already relies on through ∇ θ s ( θ ) \nabla_{\theta}s(\theta) ∇ θ s ( θ ) . As a rule, if the pilot gradient field in Sec. C.5 is incoherent away from the posterior, use the horizon schedule. If the state is fully observed and restartable, multiple shooting is a further option. If the field is already coherent, keep the full-trajectory surrogate.
Reference posteriors
We require a reference posterior p ( θ ∣ x o ) p(\theta\mid x_o) p ( θ ∣ x o ) for every (task, scenario, observation) triple in the benchmark grid, since C2ST in Section 5 is computed against it. The sampler used to obtain these samples differs by task; we always draw N r e f = 10 4 N_{\mathrm{ref}}=10^{4} N ref = 1 0 4 samples per observation and reuse them across method runs.
For Two Moons the likelihood is fully analytic on the polar variables recovered by sbibm’s TwoMoons._likelihood [18 ] . The simulator can be therefore inverted in closed form. sbibm’s reference sampler draws the simulator noise, maps it to the parameter that reproduces x o x_o x o (choosing between the two branches at random), and rejects only draws outside the prior box. Each prior scenario is redrawn independently with its own box. For eight of the ten observations the posterior lies inside [ − 1 , 1 ] 2 [-1,1]^2 [ − 1 , 1 ] 2 , so all scenarios share the same reference. For observations 5 and 10 the default box truncates one moon, removing about 57% and 12% of the wide-prior posterior mass, and the wider scenarios recover the full moon.
For SLCP the analytic likelihood induces a four-mode posterior across which Markov chains mix poorly, so we use sbibm’s shipped SIR+NSF reference sampler for all three scenarios (DEFAULT, WIDE5X, WIDE10X). On the 20×-larger WIDE10X box this sampler fails for observations 4 and 10: the posterior occupies a tiny fraction of the box, the spline-flow proposal misses parts of it, and rejection sampling stalls at an acceptance rate of about 10 − 6 10^{-6} 1 0 − 6 . For these two cases we exploit the closed-form SLCP likelihood directly. Because it depends on θ 3 \theta_3 θ 3 and θ 4 \theta_4 θ 4 only through θ 3 2 \theta_3^2 θ 3 2 and θ 4 2 \theta_4^2 θ 4 2 , the posterior is exactly symmetric under sign flips of these two coordinates, so we sample the folded posterior ( θ 3 , θ 4 ≥ 0 ) (\theta_3,\theta_4\ge0) ( θ 3 , θ 4 ≥ 0 ) and attach independent random signs afterwards, which gives the four modes exactly equal mass. A short ensemble-MCMC pilot run [42 ] on the folded posterior locates the posterior mass; a mixture of Student-t t t distributions fitted to it, plus a 5% uniform component on the box, serves as an importance proposal against the exact posterior, from which we resample 10 4 10^{4} 1 0 4 points. As a check, the same procedure applied to observation 4 under the DEFAULT prior reproduces the sbibm reference (C2ST = 0.50).
For gw_ringdown the whitened-strain log-likelihood is the analytic Gaussian − 1 2 ∥ L − 1 ( x o − s ( θ ) ) ∥ 2 2 + -\frac{1}{2}\lVert L^{-1}(x_o-s(\theta))\rVert_2^2+ − 2 1 ∥ L − 1 ( x o − s ( θ )) ∥ 2 2 + const, but the posterior has pronounced structure—periodic identifications on the phase dimensions and a near-degeneracy between A 220 A_{220} A 220 and A 221 A_{221} A 221 at low signal-to-noise—so we sample via affine-invariant ensemble MCMC with emcee [42 ] : 256 walkers initialized in a 0.01-ball around the injected truth, 10 5 10^5 1 0 5 steps per walker, 10 3 10^3 1 0 3 steps of burn-in, thinning factor 2, and an equal mixture of differential-evolution and stretch moves. Convergence is diagnosed via per-chain effective sample size (ESS ≳ 100 \gtrsim100 ≳ 100 per parameter) and trace-plot inspection, rerunning with longer chains when either flag fires.
For Bernoulli GLM, we use the reference-posterior sampler that ships with sbibm [18 ] unchanged, and replace only the prior with the scenario prior (Table 21 ). The scenarios scale the precision matrix of the sbibm smoothness prior, so the prior remains Gaussian and the sbibm sampler applies directly. That sampler is an exact Gibbs sampler based on Pólya–Gamma data augmentation [80 ] . It is initialised at the true parameters, discards 25,000 warm-up iterations and keeps every 25th draw, giving 10 4 10^4 1 0 4 samples.
For Lotka–Volterra, the default-prior reference is the sbibm reference posterior, obtained with its ODE solver and its log-normal observation model (noise scale 0.1) as the likelihood. A slice-sampling MCMC chain is started at the true parameters and run for 10 4 10^4 1 0 4 warm-up steps. A neural spline flow is then fitted to its samples and mixed with the prior (weight 0.1) to form a proposal, and rejection sampling from this proposal yields 10 4 10^4 1 0 4 exact posterior samples. The wider scenarios only widen the scale of the log-normal prior, with its location unchanged (Table 21 ). Their references are obtained by importance-resampling the default reference with weights p s ( θ ) / p 0.5 ( θ ) p_s(\theta)/p_{0.5}(\theta) p s ( θ ) / p 0.5 ( θ ) , the ratio of the scenario prior to the default prior, drawing 10 4 10^4 1 0 4 samples with replacement. Because every observation’s posterior lies well inside the default prior, the weights are nearly uniform. The effective sample size is at least 86% of N N N for the medium prior and 78% for the large prior across the ten observations, and the resampled sets contain about 6,000 to 6,400 distinct draws.
Prior scenarios and volume ratios
For each task we consider a default prior and two wider ones (one for gw_ringdown): uniform boxes for Two Moons, SLCP and gw_ringdown, and Gaussian priors with inflated covariance for Bernoulli GLM and Lotka–Volterra (Table 21 ). For each (task, scenario, obs) triple we estimate log 10 R p \log_{10} \mathcal{R}_p log 10 R p from the reference samples using a per-dimension 95%-marginal box estimator and a sample-covariance ellipsoid estimator (see Appendix § 16 ). For Gaussian priors the prior volume is measured with the same two estimators, from its covariance. The marginal estimator is the one we report throughout the main text, including the legends of the C2ST-versus-budget figures, because it is insensitive to multimodality. The ellipsoid estimator, which accounts for correlations but is inflated when the posterior has several modes (as for SLCP), is given in the table for comparison only. Across tasks the scenarios span marginal log 10 R p \log_{10} \mathcal{R}_p log 10 R p from about 0.9 to 12.
The gw_ringdown ratio changes little between the two scenarios (2.62 and 2.84), even though the wide box has about 1.4 orders of magnitude more prior volume, because the posterior widens with the prior. The overtone mode is only weakly constrained by single-detector data. Its phase ϕ 221 \phi_{221} ϕ 221 is essentially unconstrained, and the 95% width of its amplitude A 221 A_{221} A 221 grows from about 4.8 to 14, nearly filling the wider amplitude range.
Baselines
In this work we compare RSBI against two established neural-SBI methods. SNPE is configured as in sbibm [18 ] : a neural spline flow (nsf) density estimator with 50 hidden features, but with R = 2 R = 2 R = 2 with 50% of the budget allocated per round, and the sbibm training schedule.
SNVI+fKL is configured as in Glöckler et al. [7 ] : a resnet ratio estimator paired with an autoregressive spline flow proposal, but with R = 5 R = 5 R = 5 sequential rounds with 256 variational particles per round, and the β \beta β -divergence schedule released with the original implementation.
Software versions. All baselines of the main benchmark run on sbibm 1.1.1 (https://github.com/sbi-benchmark/sbibm), extended with our prior scenarios and the differentiable gravitational-wave task. SNPE runs on the sbi 0.16.0 fork released with the SNVI paper, and SNVI+fKL runs on that fork together with its sbivibm runners, both from the original implementation (https://github.com/mackelab/snvi_repo). These run under Python 3.10 and PyTorch 2.11. The Phase 2 ablations of Appendix § 12 .§ 3 use the current sbi release, 0.26.1 (https://github.com/sbi-dev/sbi), under Python 3.12 and PyTorch 2.10.
The careful reader will notice that we use fewer sequential rounds for SNPE and SNVI+fKL than mentioned in Refs. [7 , 18 ] . Indeed, the results in Figure 4 corresponding to the sbibm default slightly differ from these publications. This was done due to computational limitations. We also restrict the Phase-2 sequential posterior refinement of RSBI to a maximum of two rounds (matching SNPE) except for Lotka-Volterra for which we use three - in this case we checked the SNPE method with 3 rounds, yielding negligible difference to those recorded in the main results in § 7.2 . Furthermore, for SNPE and SNVI we use the default training procedures of the sbi package. The neural density estimator (SNPE) and the classifier-based ratio estimator underlying SNVI are trained with Adam, holding out 10% of the simulations for validation and stopping once the validation loss has not improved for 20 consecutive epochs; the weights with the best validation loss are kept. In addition, each round of SNVI fits a normalizing-flow variational posterior to the learned ratio by minimizing the forward KL divergence for at most 1,000 iterations, stopping early once the average slope of the running-mean loss over the last 50 iterations falls below 5 × 10 − 5 5 \times10^{-5} 5 × 1 0 − 5 in absolute value. We do note however that the authors of the SNVI paper mention in the appendix, that the benchmark convergence criteria were "chosen conservative to avoid early stopping". This could indicate using as many as 1000 iterations in each VI stage, therefore, differences with the published scores could potentially be a result of this.
Lastly, both the SNVI release and sbi 0.26.1 estimate the forward KL with self-normalised importance weights w i ∝ exp ( log p ~ ( θ i ) − log q ( θ i ) ) w_i \propto\exp(\log\tilde{p}(\theta_i) - \log q(\theta_i)) w i ∝ exp ( log p ~ ( θ i ) − log q ( θ i )) , where p ~ \tilde{p} p ~ is the prior times the learned ratio. They exponentiate the log-weights before normalising, so a single log-weight above about 88 overflows in single precision and the loss becomes undefined. On such a step the optimiser restores the last finite state of q q q and refits it to the prior before continuing. On wide priors, a sharp ratio estimator easily produces such log-weights, and the step can recur until the variational posterior is spread back over the prior. We found this in the Phase 2 ablation (Appendix § 12 .§ 3 ), where it affected 9 of 10 observations at 10 4 10^4 1 0 4 simulations. For that ablation we compute the weights in log space with a softmax, which gives the same estimator without overflow, and on a non-finite step we only restore the last finite state of q q q . For the main benchmark we use the released implementation unmodified. We checked that starting from the prior, the variational posterior is broad and so the log-weights stay far below the overflow threshold. Rerunning three observations at 10 4 10^4 1 0 4 simulations on the widest Two Moons prior with the stabilised weights fix gave no non-finite steps, and recovered essentially the same C2ST for the released implementation that was used to generate the main results of this paper.
Volume estimator for R p R_p R p
For each task and prior scenario we estimate the posterior support volume non-parametrically from the N r e f = 10,000 N_{\mathrm{ref}} = 10{,}000 N ref = 10 , 000 reference samples { θ ( n ) } n = 1 N r e f ⊂ R d θ \{\theta^{(n)}\}_{n=1}^{N_{\mathrm{ref}}} \subset\mathbb{R}^{d_\theta} { θ ( n ) } n = 1 N ref ⊂ R d θ . Writing μ ^ \hat{\mu} μ ^ and Σ ^ \hat{\Sigma} Σ ^ for the sample mean and covariance, the unit-Mahalanobis ellipsoid E = { θ : ( θ − μ ^ ) ⊤ Σ ^ − 1 ( θ − μ ^ ) ≤ 1 } \mathcal{E} = \{\theta: (\theta-\hat{\mu})^\top\hat{\Sigma}^{-1}(\theta-\hat{\mu}) \le1\} E = { θ : ( θ − μ ^ ) ⊤ Σ ^ − 1 ( θ − μ ^ ) ≤ 1 } has Lebesgue volume
V e l l = c d θ det Σ ^ , c d θ = π d θ / 2 Γ ( d θ 2 + 1 ) . (25) V_{\mathrm{ell}} = c_{d_\theta}\sqrt{\det\hat{\Sigma}}, \qquad c_{d_\theta} = \frac{\pi^{d_\theta/2}}{\Gamma\left(\frac{d_\theta}{2}+1\right)}. \tag*{(25)} V ell = c d θ det Σ ^ , c d θ = Γ ( 2 d θ + 1 ) π d θ /2 . (25)
where c d θ c_{d_\theta} c d θ is the volume of the unit d θ d_\theta d θ -ball and det Σ ^ \sqrt{\det\hat{\Sigma}} det Σ ^ maps it to the covariance ellipsoid. Working in log-space for numerical stability,
log V e l l = 1 2 log det Σ ^ + d θ 2 log π − log Γ ( d θ 2 + 1 ) , (26) \log V_{\mathrm{ell}} = \frac{1}{2}\log\det\hat{\Sigma} + \frac{d_\theta}{2}\log\pi- \log\Gamma\left(\frac{d_\theta}{2}+1\right), \tag*{(26)} log V ell = 2 1 log det Σ ^ + 2 d θ log π − log Γ ( 2 d θ + 1 ) , (26)
with log det Σ ^ \log\det\hat{\Sigma} log det Σ ^ computed via slogdet. For a rectangular prior with uncorrelated distributions, θ ∼ U ( ∏ i [ a i , b i ] ) \theta\sim\mathcal{U}\left(\prod_i[a_i,b_i]\right) θ ∼ U ( ∏ i [ a i , b i ] ) , the ellipsoid log-ratio is
log 10 R p e l l = 1 ln 10 ( log V p r i o r − log V e l l ) , V p r i o r = ∏ i = 1 d θ ( b i − a i ) . (27) \log_{10}R_p^{\mathrm{ell}} = \frac{1}{\ln10}\left(\log V_{\mathrm{prior}}-\log V_{\mathrm{ell}}\right), \qquad V_{\mathrm{prior}} = \prod_{i=1}^{d_\theta}(b_i-a_i). \tag*{(27)} log 10 R p ell = ln 10 1 ( log V prior − log V ell ) , V prior = i = 1 ∏ d θ ( b i − a i ) . (27)
The ellipsoid is the one-σ \sigma σ contour of the best-fit Gaussian to the reference samples and therefore captures elongation and orientation that the per-dimension marginal-box estimator V m a r g . = ∏ i ( q 0.975 ( i ) − q 0.025 ( i ) ) V_{\mathrm{marg.}} = \prod_i(q^{(i)}_{0.975}-q^{(i)}_{0.025}) V marg. = ∏ i ( q 0.975 ( i ) − q 0.025 ( i ) ) ignores. For uniform priors it is also much smaller than the marginal box, which spans about ± 2 σ \pm2\sigma ± 2 σ per dimension, so log 10 R p e l l \log_{10}R_p^{\mathrm{ell}} log 10 R p ell exceeds its marginal counterpart in Table 21 . For Gaussian priors, where prior and posterior are measured with the same estimator, these constants cancel and the two log-ratios differ by 1 2 ( log det C p r i o r − log det C ^ ) \frac{1}{2}(\log\det C_{\mathrm{prior}}-\log\det\hat{C}) 2 1 ( log det C prior − log det C ^ ) , with C C C the correlation matrices. This is positive for the isotropic Lotka–Volterra prior and negative for the strongly correlated smoothness prior of the Bernoulli GLM, whose ellipsoid ratio is therefore the smaller of the two. If Σ ^ \hat{\Sigma} Σ ^ were rank-deficient we would floor its eigenvalues at 10 − 300 10^{-300} 1 0 − 300 before taking the log-determinant. This was not needed for any cell in Table 21 .
For the Bernoulli GLM, whose prior is Gaussian, θ ∼ N ( 0 , Σ p r i o r ) \theta\sim\mathcal{N}(0,\Sigma_{\mathrm{prior}}) θ ∼ N ( 0 , Σ prior ) with Σ p r i o r = ( m B − 1 ) − 1 ( B − 1 \Sigma_{\mathrm{prior}} = (mB^{-1})^{-1}(B^{-1} Σ prior = ( m B − 1 ) − 1 ( B − 1 the smoothness precision of Lueckmann et al. [18 ] and m m m the scenario’s precision multiplier), the prior has no finite support volume, so we measure it with the same estimator as the posterior. The prior’s unit-Mahalanobis ellipsoid has volume c d θ det Σ p r i o r c_{d_\theta}\sqrt{\det\Sigma_{\mathrm{prior}}} c d θ det Σ prior , and the ball constant cancels in the ratio,
log 10 R p e l l = 1 2 ln 10 ( log det Σ p r i o r − log det Σ ^ ) , (28) \log_{10}R_p^{\mathrm{ell}} = \frac{1}{2\ln10}\left(\log\det\Sigma_{\mathrm{prior}}-\log\det\hat{\Sigma}\right), \tag*{(28)} log 10 R p ell = 2 ln 10 1 ( log det Σ prior − log det Σ ^ ) , (28)
while the marginal-box counterpart replaces the box widths b i − a i b_i-a_i b i − a i by the prior’s central 95% widths 2 z 0.975 σ i 2z_{0.975}\sigma_i 2 z 0.975 σ i , with σ i 2 = ( Σ p r i o r ) i i \sigma_i^2=(\Sigma_{\mathrm{prior}})_{ii} σ i 2 = ( Σ prior ) ii . The Lotka–Volterra prior is log-normal, log θ ∼ N ( μ 0 , s 2 I ) \log\theta\sim\mathcal{N}(\mu_0,s^2I) log θ ∼ N ( μ 0 , s 2 I ) , so we apply the same Gaussian formulae in log θ \log\theta log θ , the coordinates in which the prior is Gaussian (and in which RSBI samples), with Σ p r i o r = s 2 I \Sigma_{\mathrm{prior}}=s^2I Σ prior = s 2 I and Σ ^ \hat{\Sigma} Σ ^ the covariance of the log-transformed reference samples.
RSBI Hyperparameters
In this section we report hyperparameters used throughout the paper’s various experiments.
Main Benchmarks
Tables 22–25 list the RSBI hyperparameters of every prior × \times × budget cell in Figure 4 . The prior scenarios are defined in Table 21 .
Table 22. RSBI hyperparameters for the Two Moons results in Figure 4 . One column per prior × \times × budget cell. A value spanning several columns applies to all of them, and — marks a setting that is not used in that cell.
Table 23. RSBI hyperparameters for the Lotka–Volterra results in Figure 4. One column per prior × \times × budget cell. A value spanning several columns applies to all of them, and — marks a setting that is not used in that cell. †the Phase-1 simulations are discarded and the first NRE round trains on fresh draws from the Phase-1 sampler.
Table 24. RSBI hyperparameters for the Bernoulli GLM results in Figure 4. One column per prior × \times × budget cell. A value spanning several columns applies to all of them, and — marks a setting that is not used in that cell.
Table 25. RSBI hyperparameters for the SLCP results in Figure 4 . One column per prior × \times × budget cell. A value spanning several columns applies to all of them, and — marks a setting that is not used in that cell.
gw_ringdown
Table 26 lists the RSBI hyperparameters of the gw_ringdown experiment (Table 1 and Figure 3 ) for the two prior scenarios of Table 21 . All ten observations of a scenario use the same configuration; the sampler runs on the prior box rescaled to [ 0 , 10 ] 6 [0, 10]^6 [ 0 , 10 ] 6 , so the schedule and well-tempered settings are given in those coordinates.
Table 26. RSBI hyperparameters for the gw_ringdown results in Table 1 and Figure 3 (N s i m = 10 5 N_{\mathrm{sim}} = 10^5 N sim = 1 0 5 ). The default and wide prior scenarios share every setting.