The work of Z. He was supported by the National Natural Science Foundation of China under Grant 12671500 and by the Guangdong Basic and Applied Basic Research Foundation under Grant 2025A1515011888.
Introduction
We consider Bayesian inverse problems, in which unknown model parameters are inferred from observed data. The Bayesian approach combines prior information with a likelihood model and quantifies uncertainty through the posterior distribution [1 ] . This formulation is used, for example, for parameter estimation in differential equations [2 ] and in population genetics [3 ] . We focus on likelihoods given by conditional expectations that are costly to evaluate, and aim to reduce the simulation cost of posterior sampling.
Let x ∈ R d x \in\mathbb{R}^{d} x ∈ R d denote the unknown parameter. The posterior density is
π ( x ) = π 0 ( x ) L ( x ) Z , Z = ∫ R d π 0 ( x ) L ( x ) d x , (1) \pi(x) = \frac{\pi_{0}(x)L(x)}{\mathcal{Z}}, \qquad\mathcal{Z} = \int_{\mathbb{R}^{d}} \pi_{0}(x)L(x)\,\mathrm{d}x, \tag*{(1)} π ( x ) = Z π 0 ( x ) L ( x ) , Z = ∫ R d π 0 ( x ) L ( x ) d x , (1)
where π 0 \pi_{0} π 0 is the prior density, L L L is the likelihood or a specified likelihood approximation, and 0 < Z < ∞ 0 < \mathcal{Z} < \infty 0 < Z < ∞ .
We consider targets for which
L ( x ) = E ξ ∣ x [ ψ ( x , ξ ) ] , ξ ∼ p ( ⋅ ∣ x ) , ψ ( x , ξ ) ≥ 0. (2) L(x) = \mathbb{E}_{\xi\mid x}[\psi(x,\xi)], \qquad\xi\sim p(\cdot\mid x), \qquad\psi(x,\xi) \ge0. \tag*{(2)} L ( x ) = E ξ ∣ x [ ψ ( x , ξ )] , ξ ∼ p ( ⋅ ∣ x ) , ψ ( x , ξ ) ≥ 0. (2)
Here ξ \xi ξ denotes simulator outputs or auxiliary variables, p ( ⋅ ∣ x ) p(\cdot\mid x) p ( ⋅ ∣ x ) is their conditional distribution given x x x , and ψ ( x , ξ ) \psi(x,\xi) ψ ( x , ξ ) is a computable nonnegative weight. We assume that ξ ∼ p ( ⋅ ∣ x ) \xi\sim p(\cdot\mid x) ξ ∼ p ( ⋅ ∣ x ) can be simulated, but not that the density p ( ξ ∣ x ) p(\xi\mid x) p ( ξ ∣ x ) can be evaluated.
Two examples illustrate the representation in (I.2). In approximate Bayesian computation (ABC), simulated data receive larger weights when they agree more closely with the observations; averaging these weights gives an approximate likelihood [3 , 4 ] . In latent-variable models, averaging the conditional likelihood over the unobserved variables gives the exact likelihood, which can therefore be estimated without bias by simulation [5 ] . Section II describes both constructions in detail.
Our goal is to generate approximately unweighted samples from π \pi π when L L L cannot be evaluated directly; such samples give any posterior summary, such as E X ∼ π [ h ( X ) ] \mathbb{E}_{X\sim\pi}[h(X)] E X ∼ π [ h ( X )] , without reweighting. Representation (I.2) also permits importance-sampling estimates of posterior expectations, but these are weighted averages rather than posterior samples. Equally weighted samples can be obtained by resampling the weighted points, but they are then confined to those points, and when the importance weights degenerate, which typically happens in high dimensions or for concentrated posteriors, only a few distinct points remain [6 ] . On the other hand, estimating (I.2) at every parameter value visited by a sampler can require many costly simulations, so we seek accurate posterior samples from fewer simulator evaluations.
Most methods for intractable likelihoods are based on Markov chain or sequential Monte Carlo. ABC compares simulated and observed data through a kernel and is implemented by rejection sampling [3 ] , Markov chain Monte Carlo (MCMC) [7 ] , or sequential Monte Carlo [8 ] ; see Marin et al. [9 ] and Sisson et al. [4 ] for reviews. Synthetic likelihood replaces the likelihood of the summaries by a Gaussian fitted to simulations [10 ] , and pseudo-marginal MCMC replaces the likelihood by a nonnegative unbiased estimator while still targeting the exact posterior [5 , 11 ] . These MCMC-based methods require new simulations at every proposed parameter and produce correlated output, and the noise in the likelihood estimate must be tuned to avoid sticky chains [12 ] . Neural simulation-based inference instead trains a conditional density estimator on simulations [13 ] , at the cost of network design and training.
Diffusion-based samplers transport a simple reference distribution to the target in finite time, so independent trajectories can be run in parallel without burn-in [14 , 15 ] . Most of them learn the drift with neural networks. We build on Schrödinger–Föllmer sampling (SFS) [16 , 17 ] , a diffusion-based sampler whose drift is an explicit Gaussian expectation and can therefore be estimated by Monte Carlo without training.
SFS generates samples from a prescribed target by simulating the diffusion
d X t = b ( X t , t ) d t + d B t , X 0 = 0 , (3) \mathrm{d}X_t = b(X_t,t)\,\mathrm{d}t + \mathrm{d}B_t,\qquad X_0=0, \tag*{(3)} d X t = b ( X t , t ) d t + d B t , X 0 = 0 , (3)
where B t B_t B t is a standard d d d -dimensional Brownian motion. Let φ \varphi φ be the density of B 1 ∼ N ( 0 , I d ) B_1 \sim\mathcal{N}(0,I_d) B 1 ∼ N ( 0 , I d ) , which serves as a Gaussian reference, set f = π / φ f=\pi/\varphi f = π / φ , and choose
b ( x , t ) = E Z [ f ( Z ) ( Z − x ) ] ( 1 − t ) E Z [ f ( Z ) ] , Z ∼ N ( x , ( 1 − t ) I d ) , 0 ≤ t < 1. (4) b(x,t)=\frac{\mathbb{E}_Z[f(Z)(Z-x)]}{(1-t)\mathbb{E}_Z[f(Z)]},\qquad Z\sim\mathcal{N}(x,(1-t)I_d),\qquad0\leq t<1. \tag*{(4)} b ( x , t ) = ( 1 − t ) E Z [ f ( Z )] E Z [ f ( Z ) ( Z − x )] , Z ∼ N ( x , ( 1 − t ) I d ) , 0 ≤ t < 1. (4)
Under suitable regularity and integrability conditions, this choice gives X 1 ∼ π X_1\sim\pi X 1 ∼ π [16 , 17 ] ; the resulting process solves the Schrödinger bridge problem between the point mass at the origin and π \pi π [16 , 18 ] . Since f ( Z ) = π 0 ( Z ) L ( Z ) / ( Z φ ( Z ) ) f(Z)=\pi_0(Z)L(Z)/(\mathcal{Z}\varphi(Z)) f ( Z ) = π 0 ( Z ) L ( Z ) / ( Z φ ( Z )) , the unknown normalizing constant Z \mathcal{Z} Z cancels in the ratio. However, f f f still involves L L L , which cannot be evaluated.
We combine (I.2) with iterated expectation to write the drift as a ratio of joint expectations over a Gaussian variable Z Z Z and an auxiliary variable ξ ∼ p ( ⋅ ∣ Z ) \xi\sim p(\cdot\mid Z) ξ ∼ p ( ⋅ ∣ Z ) . The drift can then be estimated directly from simulated pairs ( Z , ξ ) (Z,\xi) ( Z , ξ ) . § 3 gives the derivation and the two resulting samplers.
Several works study the construction and numerical approximation of SFS. Huang et al. [16 ] introduced the sampler and analyzed its time discretization. Jiao et al. [17 ] established quadratic Wasserstein error bounds under smoothness and boundedness assumptions on the target density ratio, separating time-discretization and Monte Carlo (MC) errors. The MC estimator analyzed by Jiao et al. [17 ] writes the drift as E Z [ ∇ f ( Z ) ] / E Z [ f ( Z ) ] \mathbb{E}_Z[\nabla f(Z)]/\mathbb{E}_Z[f(Z)] E Z [ ∇ f ( Z )] / E Z [ f ( Z )] and thus requires the gradient of the density ratio. In (I.4), the derivative is instead moved onto the Gaussian kernel, so only values of f f f are needed; this is what later allows f f f to be replaced by simulator outputs. Their MC error bound therefore does not cover our estimator.
Drift approximation is also related to importance sampling with random weights. Self-normalized importance sampling estimates a ratio of integrals using a common set of samples; its consistency and asymptotic properties are discussed in Owen [19 Section 9.2] . Llorente et al. [20 ] studied noisy importance sampling and the effect of random target evaluations on proposal design. Riel and Bischoff [21 ] used importance sampling and particle ensembles to approximate gradients of log densities without target gradients. Importance sampling with mixture proposals was studied by Hesterberg [22 ] and Owen and Zhou [23 ] .
Quasi-Monte Carlo (QMC) and randomized QMC (RQMC) integration rules can improve on the MC error under suitable regularity assumptions [24 ] –[26 ] . For the SFS drift, however, the integrands depend on the current state and time, and reusing the same samples makes the subsequent states depend on them. An integration bound at a fixed state therefore does not by itself yield a bound on the terminal distribution.
The works above provide diffusion error estimates, importance sampling constructions, and integration rules, but they do not give a sampler for (I.2) that avoids estimating the likelihood at every parameter value. Our contributions are as follows.
Likelihood-free drift. Using iterated expectation, we write the SFS drift as a ratio of joint expectations over Gaussian variables and simulator outputs (III.11). The drift can thus be estimated from simulated pairs by MC or RQMC, without likelihood values or simulator derivatives.
Likelihood-free SFS. Estimating this drift with new samples at each time step gives likelihood-free SFS. Theorem IV.1 bounds its terminal quadratic Wasserstein distance W 2 W_2 W 2 error by O ( h + Λ h / m ) O\left(\sqrt{h}+\sqrt{\Lambda_h/m}\right) O ( h + Λ h / m ) , where h h h is the time step, m m m is the number of samples per step, and Λ h \Lambda_h Λ h , defined in (IV.3), accumulates the conditional mean-square drift-estimation error over the time grid.
Likelihood-free SFS with sample reuse. Importance sampling with a fixed proposal lets one set of M M M simulated pairs serve all time steps and trajectories, reducing the number of simulator calls from N K m NKm N K m to M M M ; in the Gaussian benchmark, 8192 calls replace about 10 7 10^7 1 0 7 . Theorem IV.2 proves that the output converges to the posterior in the quadratic Wasserstein distance W 2 W_2 W 2 under only a finite posterior second moment, and Theorem IV.3 gives a central limit theorem for the drift estimator that guides the choice of proposal.
This paper is organized as follows. Section II describes the problem setting and the likelihood representations. Section III introduces the two samplers. Section IV gives terminal error and convergence results for both, followed by the pointwise drift limit and the proposal-variance analysis. Section V reports numerical experiments, and Section VI concludes. Appendices A and B contain the proofs and supporting calculations.
Problem setting
Let x ∈ R d x \in\mathbb{R}^{d} x ∈ R d denote the unknown parameter and let y y y denote the observed data. Since y y y is fixed throughout, we suppress the dependence on y y y in the notation for the likelihood and the posterior. We consider the posterior density
π ( x ) = π 0 ( x ) L ( x ) Z , Z = ∫ R d π 0 ( u ) L ( u ) d u , (5) \pi(x) = \frac{\pi_{0}(x)L(x)}{\mathcal{Z}}, \qquad\mathcal{Z} = \int_{\mathbb{R}^{d}} \pi_{0}(u)L(u)\,\mathrm{d}u, \tag*{(5)} π ( x ) = Z π 0 ( x ) L ( x ) , Z = ∫ R d π 0 ( u ) L ( u ) d u , (5)
where π 0 \pi_{0} π 0 is a prior density and 0 < Z < ∞ 0 < \mathcal{Z} < \infty 0 < Z < ∞ . Throughout this section, L L L is a nonnegative function defining the target; it may be the exact likelihood or a specified approximation, such as an ABC likelihood.
We assume the representation
L ( x ) = E ξ ∣ x [ ψ ( x , ξ ) ] = ∫ Ξ ψ ( x , ξ ) p ( d ξ ∣ x ) , ψ ( x , ξ ) ≥ 0. (6) L(x) = \mathbb{E}_{\xi\mid x}[\psi(x,\xi)] = \int_{\Xi} \psi(x,\xi)\,p(\mathrm{d}\xi\mid x), \qquad\psi(x,\xi) \ge0. \tag*{(6)} L ( x ) = E ξ ∣ x [ ψ ( x , ξ )] = ∫ Ξ ψ ( x , ξ ) p ( d ξ ∣ x ) , ψ ( x , ξ ) ≥ 0. (6)
Here Ξ \Xi Ξ is the space of simulator outputs or auxiliary variables, p ( d ξ ∣ x ) p(\mathrm{d}\xi\mid x) p ( d ξ ∣ x ) is their conditional probability law, and ψ ( x , ξ ) \psi(x,\xi) ψ ( x , ξ ) is a measurable, computable weight; the expectation is assumed finite for all parameter values considered. When this law has a density with respect to a reference measure, we also write p ( ξ ∣ x ) p(\xi\mid x) p ( ξ ∣ x ) for that density. The samplers require draws from this law but never evaluate its density.
In ABC [3 ] , [27 ] , the model is a simulator: for given x x x , it generates synthetic data ξ ∼ p ( ⋅ ∣ x ) \xi\sim p(\mathord{\cdot} \mid x) ξ ∼ p ( ⋅ ∣ x ) in the same space Ξ \Xi Ξ as the observed data y y y , but the likelihood p ( y ∣ x ) p(y \mid x) p ( y ∣ x ) of the observed data cannot be evaluated. ABC therefore compares the simulated data ξ \xi ξ with y y y . Let S : Ξ → R r S : \Xi\to\mathbb{R}^{r} S : Ξ → R r be a summary map [28 ] , where r r r is the number of summary statistics. The vectors S ( ξ ) S(\xi) S ( ξ ) and S ( y ) S(y) S ( y ) may contain, for example, sample means or variances. We set
ψ ( x , ξ ) = K δ ( S ( ξ ) , S ( y ) ) , ξ ∼ p ( ⋅ ∣ x ) , (7) \psi(x,\xi) = K_{\delta}(S(\xi), S(y)), \qquad\xi\sim p(\mathord{\cdot} \mid x), \tag*{(7)} ψ ( x , ξ ) = K δ ( S ( ξ ) , S ( y )) , ξ ∼ p ( ⋅ ∣ x ) , (7)
where the nonnegative kernel K δ K_{\delta} K δ gives larger weights to closer matches, and its bandwidth δ > 0 \delta> 0 δ > 0 controls how much mismatch is tolerated. Equation (II.2) then defines the approximate likelihood L A B C ( x ) = E ξ ∣ x [ K δ ( S ( ξ ) , S ( y ) ) ] L_{\mathrm{ABC}}(x) = \mathbb{E}_{\xi\mid x}[K_{\delta}(S(\xi), S(y))] L ABC ( x ) = E ξ ∣ x [ K δ ( S ( ξ ) , S ( y ))] . The resulting ABC posterior, proportional to π 0 ( x ) L A B C ( x ) \pi_{0}(x)L_{\mathrm{ABC}}(x) π 0 ( x ) L ABC ( x ) , approximates the exact posterior, proportional to π 0 ( x ) p ( y ∣ x ) \pi_{0}(x)p(y \mid x) π 0 ( x ) p ( y ∣ x ) , and has two sources of error. First, it conditions on the summaries S ( y ) S(y) S ( y ) instead of the full data y y y , which loses information unless S S S is sufficient. Second, the kernel also gives weight to simulated summaries that are only close to S ( y ) S(y) S ( y ) ; under mild conditions this error vanishes as δ → 0 \delta\to0 δ → 0 .
A second example is a latent-variable model [5 ] , in which unobserved random variables ξ \xi ξ affect the data; for instance, repeated measurements on one individual may share an unobserved individual effect. If ξ \xi ξ were observed, the likelihood of y y y given ( x , ξ ) (x,\xi) ( x , ξ ) would be computable; we take this conditional likelihood as ψ ( x , ξ ) \psi(x,\xi) ψ ( x , ξ ) . Averaging it over the unobserved ξ ∼ p ( ⋅ ∣ x ) \xi\sim p(\mathord{\cdot} \mid x) ξ ∼ p ( ⋅ ∣ x ) gives the likelihood L ( x ) L(x) L ( x ) in (II.2), so here L L L is the exact likelihood, with no ABC approximation. More generally, ξ \xi ξ can be the random inputs of any nonnegative unbiased estimator of L ( x ) L(x) L ( x ) .
By contrast, for a model y = G ( x ) + ε y = G(x) + \varepsilon y = G ( x ) + ε with a deterministic forward map G G G and additive noise with density ρ \rho ρ , independent of x x x , the likelihood L ( x ) = ρ ( y − G ( x ) ) L(x) = \rho(y - G(x)) L ( x ) = ρ ( y − G ( x )) can be evaluated directly whenever G G G can, and no auxiliary simulation is needed.
A direct Monte Carlo approximation of (II.2) is
L ^ m ( x ) = 1 m ∑ i = 1 m ψ ( x , ξ i ) , ξ i ∼ i . i . d . p ( ⋅ ∣ x ) . (8) \widehat{L}_m(x) = \frac{1}{m}\sum_{i=1}^{m}\psi(x,\xi_i), \qquad\xi_i \overset{\mathrm{i.i.d.}}{\sim} p(\cdot\mid x). \tag*{(8)} L m ( x ) = m 1 i = 1 ∑ m ψ ( x , ξ i ) , ξ i ∼ i.i.d. p ( ⋅ ∣ x ) . (8)
For fixed x x x , this estimator is unbiased, E [ L ^ m ( x ) ∣ x ] = L ( x ) \mathbb{E}[\widehat{L}_m(x)\mid x] = L(x) E [ L m ( x ) ∣ x ] = L ( x ) . Its drawback is cost. Each likelihood value requires m m m simulations, and a posterior sampler needs the likelihood at many parameter values, for example at every proposed state of a Markov chain. The total number of simulations is therefore m m m times the number of likelihood evaluations, and m m m must be large enough to control the noise in each estimate.
Representation (II.2) also allows posterior expectations to be estimated without a sampler. Given a proposal density q q q , draw Z i ∼ q Z_i \sim q Z i ∼ q and ξ i ∼ p ( ⋅ ∣ Z i ) \xi_i \sim p(\cdot\mid Z_i) ξ i ∼ p ( ⋅ ∣ Z i ) , i = 1 , … , M i = 1,\ldots,M i = 1 , … , M , and set v i = π 0 ( Z i ) ψ ( Z i , ξ i ) / q ( Z i ) v_i = \pi_0(Z_i)\psi(Z_i,\xi_i)/q(Z_i) v i = π 0 ( Z i ) ψ ( Z i , ξ i ) / q ( Z i ) . Since E q [ v i g ( Z i ) ] = Z E π [ g ( X ) ] \mathbb{E}_q[v_i g(Z_i)] = Z\mathbb{E}_{\pi}[g(X)] E q [ v i g ( Z i )] = Z E π [ g ( X )] , the self-normalized estimate ∑ i v i g ( Z i ) / ∑ i v i \sum_i v_i g(Z_i)/\sum_i v_i ∑ i v i g ( Z i ) / ∑ i v i converges to E π [ g ( X ) ] \mathbb{E}_{\pi}[g(X)] E π [ g ( X )] [19 Chapter 9] . This yields weighted estimates of posterior expectations, not posterior samples. Resampling from the weighted empirical distribution ∑ i ( v i / ∑ j v j ) δ Z i \sum_i(v_i/\sum_j v_j)\delta_{Z_i} ∑ i ( v i / ∑ j v j ) δ Z i gives equally weighted samples, but only at the points Z 1 , … , Z M Z_1,\ldots,Z_M Z 1 , … , Z M , and the number of distinct points is governed by the effective sample size ( ∑ i v i ) 2 / ∑ i v i 2 (\sum_i v_i)^2/\sum_i v_i^2 ( ∑ i v i ) 2 / ∑ i v i 2 , which typically collapses as the dimension grows or as the posterior concentrates relative to q q q [6 ] .
The sampler in § 3 instead outputs unweighted samples and requires L L L only through averages over parameter values. By (II.2), such averages are joint expectations over the parameter and the simulator output, and the simulations can be generated once by importance sampling and reused throughout the sampler.
Likelihood-free SFS methods
Section III-A recalls SFS and its Euler–Maruyama discretization, in which the only quantity that cannot be computed directly is the drift. Section III-B writes the drift as a ratio of joint expectations over Gaussian variables and simulator outputs. Sections III-C and III-D give two estimators of this ratio: likelihood-free SFS draws new samples at each step, and likelihood-free SFS with sample reuse uses importance sampling so that the samples are generated once and reused across all time steps and trajectories.
Schrödinger–Föllmer sampling and its time discretization
Let
φ s ( z ) : = ( 2 π s ) − d / 2 exp ( − ∥ z ∥ 2 2 s ) , s > 0 , φ : = φ 1 , (9) \varphi_s(z) := (2\pi s)^{-d/2}\exp\left(-\frac{\lVert z\rVert^2}{2s}\right), \qquad s>0,\qquad\varphi:= \varphi_1, \tag*{(9)} φ s ( z ) := ( 2 π s ) − d /2 exp ( − 2 s ∥ z ∥ 2 ) , s > 0 , φ := φ 1 , (9)
denote the densities of N ( 0 , s I d ) \mathcal{N}(0,sI_d) N ( 0 , s I d ) , where I d I_d I d is the d × d d \times d d × d identity matrix. For functions g g g for which the integral below is finite, define
f = π φ , Q s g ( x ) = ∫ R d g ( z ) φ s ( z − x ) d z . (10) f=\frac{\pi}{\varphi}, \qquad Q_sg(x)=\int_{\mathbb{R}^d}g(z)\varphi_s(z-x)\,\mathrm{d}z. \tag*{(10)} f = φ π , Q s g ( x ) = ∫ R d g ( z ) φ s ( z − x ) d z . (10)
Thus f f f is the target density relative to the Gaussian reference φ \varphi φ , and Q s Q_s Q s is the heat semigroup. With B t B_t B t a standard d d d -dimensional Brownian motion, SFS [16 ] , [17 ] simulates
d X t = b ( X t , t ) d t + d B t , X 0 = 0 , b ( x , t ) = ∇ x log Q 1 − t f ( x ) , (11) \mathrm{d}X_t=b(X_t,t)\,\mathrm{d}t+\mathrm{d}B_t,\qquad X_0=0,\qquad b(x,t)=\nabla_x\log Q_{1-t}f(x), \tag*{(11)} d X t = b ( X t , t ) d t + d B t , X 0 = 0 , b ( x , t ) = ∇ x log Q 1 − t f ( x ) , (11)
for 0 ≤ t < 1 0\le t<1 0 ≤ t < 1 .
To see why X 1 ∼ π X_1\sim\pi X 1 ∼ π , let P 0 \mathbb{P}_0 P 0 be the law of a standard Brownian motion ( B t ) 0 ≤ t ≤ 1 (B_t)_{0\le t\le1} ( B t ) 0 ≤ t ≤ 1 with filtration ( F t ) (\mathcal{F}_t) ( F t ) , and define P ∗ \mathbb{P}_* P ∗ by
d P ∗ d P 0 = f ( B 1 ) . (12) \frac{\mathrm{d}\mathbb{P}_*}{\mathrm{d}\mathbb{P}_0}=f(B_1). \tag*{(12)} d P 0 d P ∗ = f ( B 1 ) . (12)
Since E P 0 [ f ( B 1 ) ] = ∫ f φ = ∫ π = 1 \mathbb{E}_{\mathbb{P}_0}[f(B_1)]=\int f\varphi=\int\pi=1 E P 0 [ f ( B 1 )] = ∫ f φ = ∫ π = 1 , P ∗ \mathbb{P}_* P ∗ is a probability measure, and under P ∗ \mathbb{P}_* P ∗ the endpoint B 1 B_1 B 1 has density f φ = π f\varphi=\pi f φ = π . By the Markov property, the density process is
M t = E P 0 [ f ( B 1 ) ∣ F t ] = h ( B t , t ) , h ( x , t ) = Q 1 − t f ( x ) , 0 ≤ t < 1. M_t=\mathbb{E}_{\mathbb{P}_0}[f(B_1)\mid\mathcal{F}_t]=h(B_t,t),\qquad h(x,t)=Q_{1-t}f(x),\qquad0\le t<1. M t = E P 0 [ f ( B 1 ) ∣ F t ] = h ( B t , t ) , h ( x , t ) = Q 1 − t f ( x ) , 0 ≤ t < 1.
The function h h h is positive and solves the backward heat equation ∂ t h + 1 2 Δ h = 0 \partial_t h+\frac{1}{2}\Delta h=0 ∂ t h + 2 1 Δ h = 0 , so Itô’s formula gives
d M t = ∇ x h ( B t , t ) ⊤ d B t = M t b ( B t , t ) ⊤ d B t , b = ∇ x log h . \mathrm{d}M_t=\nabla_xh(B_t,t)^\top\mathrm{d}B_t=M_tb(B_t,t)^\top\mathrm{d}B_t,\qquad b=\nabla_x\log h. d M t = ∇ x h ( B t , t ) ⊤ d B t = M t b ( B t , t ) ⊤ d B t , b = ∇ x log h .
By Girsanov’s theorem [29 Section 3.5] , B t − ∫ 0 t b ( B s , s ) d s B_t-\int_0^t b(B_s,s)\,\mathrm{d}s B t − ∫ 0 t b ( B s , s ) d s is a Brownian motion under P ∗ \mathbb{P}_* P ∗ ; that is, under P ∗ \mathbb{P}_* P ∗ the process B B B solves (III.3). Hence the solution of (III.3) has path law P ∗ \mathbb{P}_* P ∗ and, in particular, X 1 ∼ π X_1\sim\pi X 1 ∼ π . Making this rigorous up to t = 1 t=1 t = 1 requires regularity of f f f ; Appendix A-A does so for the ABC targets of Section IV-A.
The drift has an explicit form. If f ( z ) φ 1 − t ( z − x ) f(z)\varphi_{1-t}(z-x) f ( z ) φ 1 − t ( z − x ) and its x x x -gradient are dominated by an integrable function of z z z , locally uniformly in x x x , then differentiating the Gaussian kernel in Q 1 − t f ( x ) Q_{1-t}f(x) Q 1 − t f ( x ) with respect to x x x gives
b ( x , t ) = E Z [ f ( Z ) ( Z − x ) ] ( 1 − t ) E Z [ f ( Z ) ] , Z ∼ N ( x , ( 1 − t ) I d ) . (13) b(x,t)=\frac{\mathbb{E}_Z[f(Z)(Z-x)]}{(1-t)\mathbb{E}_Z[f(Z)]},\qquad Z\sim\mathcal{N}(x,(1-t)I_d). \tag*{(13)} b ( x , t ) = ( 1 − t ) E Z [ f ( Z )] E Z [ f ( Z ) ( Z − x )] , Z ∼ N ( x , ( 1 − t ) I d ) . (13)
To simulate (III.3), choose an integer K ≥ 1 K\ge1 K ≥ 1 and a time grid
0 = t 0 < ⋯ < t K = T ≤ 1 , Δ t k = t k + 1 − t k , k = 0 , … , K − 1. (14) 0=t_0<\cdots<t_K=T\le1,\qquad\Delta t_k=t_{k+1}-t_k,\qquad k=0,\ldots,K-1. \tag*{(14)} 0 = t 0 < ⋯ < t K = T ≤ 1 , Δ t k = t k + 1 − t k , k = 0 , … , K − 1. (14)
The uniform grid has t k = k T / K t_k=kT/K t k = k T / K and Δ t k = T / K \Delta t_k=T/K Δ t k = T / K . Given an estimator b ^ \widehat b b of the drift, the Euler–Maruyama scheme has the iteration
X ~ t k + 1 = X ~ t k + Δ t k b ^ ( X ~ t k , t k ) + Δ t k η k + 1 , X ~ 0 = 0 , η k + 1 ∼ i . i . d . N ( 0 , I d ) (15) \widetilde X_{t_{k+1}}=\widetilde X_{t_k}+\Delta t_k\widehat b(\widetilde X_{t_k},t_k)+\sqrt{\Delta t_k}\,\eta_{k+1},\qquad\widetilde X_0=0,\qquad\eta_{k+1}\mathrel{\mathop{\sim}^{\mathrm{i.i.d.}}}\mathcal{N}(0,I_d) \tag*{(15)} X t k + 1 = X t k + Δ t k b ( X t k , t k ) + Δ t k η k + 1 , X 0 = 0 , η k + 1 ∼ i.i.d. N ( 0 , I d ) (15)
for k = 0 , 1 , … , K − 1 k=0,1,\ldots,K-1 k = 0 , 1 , … , K − 1 . The output is X ~ T \widetilde X_T X T ; taking T = 1 T=1 T = 1 targets the posterior π \pi π .
The scheme thus requires the drift at every visited state and time. Formula (III.5) involves no derivatives of the likelihood, but it requires values of L L L through f f f , which are not available in our setting. The rest of this section addresses the estimation of b b b .
Likelihood-free representation of the drift
To remove L L L from (III.5), we use the representation (II.2). Draw
Z ∼ N ( x , ( 1 − t ) I d ) , ξ ∣ Z ∼ p ( ⋅ ∣ Z ) , (16) Z\sim\mathcal{N}(x,(1-t)I_d),\qquad\xi\mid Z\sim p(\cdot\mid Z), \tag*{(16)} Z ∼ N ( x , ( 1 − t ) I d ) , ξ ∣ Z ∼ p ( ⋅ ∣ Z ) , (16)
and define the computable weight
W ( z , ξ ) = π 0 ( z ) φ ( z ) ψ ( z , ξ ) . (17) W(z,\xi)=\frac{\pi_0(z)}{\varphi(z)}\psi(z,\xi). \tag*{(17)} W ( z , ξ ) = φ ( z ) π 0 ( z ) ψ ( z , ξ ) . (17)
For fixed z z z , the likelihood representation gives
E ξ ∣ z [ W ( z , ξ ) ] = π 0 ( z ) L ( z ) φ ( z ) = Z f ( z ) . (18) \mathbb{E}_{\xi\mid z}[W(z,\xi)]=\frac{\pi_0(z)L(z)}{\varphi(z)}=\mathcal{Z}f(z). \tag*{(18)} E ξ ∣ z [ W ( z , ξ )] = φ ( z ) π 0 ( z ) L ( z ) = Z f ( z ) . (18)
By iterated expectation,
E Z , ξ [ W ( Z , ξ ) ] = E Z [ π 0 ( Z ) φ ( Z ) E ξ ∣ Z [ ψ ( Z , ξ ) ] ] = Z E Z [ f ( Z ) ] , \mathbb{E}_{Z,\xi}[W(Z,\xi)]=\mathbb{E}_Z\left[\frac{\pi_0(Z)}{\varphi(Z)}\mathbb{E}_{\xi\mid Z}[\psi(Z,\xi)]\right]=\mathcal{Z}\mathbb{E}_Z[f(Z)], E Z , ξ [ W ( Z , ξ )] = E Z [ φ ( Z ) π 0 ( Z ) E ξ ∣ Z [ ψ ( Z , ξ )] ] = Z E Z [ f ( Z )] ,
E Z , ξ [ W ( Z , ξ ) ( Z − x ) ] = E Z [ π 0 ( Z ) φ ( Z ) ( Z − x ) E ξ ∣ Z [ ψ ( Z , ξ ) ] ] = Z E Z [ f ( Z ) ( Z − x ) ] . \mathbb{E}_{Z,\xi}[W(Z,\xi)(Z-x)]=\mathbb{E}_Z\left[\frac{\pi_0(Z)}{\varphi(Z)}(Z-x)\mathbb{E}_{\xi\mid Z}[\psi(Z,\xi)]\right]=\mathcal{Z}\mathbb{E}_Z[f(Z)(Z-x)]. E Z , ξ [ W ( Z , ξ ) ( Z − x )] = E Z [ φ ( Z ) π 0 ( Z ) ( Z − x ) E ξ ∣ Z [ ψ ( Z , ξ )] ] = Z E Z [ f ( Z ) ( Z − x )] .
Substituting these identities into (III.5), the constant Z \mathcal{Z} Z cancels and
b ( x , t ) = E Z , ξ [ W ( Z , ξ ) ( Z − x ) ] ( 1 − t ) E Z , ξ [ W ( Z , ξ ) ] . (19) b(x,t)=\frac{\mathbb{E}_{Z,\xi}[W(Z,\xi)(Z-x)]}{(1-t)\mathbb{E}_{Z,\xi}[W(Z,\xi)]}. \tag*{(19)} b ( x , t ) = ( 1 − t ) E Z , ξ [ W ( Z , ξ )] E Z , ξ [ W ( Z , ξ ) ( Z − x )] . (19)
Both expectations are taken under the joint law (III.8). We require the denominator expectation C ( x , t ) = E Z , ξ [ W ( Z , ξ ) ] C(x,t)=\mathbb{E}_{Z,\xi}[W(Z,\xi)] C ( x , t ) = E Z , ξ [ W ( Z , ξ )] to be finite and strictly positive, and E [ W ( Z , ξ ) ∥ Z − x ∥ ] < ∞ \mathbb{E}[W(Z,\xi)\lVert Z-x\rVert]<\infty E [ W ( Z , ξ ) ∥ Z − x ∥] < ∞ for the numerator.
Likelihood-free SFS
The most direct estimator of (III.11) draws $m$ independent pairs $(Z_i,\xi_i)$ according to (III.8) and sets
b ^ m ( x , t ) = m − 1 ∑ i = 1 m W ( Z i , ξ i ) ( Z i − x ) ( 1 − t ) m − 1 ∑ i = 1 m W ( Z i , ξ i ) . (20) \widehat b_m(x,t)=\frac{m^{-1}\sum_{i=1}^{m}W(Z_i,\xi_i)(Z_i-x)}{(1-t)m^{-1}\sum_{i=1}^{m}W(Z_i,\xi_i)}. \tag*{(20)} b m ( x , t ) = ( 1 − t ) m − 1 ∑ i = 1 m W ( Z i , ξ i ) m − 1 ∑ i = 1 m W ( Z i , ξ i ) ( Z i − x ) . (20)
This is a self-normalized ratio estimator [19 Section 9.2] : the numerator and denominator use the same pairs. Each Z i Z_i Z i is paired with a single draw ξ i ∼ p ( ⋅ ∣ Z i ) \xi_i\sim p(\cdot\mid Z_i) ξ i ∼ p ( ⋅ ∣ Z i ) . Using b ^ = b ^ m \widehat b=\widehat b_m b = b m in (III.7), with new pairs generated at each step according to (III.8) at ( x , t ) = ( X ~ t k , t k ) (x,t)=(\widetilde{X}_{t_k},t_k) ( x , t ) = ( X t k , t k ) , independently of future Brownian increments, gives likelihood-free SFS .
Each drift evaluation uses m m m evaluations of ψ \psi ψ , so N N N trajectories with K K K steps use N K m NKm N K m evaluations in total; one evaluation of ψ \psi ψ may itself require several simulator calls, as in § 5 -C.
The expectations in (III.11) can also be approximated by QMC or RQMC, replacing the independent uniform inputs that generate ( Z i , ξ i ) (Z_i,\xi_i) ( Z i , ξ i ) with low-discrepancy points or their randomizations. At fixed ( x , t ) (x,t) ( x , t ) , the first d d d coordinates of a QMC point u i u_i u i in the open unit cube give ( Z i ) j = x j + 1 − t Φ N − 1 ( u i , j ) (Z_i)_j=x_j+\sqrt{1-t}\,\Phi_{\mathrm{N}}^{-1}(u_{i,j}) ( Z i ) j = x j + 1 − t Φ N − 1 ( u i , j ) , j = 1 , … , d j=1,\ldots,d j = 1 , … , d , where Φ N \Phi_{\mathrm{N}} Φ N is the standard normal cumulative distribution function; the remaining coordinates generate ξ i \xi_i ξ i conditionally on Z i Z_i Z i . Since Φ N − 1 \Phi_{\mathrm{N}}^{-1} Φ N − 1 diverges at 0 and 1, the transformed integrands may be unbounded, and the classical Koksma–Hlawka bound need not apply. For such integrands, under suitable growth conditions on the integrands and their mixed derivatives, Owen [30 ] and Ouyang et al. [31 ] obtain RQMC errors of order O ( m − 1 + ε ) O(m^{-1+\varepsilon}) O ( m − 1 + ε ) in expectation for every ε > 0 \varepsilon>0 ε > 0 , compared with the MC rate O ( m − 1 / 2 ) O(m^{-1/2}) O ( m − 1/2 ) for square-integrable integrands.
Likelihood-free SFS with sample reuse
In likelihood-free SFS, the Gaussian samples at step k k k are drawn from N ( X ~ t k , ( 1 − t k ) I d ) \mathcal{N}(\widetilde{X}_{t_k},(1-t_k)I_d) N ( X t k , ( 1 − t k ) I d ) , which depends on both the current state and the time, so new Z i Z_i Z i and new simulator outputs ξ i ∼ p ( ⋅ ∣ Z i ) \xi_i\sim p(\cdot\mid Z_i) ξ i ∼ p ( ⋅ ∣ Z i ) are needed at every step and for every trajectory.
To keep the samples fixed, we use importance sampling. Choose a probability density q q q on R d \mathbb{R}^d R d , independent of ( x , t ) (x,t) ( x , t ) , such that q ( z ) > 0 q(z)>0 q ( z ) > 0 for almost every z z z with π 0 ( z ) L ( z ) > 0 \pi_0(z)L(z)>0 π 0 ( z ) L ( z ) > 0 . Draw Z ∼ q Z\sim q Z ∼ q and then ξ ∼ p ( ⋅ ∣ Z ) \xi\sim p(\cdot\mid Z) ξ ∼ p ( ⋅ ∣ Z ) . Write E q \mathbb{E}_q E q for expectation under q ( z ) d z p ( d ξ ∣ z ) q(z)\,dz\,p(d\xi\mid z) q ( z ) d z p ( d ξ ∣ z ) .
Let g g g be a scalar or vector function with E Z , ξ [ W ( Z , ξ ) ∥ g ( Z ) ∥ ] < ∞ \mathbb{E}_{Z,\xi}[W(Z,\xi)\lVert g(Z)\rVert]<\infty E Z , ξ [ W ( Z , ξ ) ∥ g ( Z )∥] < ∞ , where the expectation is under (III.8). Then
E Z , ξ [ W ( Z , ξ ) g ( Z ) ] = ∫ E ξ ∣ z [ W ( z , ξ ) ] g ( z ) ϕ 1 − t ( z − x ) d z = ∫ q ( z ) E ξ ∣ z [ W ( z , ξ ) ] ϕ 1 − t ( z − x ) q ( z ) g ( z ) d z = E q [ W ( Z , ξ ) ϕ 1 − t ( Z − x ) q ( Z ) g ( Z ) ] . (21) \begin{aligned}
\mathbb{E}_{Z,\xi}[W(Z,\xi)g(Z)] &= \int\mathbb{E}_{\xi\mid z}[W(z,\xi)]g(z)\phi_{1-t}(z-x)\,dz \\
&= \int q(z)\mathbb{E}_{\xi\mid z}[W(z,\xi)]\frac{\phi_{1-t}(z-x)}{q(z)}g(z)\,dz \\
&= \mathbb{E}_q\left[W(Z,\xi)\frac{\phi_{1-t}(Z-x)}{q(Z)}g(Z)\right]. \tag*{(21)}
\end{aligned} E Z , ξ [ W ( Z , ξ ) g ( Z )] = ∫ E ξ ∣ z [ W ( z , ξ )] g ( z ) ϕ 1 − t ( z − x ) d z = ∫ q ( z ) E ξ ∣ z [ W ( z , ξ )] q ( z ) ϕ 1 − t ( z − x ) g ( z ) d z = E q [ W ( Z , ξ ) q ( Z ) ϕ 1 − t ( Z − x ) g ( Z ) ] . (21)
The sampling law under E q \mathbb{E}_q E q no longer depends on the state or the time. Taking g = 1 g=1 g = 1 and g ( z ) = z − x g(z)=z-x g ( z ) = z − x in (III.13) gives
b ( x , t ) = E q [ W ( Z , ξ ) q ( Z ) ϕ 1 − t ( Z − x ) ( Z − x ) ] ( 1 − t ) E q [ W ( Z , ξ ) q ( Z ) ϕ 1 − t ( Z − x ) ] . (22) b(x,t)=\frac{\mathbb{E}_q\left[\dfrac{W(Z,\xi)}{q(Z)}\phi_{1-t}(Z-x)(Z-x)\right]}{(1-t)\mathbb{E}_q\left[\dfrac{W(Z,\xi)}{q(Z)}\phi_{1-t}(Z-x)\right]}. \tag*{(22)} b ( x , t ) = ( 1 − t ) E q [ q ( Z ) W ( Z , ξ ) ϕ 1 − t ( Z − x ) ] E q [ q ( Z ) W ( Z , ξ ) ϕ 1 − t ( Z − x ) ( Z − x ) ] . (22)
Draw M M M pairs once, with Z i ∼ q Z_i \sim q Z i ∼ q and ξ i ∼ p ( ⋅ ∣ Z i ) \xi_i \sim p(\mathord{\cdot}\mid Z_i) ξ i ∼ p ( ⋅ ∣ Z i ) , and write
B M = { ( Z i , ξ i ) } i = 1 M . (23) \mathcal{B}_M=\{(Z_i,\xi_i)\}_{i=1}^{M}. \tag*{(23)} B M = {( Z i , ξ i ) } i = 1 M . (23)
We store the values W ( Z i , ξ i ) / q ( Z i ) W(Z_i,\xi_i)/q(Z_i) W ( Z i , ξ i ) / q ( Z i ) . Replacing the expectations in (III.14) by sample averages gives
b ^ M , q ( x , t ) = ∑ i = 1 M W ( Z i , ξ i ) q ( Z i ) φ 1 − t ( Z i − x ) ( Z i − x ) ( 1 − t ) ∑ i = 1 M W ( Z i , ξ i ) q ( Z i ) φ 1 − t ( Z i − x ) . (24) \widehat{b}_{M,q}(x,t)=\frac{\sum_{i=1}^{M}\frac{W(Z_i,\xi_i)}{q(Z_i)}\varphi_{1-t}(Z_i-x)(Z_i-x)}{(1-t)\sum_{i=1}^{M}\frac{W(Z_i,\xi_i)}{q(Z_i)}\varphi_{1-t}(Z_i-x)}. \tag*{(24)} b M , q ( x , t ) = ( 1 − t ) ∑ i = 1 M q ( Z i ) W ( Z i , ξ i ) φ 1 − t ( Z i − x ) ∑ i = 1 M q ( Z i ) W ( Z i , ξ i ) φ 1 − t ( Z i − x ) ( Z i − x ) . (24)
At each ( x , t ) (x,t) ( x , t ) , only the Gaussian factors φ 1 − t ( Z i − x ) \varphi_{1-t}(Z_i-x) φ 1 − t ( Z i − x ) must be recomputed. Using b ^ = b ^ M , q \widehat{b}=\widehat{b}_{M,q} b = b M , q in (III.7), with the same pairs at every step and for every trajectory, gives likelihood-free SFS with sample reuse , which requires no further simulator evaluations.
For a fixed proposal q q q , likelihood-free SFS with sample reuse requires M M M simulated pairs in total for N N N trajectories of K K K steps, compared with N K m N K m N K m for likelihood-free SFS; this can substantially reduce computation time when simulator evaluations dominate the cost.
Section III-E describes the proposals used in the experiments, and Sections IV-B and IV-C analyze terminal convergence and pointwise drift estimation under sample reuse, respectively.
Choosing the sampling distribution
Any proposal q q q satisfying the support condition gives a consistent sampler, but the efficiency of sample reuse depends on q q q . We compare three proposals of increasing sophistication, two of which are fitted to the posterior from a preliminary sample. The experiments use coordinates in which the prior is standard Gaussian, π 0 = φ \pi_0=\varphi π 0 = φ . Let q p q_p q p be a density for generating preliminary samples, positive wherever φ ( z ) L ( z ) > 0 \varphi(z)L(z)>0 φ ( z ) L ( z ) > 0 , and draw ( Z j p , ξ j p ) (Z_j^p,\xi_j^p) ( Z j p , ξ j p ) , j = 1 , … , M 0 j=1,\ldots,M_0 j = 1 , … , M 0 , with Z j p ∼ q p Z_j^p\sim q_p Z j p ∼ q p and ξ j p ∣ Z j p ∼ p ( ⋅ ∣ Z j p ) \xi_j^p\mid Z_j^p\sim p(\mathord{\cdot}\mid Z_j^p) ξ j p ∣ Z j p ∼ p ( ⋅ ∣ Z j p ) . Set a ~ ( z , ξ ) = ψ ( z , ξ ) φ ( z ) / q p ( z ) \widetilde{a}(z,\xi)=\psi(z,\xi)\varphi(z)/q_p(z) a ( z , ξ ) = ψ ( z , ξ ) φ ( z ) / q p ( z ) , with sample values and normalized weights
a ~ j = a ~ ( Z j p , ξ j p ) , a j = a ~ j ∑ ℓ = 1 M 0 a ~ ℓ . (25) \widetilde{a}_j=\widetilde{a}(Z_j^p,\xi_j^p),\qquad a_j=\frac{\widetilde{a}_j}{\sum_{\ell=1}^{M_0}\widetilde{a}_\ell}. \tag*{(25)} a j = a ( Z j p , ξ j p ) , a j = ∑ ℓ = 1 M 0 a ℓ a j . (25)
These weights estimate posterior moments, since for integrable g g g
E q p [ a ~ g ( Z ) ] = ∫ φ ( z ) L ( z ) g ( z ) d z . (26) \mathbb{E}_{q_p}[\widetilde{a}\,g(Z)]=\int\varphi(z)L(z)g(z)\,\mathrm{d}z. \tag*{(26)} E q p [ a g ( Z )] = ∫ φ ( z ) L ( z ) g ( z ) d z . (26)
The proposals use the fitted moments
μ ^ = ∑ j = 1 M 0 a j Z j p , Σ ^ = ∑ j = 1 M 0 a j ( Z j p − μ ^ ) ( Z j p − μ ^ ) T 1 − ∑ j = 1 M 0 a j 2 . (27) \widehat{\mu}=\sum_{j=1}^{M_0}a_jZ_j^p,\qquad\widehat{\Sigma}=\frac{\sum_{j=1}^{M_0}a_j(Z_j^p-\widehat{\mu})(Z_j^p-\widehat{\mu})^{\mathsf{T}}}{1-\sum_{j=1}^{M_0}a_j^2}. \tag*{(27)} μ = j = 1 ∑ M 0 a j Z j p , Σ = 1 − ∑ j = 1 M 0 a j 2 ∑ j = 1 M 0 a j ( Z j p − μ ) ( Z j p − μ ) T . (27)
Writing N ( μ , Σ ) \mathcal{N}(\mu,\Sigma) N ( μ , Σ ) also for the corresponding density, we use the proposals
q 1 = N ( 0 , I d ) , q_1 = \mathcal{N}(0,I_d), q 1 = N ( 0 , I d ) ,
q 2 = N ( μ ^ , 1.25 diag ( Σ ^ ) ) , (28) q_2 = \mathcal{N}\left(\widehat{\mu},1.25\operatorname{diag}(\widehat{\Sigma})\right), \tag*{(28)} q 2 = N ( μ , 1.25 diag ( Σ ) ) , (28)
q 3 = 0.95 q 2 + 0.05 q 1 . q_3 = 0.95q_2 + 0.05q_1. q 3 = 0.95 q 2 + 0.05 q 1 .
Here diag ( Σ ^ ) \operatorname{diag}(\widehat{\Sigma}) diag ( Σ ) is the diagonal part of Σ ^ \widehat{\Sigma} Σ . The prior q 1 q_1 q 1 needs no fitting, but when the posterior is concentrated, few of its samples fall where the posterior has mass. By (III.10), the integrands in (III.14) are, on average over ξ \xi ξ , proportional to f ( z ) = π ( z ) / φ ( z ) f(z)=\pi(z)/\varphi(z) f ( z ) = π ( z ) / φ ( z ) and are therefore large where the posterior has mass; q 2 q_2 q 2 places its samples there by using the fitted posterior mean and variances. The factor 1.25 makes q 2 q_2 q 2 slightly wider than the posterior, because one fixed proposal must serve all states and times, and importance weights are more stable when the proposal is wider than the region it covers. Finally, q 3 q_3 q 3 is a defensive mixture [22 ] , [23 ] : since q 3 ( z ) ≥ 0.05 q 1 ( z ) q_3(z)\ge0.05q_1(z) q 3 ( z ) ≥ 0.05 q 1 ( z ) , its importance weights are at most 20 times those under q 1 q_1 q 1 , which protects against regions missed by the fitted Gaussian. Section IV-C analyzes how the proposal affects the drift-estimation variance.
In practice, a fitted variance that is nearly zero can be replaced by a small positive lower bound (10 − 6 10^{-6} 1 0 − 6 in Section V-C). Once fitted, the proposal is held fixed, and the samples used for drift estimation are generated independently of those used in fitting.
The local effective sample size (ESS) measures the concentration of the weights in (III.16) at a given state and time. When their sum is positive, it is defined by
ESS ( x , t ) = ( ∑ i = 1 M W ( Z i , ξ i ) q ( Z i ) φ 1 − t ( Z i − x ) ) 2 ∑ i = 1 M ( W ( Z i , ξ i ) q ( Z i ) φ 1 − t ( Z i − x ) ) 2 . (29) \operatorname{ESS}(x,t)=\frac{\left(\sum_{i=1}^{M}\frac{W(Z_i,\xi_i)}{q(Z_i)}\varphi_{1-t}(Z_i-x)\right)^2}{\sum_{i=1}^{M}\left(\frac{W(Z_i,\xi_i)}{q(Z_i)}\varphi_{1-t}(Z_i-x)\right)^2}. \tag*{(29)} ESS ( x , t ) = ∑ i = 1 M ( q ( Z i ) W ( Z i , ξ i ) φ 1 − t ( Z i − x ) ) 2 ( ∑ i = 1 M q ( Z i ) W ( Z i , ξ i ) φ 1 − t ( Z i − x ) ) 2 . (29)
It lies between 1 and M M M and does not count independent posterior outputs. Near the terminal time it can approach one even for the exact fixed-sample diffusion; Remark B.1 relates it to the terminal importance weights.
Error and convergence analysis
We measure the error in the terminal distributions of the two methods of Section III in the quadratic Wasserstein distance W 2 W_2 W 2 . For likelihood-free SFS, Theorem IV.1 separates time-discretization and drift-estimation errors. For likelihood-free SFS with sample reuse, Theorem IV.2 first identifies the fixed-sample terminal distribution and then proves convergence to the posterior. The pointwise central limit theorem in Section IV-C describes the effect of the proposal on drift estimation.
Terminal error for likelihood-free SFS
We consider ABC in this subsection, i.e., L = L A B C L = L_{\mathrm{ABC}} L = L ABC and ψ ( x , ξ ) = K δ ( S ( ξ ) , S ( y ) ) \psi(x,\xi) = K_{\delta}(S(\xi), S(y)) ψ ( x , ξ ) = K δ ( S ( ξ ) , S ( y )) ; write π A B C \pi_{\mathrm{ABC}} π ABC for the resulting posterior. Set V = − log f V = -\log f V = − log f , where f = π A B C / φ f = \pi_{\mathrm{ABC}}/\varphi f = π ABC / φ as in (III.2).
Assumption IV.1. The potential V V V belongs to C 2 ( R d ) C^{2}(\mathbb{R}^{d}) C 2 ( R d ) , and there are constants 0 ≤ κ < 1 0 \le\kappa< 1 0 ≤ κ < 1 , λ + ≥ 0 \lambda_{+} \ge0 λ + ≥ 0 , and B 0 ≥ 0 B_{0} \ge0 B 0 ≥ 0 such that, for all x ∈ R d x \in\mathbb{R}^{d} x ∈ R d ,
− κ I d ⪯ ∇ 2 V ( x ) ⪯ λ + I d , ∥ ∇ V ( 0 ) ∥ ≤ B 0 d . (30) -\kappa I_{d} \preceq\nabla^{2}V(x) \preceq\lambda_{+}I_{d}, \qquad\lVert\nabla V(0)\rVert\le B_{0}\sqrt{d}. \tag*{(30)} − κ I d ⪯ ∇ 2 V ( x ) ⪯ λ + I d , ∥ ∇ V ( 0 )∥ ≤ B 0 d . (30)
Since − log π A B C ( x ) = V ( x ) + ∥ x ∥ 2 / 2 + constant -\log\pi_{\mathrm{ABC}}(x) = V(x) + \lVert x\rVert^{2}/2 + \text{constant} − log π ABC ( x ) = V ( x ) + ∥ x ∥ 2 /2 + constant and κ < 1 \kappa< 1 κ < 1 , this assumption makes the target strongly log-concave. It also yields the spatial Lipschitz and time-increment estimates for the drift used in the proof of the terminal error bound. Appendix A-B gives sufficient conditions in terms of the ABC likelihood and the prior.
Let X ~ \widetilde{X} X be likelihood-free SFS (Section III-C) on the uniform grid (III.6) with T = 1 T = 1 T = 1 , h = 1 / K h = 1/K h = 1/ K , and t k = k h t_k = kh t k = k h . Let F t k \mathcal{F}_{t_k} F t k be the σ \sigma σ -field generated by the Brownian motion up to time t k t_k t k and all auxiliary samples generated before step k k k . We control the drift-estimation error through a nonnegative function ρ \rho ρ such that, almost surely for k = 0 , … , K − 1 k = 0,\ldots,K-1 k = 0 , … , K − 1 ,
E [ ∥ b ^ m ( X ~ t k , t k ) − b ( X ~ t k , t k ) ∥ 2 | F t k ] ≤ ρ ( t k ) d m , (31) \mathbb{E}\left[\left\lVert\widehat{b}_{m}(\widetilde{X}_{t_k},t_k)-b(\widetilde{X}_{t_k},t_k)\right\rVert^{2}\,\middle|\,\mathcal{F}_{t_k}\right] \le\frac{\rho(t_k)d}{m}, \tag*{(31)} E [ b m ( X t k , t k ) − b ( X t k , t k ) 2 F t k ] ≤ m ρ ( t k ) d , (31)
and set
Λ h = h ∑ k = 0 K − 1 ρ ( t k ) . (32) \Lambda_h = h\sum_{k=0}^{K-1}\rho(t_k). \tag*{(32)} Λ h = h k = 0 ∑ K − 1 ρ ( t k ) . (32)
Theorem IV.1. Suppose that Assumption IV.1 holds, that (IV.2) holds, and that Λ h \Lambda_h Λ h is finite for the chosen grid. Then
W 2 ( Law ( X ~ 1 ) , π A B C ) ≤ C p a t h d h + C p a t h d Λ h m , (33) W_{2}\left(\operatorname{Law}(\widetilde{X}_{1}),\pi_{\mathrm{ABC}}\right) \le C_{\mathrm{path}}\sqrt{dh}+C_{\mathrm{path}}\sqrt{\frac{d\Lambda_h}{m}}, \tag*{(33)} W 2 ( Law ( X 1 ) , π ABC ) ≤ C path d h + C path m d Λ h , (33) where C p a t h C_{\mathrm{path}} C path depends only on B 0 B_{0} B 0 , λ + \lambda_{+} λ + , and κ \kappa κ .
The two terms in (IV.4) are the time-discretization error and the accumulated drift-estimation error.
The proof in Appendix A-A couples the exact and numerical paths; it also shows that the fourth-moment conditions (A.13) are sufficient for (IV.2). No bound on Λ h \Lambda_h Λ h uniform in h h h is required. For fixed d d d , the bound tends to zero whenever h → 0 h \to0 h → 0 and Λ h / m → 0 \Lambda_h/m \to0 Λ h / m → 0 . For example, when f = 1 f = 1 f = 1 and W = 1 W = 1 W = 1 , the exact drift is zero, (III.12) has mean-square error d / [ m ( 1 − t ) ] d/[m(1-t)] d / [ m ( 1 − t )] . Taking ρ ( t ) = ( 1 − t ) − 1 \rho(t) = (1-t)^{-1} ρ ( t ) = ( 1 − t ) − 1 gives
Λ h = h ∑ k = 0 K − 1 1 1 − k h = ∑ j = 1 K 1 j ≤ 1 + log K . \Lambda_h = h\sum_{k=0}^{K-1}\frac{1}{1-kh}=\sum_{j=1}^{K}\frac{1}{j}\le1+\log K. Λ h = h k = 0 ∑ K − 1 1 − k h 1 = j = 1 ∑ K j 1 ≤ 1 + log K .
Thus (IV.4) gives an error bound of order O ( d h + d ( 1 + log ( 1 / h ) ) m ) O\left(\sqrt{dh}+\sqrt{\frac{d(1+\log(1/h))}{m}}\right) O ( d h + m d ( 1 + l o g ( 1/ h )) ) , which tends to zero for fixed d d d if h → 0 h \to0 h → 0 and log ( 1 / h ) / m → 0 \log(1/h)/m \to0 log ( 1/ h ) / m → 0 .
Theorem IV.1 measures the error relative to the ABC posterior π A B C \pi_{\mathrm{ABC}} π ABC . Relative to the posterior given the observed summary S ( y ) S(y) S ( y ) , the triangle inequality adds the ABC approximation error, which depends on the kernel, the summary, and the model, and vanishes as δ → 0 \delta\to0 δ → 0 under suitable conditions [32 ] , [33 ] ; relative to the full-data posterior, a further error due to summarization appears. Since C p a t h C_{\mathrm{path}} C path and Λ h \Lambda_h Λ h may depend on δ \delta δ , a joint limit in h h h , m m m , and δ \delta δ would require control of this dependence.
Terminal convergence with sample reuse
We now return to the general representation (II.2); f f f , W W W , b b b , and C C C refer to the corresponding target π \pi π .
For the samples B M \mathcal{B}_M B M in (III.15), define the importance weights and the weighted empirical measure
c i = W ( Z i , ξ i ) q ( Z i ) , v i = c i φ ( Z i ) = π 0 ( Z i ) ψ ( Z i , ξ i ) q ( Z i ) , c_i = \frac{W(Z_i,\xi_i)}{q(Z_i)}, \qquad v_i = c_i\varphi(Z_i) = \frac{\pi_0(Z_i)\psi(Z_i,\xi_i)}{q(Z_i)}, c i = q ( Z i ) W ( Z i , ξ i ) , v i = c i φ ( Z i ) = q ( Z i ) π 0 ( Z i ) ψ ( Z i , ξ i ) ,
a i , M = v i ∑ j = 1 M v j , ν M = ∑ i = 1 M a i , M δ Z i , (34) a_{i,M} = \frac{v_i}{\sum_{j=1}^{M}v_j}, \qquad\nu_M = \sum_{i=1}^{M} a_{i,M}\delta_{Z_i}, \tag*{(34)} a i , M = ∑ j = 1 M v j v i , ν M = i = 1 ∑ M a i , M δ Z i , (34)
whenever ∑ j v j > 0 \sum_j v_j > 0 ∑ j v j > 0 ; here δ z \delta_z δ z denotes the Dirac measure at z z z .
Let X ~ M , h \widetilde{X}^{M,h} X M , h denote the Euler scheme (III.7) with drift b ^ M , q \widehat{b}_{M,q} b M , q on a deterministic grid (III.6) with T = 1 T=1 T = 1 , driven by Brownian increments independent of the samples, and set h = max k Δ t k h=\max_k\Delta t_k h = max k Δ t k . The exact diffusion with this drift, started at zero, has terminal distribution ν M \nu_M ν M (Theorem IV.2(i) below), so the error splits as
W 2 ( L ( X ~ 1 M , h ∣ B M ) , π ) ≤ W 2 ( L ( X ~ 1 M , h ∣ B M ) , ν M ) + W 2 ( ν M , π ) . (35) W_2\left(\mathcal{L}\left(\widetilde{X}^{M,h}_1 \mid\mathcal{B}_M\right),\pi\right)
\le
W_2\left(\mathcal{L}\left(\widetilde{X}^{M,h}_1 \mid\mathcal{B}_M\right),\nu_M\right)
+
W_2(\nu_M,\pi). \tag*{(35)} W 2 ( L ( X 1 M , h ∣ B M ) , π ) ≤ W 2 ( L ( X 1 M , h ∣ B M ) , ν M ) + W 2 ( ν M , π ) . (35)
The first term is the time-discretization error for fixed samples, and the second is the importance-sampling error. For the large-sample limit, let q q q be fixed and satisfy the support condition of § 3 -D, let ( Z i , ξ i ) i ≥ 1 (Z_i,\xi_i)_{i\geq1} ( Z i , ξ i ) i ≥ 1 be an iid sequence with joint law q ( z ) d z p ( d ξ ∣ z ) q(z)\,dz\,p(d\xi\mid z) q ( z ) d z p ( d ξ ∣ z ) , and let B M \mathcal{B}_M B M consist of its first M M M pairs.
Theorem IV.2. (i) Fixed samples. Suppose that c i ∈ [ 0 , ∞ ) c_i \in[0,\infty) c i ∈ [ 0 , ∞ ) for all i i i and ∑ i v i > 0 \sum_i v_i > 0 ∑ i v i > 0 . Then the diffusion with drift b ^ M , q \widehat{b}_{M,q} b M , q , started at zero, has a unique strong solution on [ 0 , 1 ) [0,1) [ 0 , 1 ) that extends continuously to t = 1 t=1 t = 1 , its terminal law given B M \mathcal{B}_M B M is ν M \nu_M ν M , and, as h ↓ 0 h \downarrow0 h ↓ 0 ,
W 2 ( L ( X ~ 1 M , h ∣ B M ) , ν M ) ⟶ 0. (36) W_2\left(\mathcal{L}\left(\widetilde{X}^{M,h}_1 \mid\mathcal{B}_M\right),\nu_M\right)\longrightarrow0. \tag*{(36)} W 2 ( L ( X 1 M , h ∣ B M ) , ν M ) ⟶ 0. (36) (ii) Increasing sample size. Suppose that 0 < Z < ∞ 0 < \mathcal{Z} < \infty 0 < Z < ∞ and
∫ R d ∥ z ∥ 2 π ( z ) d z < ∞ . (37) \int_{\mathbb{R}^d}\lVert z\rVert^2\pi(z)\,dz < \infty. \tag*{(37)} ∫ R d ∥ z ∥ 2 π ( z ) d z < ∞. (37) Then, almost surely, ∑ i = 1 M v i > 0 \sum_{i=1}^{M}v_i > 0 ∑ i = 1 M v i > 0 for all sufficiently large M M M , and
W 2 ( ν M , π ) ⟶ 0 almost surely . (38) W_2(\nu_M,\pi)\longrightarrow0 \qquad\text{almost surely}. \tag*{(38)} W 2 ( ν M , π ) ⟶ 0 almost surely . (38)
lim M → ∞ lim h ↓ 0 W 2 ( L ( X ~ 1 M , h ∣ B M ) , π ) = 0 almost surely. (39) \lim_{M\to\infty}\lim_{h\downarrow0} W_2\left(\mathcal{L}\left(\widetilde{X}_1^{M,h}\mid\mathcal{B}_M\right),\pi\right)=0 \quad\text{almost surely.} \tag*{(39)} M → ∞ lim h ↓ 0 lim W 2 ( L ( X 1 M , h ∣ B M ) , π ) = 0 almost surely. (39)
Moreover, there is a deterministic sequence of uniform grids with h M = 1 / K M ↓ 0 h_M=1/K_M\downarrow0 h M = 1/ K M ↓ 0 such that
W 2 ( L ( X ~ 1 M , h M ∣ B M ) , π ) ⟶ 0 almost surely. (40) W_2\left(\mathcal{L}\left(\widetilde{X}_1^{M,h_M}\mid\mathcal{B}_M\right),\pi\right)\longrightarrow0 \quad\text{almost surely.} \tag*{(40)} W 2 ( L ( X 1 M , h M ∣ B M ) , π ) ⟶ 0 almost surely. (40)
Thus both terms in (IV.6) vanish, the first as h ↓ 0 h\downarrow0 h ↓ 0 for fixed samples and the second as M → ∞ M\to\infty M → ∞ . The proof in Appendix B-A identifies the fixed-sample diffusion, controls its Euler approximation up to time one, and applies the strong law of large numbers to the weighted measure ν M \nu_M ν M .
Corollary IV.1 (Unconditional terminal convergence). Under the assumptions of part (ii) of Theorem IV.2, suppose in addition that ∫ ∥ z ∥ 2 q ( z ) d z < ∞ \int\|z\|^2q(z)\,dz<\infty ∫ ∥ z ∥ 2 q ( z ) d z < ∞ and v 1 > 0 v_1>0 v 1 > 0 almost surely. Then
lim M → ∞ lim h ↓ 0 W 2 ( L ( X ~ 1 M , h ) , π ) = 0. \lim_{M\to\infty}\lim_{h\downarrow0}W_2\left(\mathcal{L}\left(\widetilde{X}_1^{M,h}\right),\pi\right)=0. M → ∞ lim h ↓ 0 lim W 2 ( L ( X 1 M , h ) , π ) = 0. Moreover, the deterministic uniform grids in (IV.11) can be chosen such that W 2 ( L ( X ~ 1 M , h M ) , π ) → 0 W_2\left(\mathcal{L}\left(\widetilde{X}_1^{M,h_M}\right),\pi\right)\to0 W 2 ( L ( X 1 M , h M ) , π ) → 0 .
The additional moment assumptions allow averaging over the sample set; the proof is in Appendix B-B. For a fitted proposal, the conditional results hold given the preliminary samples, provided the pairs used for drift estimation are conditionally iid and the assumptions hold for the fitted q q q ; unconditional statements would require moment control over the fitting step. RQMC samples and the input clipping used in § 5 -C are not covered; they would require separate analyses of the integration rule and of the resulting target approximation, respectively.
Pointwise drift error and proposal choice
Theorem IV.2 holds for any proposal satisfying the support condition, but it does not show how the proposal affects accuracy. This subsection addresses that question at a single state and time. Theorem IV.3 shows that the reused drift estimator is consistent and asymptotically normal, and its asymptotic covariance then serves as a criterion for comparing proposals. The main conclusions are that the variance-optimal proposal depends on the state and time, so that any fixed proposal is a compromise, and that the defensive mixture q 3 q_3 q 3 is never much worse than the prior q 1 q_1 q 1 ; this motivates the proposals of § 3 -E. The analysis uses standard limit theory for self-normalized importance sampling [19 Chapter 9] . We abbreviate the weight by
Ω q ( z , ξ ; x , t ) = W ( z , ξ ) q ( z ) φ 1 − t ( z − x ) . (41) \Omega_q(z,\xi;x,t)=\frac{W(z,\xi)}{q(z)}\varphi_{1-t}(z-x). \tag*{(41)} Ω q ( z , ξ ; x , t ) = q ( z ) W ( z , ξ ) φ 1 − t ( z − x ) . (41)
At fixed ( x , t ) (x,t) ( x , t ) , write Ω q = Ω q ( Z , ξ ; x , t ) \Omega_q=\Omega_q(Z,\xi;x,t) Ω q = Ω q ( Z , ξ ; x , t ) . Identity (III.13) gives E q Ω q = C ( x , t ) \mathbb{E}_q\Omega_q=C(x,t) E q Ω q = C ( x , t ) , with C C C defined in § 3 -B. Set
z ˉ ( x , t ) = x + ( 1 − t ) b ( x , t ) . (42) \bar{z}(x,t)=x+(1-t)b(x,t). \tag*{(42)} z ˉ ( x , t ) = x + ( 1 − t ) b ( x , t ) . (42)
By (III.14), z ˉ ( x , t ) = E q [ Ω q Z ] / C ( x , t ) \bar{z}(x,t)=\mathbb{E}_q[\Omega_q Z]/C(x,t) z ˉ ( x , t ) = E q [ Ω q Z ] / C ( x , t ) . As in Section IV-B, the pairs in B M \mathcal{B}_M B M are iid with law q ( z ) d z p ( d ξ ∣ z ) q(z)\,dz\,p(d\xi\mid z) q ( z ) d z p ( d ξ ∣ z ) , and q q q satisfies the support condition of § 3 -D.
Theorem IV.3. Let ( x , t ) (x,t) ( x , t ) with t < 1 t<1 t < 1 be fixed independently of B M \mathcal{B}_M B M . Suppose that 0 < C ( x , t ) < ∞ 0<C(x,t)<\infty 0 < C ( x , t ) < ∞ and
E q [ Ω q 2 ∥ Z − x 1 − t − b ( x , t ) ∥ 2 ] < ∞ . (43) \mathbb{E}_q\left[\Omega_q^2\left\|\frac{Z-x}{1-t}-b(x,t)\right\|^2\right]<\infty. \tag*{(43)} E q [ Ω q 2 1 − t Z − x − b ( x , t ) 2 ] < ∞. (43) Then, almost surely, b ^ M , q ( x , t ) \widehat{b}_{M,q}(x,t) b M , q ( x , t ) is well defined for all sufficiently large M M M , and
b ^ M , q ( x , t ) ⟶ b ( x , t ) almost surely , (44) \widehat{b}_{M,q}(x,t)\longrightarrow b(x,t)\quad\text{almost surely}, \tag*{(44)} b M , q ( x , t ) ⟶ b ( x , t ) almost surely , (44) M { b ^ M , q ( x , t ) − b ( x , t ) } ⇒ N ( 0 , Σ q ( x , t ) ) , (45) \sqrt{M}\{\widehat{b}_{M,q}(x,t)-b(x,t)\}\Rightarrow\mathcal{N}(0,\Sigma_q(x,t)), \tag*{(45)} M { b M , q ( x , t ) − b ( x , t )} ⇒ N ( 0 , Σ q ( x , t )) , (45) where ⇒ \Rightarrow ⇒ denotes convergence in distribution and
Σ q ( x , t ) = E q [ Ω q 2 ( Z − z ˉ ) ( Z − z ˉ ) T ] ( 1 − t ) 2 C ( x , t ) 2 . (46) \Sigma_q(x,t)=\frac{\mathbb{E}_q[\Omega_q^2(Z-\bar{z})(Z-\bar{z})^{\mathsf T}]}{(1-t)^2C(x,t)^2}. \tag*{(46)} Σ q ( x , t ) = ( 1 − t ) 2 C ( x , t ) 2 E q [ Ω q 2 ( Z − z ˉ ) ( Z − z ˉ ) T ] . (46)
The limiting covariance Σ q ( x , t ) \Sigma_q(x,t) Σ q ( x , t ) may be singular. The proof in Appendix B-C applies the strong law of large numbers and the central limit theorem to a centered weighted sum. These pointwise limits give no finite-sample mean-square bound and do not apply directly at numerical states, which depend on B M \mathcal{B}_M B M .
We now use tr Σ q ( x , t ) \operatorname{tr}\Sigma_q(x,t) tr Σ q ( x , t ) , the asymptotic mean-square error of the drift estimator at ( x , t ) (x,t) ( x , t ) , to compare proposals.
For fixed ( x , t ) (x,t) ( x , t ) , define
A x , t ( z ) = φ 1 − t ( z − x ) 2 E ξ ∣ z [ ( W ( z , ξ ) ) 2 ∥ z − z ˉ ( x , t ) ∥ 2 ] , A_{x,t}(z)=\varphi_{1-t}(z-x)^2\mathbb{E}_{\xi\mid z}\left[(W(z,\xi))^2\|z-\bar{z}(x,t)\|^2\right], A x , t ( z ) = φ 1 − t ( z − x ) 2 E ξ ∣ z [ ( W ( z , ξ ) ) 2 ∥ z − z ˉ ( x , t ) ∥ 2 ] ,
V x , t ( q ) = ∫ A x , t ( z ) q ( z ) d z . (47) \mathcal{V}_{x,t}(q)=\int\frac{A_{x,t}(z)}{q(z)}\,dz. \tag*{(47)} V x , t ( q ) = ∫ q ( z ) A x , t ( z ) d z . (47)
Then
tr Σ q ( x , t ) = V x , t ( q ) ( 1 − t ) 2 C ( x , t ) 2 . (48) \operatorname{tr}\Sigma_q(x,t)=\frac{\mathcal{V}_{x,t}(q)}{(1-t)^2C(x,t)^2}. \tag*{(48)} tr Σ q ( x , t ) = ( 1 − t ) 2 C ( x , t ) 2 V x , t ( q ) . (48)
The denominator does not depend on q q q . When 0 < ∫ A x , t < ∞ 0<\int\sqrt{A_{x,t}}<\infty 0 < ∫ A x , t < ∞ and the support condition holds, the Cauchy–Schwarz inequality shows that the trace is minimized by
q x , t ∗ ( z ) = A x , t ( z ) ∫ A x , t ( u ) d u . (49) q^*_{x,t}(z)=\frac{\sqrt{A_{x,t}(z)}}{\int\sqrt{A_{x,t}(u)}\,du}. \tag*{(49)} q x , t ∗ ( z ) = ∫ A x , t ( u ) d u A x , t ( z ) . (49)
Since q x , t ∗ q^*_{x,t} q x , t ∗ depends on ( x , t ) (x,t) ( x , t ) , on the unknown z ˉ ( x , t ) \bar{z}(x,t) z ˉ ( x , t ) , and on a conditional second moment of the simulator weight, a single fitted proposal cannot in general minimize this variance at all states and times. A proposal fitted to the posterior, such as q 2 q_2 q 2 , is a practical compromise, since the numerical states concentrate near the posterior mass as t → 1 t\to1 t → 1 .
For the mixture q 3 q_3 q 3 in (III.20), the bound q 3 ( z ) ≥ 0.05 q 1 ( z ) q_3(z)\geq0.05q_1(z) q 3 ( z ) ≥ 0.05 q 1 ( z ) gives
V x , t ( q 3 ) ≤ 20 V x , t ( q 1 ) (50) \mathcal{V}_{x,t}(q_3)\leq20\mathcal{V}_{x,t}(q_1) \tag*{(50)} V x , t ( q 3 ) ≤ 20 V x , t ( q 1 ) (50)
provided V x , t ( q 1 ) < ∞ V_{x,t}(q_1)<\infty V x , t ( q 1 ) < ∞ . Thus, at every state and time, the asymptotic mean-square drift error under q 3 q_3 q 3 is at most 20 times that under the prior q 1 q_1 q 1 , whatever the quality of the fitted component.
These calculations are detailed in Appendix B-D.
Numerical experiments
With sample reuse, trajectories driven by independent Brownian paths are conditionally independent given B M \mathcal{B}_M B M but generally dependent unconditionally. Increasing N N N therefore does not reduce the error due to B M \mathcal{B}_M B M ; assessing its variability requires independent repetitions of the sample generation. In the tables and figures, MC–SFS and RQMC–SFS denote likelihood-free SFS whose drift samples are generated by MC and RQMC, respectively, and q j q_j q j –MC and q j q_j q j –RQMC denote likelihood-free SFS with sample reuse, proposal q j q_j q j in (III.20), and MC or RQMC samples.
Analytic Gaussian ABC benchmark
This model has a closed-form ABC posterior and SFS drift, which allows us to assess drift estimation, sample reuse, and Euler discretization, including for a concentrated posterior. Let x ∼ N ( 0 , I 2 ) x\sim\mathcal{N}(0,I_2) x ∼ N ( 0 , I 2 ) and
ξ ∣ x ∼ N ( x , σ 2 I 2 ) , y = ( 2 , − 2 ) T , σ = 0.5. (51) \xi\mid x\sim\mathcal{N}(x,\sigma^2 I_2), \qquad y=(2,-2)^\mathsf{T}, \qquad\sigma=0.5. \tag*{(51)} ξ ∣ x ∼ N ( x , σ 2 I 2 ) , y = ( 2 , − 2 ) T , σ = 0.5. (51)
We use the identity summary and the Gaussian ABC kernel
ψ ( x , ξ ) = exp [ − ∥ ξ − y ∥ 2 2 δ ] , δ = 0.25. (52) \psi(x,\xi)=\exp\left[-\frac{\lVert\xi-y\rVert^2}{2\delta}\right], \qquad\delta=0.25. \tag*{(52)} ψ ( x , ξ ) = exp [ − 2 δ ∥ ξ − y ∥ 2 ] , δ = 0.25. (52)
Here δ \delta δ is a bandwidth on the variance scale. Since ψ ( x , ξ ) = ( 2 π δ ) d / 2 φ δ ( ξ − y ) \psi(x,\xi)=(2\pi\delta)^{d/2}\varphi_\delta(\xi-y) ψ ( x , ξ ) = ( 2 π δ ) d /2 φ δ ( ξ − y ) , Gaussian convolution gives
L A B C ( x ) = ( 2 π δ ) d / 2 ∫ R d φ δ ( ξ − y ) φ σ 2 ( ξ − x ) d ξ = ( δ δ + σ 2 ) d / 2 exp [ − ∥ x − y ∥ 2 2 ( δ + σ 2 ) ] , d = 2. (53) \begin{aligned}
L_{\mathrm{ABC}}(x)=(2\pi\delta)^{d/2}\int_{\mathbb{R}^d}\varphi_\delta(\xi-y)\varphi_{\sigma^2}(\xi-x)\,\mathrm{d}\xi\\
&=\left(\frac{\delta}{\delta+\sigma^2}\right)^{d/2}\exp\left[-\frac{\lVert x-y\rVert^2}{2(\delta+\sigma^2)}\right], \qquad d=2. \tag*{(53)}
\end{aligned} L ABC ( x ) = ( 2 π δ ) d /2 ∫ R d φ δ ( ξ − y ) φ σ 2 ( ξ − x ) d ξ = ( δ + σ 2 δ ) d /2 exp [ − 2 ( δ + σ 2 ) ∥ x − y ∥ 2 ] , d = 2. (53)
Set v = σ 2 + δ = 0.5 v=\sigma^2+\delta=0.5 v = σ 2 + δ = 0.5 . Multiplying by the prior and completing the square gives
π A B C = N ( μ ⋆ , Σ ⋆ ) , μ ⋆ = y 1 + v = ( 4 / 3 , − 4 / 3 ) T , Σ ⋆ = v 1 + v I 2 = 1 3 I 2 . (54) \pi_{\mathrm{ABC}}=\mathcal{N}(\mu_\star,\Sigma_\star), \qquad\mu_\star=\frac{y}{1+v}=(4/3,-4/3)^\mathsf{T}, \qquad\Sigma_\star=\frac{v}{1+v}I_2=\frac{1}{3}I_2. \tag*{(54)} π ABC = N ( μ ⋆ , Σ ⋆ ) , μ ⋆ = 1 + v y = ( 4/3 , − 4/3 ) T , Σ ⋆ = 1 + v v I 2 = 3 1 I 2 . (54)
Since f = L A B C / Z f=L_{\mathrm{ABC}}/Z f = L ABC / Z , a second Gaussian convolution yields
Q 1 − t f ( x ) = c G ( t ) exp [ − ∥ x − y ∥ 2 2 ( v + 1 − t ) ] , (55) Q_{1-t}f(x)=c_G(t)\exp\left[-\frac{\lVert x-y\rVert^2}{2(v+1-t)}\right], \tag*{(55)} Q 1 − t f ( x ) = c G ( t ) exp [ − 2 ( v + 1 − t ) ∥ x − y ∥ 2 ] , (55)
where c G ( t ) > 0 c_G(t)>0 c G ( t ) > 0 is independent of x x x . Hence
b ( x , t ) = ∇ x log Q 1 − t f ( x ) = y − x v + 1 − t . (56) b(x,t)=\nabla_x\log Q_{1-t}f(x)=\frac{y-x}{v+1-t}. \tag*{(56)} b ( x , t ) = ∇ x log Q 1 − t f ( x ) = v + 1 − t y − x . (56)
The density-ratio potential satisfies
V ( x ) = − log f ( x ) = ∥ x − y ∥ 2 2 v + constant , ∇ 2 V ( x ) = 2 I 2 , ∇ V ( 0 ) = − 2 y . (57) V(x)=-\log f(x)=\frac{\lVert x-y\rVert^2}{2v}+\text{constant},\qquad\nabla^2V(x)=2I_2,\qquad\nabla V(0)=-2y. \tag*{(57)} V ( x ) = − log f ( x ) = 2 v ∥ x − y ∥ 2 + constant , ∇ 2 V ( x ) = 2 I 2 , ∇ V ( 0 ) = − 2 y . (57)
Thus Assumption IV.1 holds with κ = 0 \kappa=0 κ = 0 , λ + = 2 \lambda_+=2 λ + = 2 , and B 0 = 4 B_0=4 B 0 = 4 , since ∥ ∇ V ( 0 ) ∥ = 4 2 = B 0 d \lVert\nabla V(0)\rVert=4\sqrt{2}=B_0\sqrt{d} ∥ ∇ V ( 0 )∥ = 4 2 = B 0 d .
However, for every finite m m m , no deterministic ρ \rho ρ satisfies (IV.2) uniformly over the states x x x . Indeed, with τ = 1 − t \tau=1-t τ = 1 − t and G i = ( Z i − x ) / τ G_i=(Z_i-x)/\sqrt{\tau} G i = ( Z i − x ) / τ , the estimator b ^ m \widehat b_m b m is a convex combination of G i / τ G_i/\sqrt{\tau} G i / τ , so
E ∥ b ^ m ( x , t ) ∥ 2 ≤ m d τ , ( E ∥ b ^ m ( x , t ) − b ( x , t ) ∥ 2 ) 1 / 2 ≥ ( ∥ y − x ∥ v + τ − m d τ ) + . \mathbb{E}\lVert\widehat b_m(x,t)\rVert^2\leq\frac{md}{\tau},\qquad\left(\mathbb{E}\lVert\widehat b_m(x,t)-b(x,t)\rVert^2\right)^{1/2}\geq\left(\frac{\lVert y-x\rVert}{v+\tau}-\sqrt{\frac{md}{\tau}}\right)_+. E ∥ b m ( x , t ) ∥ 2 ≤ τ m d , ( E ∥ b m ( x , t ) − b ( x , t ) ∥ 2 ) 1/2 ≥ ( v + τ ∥ y − x ∥ − τ m d ) + .
where ( u ) + = max { u , 0 } (u)_+=\max\{u,0\} ( u ) + = max { u , 0 } . The lower bound diverges as ∥ x ∥ → ∞ \lVert x\rVert\to\infty ∥ x ∥ → ∞ . Since every Euler state after the first step has full support, owing to its independent Gaussian increment, no finite deterministic ρ ( t k ) \rho(t_k) ρ ( t k ) satisfies (IV.2) at those steps. Theorem IV.1 therefore does not directly cover this experiment, although this does not indicate that the algorithm fails. All comparisons target the ABC posterior with δ = 0.25 \delta=0.25 δ = 0.25 and do not test the limit δ → 0 \delta\to0 δ → 0 .
Let ( μ ^ o u t , Σ ^ o u t ) (\widehat\mu_{\mathrm{out}},\widehat\Sigma_{\mathrm{out}}) ( μ out , Σ out ) be the sample mean and covariance of the output, and ( μ r e f , Σ r e f ) (\mu_{\mathrm{ref}},\Sigma_{\mathrm{ref}}) ( μ ref , Σ ref ) the analytic reference moments, namely ( μ ⋆ , Σ ⋆ ) (\mu_\star,\Sigma_\star) ( μ ⋆ , Σ ⋆ ) here and ( μ 100 , Σ 100 ) (\mu_{100},\Sigma_{100}) ( μ 100 , Σ 100 ) in the concentrated case below. Define
E μ = ∥ μ ^ o u t − μ r e f ∥ tr Σ r e f , E Σ = ∥ Σ ^ o u t − Σ r e f ∥ F ∥ Σ r e f ∥ F . (58) E_\mu=\frac{\lVert\widehat\mu_{\mathrm{out}}-\mu_{\mathrm{ref}}\rVert}{\sqrt{\operatorname{tr}\Sigma_{\mathrm{ref}}}},\qquad E_\Sigma=\frac{\lVert\widehat\Sigma_{\mathrm{out}}-\Sigma_{\mathrm{ref}}\rVert_F}{\lVert\Sigma_{\mathrm{ref}}\rVert_F}. \tag*{(58)} E μ = tr Σ ref ∥ μ out − μ ref ∥ , E Σ = ∥ Σ ref ∥ F ∥ Σ out − Σ ref ∥ F . (58)
When the full output covariance was recorded, we also report
W 2 , G 2 = ∥ μ ^ o u t − μ r e f ∥ 2 + tr [ Σ ^ o u t + Σ r e f − 2 ( Σ r e f 1 / 2 Σ ^ o u t Σ r e f 1 / 2 ) 1 / 2 ] . (59) W_{2,G}^2=\lVert\widehat\mu_{\mathrm{out}}-\mu_{\mathrm{ref}}\rVert^2+\operatorname{tr}\left[\widehat\Sigma_{\mathrm{out}}+\Sigma_{\mathrm{ref}}-2\left(\Sigma_{\mathrm{ref}}^{1/2}\widehat\Sigma_{\mathrm{out}}\Sigma_{\mathrm{ref}}^{1/2}\right)^{1/2}\right]. \tag*{(59)} W 2 , G 2 = ∥ μ out − μ ref ∥ 2 + tr [ Σ out + Σ ref − 2 ( Σ ref 1/2 Σ out Σ ref 1/2 ) 1/2 ] . (59)
where the matrix square roots are positive semidefinite. This is the squared W 2 W_2 W 2 distance between the Gaussian distributions with the estimated and the reference moments [34 ] ; for non-Gaussian output it does not measure the full distributional error.
As a baseline, we ran the preconditioned Crank–Nicolson (pCN) sampler [35 ] with the closed-form ABC likelihood for 100000 steps, discarding the first 20000 and retaining every twentieth subsequent state. Because this baseline uses the closed-form likelihood, it is not an equal-cost, simulator-only comparison; the reference moments are taken from the closed-form posterior, not from the pCN output.
For likelihood-free SFS, we used N = 1000 N=1000 N = 1000 outputs, K = 80 K=80 K = 80 steps, and m ∈ { 64 , 128 , 256 } m\in\{64,128,256\} m ∈ { 64 , 128 , 256 } pairs per drift evaluation. The grid (III.6) had T = 1 T=1 T = 1 and
t k = 1 − ( 1 − k K ) 3 , k = 0 , … , K . (60) t_k=1-\left(1-\frac{k}{K}\right)^3,\qquad k=0,\ldots,K. \tag*{(60)} t k = 1 − ( 1 − K k ) 3 , k = 0 , … , K . (60)
so the steps shorten near time one, unlike the uniform grid of Theorem IV.1. Table I reports the resulting posterior moment errors, averaged over five independent repetitions.
Table I. Posterior moment errors for likelihood-free SFS in the Gaussian ABC model
As Table I and Figure 1 (b) show, both moment errors decreased with m m m , and RQMC gave lower errors than MC for every m m m ; neither reached the accuracy of the pCN baseline, which uses the closed-form likelihood. With K K K and N N N fixed, these errors combine the drift-estimation error, the time-discretization error, and the Monte Carlo error due to the finite number N N N of outputs. Increasing m m m reduces only the drift-estimation error, so the comparison does not isolate a discretization or terminal convergence rate.
Figure 1. Gaussian ABC validation with variance bandwidth δ = 0.25 \delta= 0.25 δ = 0.25 . (a) Analytic first-coordinate posterior density and kernel density estimates from one run per SFS method (N = 3000 N = 3000 N = 3000 , K = 80 K = 80 K = 80 , m = 256 m = 256 m = 256 ). SFS with the closed-form likelihood uses the weights L A B C ( Z i ) L_{\mathrm{ABC}}(Z_i) L ABC ( Z i ) at Gaussian samples Z i Z_i Z i , rather than the exact drift. (b) Mean normalized posterior-location error E μ E_\mu E μ versus the sample size m m m for likelihood-free MC–SFS and RQMC–SFS (N = 1000 N = 1000 N = 1000 , K = 80 K = 80 K = 80 ). Points are means and error bars are sample standard deviations across five independent repetitions. Both panels use (V.10).
Figure 1 (a) compares the analytic first-coordinate marginal N ( 4 / 3 , 1 / 3 ) \mathcal{N}(4/3,1/3) N ( 4/3 , 1/3 ) with three separate SFS runs, each using N = 3000 N = 3000 N = 3000 , K = 80 K = 80 K = 80 , m = 256 m = 256 m = 256 , and (V.10). In each run, the drift is a self-normalized weighted mean of ( Z i − x ) / ( 1 − t ) (Z_i-x)/(1-t) ( Z i − x ) / ( 1 − t ) with Z i = x + 1 − t η i Z_i = x + \sqrt{1-t}\,\eta_i Z i = x + 1 − t η i . For comparison, SFS with the closed-form likelihood uses independent standard Gaussian η i \eta_i η i and weights L A B C ( Z i ) L_{\mathrm{ABC}}(Z_i) L ABC ( Z i ) ; it still estimates the drift by Monte Carlo rather than using (V.6). Likelihood-free MC–SFS uses independent standard Gaussian η i \eta_i η i , ζ i ∈ R 2 \zeta_i \in\mathbb{R}^2 ζ i ∈ R 2 , one output ξ i = Z i + σ ζ i \xi_i = Z_i + \sigma\zeta_i ξ i = Z i + σ ζ i per Z i Z_i Z i , and weights ψ ( Z i , ξ i ) \psi(Z_i,\xi_i) ψ ( Z i , ξ i ) . RQMC–SFS uses the same weights with a four-dimensional Sobol rule, scrambled independently at each drift evaluation and mapped through the inverse normal distribution function. All Brownian increments are pseudorandom. The kernel density estimates use a Gaussian kernel with bandwidth 1.06 s N − 1 / 5 1.06\,sN^{-1/5} 1.06 s N − 1/5 , where s s s is the sample standard deviation of the run. These runs are separate from the five-repetition comparison in panel (b).
For likelihood-free SFS with sample reuse, we generated M ∈ { 1024 , 4096 , 16384 } M \in\{1024, 4096, 16384\} M ∈ { 1024 , 4096 , 16384 } pairs using scrambled Sobol points. The proposals follow (III.20), with M 0 = 4096 M_0 = 4096 M 0 = 4096 independent preliminary samples for fitting q 2 q_2 q 2 and q 3 q_3 q 3 . Each configuration used N = 1000 N = 1000 N = 1000 trajectories on the uniform grid (III.6) with T = 1 T = 1 T = 1 and K = 80 K = 80 K = 80 , and was repeated five times.
In each repetition, the drift RMSE is the square root of the squared Euclidean error relative to the analytic drift (V.6), averaged over all N N N trajectories and all update times t 0 , … , t K − 1 t_0,\ldots,t_{K-1} t 0 , … , t K − 1 . For each M M M and proposal, Table II reports the moment errors, W 2 , G W_{2,G} W 2 , G , this drift RMSE averaged over the five repetitions, and the number of simulator calls.
Table 2. Gaussian ABC: Errors and simulator calls for likelihood-free SFS with sample reuse; calls include proposal fitting where applicable
At M = 4096 M = 4096 M = 4096 , q 2 q_2 q 2 and q 3 q_3 q 3 reduced W 2 , G W_{2,G} W 2 , G from 0.0478 to 0.0302 and 0.0299, respectively (about 37%), and the drift RMSE from 0.6492 to 0.2966 and 0.3584. Increasing M M M reduced the drift RMSE for every proposal, but the terminal errors did not decrease monotonically: for the fitted proposals, W 2 , G W_{2,G} W 2 , G did not decrease from M = 4096 M = 4096 M = 4096 to M = 16384 M = 16384 M = 16384 . Since the drift errors are measured along each method’s own trajectories, they do not compare estimates at common states. Figure 2 shows the corresponding terminal samples at M = 4096 M = 4096 M = 4096 for the three proposals, together with the 95% probability contour of the analytic posterior.
Figure 2. Terminal samples for the three proposals at M = 4096 M = 4096 M = 4096 . Black ellipses show the analytic posterior’s 95% probability contour; crosses mark its mean.
Likelihood-free SFS with m = 128 m = 128 m = 128 used N K m = 10 240 000 NKm = 10\,240\,000 N K m = 10 240 000 simulator calls, compared with M 0 + M = 8192 M_0 + M = 8192 M 0 + M = 8192 for sample reuse with M = 4096 M = 4096 M = 4096 and either fitted proposal; q 1 q_1 q 1 requires no preliminary samples. These counts measure simulator savings, not reductions of the same size in arithmetic or runtime, since Ω q ( Z i , ξ i ; x , t ) \Omega_q(Z_i, \xi_i; x, t) Ω q ( Z i , ξ i ; x , t ) must be recomputed at each state and time. Because the samples are scrambled Sobol points rather than iid draws, Theorems IV.3 and IV.2 do not directly cover this experiment.
To examine a concentrated posterior, we next let each simulator call generate n = 50 n = 50 n = 50 independent bivariate observations ξ 1 , … , ξ n \xi_1, \ldots, \xi_n ξ 1 , … , ξ n , so that each simulated data set has 2 n = 100 2n = 100 2 n = 100 coordinates, while keeping two parameters and the two-dimensional summary ξ ˉ = n − 1 ∑ j = 1 n ξ j \bar{\xi} = n^{-1}\sum_{j=1}^{n}\xi_j ξ ˉ = n − 1 ∑ j = 1 n ξ j . With observed mean y 0 = ( 2 , − 2 ) T y_0 = (2, -2)^{\mathsf{T}} y 0 = ( 2 , − 2 ) T ,
ξ ˉ ∣ x ∼ N ( x , σ 2 I 2 / n ) . (61) \bar{\xi} \mid x \sim\mathcal{N}\left(x, \sigma^2 I_2/n\right). \tag*{(61)} ξ ˉ ∣ x ∼ N ( x , σ 2 I 2 / n ) . (61)
With the bandwidth δ S = δ / n = 0.005 \delta_S = \delta/n = 0.005 δ S = δ / n = 0.005 for the summary, the variance v = σ 2 + δ v = \sigma^2 + \delta v = σ 2 + δ in the posterior formula is replaced by ( σ 2 + δ ) / n = v / n (\sigma^2 + \delta)/n = v/n ( σ 2 + δ ) / n = v / n , which gives
μ 100 = n n + v y 0 = ( 1.980198 , − 1.980198 ) T , (62) \mu_{100} = \frac{n}{n+v}y_0 = (1.980198, -1.980198)^{\mathsf{T}}, \tag*{(62)} μ 100 = n + v n y 0 = ( 1.980198 , − 1.980198 ) T , (62)
Σ 100 = v n + v I 2 = 0.00990099 I 2 . (63) \Sigma_{100} = \frac{v}{n+v}I_2 = 0.00990099I_2. \tag*{(63)} Σ 100 = n + v v I 2 = 0.00990099 I 2 . (63)
Table III. Posterior errors for n = 50 n = 50 n = 50 bivariate observations (100 data coordinates) and the summary ξ ˉ ∈ R 2 \bar{\xi} \in\mathbb{R}^2 ξ ˉ ∈ R 2
The subscript 100 refers to the 2 n = 100 2n = 100 2 n = 100 data coordinates, not to the number of parameters. We used N = 1000 N = 1000 N = 1000 , m = 128 m = 128 m = 128 , M = M 0 = 4096 M = M_0 = 4096 M = M 0 = 4096 , and five repetitions on the uniform grid (III.6) with T = 1 T = 1 T = 1 and K = 80 K = 80 K = 80 .
Table III compares the posterior errors and simulator calls of all methods, and Figure 3 shows the corresponding posterior projections against the closed-form 95% ellipses.
Figure 3. Posterior projections for n = 50 n = 50 n = 50 bivariate observations (100 data coordinates). Reusing samples drawn from fitted proposals improves centering, while the uniform grid leaves visible variance inflation relative to the closed-form 95% ellipses.
The fitted proposals improved the centering, but the relative covariance error remained about 0.84. Part of this error is due to the time discretization. With h = Δ t k = 1 / 80 h = \Delta t_k = 1/80 h = Δ t k = 1/80 , the final Euler update is
X ~ 1 = X ~ 1 p r e + h η K , X ~ 1 p r e = X ~ 1 − h + h b ^ ( X ~ 1 − h , 1 − h ) , (64) \widetilde{X}_1 = \widetilde{X}_1^{\mathrm{pre}} + \sqrt{h}\,\eta_K,\qquad\widetilde{X}_1^{\mathrm{pre}} = \widetilde{X}_{1-h} + h\hat{b}(\widetilde{X}_{1-h},1-h), \tag*{(64)} X 1 = X 1 pre + h η K , X 1 pre = X 1 − h + h b ^ ( X 1 − h , 1 − h ) , (64)
where b ^ \hat{b} b ^ is the drift estimator used and η K ∼ N ( 0 , I 2 ) \eta_K \sim\mathcal{N}(0,I_2) η K ∼ N ( 0 , I 2 ) is independent of the preceding state, the samples, and the drift estimate. If second moments exist, then
Cov ( X ~ 1 ) = Cov ( X ~ 1 p r e ) + h I 2 ⪰ h I 2 = 0.0125 I 2 . (65) \operatorname{Cov}(\widetilde{X}_1) = \operatorname{Cov}(\widetilde{X}_1^{\mathrm{pre}}) + hI_2 \succeq hI_2 = 0.0125I_2. \tag*{(65)} Cov ( X 1 ) = Cov ( X 1 pre ) + h I 2 ⪰ h I 2 = 0.0125 I 2 . (65)
This lower bound exceeds Σ 100 = 0.00990099 I 2 \Sigma_{100} = 0.00990099I_2 Σ 100 = 0.00990099 I 2 , so on this grid the covariance of the Euler output cannot equal the target covariance, even with the exact drift. The bound identifies a discretization contribution that more reused samples cannot remove; it does not account for the entire observed covariance error.
This experiment varies the posterior concentration, not the parameter dimension, and therefore does not test the dependence on d d d in the theory.
M/G/1 queueing model
The M/G/1 benchmark [36 ] tests likelihood-free integration with a piecewise-smooth simulator and high-dimensional random input. For an initially empty, single-server, first-come-first-served queue, set a 0 = d 0 = 0 a_0 = d_0 = 0 a 0 = d 0 = 0 and
T i s v c = θ 1 + θ 2 U i , U i ∼ Unif ( 0 , 1 ) , T i i a = − log ( 1 − U i ′ ) / θ 3 , U i ′ ∼ Unif ( 0 , 1 ) , a i = a i − 1 + T i i a , d i = max ( a i , d i − 1 ) + T i s v c , Y i = d i − d i − 1 . (66) \begin{aligned}
T_i^{\mathrm{svc}} &= \theta_1 + \theta_2 U_i, \qquad U_i \sim\operatorname{Unif}(0,1),\\
T_i^{\mathrm{ia}} &= -\log(1-U_i^{\prime})/\theta_3, \qquad U_i^{\prime} \sim\operatorname{Unif}(0,1),\\
a_i &= a_{i-1} + T_i^{\mathrm{ia}}, \qquad d_i = \max(a_i,d_{i-1}) + T_i^{\mathrm{svc}},\\
Y_i &= d_i-d_{i-1}. \tag*{(66)}
\end{aligned} T i svc T i ia a i Y i = θ 1 + θ 2 U i , U i ∼ Unif ( 0 , 1 ) , = − log ( 1 − U i ′ ) / θ 3 , U i ′ ∼ Unif ( 0 , 1 ) , = a i − 1 + T i ia , d i = max ( a i , d i − 1 ) + T i svc , = d i − d i − 1 . (66)
All uniforms are mutually independent, and T i s v c T_i^{\mathrm{svc}} T i svc , T i i a T_i^{\mathrm{ia}} T i ia , a i a_i a i , d i d_i d i , and Y i Y_i Y i denote the service, interarrival, arrival, departure, and interdeparture times, respectively. Service times are uniform on ( θ 1 , θ 1 + θ 2 ) (\theta_1,\theta_1+\theta_2) ( θ 1 , θ 1 + θ 2 ) , so θ 2 \theta_2 θ 2 is the interval width, and θ 3 \theta_3 θ 3 is the arrival rate. The data are 50 interdeparture times simulated at θ = ( 1 , 4 , 0.2 ) \theta= (1,4,0.2) θ = ( 1 , 4 , 0.2 ) , and the priors are independent, with θ 1 , θ 2 ∼ Unif ( 0 , 10 ) \theta_1,\theta_2 \sim\operatorname{Unif}(0,10) θ 1 , θ 2 ∼ Unif ( 0 , 10 ) and θ 3 ∼ Unif ( 0 , 1 / 3 ) \theta_3 \sim\operatorname{Unif}(0,1/3) θ 3 ∼ Unif ( 0 , 1/3 ) . The likelihood is that of this finite sequence from an initially empty queue, not of a stationary queue. SFS runs in standard Gaussian coordinates x ∈ R 3 x \in\mathbb{R}^3 x ∈ R 3 , mapped to the original parameters θ \theta θ by
θ = F ( x ) = ( 10 Φ N ( x 1 ) , 10 Φ N ( x 2 ) , Φ N ( x 3 ) / 3 ) T , (67) \theta= F(x) = \left(10\Phi_N(x_1),10\Phi_N(x_2),\Phi_N(x_3)/3\right)^{\mathsf{T}}, \tag*{(67)} θ = F ( x ) = ( 10 Φ N ( x 1 ) , 10 Φ N ( x 2 ) , Φ N ( x 3 ) /3 ) T , (67)
By the probability integral transform, x ∼ N ( 0 , I 3 ) x \sim\mathcal{N}(0,I_3) x ∼ N ( 0 , I 3 ) yields the stated priors.
We used the five-dimensional summary
S ( Y ) = ( log Q 0 , log Q 0.25 , log Q 0.5 , log Q 0.75 , log Q 1 ) T , (68) S(Y) = \left(\log Q_0,\log Q_{0.25},\log Q_{0.5},\log Q_{0.75},\log Q_1\right)^{\mathsf{T}}, \tag*{(68)} S ( Y ) = ( log Q 0 , log Q 0.25 , log Q 0.5 , log Q 0.75 , log Q 1 ) T , (68)
where Q p Q_p Q p is the empirical p p p -quantile of the 50 interdeparture times. The observed summary and componentwise scales were
s o b s = ( 0.0929 , 0.8333 , 1.4484 , 1.9773 , 3.1510 ) T , s_{\mathrm{obs}} = (0.0929,0.8333,1.4484,1.9773,3.1510)^{\mathsf{T}}, s obs = ( 0.0929 , 0.8333 , 1.4484 , 1.9773 , 3.1510 ) T ,
s s c = ( 0.1049 , 0.1336 , 0.1006 , 0.1893 , 0.2918 ) T , D s c = diag ( s s c ) . (69) s_{\mathrm{sc}} = (0.1049,0.1336,0.1006,0.1893,0.2918)^{\mathsf{T}}, \qquad D_{\mathrm{sc}} = \operatorname{diag}(s_{\mathrm{sc}}). \tag*{(69)} s sc = ( 0.1049 , 0.1336 , 0.1006 , 0.1893 , 0.2918 ) T , D sc = diag ( s sc ) . (69)
Let ω = ( U 1 , U 1 ′ , … , U 50 , U 50 ′ ) \omega= (U_1,U'_1,\ldots,U_{50},U'_{50}) ω = ( U 1 , U 1 ′ , … , U 50 , U 50 ′ ) be the 100-dimensional simulator input, playing the role of ξ \xi ξ in Section II. The ABC weight is
ψ ( x , ω ) = exp [ − ∥ D s c − 1 { S ( Y ( F ( x ) , ω ) ) − s o b s } ∥ 2 2 δ ] . (70) \psi(x,\omega) = \exp\left[-\frac{\left\|D_{\mathrm{sc}}^{-1}\{S(Y(F(x),\omega))-s_{\mathrm{obs}}\}\right\|^2}{2\delta}\right]. \tag*{(70)} ψ ( x , ω ) = exp [ − 2 δ D sc − 1 { S ( Y ( F ( x ) , ω )) − s obs } 2 ] . (70)
Thus L A B C ( x ) = E ω [ ψ ( x , ω ) ] L_{\mathrm{ABC}}(x)=\mathbb{E}_{\omega}[\psi(x,\omega)] L ABC ( x ) = E ω [ ψ ( x , ω )] , with parameter dimension d = 3 d=3 d = 3 and summary dimension r = 5 r=5 r = 5 . The bandwidth was calibrated on 65536 RQMC points from the prior so that the ESS of the ABC weights equals 5% of the number of points, which gives δ = 7.211990 \delta=7.211990 δ = 7.211990 ; the fractions 0.02 and 0.10 would give δ = 3.307955 \delta=3.307955 δ = 3.307955 and 15.028490. All reported comparisons use δ = 7.211990 \delta=7.211990 δ = 7.211990 . The ESS fraction measures weight concentration rather than an ABC acceptance rate; once chosen, the bandwidth and the scaling fix the ABC target.
The maximum and quantile operations make the simulator output only piecewise smooth in θ \theta θ , although the integrated likelihood may still be smooth. Assumption IV.1 and the moment conditions (A.13) have not been verified, so Theorem IV.1 is not known to apply; the comparisons also do not test the ABC bandwidth limit or the full-data posterior.
The numerical reference was computed from 524288 RQMC points drawn from the prior, retaining 32768 weighted samples. Two independent reference sets had ESS 26082 and 25944, a sliced Wasserstein discrepancy of 0.0734 between them, and a classifier two-sample test accuracy near 0.5. These checks indicate reproducibility but do not make the reference exact. Proposal fitting used M 0 = 16384 M_0=16384 M 0 = 16384 preliminary points with ESS 821.46, giving
μ ^ = ( − 1.28810 , − 0.06673 , 0.41174 ) T , \hat{\mu} = (-1.28810,-0.06673,0.41174)^{\mathsf{T}}, μ ^ = ( − 1.28810 , − 0.06673 , 0.41174 ) T ,
( ( Σ ^ q 2 ) 11 , ( Σ ^ q 2 ) 22 , ( Σ ^ q 2 ) 33 ) = ( 0.06398 , 0.88107 , 0.46020 ) . (71) \left((\hat{\Sigma}_{q_2})_{11},(\hat{\Sigma}_{q_2})_{22},(\hat{\Sigma}_{q_2})_{33}\right) = (0.06398,0.88107,0.46020). \tag*{(71)} ( ( Σ ^ q 2 ) 11 , ( Σ ^ q 2 ) 22 , ( Σ ^ q 2 ) 33 ) = ( 0.06398 , 0.88107 , 0.46020 ) . (71)
Here μ ^ \widehat{\mu} μ is fitted as in § 3 -E, and Σ ^ q 2 \widehat{\Sigma}_{q_2} Σ q 2 is the proposal covariance, not a posterior covariance estimate; the proposals are defined in (III.20). Each combination of proposal and M M M used N = 384 N = 384 N = 384 outputs, the uniform grid (III.6) with T = 1 T = 1 T = 1 and K = 64 K = 64 K = 64 , and twelve repetitions. Error bars in the figures are 1.96 standard errors of the mean; errors of the terminal distribution are reported as mean ± \pm ± standard deviation. Each RQMC point has 3 + 100 = 103 3 + 100 = 103 3 + 100 = 103 coordinates, or 104 for q 3 q_3 q 3 , which uses one more coordinate to select the mixture component; this is much larger than the parameter dimension d = 3 d = 3 d = 3 .
Let ( μ r e f θ , Σ r e f θ ) (\mu^\theta_{\mathrm{ref}}, \Sigma^\theta_{\mathrm{ref}}) ( μ ref θ , Σ ref θ ) be the numerical reference moments in the original coordinates θ = F ( x ) \theta= F(x) θ = F ( x ) , and set D r e f = diag ( ( Σ r e f θ ) 11 , … , ( Σ r e f θ ) d d ) D_{\mathrm{ref}} = \operatorname{diag}\left(\sqrt{(\Sigma^\theta_{\mathrm{ref}})_{11}}, \ldots, \sqrt{(\Sigma^\theta_{\mathrm{ref}})_{dd}}\right) D ref = diag ( ( Σ ref θ ) 11 , … , ( Σ ref θ ) dd ) . Both sliced W 2 W_2 W 2 [37 ] and W 2 , G W_{2,G} W 2 , G in (V.9) are computed in the standardized coordinates D r e f − 1 ( θ − μ r e f θ ) D_{\mathrm{ref}}^{-1}(\theta- \mu^\theta_{\mathrm{ref}}) D ref − 1 ( θ − μ ref θ ) for both output and reference samples. Sliced W 2 W_2 W 2 uses 384 random projection directions with 384 samples per direction. For the output moments ( μ ^ o u t θ , Σ ^ o u t θ ) (\widehat{\mu}^\theta_{\mathrm{out}}, \widehat{\Sigma}^\theta_{\mathrm{out}}) ( μ out θ , Σ out θ ) , define
E μ , θ = ∥ D r e f − 1 ( μ ^ o u t θ − μ r e f θ ) ∥ d , E Σ , θ = ∥ D r e f − 1 ( Σ ^ o u t θ − Σ r e f θ ) D r e f − 1 ∥ F ∥ D r e f − 1 Σ r e f θ D r e f − 1 ∥ F . (72) \begin{aligned}
E_{\mu,\theta} &= \frac{\left\|D_{\mathrm{ref}}^{-1}\left(\widehat{\mu}^\theta_{\mathrm{out}}-\mu^\theta_{\mathrm{ref}}\right)\right\|}{\sqrt{d}}, \\
E_{\Sigma,\theta} &= \frac{\left\|D_{\mathrm{ref}}^{-1}\left(\widehat{\Sigma}^\theta_{\mathrm{out}}-\Sigma^\theta_{\mathrm{ref}}\right)D_{\mathrm{ref}}^{-1}\right\|_F}{\left\|D_{\mathrm{ref}}^{-1}\Sigma^\theta_{\mathrm{ref}}D_{\mathrm{ref}}^{-1}\right\|_F}. \tag*{(72)}
\end{aligned} E μ , θ E Σ , θ = d D ref − 1 ( μ out θ − μ ref θ ) , = D ref − 1 Σ ref θ D ref − 1 F D ref − 1 ( Σ out θ − Σ ref θ ) D ref − 1 F . (72)
Here d = 3 d = 3 d = 3 ; this coordinatewise standardization differs from the normalization in § 5 -A. These projection and moment diagnostics differ from the full W 2 W_2 W 2 distance in Theorem IV.1; moreover, they concern the distribution of θ = F ( x ) \theta= F(x) θ = F ( x ) , whereas the theorem uses Gaussian coordinates.
For the drift comparisons, we fixed six points x j x_j x j in Gaussian coordinates and evaluated the drift at the states t x j \sqrt{t}x_j t x j for
t ∈ { 0 , 0.25 , 0.5 , 0.75 , 0.9 , 0.975 } . (73) t \in\{0, 0.25, 0.5, 0.75, 0.9, 0.975\}. \tag*{(73)} t ∈ { 0 , 0.25 , 0.5 , 0.75 , 0.9 , 0.975 } . (73)
These 36 state–time pairs are prescribed rather than taken from SFS trajectories. Two independent RQMC reference estimates, each with 65536 points per evaluation, differed by 0.01476 on average; this indicates the numerical uncertainty of the reference but is not a rigorous error bound. MC and RQMC used m m m from 32 to 2048, with 24 repetitions per setting. Write ( A r e f , C r e f , b r e f ) (A_{\mathrm{ref}}, C_{\mathrm{ref}}, b_{\mathrm{ref}}) ( A ref , C ref , b ref ) for the reference values of the numerator, denominator, and drift, and ( A ^ , C ^ , b ^ ) (\widehat{A}, \widehat{C}, \widehat{b}) ( A , C , b ) for the corresponding estimates in (III.12). In each repetition, the RMSEs are root mean squares over the 36 evaluations of
∥ A ^ − A r e f ∥ C r e f , ∣ C ^ C r e f − 1 ∣ , ∥ b ^ − b r e f ∥ . (74) \frac{\left\|\widehat{A}-A_{\mathrm{ref}}\right\|}{C_{\mathrm{ref}}}, \qquad\left|\frac{\widehat{C}}{C_{\mathrm{ref}}}-1\right|, \qquad\left\|\widehat{b}-b_{\mathrm{ref}}\right\|. \tag*{(74)} C ref A − A ref , C ref C − 1 , b − b ref . (74)
The exponents γ \gamma γ in Table IV are obtained by fitting error ( m ) ≈ c m − γ \operatorname{error}(m) \approx c m^{-\gamma} error ( m ) ≈ c m − γ over the tested sizes; they are empirical, not asymptotic, rates.
Table IV. Empirical error-decay exponents and errors at m = 2048 m = 2048 m = 2048 for M/G/1 drift estimation, with errors defined by (V.23). Exponents are the fitted decay rates γ > 0 \gamma> 0 γ > 0 .
RQMC gave larger exponents for all three errors and reduced the drift error by 47.4% at m = 2048 m = 2048 m = 2048 . For both methods, the exponent for the ratio was smaller than those for the numerator and denominator. Identity (A.10) shows that the ratio error depends jointly on both integration errors and on the random denominator, but it does not predict this difference in fitted exponents. Comparisons at finitely many states establish neither the uniform moment conditions (A.13) nor uniform error control along trajectories with sample reuse.
Figure 4 shows the three errors over all tested sample sizes m m m , from which the exponents in Table IV are fitted.
Figure 4. M/G/1 drift-estimation errors versus the sample size m m m , using (V.23) over the 36 prescribed evaluations. Panels show (a) numerator error scaled by the reference denominator, (b) relative denominator error, and (c) self-normalized drift error. Curves are means across 24 independent repetitions; shaded bands have half-width 1.96 times the standard error of those means. Vertical axes are logarithmic.
The terminal comparisons used RQMC samples with M ∈ { 512 , 1024 , 2048 , 4096 } M \in\{512, 1024, 2048, 4096\} M ∈ { 512 , 1024 , 2048 , 4096 } for all three proposals, together with independent MC sample sets for q 2 q_2 q 2 and q 3 q_3 q 3 at M = 4096 M = 4096 M = 4096 . Tables V and VI report the posterior moments of θ \theta θ and the errors of the terminal distribution.
At M = 4096 M = 4096 M = 4096 , RQMC with q 2 q_2 q 2 and q 3 q_3 q 3 increased the local ESS (III.21) at the last drift evaluation about 7.5- and 7.2-fold relative to q 1 q_1 q 1 , yet q 2 q_2 q 2 reduced the sliced W 2 W_2 W 2 error by only about 3%, compared with 23.4% at M = 512 M = 512 M = 512 . More balanced local weights thus did not yield a proportional terminal improvement.
Table V. M/G/1 posterior means and standard deviations of θ \theta θ at M = 4096 M = 4096 M = 4096 . Standard deviations are in parentheses.
Table VI. M/G/1 errors of the terminal distribution at M = 4096 M = 4096 M = 4096 . Both distances are computed for D r e f − 1 ( θ − μ r e f θ ) D_{\mathrm{ref}}^{-1}(\theta-\mu_{\mathrm{ref}}^\theta) D ref − 1 ( θ − μ ref θ ) . Sliced W 2 W_2 W 2 is reported as mean ± \pm ± standard deviation across repetitions; local ESS is averaged at the final drift evaluation. Calls include the preliminary samples but exclude bandwidth calibration.
Local ESS concerns the conditional weights, not the overall terminal importance weights or the number of independent outputs (Remark B.1).
Two-sided paired t t t -tests on the sliced-W 2 W_2 W 2 errors over the 12 matched RQMC and MC repetitions gave p = 0.823 p = 0.823 p = 0.823 for q 2 q_2 q 2 and p = 0.609 p = 0.609 p = 0.609 for q 3 q_3 q 3 ; no terminal difference was detected, although this does not establish equivalence. Thus RQMC improved drift estimation but showed no terminal advantage here, where the error of the numerical reference, the time-discretization error, the Monte Carlo error due to finitely many outputs, and the error from sharing one sample set B M \mathcal{B}_M B M among all trajectories also contribute.
For the fitted proposals, the call counts include the M 0 = 16384 M_0 = 16384 M 0 = 16384 preliminary samples. Adding the 65536 bandwidth-calibration calls gives 69632 calls for q 1 q_1 q 1 and 86016 for q 2 q_2 q 2 or q 3 q_3 q 3 at M = 4096 M = 4096 M = 4096 , excluding the reference computation.
Figure 5 compares the marginal densities and pairwise projections of the SFS output at M = 4096 M = 4096 M = 4096 with the numerical reference, and Figure 6 shows how the terminal errors change with the number M M M of reused samples.
Figure 5. M/G/1 marginal densities and pairwise projections at M = 4096 M = 4096 M = 4096 . The numerical ABC reference is compared with SFS output using sample reuse with proposals q 2 q_2 q 2 and q 3 q_3 q 3 .
Figure 6. M/G/1 terminal errors versus the number of reused samples. Sliced W 2 W_2 W 2 is computed for D r e f − 1 ( θ − μ r e f θ ) D_{\mathrm{ref}}^{-1}(\theta-\mu_{\mathrm{ref}}^\theta) D ref − 1 ( θ − μ ref θ ) ; mean and covariance errors are defined in (V.21). Error bars are 1.96 standard errors of the mean over 12 repetitions. Light purple, dark blue, and green curves correspond to q 1 q_1 q 1 , q 2 q_2 q 2 , and q 3 q_3 q 3 , respectively.
Ohio random-intercept logistic model
The geepack::ohio data contain binary wheezing responses for 537 children at four visits each, ages 7–10, giving 2148 records [38 ] , [39 ] . This example uses a randomized likelihood estimator without ABC smoothing. Ages were centered at 9, so A i j ∈ { − 2 , − 1 , 0 , 1 } A_{ij} \in\{-2,-1,0,1\} A ij ∈ { − 2 , − 1 , 0 , 1 } ; the maternal-smoking and wheezing proportions are 0.34823 and 0.15177. For child i i i and visit j j j , we use the random-intercept logistic model, a generalized linear mixed model (GLMM),
Y i j ∣ β , α i ∼ Bernoulli ( p i j ) , logit ( p i j ) = β 1 + β 2 A i j + β 3 S i + α i , α i ∣ τ ∼ N ( 0 , τ 2 ) , β ∼ N ( 0 , 50 I 3 ) , τ ∼ Gamma ( 1 , 0.1 ) . (75) \begin{aligned}
Y_{ij} \mid\beta,\alpha_i &\sim\operatorname{Bernoulli}(p_{ij}), \\
\operatorname{logit}(p_{ij}) &= \beta_1 + \beta_2 A_{ij} + \beta_3 S_i + \alpha_i, \\
\alpha_i \mid\tau&\sim\mathcal{N}(0,\tau^2), \\
\beta&\sim\mathcal{N}(0,50I_3), \qquad\tau\sim\operatorname{Gamma}(1,0.1). \tag*{(75)}
\end{aligned} Y ij ∣ β , α i logit ( p ij ) α i ∣ τ β ∼ Bernoulli ( p ij ) , = β 1 + β 2 A ij + β 3 S i + α i , ∼ N ( 0 , τ 2 ) , ∼ N ( 0 , 50 I 3 ) , τ ∼ Gamma ( 1 , 0.1 ) . (75)
Responses are conditionally independent given the parameters and random effects, and the random effects are independent across children. Here S i S_i S i is the maternal-smoking indicator (not a summary map), and β 1 \beta_1 β 1 , β 2 \beta_2 β 2 , and β 3 \beta_3 β 3 are the intercept, age effect, and smoking effect. The Gamma prior is parameterized by shape and rate and is placed on the standard deviation τ \tau τ , not on τ 2 \tau^2 τ 2 . SFS runs in x ∈ R 4 x \in\mathbb{R}^4 x ∈ R 4 , with
β j = 50 x j ( j = 1 , 2 , 3 ) , τ = − 10 log { 1 − Φ N ( x 4 ) } . (76) \beta_j = \sqrt{50}x_j \quad(j=1,2,3), \qquad\tau= -10\log\{1-\Phi_N(x_4)\}. \tag*{(76)} β j = 50 x j ( j = 1 , 2 , 3 ) , τ = − 10 log { 1 − Φ N ( x 4 )} . (76)
Since Gamma( 1 , 0.1 ) (1,0.1) ( 1 , 0.1 ) is the exponential distribution with rate 0.1, the second map in (V.25) gives the stated prior on τ \tau τ . In this section, τ \tau τ denotes the random-effect scale, not the remaining diffusion time 1 − t 1-t 1 − t . The 537 random intercepts are integrated out, so the state dimension is d = 4 d=4 d = 4 .
For fixed x x x , with β \beta β and τ \tau τ defined by (V.25), set
η i j ( a ) = β 1 + β 2 A i j + β 3 S i + a , p i j ( a ) = { 1 + exp [ − η i j ( a ) ] } − 1 , g i ( a ; x ) = exp { ∑ j = 1 4 [ Y i j η i j ( a ) − log ( 1 + e η i j ( a ) ) ] } . (77) \begin{aligned}
\eta_{ij}(a) &= \beta_1+\beta_2 A_{ij}+\beta_3 S_i+a,\\
p_{ij}(a) &= \{1+\exp[-\eta_{ij}(a)]\}^{-1},\\
g_i(a;x) &= \exp\left\{\sum_{j=1}^{4}[Y_{ij}\eta_{ij}(a)-\log(1+e^{\eta_{ij}(a)})]\right\}. \tag*{(77)}
\end{aligned} η ij ( a ) p ij ( a ) g i ( a ; x ) = β 1 + β 2 A ij + β 3 S i + a , = { 1 + exp [ − η ij ( a )] } − 1 , = exp { j = 1 ∑ 4 [ Y ij η ij ( a ) − log ( 1 + e η ij ( a ) )] } . (77)
Integrating out each random intercept gives
L i ( x ) = ∫ R g i ( a ; x ) φ τ 2 ( a ) d a , L ( x ) = ∏ i = 1 537 L i ( x ) . (78) L_i(x)=\int_{\mathbb{R}}g_i(a;x)\varphi_{\tau^2}(a)\,da,\qquad L(x)=\prod_{i=1}^{537}L_i(x). \tag*{(78)} L i ( x ) = ∫ R g i ( a ; x ) φ τ 2 ( a ) d a , L ( x ) = i = 1 ∏ 537 L i ( x ) . (78)
where φ τ 2 \varphi_{\tau^2} φ τ 2 is the N ( 0 , τ 2 ) \mathcal{N}(0,\tau^2) N ( 0 , τ 2 ) density. To place importance samples near the mode of each integrand, we write its logarithm, up to an additive constant independent of a a a , as
ℓ i ( a ) = ∑ j = 1 4 [ Y i j η i j ( a ) − log ( 1 + e η i j ( a ) ) ] − a 2 2 τ 2 . (79) \ell_i(a) = \sum_{j=1}^{4}\left[Y_{ij}\eta_{ij}(a)-\log\left(1+e^{\eta_{ij}(a)}\right)\right]-\frac{a^2}{2\tau^2}. \tag*{(79)} ℓ i ( a ) = j = 1 ∑ 4 [ Y ij η ij ( a ) − log ( 1 + e η ij ( a ) ) ] − 2 τ 2 a 2 . (79)
Since ∂ a log ( 1 + e η i j ( a ) ) = p i j ( a ) \partial_a\log\left(1+e^{\eta_{ij}(a)}\right)=p_{ij}(a) ∂ a log ( 1 + e η ij ( a ) ) = p ij ( a ) and ∂ a p i j ( a ) = p i j ( a ) { 1 − p i j ( a ) } \partial_a p_{ij}(a)=p_{ij}(a)\{1-p_{ij}(a)\} ∂ a p ij ( a ) = p ij ( a ) { 1 − p ij ( a )} ,
ℓ i ′ ( a ) = ∑ j = 1 4 [ Y i j − p i j ( a ) ] − a / τ 2 , (80) \ell_i'(a) = \sum_{j=1}^{4}\left[Y_{ij}-p_{ij}(a)\right]-a/\tau^2, \tag*{(80)} ℓ i ′ ( a ) = j = 1 ∑ 4 [ Y ij − p ij ( a ) ] − a / τ 2 , (80)
− ℓ i ′ ′ ( a ) = ∑ j = 1 4 p i j ( a ) [ 1 − p i j ( a ) ] + τ − 2 > 0. (81) -\ell_i''(a) = \sum_{j=1}^{4}p_{ij}(a)\left[1-p_{ij}(a)\right]+\tau^{-2}>0. \tag*{(81)} − ℓ i ′′ ( a ) = j = 1 ∑ 4 p ij ( a ) [ 1 − p ij ( a ) ] + τ − 2 > 0. (81)
Since ℓ i \ell_i ℓ i is strictly concave, its mode can be computed by Newton’s method, started at zero with iterates clipped to [ − 5 , 5 ] [-5,5] [ − 5 , 5 ] . With the resulting estimate a ^ i \widehat a_i a i and c i = − ℓ i ′ ′ ( a ^ i ) > 0 c_i=-\ell_i''(\widehat a_i)>0 c i = − ℓ i ′′ ( a i ) > 0 , we use the proposal
q R E , i ( a ∣ x ) = N ( a ; a ^ i , 1.2 2 / c i ) . (82) q_{\mathrm{RE},i}(a\mid x)=\mathcal{N}\left(a;\widehat a_i,1.2^2/c_i\right). \tag*{(82)} q RE , i ( a ∣ x ) = N ( a ; a i , 1. 2 2 / c i ) . (82)
This one-dimensional proposal for the random effect is distinct from the parameter proposal q q q . Its location and scale follow the Laplace principle [40 ] , with the standard deviation inflated by the factor 1.2; it serves only as an importance proposal, not as a Laplace approximation of the likelihood.
With J = 16 J=16 J = 16 , let α i r = a ^ i + 1.2 c i − 1 / 2 ε i r \alpha_{ir}=\widehat a_i+1.2c_i^{-1/2}\varepsilon_{ir} α i r = a i + 1.2 c i − 1/2 ε i r , where, ideally, the inputs ε i r \varepsilon_{ir} ε i r are independent standard normal variables, and define
L ^ i ( x ) = 1 J ∑ r = 1 J g i ( α i r ; x ) φ τ 2 ( α i r ) q R E , i ( α i r ∣ x ) , \widehat L_i(x)=\frac{1}{J}\sum_{r=1}^{J}\frac{g_i(\alpha_{ir};x)\varphi_{\tau^2}(\alpha_{ir})}{q_{\mathrm{RE},i}(\alpha_{ir}\mid x)}, L i ( x ) = J 1 r = 1 ∑ J q RE , i ( α i r ∣ x ) g i ( α i r ; x ) φ τ 2 ( α i r ) ,
ψ ( x , ω ) = ∏ i = 1 537 L ^ i ( x ) . (83) \psi(x,\omega)=\prod_{i=1}^{537}\widehat L_i(x). \tag*{(83)} ψ ( x , ω ) = i = 1 ∏ 537 L i ( x ) . (83)
The vector ω \omega ω collects the underlying uniform inputs, as in § 5 -B. Since the Gaussian proposal has full support, the importance-sampling identity [19 Chapter 9] gives
E [ L ^ i ( x ) ∣ x ] = ∫ R g i ( a ; x ) φ τ 2 ( a ) q R E , i ( a ∣ x ) q R E , i ( a ∣ x ) d a = L i ( x ) . (84) \begin{aligned}
\mathbb{E}\left[\widehat L_i(x)\mid x\right]=\int_{\mathbb{R}}\frac{g_i(a;x)\varphi_{\tau^2}(a)}{q_{\mathrm{RE},i}(a\mid x)}q_{\mathrm{RE},i}(a\mid x)\,\mathrm{d}a \\
&=L_i(x). \tag*{(84)}
\end{aligned} E [ L i ( x ) ∣ x ] = ∫ R q RE , i ( a ∣ x ) g i ( a ; x ) φ τ 2 ( a ) q RE , i ( a ∣ x ) d a = L i ( x ) . (84)
Since the randomizations are conditionally independent across children,
E [ ψ ( x , ω ) ∣ x ] = ∏ i = 1 537 E [ L ^ i ( x ) ∣ x ] = L ( x ) . (85) \mathbb{E}\left[\psi(x,\omega)\mid x\right]=\prod_{i=1}^{537}\mathbb{E}\left[\widehat L_i(x)\mid x\right]=L(x). \tag*{(85)} E [ ψ ( x , ω ) ∣ x ] = i = 1 ∏ 537 E [ L i ( x ) ∣ x ] = L ( x ) . (85)
Thus the ideal weight is nonnegative and unbiased even if the mode is only approximate, provided the same proposal is used for sampling and for evaluating the weight; the accuracy of the mode affects efficiency and higher moments, but not this identity. Unbiasedness does not extend to log ψ \log\psi log ψ or to the self-normalized drift, and it gives no mean-square bound for the drift. The sums and products are evaluated with log-sum-exp arithmetic for numerical stability.
In fact, the ideal unclipped weight has infinite second moment on an open set of parameters. At x = 0 x = 0 x = 0 , we have β = 0 \beta= 0 β = 0 and τ = 10 log 2 \tau= 10\log2 τ = 10 log 2 . For a child with two positive outcomes, a ^ i = 0 \widehat{a}_i = 0 a i = 0 , the proposal variance is s i 2 = 1.2 2 / ( 1 + τ − 2 ) s_i^2 = 1.2^2/(1+\tau^{-2}) s i 2 = 1. 2 2 / ( 1 + τ − 2 ) , and, as ∣ a ∣ → ∞ \lvert a\rvert\to\infty ∣ a ∣ → ∞ ,
log g i ( a ; x ) 2 φ τ 2 ( a ) 2 q R E , i ( a ∣ x ) 2 = ( 1 2 s i 2 − 1 τ 2 ) a 2 − 4 ∣ a ∣ + O ( 1 ) . \log\frac{g_i(a;x)^2\varphi_{\tau^2}(a)^2}{q_{\mathrm{RE},i}(a\mid x)^2} = \left(\frac{1}{2s_i^2}-\frac{1}{\tau^2}\right)a^2 - 4\lvert a\rvert+ O(1). log q RE , i ( a ∣ x ) 2 g i ( a ; x ) 2 φ τ 2 ( a ) 2 = ( 2 s i 2 1 − τ 2 1 ) a 2 − 4 ∣ a ∣ + O ( 1 ) .
The quadratic coefficient is positive, such children occur in the data, and the strict inequality persists in a neighborhood of x = 0 x = 0 x = 0 . Since q 2 q_2 q 2 and q 3 q_3 q 3 have full support, condition (IV.14) of Theorem IV.3 fails for the ideal weights, although the first-moment identity (V.33) still holds.
Each weight uses 537 × 16 = 8592 537 \times16 = 8592 537 × 16 = 8592 Gaussian inputs in addition to the parameter coordinates. The parameter and auxiliary inputs are generated from scrambled Sobol points with separate random seeds. Ideally, the auxiliary coordinates of each point are uniform on the unit cube and independent of its parameter coordinates, so that the inputs for different children are independent. Dependence across points preserves the expectation identity but violates the iid assumption of Theorem IV.3. Before the inverse-normal transform, the uniforms are clipped to [ 10 − 12 , 1 − 10 − 12 ] [10^{-12},1-10^{-12}] [ 1 0 − 12 , 1 − 1 0 − 12 ] , which introduces an unquantified deviation from the ideal unbiased estimator. For this model, we establish neither an RQMC rate in this high dimension nor Assumption IV.1 or the moment conditions (A.13).
The numerical reference combines 128-point Gauss–Hermite quadrature for each child’s integral [41 ] with four pCN chains [35 ] , each run for 20000 steps, with the first 4000 discarded and every fourth subsequent state retained, giving 16000 samples in total. Acceptance rates were 0.235–0.261, split-R ^ \widehat{R} R ranged from 1.0011 to 1.0039, and bulk ESS from 627 to 6949. These chain diagnostics do not bound the quadrature error, so the reference is not exact.
The density q p q_p q p for preliminary sampling was constructed from three successive sets of 4096 points, after which 32768 preliminary points were drawn from q p q_p q p ; fitting thus cost 3 × 4096 + 32768 = 45056 3 \times4096 + 32768 = 45056 3 × 4096 + 32768 = 45056 weight evaluations, and the preliminary sample had ESS 985.19. For each of q 2 q_2 q 2 and q 3 q_3 q 3 , three independent repetitions used M = 16384 M = 16384 M = 16384 pairs and N = 512 N = 512 N = 512 outputs each. The proposal q 1 q_1 q 1 was excluded because preliminary checks showed that it covered the posterior poorly. The fitted variances were bounded below by 10 − 6 10^{-6} 1 0 − 6 after the inflation in (III.20).
The time grid (III.6) was refined near its endpoint:
t k = T [ 1 − ( 1 − k K ) 4 ] , k = 0 , … , K , K = 80 , T = 0.999999. (86) t_k = T\left[1-\left(1-\frac{k}{K}\right)^4\right], \qquad k = 0,\ldots,K, \qquad K = 80,\qquad T = 0.999999. \tag*{(86)} t k = T [ 1 − ( 1 − K k ) 4 ] , k = 0 , … , K , K = 80 , T = 0.999999. (86)
The output is taken at T < 1 T<1 T < 1 , and the last step is
Δ t K − 1 = t K − t K − 1 = T K 4 ≈ 2.44 × 10 − 8 . (87) \Delta t_{K-1}=t_K-t_{K-1}=\frac{T}{K^4}\approx2.44\times10^{-8}. \tag*{(87)} Δ t K − 1 = t K − t K − 1 = K 4 T ≈ 2.44 × 1 0 − 8 . (87)
By (III.7), the noise in the last step has covariance Δ t K − 1 I 4 \Delta t_{K-1}I_4 Δ t K − 1 I 4 in Gaussian coordinates. Under (V.25), its standard deviation in each β j \beta_j β j , which is linear in x j x_j x j , is
50 Δ t K − 1 ≈ 0.00110 , (88) \sqrt{50\Delta t_{K-1}}\approx0.00110, \tag*{(88)} 50Δ t K − 1 ≈ 0.00110 , (88)
well below the reference posterior standard deviation 0.0671 of β 2 \beta_2 β 2 ; a uniform grid with the same K K K and T T T would give 50 T / K ≈ 0.791 \sqrt{50T/K}\approx0.791 50 T / K ≈ 0.791 . The calculation does not apply directly to τ \tau τ , which depends nonlinearly on x 4 x_4 x 4 . Small final-step noise does not by itself guarantee accuracy along the whole trajectory; a guarantee for this configuration would require error control for nonuniform grids, a time cutoff, and sample reuse, which Theorem IV.1 (uniform grid, terminal time one) does not provide.
The diagnostics use θ = ( β 1 , β 2 , β 3 , τ 2 ) ⊤ \theta=(\beta_1,\beta_2,\beta_3,\tau^2)^\top θ = ( β 1 , β 2 , β 3 , τ 2 ) ⊤ , that is, they report the random-effect variance, although the prior and the transformation are defined for τ \tau τ . For the numerical reference moments ( μ r e f θ , Σ r e f θ ) (\mu_{\mathrm{ref}}^\theta,\Sigma_{\mathrm{ref}}^\theta) ( μ ref θ , Σ ref θ ) , define
D r e f = diag ( ( Σ r e f θ ) 11 , … , ( Σ r e f θ ) 44 ) . (89) D_{\mathrm{ref}}=\operatorname{diag}\left(\sqrt{(\Sigma_{\mathrm{ref}}^\theta)_{11}},\ldots,\sqrt{(\Sigma_{\mathrm{ref}}^\theta)_{44}}\right). \tag*{(89)} D ref = diag ( ( Σ ref θ ) 11 , … , ( Σ ref θ ) 44 ) . (89)
For the output moments ( μ ^ o u t θ , Σ ^ o u t θ ) (\widehat{\mu}_{\mathrm{out}}^\theta,\widehat{\Sigma}_{\mathrm{out}}^\theta) ( μ out θ , Σ out θ ) , set Σ ^ o u t , s = D r e f − 1 Σ ^ o u t θ D r e f − 1 \widehat{\Sigma}_{\mathrm{out},s}=D_{\mathrm{ref}}^{-1}\widehat{\Sigma}_{\mathrm{out}}^\theta D_{\mathrm{ref}}^{-1} Σ out , s = D ref − 1 Σ out θ D ref − 1 and Σ r e f , s = D r e f − 1 Σ r e f θ D r e f − 1 \Sigma_{\mathrm{ref},s}=D_{\mathrm{ref}}^{-1}\Sigma_{\mathrm{ref}}^\theta D_{\mathrm{ref}}^{-1} Σ ref , s = D ref − 1 Σ ref θ D ref − 1 . The errors are
E μ , θ = ∥ D r e f − 1 ( μ ^ o u t θ − μ r e f θ ) ∥ d , E Σ , θ = ∥ Σ ^ o u t , s − Σ r e f , s ∥ F ∥ Σ r e f , s ∥ F . (90) E_{\mu,\theta}=\frac{\left\|D_{\mathrm{ref}}^{-1}(\widehat{\mu}_{\mathrm{out}}^\theta-\mu_{\mathrm{ref}}^\theta)\right\|}{\sqrt{d}},
\qquad
E_{\Sigma,\theta}=\frac{\left\|\widehat{\Sigma}_{\mathrm{out},s}-\Sigma_{\mathrm{ref},s}\right\|_F}{\left\|\Sigma_{\mathrm{ref},s}\right\|_F}. \tag*{(90)} E μ , θ = d D ref − 1 ( μ out θ − μ ref θ ) , E Σ , θ = ∥ Σ ref , s ∥ F Σ out , s − Σ ref , s F . (90)
Here d = 4 d=4 d = 4 . The diagnostic W 2 , G s t d W^{\mathrm{std}}_{2,G} W 2 , G std is (V.9) computed after the same reference centering and coordinatewise standardization; its scale therefore differs from that in Section V-A, and it is not the full W 2 W_2 W 2 distance between the joint distributions.
Table VII reports the error diagnostics and the local ESS at the last drift evaluation for q 2 q_2 q 2 and q 3 q_3 q 3 , and Table VIII compares the resulting posterior means and standard deviations with the reference.
Table VII. Ohio GLMM posterior diagnostics. The three error measures are reported as means ± \pm ± standard deviations across three independent sets of samples used for drift estimation. Local ESS (III.21) measures weight concentration at the last drift evaluation; it is distinct from the overall terminal importance-weight ESS in Remark B.1.
Table VIII. Ohio GLMM posterior means (standard deviations) and the ratio of the q 3 q_3 q 3 standard deviation to the reference value. For q 2 q_2 q 2 and q 3 q_3 q 3 , each entry averages the means and sample standard deviations (divisor N − 1 N - 1 N − 1 ) computed separately in the three runs. The SD ratio is the average of the three ratios of the q 3 q_3 q 3 standard deviation to the reference value.
The mixture q 3 q_3 q 3 gave smaller values of all three error diagnostics and less variation across sample sets; its marginal standard-deviation ratios ranged from 1.003 to 1.089, with the largest deviation for β 2 \beta_2 β 2 . With only three repetitions per proposal, these comparisons are descriptive and do not show that q 3 q_3 q 3 is better in general. The local ESS (III.21) at the last drift evaluation was about 1.02 for both proposals; by Remark B.1, this occurs even under exact sampling of the fixed-sample terminal measure and does not by itself indicate posterior error or instability. Finally, agreement of moments and one-dimensional marginals does not establish accuracy of the joint distribution.
Figure 7 compares the marginal posterior densities of β 1 \beta_1 β 1 , β 2 \beta_2 β 2 , β 3 \beta_3 β 3 , and τ 2 \tau^2 τ 2 obtained with q 2 q_2 q 2 and q 3 q_3 q 3 with the reference computed by Gauss–Hermite quadrature and pCN.
Figure 7. Ohio GLMM posterior marginals. For each of q 2 q_2 q 2 and q 3 q_3 q 3 , the 3 × 512 = 1536 3 \times512 = 1536 3 × 512 = 1536 outputs of the three runs are pooled into one kernel density estimate. The reference density, computed by Gauss–Hermite quadrature and pCN, uses the 16000 retained samples. Panels (a)–(d) show β 1 \beta_1 β 1 , β 2 \beta_2 β 2 , β 3 \beta_3 β 3 , and τ 2 \tau^2 τ 2 , respectively.
Proposal fitting and the six sample sets required 45056 + 6 × 16384 = 143360 45056 + 6 \times16384 = 143360 45056 + 6 × 16384 = 143360 evaluations of ψ \psi ψ , each comprising 537 × 16 537 \times16 537 × 16 random-effect importance terms. These counts are not directly comparable with the M/G/1 simulator calls and do not measure wall-clock time; the reference computation and the state-dependent weight calculations are additional costs.
Conclusion
We developed a likelihood-free Schrödinger–Föllmer sampler whose drift is estimated jointly from Gaussian samples and simulator outputs, requiring neither pointwise likelihood values nor simulator derivatives. Importance sampling with a fixed proposal allows a single set of simulator outputs to be reused across time steps and trajectories; this reduces the number of simulator calls, although the state-dependent weights must still be recomputed.
For likelihood-free SFS, we gave sufficient curvature conditions for ABC targets, established drift regularity, and derived a terminal error bound that separates integration and discretization errors. For sample reuse with iid samples, we proved pointwise consistency and asymptotic normality of the drift estimator, which yield a variance-based proposal criterion, and terminal consistency under a finite posterior second moment; unconditional W 2 W_2 W 2 convergence requires additional moments of the proposal. Some of these assumptions are restrictive: the Gaussian benchmark violates the state-uniform mean-square assumption of Theorem IV.1, and the ideal Ohio weights violate the second-moment assumption of the drift CLT. Neither assumption is needed for terminal consistency under sample reuse.
In the experiments, sample reuse greatly reduced the number of simulator calls, and RQMC improved drift estimation. Fitted proposals improved centering and local weight balance, but these gains did not always translate into proportional terminal improvements: the concentrated Gaussian example is limited by an Euler covariance floor, and the M/G/1 example showed no terminal advantage of RQMC. The Ohio example applies the construction to an unbiased randomized likelihood without ABC. Quantitative terminal bounds in the sample size and time step under sample reuse, covering the RQMC rules, clipped inputs, and time cutoff used in the experiments, remain to be developed.
Appendix A: Proofs for likelihood-free SFS
Throughout this appendix, Assumption IV.1 holds and we write
τ = 1 − t , L V = max { λ + , κ } , L b = L V 1 − κ . \tau= 1 - t,\qquad L_V = \max\{\lambda_+,\kappa\},\qquad L_b = \frac{L_V}{1-\kappa}. τ = 1 − t , L V = max { λ + , κ } , L b = 1 − κ L V .
In particular, ∥ ∇ 2 V ( z ) ∥ o p ≤ L V \lVert\nabla^2 V(z)\rVert_{\mathrm{op}} \le L_V ∥ ∇ 2 V ( z ) ∥ op ≤ L V for all z ∈ R d z \in\mathbb{R}^d z ∈ R d .
Proof of Theorem IV.1
The proof is based on a synchronous coupling of the exact diffusion and the Euler scheme. We first establish the regularity of the exact drift in Lemma A.1 and bound the drift-estimation error at a fixed state in Lemma A.2; the two lemmas are then combined through a discrete Gronwall argument. For ( x , t ) ∈ R d × [ 0 , 1 ) (x,t) \in\mathbb{R}^{d} \times[0,1) ( x , t ) ∈ R d × [ 0 , 1 ) , define the probability measure μ x , t \mu_{x,t} μ x , t and its potential U x , t U_{x,t} U x , t by
μ x , t ( d z ) = f ( z ) φ τ ( z − x ) Q τ f ( x ) d z , U x , t ( z ) = V ( z ) + ∥ z − x ∥ 2 2 τ . (91) \mu_{x,t}(\mathrm{d}z) = \frac{f(z)\varphi_{\tau}(z-x)}{Q_{\tau}f(x)} \,\mathrm{d}z,\qquad U_{x,t}(z) = V(z) + \frac{\lVert z-x\rVert^{2}}{2\tau}. \tag*{(91)} μ x , t ( d z ) = Q τ f ( x ) f ( z ) φ τ ( z − x ) d z , U x , t ( z ) = V ( z ) + 2 τ ∥ z − x ∥ 2 . (91)
Since f = e − V f=e^{-V} f = e − V , the function U x , t U_{x,t} U x , t is the negative log-density of μ x , t \mu_{x,t} μ x , t up to an additive constant.
Lemma A.1. The drift b b b extends continuously to t = 1 t=1 t = 1 by b ( x , 1 ) = − ∇ V ( x ) b(x,1)=-\nabla V(x) b ( x , 1 ) = − ∇ V ( x ) , the equation (III.3) has a unique strong solution on [ 0 , 1 ] [0,1] [ 0 , 1 ] , and X 1 ∼ π A B C X_{1}\sim\pi_{\mathrm{ABC}} X 1 ∼ π ABC . Moreover, there are constants B ∗ B_{*} B ∗ and C X C_{X} C X , depending only on B 0 B_{0} B 0 , λ + \lambda_{+} λ + , and κ \kappa κ , such that for all x , x ′ ∈ R d x,x'\in\mathbb{R}^{d} x , x ′ ∈ R d and 0 ≤ s ≤ t ≤ 1 0\leq s\leq t\leq1 0 ≤ s ≤ t ≤ 1 ,
∥ b ( x , t ) − b ( x ′ , t ) ∥ ≤ L b ∥ x − x ′ ∥ , (92) \lVert b(x,t)-b(x',t)\rVert\leq L_{b}\lVert x-x'\rVert, \tag*{(92)} ∥ b ( x , t ) − b ( x ′ , t )∥ ≤ L b ∥ x − x ′ ∥ , (92) ∥ b ( x , t ) ∥ ≤ B ∗ d + L b ∥ x ∥ , (93) \lVert b(x,t)\rVert\leq B_{*}\sqrt{d}+L_{b}\lVert x\rVert, \tag*{(93)} ∥ b ( x , t )∥ ≤ B ∗ d + L b ∥ x ∥ , (93) sup 0 ≤ u ≤ 1 E ∥ X u ∥ 2 ≤ C X d , (94) \sup_{0\leq u\leq1}\mathbb{E}\lVert X_{u}\rVert^{2}\leq C_{X}d, \tag*{(94)} 0 ≤ u ≤ 1 sup E ∥ X u ∥ 2 ≤ C X d , (94) E ∥ b ( X t , t ) − b ( X s , s ) ∥ 2 ≤ d L b 2 ( t − s ) . (95) \mathbb{E}\lVert b(X_{t},t)-b(X_{s},s)\rVert^{2}\leq dL_{b}^{2}(t-s). \tag*{(95)} E ∥ b ( X t , t ) − b ( X s , s ) ∥ 2 ≤ d L b 2 ( t − s ) . (95)
Proof. We first prove (A.2) and (A.3) for t < 1 t<1 t < 1 . By Assumption IV.1, ∇ z 2 U x , t ⪰ ( τ − 1 − κ ) I d \nabla_{z}^{2}U_{x,t}\succeq(\tau^{-1}-\kappa)I_{d} ∇ z 2 U x , t ⪰ ( τ − 1 − κ ) I d with τ − 1 − κ ≥ 1 − κ > 0 \tau^{-1}-\kappa\geq1-\kappa>0 τ − 1 − κ ≥ 1 − κ > 0 , so μ x , t \mu_{x,t} μ x , t is strongly log-concave. By the Brascamp–Lieb variance inequality, its Poincaré constant satisfies
C P ( x , t ) ≤ ( τ − 1 − κ ) − 1 = τ 1 − κ τ . (96) C_{P}(x,t)\leq(\tau^{-1}-\kappa)^{-1}=\frac{\tau}{1-\kappa\tau}. \tag*{(96)} C P ( x , t ) ≤ ( τ − 1 − κ ) − 1 = 1 − κ τ τ . (96)
By Taylor’s formula, V ( z ) ≥ V ( 0 ) + ∇ V ( 0 ) ⊤ z − κ ∥ z ∥ 2 / 2 V(z)\geq V(0)+\nabla V(0)^{\top}z-\kappa\lVert z\rVert^{2}/2 V ( z ) ≥ V ( 0 ) + ∇ V ( 0 ) ⊤ z − κ ∥ z ∥ 2 /2 . Since τ − 1 > κ \tau^{-1}>\kappa τ − 1 > κ , the function f ( z ) φ τ ( z − x ) f(z)\varphi_{\tau}(z-x) f ( z ) φ τ ( z − x ) and its derivatives in x x x are dominated by Gaussian functions of z z z , locally uniformly in x x x . Hence we may differentiate under the integral sign and integrate by parts. Differentiating Q τ f Q_{\tau}f Q τ f gives b ( x , t ) = E μ x , t [ ( Z − x ) / τ ] b(x,t)=\mathbb{E}_{\mu_{x,t}}[(Z-x)/\tau] b ( x , t ) = E μ x , t [( Z − x ) / τ ] , and integrating ∇ z e − U x , t \nabla_{z}e^{-U_{x,t}} ∇ z e − U x , t over R d \mathbb{R}^{d} R d gives E μ x , t ∇ U x , t ( Z ) = 0 \mathbb{E}_{\mu_{x,t}}\nabla U_{x,t}(Z)=0 E μ x , t ∇ U x , t ( Z ) = 0 . It follows that
b ( x , t ) = E μ x , t Z − x τ = − E μ x , t ∇ V ( Z ) . (97) b(x,t)=\mathbb{E}_{\mu_{x,t}}\frac{Z-x}{\tau}=-\mathbb{E}_{\mu_{x,t}}\nabla V(Z). \tag*{(97)} b ( x , t ) = E μ x , t τ Z − x = − E μ x , t ∇ V ( Z ) . (97)
Differentiating the density of μ x , t \mu_{x,t} μ x , t with respect to x x x in (A.7) gives
∇ x b ( x , t ) = − τ − 1 Cov μ x , t ( ∇ V ( Z ) , Z ) . \nabla_{x}b(x,t)=-\tau^{-1}\operatorname{Cov}_{\mu_{x,t}}(\nabla V(Z),Z). ∇ x b ( x , t ) = − τ − 1 Cov μ x , t ( ∇ V ( Z ) , Z ) .
For unit vectors u , v ∈ R d u,v\in\mathbb{R}^{d} u , v ∈ R d , the Cauchy–Schwarz inequality and (A.6) give
∣ u ⊤ ∇ x b ( x , t ) v ∣ ≤ 1 τ Var μ x , t ( u ⊤ ∇ V ( Z ) ) 1 / 2 Var μ x , t ( v ⊤ Z ) 1 / 2 ≤ C P ( x , t ) L V τ ≤ L V 1 − κ τ ≤ L b . (98) \left|u^{\top}\nabla_{x}b(x,t)v\right|
\leq\frac{1}{\tau}\operatorname{Var}_{\mu_{x,t}}(u^{\top}\nabla V(Z))^{1/2}\operatorname{Var}_{\mu_{x,t}}(v^{\top}Z)^{1/2}
\leq\frac{C_{P}(x,t)L_{V}}{\tau}
\leq\frac{L_{V}}{1-\kappa\tau}
\leq L_{b}. \tag*{(98)} u ⊤ ∇ x b ( x , t ) v ≤ τ 1 Var μ x , t ( u ⊤ ∇ V ( Z ) ) 1/2 Var μ x , t ( v ⊤ Z ) 1/2 ≤ τ C P ( x , t ) L V ≤ 1 − κ τ L V ≤ L b . (98)
where in the second inequality we use Var ( u ⊤ ∇ V ( Z ) ) ≤ C P E ∥ ∇ 2 V ( Z ) u ∥ 2 ≤ C P L V 2 \operatorname{Var}(u^{\top}\nabla V(Z)) \le C_P\mathbb{E}\lVert\nabla^2V(Z)u\rVert^2 \le C_PL_V^2 Var ( u ⊤ ∇ V ( Z )) ≤ C P E ∥ ∇ 2 V ( Z ) u ∥ 2 ≤ C P L V 2 and Var ( v ⊤ Z ) ≤ C P \operatorname{Var}(v^{\top}Z) \le C_P Var ( v ⊤ Z ) ≤ C P . Therefore, (A.2) holds for t < 1 t<1 t < 1 .
For (A.3), it suffices to bound b ( 0 , t ) b(0,t) b ( 0 , t ) . Let z t ∗ z_t^* z t ∗ be the minimizer of U 0 , t U_{0,t} U 0 , t . Since U 0 , t U_{0,t} U 0 , t is ( 1 − κ ) (1-\kappa) ( 1 − κ ) -strongly convex and ∇ U 0 , t ( 0 ) = ∇ V ( 0 ) \nabla U_{0,t}(0)=\nabla V(0) ∇ U 0 , t ( 0 ) = ∇ V ( 0 ) , we have ∥ z t ∗ ∥ ≤ ∥ ∇ V ( 0 ) ∥ / ( 1 − κ ) ≤ B 0 d / ( 1 − κ ) \lVert z_t^*\rVert\le\lVert\nabla V(0)\rVert/(1-\kappa)\le B_0\sqrt{d}/(1-\kappa) ∥ z t ∗ ∥ ≤ ∥ ∇ V ( 0 )∥ / ( 1 − κ ) ≤ B 0 d / ( 1 − κ ) . Integrating ∇ z ⋅ [ ( z − z t ∗ ) e − U 0 , t ( z ) ] \nabla_z\cdot[(z-z_t^*)e^{-U_{0,t}(z)}] ∇ z ⋅ [( z − z t ∗ ) e − U 0 , t ( z ) ] over R d \mathbb{R}^d R d gives d = E μ 0 , t [ ( Z − z t ∗ ) ⊤ ∇ U 0 , t ( Z ) ] ≥ ( 1 − κ ) E μ 0 , t ∥ Z − z t ∗ ∥ 2 d=\mathbb{E}_{\mu_{0,t}}[(Z-z_t^*)^{\top}\nabla U_{0,t}(Z)]\ge(1-\kappa)\mathbb{E}_{\mu_{0,t}}\lVert Z-z_t^*\rVert^2 d = E μ 0 , t [( Z − z t ∗ ) ⊤ ∇ U 0 , t ( Z )] ≥ ( 1 − κ ) E μ 0 , t ∥ Z − z t ∗ ∥ 2 . Therefore,
E μ 0 , t ∥ Z ∥ ≤ ( B 0 1 − κ + 1 1 − κ ) d . (99) \mathbb{E}_{\mu_{0,t}}\lVert Z\rVert\le\left(\frac{B_0}{1-\kappa}+\frac{1}{\sqrt{1-\kappa}}\right)\sqrt{d}. \tag*{(99)} E μ 0 , t ∥ Z ∥ ≤ ( 1 − κ B 0 + 1 − κ 1 ) d . (99)
Since ∥ ∇ V ( z ) ∥ ≤ B 0 d + L V ∥ z ∥ \lVert\nabla V(z)\rVert\le B_0\sqrt{d}+L_V\lVert z\rVert ∥ ∇ V ( z )∥ ≤ B 0 d + L V ∥ z ∥ , (A.7) and (A.9) give ∥ b ( 0 , t ) ∥ ≤ B ∗ d \lVert b(0,t)\rVert\le B_*\sqrt{d} ∥ b ( 0 , t )∥ ≤ B ∗ d with B ∗ = B 0 + L V [ B 0 / ( 1 − κ ) + ( 1 − κ ) − 1 / 2 ] B_*=B_0+L_V[B_0/(1-\kappa)+(1-\kappa)^{-1/2}] B ∗ = B 0 + L V [ B 0 / ( 1 − κ ) + ( 1 − κ ) − 1/2 ] . Together with (A.2), this proves (A.3) for t < 1 t<1 t < 1 .
We next extend the drift to t = 1 t=1 t = 1 . Let z x , t ∗ z_{x,t}^* z x , t ∗ be the minimizer of U x , t U_{x,t} U x , t . The same strong convexity and integration-by-parts arguments give
∥ z x , t ∗ − x ∥ ≤ τ ∥ ∇ V ( x ) ∥ 1 − κ τ , E μ x , t ∥ Z − z x , t ∗ ∥ 2 ≤ d τ 1 − κ τ . \lVert z_{x,t}^*-x\rVert\le\frac{\tau\lVert\nabla V(x)\rVert}{1-\kappa\tau},\qquad\mathbb{E}_{\mu_{x,t}}\lVert Z-z_{x,t}^*\rVert^2\le\frac{d\tau}{1-\kappa\tau}. ∥ z x , t ∗ − x ∥ ≤ 1 − κ τ τ ∥ ∇ V ( x )∥ , E μ x , t ∥ Z − z x , t ∗ ∥ 2 ≤ 1 − κ τ d τ .
By (A.7) and the Lipschitz continuity of ∇ V \nabla V ∇ V , ∥ b ( x , t ) + ∇ V ( x ) ∥ ≤ L V E μ x , t ∥ Z − x ∥ → 0 \lVert b(x,t)+\nabla V(x)\rVert\le L_V\mathbb{E}_{\mu_{x,t}}\lVert Z-x\rVert\to0 ∥ b ( x , t ) + ∇ V ( x )∥ ≤ L V E μ x , t ∥ Z − x ∥ → 0 locally uniformly in x x x as t ↑ 1 t\uparrow1 t ↑ 1 . Hence b b b extends continuously to t = 1 t=1 t = 1 by b ( x , 1 ) = − ∇ V ( x ) b(x,1)=-\nabla V(x) b ( x , 1 ) = − ∇ V ( x ) , and (A.2) and (A.3) remain valid at t = 1 t=1 t = 1 . Since the drift is globally Lipschitz in x x x with linear growth, uniformly in t ∈ [ 0 , 1 ] t\in[0,1] t ∈ [ 0 , 1 ] , (III.3) has a unique strong solution on [ 0 , 1 ] [0,1] [ 0 , 1 ] . By Itô’s formula for ∥ X t ∥ 2 \lVert X_t\rVert^2 ∥ X t ∥ 2 , (A.3), and Gronwall’s inequality, we obtain (A.4).
To prove (A.5), note that for t < 1 t<1 t < 1 the function h = Q τ f h=Q_{\tau}f h = Q τ f solves the backward heat equation ∂ t h + 1 2 Δ h = 0 \partial_t h+\frac{1}{2}\Delta h=0 ∂ t h + 2 1 Δ h = 0 . Hence b = ∇ x log h b=\nabla_x\log h b = ∇ x log h satisfies ∂ t b + 1 2 Δ b + ( ∇ x b ) b = 0 \partial_t b+\frac{1}{2}\Delta b+(\nabla_xb)b=0 ∂ t b + 2 1 Δ b + ( ∇ x b ) b = 0 , and Itô’s formula gives
d b ( X t , t ) = ∇ x b ( X t , t ) d B t . {}
db(X_t,t)=\nabla_xb(X_t,t)\,dB_t. d b ( X t , t ) = ∇ x b ( X t , t ) d B t .
Since ∥ ∇ x b ∥ F 2 ≤ d ∥ ∇ x b ∥ o p 2 ≤ d L b 2 \lVert\nabla_xb\rVert_F^2\le d\lVert\nabla_xb\rVert_{\mathrm{op}}^2\le dL_b^2 ∥ ∇ x b ∥ F 2 ≤ d ∥ ∇ x b ∥ op 2 ≤ d L b 2 by (A.8), it follows from the Itô isometry that (A.5) holds for t < 1 t<1 t < 1 ; the case t = 1 t=1 t = 1 follows by continuity.
It remains to identify the terminal law. Let B 0 B^0 B 0 be a Brownian motion under a probability measure P 0 \mathbb{P}_0 P 0 , and define d P ∗ = f ( B 1 0 ) d P 0 d\mathbb{P}_*=f(B_1^0)d\mathbb{P}_0 d P ∗ = f ( B 1 0 ) d P 0 , which is a probability measure because ∫ f φ = 1 \int f\varphi=1 ∫ f φ = 1 . Its density process is M t = Q 1 − t f ( B t 0 ) M_t=Q_{1-t}f(B_t^0) M t = Q 1 − t f ( B t 0 ) , and d M t = M t b ( B t 0 , t ) ⊤ d B t 0 dM_t=M_tb(B_t^0,t)^{\top}dB_t^0 d M t = M t b ( B t 0 , t ) ⊤ d B t 0 . By Girsanov’s theorem on [ 0 , T ] [0,T] [ 0 , T ] for each T < 1 T<1 T < 1 , the process B 0 B^0 B 0 solves (III.3) under P ∗ \mathbb{P}_* P ∗ . Pathwise uniqueness implies uniqueness in law, so X X X and B 0 B^0 B 0 under P ∗ \mathbb{P}_* P ∗ have the same law on [ 0 , T ] [0,T] [ 0 , T ] for every T < 1 T<1 T < 1 ; by path continuity, X 1 X_1 X 1 has the law of B 1 0 B_1^0 B 1 0 under P ∗ \mathbb{P}_* P ∗ , whose density is f φ = π A B C f\varphi=\pi_{\mathrm{ABC}} f φ = π ABC . This proves the desired result. □ \square □
We next bound the drift-estimation error at a fixed state. For fixed ( x , t ) (x,t) ( x , t ) with t < 1 t<1 t < 1 , let A ( x , t ) = E [ W ( Z , ξ ) ( Z − x ) ] A(x,t)=\mathbb{E}[W(Z,\xi)(Z-x)] A ( x , t ) = E [ W ( Z , ξ ) ( Z − x )] , with expectation under (III.8), let C ( x , t ) C(x,t) C ( x , t ) be as in § 3 -B, and let A ^ m \widehat{A}_m A m and C ^ m \widehat{C}_m C m be the corresponding sample averages in (III.12). Direct subtraction gives
b ^ m − b = C ( A ^ m − A ) − A ( C ^ m − C ) ( 1 − t ) C C ^ m . (100) \widehat{b}_m-b=\frac{C(\widehat{A}_m-A)-A(\widehat{C}_m-C)}{(1-t)C\widehat{C}_m}. \tag*{(100)} b m − b = ( 1 − t ) C C m C ( A m − A ) − A ( C m − C ) . (100)
Define
χ x , t ( Z , ξ ) = W ( Z , ξ ) C ( x , t ) { Z − x 1 − t − b ( x , t ) } . (101) \chi_{x,t}(Z,\xi)=\frac{W(Z,\xi)}{C(x,t)}\left\{\frac{Z-x}{1-t}-b(x,t)\right\}. \tag*{(101)} χ x , t ( Z , ξ ) = C ( x , t ) W ( Z , ξ ) { 1 − t Z − x − b ( x , t ) } . (101)
Since A = ( 1 − t ) C b A=(1-t)Cb A = ( 1 − t ) C b , we have E χ x , t ( Z , ξ ) = 0 \mathbb{E}\chi_{x,t}(Z,\xi)=0 E χ x , t ( Z , ξ ) = 0 , and (A.10) can be rewritten as
b ^ m − b = C C ^ m 1 m ∑ i = 1 m χ x , t ( Z i , ξ i ) , (102) \widehat{b}_m-b=\frac{C}{\widehat{C}_m}\frac{1}{m}\sum_{i=1}^{m}\chi_{x,t}(Z_i,\xi_i), \tag*{(102)} b m − b = C m C m 1 i = 1 ∑ m χ x , t ( Z i , ξ i ) , (102)
which is the vector form of the self-normalized ratio identity [19 Section 9.2] .
Lemma A.2. Fix ( x , t ) (x,t) ( x , t ) with t < 1 t<1 t < 1 and 0 < C ( x , t ) < ∞ 0<C(x,t)<\infty 0 < C ( x , t ) < ∞ , let ( Z i , ξ i ) (Z_i,\xi_i) ( Z i , ξ i ) , i = 1 , … , m i=1,\ldots,m i = 1 , … , m , be independent with law (III.8), and suppose that C ^ m ( x , t ) > 0 \widehat{C}_m(x,t)>0 C m ( x , t ) > 0 almost surely. If there are finite D ( t ) , R ( t ) ≥ 0 D(t),R(t)\geq0 D ( t ) , R ( t ) ≥ 0 such that
E [ ( C ( x , t ) C ^ m ( x , t ) ) 4 ] ≤ D ( t ) , E ∥ χ x , t ( Z , ξ ) ∥ 4 ≤ R ( t ) d 2 , (103) \mathbb{E}\left[\left(\frac{C(x,t)}{\widehat{C}_m(x,t)}\right)^4\right]\leq D(t),\qquad\mathbb{E}\lVert\chi_{x,t}(Z,\xi)\rVert^4\leq R(t)d^2, \tag*{(103)} E [ ( C m ( x , t ) C ( x , t ) ) 4 ] ≤ D ( t ) , E ∥ χ x , t ( Z , ξ ) ∥ 4 ≤ R ( t ) d 2 , (103) then
E ∥ b ^ m ( x , t ) − b ( x , t ) ∥ 2 ≤ ρ ( t ) d m , ρ ( t ) = 3 D ( t ) R ( t ) . (104) \mathbb{E}\lVert\widehat{b}_m(x,t)-b(x,t)\rVert^2\leq\frac{\rho(t)d}{m},\qquad\rho(t)=\sqrt{3D(t)R(t)}. \tag*{(104)} E ∥ b m ( x , t ) − b ( x , t ) ∥ 2 ≤ m ρ ( t ) d , ρ ( t ) = 3 D ( t ) R ( t ) . (104)
Proof. Write χ i = χ x , t ( Z i , ξ i ) \chi_i=\chi_{x,t}(Z_i,\xi_i) χ i = χ x , t ( Z i , ξ i ) . Since the χ i \chi_i χ i are independent and centered, expanding the fourth power, we have
E ∥ ∑ i = 1 m χ i ∥ 4 = ∑ i = 1 m E ∥ χ i ∥ 4 + ∑ i ≠ l ( E ∥ χ i ∥ 2 E ∥ χ l ∥ 2 + 2 E ( χ i ⊤ χ l ) 2 ) ≤ 3 m 2 R ( t ) d 2 , \mathbb{E}\left\lVert\sum_{i=1}^{m}\chi_i\right\rVert^4=\sum_{i=1}^{m}\mathbb{E}\lVert\chi_i\rVert^4+\sum_{i\ne l}\left(\mathbb{E}\lVert\chi_i\rVert^2\mathbb{E}\lVert\chi_l\rVert^2+2\mathbb{E}(\chi_i^\top\chi_l)^2\right)\leq3m^2R(t)d^2, E i = 1 ∑ m χ i 4 = i = 1 ∑ m E ∥ χ i ∥ 4 + i = l ∑ ( E ∥ χ i ∥ 2 E ∥ χ l ∥ 2 + 2 E ( χ i ⊤ χ l ) 2 ) ≤ 3 m 2 R ( t ) d 2 ,
where the inequality used E ( χ i ⊤ χ l ) 2 ≤ E ∥ χ i ∥ 2 E ∥ χ l ∥ 2 ≤ R ( t ) d 2 \mathbb{E}(\chi_i^\top\chi_l)^2\leq\mathbb{E}\lVert\chi_i\rVert^2\mathbb{E}\lVert\chi_l\rVert^2\leq R(t)d^2 E ( χ i ⊤ χ l ) 2 ≤ E ∥ χ i ∥ 2 E ∥ χ l ∥ 2 ≤ R ( t ) d 2 for i ≠ l i\ne l i = l . By (A.12) and the Cauchy–Schwarz inequality, we obtain
E ∥ b ^ m − b ∥ 2 ≤ D ( t ) 1 / 2 ( E ∥ 1 m ∑ i = 1 m χ i ∥ 4 ) 1 / 2 ≤ 3 D ( t ) R ( t ) d m . (105) \mathbb{E}\lVert\widehat{b}_m-b\rVert^2\leq D(t)^{1/2}\left(\mathbb{E}\left\lVert\frac{1}{m}\sum_{i=1}^{m}\chi_i\right\rVert^4\right)^{1/2}\leq\frac{\sqrt{3D(t)R(t)}\,d}{m}. \tag*{(105)} E ∥ b m − b ∥ 2 ≤ D ( t ) 1/2 E m 1 i = 1 ∑ m χ i 4 1/2 ≤ m 3 D ( t ) R ( t ) d . (105)
This proves the desired result. □
If (A.13) holds uniformly over the states visited by the scheme, then conditioning on F t k \mathcal{F}_{t_k} F t k and applying Lemma A.2 at ( x , t ) = ( X ~ t k , t k ) (x,t)=(\widetilde{X}_{t_k},t_k) ( x , t ) = ( X t k , t k ) gives (IV.2). Based on Lemmas A.1 and A.2, we prove Theorem IV.1.
Proof of Theorem IV.1. We couple the exact diffusion and the scheme by using the same Brownian increments [16 ] . Let e k = X t k − X ~ t k e_k=X_{t_k}-\widetilde{X}_{t_k} e k = X t k − X t k , so that e 0 = 0 e_0=0 e 0 = 0 . Subtracting the Euler update (III.7) from the integral form of (III.3) gives e k + 1 = e k + h a k + R k + h ζ k e_{k+1}=e_k+ha_k+\mathcal{R}_k+h\zeta_k e k + 1 = e k + h a k + R k + h ζ k , where
a k = b ( X t k , t k ) − b ( X ~ t k , t k ) , a_k=b(X_{t_k},t_k)-b(\widetilde{X}_{t_k},t_k), a k = b ( X t k , t k ) − b ( X t k , t k ) ,
R k = ∫ t k t k + 1 [ b ( X u , u ) − b ( X t k , t k ) ] d u , \mathcal{R}_k=\int_{t_k}^{t_{k+1}}[b(X_u,u)-b(X_{t_k},t_k)]\,\mathrm{d}u, R k = ∫ t k t k + 1 [ b ( X u , u ) − b ( X t k , t k )] d u ,
ζ k = b ( X ~ t k , t k ) − b ^ m ( X ~ t k , t k ) . (106) \zeta_k=b(\widetilde{X}_{t_k},t_k)-\widehat{b}_m(\widetilde{X}_{t_k},t_k). \tag*{(106)} ζ k = b ( X t k , t k ) − b m ( X t k , t k ) . (106)
It suffices to estimate each E ∥ e k + 1 ∥ 2 \mathbb{E}\lVert e_{k+1}\rVert^2 E ∥ e k + 1 ∥ 2 in terms of E ∥ e k ∥ 2 \mathbb{E}\lVert e_k\rVert^2 E ∥ e k ∥ 2 . For h ≤ 1 h \le1 h ≤ 1 , Young’s inequality 2 u T v ≤ h ∥ u ∥ 2 + h − 1 ∥ v ∥ 2 2u^{\mathsf{T}}v \le h\lVert u\rVert^2 + h^{-1}\lVert v\rVert^2 2 u T v ≤ h ∥ u ∥ 2 + h − 1 ∥ v ∥ 2 yields
∥ e k + 1 ∥ 2 ≤ ( 1 + h ) ∥ e k + h a k ∥ 2 + 4 h ( ∥ R k ∥ 2 + h 2 ∥ ζ k ∥ 2 ) . \lVert e_{k+1}\rVert^2 \le(1+h)\lVert e_k+ha_k\rVert^2+\frac{4}{h}\left(\lVert R_k\rVert^2+h^2\lVert\zeta_k\rVert^2\right). ∥ e k + 1 ∥ 2 ≤ ( 1 + h ) ∥ e k + h a k ∥ 2 + h 4 ( ∥ R k ∥ 2 + h 2 ∥ ζ k ∥ 2 ) .
Taking expectations, we have
E ∥ e k + 1 ∥ 2 ≤ ( 1 + h ) ( 1 + h L b ) 2 E ∥ e k ∥ 2 + 4 h ⋅ d L b 2 h 3 2 + 4 h ⋅ ρ ( t k ) d m ≤ ( 1 + c 0 h ) E ∥ e k ∥ 2 + 2 d L b 2 h 2 + 4 d h ρ ( t k ) m , \begin{aligned}
\mathbb{E}\lVert e_{k+1}\rVert^2 \le(1+h)(1+hL_b)^2\mathbb{E}\lVert e_k\rVert^2+\frac{4}{h}\cdot\frac{dL_b^2h^3}{2}+4h\cdot\frac{\rho(t_k)d}{m} \\
&\le(1+c_0h)\mathbb{E}\lVert e_k\rVert^2+2dL_b^2h^2+\frac{4dh\rho(t_k)}{m},
\end{aligned} E ∥ e k + 1 ∥ 2 ≤ ( 1 + h ) ( 1 + h L b ) 2 E ∥ e k ∥ 2 + h 4 ⋅ 2 d L b 2 h 3 + 4 h ⋅ m ρ ( t k ) d ≤ ( 1 + c 0 h ) E ∥ e k ∥ 2 + 2 d L b 2 h 2 + m 4 d h ρ ( t k ) ,
where the first inequality follows from ∥ e k + h a k ∥ ≤ ( 1 + h L b ) ∥ e k ∥ \lVert e_k+ha_k\rVert\le(1+hL_b)\lVert e_k\rVert ∥ e k + h a k ∥ ≤ ( 1 + h L b ) ∥ e k ∥ by (A.2), from E ∥ R k ∥ 2 ≤ h ∫ 0 h d L b 2 u d u = d L b 2 h 3 / 2 \mathbb{E}\lVert R_k\rVert^2\le h\int_0^h dL_b^2u\,du=dL_b^2h^3/2 E ∥ R k ∥ 2 ≤ h ∫ 0 h d L b 2 u d u = d L b 2 h 3 /2 by the Cauchy–Schwarz inequality and (A.5), and from E ∥ ζ k ∥ 2 ≤ ρ ( t k ) d / m \mathbb{E}\lVert\zeta_k\rVert^2\le\rho(t_k)d/m E ∥ ζ k ∥ 2 ≤ ρ ( t k ) d / m by (IV.2), and the second inequality follows from ( 1 + h ) ( 1 + h L b ) 2 ≤ 1 + c 0 h (1+h)(1+hL_b)^2\le1+c_0h ( 1 + h ) ( 1 + h L b ) 2 ≤ 1 + c 0 h with c 0 = 1 + 4 L b + 2 L b 2 c_0=1+4L_b+2L_b^2 c 0 = 1 + 4 L b + 2 L b 2 .
Iterating from e 0 = 0 e_0=0 e 0 = 0 and using ( 1 + c 0 h ) K ≤ e c 0 (1+c_0h)^K\le e^{c_0} ( 1 + c 0 h ) K ≤ e c 0 and K h = 1 Kh=1 K h = 1 , we obtain
E ∥ e K ∥ 2 ≤ e c 0 ( 2 d L b 2 h + 4 d Λ h m ) ≤ C c p l ( d h + d Λ h m ) , (107) \mathbb{E}\lVert e_K\rVert^2\le e^{c_0}\left(2dL_b^2h+\frac{4d\Lambda_h}{m}\right)\le C_{\mathrm{cpl}}\left(dh+\frac{d\Lambda_h}{m}\right), \tag*{(107)} E ∥ e K ∥ 2 ≤ e c 0 ( 2 d L b 2 h + m 4 d Λ h ) ≤ C cpl ( d h + m d Λ h ) , (107)
for the constant C c p l = e c 0 max { 2 L b 2 , 4 } C_{\mathrm{cpl}}=e^{c_0}\max\{2L_b^2,4\} C cpl = e c 0 max { 2 L b 2 , 4 } depending only on L b L_b L b . Since X 1 ∼ π A B C X_1\sim\pi_{\mathrm{ABC}} X 1 ∼ π ABC by Lemma A.1 , we have W 2 2 ( Law ( X ~ 1 ) , π A B C ) ≤ E ∥ e K ∥ 2 W_2^2(\operatorname{Law}(\widetilde{X}_1),\pi_{\mathrm{ABC}})\le\mathbb{E}\lVert e_K\rVert^2 W 2 2 ( Law ( X 1 ) , π ABC ) ≤ E ∥ e K ∥ 2 . The desired result follows from (A.17) with C p a t h = C c p l 1 / 2 C_{\mathrm{path}}=C_{\mathrm{cpl}}^{1/2} C path = C cpl 1/2 . □
Verifying the ABC curvature condition
We give sufficient conditions on the simulator density and the prior for Assumption IV.1. The derivatives below are used only to verify this assumption; the sampler does not require them. For fixed y y y and δ > 0 \delta>0 δ > 0 , write K ( ξ ) = K δ ( S ( ξ ) , S ( y ) ) K(\xi)=K_\delta(S(\xi),S(y)) K ( ξ ) = K δ ( S ( ξ ) , S ( y )) and L = L A B C > 0 L=L_{\mathrm{ABC}}>0 L = L ABC > 0 , and define
s x ( ξ ) = ∇ x log p ( ξ ∣ x ) , H x ( ξ ) = ∇ x 2 log p ( ξ ∣ x ) , ν x , δ ( d ξ ) = K ( ξ ) p ( ξ ∣ x ) L ( x ) d ξ . (108) s_x(\xi)=\nabla_x\log p(\xi\mid x),\qquad H_x(\xi)=\nabla_x^2\log p(\xi\mid x),\qquad\nu_{x,\delta}(d\xi)=\frac{K(\xi)p(\xi\mid x)}{L(x)}d\xi. \tag*{(108)} s x ( ξ ) = ∇ x log p ( ξ ∣ x ) , H x ( ξ ) = ∇ x 2 log p ( ξ ∣ x ) , ν x , δ ( d ξ ) = L ( x ) K ( ξ ) p ( ξ ∣ x ) d ξ . (108)
Lemma A.3. Suppose that p ( ξ ∣ x ) > 0 p(\xi\mid x)>0 p ( ξ ∣ x ) > 0 on a support independent of x x x , that p ( ξ ∣ ⋅ ) ∈ C 2 ( R d ) p(\xi\mid\cdot)\in C^2(\mathbb{R}^d) p ( ξ ∣ ⋅ ) ∈ C 2 ( R d ) for almost every ξ \xi ξ , and that K p Kp K p , K ∇ p K\nabla p K ∇ p , K ∇ 2 p K\nabla^2p K ∇ 2 p , and K ∥ ∇ p ∥ 2 / p K\lVert\nabla p\rVert^2/p K ∥ ∇ p ∥ 2 / p are dominated by integrable functions of ξ \xi ξ , locally uniformly in x x x . Suppose also that there are constants 0 ≤ κ < 1 0\le\kappa<1 0 ≤ κ < 1 and λ + , Γ 0 ≥ 0 \lambda_+,\Gamma_0\ge0 λ + , Γ 0 ≥ 0 such that, for every x ∈ R d x\in\mathbb{R}^d x ∈ R d and ν = ν x , δ \nu=\nu_{x,\delta} ν = ν x , δ ,
λ max ( E ν H x + Cov ν ( s x ) ) ≤ κ , − λ min ( E ν H x ) ≤ λ + , ∥ E ν 0 , δ s 0 ∥ ≤ Γ 0 d . (109) \lambda_{\max}\left(\mathbb{E}_{\nu}H_x+\operatorname{Cov}_{\nu}(s_x)\right)\le\kappa,\qquad-\lambda_{\min}\left(\mathbb{E}_{\nu}H_x\right)\le\lambda_+,\qquad\left\lVert\mathbb{E}_{\nu_{0,\delta}}s_0\right\rVert\le\Gamma_0\sqrt{d}. \tag*{(109)} λ m a x ( E ν H x + Cov ν ( s x ) ) ≤ κ , − λ m i n ( E ν H x ) ≤ λ + , E ν 0 , δ s 0 ≤ Γ 0 d . (109) If π 0 = φ \pi_0=\varphi π 0 = φ , then Assumption IV.1 holds with B 0 = Γ 0 B_0=\Gamma_0 B 0 = Γ 0 .
Proof. By the domination assumptions, we may differentiate L ( x ) = ∫ K ( ξ ) p ( ξ ∣ x ) d ξ L(x)=\int K(\xi)p(\xi\mid x)\,d\xi L ( x ) = ∫ K ( ξ ) p ( ξ ∣ x ) d ξ twice under the integral sign, which gives
∇ [ − log L ( x ) ] = − E ν x , δ s x ( ξ ) , (110) \nabla[-\log L(x)]=-\mathbb{E}_{\nu_{x,\delta}}s_x(\xi), \tag*{(110)} ∇ [ − log L ( x )] = − E ν x , δ s x ( ξ ) , (110)
∇ 2 [ − log L ( x ) ] = − E ν x , δ H x ( ξ ) − Cov ν x , δ ( s x ( ξ ) ) . (111) \nabla^2[-\log L(x)]=-\mathbb{E}_{\nu_{x,\delta}}H_x(\xi)-\operatorname{Cov}_{\nu_{x,\delta}}(s_x(\xi)). \tag*{(111)} ∇ 2 [ − log L ( x )] = − E ν x , δ H x ( ξ ) − Cov ν x , δ ( s x ( ξ )) . (111)
By the first condition in (A.19), ∇ 2 [ − log L ] ⪰ − κ I d \nabla^{2}[-\log L] \succeq-\kappa I_d ∇ 2 [ − log L ] ⪰ − κ I d . Since covariance matrices are positive semidefinite, the second condition gives ∇ 2 [ − log L ] ⪯ − E ν H x ⪯ λ + I d \nabla^{2}[-\log L] \preceq-\mathbb{E}_{\nu}H_x \preceq\lambda_{+}I_d ∇ 2 [ − log L ] ⪯ − E ν H x ⪯ λ + I d , and the third gives ∥ ∇ [ − log L ] ( 0 ) ∥ ≤ Γ 0 d \lVert\nabla[-\log L](0)\rVert\le\Gamma_0\sqrt{d} ∥ ∇ [ − log L ] ( 0 )∥ ≤ Γ 0 d .
For a positive C 2 C^2 C 2 prior, V = − log ( π 0 L / φ ) V=-\log(\pi_0L/\varphi) V = − log ( π 0 L / φ ) up to an additive constant, so
∇ 2 V = ∇ 2 [ − log L ] + ∇ 2 [ − log π 0 ] − I d . (112) \nabla^{2}V=\nabla^{2}[-\log L]+\nabla^{2}[-\log\pi_0]-I_d. \tag*{(112)} ∇ 2 V = ∇ 2 [ − log L ] + ∇ 2 [ − log π 0 ] − I d . (112)
If π 0 = φ \pi_0=\varphi π 0 = φ , the last two terms cancel and ∇ V = ∇ [ − log L ] \nabla V=\nabla[-\log L] ∇ V = ∇ [ − log L ] . This proves the desired result.
For other priors, the curvature of − log π 0 -\log\pi_0 − log π 0 enters through (A.22), and the gradient at the origin is bounded by using ∇ V ( 0 ) = − E ν 0 , δ s 0 − ∇ log π 0 ( 0 ) \nabla V(0)=-\mathbb{E}_{\nu_{0,\delta}}s_0-\nabla\log\pi_0(0) ∇ V ( 0 ) = − E ν 0 , δ s 0 − ∇ log π 0 ( 0 ) .
Appendix B: Proofs for likelihood-free SFS with sample reuse
Proof of Theorem IV.2
Throughout this subsection, the samples are fixed until the final step of the proof, and all expectations are conditional on them. Set
H t ( x ) = ∑ i = 1 M c i φ 1 − t ( Z i − x ) , 0 ≤ t < 1. H_t(x)=\sum_{i=1}^{M}c_i\varphi_{1-t}(Z_i-x),\qquad0\le t<1. H t ( x ) = i = 1 ∑ M c i φ 1 − t ( Z i − x ) , 0 ≤ t < 1.
If ∑ i v i > 0 \sum_i v_i>0 ∑ i v i > 0 , then H t > 0 H_t>0 H t > 0 , ∂ t H t = − 1 2 Δ H t \partial_tH_t=-\frac{1}{2}\Delta H_t ∂ t H t = − 2 1 Δ H t , and b ^ M , q = ∇ log H t \widehat b_{M,q}=\nabla\log H_t b M , q = ∇ log H t . Consider the diffusion
d X t M = b ^ M , q ( X t M , t ) d t + d B t , X 0 M = 0. \mathrm{d}X_t^M=\widehat b_{M,q}(X_t^M,t)\,\mathrm{d}t+\mathrm{d}B_t,\qquad X_0^M=0. d X t M = b M , q ( X t M , t ) d t + d B t , X 0 M = 0.
Lemma B.1. Suppose that c i ∈ [ 0 , ∞ ) c_i\in[0,\infty) c i ∈ [ 0 , ∞ ) for all i i i and ∑ i v i > 0 \sum_i v_i>0 ∑ i v i > 0 . Then X M X^M X M has a unique strong solution on [ 0 , 1 ) [0,1) [ 0 , 1 ) , and its path distribution is that of t Z I + β t tZ_I+\beta_t t Z I + β t , where P ( I = i ) = a i , M \mathbb{P}(I=i)=a_{i,M} P ( I = i ) = a i , M and β \beta β is an independent standard Brownian bridge from zero to zero. In particular, X M X^M X M extends continuously to t = 1 t=1 t = 1 ,
L ( X t M ∣ B M ) = ∑ i = 1 M a i , M N ( t Z i , t ( 1 − t ) I d ) , 0 < t < 1 , L ( X 1 M ∣ B M ) = ν M , (113) \mathcal{L}(X_t^M\mid B_M)=\sum_{i=1}^{M}a_{i,M}\mathcal{N}\left(tZ_i,t(1-t)I_d\right),\qquad0<t<1,\qquad\mathcal{L}(X_1^M\mid B_M)=\nu_M, \tag*{(113)} L ( X t M ∣ B M ) = i = 1 ∑ M a i , M N ( t Z i , t ( 1 − t ) I d ) , 0 < t < 1 , L ( X 1 M ∣ B M ) = ν M , (113) and, for 0 ≤ T < 1 0\le T<1 0 ≤ T < 1 ,
W 2 2 ( L ( X T M ∣ B M ) , ν M ) ≤ ( 1 − T ) 2 ∑ i = 1 M a i , M ∥ Z i ∥ 2 + d T ( 1 − T ) . (114) W_2^2\left(\mathcal{L}(X_T^M\mid B_M),\nu_M\right)\le(1-T)^2\sum_{i=1}^{M}a_{i,M}\lVert Z_i\rVert^2+dT(1-T). \tag*{(114)} W 2 2 ( L ( X T M ∣ B M ) , ν M ) ≤ ( 1 − T ) 2 i = 1 ∑ M a i , M ∥ Z i ∥ 2 + d T ( 1 − T ) . (114)
Proof. The proof is based on an explicit construction of the transition densities. For 0 ≤ s < t < 1 0\le s<t<1 0 ≤ s < t < 1 , define
p s , t ( x , z ) = φ t − s ( z − x ) H t ( z ) H s ( x ) . p_{s,t}(x,z)=\varphi_{t-s}(z-x)\frac{H_t(z)}{H_s(x)}. p s , t ( x , z ) = φ t − s ( z − x ) H s ( x ) H t ( z ) .
By the Gaussian convolution identity ∫ φ t − s ( z − x ) φ 1 − t ( Z i − z ) d z = φ 1 − s ( Z i − x ) \int\varphi_{t-s}(z-x)\varphi_{1-t}(Z_i-z)\,\mathrm{d}z=\varphi_{1-s}(Z_i-x) ∫ φ t − s ( z − x ) φ 1 − t ( Z i − z ) d z = φ 1 − s ( Z i − x ) , each p s , t ( x , ⋅ ) p_{s,t}(x,\cdot) p s , t ( x , ⋅ ) integrates to one. Hence these transition densities define a Markov process with drift ∇ log H t \nabla\log H_t ∇ log H t , whose density at time t t t , when started at zero, is φ t ( x ) H t ( x ) / H 0 ( 0 ) \varphi_t(x)H_t(x)/H_0(0) φ t ( x ) H t ( x ) / H 0 ( 0 ) . The identity
φ t ( x ) φ 1 − t ( Z i − x ) = φ ( Z i ) φ t ( 1 − t ) ( x − t Z i ) \varphi_t(x)\varphi_{1-t}(Z_i-x)=\varphi(Z_i)\varphi_{t(1-t)}(x-tZ_i) φ t ( x ) φ 1 − t ( Z i − x ) = φ ( Z i ) φ t ( 1 − t ) ( x − t Z i )
and H 0 ( 0 ) = ∑ i v i H_0(0)=\sum_i v_i H 0 ( 0 ) = ∑ i v i give (B.1) for 0 < t < 1 0<t<1 0 < t < 1 . The same transition densities arise by first choosing I I I with probabilities a i , M a_{i,M} a i , M and then running a Brownian bridge from zero to Z I Z_I Z I , which identifies the path distribution and its continuous endpoint. On every [ 0 , T ] [0,T] [ 0 , T ] with T < 1 T<1 T < 1 , the drift is smooth, globally Lipschitz in space, and of linear growth; see (B.3) below. Existence and pathwise uniqueness therefore identify this process with the strong solution. Finally, coupling X T M = T Z I + β T X_T^M=T Z_I+\beta_T X T M = T Z I + β T with Z I Z_I Z I and using the independence of β T \beta_T β T and Z I Z_I Z I together with E ∥ β T ∥ 2 = d T ( 1 − T ) \mathbb{E}\lVert\beta_T\rVert^2=dT(1-T) E ∥ β T ∥ 2 = d T ( 1 − T ) gives (B.2). This proves the desired result.
We now prove the Euler convergence (IV.7). Removing the terms with c i = 0 c_i=0 c i = 0 and merging coincident nodes, adding their coefficients and terminal probabilities, changes neither b ^ M , q \widehat b_{M,q} b M , q nor ν M \nu_M ν M . Without loss of generality, we therefore assume that ( Z i , c i , a i , M ) i = 1 n (Z_i,c_i,a_{i,M})_{i=1}^n ( Z i , c i , a i , M ) i = 1 n consists of distinct nodes with positive coefficients. Write
s = 1 − t , w i ( x , t ) = c i ϕ s ( Z i − x ) H t ( x ) , m M ( x , t ) = ∑ i = 1 n w i ( x , t ) Z i , R = max i ∥ Z i ∥ . s=1-t,\qquad w_i(x,t)=\frac{c_i\phi_s(Z_i-x)}{H_t(x)},\qquad m_M(x,t)=\sum_{i=1}^n w_i(x,t)Z_i,\qquad R=\max_i\lVert Z_i\rVert. s = 1 − t , w i ( x , t ) = H t ( x ) c i ϕ s ( Z i − x ) , m M ( x , t ) = i = 1 ∑ n w i ( x , t ) Z i , R = i max ∥ Z i ∥ .
Then b ^ M , q ( x , t ) = ( m M ( x , t ) − x ) / s \widehat b_{M,q}(x,t)=(m_M(x,t)-x)/s b M , q ( x , t ) = ( m M ( x , t ) − x ) / s and ∥ m M ( x , t ) ∥ ≤ R \lVert m_M(x,t)\rVert\le R ∥ m M ( x , t )∥ ≤ R . Since ∇ x w i = w i ( Z i − m M ) / s \nabla_xw_i=w_i(Z_i-m_M)/s ∇ x w i = w i ( Z i − m M ) / s , we have
∇ x b ^ M , q ( x , t ) = − I d s + Cov w ( x , t ) ( Z ) s 2 , ∥ ∇ x b ^ M , q ( x , t ) ∥ o p ≤ L ε : = ε − 1 + R 2 ε − 2 , t ≤ 1 − ε , (115) \nabla_x\widehat b_{M,q}(x,t)=-\frac{I_d}{s}+\frac{\operatorname{Cov}_{w(x,t)}(Z)}{s^2},\qquad\lVert\nabla_x\widehat b_{M,q}(x,t)\rVert_{\mathrm{op}}\le L_\varepsilon:=\varepsilon^{-1}+R^2\varepsilon^{-2},\qquad t\le1-\varepsilon, \tag*{(115)} ∇ x b M , q ( x , t ) = − s I d + s 2 Cov w ( x , t ) ( Z ) , ∥ ∇ x b M , q ( x , t ) ∥ op ≤ L ε := ε − 1 + R 2 ε − 2 , t ≤ 1 − ε , (115)
where Cov w ( x , t ) \operatorname{Cov}_{w(x,t)} Cov w ( x , t ) denotes the covariance under the probabilities w i ( x , t ) w_i(x,t) w i ( x , t ) .
Lemma B.2. Under the assumptions of Lemma B.1, let X ~ M , h \widetilde X^{M,h} X M , h be the Euler scheme with drift b ^ M , q \widehat b_{M,q} b M , q on a deterministic grid with t K = 1 t_K=1 t K = 1 and mesh h h h , driven by the same Brownian motion as X M X^M X M . Then E ∥ X ~ 1 M , h − X 1 M ∥ 2 → 0 \mathbb{E}\lVert\widetilde X^{M,h}_1-X^M_1\rVert^2\to0 E ∥ X 1 M , h − X 1 M ∥ 2 → 0 as h ↓ 0 h\downarrow0 h ↓ 0 . In particular, (IV.7) holds.
Proof. Write Y k = X ~ t k M , h Y_k=\widetilde X^{M,h}_{t_k} Y k = X t k M , h . Since W 2 2 ( L ( Y K ∣ B M ) , ν M ) ≤ E ∥ Y K − X 1 M ∥ 2 W_2^2(\mathcal{L}(Y_K\mid B_M),\nu_M)\le\mathbb{E}\lVert Y_K-X^M_1\rVert^2 W 2 2 ( L ( Y K ∣ B M ) , ν M ) ≤ E ∥ Y K − X 1 M ∥ 2 , it suffices to prove the first statement. The proof is based on splitting [ 0 , 1 ] [0,1] [ 0 , 1 ] into an initial interval [ 0 , T h ] [0,T_h] [ 0 , T h ] , on which the drift is Lipschitz, and a terminal interval [ T h , 1 ] [T_h,1] [ T h , 1 ] , on which the iterates are attracted to the endpoint Z I Z_I Z I .
We first estimate the error on the initial interval. Fix ε ∈ ( 0 , 1 ) \varepsilon\in(0,1) ε ∈ ( 0 , 1 ) , and let T h = t j T_h=t_j T h = t j be the last grid point not exceeding 1 − ε 1-\varepsilon 1 − ε . For h ≤ ε h\le\varepsilon h ≤ ε , we have ε ≤ 1 − T h ≤ 2 ε \varepsilon\le1-T_h\le2\varepsilon ε ≤ 1 − T h ≤ 2 ε . The backward heat equation for H H H implies
∂ t b ^ M , q + 1 2 Δ b ^ M , q + ( ∇ x b ^ M , q ) b ^ M , q = 0 , \partial_t\widehat b_{M,q}+\frac{1}{2}\Delta\widehat b_{M,q}+(\nabla_x\widehat b_{M,q})\widehat b_{M,q}=0, ∂ t b M , q + 2 1 Δ b M , q + ( ∇ x b M , q ) b M , q = 0 ,
so Itô’s formula gives d b ^ M , q ( X t M , t ) = ∇ x b ^ M , q ( X t M , t ) d B t \mathrm{d}\widehat b_{M,q}(X_t^M,t)=\nabla_x\widehat b_{M,q}(X_t^M,t)\,\mathrm{d}B_t d b M , q ( X t M , t ) = ∇ x b M , q ( X t M , t ) d B t . By (B.3) and the Itô isometry,
∥ b ^ M , q ( X u M , u ) − b ^ M , q ( X v M , v ) ∥ L 2 ≤ d L ε u − v , 0 ≤ v ≤ u ≤ T h . \left\lVert\widehat b_{M,q}(X_u^M,u)-\widehat b_{M,q}(X_v^M,v)\right\rVert_{L^2}\le\sqrt{d}\,L_\varepsilon\sqrt{u-v},\qquad0\le v\le u\le T_h. b M , q ( X u M , u ) − b M , q ( X v M , v ) L 2 ≤ d L ε u − v , 0 ≤ v ≤ u ≤ T h .
Let e k = ∥ Y k − X t k M ∥ L 2 e_k=\lVert Y_k-X^M_{t_k}\rVert_{L^2} e k = ∥ Y k − X t k M ∥ L 2 . Subtracting the Euler update from the integral equation and using Minkowski’s inequality, we obtain
e k + 1 ≤ ( 1 + L ε Δ t k ) e k + 2 3 d L ε ( Δ t k ) 3 / 2 , k < j . e_{k+1}\le(1+L_\varepsilon\Delta t_k)e_k+\frac{2}{3}\sqrt{d}\,L_\varepsilon(\Delta t_k)^{3/2},\qquad k<j. e k + 1 ≤ ( 1 + L ε Δ t k ) e k + 3 2 d L ε ( Δ t k ) 3/2 , k < j .
Since e 0 = 0 e_0=0 e 0 = 0 and ∑ k < j ( Δ t k ) 3 / 2 ≤ T h h \sum_{k<j}(\Delta t_k)^{3/2}\leq T_h\sqrt{h} ∑ k < j ( Δ t k ) 3/2 ≤ T h h , the discrete Gronwall inequality yields
e j ≤ 2 5 d L ε T h e L ε T h h . (116) e_j\leq\frac{2}{5}\sqrt{d}\,L_\varepsilon T_h e^{L_\varepsilon T_h}\sqrt{h}. \tag*{(116)} e j ≤ 5 2 d L ε T h e L ε T h h . (116)
It remains to control the terminal interval [ T h , 1 ] [T_h,1] [ T h , 1 ] . If n = 1 n=1 n = 1 , then m M ≡ Z 1 m_M\equiv Z_1 m M ≡ Z 1 , and the last update gives
Y K = Z 1 + B 1 − B t K − 1 , Y_K=Z_1+B_1-B_{t_{K-1}}, Y K = Z 1 + B 1 − B t K − 1 ,
so that E ∥ Y K − X 1 M ∥ 2 ≤ d h \mathbb{E}\lVert Y_K-X_1^M\rVert^2\leq dh E ∥ Y K − X 1 M ∥ 2 ≤ d h . Now let n ≥ 2 n\geq2 n ≥ 2 , set δ ∗ = min i ≠ l ∥ Z i − Z l ∥ > 0 \delta_*=\min_{i\ne l}\lVert Z_i-Z_l\rVert>0 δ ∗ = min i = l ∥ Z i − Z l ∥ > 0 , and fix 0 < ℓ < δ ∗ / 8 0<\ell<\delta_*/8 0 < ℓ < δ ∗ /8 . If ∥ x − Z i ∥ ≤ 2 ℓ \lVert x-Z_i\rVert\leq2\ell ∥ x − Z i ∥ ≤ 2 ℓ , then ∥ Z l − x ∥ ≥ 3 4 ∥ Z l − Z i ∥ \lVert Z_l-x\rVert\geq\frac{3}{4}\lVert Z_l-Z_i\rVert ∥ Z l − x ∥ ≥ 4 3 ∥ Z l − Z i ∥ and ∥ Z i − x ∥ ≤ 1 4 ∥ Z l − Z i ∥ \lVert Z_i-x\rVert\leq\frac{1}{4}\lVert Z_l-Z_i\rVert ∥ Z i − x ∥ ≤ 4 1 ∥ Z l − Z i ∥ for l ≠ i l\ne i l = i , so
∥ Z l − x ∥ 2 − ∥ Z i − x ∥ 2 ≥ 1 2 ∥ Z l − Z i ∥ 2 . \lVert Z_l-x\rVert^2-\lVert Z_i-x\rVert^2\geq\frac{1}{2}\lVert Z_l-Z_i\rVert^2. ∥ Z l − x ∥ 2 − ∥ Z i − x ∥ 2 ≥ 2 1 ∥ Z l − Z i ∥ 2 .
Since m M − Z i = ∑ l ≠ i w l ( Z l − Z i ) m_M-Z_i=\sum_{l\ne i}w_l(Z_l-Z_i) m M − Z i = ∑ l = i w l ( Z l − Z i ) and w l ≤ w l / w i w_l\leq w_l/w_i w l ≤ w l / w i , it follows that
∥ m M ( x , t ) − Z i ∥ ≤ Γ M exp ( − δ ∗ 2 4 ( 1 − t ) ) , Γ M = max i ∑ l ≠ i c l c i ∥ Z l − Z i ∥ . (117) \lVert m_M(x,t)-Z_i\rVert\leq\Gamma_M\exp\left(-\frac{\delta_*^2}{4(1-t)}\right),\qquad\Gamma_M=\max_i\sum_{l\ne i}\frac{c_l}{c_i}\lVert Z_l-Z_i\rVert. \tag*{(117)} ∥ m M ( x , t ) − Z i ∥ ≤ Γ M exp ( − 4 ( 1 − t ) δ ∗ 2 ) , Γ M = i max l = i ∑ c i c l ∥ Z l − Z i ∥ . (117)
whenever ∥ x − Z i ∥ ≤ 2 ℓ \lVert x-Z_i\rVert\leq2\ell ∥ x − Z i ∥ ≤ 2 ℓ . By Lemma B.1 , X 1 M = Z I X_1^M=Z_I X 1 M = Z I for a random index I I I with P ( I = i ) = a i , M \mathbb{P}(I=i)=a_{i,M} P ( I = i ) = a i , M . Set η ε = Γ M exp [ − δ ∗ 2 / ( 8 ε ) ] \eta_\varepsilon=\Gamma_M\exp[-\delta_*^2/(8\varepsilon)] η ε = Γ M exp [ − δ ∗ 2 / ( 8 ε )] , and take ε \varepsilon ε small enough that η ε ≤ ℓ / 4 \eta_\varepsilon\leq\ell/4 η ε ≤ ℓ /4 . Consider the event
G h , ε = { ∥ Y j − X T h M ∥ ≤ ℓ / 4 , ∥ X T h M − Z I ∥ ≤ ℓ / 4 , sup T h ≤ u ≤ 1 ∥ B u − B T h ∥ ≤ ℓ / 8 } . G_{h,\varepsilon}=\left\{\lVert Y_j-X_{T_h}^M\rVert\leq\ell/4,\quad\lVert X_{T_h}^M-Z_I\rVert\leq\ell/4,\quad\sup_{T_h\leq u\leq1}\lVert B_u-B_{T_h}\rVert\leq\ell/8\right\}. G h , ε = { ∥ Y j − X T h M ∥ ≤ ℓ /4 , ∥ X T h M − Z I ∥ ≤ ℓ /4 , T h ≤ u ≤ 1 sup ∥ B u − B T h ∥ ≤ ℓ /8 } .
By (B.2) and 1 − T h ≤ 2 ε 1-T_h\leq2\varepsilon 1 − T h ≤ 2 ε , we have E ∥ X T h M − Z I ∥ 2 ≤ 4 R 2 ε 2 + 2 d ε \mathbb{E}\lVert X_{T_h}^M-Z_I\rVert^2\leq4R^2\varepsilon^2+2d\varepsilon E ∥ X T h M − Z I ∥ 2 ≤ 4 R 2 ε 2 + 2 d ε . By Markov’s inequality and Doob’s maximal inequality, we have
P ( G h , ε c ) ≤ 16 e j 2 + 64 R 2 ε 2 + 544 d ε ℓ 2 . (118) \mathbb{P}(G_{h,\varepsilon}^{c})\leq\frac{16e_j^2+64R^2\varepsilon^2+544d\varepsilon}{\ell^2}. \tag*{(118)} P ( G h , ε c ) ≤ ℓ 2 16 e j 2 + 64 R 2 ε 2 + 544 d ε . (118)
We note that this bound does not use independence between I I I and the subsequent Brownian increments.
Write α k = Δ t k / ( 1 − t k ) ∈ ( 0 , 1 ] \alpha_k=\Delta t_k/(1-t_k)\in(0,1] α k = Δ t k / ( 1 − t k ) ∈ ( 0 , 1 ] . The remaining Euler updates satisfy
Y k + 1 − Z I = ( 1 − α k ) ( Y k − Z I ) + α k { m M ( Y k , t k ) − Z I } + ( B t k + 1 − B t k ) , k ≥ j . (119) Y_{k+1}-Z_I=(1-\alpha_k)(Y_k-Z_I)+\alpha_k\{m_M(Y_k,t_k)-Z_I\}+(B_{t_{k+1}}-B_{t_k}),\qquad k\geq j. \tag*{(119)} Y k + 1 − Z I = ( 1 − α k ) ( Y k − Z I ) + α k { m M ( Y k , t k ) − Z I } + ( B t k + 1 − B t k ) , k ≥ j . (119)
Iterating (B.7) from j j j to k k k , we find that the noise increments B t l + 1 − B t l B_{t_{l+1}}-B_{t_l} B t l + 1 − B t l , j ≤ l < k j\leq l<k j ≤ l < k , enter with the coefficients ∏ u = l + 1 k − 1 ( 1 − α u ) \prod_{u=l+1}^{k-1}(1-\alpha_u) ∏ u = l + 1 k − 1 ( 1 − α u ) , which are deterministic, nondecreasing in l l l , and between zero and one. By Abel summation, the noise contribution is bounded by 2 sup T h ≤ u ≤ 1 ∥ B u − B T h ∥ 2\sup_{T_h\leq u\leq1}\lVert B_u-B_{T_h}\rVert 2 sup T h ≤ u ≤ 1 ∥ B u − B T h ∥ . The drift residuals enter with nonnegative coefficients whose sum is at most one. Hence, if Y j , … , Y k − 1 Y_j,\ldots,Y_{k-1} Y j , … , Y k − 1 lie within distance 2 ℓ 2\ell 2 ℓ of Z I Z_I Z I , then (B.5) bounds each drift residual by η ε \eta_\varepsilon η ε , and on G h , ε G_{h,\varepsilon} G h , ε ,
∥ Y k − Z I ∥ ≤ ∥ Y j − Z I ∥ + η ε + 2 sup T h ≤ u ≤ 1 ∥ B u − B T h ∥ ≤ ℓ 2 + ℓ 4 + ℓ 4 = ℓ . \lVert Y_k-Z_I\rVert\leq\lVert Y_j-Z_I\rVert+\eta_\varepsilon+2\sup_{T_h\leq u\leq1}\lVert B_u-B_{T_h}\rVert\leq\frac{\ell}{2}+\frac{\ell}{4}+\frac{\ell}{4}=\ell. ∥ Y k − Z I ∥ ≤ ∥ Y j − Z I ∥ + η ε + 2 T h ≤ u ≤ 1 sup ∥ B u − B T h ∥ ≤ 2 ℓ + 4 ℓ + 4 ℓ = ℓ .
By induction, on G h , ε G_{h,\varepsilon} G h , ε the iterates never leave the 2 ℓ 2\ell 2 ℓ -neighborhood of Z I Z_I Z I . Since α K − 1 = 1 \alpha_{K-1}=1 α K − 1 = 1 , the final update is Y K = m M ( Y K − 1 , t K − 1 ) + B 1 − B t K − 1 Y_K=m_M(Y_{K-1},t_{K-1})+B_1-B_{t_{K-1}} Y K = m M ( Y K − 1 , t K − 1 ) + B 1 − B t K − 1 , and ∥ m M ( Y K − 1 , t K − 1 ) − Z I ∥ \lVert m_M(Y_{K-1},t_{K-1})-Z_I\rVert ∥ m M ( Y K − 1 , t K − 1 ) − Z I ∥ is at most η ε \eta_\varepsilon η ε on G h , ε G_{h,\varepsilon} G h , ε and at most 2 R 2R 2 R everywhere. Therefore,
E ∥ Y K − X 1 M ∥ 2 ≤ 2 η ε 2 + 8 R 2 P ( G h , ε c ) + 2 d h . (120) \mathbb{E}\lVert Y_K-X_1^M\rVert^2\leq2\eta_\varepsilon^2+8R^2\mathbb{P}(G_{h,\varepsilon}^{c})+2dh. \tag*{(120)} E ∥ Y K − X 1 M ∥ 2 ≤ 2 η ε 2 + 8 R 2 P ( G h , ε c ) + 2 d h . (120)
We now conclude. For fixed ε \varepsilon ε , e j → 0 e_j \to0 e j → 0 as h ↓ 0 h \downarrow0 h ↓ 0 by (B.4). Taking the limit superior as h ↓ 0 h \downarrow0 h ↓ 0 in (B.8) and using (B.6), we obtain
lim sup h ↓ 0 E ∥ Y K − X 1 M ∥ 2 ≤ 2 η ε 2 + 8 R 2 ( 64 R 2 ε 2 + 544 d ε ) ℓ 2 . \limsup_{h\downarrow0}\mathbb{E}\lVert Y_K-X_1^M\rVert^2 \le2\eta_\varepsilon^2+\frac{8R^2(64R^2\varepsilon^2+544d\varepsilon)}{\ell^2}. h ↓ 0 lim sup E ∥ Y K − X 1 M ∥ 2 ≤ 2 η ε 2 + ℓ 2 8 R 2 ( 64 R 2 ε 2 + 544 d ε ) .
Letting ε ↓ 0 \varepsilon\downarrow0 ε ↓ 0 , the right-hand side tends to zero. This proves the desired result.
Based on Lemmas B.1 and B.2, we prove Theorem IV.2.
Proof of Theorem IV.2. Part (i) follows from Lemmas B.1 and B.2. It remains to prove part (ii). Write m 2 ( π ) = ∫ R d ∥ z ∥ 2 π ( z ) d z m_2(\pi)=\int_{\mathbb{R}^d}\lVert z\rVert^2\pi(z)\,dz m 2 ( π ) = ∫ R d ∥ z ∥ 2 π ( z ) d z . We first prove (IV.9). By Tonelli’s theorem and (II.2), we have, for bounded measurable g g g ,
E q v 1 = Z , E q [ v 1 g ( Z 1 ) ] = Z ∫ g ( z ) π ( z ) d z , E q [ v 1 ∥ Z 1 ∥ 2 ] = Z m 2 ( π ) , \mathbb{E}_qv_1=\mathcal{Z},\qquad\mathbb{E}_q[v_1g(Z_1)]=\mathcal{Z}\int g(z)\pi(z)\,dz,\qquad\mathbb{E}_q[v_1\lVert Z_1\rVert^2]=\mathcal{Z}m_2(\pi), E q v 1 = Z , E q [ v 1 g ( Z 1 )] = Z ∫ g ( z ) π ( z ) d z , E q [ v 1 ∥ Z 1 ∥ 2 ] = Z m 2 ( π ) ,
where the last identity uses nonnegativity. All these variables are integrable. Applying the strong law of large numbers simultaneously to a countable convergence-determining class of bounded continuous functions and to v 1 ∥ Z 1 ∥ 2 v_1\lVert Z_1\rVert^2 v 1 ∥ Z 1 ∥ 2 , we obtain, almost surely,
1 M ∑ i = 1 M v i → Z > 0 , ν M ⇒ π , ∫ ∥ z ∥ 2 ν M ( d z ) → m 2 ( π ) . \frac{1}{M}\sum_{i=1}^{M}v_i\to\mathcal{Z}>0,\qquad\nu_M\Rightarrow\pi,\qquad\int\lVert z\rVert^2\nu_M(dz)\to m_2(\pi). M 1 i = 1 ∑ M v i → Z > 0 , ν M ⇒ π , ∫ ∥ z ∥ 2 ν M ( d z ) → m 2 ( π ) .
In particular, ∑ i = 1 M v i > 0 \sum_{i=1}^{M}v_i>0 ∑ i = 1 M v i > 0 for all sufficiently large M M M . Since weak convergence together with convergence of second moments is equivalent to convergence in W 2 W_2 W 2 , this proves (IV.9). By the triangle inequality,
W 2 ( L ( X ~ 1 M , h ∣ B M ) , π ) ≤ W 2 ( L ( X ~ 1 M , h ∣ B M ) , ν M ) + W 2 ( ν M , π ) , W_2\bigl(\mathcal{L}(\widetilde X_1^{M,h}\mid B_M),\pi\bigr)\le W_2\bigl(\mathcal{L}(\widetilde X_1^{M,h}\mid B_M),\nu_M\bigr)+W_2(\nu_M,\pi), W 2 ( L ( X 1 M , h ∣ B M ) , π ) ≤ W 2 ( L ( X 1 M , h ∣ B M ) , ν M ) + W 2 ( ν M , π ) ,
and (IV.10) follows from (IV.7) and (IV.9).
For the diagonal limit, let D M , K D_{M,K} D M , K denote the conditional W 2 W_2 W 2 distance in (IV.7) on the uniform grid with K K K steps. On the event { ∑ i = 1 M v i > 0 } \{\sum_{i=1}^{M}v_i>0\} { ∑ i = 1 M v i > 0 } , D M , K → 0 D_{M,K}\to0 D M , K → 0 as K → ∞ K\to\infty K → ∞ for each fixed M M M . By bounded convergence, we can choose deterministic increasing integers K M ≥ M K_M\ge M K M ≥ M such that
P ( ∑ i = 1 M v i > 0 , D M , K M > 1 / M ) ≤ 2 − M . \mathbb{P}\left(\sum_{i=1}^{M}v_i>0,\ D_{M,K_M}>1/M\right)\le2^{-M}. P ( i = 1 ∑ M v i > 0 , D M , K M > 1/ M ) ≤ 2 − M .
By the Borel–Cantelli lemma and the eventual positivity of ∑ i v i \sum_i v_i ∑ i v i , almost surely D M , K M ≤ 1 / M D_{M,K_M}\le1/M D M , K M ≤ 1/ M for all sufficiently large M M M . Together with (IV.9) and the triangle inequality, this proves (IV.11).
Proof of Corollary IV.1
Proof. We first prove that E W 2 2 ( ν M , π ) → 0 \mathbb{E}W_2^2(\nu_M,\pi)\to0 E W 2 2 ( ν M , π ) → 0 . Set S M = ∑ i = 1 M v i S_M=\sum_{i=1}^{M}v_i S M = ∑ i = 1 M v i , R M = max i ≤ M ∥ Z i ∥ R_M=\max_{i\le M}\lVert Z_i\rVert R M = max i ≤ M ∥ Z i ∥ , and J M = ∫ ∥ z ∥ 2 ν M ( d z ) J_M=\int\lVert z\rVert^2\nu_M(dz) J M = ∫ ∥ z ∥ 2 ν M ( d z ) . Since v 1 > 0 v_1>0 v 1 > 0 almost surely, S M > 0 S_M>0 S M > 0 and ν M \nu_M ν M is defined for every M M M . We first show that ( J M ) (J_M) ( J M ) is uniformly integrable. On G M = { S M ≥ M Z / 2 } G_M=\{S_M\ge M\mathcal{Z}/2\} G M = { S M ≥ M Z /2 } , we have
J M ≤ 2 Z 1 M ∑ i = 1 M v i ∥ Z i ∥ 2 , J_M\le\frac{2}{\mathcal{Z}}\frac{1}{M}\sum_{i=1}^{M}v_i\lVert Z_i\rVert^2, J M ≤ Z 2 M 1 i = 1 ∑ M v i ∥ Z i ∥ 2 ,
and the sample average on the right converges in L 1 L^{1} L 1 , so it is uniformly integrable. On G M c G_{M}^{c} G M c , we use J M ≤ R M 2 ≤ ∑ i = 1 M ∥ Z i ∥ 2 J_{M} \le R_{M}^{2} \le\sum_{i=1}^{M}\lVert Z_{i}\rVert^{2} J M ≤ R M 2 ≤ ∑ i = 1 M ∥ Z i ∥ 2 . Since v i ≥ 0 v_{i} \ge0 v i ≥ 0 , we have S M ≥ ∑ l ≠ i v l S_{M} \ge\sum_{l\ne i} v_{l} S M ≥ ∑ l = i v l , and independence gives
E [ R M 2 1 G M c ] ≤ M E q ∥ Z 1 ∥ 2 P ( ∑ i = 1 M − 1 v i < M Z / 2 ) ≤ M E q ∥ Z 1 ∥ 2 ϑ M − 1 , M ≥ 3 , \mathbb{E}\left[R_{M}^{2}\mathbf{1}_{G_{M}^{c}}\right] \le M\mathbb{E}_{q}\lVert Z_{1}\rVert^{2}\mathbb{P}\left(\sum_{i=1}^{M-1}v_{i}<M\mathcal{Z}/2\right) \le M\mathbb{E}_{q}\lVert Z_{1}\rVert^{2}\vartheta^{M-1}, \qquad M\ge3, E [ R M 2 1 G M c ] ≤ M E q ∥ Z 1 ∥ 2 P ( i = 1 ∑ M − 1 v i < M Z /2 ) ≤ M E q ∥ Z 1 ∥ 2 ϑ M − 1 , M ≥ 3 ,
for some ϑ < 1 \vartheta< 1 ϑ < 1 . Indeed, since E v 1 = Z > 0 \mathbb{E}v_{1}=\mathcal{Z}>0 E v 1 = Z > 0 , there is θ > 0 \theta>0 θ > 0 such that ϑ : = e 3 θ Z / 4 E e − θ v 1 < 1 \vartheta:= e^{3\theta\mathcal{Z}/4}\mathbb{E}e^{-\theta v_{1}}<1 ϑ := e 3 θ Z /4 E e − θ v 1 < 1 , and the bound follows from the exponential Markov inequality and M / 2 ≤ 3 ( M − 1 ) / 4 M/2\le3(M-1)/4 M /2 ≤ 3 ( M − 1 ) /4 for M ≥ 3 M\ge3 M ≥ 3 . Hence the contribution of G M c G_{M}^{c} G M c tends to zero in L 1 L^{1} L 1 , and ( J M ) (J_{M}) ( J M ) is uniformly integrable. Since W 2 2 ( ν M , π ) ≤ 2 J M + 2 m 2 ( π ) W_{2}^{2}(\nu_{M},\pi)\le2J_{M}+2m_{2}(\pi) W 2 2 ( ν M , π ) ≤ 2 J M + 2 m 2 ( π ) , (IV.9) and uniform integrability yield E W 2 2 ( ν M , π ) → 0 \mathbb{E}W_{2}^{2}(\nu_{M},\pi)\to0 E W 2 2 ( ν M , π ) → 0 .
Next, for fixed M M M , coupling the last Euler update with an independent draw from ν M \nu_{M} ν M , conditionally on B M \mathcal{B}_{M} B M , gives
D M , h 2 : = W 2 2 ( L ( X ~ 1 M , h ∣ B M ) , ν M ) ≤ 4 R M 2 + d h . D_{M,h}^{2}:=W_{2}^{2}\left(\mathcal{L}(\widetilde{X}_{1}^{M,h}\mid\mathcal{B}_{M}),\nu_{M}\right)\le4R_{M}^{2}+dh. D M , h 2 := W 2 2 ( L ( X 1 M , h ∣ B M ) , ν M ) ≤ 4 R M 2 + d h .
Since E R M 2 ≤ M E q ∥ Z 1 ∥ 2 < ∞ \mathbb{E}R_{M}^{2}\le M\mathbb{E}_{q}\lVert Z_{1}\rVert^{2}<\infty E R M 2 ≤ M E q ∥ Z 1 ∥ 2 < ∞ , (IV.7) and dominated convergence give E D M , h 2 → 0 \mathbb{E}D_{M,h}^{2}\to0 E D M , h 2 → 0 as h ↓ 0 h\downarrow0 h ↓ 0 . Mixing the conditional couplings over B M \mathcal{B}_{M} B M , we obtain
W 2 ( L ( X ~ 1 M , h ) , π ) ≤ ( E D M , h 2 ) 1 / 2 + ( E W 2 2 ( ν M , π ) ) 1 / 2 , W_{2}\left(\mathcal{L}(\widetilde{X}_{1}^{M,h}),\pi\right)\le\left(\mathbb{E}D_{M,h}^{2}\right)^{1/2}+\left(\mathbb{E}W_{2}^{2}(\nu_{M},\pi)\right)^{1/2}, W 2 ( L ( X 1 M , h ) , π ) ≤ ( E D M , h 2 ) 1/2 + ( E W 2 2 ( ν M , π ) ) 1/2 ,
which proves the iterated limit. For the diagonal limit, we choose the deterministic K M K_{M} K M larger if necessary so that, in addition to the probability bound in the proof of Theorem IV.2, E D M , 1 / K M 2 ≤ 2 − M \mathbb{E}D_{M,1/K_{M}}^{2}\le2^{-M} E D M , 1/ K M 2 ≤ 2 − M . This proves the desired result.
Proof of Theorem IV.3
Proof. We suppress the dependence on the fixed ( x , t ) (x,t) ( x , t ) and define
χ i q = Ω q ( Z i , ξ i ; x , t ) C ( x , t ) { Z i − x 1 − t − b ( x , t ) } , C ^ M , q = 1 M ∑ i = 1 M Ω q ( Z i , ξ i ; x , t ) . (121) \chi_{i}^{q}=\frac{\Omega_{q}(Z_{i},\xi_{i};x,t)}{C(x,t)}\left\{\frac{Z_{i}-x}{1-t}-b(x,t)\right\}, \qquad\widehat{C}_{M,q}=\frac{1}{M}\sum_{i=1}^{M}\Omega_{q}(Z_{i},\xi_{i};x,t). \tag*{(121)} χ i q = C ( x , t ) Ω q ( Z i , ξ i ; x , t ) { 1 − t Z i − x − b ( x , t ) } , C M , q = M 1 i = 1 ∑ M Ω q ( Z i , ξ i ; x , t ) . (121)
By (III.13) and (III.14), E q χ i q = 0 \mathbb{E}_{q}\chi_{i}^{q}=0 E q χ i q = 0 , and by (IV.14), E q ∥ χ i q ∥ 2 < ∞ \mathbb{E}_{q}\lVert\chi_{i}^{q}\rVert^{2}<\infty E q ∥ χ i q ∥ 2 < ∞ . Direct subtraction of the two ratios gives
b ^ M , q − b = C ( x , t ) C ^ M , q 1 M ∑ i = 1 M χ i q . (122) \widehat{b}_{M,q}-b=\frac{C(x,t)}{\widehat{C}_{M,q}}\frac{1}{M}\sum_{i=1}^{M}\chi_{i}^{q}. \tag*{(122)} b M , q − b = C M , q C ( x , t ) M 1 i = 1 ∑ M χ i q . (122)
By the strong law of large numbers, C ^ M , q → C ( x , t ) > 0 \widehat{C}_{M,q}\to C(x,t)>0 C M , q → C ( x , t ) > 0 and M − 1 ∑ i χ i q → 0 M^{-1}\sum_{i}\chi_{i}^{q}\to0 M − 1 ∑ i χ i q → 0 almost surely. Hence b ^ M , q \widehat{b}_{M,q} b M , q is well defined for all sufficiently large M M M , and (IV.15) follows from (B.10). By the multivariate central limit theorem,
M − 1 / 2 ∑ i = 1 M χ i q ⇒ N ( 0 , E q [ χ 1 q ( χ 1 q ) T ] ) . (123) M^{-1/2}\sum_{i=1}^{M}\chi_{i}^{q}\Rightarrow\mathcal{N}\left(0,\mathbb{E}_{q}\left[\chi_{1}^{q}(\chi_{1}^{q})^{\mathsf{T}}\right]\right). \tag*{(123)} M − 1/2 i = 1 ∑ M χ i q ⇒ N ( 0 , E q [ χ 1 q ( χ 1 q ) T ] ) . (123)
Since χ i q = Ω q ( Z i , ξ i ; x , t ) ( Z i − z ˉ ) / [ ( 1 − t ) C ( x , t ) ] \chi_{i}^{q}=\Omega_{q}(Z_{i},\xi_{i};x,t)(Z_{i}-\bar{z})/[(1-t)C(x,t)] χ i q = Ω q ( Z i , ξ i ; x , t ) ( Z i − z ˉ ) / [( 1 − t ) C ( x , t )] , Slutsky’s theorem applied to (B.10) gives (IV.16) with covariance (IV.17). This proves the desired result.
Variance and effective sample size
By (IV.17) and the definition of Ω q \Omega_q Ω q ,
E q [ Ω q 2 ∥ Z − z ˉ ∥ 2 ] = ∫ q ( z ) φ 1 − t ( z − x ) 2 q ( z ) 2 E ξ ∣ z [ W ( z , ξ ) 2 ] ∥ z − z ˉ ∥ 2 d z = V x , t ( q ) , \mathbb{E}_q\left[\Omega_q^2\lVert Z-\bar{z}\rVert^2\right]=\int q(z)\frac{\varphi_{1-t}(z-x)^2}{q(z)^2}\mathbb{E}_{\xi\mid z}\left[W(z,\xi)^2\right]\lVert z-\bar{z}\rVert^2\,\mathrm{d}z=V_{x,t}(q), E q [ Ω q 2 ∥ Z − z ˉ ∥ 2 ] = ∫ q ( z ) q ( z ) 2 φ 1 − t ( z − x ) 2 E ξ ∣ z [ W ( z , ξ ) 2 ] ∥ z − z ˉ ∥ 2 d z = V x , t ( q ) ,
which gives (IV.19). For any proposal q q q satisfying the support condition, the Cauchy–Schwarz inequality gives
( ∫ A x , t ( z ) d z ) 2 ≤ ( ∫ A x , t ( z ) q ( z ) d z ) ( ∫ q ( z ) d z ) = V x , t ( q ) , (124) \left(\int\sqrt{A_{x,t}(z)}\,\mathrm{d}z\right)^2\leq\left(\int\frac{A_{x,t}(z)}{q(z)}\,\mathrm{d}z\right)\left(\int q(z)\,\mathrm{d}z\right)=V_{x,t}(q), \tag*{(124)} ( ∫ A x , t ( z ) d z ) 2 ≤ ( ∫ q ( z ) A x , t ( z ) d z ) ( ∫ q ( z ) d z ) = V x , t ( q ) , (124)
with equality for q = q x , t ∗ q=q^*_{x,t} q = q x , t ∗ in (IV.20) whenever q x , t ∗ q^*_{x,t} q x , t ∗ satisfies the support condition. Finally, q 3 ≥ 0.05 q 1 q_3\geq0.05q_1 q 3 ≥ 0.05 q 1 gives A x , t / q 3 ≤ 20 A x , t / q 1 A_{x,t}/q_3\leq20A_{x,t}/q_1 A x , t / q 3 ≤ 20 A x , t / q 1 , and integrating proves (IV.21).
Remark B.1 (Local ESS and terminal importance weights). In the bridge representation of Lemma B.1 , Bayes’ formula and the Gaussian density identity in its proof give
P ( I = i ∣ X t M = x , B M ) = c i φ 1 − t ( Z i − x ) ∑ j c j φ 1 − t ( Z j − x ) . (125) \mathbb{P}(I=i\mid X_t^M=x,B_M)=\frac{c_i\varphi_{1-t}(Z_i-x)}{\sum_j c_j\varphi_{1-t}(Z_j-x)}. \tag*{(125)} P ( I = i ∣ X t M = x , B M ) = ∑ j c j φ 1 − t ( Z j − x ) c i φ 1 − t ( Z i − x ) . (125)
Thus (III.21) measures the concentration of the conditional endpoint probabilities. In contrast, the effective sample size of the terminal importance weights is
ESS t e r m = ( ∑ i a i , M 2 ) − 1 = ESS ( 0 , 0 ) . \operatorname{ESS}_{\mathrm{term}}=\left(\sum_i a_{i,M}^2\right)^{-1}=\operatorname{ESS}(0,0). ESS term = ( i ∑ a i , M 2 ) − 1 = ESS ( 0 , 0 ) .
If the nodes with positive weights are distinct, then X t M → Z I X_t^M\to Z_I X t M → Z I almost surely and, for j ≠ I j\ne I j = I ,
c j φ 1 − t ( Z j − X t M ) c I φ 1 − t ( Z I − X t M ) = c j c I exp { − ∥ Z j − X t M ∥ 2 − ∥ Z I − X t M ∥ 2 2 ( 1 − t ) } ⟶ 0. \frac{c_j\varphi_{1-t}(Z_j-X_t^M)}{c_I\varphi_{1-t}(Z_I-X_t^M)}=\frac{c_j}{c_I}\exp\left\{-\frac{\lVert Z_j-X_t^M\rVert^2-\lVert Z_I-X_t^M\rVert^2}{2(1-t)}\right\}\longrightarrow0. c I φ 1 − t ( Z I − X t M ) c j φ 1 − t ( Z j − X t M ) = c I c j exp { − 2 ( 1 − t ) ∥ Z j − X t M ∥ 2 − ∥ Z I − X t M ∥ 2 } ⟶ 0.
Consequently, ESS ( X t M , t ) → 1 \operatorname{ESS}(X_t^M,t)\to1 ESS ( X t M , t ) → 1 almost surely; for repeated nodes, the same conclusion holds after combining the weights at each distinct node. A low local ESS near time one therefore does not by itself indicate posterior error, a lack of independent outputs, or numerical instability, since it also occurs under exact sampling of ν M \nu_M ν M .