Introduction
A generative model specifies a mechanism: an observation is a deterministic transform, indexed by a structural parameter θ \theta θ Y = G ( θ , X ) Y = G(\theta, X) Y = G ( θ , X ) X ∼ ν X \sim\nu X ∼ ν X X X Y Y Y where it fails. For many generative models the likelihood is unavailable or intractable, while simulation from G G G [33 Eq. (1.9] .
The procedure splits the observations into an estimation block ξ 0 \xi_0 ξ 0 ξ 1 \xi_1 ξ 1
Estimation stage. Using ξ 0 \xi_0 ξ 0 θ \theta θ θ ~ \tilde{\theta} θ ~ [5 ] (Bassetti et al. 2006), computed by sample average approximation [36 ] . Minimum-distance estimators of this kind are robust [11 ] (Bassetti and Regazzini 2006), so a mild misfit of the noise law does not drag θ ~ \tilde{\theta} θ ~
Recourse stage. With θ \theta θ θ ~ \tilde{\theta} θ ~ x ˉ 1 , … , x ˉ N 1 \bar{x}_1,\ldots,\bar{x}_{N_1} x ˉ 1 , … , x ˉ N 1 mesh ), and push each through the fitted mechanism G ( θ ~ , ⋅ ) G(\tilde{\theta},\cdot) G ( θ ~ , ⋅ ) z ℓ = G ( θ ~ , x ˉ j ) z_{\ell}=G(\tilde{\theta},\bar{x}_j) z ℓ = G ( θ ~ , x ˉ j ) atoms of the fitted model, interpreted as typical observations the fitted model can produce, one for each representative noise value (noise values that produce the same observation are merged into one atom). Each atom owns a cell , the set of observations closer to it than to any other atom. The fitted model assigns a probability to every cell and the held-out block gives an empirical frequency for every cell. The recourse decision is the vector w \mathbf{w} w w \mathbf{w} w θ ~ \tilde{\theta} θ ~
The split is what makes this test calibrated, and splitting is familiar in Bayesian model criticism. In Moran et al. [29 ] and Li and Huggins [27 ] the split repairs a calibration defect as the posterior predictive p p p [17 ] , and holding out a block restores uniformity. There the two halves play the roles of “train” and “test” for one and the same object, the predictive distribution. Our two halves instead determine two different decisions. ξ 0 \xi_0 ξ 0 θ \theta θ ν \nu ν ξ 1 \xi_1 ξ 1 w \mathbf{w} w θ \theta θ θ ~ \tilde{\theta} θ ~ θ ~ \tilde{\theta} θ ~ ξ 0 \xi_0 ξ 0 θ ~ \tilde{\theta} θ ~ θ ~ \tilde{\theta} θ ~ G G G ν \nu ν
The program of criticizing a model by confronting it with data it was not fitted to descends from Box [8 ] , who separated estimation within a model from criticism of the model itself, and from the posterior predictive assessment of Gelman et al. [18 ] . The resulting p p p [4 ] , and it motivates the split constructions of Moran et al. [29 ] and Li and Huggins [27 ] discussed above. Our structural stance is closest to Covington and Miller [10 ] , who also criticize a model through a parametrization of its noise rather than through a statistic of the observations.
The paper is organized as follows. § 2 recalls the Wasserstein distance and two-stage stochastic programs, and casts model criticism as a two-stage program (§ 2.3 ). § 3 constructs the ingredients: the structural model, including stochastic processes, the split, the synthetic noise of the estimation stage, and the recourse mesh. § 4 develops the two stages, the prior test, the diagnostic and the calibrated test. § 5 states the assumptions under which both stages are consistent. § 6 illustrates the procedure on four models. § 7 describes the algorithms, and § 8 discusses limitations and extensions. Proofs, and the verification of the assumptions for the examples are in the appendix.
Preliminaries
The space of measures and the Wasserstein distance
Throughout, the sample space S S S d d d S = R d S = \mathbb{R}^{d} S = R d L 1 ( [ 0 , T ] ; R d ) L^{1}([0,T];\mathbb{R}^{d}) L 1 ([ 0 , T ] ; R d ) L 2 ( [ 0 , T ] ; R d ) L^{2}([0,T];\mathbb{R}^{d}) L 2 ([ 0 , T ] ; R d ) C ( [ 0 , T ] ; R d ) C([0,T];\mathbb{R}^{d}) C ([ 0 , T ] ; R d ) S S S d ( ω 1 , ω 2 ) = ∥ ω 1 − ω 2 ∥ d(\omega_{1},\omega_{2}) = \lVert\omega_{1}-\omega_{2}\rVert d ( ω 1 , ω 2 ) = ∥ ω 1 − ω 2 ∥ P ( S ) \mathcal{P}(S) P ( S ) S S S
P 1 ( S ) : = { μ ∈ P ( S ) : ∫ S d ( ω 0 , ω ) μ ( d ω ) < ∞ for some (all) ω 0 ∈ S } . \mathcal{P}_{1}(S) := \left\{ \mu\in\mathcal{P}(S) : \int_{S} d(\omega_{0},\omega) \,\mu(\mathrm{d}\omega) < \infty\text{ for some (all) } \omega_{0} \in S \right\}. P 1 ( S ) := { μ ∈ P ( S ) : ∫ S d ( ω 0 , ω ) μ ( d ω ) < ∞ for some (all) ω 0 ∈ S } . For μ , ν ∈ P 1 ( S ) \mu,\nu\in\mathcal{P}_{1}(S) μ , ν ∈ P 1 ( S )
W 1 ( μ , ν ) : = inf π ∈ Π ( μ , ν ) ∫ S × S d ( ω 1 , ω 2 ) π ( d ω 1 , d ω 2 ) , (1) W_{1}(\mu,\nu) := \inf_{\pi\in\Pi(\mu,\nu)} \int_{S \times S} d(\omega_{1},\omega_{2}) \,\pi(\mathrm{d}\omega_{1},\mathrm{d}\omega_{2}), \tag*{(1)} W 1 ( μ , ν ) := π ∈ Π ( μ , ν ) inf ∫ S × S d ( ω 1 , ω 2 ) π ( d ω 1 , d ω 2 ) , (1) the infimum being over the set of couplings Π \Pi Π μ \mu μ ν \nu ν [33 , 41 Ch. 2] ,
W 1 ( μ , ν ) = sup { ∫ S f d ( μ − ν ) : ∥ f ∥ L i p ≤ 1 } . (2) W_{1}(\mu,\nu) = \sup\left\{ \int_{S} f \,\mathrm{d}(\mu-\nu) : \lVert f\rVert_{\mathrm{Lip}} \le1 \right\}. \tag*{(2)} W 1 ( μ , ν ) = sup { ∫ S f d ( μ − ν ) : ∥ f ∥ Lip ≤ 1 } . (2) The metric space ( P 1 ( S ) , W 1 ) (\mathcal{P}_{1}(S), W_{1}) ( P 1 ( S ) , W 1 ) C C C S S S molecular probability measures with rational coefficients on C C C
{ ∑ i = 1 n q i δ c i ∣ c i ∈ C , 0 ≤ q i ∈ Q , ∑ i = 1 n q i = 1 , n ∈ N } , (3) \left\{ \sum_{i=1}^{n} q_{i}\delta_{c_{i}} \mid c_{i} \in C,\ 0 \le q_{i} \in\mathbb{Q},\ \sum_{i=1}^{n} q_{i} = 1,\ n \in\mathbb{N} \right\}, \tag*{(3)} { i = 1 ∑ n q i δ c i ∣ c i ∈ C , 0 ≤ q i ∈ Q , i = 1 ∑ n q i = 1 , n ∈ N } , (3) is a countable dense subset of ( P 1 ( S ) , W 1 ) (\mathcal{P}_{1}(S), W_{1}) ( P 1 ( S ) , W 1 ) [12 ] and Düll et al. [13 ] for more details. In contrast, P 1 ( S ) \mathcal{P}_{1}(S) P 1 ( S ) ∥ ⋅ ∥ T V \lVert\cdot\rVert_{\mathrm{TV}} ∥ ⋅ ∥ TV W 1 W_{1} W 1 P 1 ( S ) \mathcal{P}_{1}(S) P 1 ( S ) [41 ] .
Example 1. For a ≠ b a \ne b a = b ∥ δ a − δ b ∥ T V = 1 \lVert\delta_{a}-\delta_{b}\rVert_{\mathrm{TV}} = 1 ∥ δ a − δ b ∥ TV = 1 a n → b a_{n} \to b a n → b S S S ∥ δ a n − δ b ∥ T V → 0 \lVert\delta_{a_{n}}-\delta_{b}\rVert_{\mathrm{TV}} \to0 ∥ δ a n − δ b ∥ TV → 0 W 1 ( δ a n , δ b ) = d ( a n , b ) W_{1}(\delta_{a_{n}},\delta_{b}) = d(a_{n},b) W 1 ( δ a n , δ b ) = d ( a n , b ) a n → b a_{n} \to b a n → b W 1 ( δ a n , δ b ) → 0 W_{1}(\delta_{a_{n}},\delta_{b}) \to0 W 1 ( δ a n , δ b ) → 0
Two-stage stochastic programs
A stochastic optimization problem selects a decision to optimize a cost that depends on uncertainty whose distribution is known. In the canonical form of Pflug and Pichler [33 ] ,
min x ∈ X R P [ Q ( x , ξ ) ] , (4) \min_{x \in\mathbb{X}} R_{P}[Q(x,\xi)], \tag*{(4)} x ∈ X min R P [ Q ( x , ξ )] , (4) ξ ∼ P \xi\sim P ξ ∼ P x ∈ X x \in\mathbb{X} x ∈ X Q ( x , ξ ) Q(x,\xi) Q ( x , ξ ) R P R_{P} R P risk functional summarizing the random cost by a scalar. Taking R P = E P R_{P} = \mathbb{E}_{P} R P = E P risk-neutral program . When P P P sample average approximation (SAA) replaces it by the empirical measure of scenarios drawn once and for all from P P P [36 ] .
Decisions are random variables constrained to use only the information available when they are taken. A filtration F 0 ⊆ F 1 ⊆ ⋯ \mathcal{F}_{0} \subseteq\mathcal{F}_{1} \subseteq\cdots F 0 ⊆ F 1 ⊆ ⋯ t t t F t \mathcal{F}_{t} F t here-and-now decision, fixed before the relevant uncertainty is observed, and a wait-and-see decision, taken after the realization and so free to depend on it. A two-stage recourse program fixes x 0 x_{0} x 0 ξ 1 \xi_{1} ξ 1 x 1 ∈ X 1 ( x 0 , ξ 1 ) x_{1} \in\mathbb{X}_{1}(x_{0},\xi_{1}) x 1 ∈ X 1 ( x 0 , ξ 1 ) § 2.3 casts model criticism in this form.
Model criticism as a two-stage program
We ask two questions in sequence. First, which structure within the model class describes the data best, when the noise law is taken exactly as the modeler wrote it down? Second, with that structure fixed, how do new data redistribute the mass that the fitted model assigns to its own support? We answer the two questions by the two stages of a stochastic program on a sample split ( ξ 0 , ξ 1 ) (\xi_0,\xi_1) ( ξ 0 , ξ 1 )
Table 1.
Written out, the program is
θ ~ ∈ arg min θ ∈ K W 1 ( P ^ ξ 0 , P θ ) , w ~ ∈ arg min w ∈ Δ L ( θ ~ ) − 1 W 1 ( P ^ ξ 1 , ∑ ℓ w ℓ δ z ℓ ( θ ~ ) ) , (5) \begin{aligned}
\tilde{\theta} &\in\underset{\theta\in K}{\arg\min}\; W_1\left(\widehat{P}_{\xi_0},P_\theta\right), \qquad
\tilde{\mathbf{w}} \in\underset{\mathbf{w}\in\Delta^{L(\tilde{\theta})-1}}{\arg\min}\; W_1\left(\widehat{P}_{\xi_1},\sum_{\ell}w_\ell\delta_{z_\ell(\tilde{\theta})}\right), \tag*{(5)}
\end{aligned} θ ~ ∈ θ ∈ K arg min W 1 ( P ξ 0 , P θ ) , w ~ ∈ w ∈ Δ L ( θ ~ ) − 1 arg min W 1 ( P ξ 1 , ℓ ∑ w ℓ δ z ℓ ( θ ~ ) ) , (5) where z 1 ( θ ~ ) , … , z L ( θ ~ ) ( θ ~ ) z_1(\tilde{\theta}),\ldots,z_{L(\tilde{\theta})}(\tilde{\theta}) z 1 ( θ ~ ) , … , z L ( θ ~ ) ( θ ~ ) G ( θ ~ , ⋅ ) G(\tilde{\theta},\cdot) G ( θ ~ , ⋅ ) § 3.5 ). The first stage optimizes over the model class and the second optimizes within the fitted model P θ ~ P_{\tilde{\theta}} P θ ~ ξ 1 \xi_1 ξ 1 w v o r ( θ ~ ) w^{\mathrm{vor}}(\tilde{\theta}) w vor ( θ ~ ) w ~ \tilde{\mathbf{w}} w ~ w v o r ( θ ~ ) w^{\mathrm{vor}}(\tilde{\theta}) w vor ( θ ~ ) § 2.2 , θ \theta θ ξ 1 \xi_1 ξ 1 F 0 \mathcal{F}_0 F 0 w \mathbf{w} w ξ 1 \xi_1 ξ 1
We note a difference from a classical recourse program. In the classical setting, the first-stage decision anticipates the recourse and minimizes the expected total cost.
Q 0 ( x 0 ) + E [ min x 1 Q 1 ( x 0 , x 1 ; ξ 1 ) ] Q_0(x_0)+\mathbb{E}\left[\min_{x_1}Q_1(x_0,x_1;\xi_1)\right] Q 0 ( x 0 ) + E [ x 1 min Q 1 ( x 0 , x 1 ; ξ 1 ) ] Here the first stage minimizes its own cost alone. The program is sequential , and the first stage is myopic by design. Were θ \theta θ θ ~ \tilde{\theta} θ ~ § 4 develops both stages, and Figure 1 summarizes the procedure.
Figure 1. The procedure at a glance. Top: the estimation block and the synthetic noise determine θ ~ \tilde{\theta} θ ~ G ( θ ~ , ⋅ ) G(\tilde{\theta},\cdot) G ( θ ~ , ⋅ ) w ~ ℓ \tilde{w}_{\ell} w ~ ℓ
The structural model, the split, synthetic noise, and the recourse mesh
Mechanism
Following Covington and Miller [10 ] , we posit that the data-generating mechanism admits a structural representation. The structural parameter θ \theta θ Π \Pi Π Θ \Theta Θ ρ \rho ρ
Π ( Θ ) = 1 , θ 0 ∈ Θ . (6) \Pi(\Theta)=1,\qquad\theta_0\in\Theta. \tag*{(6)} Π ( Θ ) = 1 , θ 0 ∈ Θ. (6) Throughout, ∥ θ − θ ′ ∥ \lVert\theta-\theta'\rVert ∥ θ − θ ′ ∥ ρ ( θ , θ ′ ) \rho(\theta,\theta') ρ ( θ , θ ′ ) Θ ⊆ R p \Theta\subseteq\mathbb{R}^{p} Θ ⊆ R p θ \theta θ ν \nu ν noise space X \mathcal{X} X ν ∈ P ( X ) \nu\in\mathcal{P}(\mathcal{X}) ν ∈ P ( X )
X 1 , … , X n ∼ i . i . d . ν , Y t = G ( θ 0 , X t ) ∈ S , t = 1 , … , n . (7) X_1,\ldots,X_n \mathrel{\overset{\mathrm{i.i.d.}}{\sim}} \nu,\qquad Y_t=G(\theta_0,X_t)\in S,\qquad t=1,\ldots,n. \tag*{(7)} X 1 , … , X n ∼ i.i.d. ν , Y t = G ( θ 0 , X t ) ∈ S , t = 1 , … , n . (7) The procedure uses ν \nu ν § 3.4 ), and compute a finite mesh of the noise space from simulated draws in the recourse stage (§ 3.5 ). The mechanism G : Θ × X → S G:\Theta\times\mathcal{X}\to S G : Θ × X → S
Stochastic processes as mechanisms
For a stochastic process, an observation Y t Y_t Y t 0 < t 1 < ⋯ < t J ≤ T 0<t_1<\cdots<t_J\leq T 0 < t 1 < ⋯ < t J ≤ T S ⊆ R d J S\subseteq\mathbb{R}^{dJ} S ⊆ R dJ [ 0 , T ] [0,T] [ 0 , T ] S S S Y 1 , … , Y n Y_1,\ldots,Y_n Y 1 , … , Y n
Continuous-time Markov chains. Let Z Z Z E ⊆ Z d E\subseteq\mathbb{Z}^d E ⊆ Z d ζ r ∈ Z d \zeta_r\in\mathbb{Z}^d ζ r ∈ Z d λ r ( θ , Z ) ≥ 0 \lambda_r(\theta,Z)\geq0 λ r ( θ , Z ) ≥ 0 r = 1 , … , R r=1,\ldots,R r = 1 , … , R Z Z Z
Z ( t ) = z 0 + ∑ r = 1 R ζ r ∫ 0 t ∫ 0 ∞ 1 { u ≤ λ r ( θ , Z ( s − ) ) } N r ( d s , d u ) , t ∈ [ 0 , T ] , (8) Z(t)=z_0+\sum_{r=1}^{R}\zeta_r\int_0^t\int_0^\infty\mathbf{1}\{u\leq\lambda_r(\theta,Z(s-))\}N_r(ds,du),\qquad t\in[0,T], \tag*{(8)} Z ( t ) = z 0 + r = 1 ∑ R ζ r ∫ 0 t ∫ 0 ∞ 1 { u ≤ λ r ( θ , Z ( s − ))} N r ( d s , d u ) , t ∈ [ 0 , T ] , (8) where N 1 , … , N R N_1,\ldots,N_R N 1 , … , N R [ 0 , T ] × [ 0 , ∞ ) [0,T]\times[0,\infty) [ 0 , T ] × [ 0 , ∞ ) d s d u \mathrm{d}s\,\mathrm{d}u d s d u [24 , 16 Ch. II] . Each atom ( s , u ) (s,u) ( s , u ) N r N_r N r r r r s s s u ≤ λ r ( θ , Z ( s − ) ) u\leq\lambda_r(\theta,Z(s-)) u ≤ λ r ( θ , Z ( s − )) x = ( N 1 , … , N R ) x=(N_1,\ldots,N_R) x = ( N 1 , … , N R ) d = 1 d=1 d = 1 ζ 1 = + 1 \zeta_1=+1 ζ 1 = + 1 ζ 2 = − 1 \zeta_2=-1 ζ 2 = − 1 [25 ] and stochastic chemical reaction networks are the general case [19 , 1 ] . Paths can be recorded in L 1 ( [ 0 , T ] ; R d ) L^1([0,T];\mathbb{R}^d) L 1 ([ 0 , T ] ; R d ) [6 Sec. 12] .
Diffusions. Let Z Z Z d Z ( t ) = b ( θ , Z ( t ) ) d t + σ ( θ , Z ( t ) ) d W ( t ) \mathrm{d}Z(t)=b(\theta,Z(t))\mathrm{d}t+\sigma(\theta,Z(t))\mathrm{d}W(t) d Z ( t ) = b ( θ , Z ( t )) d t + σ ( θ , Z ( t )) d W ( t ) Z ( 0 ) = z 0 ∈ R d Z(0)=z_0\in\mathbb{R}^d Z ( 0 ) = z 0 ∈ R d W W W m m m X = C 0 ( [ 0 , T ] ; R m ) \mathcal{X}=C_0([0,T];\mathbb{R}^m) X = C 0 ([ 0 , T ] ; R m ) ν \nu ν R d J \mathbb{R}^{dJ} R dJ C ( [ 0 , T ] ; R d ) C([0,T];\mathbb{R}^d) C ([ 0 , T ] ; R d ) L 2 ( [ 0 , T ] ; R d ) L^2([0,T];\mathbb{R}^d) L 2 ([ 0 , T ] ; R d )
The split
Partition the n n n
ξ 0 = ( y 1 , … , y n 0 ) , ξ 1 = ( y n 0 + 1 , … , y n ) , n 0 + n 1 = n , (9) \xi_0=(y_1,\ldots,y_{n_0}),\qquad\xi_1=(y_{n_0+1},\ldots,y_n),\qquad n_0+n_1=n, \tag*{(9)} ξ 0 = ( y 1 , … , y n 0 ) , ξ 1 = ( y n 0 + 1 , … , y n ) , n 0 + n 1 = n , (9) with empirical measures P ^ ξ 0 = 1 n 0 ∑ t ≤ n 0 δ y t \widehat{P}_{\xi_0}=\frac{1}{n_0}\sum_{t\leq n_0}\delta_{y_t} P ξ 0 = n 0 1 ∑ t ≤ n 0 δ y t P ^ ξ 1 = 1 n 1 ∑ t > n 0 δ y t \widehat{P}_{\xi_1}=\frac{1}{n_1}\sum_{t>n_0}\delta_{y_t} P ξ 1 = n 1 1 ∑ t > n 0 δ y t n 0 = n 0 ( n ) n_0=n_0(n) n 0 = n 0 ( n ) n 1 = n 1 ( n ) n_1=n_1(n) n 1 = n 1 ( n ) n n n
n 0 ∧ n 1 ⟶ ∞ , n 1 log n ⟶ ∞ , (10) n_0\wedge n_1\longrightarrow\infty,\qquad\frac{n_1}{\log n}\longrightarrow\infty, \tag*{(10)} n 0 ∧ n 1 ⟶ ∞ , log n n 1 ⟶ ∞ , (10) the second condition holding for instance when n 1 n_1 n 1 n n n
Synthetic noise for the estimation stage
The model law P θ P_\theta P θ m = m ( n 0 ) ∈ N m=m(n_0)\in\mathbb{N} m = m ( n 0 ) ∈ N
x ~ 1 , … , x ~ m ∼ i . i . d . ν , P ^ θ , m : = 1 m ∑ i = 1 m δ G ( θ , x ~ i ) . (11) \widetilde{x}_1,\ldots,\widetilde{x}_m\mathrel{\overset{\mathrm{i.i.d.}}{\sim}}\nu,\qquad\widehat{P}_{\theta,m}:=\frac{1}{m}\sum_{i=1}^{m}\delta_{G(\theta,\widetilde{x}_i)}. \tag*{(11)} x 1 , … , x m ∼ i.i.d. ν , P θ , m := m 1 i = 1 ∑ m δ G ( θ , x i ) . (11) The same noise draws are used for every θ \theta θ P ^ θ , m \widehat{P}_{\theta,m} P θ , m sample average approximation (SAA) of P θ P_\theta P θ [36 Ch. 5] , and it plays the role of the scenario generation step of Pflug and Pichler [33 ] . It is also the single-replicate Monte Carlo approximation of the minimum expected Wasserstein estimator used in the computations of Bernton et al. [5 ] , where the noise is likewise drawn once and θ \theta θ
Together with the split, the synthetic sample forms the filtration
F 0 = σ ( ξ 0 , x ~ 1 : m ) ⊂ F 1 = σ ( ξ 0 , x ~ 1 : m , ξ 1 ) . (12) \mathcal{F}_0=\sigma(\xi_0,\widetilde{x}_{1:m})\subset\mathcal{F}_1=\sigma(\xi_0,\widetilde{x}_{1:m},\xi_1). \tag*{(12)} F 0 = σ ( ξ 0 , x 1 : m ) ⊂ F 1 = σ ( ξ 0 , x 1 : m , ξ 1 ) . (12) Because x ~ 1 : m \widetilde{x}_{1:m} x 1 : m F 0 \mathcal{F}_0 F 0 ξ 1 \xi_1 ξ 1
The recourse mesh: the N 1 N_1 N 1
The recourse stage needs a fixed, finite set of model atoms at which a departure from the model can be reported. We obtain them by pushing finitely many noise values through the fitted mechanism, and we choose these noise values to represent ν \nu ν N 1 N_1 N 1 [20 ] .
Optimal quantization. Fix a quantization (pseudo)metric d Q d_Q d Q X \mathcal{X} X X × X \mathcal{X} \times\mathcal{X} X × X r ≥ 1 r \ge1 r ≥ 1 Γ = { x ˉ 1 , … , x ˉ N 1 } ⊂ X \Gamma= \{\bar{x}_1,\ldots,\bar{x}_{N_1}\} \subset\mathcal{X} Γ = { x ˉ 1 , … , x ˉ N 1 } ⊂ X
C j : = { x ∈ X : j = arg min j ′ d Q ( x , x ˉ j ′ ) } , ties broken by lowest index, (13) C_j := \left\{x \in\mathcal{X}: j = \underset{j'}{\arg\min}\ d_Q(x,\bar{x}_{j'})\right\}, \qquad\text{ties broken by lowest index,} \tag*{(13)} C j := { x ∈ X : j = j ′ arg min d Q ( x , x ˉ j ′ ) } , ties broken by lowest index, (13) be its Voronoi cells, q j : = ν ( C j ) q_j := \nu(C_j) q j := ν ( C j )
e r ( Γ ; ν ) : = ( ∫ X min j ≤ N 1 d Q ( x , x ˉ j ) r ν ( d x ) ) 1 / r (14) e_r(\Gamma;\nu) := \left(\int_{\mathcal{X}} \min_{j \le N_1} d_Q(x,\bar{x}_j)^r \,\nu(dx)\right)^{1/r} \tag*{(14)} e r ( Γ ; ν ) := ( ∫ X j ≤ N 1 min d Q ( x , x ˉ j ) r ν ( d x ) ) 1/ r (14) its quantization error. Optimal quantizers exist for X = R k \mathcal{X} = \mathbb{R}^k X = R k d Q d_Q d Q
∫ d Q ( x 0 , x ) r ν ( d x ) < ∞ \int d_Q(x_0,x)^r \,\nu(dx) < \infty ∫ d Q ( x 0 , x ) r ν ( d x ) < ∞ [20 , 28 ] . In a general Polish space an optimal quantizer need not exist.
Construction from simulated noise. Because ν \nu ν
x ( 1 ) , … , x ( M ) ∼ i . i . d . ν , (15) x^{(1)},\ldots,x^{(M)} \overset{\mathrm{i.i.d.}}{\sim} \nu, \tag*{(15)} x ( 1 ) , … , x ( M ) ∼ i.i.d. ν , (15) which can also be subsampled from the m m m M < m M < m M < m N 1 N_1 N 1
Γ ^ ∈ arg min Γ ⊆ { x ( 1 ) , … , x ( M ) } ∣ Γ ∣ = N 1 { 1 M ∑ i ≤ M min x ˉ ∈ Γ d Q ( x ( i ) , x ˉ ) r } , (16) \widehat{\Gamma} \in\underset{\substack{\Gamma\subseteq\{x^{(1)},\ldots,x^{(M)}\}\\|\Gamma|=N_1}}{\arg\min} \left\{\frac{1}{M}\sum_{i \le M}\min_{\bar{x}\in\Gamma} d_Q(x^{(i)},\bar{x})^r\right\}, \tag*{(16)} Γ ∈ Γ ⊆ { x ( 1 ) , … , x ( M ) } ∣Γ∣ = N 1 arg min { M 1 i ≤ M ∑ x ˉ ∈ Γ min d Q ( x ( i ) , x ˉ ) r } , (16) computed approximately by Algorithm 2 of § 7.3 . Every mesh point is a genuine draw from ν \nu ν G ( θ , x ˉ j ) G(\theta,\bar{x}_j) G ( θ , x ˉ j )
Raw and merged atoms. The raw atoms are G ( θ , x ˉ j ) G(\theta,\bar{x}_j) G ( θ , x ˉ j ) j = 1 , … , N 1 j=1,\ldots,N_1 j = 1 , … , N 1 G G G z 1 ( θ ) , … , z L ( θ ) ( θ ) z_1(\theta),\ldots,z_{L(\theta)}(\theta) z 1 ( θ ) , … , z L ( θ ) ( θ ) distinct values among { G ( θ , x ˉ j ) } j ≤ N 1 \{G(\theta,\bar{x}_j)\}_{j\le N_1} { G ( θ , x ˉ j ) } j ≤ N 1
I ℓ ( θ ) : = { j ≤ N 1 : G ( θ , x ˉ j ) = z ℓ ( θ ) } , ℓ = 1 , … , L ( θ ) , (17) I_\ell(\theta) := \{j \le N_1 : G(\theta,\bar{x}_j) = z_\ell(\theta)\}, \qquad\ell= 1,\ldots,L(\theta), \tag*{(17)} I ℓ ( θ ) := { j ≤ N 1 : G ( θ , x ˉ j ) = z ℓ ( θ )} , ℓ = 1 , … , L ( θ ) , (17) be the fibers , labelled so that min I 1 ( θ ) < min I 2 ( θ ) < ⋯ \min I_1(\theta) < \min I_2(\theta) < \cdots min I 1 ( θ ) < min I 2 ( θ ) < ⋯ molecular model measure on the merged atoms is
P θ , w : = ∑ ℓ = 1 L ( θ ) w ℓ δ z ℓ ( θ ) , w ∈ Δ L ( θ ) − 1 , (18) P_{\theta,\mathbf{w}} := \sum_{\ell=1}^{L(\theta)} w_\ell\delta_{z_\ell(\theta)}, \qquad\mathbf{w} \in\Delta^{L(\theta)-1}, \tag*{(18)} P θ , w := ℓ = 1 ∑ L ( θ ) w ℓ δ z ℓ ( θ ) , w ∈ Δ L ( θ ) − 1 , (18) where Δ L − 1 = { w ∈ [ 0 , 1 ] L : ∑ ℓ w ℓ = 1 } \Delta^{L-1} = \{\mathbf{w} \in[0,1]^L : \sum_\ell w_\ell= 1\} Δ L − 1 = { w ∈ [ 0 , 1 ] L : ∑ ℓ w ℓ = 1 } w \mathbf{w} w model's weights , living on observation-space atoms. When G G G x x x L ( θ ) = N 1 L(\theta) = N_1 L ( θ ) = N 1 P θ P_\theta P θ L ( θ ) L(\theta) L ( θ )
Model criticism procedure
The sample split (9) splits the decision as well. The structural parameter is chosen first with the noise law held at its assumed value, and the weights are chosen second with the structure held at its fitted value. The here-and-now decision is therefore the structural parameter θ \theta θ w \mathbf{w} w
X 0 ( ξ 0 ) : = K ⊆ Θ , X 1 ( θ ~ , ξ 1 ) : = Δ L ( θ ~ ) − 1 , (19) X_0(\xi_0) := K \subseteq\Theta,\qquad X_1(\widetilde{\theta},\xi_1) := \Delta^{L(\widetilde{\theta})-1}, \tag*{(19)} X 0 ( ξ 0 ) := K ⊆ Θ , X 1 ( θ , ξ 1 ) := Δ L ( θ ) − 1 , (19) with K K K Assumption 3 . The stage costs are Wasserstein discrepancies against the two blocks,
Q 0 ( x 0 ; ξ 0 ) : = W 1 ( P ^ ξ 0 , P ^ θ , m ) , Q 1 ( x 0 , x 1 ; ξ 1 ) : = W 1 ( P ^ ξ 1 , P θ ~ , w ) , (20) Q_0(x_0;\xi_0) := W_1(\widehat{P}_{\xi_0},\widehat{P}_{\theta,m}),\qquad Q_1(x_0,x_1;\xi_1) := W_1(\widehat{P}_{\xi_1},P_{\widetilde{\theta},\mathbf{w}}), \tag*{(20)} Q 0 ( x 0 ; ξ 0 ) := W 1 ( P ξ 0 , P θ , m ) , Q 1 ( x 0 , x 1 ; ξ 1 ) := W 1 ( P ξ 1 , P θ , w ) , (20) the first formed against the synthetic sample (11) and the second on the merged atoms (18) . Ideally, we would like to compute W 1 ( P ^ ξ 0 , P θ ) W_1(\widehat{P}_{\xi_0},P_\theta) W 1 ( P ξ 0 , P θ ) P θ P_\theta P θ P θ P_\theta P θ P ^ θ , m \widehat{P}_{\theta,m} P θ , m
We explain the design of the feasible sets (19) . First, the estimation stage may not touch the weights. Simulating from ν \nu ν assumed instead of estimated. Second, the recourse stage may not revise the structure. Were θ \theta θ
Estimation stage
Let θ 0 \theta_0 θ 0 Π \Pi Π y 1 : n y_{1:n} y 1 : n (7) . We solve the estimation-stage term alone,
θ ~ n 0 ∈ ε n 0 -arg min θ ∈ K W 1 ( P ^ ξ 0 , P ^ θ , m ) , m = m ( n 0 ) , (21) \widetilde{\theta}_{n_0} \in\varepsilon_{n_0}\text{-arg min}_{\theta\in K} W_1(\widehat{P}_{\xi_0},\widehat{P}_{\theta,m}),\qquad m=m(n_0), \tag*{(21)} θ n 0 ∈ ε n 0 -arg min θ ∈ K W 1 ( P ξ 0 , P θ , m ) , m = m ( n 0 ) , (21) where ε -arg min θ f : = { θ : f ( θ ) ≤ ε + inf f } \varepsilon\text{-arg min}_{\theta} f := \{\theta:f(\theta)\leq\varepsilon+\inf f\} ε -arg min θ f := { θ : f ( θ ) ≤ ε + inf f } ε n 0 ↓ 0 \varepsilon_{n_0}\downarrow0 ε n 0 ↓ 0 θ ~ n 0 \widetilde{\theta}_{n_0} θ n 0 SAA minimum Wasserstein estimator .
When the observations are vector-valued, S = R s S=\mathbb{R}^{s} S = R s s > 1 s>1 s > 1 (21) is a transport linear program between n 0 n_0 n 0 m m m § 7.1 ), and the search of Algorithm 1 needs hundreds of evaluations. For efficiency we then replace W 1 W_1 W 1 W 1 W_1 W 1 [30 ] ). This changes the estimator, whose consistency is then that of minimum sliced-Wasserstein estimators ([30 ] ) rather than Theorem 1.
For large n 0 n_0 n 0 θ 0 \theta_0 θ 0 Π \Pi Π θ \theta θ Π \Pi Π α = 0.05 \alpha=0.05 α = 0.05
H 0 : θ ∼ Π against H 1 = ¬ H 0 . H_0:\ \theta\sim\Pi\ \text{against}\ H_1=\neg H_0. H 0 : θ ∼ Π against H 1 = ¬ H 0 . Since we observe only one instantiation of the structural parameter θ 0 \theta_0 θ 0 Y 1 , … , Y n Y_1,\ldots,Y_n Y 1 , … , Y n T ( θ ) T(\theta) T ( θ ) T ( θ ) = ∥ θ ∥ T(\theta)=\lVert\theta\rVert T ( θ ) = ∥ θ ∥ F ( t ) : = Π ( T ( θ ) ≤ t ) F(t):=\Pi(T(\theta)\leq t) F ( t ) := Π ( T ( θ ) ≤ t ) F ( t − ) : = Π ( T ( θ ) < t ) F(t^-):=\Pi(T(\theta)<t) F ( t − ) := Π ( T ( θ ) < t )
U = F ( t − ) + V Π ( T ( θ ) = t ) , t = T ( θ ~ n 0 ) , V ∼ Unif ( 0 , 1 ) independent of the data , U=F(t^-)+V\Pi(T(\theta)=t),\qquad t=T(\widetilde{\theta}_{n_0}),\qquad V\sim\operatorname{Unif}(0,1)\ \text{independent of the data}, U = F ( t − ) + V Π ( T ( θ ) = t ) , t = T ( θ n 0 ) , V ∼ Unif ( 0 , 1 ) independent of the data , and the p p p
p = 2 min { U , 1 − U } . p = 2\min\{U, 1-U\}. p = 2 min { U , 1 − U } . The central estimation-stage result is the strong consistency of θ ~ n 0 \widetilde{\theta}_{n_0} θ n 0 § 12 ). Under Assumptions 1–3, for any measurable ε n 0 \varepsilon_{n_0} ε n 0 θ ~ n 0 \widetilde{\theta}_{n_0} θ n 0 (21) with ε n 0 ↓ 0 \varepsilon_{n_0} \downarrow0 ε n 0 ↓ 0 m = m ( n 0 ) → ∞ m=m(n_0)\to\infty m = m ( n 0 ) → ∞
lim n 0 → ∞ ∥ θ ~ n 0 − θ 0 ∥ = 0 almost surely . \lim_{n_0\to\infty}\left\|\widetilde{\theta}_{n_0}-\theta_0\right\|=0 \quad\text{almost surely}. n 0 → ∞ lim θ n 0 − θ 0 = 0 almost surely . Hence, θ ~ n 0 \widetilde{\theta}_{n_0} θ n 0 θ 0 \theta_0 θ 0 n 0 n_0 n 0 θ ~ n 0 \widetilde{\theta}_{n_0} θ n 0 Π \Pi Π G G G G G G § 6.2 ).
To protect the influence of noise law misspecification on the structural parameter. We show the estimator θ ~ n 0 \widetilde{\theta}_{n_0} θ n 0 T T T ( P 1 ( S ) , W 1 ) (\mathcal{P}_1(S),W_1) ( P 1 ( S ) , W 1 ) T ( Q ) ∈ arg min θ ∈ K W 1 ( Q , P θ ) T(Q)\in\arg\min_{\theta\in K}W_1(Q,P_\theta) T ( Q ) ∈ arg min θ ∈ K W 1 ( Q , P θ ) θ ~ n 0 \widetilde{\theta}_{n_0} θ n 0 T ( P ^ ξ 0 ) T(\widehat{P}_{\xi_0}) T ( P ξ 0 ) T T T P θ 0 P_{\theta_0} P θ 0 [21 ] ). In Proposition 2 we show that if Q k → P θ 0 Q_k\to P_{\theta_0} Q k → P θ 0 W 1 W_1 W 1 θ ↦ W 1 ( Q k , P θ ) \theta\mapsto W_1(Q_k,P_\theta) θ ↦ W 1 ( Q k , P θ ) K K K θ 0 \theta_0 θ 0 [23 ] ), namely continuity of T T T ( P 1 ( S ) , W 1 ) (\mathcal{P}_1(S),W_1) ( P 1 ( S ) , W 1 ) W 1 W_1 W 1 [5 ] .
Recourse stage
Fix θ ~ : = θ ~ n 0 \widetilde{\theta}:=\widetilde{\theta}_{n_0} θ := θ n 0 z ℓ : = z ℓ ( θ ~ ) z_\ell:=z_\ell(\widetilde{\theta}) z ℓ := z ℓ ( θ ) ℓ = 1 , … , L : = L ( θ ~ ) \ell=1,\ldots,L:=L(\widetilde{\theta}) ℓ = 1 , … , L := L ( θ ) § 3.5 . The recourse-stage problem is
Q ( θ ~ ) = min w ∈ Δ L − 1 W 1 ( P ^ ξ 1 , P θ ~ , w ) . (22) Q(\widetilde{\theta})=\min_{w\in\Delta^{L-1}}W_1(\widehat{P}_{\xi_1},P_{\widetilde{\theta},w}). \tag*{(22)} Q ( θ ) = w ∈ Δ L − 1 min W 1 ( P ξ 1 , P θ , w ) . (22) The recourse problem is solved by Voronoi recourse (Pflug and Pichler, [33 ] ). The minimizer and the value of Equation (22) are
w ~ ℓ = 1 n 1 # { y ∈ ξ 1 : ℓ = arg min ℓ ′ d ( y , z ℓ ′ ) } , Q ( θ ~ ) = 1 n 1 ∑ y ∈ ξ 1 min ℓ d ( y , z ℓ ) . (23) \widetilde{w}_\ell=\frac{1}{n_1}\#\left\{y\in\xi_1:\ell=\arg\min_{\ell'}d(y,z_{\ell'})\right\},\qquad
Q(\widetilde{\theta})=\frac{1}{n_1}\sum_{y\in\xi_1}\min_\ell d(y,z_\ell). \tag*{(23)} w ℓ = n 1 1 # { y ∈ ξ 1 : ℓ = arg ℓ ′ min d ( y , z ℓ ′ ) } , Q ( θ ) = n 1 1 y ∈ ξ 1 ∑ ℓ min d ( y , z ℓ ) . (23) ties broken by lowest label. Define the observation-space Voronoi cells of the merged atoms and the model’s Voronoi masses
w ℓ v o r ( θ ) : = P θ ( V ℓ ( θ ) ) , V ℓ ( θ ) : = { y ∈ S : ℓ = arg min ℓ ′ d ( y , z ℓ ′ ( θ ) ) } . (24) w_\ell^{\mathrm{vor}}(\theta):=P_\theta(V_\ell(\theta)),\qquad
V_\ell(\theta):=\left\{y\in S:\ell=\arg\min_{\ell'}d(y,z_{\ell'}(\theta))\right\}. \tag*{(24)} w ℓ vor ( θ ) := P θ ( V ℓ ( θ )) , V ℓ ( θ ) := { y ∈ S : ℓ = arg ℓ ′ min d ( y , z ℓ ′ ( θ )) } . (24) ties broken by lowest label. If P θ P_\theta P θ P θ P_\theta P θ P θ P_\theta P θ
w ℓ v o r ( θ ) = P θ ( { z ℓ ( θ ) } ) = ν { x ∈ X : G ( θ , x ) = z ℓ ( θ ) } , (25) w_\ell^{\mathrm{vor}}(\theta)=P_\theta(\{z_\ell(\theta)\})=\nu\{x\in\mathcal{X}:G(\theta,x)=z_\ell(\theta)\}, \tag*{(25)} w ℓ vor ( θ ) = P θ ({ z ℓ ( θ )}) = ν { x ∈ X : G ( θ , x ) = z ℓ ( θ )} , (25) the probability mass function of the model at the atom. In that case (23) is the relative frequency of each support value in the held-out block, and held-out values outside the model’s support are counted into the nearest atom, typically an extreme one.
At the recourse stage we localize the residual misspecification in the noise law. In Theorem 2, Appendix § 12 , we show that almost surely, as n 0 , n 1 → ∞ n_0,n_1\to\infty n 0 , n 1 → ∞ L ( θ 0 ) L(\theta_0) L ( θ 0 ) ℓ \ell ℓ
w ~ ℓ → w ℓ v o r ( θ 0 ) , w ℓ v o r ( θ ~ n 0 ) → w ℓ v o r ( θ 0 ) , ∑ ℓ w ~ ℓ δ z ℓ ( θ ~ n 0 ) → W 1 ∑ ℓ w ℓ v o r ( θ 0 ) δ z ℓ ( θ 0 ) . \widetilde{w}_{\ell}\to w_{\ell}^{\mathrm{vor}}(\theta_0),\qquad w_{\ell}^{\mathrm{vor}}(\widetilde{\theta}_{n_0})\to w_{\ell}^{\mathrm{vor}}(\theta_0),\qquad\sum_{\ell}\widetilde{w}_{\ell}\delta_{z_{\ell}(\widetilde{\theta}_{n_0})}\xrightarrow{\mathrm{W}_1}\sum_{\ell}w_{\ell}^{\mathrm{vor}}(\theta_0)\delta_{z_{\ell}(\theta_0)}. w ℓ → w ℓ vor ( θ 0 ) , w ℓ vor ( θ n 0 ) → w ℓ vor ( θ 0 ) , ℓ ∑ w ℓ δ z ℓ ( θ n 0 ) W 1 ℓ ∑ w ℓ vor ( θ 0 ) δ z ℓ ( θ 0 ) . Condition on F 0 \mathcal{F}_0 F 0 θ ~ n 0 \widetilde{\theta}_{n_0} θ n 0 V ℓ ( θ ~ n 0 ) V_{\ell}(\widetilde{\theta}_{n_0}) V ℓ ( θ n 0 )
p ℓ : = P ( Y ∈ V ℓ ( θ ~ n 0 ) ∣ F 0 ) , ℓ = 1 , … , L , (26) p_{\ell}:=\mathbb{P}\left(Y\in V_{\ell}(\widetilde{\theta}_{n_0})\mid\mathcal{F}_0\right),\qquad\ell=1,\ldots,L, \tag*{(26)} p ℓ := P ( Y ∈ V ℓ ( θ n 0 ) ∣ F 0 ) , ℓ = 1 , … , L , (26) for the resulting cell probabilities. Since the cells partition S S S ∑ ℓ p ℓ = 1 \sum_{\ell}p_{\ell}=1 ∑ ℓ p ℓ = 1 (23) counts each held-out observation into exactly one of them, so by construction
( n 1 w ~ 1 , … , n 1 w ~ L ) ∣ F 0 ∼ Multinomial ( n 1 ; p 1 , … , p L ) . (27) (n_1\widetilde{w}_1,\ldots,n_1\widetilde{w}_L)\mid\mathcal{F}_0\sim\operatorname{Multinomial}(n_1;p_1,\ldots,p_L). \tag*{(27)} ( n 1 w 1 , … , n 1 w L ) ∣ F 0 ∼ Multinomial ( n 1 ; p 1 , … , p L ) . (27) The counts are not independent across cells as they sum to n 1 n_1 n 1
sd ( w ~ ℓ ∣ F 0 ) = p ℓ ( 1 − p ℓ ) n 1 , corr ( w ~ ℓ , w ~ ℓ ′ ∣ F 0 ) = − p ℓ p ℓ ′ ( 1 − p ℓ ) ( 1 − p ℓ ′ ) . (28) \operatorname{sd}(\widetilde{w}_{\ell}\mid\mathcal{F}_0)=\sqrt{\frac{p_{\ell}(1-p_{\ell})}{n_1}},\qquad\operatorname{corr}(\widetilde{w}_{\ell},\widetilde{w}_{\ell'}\mid\mathcal{F}_0)=-\sqrt{\frac{p_{\ell}p_{\ell'}}{(1-p_{\ell})(1-p_{\ell'})}}. \tag*{(28)} sd ( w ℓ ∣ F 0 ) = n 1 p ℓ ( 1 − p ℓ ) , corr ( w ℓ , w ℓ ′ ∣ F 0 ) = − ( 1 − p ℓ ) ( 1 − p ℓ ′ ) p ℓ p ℓ ′ . (28) Replacing p ℓ p_{\ell} p ℓ w ℓ v o r ( θ ~ n 0 ) w_{\ell}^{\mathrm{vor}}(\widetilde{\theta}_{n_0}) w ℓ vor ( θ n 0 ) 1 / N 1 1/N_1 1/ N 1 O ( N 1 − 1 ) O(N_1^{-1}) O ( N 1 − 1 ) (25) and need not be small, so the correlations should be kept in mind when a few atoms carry most of the mass. The signal-to-noise ratio of cell ℓ \ell ℓ
w ℓ v o r ( θ ~ n 0 ) sd ( w ~ ℓ ∣ F 0 ) ≈ n 1 w ℓ v o r ( θ ~ n 0 ) 1 − w ℓ v o r ( θ ~ n 0 ) ≥ e ℓ , e ℓ : = n 1 w ℓ v o r ( θ ~ n 0 ) , \frac{w_{\ell}^{\mathrm{vor}}(\widetilde{\theta}_{n_0})}{\operatorname{sd}(\widetilde{w}_{\ell}\mid\mathcal{F}_0)}\approx\sqrt{\frac{n_1w_{\ell}^{\mathrm{vor}}(\widetilde{\theta}_{n_0})}{1-w_{\ell}^{\mathrm{vor}}(\widetilde{\theta}_{n_0})}}\ge\sqrt{e_{\ell}},\qquad e_{\ell}:=n_1w_{\ell}^{\mathrm{vor}}(\widetilde{\theta}_{n_0}), sd ( w ℓ ∣ F 0 ) w ℓ vor ( θ n 0 ) ≈ 1 − w ℓ vor ( θ n 0 ) n 1 w ℓ vor ( θ n 0 ) ≥ e ℓ , e ℓ := n 1 w ℓ vor ( θ n 0 ) , the square root of the expected count. In the continuous case e ℓ e_{\ell} e ℓ n 1 / N 1 n_1/N_1 n 1 / N 1 N 1 N_1 N 1 e ℓ e_{\ell} e ℓ [33 ] for optimal quantization. This also guarantees the signal-to-noise ratio is at least 3, which satisfies the limit of detection standard in Borman and Elder [7 ] .
The natural test compares the observed counts in Equation (27) with the counts the fitted model predicts. Conditionally on F 0 \mathcal{F}_0 F 0
H 0 : p ℓ = w ℓ v o r ( θ ~ n 0 ) for every ℓ = 1 , … , L , H 1 : p ℓ ≠ w ℓ v o r ( θ ~ n 0 ) for some ℓ . (29) H_0:p_{\ell}=w_{\ell}^{\mathrm{vor}}(\widetilde{\theta}_{n_0})\ \text{for every }\ell=1,\ldots,L,\qquad H_1:p_{\ell}\ne w_{\ell}^{\mathrm{vor}}(\widetilde{\theta}_{n_0})\ \text{for some }\ell. \tag*{(29)} H 0 : p ℓ = w ℓ vor ( θ n 0 ) for every ℓ = 1 , … , L , H 1 : p ℓ = w ℓ vor ( θ n 0 ) for some ℓ . (29) H 0 H_0 H 0
T n 1 : = ∑ ℓ = 1 L ( n 1 w ~ ℓ − n 1 w ℓ v o r ( θ ~ n 0 ) ) 2 n 1 w ℓ v o r ( θ ~ n 0 ) , T n 1 ∣ F 0 ⟹ χ L − 1 2 (30) T_{n_1}:=\sum_{\ell=1}^{L}\frac{(n_1\widetilde{w}_{\ell}-n_1w_{\ell}^{\mathrm{vor}}(\widetilde{\theta}_{n_0}))^2}{n_1w_{\ell}^{\mathrm{vor}}(\widetilde{\theta}_{n_0})},\qquad T_{n_1}\mid\mathcal{F}_0\Longrightarrow\chi^2_{L-1} \tag*{(30)} T n 1 := ℓ = 1 ∑ L n 1 w ℓ vor ( θ n 0 ) ( n 1 w ℓ − n 1 w ℓ vor ( θ n 0 ) ) 2 , T n 1 ∣ F 0 ⟹ χ L − 1 2 (30) under H 0 H_0 H 0 n 1 → ∞ n_1 \to\infty n 1 → ∞ T n 1 T_{n_1} T n 1 ( 1 − α ) (1-\alpha) ( 1 − α ) χ L − 1 2 \chi^2_{L-1} χ L − 1 2 calibrated by the split as θ ~ n 0 \widetilde{\theta}_{n_0} θ n 0 F 0 \mathcal{F}_0 F 0 ξ 1 \xi_1 ξ 1 F 0 \mathcal{F}_0 F 0 [9 ] , adjacent cells are pooled until every expected count e ℓ e_\ell e ℓ
If we reject the null hypothesis, the optimized recourse weight profile w ~ \widetilde{w} w w ~ \widetilde{w} w w vor ( θ ~ n 0 ) w^{\operatorname{vor}}(\widetilde{\theta}_{n_0}) w vor ( θ n 0 ) (28) widened from one standard deviation to 1 − α 1-\alpha 1 − α (28) are negligible, we may treat the bars as bona fide 100 ( 1 − α ) % 100(1-\alpha)\% 100 ( 1 − α ) % w ~ \widetilde{w} w w vor ( θ ~ n 0 ) w^{\operatorname{vor}}(\widetilde{\theta}_{n_0}) w vor ( θ n 0 )
Assumptions
In this section we collect assumptions that are sufficient for propositions, lemmas and theorems in the Appendix.
Assumption 1 (L 1 L^1 L 1
(i) (envelope ) for some (hence every) y 0 ∈ S y_0 \in S y 0 ∈ S K ′ ⊆ Θ K' \subseteq\Theta K ′ ⊆ Θ
∫ X sup θ ∈ K ′ d ( y 0 , G ( θ , x ) ) ν ( d x ) < ∞ ; \int_{\mathcal{X}} \sup_{\theta\in K'} d(y_0,G(\theta,x)) \,\nu(\mathrm{d}x) < \infty; ∫ X θ ∈ K ′ sup d ( y 0 , G ( θ , x )) ν ( d x ) < ∞ ; (ii) (continuity in L 1 ( ν ) L^1(\nu) L 1 ( ν ) ) for every θ ∈ Θ \theta\in\Theta θ ∈ Θ
ω θ ( δ ) : = ∫ X sup θ ′ ∈ Θ : ∥ θ ′ − θ ∥ < δ d ( G ( θ ′ , x ) , G ( θ , x ) ) ν ( d x ) ⟶ 0 ( δ ↓ 0 ) . \omega_\theta(\delta) := \int_{\mathcal{X}} \sup_{\substack{\theta' \in\Theta:\\ \lVert\theta'-\theta\rVert<\delta}} d(G(\theta',x),G(\theta,x)) \,\nu(\mathrm{d}x) \longrightarrow0 \quad(\delta\downarrow0). ω θ ( δ ) := ∫ X θ ′ ∈ Θ : ∥ θ ′ − θ ∥ < δ sup d ( G ( θ ′ , x ) , G ( θ , x )) ν ( d x ) ⟶ 0 ( δ ↓ 0 ) . Assumption 1(i) implies E d ( y 0 , Y 1 ) < ∞ \mathbb{E}d(y_0,Y_1)<\infty E d ( y 0 , Y 1 ) < ∞ P θ ∈ P 1 ( S ) P_\theta\in\mathcal{P}_1(S) P θ ∈ P 1 ( S ) K ′ K' K ′ θ ↦ P θ \theta\mapsto P_\theta θ ↦ P θ W 1 W_1 W 1 Lemma 1 ), which is equivalent to Assumption 2.2 of Bernton et al. [5 ] .
Assumption 2 (Identifiability). θ ≠ θ ′ ⇒ W 1 ( P θ , P θ ′ ) > 0 \theta\ne\theta' \Rightarrow W_1(P_\theta,P_{\theta'})>0 θ = θ ′ ⇒ W 1 ( P θ , P θ ′ ) > 0
Assumption 3 (Compact search set). The estimation stage (21) is solved over a compact set K ⊆ Θ K \subseteq\Theta K ⊆ Θ θ 0 ∈ K \theta_0 \in K θ 0 ∈ K
Assumption 3 implies Assumption 2.3 of Bernton et al. [5 ] on the search set, and it serves synthetic approximation accuracy. P ^ θ , m \widehat{P}_{\theta,m} P θ , m P θ P_\theta P θ Lemma 3 ). In practice K K K
The recourse assumptions concern the merged atoms of § 3.5 at the fixed mesh { x ˉ j } j ≤ N 1 \{\bar{x}_j\}_{j\le N_1} { x ˉ j } j ≤ N 1 exceptional set
D : = { θ ∈ Θ : the fiber partition { I ℓ ( ⋅ ) } is not constant on any neighborhood of θ } ∪ ⋃ j ≤ N 1 { θ ∈ Θ : G ( ⋅ , x ˉ j ) is discontinuous at θ } . (31) D := \left\{\theta\in\Theta: \text{the fiber partition } \{I_\ell(\cdot)\} \text{ is not constant on any neighborhood of } \theta\right\}
\cup\bigcup_{j\le N_1}\left\{\theta\in\Theta: G(\cdot,\bar{x}_j) \text{ is discontinuous at } \theta\right\}. \tag*{(31)} D := { θ ∈ Θ : the fiber partition { I ℓ ( ⋅ )} is not constant on any neighborhood of θ } ∪ j ≤ N 1 ⋃ { θ ∈ Θ : G ( ⋅ , x ˉ j ) is discontinuous at θ } . (31) Assumption 4 (Local stability of the merged atoms). The prior does not charge the exceptional set, Π ( D ) = 0 \Pi(D)=0 Π ( D ) = 0
Table 1 (PDF p. 12). Examples and the condition giving Assumption 5.
For θ 0 ∉ D \theta_0 \notin D θ 0 ∈ / D U U U θ 0 \theta_0 θ 0 L ( θ ) = L ( θ 0 ) L(\theta)=L(\theta_0) L ( θ ) = L ( θ 0 ) z ℓ ( θ ) = G ( θ , x ˉ j ) z_\ell(\theta)=G(\theta,\bar{x}_j) z ℓ ( θ ) = G ( θ , x ˉ j ) j ∈ I ℓ ( θ 0 ) j\in I_\ell(\theta_0) j ∈ I ℓ ( θ 0 ) θ 0 \theta_0 θ 0 θ ∈ U \theta\in U θ ∈ U V ℓ ( θ ) V_\ell(\theta) V ℓ ( θ ) (24) are therefore defined with a common number of atoms and common labels, and the following maps make sense.
Assumption 5 (Continuity of the cell masses). Almost surely under Π \Pi Π ℓ ≤ L ( θ 0 ) \ell\le L(\theta_0) ℓ ≤ L ( θ 0 )
θ ↦ P θ 0 ( V ℓ ( θ ) ) and θ ↦ P θ ( V ℓ ( θ ) ) \theta\mapsto P_{\theta_0}(V_\ell(\theta)) \qquad\text{and} \qquad\theta\mapsto P_\theta(V_\ell(\theta)) θ ↦ P θ 0 ( V ℓ ( θ )) and θ ↦ P θ ( V ℓ ( θ )) are continuous at θ 0 \theta_0 θ 0
Assumption 5 is exactly what the recourse consistency theorem uses. The first map controls the population target of the recourse weights, the second the model’s own Voronoi masses. It is not checked directly; instead, either of the following two conditions implies it (Lemma 6 , Appendix § 12 ). The first suits continuous observations and atoms that move with θ \theta θ
(5a) (Boundary-null cells. ) Almost surely under Π \Pi Π P θ 0 P_{\theta_0} P θ 0
P θ 0 ( ⋃ ℓ ≠ ℓ ′ { y ∈ S : d ( y , z ℓ ( θ 0 ) ) = d ( y , z ℓ ′ ( θ 0 ) ) } ) = 0. P_{\theta_0}\left(\bigcup_{\ell\ne\ell'}\{y\in S:d(y,z_\ell(\theta_0))=d(y,z_{\ell'}(\theta_0))\}\right)=0. P θ 0 ℓ = ℓ ′ ⋃ { y ∈ S : d ( y , z ℓ ( θ 0 )) = d ( y , z ℓ ′ ( θ 0 ))} = 0. (5b) (Fixed atoms on a discrete support. ) Almost surely under Π \Pi Π U ′ U' U ′ θ 0 \theta_0 θ 0 Y ⊂ S \mathcal{Y}\subset S Y ⊂ S η : = inf { d ( y , y ′ ) : y ≠ y ′ ∈ Y } > 0 \eta:=\inf\{d(y,y'):y\ne y'\in\mathcal{Y}\}>0 η := inf { d ( y , y ′ ) : y = y ′ ∈ Y } > 0 z ℓ ( θ ) = z ℓ ( θ 0 ) z_\ell(\theta)=z_\ell(\theta_0) z ℓ ( θ ) = z ℓ ( θ 0 ) ℓ \ell ℓ θ ∈ U ′ \theta\in U' θ ∈ U ′ P θ ( Y ) = 1 P_\theta(\mathcal{Y})=1 P θ ( Y ) = 1 θ ∈ U ′ \theta\in U' θ ∈ U ′
The verification of Assumptions 1–5 for a location–scale family, for Poisson counts and for Ornstein–Uhlenbeck paths is given in Appendix § 13 . Table 1 summarizes which condition gives Assumption 5 in each example of the paper. Continuous observations whose atoms move with θ \theta θ
Experiments
We illustrate the procedure on four models: a Gaussian location–scale family, the number of segregating sites under Kingman’s coalescent, a pure birth process, and an Ornstein–Uhlenbeck diffusion. In every example the noise is simulated in the coordinates in which the mechanism is written, and the recourse mesh is the N 1 N_1 N 1 § 3.5 ). The estimation stage is Algorithm 1. For scalar observations the objective is the exact W 1 W_1 W 1 [30 ] . The quantization metric d Q d_Q d Q r = 2 r=2 r = 2 d d d w ℓ v o r ( θ ~ n 0 ) w_\ell^{\mathrm{vor}}(\tilde{\theta}_{n_0}) w ℓ vor ( θ ~ n 0 ) (27) ; cells above the interval are reported as excess and cells below it as deficit . In each example the structural parameter θ 0 \theta_0 θ 0
Table 2. Summary of the experiments: noise law, quantization metric d Q d_Q d Q d d d p p p
Table 3. Location–scale family: estimation stage. Minimum-W 1 W_1 W 1 ( μ ⋆ , σ ⋆ ) = ( 0.684 , 0.876 ) (\mu_\star,\sigma_\star)=(0.684,0.876) ( μ ⋆ , σ ⋆ ) = ( 0.684 , 0.876 )
Gaussian location–scale family
Model and misspecifications. The mechanism is
Y = G ( θ , X ) = μ + σ X , X ∼ ν = N ( 0 , 1 ) , θ = ( μ , σ 2 ) , Y=G(\theta,X)=\mu+\sigma X,\qquad X\sim\nu=\mathcal{N}(0,1),\qquad\theta=(\mu,\sigma^2), Y = G ( θ , X ) = μ + σ X , X ∼ ν = N ( 0 , 1 ) , θ = ( μ , σ 2 ) , with the normal–inverse-gamma prior
σ 2 ∼ InvGamma ( 3 , 2 ) ; μ ∣ σ 2 ∼ N ( 0 , σ 2 ) . \sigma^2\sim\operatorname{InvGamma}(3,2);\qquad\mu\mid\sigma^2\sim\mathcal{N}(0,\sigma^2). σ 2 ∼ InvGamma ( 3 , 2 ) ; μ ∣ σ 2 ∼ N ( 0 , σ 2 ) . The single draw is θ 0 = ( 0.684 , 0.767 ) \theta_0=(0.684,0.767) θ 0 = ( 0.684 , 0.767 ) n = 5000 n=5000 n = 5000 n 0 = 4000 n_0=4000 n 0 = 4000 n 1 = 1000 n_1=1000 n 1 = 1000 m = 10 4 m=10^4 m = 1 0 4 N 1 = 100 N_1=100 N 1 = 100 M = 2000 M=2000 M = 2000 n 1 / N 1 = 10 n_1/N_1=10 n 1 / N 1 = 10 d Q ( x , x ′ ) = ∣ x − x ′ ∣ d_Q(x,x')=|x-x'| d Q ( x , x ′ ) = ∣ x − x ′ ∣ X = R \mathcal{X}=\mathbb{R} X = R d ( y , y ′ ) = ∣ y − y ′ ∣ d(y,y')=|y-y'| d ( y , y ′ ) = ∣ y − y ′ ∣ S = R S=\mathbb{R} S = R
Three laws generate the data: the model’s own law N ( μ ⋆ , σ ⋆ 2 ) \mathcal{N}(\mu_\star,\sigma_\star^2) N ( μ ⋆ , σ ⋆ 2 ) μ ⋆ + σ ⋆ t 2 \mu_\star+\sigma_\star t_2 μ ⋆ + σ ⋆ t 2 N ( μ ⋆ + 10 σ ⋆ , 0.5 2 ) \mathcal{N}(\mu_\star+10\sigma_\star,0.5^2) N ( μ ⋆ + 10 σ ⋆ , 0. 5 2 )
Estimation stage and prior test. Table 3 compares the minimum-W 1 W_1 W 1 t 2 t_2 t 2 W 1 W_1 W 1 σ ~ = 1.27 \tilde{\sigma}=1.27 σ ~ = 1.27 σ ^ = 3.23 \hat{\sigma}=3.23 σ ^ = 3.23 σ \sigma σ μ \mu μ W 1 W_1 W 1
Prior misspecification. The prior test of § 4 scores ∥ θ ~ ∥ \lVert\tilde{\theta}\rVert ∥ θ ~ ∥ θ 0 \theta_0 θ 0 p = 0.99 p = 0.99 p = 0.99 θ ~ \tilde{\theta} θ ~ ( α 0 , β 0 , m 0 , κ 0 ) (\alpha_0,\beta_0,m_0,\kappa_0) ( α 0 , β 0 , m 0 , κ 0 ) Figure 2 ). Only the two gross misspecifications are rejected at the 5% level: a prior mean shifted to m 0 = 5 m_0 = 5 m 0 = 5 ( α 0 , β 0 ) = ( 10 , 30 ) (\alpha_0,\beta_0) = (10,30) ( α 0 , β 0 ) = ( 10 , 30 ) p < 10 − 4 p < 10^{-4} p < 1 0 − 4 m 0 = 2 m_0 = 2 m 0 = 2 p = 0.09 p = 0.09 p = 0.09 σ 2 \sigma^2 σ 2 p = 0.13 p = 0.13 p = 0.13 κ 0 = 25 \kappa_0 = 25 κ 0 = 25 p = 0.61 p = 0.61 p = 0.61 θ 0 \theta_0 θ 0 θ ~ \tilde{\theta} θ ~ ∥ θ ∥ \lVert\theta\rVert ∥ θ ∥ ∥ θ ∥ \lVert\theta\rVert ∥ θ ∥
Figure 2. Gaussian location–scale family: prior law of ∥ θ ∥ \lVert\theta\rVert ∥ θ ∥ ∥ θ ~ ∥ \lVert\tilde{\theta}\rVert ∥ θ ~ ∥ ( α 0 , β 0 , m 0 , κ 0 ) = ( 3 , 2 , 0 , 1 ) (\alpha_0,\beta_0,m_0,\kappa_0) = (3, 2, 0, 1) ( α 0 , β 0 , m 0 , κ 0 ) = ( 3 , 2 , 0 , 1 )
Recourse stage. Figure 3 shows the recourse weights against the model’s Voronoi masses and Figure 4 their ratio on a logarithmic scale with a simulated 95% null envelope. Under correct specification 4 of the 100 cells fall outside their pointwise interval, against 3.05 expected, and the Pearson test gives p = 0.59 p = 0.59 p = 0.59 t 2 t_2 t 2 p = 6 × 10 − 26 p = 6 \times10^{-26} p = 6 × 1 0 − 26 6 × 10 − 20 6 \times10^{-20} 6 × 1 0 − 20
Figure 3. Gaussian location–scale family: optimized recourse weights w ~ ℓ \tilde{w}_{\ell} w ~ ℓ w ℓ v o r ( θ ~ n 0 ) w_{\ell}^{\mathrm{vor}}(\tilde{\theta}_{n0}) w ℓ vor ( θ ~ n 0 ) normal , t2 and contam data. Red diamonds: cells above the interval (excess); blue diamonds: cells below it (deficit). Cells are ordered by their atom.
Figure 4. Gaussian location–scale family: w ~ ℓ / w ℓ v o r ( θ ~ n 0 ) \tilde{w}_{\ell}/w_{\ell}^{\mathrm{vor}}(\tilde{\theta}_{n0}) w ~ ℓ / w ℓ vor ( θ ~ n 0 ) θ ~ n 0 \tilde{\theta}_{n0} θ ~ n 0
Calibration and power. Over 400 replicate data sets of the whole pipeline, with the synthetic sample and the mesh held fixed, the rejection rate at the 5% level is 0.040 (Monte Carlo standard error 0.010) under correct specification, and 1.00 under both alternatives (Figure 5 ).
Figure 5. Gaussian location–scale family: rejection rate of the recourse χ 2 \chi^2 χ 2 normal bar is the size.
Segregating sites under the coalescent
Model and misspecification. A sample of n = 10 n = 10 n = 10 [26 ] , and mutations fall on its branches as a Poisson process. We write the noise as the coalescence exponentials E 2 , … , E n E_2,\ldots,E_n E 2 , … , E n N N N
S n = G ( θ , x ) = N ( θ L n / 2 ) , L n = ∑ k = 2 n 2 E k k − 1 , S_n = G(\theta,x) = N(\theta L_n/2), \qquad L_n = \sum_{k=2}^{n}\frac{2E_k}{k-1}, S n = G ( θ , x ) = N ( θ L n /2 ) , L n = k = 2 ∑ n k − 1 2 E k , the number of clock arrivals by internal time θ L n / 2 \theta L_n/2 θ L n /2 S n ∣ L n ∼ Poisson ( θ L n / 2 ) S_n \mid L_n \sim\operatorname{Poisson}(\theta L_n/2) S n ∣ L n ∼ Poisson ( θ L n /2 ) [42 ] . Observations are counts at independent loci. The probability mass function of S n S_n S n N 1 = 300 N_1 = 300 N 1 = 300 M = 2500 M = 2500 M = 2500 d Q d_Q d Q log L n \log L_n log L n log R \log R log R R = 10 / P 10 R = 10/P_{10} R = 10/ P 10 P 10 P_{10} P 10
( ( E 2 , … , E 10 ) ⊤ , ( ϵ 1 , … ) ⊤ ) ∈ ( 0 , ∞ ) 9 × ( 0 , ∞ ) N , \left((E_2,\ldots,E_{10})^\top,(\epsilon_1,\ldots)^\top\right)\in(0,\infty)^9\times(0,\infty)^{\mathbb{N}}, ( ( E 2 , … , E 10 ) ⊤ , ( ϵ 1 , … ) ⊤ ) ∈ ( 0 , ∞ ) 9 × ( 0 , ∞ ) N , where ϵ k ∼ i . i . d . Exp ( 1 ) \epsilon_k \overset{\mathrm{i.i.d.}}{\sim} \operatorname{Exp}(1) ϵ k ∼ i.i.d. Exp ( 1 ) d ( y , y ′ ) = ∣ y − y ′ ∣ d(y,y') = |y-y'| d ( y , y ′ ) = ∣ y − y ′ ∣ S = R S = \mathbb{R} S = R LogNormal ( log 5 , 0.5 2 ) \operatorname{LogNormal}(\log5,0.5^2) LogNormal ( log 5 , 0. 5 2 ) θ 0 = 8.48 \theta_0 = 8.48 θ 0 = 8.48 n 0 = 4000 n_0 = 4000 n 0 = 4000 n 1 = 1000 n_1 = 1000 n 1 = 1000 m = 10 4 m = 10^4 m = 1 0 4 g = 0.7 g = 0.7 g = 0.7 [37 ] . Backwards in time k k k ( k 2 ) e g t \binom{k}{2}e^{gt} ( 2 k ) e g t
Estimation stage. The minimum-W 1 W_1 W 1 θ ~ \tilde{\theta} θ ~ p = 0.30 p = 0.30 p = 0.30 p = 0.29 p = 0.29 p = 0.29 θ 0 \theta_0 θ 0
Recourse stage. The 300 mesh points merge into 63 count atoms under correct specification and 46 under growth (Figure 6 ). Under correct specification no atom is flagged and the Pearson test gives p = 0.83 p = 0.83 p = 0.83 p = 3 × 10 − 9 p = 3 \times10^{-9} p = 3 × 1 0 − 9 Figure 7 ) shows how the misspecification appears in the noise coordinates of the fitted model. The mesh points in the excess fibers have long genealogies (median L n = 6.5 L_n = 6.5 L n = 6.5 L n = 3.0 L_n = 3.0 L n = 3.0
Figure 6. Segregating sites: recourse weights on the merged count atoms against the exact Voronoi masses, for the constant-size (correct) and exponential-growth (growth) data. Red and blue diamonds: excess and deficit at the pointwise 95% level.
Figure 7. Segregating sites: the recourse lifted to the noise mesh (Remark 1). Each mesh point is placed at its genealogy length L n L_n L n R R R log 2 ( w ~ ℓ / w v o r ) \log_{2}(\tilde{w}_{\ell}/w^{\mathrm{vor}}) log 2 ( w ~ ℓ / w vor ) L n R L_nR L n R
Yule process
Model and misspecification. A population starts from z 0 = 5 z_0 = 5 z 0 = 5 λ \lambda λ t = 1 , 2 , 3 , 4 t = 1, 2, 3, 4 t = 1 , 2 , 3 , 4 Y = ( Z ( 1 ) , … , Z ( 4 ) ) ∈ R 4 Y = (Z(1), \ldots, Z(4)) \in\mathbb{R}^4 Y = ( Z ( 1 ) , … , Z ( 4 )) ∈ R 4 θ = λ \theta= \lambda θ = λ (8) with a single reaction channel,
Z ( t ) = z 0 + ∫ 0 t ∫ 0 ∞ 1 { u ≤ λ Z ( s − ) } N ( d s , d u ) , Z(t) = z_0 + \int_0^t \int_0^\infty\mathbf{1}\{u \le\lambda Z(s-)\} N(\mathrm{d}s,\mathrm{d}u), Z ( t ) = z 0 + ∫ 0 t ∫ 0 ∞ 1 { u ≤ λ Z ( s − )} N ( d s , d u ) , and the noise is the Poisson sample path N N N [ 0 , 4 ] × [ 0 , 150 ] [0,4] \times[0,150] [ 0 , 4 ] × [ 0 , 150 ] ( s , u ) (s,u) ( s , u ) s s s u ≤ λ Z ( s − ) u \le\lambda Z(s-) u ≤ λ Z ( s − ) m = 2000 m = 2000 m = 2000 d Q d_Q d Q [ 0 , 4 ] × [ 0 , 32 ] [0,4] \times[0,32] [ 0 , 4 ] × [ 0 , 32 ] S = R 4 S = \mathbb{R}^4 S = R 4 ( Z ( 1 ) , … , Z ( 4 ) ) (Z(1), \ldots, Z(4)) ( Z ( 1 ) , … , Z ( 4 )) λ ∼ LogNormal ( log 0.5 , 0.15 2 ) \lambda\sim\operatorname{LogNormal}(\log0.5, 0.15^2) λ ∼ LogNormal ( log 0.5 , 0.1 5 2 ) λ 0 = 0.576 \lambda_0 = 0.576 λ 0 = 0.576 n 0 = 4000 n_0 = 4000 n 0 = 4000 n 1 = 1000 n_1 = 1000 n 1 = 1000 [19 ] . The misspecification is a state-dependent birth rate. In saturating the per-capita birth rate is λ 0 K / ( K + z ) \lambda_0K/(K + z) λ 0 K / ( K + z ) K = 10 K = 10 K = 10 z ≪ K z \ll K z ≪ K z ≫ K z \gg K z ≫ K
Estimation stage. Under correct specification λ ~ = 0.571 \tilde{\lambda} = 0.571 λ ~ = 0.571 p p p λ 0 \lambda_0 λ 0 λ ~ = 0.297 \tilde{\lambda} = 0.297 λ ~ = 0.297 p = 0.0005 p = 0.0005 p = 0.0005
Recourse stage. The mesh merges into 149 and 135 distinct count vectors. Under correct specification 4 cells are flagged against 4.2 expected and the Pearson test gives p = 0.34 p = 0.34 p = 0.34 p = 2 × 10 − 54 p = 2 \times10^{-54} p = 2 × 1 0 − 54 Figure 8 ). The held-out trajectories grow too fast early and too slowly late for any Yule process. Drawn as trajectories (Figure 9 ), the excess atoms are nearly linear paths and the deficit atoms are the accelerating. Hence, the recourse locates the failure in the shape of the growth curve, which is exactly what the state dependence of the birth rate changes. The mesh points are sample paths of the Poisson random measure N N N
Figure 8. Yule process: recourse weights on the merged count vectors, ordered by Z ( 4 ) Z(4) Z ( 4 )
Figure 9. Yule process: atoms drawn as count trajectories. Top: atoms in observation space. Bottom: the corresponding mesh paths in noise space. Red: excess; blue: deficit; grey: inside.
N λ ~ ( t ) = # { ( s , u ) ∈ N : s ≤ t , u ≤ λ ~ z 0 e λ ~ s } . N_{\tilde{\lambda}}(t) = \#\{(s,u) \in N : s \le t,\ u \le\tilde{\lambda} z_0 e^{\tilde{\lambda}s}\}. N λ ~ ( t ) = # {( s , u ) ∈ N : s ≤ t , u ≤ λ ~ z 0 e λ ~ s } . In other words, N N N z 0 e λ ~ s z_0e^{\tilde{\lambda}s} z 0 e λ ~ s y y y Z Z Z λ Z \lambda Z λ Z
Ornstein–Uhlenbeck diffusion
Model and misspecification. The observation is the Ornstein–Uhlenbeck path d Y = − a Y d t + σ d W dY=-aY\,dt+\sigma\,dW d Y = − aY d t + σ d W Y ( 0 ) = 0 Y(0)=0 Y ( 0 ) = 0 [39 ] ), written through its Itô map
Y = G ( θ , x ) = σ ( x − a K a x ) , ( K a x ) ( t ) = ∫ 0 t e − a ( t − u ) x ( u ) d u , θ = ( a , σ ) , Y=G(\theta,x)=\sigma\left(x-aK_a x\right),\qquad(K_a x)(t)=\int_0^t e^{-a(t-u)}x(u)\,du,\qquad\theta=(a,\sigma), Y = G ( θ , x ) = σ ( x − a K a x ) , ( K a x ) ( t ) = ∫ 0 t e − a ( t − u ) x ( u ) d u , θ = ( a , σ ) , applied to a Brownian path x x x [ 0 , 2 ] [0,2] [ 0 , 2 ] K a x K_a x K a x m = 10 4 m=10^4 m = 1 0 4 d Q d_Q d Q L 2 L^2 L 2 S = R 10 S=\mathbb{R}^{10} S = R 10 L 2 ( [ 0 , 2 ] ) L^2([0,2]) L 2 ([ 0 , 2 ]) a ∼ LogNormal ( log 2 , 0.3 2 ) a\sim\operatorname{LogNormal}(\log2,0.3^2) a ∼ LogNormal ( log 2 , 0. 3 2 ) σ ∼ LogNormal ( 0 , 0.3 2 ) \sigma\sim\operatorname{LogNormal}(0,0.3^2) σ ∼ LogNormal ( 0 , 0. 3 2 ) θ 0 = ( 2.06 , 1.08 ) \theta_0=(2.06,1.08) θ 0 = ( 2.06 , 1.08 ) n 0 = 4000 n_0=4000 n 0 = 4000 n 1 = 1000 n_1=1000 n 1 = 1000 G ( θ 0 , X ) G(\theta_0,X) G ( θ 0 , X ) X X X d X = − 1.5 X d t + d W dX=-1.5X\,dt+dW d X = − 1.5 X d t + d W X ( 0 ) = 0 X(0)=0 X ( 0 ) = 0
Estimation stage. Under correct specification θ ~ = ( 2.02 , 1.07 ) \tilde{\theta}=(2.02,1.07) θ ~ = ( 2.02 , 1.07 ) p = 0.998 p=0.998 p = 0.998 0.95 0.95 0.95 θ 0 \theta_0 θ 0 θ ~ = ( 5.00 , 1.22 ) \tilde{\theta}=(5.00,1.22) θ ~ = ( 5.00 , 1.22 ) p = 0.002 p=0.002 p = 0.002
Recourse stage. Under correct specification 4 of the 80 cells are flagged against 2.7 expected and the Pearson test gives p = 0.04 p=0.04 p = 0.04 p p p p = 2 × 10 − 12 p=2\times10^{-12} p = 2 × 1 0 − 12 Figure 10 ). Because the noise is a Brownian path, the diagnostic is read off paths (§ 4.2 ). In noise space (Figure 11 , bottom row) the deficit cells are exactly the mesh paths with large excursions and the excess cells the paths that stay near the origin. The recourse recovers that the driving noise is more concentrated than Wiener measure.
Figure 10. Ornstein–Uhlenbeck diffusion: recourse weights on the mesh paths, ordered by the end point Z ( T ) Z(T) Z ( T )
Figure 11. Ornstein–Uhlenbeck diffusion: cells outside the pointwise 95% band, as paths. Top: atoms in observation space. Bottom: the corresponding mesh paths in noise space. Red: excess; blue: deficit; grey: inside.
Computation
The estimation stage minimizes W 1 ( P ^ ξ 0 , P ^ θ , m ) W_1(\widehat{P}_{\xi_0},\widehat{P}_{\theta,m}) W 1 ( P ξ 0 , P θ , m ) θ ∈ K \theta\in K θ ∈ K W 1 ( P ^ ξ 1 , P ~ θ ~ n 0 , w ) W_1(\widehat{P}_{\xi_1},\widetilde{P}_{\widetilde{\theta}_{n_0},w}) W 1 ( P ξ 1 , P θ n 0 , w ) w ∈ Δ L − 1 w \in\Delta^{L-1} w ∈ Δ L − 1
Evaluating the estimation-stage objective
Write a = ( a t ) a=(a_t) a = ( a t ) P ^ ξ 0 \widehat{P}_{\xi_0} P ξ 0 b = ( b i ) b=(b_i) b = ( b i ) P ^ θ , m \widehat{P}_{\theta,m} P θ , m P ^ ξ 1 \widehat{P}_{\xi_1} P ξ 1 b = w b=w b = w C t i = d ( y t , G ( θ , x ~ i ) ) C_{ti}=d(y_t,G(\theta,\widetilde{x}_i)) C t i = d ( y t , G ( θ , x i ))
W 1 ( P ^ ξ 0 , P ^ θ , m ) = min π ≥ 0 ⟨ C , π ⟩ s.t. π 1 = a , π T 1 = b , (32) W_1(\widehat{P}_{\xi_0},\widehat{P}_{\theta,m})=\min_{\pi\geq0}\langle C,\pi\rangle\quad\text{s.t.}\quad\pi\mathbf{1}=a,\ \pi^\mathsf{T}\mathbf{1}=b, \tag*{(32)} W 1 ( P ξ 0 , P θ , m ) = π ≥ 0 min ⟨ C , π ⟩ s.t. π 1 = a , π T 1 = b , (32) which is solved by the transportation (network) simplex or any linear-programming solver [32 ] . A standard implementation is [14 ] .
High-dimensional approximation: sliced W 1 W_1 W 1 For s > 1 s>1 s > 1 (32) has n 0 m n_0m n 0 m
SW 1 ( P ^ ξ 0 , P ^ θ , m ) : = 1 P ∑ p = 1 P W 1 ( ( π p ) # P ^ ξ 0 , ( π p ) # P ^ θ , m ) , π p ( y ) = ⟨ v p , y ⟩ , \operatorname{SW}_1(\widehat{P}_{\xi_0},\widehat{P}_{\theta,m}):=\frac{1}{P}\sum_{p=1}^{P}W_1\left((\pi_p)_\#\widehat{P}_{\xi_0},(\pi_p)_\#\widehat{P}_{\theta,m}\right),\qquad\pi_p(y)=\langle v_p,y\rangle, SW 1 ( P ξ 0 , P θ , m ) := P 1 p = 1 ∑ P W 1 ( ( π p ) # P ξ 0 , ( π p ) # P θ , m ) , π p ( y ) = ⟨ v p , y ⟩ , over a fixed set of unit directions v 1 , … , v P v_1,\ldots,v_P v 1 , … , v P W 1 ( μ , ν ) = ∫ R ∣ F μ − F ν ∣ W_1(\mu,\nu)=\int_{\mathbb{R}}|F_\mu-F_\nu| W 1 ( μ , ν ) = ∫ R ∣ F μ − F ν ∣
Estimation stage: a derivative-free search over θ \theta θ
The objective F n 0 ( θ ) : = W 1 ( P ^ ξ 0 , P ^ θ , m ) F_{n_0}(\theta):=W_1(\widehat{P}_{\xi_0},\widehat{P}_{\theta,m}) F n 0 ( θ ) := W 1 ( P ξ 0 , P θ , m ) θ \theta θ F n 0 F_{n_0} F n 0 § 7.1 . The grid phase locates the global basin, which matters because F n 0 F_{n_0} F n 0 [31 ] refines within it. The search is embarrassingly parallel.
Recourse mesh: computing the N 1 N_1 N 1
The mesh of § 3.5 is the N 1 N_1 N 1 (16) , which minimizes the empirical distortion
e r ( Γ ; ν M ) r = M − 1 ∑ i ≤ M min j ≤ N 1 d Q ( x ( i ) , x ˉ j ) r , ν M : = M − 1 ∑ i ≤ M δ x ( i ) , e_r(\Gamma;\nu_M)^r=M^{-1}\sum_{i\leq M}\min_{j\leq N_1}d_Q(x^{(i)},\bar{x}_j)^r,\qquad\nu_M:=M^{-1}\sum_{i\leq M}\delta_{x^{(i)}}, e r ( Γ ; ν M ) r = M − 1 i ≤ M ∑ j ≤ N 1 min d Q ( x ( i ) , x ˉ j ) r , ν M := M − 1 i ≤ M ∑ δ x ( i ) , over meshes of N 1 N_1 N 1
Recourse-stage algorithm
With θ ~ n 0 \widetilde{\theta}_{n_0} θ n 0 (23) [33 Lem. 4.9] says the optimal weights are nearest-atom counts and the value is the mean nearest-atom distance, so the entire stage is a merge followed by a nearest-neighbour pass over the held-out block ξ 1 \xi_1 ξ 1
Algorithm 1 Estimation stage: derivative-free search for Equation (21) .
Require: estimation block ξ 0 \xi_0 ξ 0 x ~ 1 : m \tilde{x}_{1:m} x ~ 1 : m ν \nu ν K K K θ ( 1 ) , … , θ ( G ) ∈ K \theta^{(1)},\ldots,\theta^{(G)} \in K θ ( 1 ) , … , θ ( G ) ∈ K h h h r ≤ G r \le G r ≤ G ϵ > 0 \epsilon> 0 ϵ > 0
for g = 1 , … , G g = 1,\ldots,G g = 1 , … , G
F g ← W 1 ( P ^ ξ 0 , P ^ θ ( g ) , m ) F_g \leftarrow W_1(\widehat{P}_{\xi_0},\widehat{P}_{\theta^{(g)},m}) F g ← W 1 ( P ξ 0 , P θ ( g ) , m ) § 7.1 )
end for
let ϑ ( 1 ) , … , ϑ ( r ) \vartheta^{(1)},\ldots,\vartheta^{(r)} ϑ ( 1 ) , … , ϑ ( r ) r r r F g F_g F g
for c = 1 , … , r c = 1,\ldots,r c = 1 , … , r
run Nelder–Mead on F n 0 F_{n_0} F n 0 ϑ ( c ) \vartheta^{(c)} ϑ ( c ) F n 0 : = + ∞ F_{n_0} := +\infty F n 0 := + ∞ K K K ϵ \epsilon ϵ ϑ ^ ( c ) \widehat{\vartheta}^{(c)} ϑ ( c )
end for
return θ ~ n 0 \widetilde{\theta}_{n_0} θ n 0 { θ ( g ) } g ≤ G ∪ { ϑ ^ ( c ) } c ≤ r \{\theta^{(g)}\}_{g \le G} \cup\{\widehat{\vartheta}^{(c)}\}_{c \le r} { θ ( g ) } g ≤ G ∪ { ϑ ( c ) } c ≤ r
Algorithm 2 Recourse mesh: approximate N 1 N_1 N 1
Require: synthetic noise x ~ 1 : m \tilde{x}_{1:m} x ~ 1 : m ν \nu ν N 1 N_1 N 1 M ≤ m M \le m M ≤ m d Q d_Q d Q r ≥ 1 r \ge1 r ≥ 1 ϵ > 0 \epsilon> 0 ϵ > 0
draw a random subsample x ( 1 ) , … , x ( M ) x^{(1)},\ldots,x^{(M)} x ( 1 ) , … , x ( M ) x ~ 1 : m \tilde{x}_{1:m} x ~ 1 : m M M M
(seeding, k-medoids++) x ˉ 1 ← x ( i ) \bar{x}_1 \leftarrow x^{(i)} x ˉ 1 ← x ( i ) i i i { 1 , … , M } \{1,\ldots,M\} { 1 , … , M } j = 2 , … , N 1 j = 2,\ldots,N_1 j = 2 , … , N 1 x ˉ j ← x ( i ) \bar{x}_j \leftarrow x^{(i)} x ˉ j ← x ( i ) min j ′ < j d Q ( x ( i ) , x ˉ j ′ ) r \min_{j' < j}d_Q(x^{(i)},\bar{x}_{j'})^r min j ′ < j d Q ( x ( i ) , x ˉ j ′ ) r
repeat
(assignment) assign each x ( i ) x^{(i)} x ( i )
(medoid update) replace each x ˉ j \bar{x}_j x ˉ j ∑ i ∈ C j d Q ( x ( i ) , ⋅ ) r \sum_{i \in C_j}d_Q(x^{(i)},\cdot)^r ∑ i ∈ C j d Q ( x ( i ) , ⋅ ) r
(swap) replace a mesh point x ˉ j \bar{x}_j x ˉ j x ( i ) x^{(i)} x ( i )
until the empirical distortion decreases by less than ϵ \epsilon ϵ
q j ← q_j \leftarrow q j ← C j C_j C j ν \nu ν
return mesh { x ˉ j } j ≤ N 1 \{\bar{x}_j\}_{j \le N_1} { x ˉ j } j ≤ N 1 { q j } j ≤ N 1 \{q_j\}_{j \le N_1} { q j } j ≤ N 1
Algorithm 3 Recourse stage: closed-form solution of Equation (22) and the calibrated test.
Require: held-out block ξ 1 \xi_1 ξ 1 θ ~ n 0 \widetilde{\theta}_{n_0} θ n 0 { x ˉ j } j = 1 N 1 \{\bar{x}_j\}_{j=1}^{N_1} { x ˉ j } j = 1 N 1
raw atoms G ( θ ~ n 0 , x ˉ j ) G(\widetilde{\theta}_{n_0},\bar{x}_j) G ( θ n 0 , x ˉ j ) j = 1 , … , N 1 j = 1,\ldots,N_1 j = 1 , … , N 1
(merge) sort or hash the raw atoms and group equal values into the fibers I ℓ I_\ell I ℓ z 1 , … , z L z_1,\ldots,z_L z 1 , … , z L
build a nearest-neighbour index on { z ℓ } \{z_\ell\} { z ℓ }
for t ∈ ξ 1 t \in\xi_1 t ∈ ξ 1
ℓ ( t ) ← arg min ℓ d ( y t , z ℓ ) \ell(t) \leftarrow\arg\min_\ell d(y_t,z_\ell) ℓ ( t ) ← arg min ℓ d ( y t , z ℓ ) d t ← d ( y t , z ℓ ( t ) ) d_t \leftarrow d(y_t,z_{\ell(t)}) d t ← d ( y t , z ℓ ( t ) )
end for
w ~ ℓ ← # { t : ℓ ( t ) = ℓ } / n 1 \widetilde{w}_\ell\leftarrow\#\{t:\ell(t)=\ell\}/n_1 w ℓ ← # { t : ℓ ( t ) = ℓ } / n 1 ℓ = 1 , … , L \ell= 1,\ldots,L ℓ = 1 , … , L Q ( θ ~ n 0 ) ← n 1 − 1 ∑ t ∈ ξ 1 d t Q(\widetilde{\theta}_{n_0}) \leftarrow n_1^{-1}\sum_{t\in\xi_1}d_t Q ( θ n 0 ) ← n 1 − 1 ∑ t ∈ ξ 1 d t
w ℓ v o r ( θ ~ n 0 ) ← P θ ~ n 0 ( V ℓ ( θ ~ n 0 ) ) w_\ell^{\mathrm{vor}}(\widetilde{\theta}_{n_0}) \leftarrow P_{\widetilde{\theta}_{n_0}}(V_\ell(\widetilde{\theta}_{n_0})) w ℓ vor ( θ n 0 ) ← P θ n 0 ( V ℓ ( θ n 0 )) ℓ = 1 , … , L \ell= 1,\ldots,L ℓ = 1 , … , L P θ ~ n 0 P_{\widetilde{\theta}_{n_0}} P θ n 0 { G ( θ ~ n 0 , X ) = z ℓ } \{G(\widetilde{\theta}_{n_0},X)=z_\ell\} { G ( θ n 0 , X ) = z ℓ }
pool cells with small expected count n 1 w ℓ v o r ( θ ~ n 0 ) n_1w_\ell^{\mathrm{vor}}(\widetilde{\theta}_{n_0}) n 1 w ℓ vor ( θ n 0 ) T n 1 T_{n_1} T n 1 χ 2 \chi^2 χ 2 p p p
return weights w ~ \widetilde{w} w Q ( θ ~ n 0 ) Q(\widetilde{\theta}_{n_0}) Q ( θ n 0 ) w ~ ℓ \widetilde{w}_\ell w ℓ w ℓ v o r ( θ ~ n 0 ) w_\ell^{\mathrm{vor}}(\widetilde{\theta}_{n_0}) w ℓ vor ( θ n 0 )
Discussion
We have cast model criticism for structural models Y = G ( θ , X ) Y = G(\theta, X) Y = G ( θ , X )
Ratmann et al. [34 ] proposed model criticism within approximate Bayesian computation. The ABC tolerance is replaced by an error term on each summary statistic, and the posterior of these errors, computed jointly with the parameters, signals which summaries the model fails to reproduce. That approach shares our setting, models that can be simulated but whose likelihood is not available, and our aim of reporting where a model fails. It differs in two respects. First, its diagnostic is expressed in the summary statistics chosen by the analyst, whereas ours is expressed on the atoms of the fitted model, i.e. on regions of the noise space, and requires no summary. The price is that the diagnostic is resolved only up to the fibers of the fitted mechanism (Remark 1). Second, the posterior of the errors is a Bayesian summary without calibration, whereas the split makes the recourse counts conditionally multinomial and the Pearson test calibrated.
We conclude with limitations. The theory requires i.i.d. replicates, so a single long trajectory of a process is not covered. The recourse identifies the noise law only on the σ \sigma σ n 1 / N 1 n_1/N_1 n 1 / N 1 θ ~ \tilde{\theta} θ ~ n 1 / n 0 n_1/n_0 n 1 / n 0
Code Availability
Acknowledgments
The author thanks Professors David Blei and Simon Tavaré for their helpful comments on this manuscript. RYW acknowledges support from the Herbert and Florence Irving Institute for Cancer Dynamics at Columbia University.
Competing Interests
The author declares no competing interests.
Proofs
Estimation stage
Proposition 1 (Wasserstein strong law). Let Assumption 1(i) hold and let Y 1 , Y 2 , … ∼ i i d P θ 0 Y_1,Y_2,\ldots\overset{\mathrm{iid}}{\sim} P_{\theta_0} Y 1 , Y 2 , … ∼ iid P θ 0
P { lim m → ∞ W 1 ( 1 m ∑ t = 1 m δ Y t , P θ 0 ) = 0 } = 1. \mathbb{P}\left\{\lim_{m\to\infty} W_1\left(\frac{1}{m}\sum_{t=1}^{m}\delta_{Y_t},P_{\theta_0}\right)=0\right\}=1. P { m → ∞ lim W 1 ( m 1 t = 1 ∑ m δ Y t , P θ 0 ) = 0 } = 1. Consequently, W 1 ( P ^ ξ 0 , P θ 0 ) → 0 W_1(\widehat{P}_{\xi_0},P_{\theta_0})\to0 W 1 ( P ξ 0 , P θ 0 ) → 0 n 0 → ∞ n_0\to\infty n 0 → ∞ W 1 ( P ^ ξ 1 , P θ 0 ) → 0 W_1(\widehat{P}_{\xi_1},P_{\theta_0})\to0 W 1 ( P ξ 1 , P θ 0 ) → 0 n 1 → ∞ n_1\to\infty n 1 → ∞ P θ 0 P_{\theta_0} P θ 0 P θ P_\theta P θ θ ∈ Θ \theta\in\Theta θ ∈ Θ G ( θ , x ~ 1 : m ) G(\theta,\widetilde{x}_{1:m}) G ( θ , x 1 : m )
Proof of Proposition 1. This is Villani [41 ] combined with the strong law of large numbers for the empirical measure and its first moment. Quantitative rates are available under higher moments (Fournier and Guillin, [15 ] ). □
Lemma 1 (Wasserstein-continuity of the model map). Under Assumption 1 , for all θ , θ ′ ∈ Θ \theta,\theta'\in\Theta θ , θ ′ ∈ Θ ∥ θ ′ − θ ∥ < δ \lVert\theta'-\theta\rVert<\delta ∥ θ ′ − θ ∥ < δ
W 1 ( P θ , P θ ′ ) ≤ ∫ X d ( G ( θ , x ) , G ( θ ′ , x ) ) ν ( d x ) ≤ ω θ ( δ ) . W_1(P_\theta,P_{\theta'})\le\int_{\mathcal{X}}d(G(\theta,x),G(\theta',x))\,\nu(dx)\le\omega_\theta(\delta). W 1 ( P θ , P θ ′ ) ≤ ∫ X d ( G ( θ , x ) , G ( θ ′ , x )) ν ( d x ) ≤ ω θ ( δ ) . In particular θ k → θ \theta_k\to\theta θ k → θ W 1 ( P θ k , P θ ) → 0 W_1(P_{\theta_k},P_\theta)\to0 W 1 ( P θ k , P θ ) → 0 P θ k ⇒ P θ P_{\theta_k}\Rightarrow P_\theta P θ k ⇒ P θ
Proof of Lemma 1. The law of ( G ( θ , X ) , G ( θ ′ , X ) ) (G(\theta,X),G(\theta',X)) ( G ( θ , X ) , G ( θ ′ , X )) X ∼ ν X\sim\nu X ∼ ν P θ P_\theta P θ P θ ′ P_{\theta'} P θ ′
W 1 ( P θ , P θ ′ ) = inf π ∈ Π ( ν , ν ) ∫ X × X d ( G ( θ , x 1 ) , G ( θ ′ , x 2 ) ) π ( d x 1 , d x 2 ) , W_1(P_\theta,P_{\theta'})=\inf_{\pi\in\Pi(\nu,\nu)}\int_{\mathcal{X}\times\mathcal{X}}d(G(\theta,x_1),G(\theta',x_2))\,\pi(dx_1,dx_2), W 1 ( P θ , P θ ′ ) = π ∈ Π ( ν , ν ) inf ∫ X × X d ( G ( θ , x 1 ) , G ( θ ′ , x 2 )) π ( d x 1 , d x 2 ) , the first inequality holds. The second is the definition of ω θ \omega_\theta ω θ Assumption 1(ii) , and W 1 W_1 W 1 [41 ] ). □
Lemma 2 (Continuity of the exact objective and the synthetic error). For every fixed Q ∈ P 1 ( S ) Q\in\mathcal{P}_1(S) Q ∈ P 1 ( S ) θ ↦ W 1 ( Q , P θ ) \theta\mapsto W_1(Q,P_\theta) θ ↦ W 1 ( Q , P θ ) Θ \Theta Θ Assumption 1 , and for every θ \theta θ m m m
∣ W 1 ( Q , P ^ θ , m ) − W 1 ( Q , P θ ) ∣ ≤ W 1 ( P θ , P ^ θ , m ) . \left|W_1(Q,\widehat{P}_{\theta,m})-W_1(Q,P_\theta)\right|\le W_1(P_\theta,\widehat{P}_{\theta,m}). W 1 ( Q , P θ , m ) − W 1 ( Q , P θ ) ≤ W 1 ( P θ , P θ , m ) . Proof of Lemma 2. Both statements follow from the reverse triangle inequality,
∣ W 1 ( Q , μ ) − W 1 ( Q , μ ′ ) ∣ ≤ W 1 ( μ , μ ′ ) , \left|W_1(Q,\mu)-W_1(Q,\mu')\right|\le W_1(\mu,\mu'), ∣ W 1 ( Q , μ ) − W 1 ( Q , μ ′ ) ∣ ≤ W 1 ( μ , μ ′ ) , with ( μ , μ ′ ) = ( P θ , P θ ′ ) (\mu,\mu')=(P_\theta,P_{\theta'}) ( μ , μ ′ ) = ( P θ , P θ ′ ) ( μ , μ ′ ) = ( P ^ θ , m , P θ ) (\mu,\mu')=(\widehat{P}_{\theta,m},P_\theta) ( μ , μ ′ ) = ( P θ , m , P θ )
Lemma 3 (Uniform synthetic approximation). Let Assumption 1 hold and let K ⊆ Θ K\subseteq\Theta K ⊆ Θ
Δ m : = sup θ ∈ K W 1 ( P θ , P ^ θ , m ) ⟶ 0 almost surely as m → ∞ . \Delta_m:=\sup_{\theta\in K}W_1(P_\theta,\widehat{P}_{\theta,m})\longrightarrow0\qquad\text{almost surely as }m\to\infty. Δ m := θ ∈ K sup W 1 ( P θ , P θ , m ) ⟶ 0 almost surely as m → ∞. Proof of Lemma 3. For θ ∈ K \theta\in K θ ∈ K δ > 0 \delta>0 δ > 0
Δ θ , δ ( x ) : = sup θ ′ ∈ Θ , ∥ θ ′ − θ ∥ < δ d ( G ( θ ′ , x ) , G ( θ , x ) ) , \Delta_{\theta,\delta}(x):=\sup_{\theta'\in\Theta,\ \lVert\theta'-\theta\rVert<\delta}d(G(\theta',x),G(\theta,x)), Δ θ , δ ( x ) := θ ′ ∈ Θ , ∥ θ ′ − θ ∥ < δ sup d ( G ( θ ′ , x ) , G ( θ , x )) , such that ∫ Δ θ , δ d ν = ω θ ( δ ) \int\Delta_{\theta,\delta}\,d\nu= \omega_\theta(\delta) ∫ Δ θ , δ d ν = ω θ ( δ ) δ \delta δ θ ′ \theta' θ ′ ∥ θ ′ − θ ∥ < δ \lVert\theta' - \theta\rVert< \delta ∥ θ ′ − θ ∥ < δ Lemma 1 , and the coupling of P ^ θ , m \widehat{P}_{\theta,m} P θ , m P ^ θ ′ , m \widehat{P}_{\theta',m} P θ ′ , m G ( θ , x ~ i ) G(\theta,\widetilde{x}_i) G ( θ , x i ) G ( θ ′ , x ~ i ) G(\theta',\widetilde{x}_i) G ( θ ′ , x i )
W 1 ( P θ ′ , P ^ θ ′ , m ) ≤ W 1 ( P θ ′ , P θ ) + W 1 ( P θ , P ^ θ , m ) + W 1 ( P ^ θ , m , P ^ θ ′ , m ) ≤ ω θ ( δ ) + W 1 ( P θ , P ^ θ , m ) + 1 m ∑ i = 1 m Δ θ , δ ( x ~ i ) . \begin{aligned}
W_1(P_{\theta'},\widehat{P}_{\theta',m}) \le W_1(P_{\theta'},P_\theta) + W_1(P_\theta,\widehat{P}_{\theta,m}) + W_1(\widehat{P}_{\theta,m},\widehat{P}_{\theta',m}) \\
&\le\omega_\theta(\delta) + W_1(P_\theta,\widehat{P}_{\theta,m}) + \frac{1}{m}\sum_{i=1}^{m}\Delta_{\theta,\delta}(\widetilde{x}_i).
\end{aligned} W 1 ( P θ ′ , P θ ′ , m ) ≤ W 1 ( P θ ′ , P θ ) + W 1 ( P θ , P θ , m ) + W 1 ( P θ , m , P θ ′ , m ) ≤ ω θ ( δ ) + W 1 ( P θ , P θ , m ) + m 1 i = 1 ∑ m Δ θ , δ ( x i ) . Note the same x ~ i \widetilde{x}_i x i Proposition 1 applied to P θ P_\theta P θ Δ θ , δ ( x ~ i ) \Delta_{\theta,\delta}(\widetilde{x}_i) Δ θ , δ ( x i )
lim sup m → ∞ sup ∥ θ ′ − θ ∥ < δ W 1 ( P θ ′ , P ^ θ ′ , m ) ≤ 2 ω θ ( δ ) . \limsup_{m\to\infty}\ \sup_{\lVert\theta'-\theta\rVert<\delta} W_1(P_{\theta'},\widehat{P}_{\theta',m}) \le2\omega_\theta(\delta). m → ∞ lim sup ∥ θ ′ − θ ∥ < δ sup W 1 ( P θ ′ , P θ ′ , m ) ≤ 2 ω θ ( δ ) . Fix ϵ > 0 \epsilon> 0 ϵ > 0 θ ∈ K \theta\in K θ ∈ K δ θ > 0 \delta_\theta> 0 δ θ > 0 ω θ ( δ θ ) ≤ ϵ / 2 \omega_\theta(\delta_\theta) \le\epsilon/2 ω θ ( δ θ ) ≤ ϵ /2 { θ ′ ∣ ∥ θ ′ − θ q ∥ < δ θ q } \{\theta' \mid\lVert\theta' - \theta_q\rVert< \delta_{\theta_q}\} { θ ′ ∣ ∥ θ ′ − θ q ∥ < δ θ q } K K K lim sup m Δ m ≤ ϵ \limsup_m \Delta_m \le\epsilon lim sup m Δ m ≤ ϵ ϵ ↓ 0 \epsilon\downarrow0 ϵ ↓ 0
Lemma 4 (Inverse continuity at θ 0 \theta_0 θ 0 Under Assumptions 1, 2 and 3 , if θ k ∈ K \theta_k \in K θ k ∈ K
W 1 ( P θ 0 , P θ k ) → 0 , then ∥ θ k − θ 0 ∥ → 0. W_1(P_{\theta_0},P_{\theta_k}) \to0,\qquad\text{then}\qquad\lVert\theta_k-\theta_0\rVert\to0. W 1 ( P θ 0 , P θ k ) → 0 , then ∥ θ k − θ 0 ∥ → 0. Proof of Lemma 4. Since K K K ( θ k ) (\theta_k) ( θ k ) θ k i → θ ∗ ∈ K \theta_{k_i} \to\theta^* \in K θ k i → θ ∗ ∈ K Lemma 1 , W 1 ( P θ k i , P θ ∗ ) → 0 W_1(P_{\theta_{k_i}},P_{\theta^*}) \to0 W 1 ( P θ k i , P θ ∗ ) → 0 W 1 ( P θ 0 , P θ ∗ ) ≤ W 1 ( P θ 0 , P θ k i ) + W 1 ( P θ k i , P θ ∗ ) → 0 W_1(P_{\theta_0},P_{\theta^*}) \le W_1(P_{\theta_0},P_{\theta_{k_i}}) + W_1(P_{\theta_{k_i}},P_{\theta^*}) \to0 W 1 ( P θ 0 , P θ ∗ ) ≤ W 1 ( P θ 0 , P θ k i ) + W 1 ( P θ k i , P θ ∗ ) → 0 θ ∗ = θ 0 \theta^* = \theta_0 θ ∗ = θ 0 θ k → θ 0 \theta_k \to\theta_0 θ k → θ 0
Theorem 1 (Strong consistency of θ ~ n 0 \widetilde{\theta}_{n_0} θ n 0 Suppose Assumptions 1–3 hold, let the synthetic sample size satisfy m = m ( n 0 ) → ∞ m=m(n_0)\to\infty m = m ( n 0 ) → ∞ θ ~ n 0 \widetilde{\theta}_{n_0} θ n 0 ϵ n 0 \epsilon_{n_0} ϵ n 0 (21) with ϵ n 0 ↓ 0 \epsilon_{n_0}\downarrow0 ϵ n 0 ↓ 0
∥ θ ~ n 0 − θ 0 ∥ ⟶ 0 almost surely as n 0 → ∞ . \lVert\widetilde{\theta}_{n_0}-\theta_0\rVert\longrightarrow0 \qquad\text{almost surely as } n_0\to\infty. ∥ θ n 0 − θ 0 ∥ ⟶ 0 almost surely as n 0 → ∞. Proof of Theorem 1. Work on the intersection of the almost sure events of Proposition 1 and Lemma 3 , and write Δ : = Δ m ( n 0 ) \Delta:= \Delta_{m(n_0)} Δ := Δ m ( n 0 ) Lemma 2 , for every θ ∈ K \theta\in K θ ∈ K ∣ W 1 ( P ^ ξ 0 , P ^ θ , m ) − W 1 ( P ^ ξ 0 , P θ ) ∣ ≤ Δ |W_1(\widehat{P}_{\xi_0},\widehat{P}_{\theta,m})-W_1(\widehat{P}_{\xi_0},P_\theta)| \le\Delta ∣ W 1 ( P ξ 0 , P θ , m ) − W 1 ( P ξ 0 , P θ ) ∣ ≤ Δ θ 0 , θ ~ n 0 ∈ K \theta_0,\widetilde{\theta}_{n_0}\in K θ 0 , θ n 0 ∈ K
W 1 ( P θ 0 , P θ ~ n 0 ) ≤ W 1 ( P θ 0 , P ^ ξ 0 ) + W 1 ( P ^ ξ 0 , P ^ θ ~ n 0 , m ) + Δ ≤ W 1 ( P θ 0 , P ^ ξ 0 ) + W 1 ( P ^ ξ 0 , P ^ θ 0 , m ) + ϵ n 0 + Δ W_1(P_{\theta_0},P_{\widetilde{\theta}_{n_0}}) \le W_1(P_{\theta_0},\widehat{P}_{\xi_0}) + W_1(\widehat{P}_{\xi_0},\widehat{P}_{\widetilde{\theta}_{n_0},m}) + \Delta\le W_1(P_{\theta_0},\widehat{P}_{\xi_0}) + W_1(\widehat{P}_{\xi_0},\widehat{P}_{\theta_0,m}) + \epsilon_{n_0} + \Delta W 1 ( P θ 0 , P θ n 0 ) ≤ W 1 ( P θ 0 , P ξ 0 ) + W 1 ( P ξ 0 , P θ n 0 , m ) + Δ ≤ W 1 ( P θ 0 , P ξ 0 ) + W 1 ( P ξ 0 , P θ 0 , m ) + ϵ n 0 + Δ ≤ 2 W 1 ( P θ 0 , P ^ ξ 0 ) + ϵ n 0 + 2 Δ ⟶ 0. \le2W_1(P_{\theta_0},\widehat{P}_{\xi_0})+\epsilon_{n_0}+2\Delta\longrightarrow0. ≤ 2 W 1 ( P θ 0 , P ξ 0 ) + ϵ n 0 + 2Δ ⟶ 0. Lemma 4 concludes ∥ θ ~ n 0 − θ 0 ∥ ⟶ 0 \lVert\widetilde{\theta}_{n_0}-\theta_0\rVert\longrightarrow0 ∥ θ n 0 − θ 0 ∥ ⟶ 0 θ ~ n 0 \widetilde{\theta}_{n_0} θ n 0 ( ϵ n 0 + 2 Δ ) (\epsilon_{n_0}+2\Delta) ( ϵ n 0 + 2Δ ) θ ↦ W 1 ( P ^ ξ 0 , P θ ) \theta\mapsto W_1(\widehat{P}_{\xi_0},P_\theta) θ ↦ W 1 ( P ξ 0 , P θ ) K K K ϵ n \epsilon_n ϵ n
Proposition 2 (Qualitative robustness at the model). Let Assumptions 1–3 hold, let Q k ∈ P 1 ( S ) Q_k\in\mathcal{P}_1(S) Q k ∈ P 1 ( S ) W 1 ( Q k , P θ 0 ) → 0 W_1(Q_k,P_{\theta_0})\to0 W 1 ( Q k , P θ 0 ) → 0 θ k ∈ K \theta_k\in K θ k ∈ K W 1 ( Q k , P θ k ) ≤ inf θ ∈ K W 1 ( Q k , P θ ) + ϵ k W_1(Q_k,P_{\theta_k})\le\inf_{\theta\in K}W_1(Q_k,P_\theta)+\epsilon_k W 1 ( Q k , P θ k ) ≤ inf θ ∈ K W 1 ( Q k , P θ ) + ϵ k ϵ k → 0 \epsilon_k\to0 ϵ k → 0 θ k → θ 0 \theta_k\to\theta_0 θ k → θ 0 P θ P_\theta P θ P ^ θ , m \widehat{P}_{\theta,m} P θ , m θ k \theta_k θ k m = m ( k ) → ∞ m=m(k)\to\infty m = m ( k ) → ∞
Proof of Proposition 2. Since θ 0 ∈ K \theta_0 \in K θ 0 ∈ K
W 1 ( P θ 0 , P θ k ) ≤ W 1 ( P θ 0 , Q k ) + W 1 ( Q k , P θ k ) ≤ W 1 ( P θ 0 , Q k ) + W 1 ( Q k , P θ 0 ) + ε k → 0 , W_1(P_{\theta_0}, P_{\theta_k}) \le W_1(P_{\theta_0}, Q_k) + W_1(Q_k, P_{\theta_k}) \le W_1(P_{\theta_0}, Q_k) + W_1(Q_k, P_{\theta_0}) + \varepsilon_k \to0, W 1 ( P θ 0 , P θ k ) ≤ W 1 ( P θ 0 , Q k ) + W 1 ( Q k , P θ k ) ≤ W 1 ( P θ 0 , Q k ) + W 1 ( Q k , P θ 0 ) + ε k → 0 , and Lemma 4 gives θ k → θ 0 \theta_k \to\theta_0 θ k → θ 0 ε k \varepsilon_k ε k θ ↦ W 1 ( Q k , P ~ θ , m ) \theta\mapsto W_1(Q_k, \widetilde{P}_{\theta,m}) θ ↦ W 1 ( Q k , P θ , m ) K K K ( ε k + 2 Δ m ( k ) ) (\varepsilon_k + 2\Delta_{m(k)}) ( ε k + 2 Δ m ( k ) ) θ ↦ W 1 ( Q k , P θ ) \theta\mapsto W_1(Q_k, P_\theta) θ ↦ W 1 ( Q k , P θ ) Δ m \Delta_m Δ m
Recourse stage
Lemma 5 (Conditional concentration of the recourse weights). Let the data be i.i.d. from P θ 0 P_{\theta_0} P θ 0 p ℓ : = P θ 0 ( V ℓ ( θ ~ n 0 ) ) p_\ell:= P_{\theta_0}(V_\ell(\widetilde{\theta}_{n_0})) p ℓ := P θ 0 ( V ℓ ( θ n 0 )) ℓ ≤ L ( θ ~ n 0 ) \ell\le L(\widetilde{\theta}_{n_0}) ℓ ≤ L ( θ n 0 )
max ℓ ≤ L ( θ ~ n 0 ) ∣ w ~ ℓ − p ℓ ∣ ⟶ 0 ( n → ∞ ) . \max_{\ell\le L(\widetilde{\theta}_{n_0})} \left|\widetilde{w}_\ell- p_\ell\right| \longrightarrow0 \qquad(n \to\infty). ℓ ≤ L ( θ n 0 ) max ∣ w ℓ − p ℓ ∣ ⟶ 0 ( n → ∞ ) . Proof of Lemma 5. Given F 0 \mathcal{F}_0 F 0 θ ~ n 0 \widetilde{\theta}_{n_0} θ n 0
V 1 ( θ ~ n 0 ) , … , V L ( θ ~ n 0 ) ( θ ~ n 0 ) V_1(\widetilde{\theta}_{n_0}), \ldots, V_{L(\widetilde{\theta}_{n_0})}(\widetilde{\theta}_{n_0}) V 1 ( θ n 0 ) , … , V L ( θ n 0 ) ( θ n 0 ) are fixed Borel sets partitioning S S S ξ 1 \xi_1 ξ 1 P θ 0 P_{\theta_0} P θ 0 F 0 \mathcal{F}_0 F 0 n 1 w ~ ℓ ∣ F 0 ∼ Binomial ( n 1 , p ℓ ) n_1\widetilde{w}_\ell\mid\mathcal{F}_0 \sim\operatorname{Binomial}(n_1,p_\ell) n 1 w ℓ ∣ F 0 ∼ Binomial ( n 1 , p ℓ ) (27) . Hoeffding’s inequality [22 ] with a union bound over ℓ ≤ L ( θ ~ n 0 ) ≤ N 1 \ell\le L(\widetilde{\theta}_{n_0}) \le N_1 ℓ ≤ L ( θ n 0 ) ≤ N 1 ϵ > 0 \epsilon> 0 ϵ > 0
P ( max ℓ ≤ L ( θ ~ n 0 ) ∣ w ~ ℓ − p ℓ ∣ > ϵ | F 0 ) ≤ ∑ ℓ ≤ L ( θ ~ n 0 ) P ( ∣ n 1 w ~ ℓ − n 1 p ℓ ∣ > n 1 ϵ | F 0 ) ≤ 2 N 1 e − 2 n 1 ϵ 2 . \mathbb{P}\left(\max_{\ell\le L(\widetilde{\theta}_{n_0})} \left|\widetilde{w}_\ell- p_\ell\right| > \epsilon\,\middle|\, \mathcal{F}_0\right) \le\sum_{\ell\le L(\widetilde{\theta}_{n_0})} \mathbb{P}\left(\left|n_1\widetilde{w}_\ell- n_1p_\ell\right| > n_1\epsilon\,\middle|\, \mathcal{F}_0\right) \le2N_1e^{-2n_1\epsilon^2}. P ( ℓ ≤ L ( θ n 0 ) max ∣ w ℓ − p ℓ ∣ > ϵ F 0 ) ≤ ℓ ≤ L ( θ n 0 ) ∑ P ( ∣ n 1 w ℓ − n 1 p ℓ ∣ > n 1 ϵ ∣ F 0 ) ≤ 2 N 1 e − 2 n 1 ϵ 2 . Under regime (10),
n 0 ∧ n 1 → ∞ ; n 1 log ( n ) → ∞ . n_0 \wedge n_1 \to\infty; \qquad\frac{n_1}{\log(n)} \to\infty. n 0 ∧ n 1 → ∞ ; log ( n ) n 1 → ∞. Hence, n 1 ( n ) ≥ ( 1 / ϵ 2 ) log n n_1(n) \ge(1/\epsilon^2)\log n n 1 ( n ) ≥ ( 1/ ϵ 2 ) log n n n n
∑ n 2 N 1 e − 2 n 1 ( n ) ϵ 2 ≤ C + ∑ n 2 N 1 n − 2 < ∞ . \sum_n 2N_1e^{-2n_1(n)\epsilon^2} \le C + \sum_n 2N_1n^{-2} < \infty. n ∑ 2 N 1 e − 2 n 1 ( n ) ϵ 2 ≤ C + n ∑ 2 N 1 n − 2 < ∞. The Borel–Cantelli lemma gives lim sup n max ℓ ∣ w ~ ℓ − p ℓ ∣ ≤ ϵ \limsup_n \max_\ell|\widetilde{w}_\ell-p_\ell| \le\epsilon lim sup n max ℓ ∣ w ℓ − p ℓ ∣ ≤ ϵ ϵ = 1 / q \epsilon= 1/q ϵ = 1/ q q ∈ N q \in\mathbb{N} q ∈ N
Lemma 6 (Sufficient conditions for continuity of the cell masses). Let Assumptions 1 and 4 hold and let θ 0 ∉ D \theta_0 \notin D θ 0 ∈ / D θ 0 \theta_0 θ 0 ℓ ≤ L ( θ 0 ) \ell\le L(\theta_0) ℓ ≤ L ( θ 0 ) θ ↦ P θ 0 ( V ℓ ( θ ) ) \theta\mapsto P_{\theta_0}(V_\ell(\theta)) θ ↦ P θ 0 ( V ℓ ( θ )) θ ↦ P θ ( V ℓ ( θ ) ) \theta\mapsto P_\theta(V_\ell(\theta)) θ ↦ P θ ( V ℓ ( θ )) θ 0 \theta_0 θ 0
Proof of Lemma 6. Since θ 0 ∉ D \theta_0 \notin D θ 0 ∈ / D U U U θ 0 \theta_0 θ 0
I ℓ ( θ ) = I ℓ ( θ 0 ) , ∀ θ ∈ U , ℓ ≤ L ( θ 0 ) . I_\ell(\theta) = I_\ell(\theta_0), \qquad\forall\theta\in U,\ \ell\le L(\theta_0). I ℓ ( θ ) = I ℓ ( θ 0 ) , ∀ θ ∈ U , ℓ ≤ L ( θ 0 ) . Let θ k → θ 0 \theta_k \to\theta_0 θ k → θ 0 θ k ∈ U \theta_k \in U θ k ∈ U
z ℓ ( θ k ) = G ( θ k , x ˉ j ) → z ℓ ( θ 0 ) , ∀ j ∈ I ℓ ( θ 0 ) , ℓ ≤ L ( θ 0 ) . z_\ell(\theta_k) = G(\theta_k,\bar{x}_j) \to z_\ell(\theta_0), \qquad\forall j \in I_\ell(\theta_0),\ \ell\le L(\theta_0). z ℓ ( θ k ) = G ( θ k , x ˉ j ) → z ℓ ( θ 0 ) , ∀ j ∈ I ℓ ( θ 0 ) , ℓ ≤ L ( θ 0 ) . Under (5a). Let y y y
y ∉ { d ( y , z ℓ ( θ 0 ) ) = d ( y , z ℓ ′ ( θ 0 ) ) , ∀ ℓ ′ ≠ ℓ } . y \notin\left\{d(y,z_\ell(\theta_0)) = d(y,z_{\ell'}(\theta_0)),\ \forall\ell' \ne\ell\right\}. y ∈ / { d ( y , z ℓ ( θ 0 )) = d ( y , z ℓ ′ ( θ 0 )) , ∀ ℓ ′ = ℓ } . The inequalities d ( y , z ℓ ( θ 0 ) ) ≤ d ( y , z ℓ ′ ( θ 0 ) ) d(y,z_{\ell}(\theta_0)) \leq d(y,z_{\ell'}(\theta_0)) d ( y , z ℓ ( θ 0 )) ≤ d ( y , z ℓ ′ ( θ 0 )) z 1 : L ( θ k ) z_{1:L}(\theta_k) z 1 : L ( θ k ) k k k
1 V ℓ ( θ k ) ( y ) → 1 V ℓ ( θ 0 ) ( y ) . \mathbf{1}_{V_{\ell}(\theta_k)}(y) \to\mathbf{1}_{V_{\ell}(\theta_0)}(y). 1 V ℓ ( θ k ) ( y ) → 1 V ℓ ( θ 0 ) ( y ) . By (5a) this holds P θ 0 P_{\theta_0} P θ 0
P θ 0 ( V ℓ ( θ k ) ) → P θ 0 ( V ℓ ( θ 0 ) ) . P_{\theta_0}(V_{\ell}(\theta_k)) \to P_{\theta_0}(V_{\ell}(\theta_0)). P θ 0 ( V ℓ ( θ k )) → P θ 0 ( V ℓ ( θ 0 )) . For the second map write P θ k ( V ℓ ( θ k ) ) = ν { x : G ( θ k , x ) ∈ V ℓ ( θ k ) } P_{\theta_k}(V_{\ell}(\theta_k))=\nu\{x:G(\theta_k,x)\in V_{\ell}(\theta_k)\} P θ k ( V ℓ ( θ k )) = ν { x : G ( θ k , x ) ∈ V ℓ ( θ k )} Lemma 1 ,
∫ d ( G ( θ k , x ) , G ( θ 0 , x ) ) ν ( d x ) → 0 , \int d(G(\theta_k,x),G(\theta_0,x))\,\nu(dx)\to0, ∫ d ( G ( θ k , x ) , G ( θ 0 , x )) ν ( d x ) → 0 , so every subsequence has a further subsequence along which G ( θ k , x ) → G ( θ 0 , x ) G(\theta_k,x)\to G(\theta_0,x) G ( θ k , x ) → G ( θ 0 , x ) ν \nu ν x x x
ν ( { x ∣ G ( θ 0 , x ) off the bisectors of z 1 : L ( θ 0 ) } ) = 1. \nu\left(\{x\mid G(\theta_0,x)\text{ off the bisectors of }z_{1:L}(\theta_0)\}\right)=1. ν ( { x ∣ G ( θ 0 , x ) off the bisectors of z 1 : L ( θ 0 )} ) = 1. The strict inequalities above are preserved for the pair ( G ( θ k , x ) , z 1 : L ( θ k ) ) (G(\theta_k,x),z_{1:L}(\theta_k)) ( G ( θ k , x ) , z 1 : L ( θ k ))
1 { G ( θ k , x ) ∈ V ℓ ( θ k ) } → 1 { G ( θ 0 , x ) ∈ V ℓ ( θ 0 ) } . \mathbf{1}\{G(\theta_k,x)\in V_{\ell}(\theta_k)\}\to\mathbf{1}\{G(\theta_0,x)\in V_{\ell}(\theta_0)\}. 1 { G ( θ k , x ) ∈ V ℓ ( θ k )} → 1 { G ( θ 0 , x ) ∈ V ℓ ( θ 0 )} . Dominated convergence along the sub-subsequence, and the subsequence principle, give
P θ k ( V ℓ ( θ k ) ) → P θ 0 ( V ℓ ( θ 0 ) ) . P_{\theta_k}(V_{\ell}(\theta_k))\to P_{\theta_0}(V_{\ell}(\theta_0)). P θ k ( V ℓ ( θ k )) → P θ 0 ( V ℓ ( θ 0 )) . Under (5b). For large k k k θ k ∈ U ∩ U ′ \theta_k\in U\cap U' θ k ∈ U ∩ U ′
V ℓ ( θ k ) = V ℓ ( θ 0 ) = : V ℓ , V_{\ell}(\theta_k)=V_{\ell}(\theta_0)=:V_{\ell}, V ℓ ( θ k ) = V ℓ ( θ 0 ) =: V ℓ , are fixed sets. The first map is therefore eventually constant. For the second, write
p θ ( y ) : = P θ ( { y } ) , ∀ y ∈ Y . p_{\theta}(y):=P_{\theta}(\{y\}),\qquad\forall y\in\mathcal{Y}. p θ ( y ) := P θ ({ y }) , ∀ y ∈ Y . The function f y ( z ) : = max { 0 , 1 − 2 d ( z , y ) / η } f_y(z):=\max\{0,1-2d(z,y)/\eta\} f y ( z ) := max { 0 , 1 − 2 d ( z , y ) / η } y y y Y \mathcal{Y} Y p θ ( y ) = ∫ f y d P θ p_{\theta}(y)=\int f_y\,\mathrm{d}P_{\theta} p θ ( y ) = ∫ f y d P θ θ ∈ U ′ \theta\in U' θ ∈ U ′ Lemma 1 , P θ k ⇒ P θ 0 P_{\theta_k}\Rightarrow P_{\theta_0} P θ k ⇒ P θ 0 p θ k ( y ) → p θ 0 ( y ) p_{\theta_k}(y)\to p_{\theta_0}(y) p θ k ( y ) → p θ 0 ( y ) y ∈ Y y\in\mathcal{Y} y ∈ Y ∑ y ∈ Y p θ ( y ) = 1 \sum_{y\in\mathcal{Y}}p_{\theta}(y)=1 ∑ y ∈ Y p θ ( y ) = 1 θ ∈ U ′ \theta\in U' θ ∈ U ′ [35 ] ) applied on Y \mathcal{Y} Y ∑ y ∈ Y ∣ p θ k ( y ) − p θ 0 ( y ) ∣ → 0 \sum_{y\in\mathcal{Y}}|p_{\theta_k}(y)-p_{\theta_0}(y)|\to0 ∑ y ∈ Y ∣ p θ k ( y ) − p θ 0 ( y ) ∣ → 0
∣ P θ k ( V ℓ ) − P θ 0 ( V ℓ ) ∣ ≤ ∑ y ∈ Y ∣ p θ k ( y ) − p θ 0 ( y ) ∣ ⟶ 0. \left|P_{\theta_k}(V_{\ell})-P_{\theta_0}(V_{\ell})\right|
\leq\sum_{y\in\mathcal{Y}}|p_{\theta_k}(y)-p_{\theta_0}(y)|\longrightarrow0. ∣ P θ k ( V ℓ ) − P θ 0 ( V ℓ ) ∣ ≤ y ∈ Y ∑ ∣ p θ k ( y ) − p θ 0 ( y ) ∣ ⟶ 0. Theorem 2 (Recourse weight consistency). Let Assumptions 1–5 hold for the fixed mesh { x ˉ j } j ≤ N 1 \{\bar{x}_j\}_{j\leq N_1} { x ˉ j } j ≤ N 1 θ ~ n 0 \tilde{\theta}_{n_0} θ ~ n 0 (10) , L ( θ ~ n 0 ) = L ( θ 0 ) = : L L(\tilde{\theta}_{n_0})=L(\theta_0)=:L L ( θ ~ n 0 ) = L ( θ 0 ) =: L n 0 n_0 n 0 ℓ ≤ L \ell\leq L ℓ ≤ L
w ~ ℓ ⟶ w ℓ v o r ( θ 0 ) , w ℓ v o r ( θ ~ n 0 ) ⟶ w ℓ v o r ( θ 0 ) , \tilde{w}_{\ell}\longrightarrow w_{\ell}^{\mathrm{vor}}(\theta_0),\qquad
w_{\ell}^{\mathrm{vor}}(\tilde{\theta}_{n_0})\longrightarrow w_{\ell}^{\mathrm{vor}}(\theta_0), w ~ ℓ ⟶ w ℓ vor ( θ 0 ) , w ℓ vor ( θ ~ n 0 ) ⟶ w ℓ vor ( θ 0 ) , and
W 1 ( ∑ ℓ w ~ ℓ δ z ℓ ( θ ~ n 0 ) , ∑ ℓ w ℓ v o r ( θ 0 ) δ z ℓ ( θ 0 ) ) ⟶ 0. W_1\left(\sum_{\ell}\tilde{w}_{\ell}\delta_{z_{\ell}(\tilde{\theta}_{n_0})},\sum_{\ell}w_{\ell}^{\mathrm{vor}}(\theta_0)\delta_{z_{\ell}(\theta_0)}\right)\longrightarrow0. W 1 ( ℓ ∑ w ~ ℓ δ z ℓ ( θ ~ n 0 ) , ℓ ∑ w ℓ vor ( θ 0 ) δ z ℓ ( θ 0 ) ) ⟶ 0. Proof of Theorem 2. By Assumption 4, θ 0 ∉ D \theta_0 \notin D θ 0 ∈ / D U U U θ 0 \theta_0 θ 0 θ ~ n 0 ∈ U \tilde{\theta}_{n_0} \in U θ ~ n 0 ∈ U n 0 n_0 n 0 L ( θ ~ n 0 ) = L L(\tilde{\theta}_{n_0})=L L ( θ ~ n 0 ) = L θ ~ n 0 \tilde{\theta}_{n_0} θ ~ n 0 θ 0 \theta_0 θ 0 z ℓ ( θ ~ n 0 ) = G ( θ ~ n 0 , x ˉ j ) z_\ell(\tilde{\theta}_{n_0})=G(\tilde{\theta}_{n_0},\bar{x}_j) z ℓ ( θ ~ n 0 ) = G ( θ ~ n 0 , x ˉ j ) j ∈ I ℓ ( θ 0 ) j\in I_\ell(\theta_0) j ∈ I ℓ ( θ 0 ) z ℓ ( θ ~ n 0 ) → z ℓ ( θ 0 ) z_\ell(\tilde{\theta}_{n_0})\to z_\ell(\theta_0) z ℓ ( θ ~ n 0 ) → z ℓ ( θ 0 )
Fix ℓ \ell ℓ
∣ w ~ ℓ − w ℓ vor ( θ 0 ) ∣ ≤ ∣ w ~ ℓ − P θ 0 ( V ℓ ( θ ~ n 0 ) ) ∣ + ∣ P θ 0 ( V ℓ ( θ ~ n 0 ) ) − P θ 0 ( V ℓ ( θ 0 ) ) ∣ . \left|\tilde{w}_\ell-w_\ell^{\operatorname{vor}}(\theta_0)\right|
\leq\left|\tilde{w}_\ell-P_{\theta_0}(V_\ell(\tilde{\theta}_{n_0}))\right|+\left|P_{\theta_0}(V_\ell(\tilde{\theta}_{n_0}))-P_{\theta_0}(V_\ell(\theta_0))\right|. ∣ w ~ ℓ − w ℓ vor ( θ 0 ) ∣ ≤ w ~ ℓ − P θ 0 ( V ℓ ( θ ~ n 0 )) + P θ 0 ( V ℓ ( θ ~ n 0 )) − P θ 0 ( V ℓ ( θ 0 )) . The first term tends to zero almost surely by Lemma 5 . The second tends to zero because θ ~ n 0 → θ 0 \tilde{\theta}_{n_0}\to\theta_0 θ ~ n 0 → θ 0 θ ↦ P θ 0 ( V ℓ ( θ ) ) \theta\mapsto P_{\theta_0}(V_\ell(\theta)) θ ↦ P θ 0 ( V ℓ ( θ )) θ 0 \theta_0 θ 0
w ℓ vor ( θ ~ n 0 ) = P θ ~ n 0 ( V ℓ ( θ ~ n 0 ) ) → P θ 0 ( V ℓ ( θ 0 ) ) = w ℓ vor ( θ 0 ) w_\ell^{\operatorname{vor}}(\tilde{\theta}_{n_0})=P_{\tilde{\theta}_{n_0}}(V_\ell(\tilde{\theta}_{n_0}))\to P_{\theta_0}(V_\ell(\theta_0))=w_\ell^{\operatorname{vor}}(\theta_0) w ℓ vor ( θ ~ n 0 ) = P θ ~ n 0 ( V ℓ ( θ ~ n 0 )) → P θ 0 ( V ℓ ( θ 0 )) = w ℓ vor ( θ 0 ) by the continuity of the second map in Assumption 5.
By the triangle inequality, coupling atom ℓ \ell ℓ ℓ \ell ℓ W 1 W_1 W 1
W 1 ( ∑ ℓ w ~ ℓ δ z ℓ ( θ ~ n 0 ) , ∑ ℓ w ℓ vor δ z ℓ ( θ 0 ) ) ≤ max ℓ d ( z ℓ ( θ ~ n 0 ) , z ℓ ( θ 0 ) ) + diam { z 1 : L ( θ 0 ) } ∑ ℓ ∣ w ~ ℓ − w ℓ vor ( θ 0 ) ∣ → 0. W_1\left(\sum_\ell\tilde{w}_\ell\delta_{z_\ell(\tilde{\theta}_{n_0})},\sum_\ell w_\ell^{\operatorname{vor}}\delta_{z_\ell(\theta_0)}\right)
\leq\max_\ell d(z_\ell(\tilde{\theta}_{n_0}),z_\ell(\theta_0))+\operatorname{diam}\{z_{1:L}(\theta_0)\}\sum_\ell\left|\tilde{w}_\ell-w_\ell^{\operatorname{vor}}(\theta_0)\right|\to0. W 1 ( ℓ ∑ w ~ ℓ δ z ℓ ( θ ~ n 0 ) , ℓ ∑ w ℓ vor δ z ℓ ( θ 0 ) ) ≤ ℓ max d ( z ℓ ( θ ~ n 0 ) , z ℓ ( θ 0 )) + diam { z 1 : L ( θ 0 )} ℓ ∑ ∣ w ~ ℓ − w ℓ vor ( θ 0 ) ∣ → 0. Verification of the assumptions for the examples
This appendix verifies Assumptions 1–5 for three families; Table 1 summarizes the results.
Example 2 (Location–scale family). Let S = R S=\mathbb{R} S = R Θ = R × ( 0 , ∞ ) \Theta=\mathbb{R}\times(0,\infty) Θ = R × ( 0 , ∞ ) θ = ( μ , σ ) \theta=(\mu,\sigma) θ = ( μ , σ )
Y = μ + σ Z , Y=\mu+\sigma Z, Y = μ + σ Z , where Z ∈ L 1 Z\in L^1 Z ∈ L 1 X = R \mathcal{X}=\mathbb{R} X = R ν \nu ν Z Z Z G ( θ , x ) = μ + σ x G(\theta,x)=\mu+\sigma x G ( θ , x ) = μ + σ x ∥ θ ′ − θ ∥ < δ \|\theta'-\theta\|<\delta ∥ θ ′ − θ ∥ < δ
∣ G ( θ ′ , x ) − G ( θ , x ) ∣ ≤ δ ( 1 + ∣ x ∣ ) , |G(\theta',x)-G(\theta,x)|\leq\delta(1+|x|), ∣ G ( θ ′ , x ) − G ( θ , x ) ∣ ≤ δ ( 1 + ∣ x ∣ ) , so ω θ ( δ ) ≤ δ ( 1 + E ∣ Z ∣ ) \omega_\theta(\delta)\leq\delta(1+\mathbb{E}|Z|) ω θ ( δ ) ≤ δ ( 1 + E ∣ Z ∣ ) K ′ K' K ′ c K ′ ( 1 + ∣ x ∣ ) c_{K'}(1+|x|) c K ′ ( 1 + ∣ x ∣ ) ν \nu ν μ = E Y \mu=\mathbb{E}Y μ = E Y σ = E ∣ Y − μ ∣ / E ∣ Z ∣ \sigma=\mathbb{E}|Y-\mu|/\mathbb{E}|Z| σ = E ∣ Y − μ ∣/ E ∣ Z ∣ G ( θ , ⋅ ) G(\theta,\cdot) G ( θ , ⋅ ) θ \theta θ θ \theta θ D = ∅ D=\varnothing D = ∅ P θ 0 P_{\theta_0} P θ 0
Example 3 (Poisson counts). Let S = R S=\mathbb{R} S = R Θ = ( 0 , ∞ ) \Theta=(0,\infty) Θ = ( 0 , ∞ ) [ 0 , ∞ ) [0,\infty) [ 0 , ∞ ) x = ( P 1 , P 2 , … ) x=(P_1,P_2,\ldots) x = ( P 1 , P 2 , … ) 0 < P 1 < P 2 < ⋯ 0<P_1<P_2<\cdots 0 < P 1 < P 2 < ⋯ ν \nu ν λ \lambda λ
G ( λ , x ) = N x ( λ ) : = # { k : P k ≤ λ } , G(\lambda,x)=N_x(\lambda):=\#\{k:P_k\leq\lambda\}, G ( λ , x ) = N x ( λ ) := # { k : P k ≤ λ } , so that P λ = Poisson ( λ ) P_\lambda=\operatorname{Poisson}(\lambda) P λ = Poisson ( λ ) G ( λ , ⋅ ) G(\lambda,\cdot) G ( λ , ⋅ ) F λ − 1 ( U ) F_\lambda^{-1}(U) F λ − 1 ( U ) U U U λ ↦ N x ( λ ) \lambda\mapsto N_x(\lambda) λ ↦ N x ( λ ) δ < λ \delta<\lambda δ < λ
ω λ ( δ ) ≤ E [ N ( λ + δ ) − N ( λ − δ ) ] = 2 δ , \omega_\lambda(\delta)\leq\mathbb{E}[N(\lambda+\delta)-N(\lambda-\delta)]=2\delta, ω λ ( δ ) ≤ E [ N ( λ + δ ) − N ( λ − δ )] = 2 δ , and on K ′ ⊆ [ a , b ] K' \subseteq[a,b] K ′ ⊆ [ a , b ] N x ( b ) N_x(b) N x ( b ) E N ( b ) = b < ∞ \mathbb{E}N(b)=b<\infty E N ( b ) = b < ∞ W 1 ( P λ , P λ ′ ) ≥ ∣ λ − λ ′ ∣ W_1(P_\lambda,P_{\lambda'})\geq|\lambda-\lambda'| W 1 ( P λ , P λ ′ ) ≥ ∣ λ − λ ′ ∣ y ↦ y y\mapsto y y ↦ y
For a fixed mesh point x ˉ j \bar{x}_j x ˉ j λ ↦ G ( λ , x ˉ j ) \lambda\mapsto G(\lambda,\bar{x}_j) λ ↦ G ( λ , x ˉ j ) x ˉ j \bar{x}_j x ˉ j
D ⊆ ⋃ j ≤ N 1 { P 1 ( x ˉ j ) , P 2 ( x ˉ j ) , … } . D\subseteq\bigcup_{j\leq N_1}\{P_1(\bar{x}_j),P_2(\bar{x}_j),\ldots\}. D ⊆ j ≤ N 1 ⋃ { P 1 ( x ˉ j ) , P 2 ( x ˉ j ) , … } . This set is Lebesgue-null, hence Π \Pi Π Π \Pi Π D D D P λ ( N 0 ) = 1 P_\lambda(\mathcal{N}_0)=1 P λ ( N 0 ) = 1 λ \lambda λ Y = N 0 \mathcal{Y}=\mathcal{N}_0 Y = N 0 η = 1 \eta=1 η = 1
Example 4 (Ornstein–Uhlenbeck process). Let X = C 0 ( [ 0 , T ] ) \mathcal{X}=C_0([0,T]) X = C 0 ([ 0 , T ]) ν \nu ν S = L 2 ( [ 0 , T ] ) S=L^2([0,T]) S = L 2 ([ 0 , T ]) θ = ( a , σ ) ∈ Θ = ( 0 , ∞ ) 2 \theta=(a,\sigma)\in\Theta=(0,\infty)^2 θ = ( a , σ ) ∈ Θ = ( 0 , ∞ ) 2 x ∈ X x\in\mathcal{X} x ∈ X
G ( θ , x ) = σ ( x − a K a x ) , ( K a x ) ( t ) : = ∫ 0 t e − a ( t − u ) x ( u ) d u . G(\theta,x)=\sigma(x-aK_a x),\qquad(K_a x)(t):=\int_0^t e^{-a(t-u)}x(u)\,\mathrm{d}u. G ( θ , x ) = σ ( x − a K a x ) , ( K a x ) ( t ) := ∫ 0 t e − a ( t − u ) x ( u ) d u . Integration by parts gives G ( θ , W ) ( t ) = σ ∫ 0 t e − a ( t − u ) d W ( u ) G(\theta,W)(t)=\sigma\int_0^t e^{-a(t-u)}\,\mathrm{d}W(u) G ( θ , W ) ( t ) = σ ∫ 0 t e − a ( t − u ) d W ( u ) P θ P_\theta P θ
σ 2 2 a ( e − a ∣ t − u ∣ − e − a ( t + u ) ) . \frac{\sigma^2}{2a}\left(e^{-a|t-u|}-e^{-a(t+u)}\right). 2 a σ 2 ( e − a ∣ t − u ∣ − e − a ( t + u ) ) . Distinct θ \theta θ L 2 ( [ 0 , T ] ) L^2([0,T]) L 2 ([ 0 , T ]) K a K_a K a T T T ∥ K a − K a ′ ∥ ≤ T 2 ∣ a − a ′ ∣ \lVert K_a-K_{a'}\rVert\leq T^2|a-a'| ∥ K a − K a ′ ∥ ≤ T 2 ∣ a − a ′ ∣ K ′ ⊂ Θ K'\subset\Theta K ′ ⊂ Θ c K ′ c_{K'} c K ′
∥ G ( θ , x ) − G ( θ ′ , x ) ∥ L 2 ≤ c K ′ ∥ θ − θ ′ ∥ ∥ x ∥ L 2 and ∥ G ( θ , x ) ∥ L 2 ≤ c K ′ ∥ x ∥ L 2 . \lVert G(\theta,x)-G(\theta',x)\rVert_{L^2}\leq c_{K'}\lVert\theta-\theta'\rVert\lVert x\rVert_{L^2}\quad\text{and}\quad\lVert G(\theta,x)\rVert_{L^2}\leq c_{K'}\lVert x\rVert_{L^2}. ∥ G ( θ , x ) − G ( θ ′ , x ) ∥ L 2 ≤ c K ′ ∥ θ − θ ′ ∥ ∥ x ∥ L 2 and ∥ G ( θ , x ) ∥ L 2 ≤ c K ′ ∥ x ∥ L 2 . Since ∥ x ∥ L 2 ≤ T ∥ x ∥ ∞ \lVert x\rVert_{L^2}\leq\sqrt{T}\lVert x\rVert_\infty ∥ x ∥ L 2 ≤ T ∥ x ∥ ∞ E ∥ W ∥ ∞ < ∞ \mathbb{E}\lVert W\rVert_\infty<\infty E ∥ W ∥ ∞ < ∞ ω θ ( δ ) ≤ c δ \omega_\theta(\delta)\leq c\delta ω θ ( δ ) ≤ cδ I − a K a I-aK_a I − a K a L 2 ( [ 0 , T ] ) L^2([0,T]) L 2 ([ 0 , T ]) C 0 ( [ 0 , T ] ) C_0([0,T]) C 0 ([ 0 , T ]) L 2 L^2 L 2 G ( ⋅ , x ˉ j ) G(\cdot,\bar{x}_j) G ( ⋅ , x ˉ j ) D = ∅ D=\varnothing D = ∅ P θ 0 P_{\theta_0} P θ 0 σ 0 2 B C W B ∗ \sigma_0^2BC_WB^* σ 0 2 B C W B ∗ B = I − a 0 K a 0 B=I-a_0K_{a_0} B = I − a 0 K a 0 C W C_W C W
