Figure 1. Illustration of two independent samples of 1,000 points in [ 0 , 1 ] [0,1] [ 0 , 1 ] , matched using a near-optimal mapping. Each color identifies a pair of corresponding intervals whose points are matched bijectively.
Introduction
How well can two independent random samples X , Y X,Y X , Y of n n n points from a metric measure space be matched while preserving pairwise distances? We study this question through the minimum distortion between X X X and Y Y Y , viewed as finite metric spaces with the induced metric. Our aim is to understand the typical growth of this distortion with the sample size.
For finite metric spaces X , Y X,Y X , Y of the same cardinality with at least two points, and a bijection f : X → Y f:X\to Y f : X → Y , define
Lip ( f ) = max x ≠ x ′ d Y ( f ( x ) , f ( x ′ ) ) d X ( x , x ′ ) , dist ( f ) = Lip ( f ) Lip ( f − 1 ) , \operatorname{Lip}(f)=\max_{x\ne x'}\frac{d_Y(f(x),f(x'))}{d_X(x,x')},\qquad\operatorname{dist}(f)=\operatorname{Lip}(f)\operatorname{Lip}(f^{-1}), Lip ( f ) = x = x ′ max d X ( x , x ′ ) d Y ( f ( x ) , f ( x ′ )) , dist ( f ) = Lip ( f ) Lip ( f − 1 ) ,
c ( X , Y ) = min { dist ( f ) : f : X → Y is bijective } . c(X,Y)=\min\{\operatorname{dist}(f):f:X\to Y\text{ is bijective}\}. c ( X , Y ) = min { dist ( f ) : f : X → Y is bijective } .
The distortion of a bijection is the product of its maximum expansion and contraction factors. In particular, it is invariant under rescaling either metric. Thus c ( X , Y ) c(X,Y) c ( X , Y ) measures how closely the two metric spaces can be matched.
The minimum distortion between two independent samples can be thought of as a notion of metric variance . It is well known that, for a square-integrable real random variable A A A and an independent copy A ′ A' A ′ , we have Var ( A ) = 1 2 E [ ( A − A ′ ) 2 ] \operatorname{Var}(A)=\frac{1}{2}\mathbb{E}[(A-A')^2] Var ( A ) = 2 1 E [( A − A ′ ) 2 ] . This identity expresses variance through the discrepancy between independent copies, without referring to their mean. Similarly, comparing X X X and Y Y Y through c ( X , Y ) c(X,Y) c ( X , Y ) provides a multiplicative measure of the variability of a random finite metric space, without requiring a definition of its expected metric space.
Main result
In this work, we study the minimum distortion between two copies, X X X and Y Y Y , each consisting of n n n points drawn uniformly and independently from [ 0 , 1 ] [0,1] [ 0 , 1 ] . We show that even this one-dimensional setting is highly non-trivial and exhibits rich geometric structure. Our main result is the following theorem:
Theorem 1.1. There exists an absolute constant C > 0 C>0 C > 0 such that, with high probability,
n 2 / 3 ( log n ) C ≤ c ( X , Y ) ≤ n 2 / 3 ( log n ) C . \frac{n^{2/3}}{(\log n)^C}\le c(X,Y)\le n^{2/3}(\log n)^C. ( log n ) C n 2/3 ≤ c ( X , Y ) ≤ n 2/3 ( log n ) C .
Why typical distortion? The expected distortion is a natural first statistic to consider. In our setting, however, it fails to capture the typical behavior, as E [ c ( X , Y ) ] = ∞ \mathbb{E}[c(X,Y)]=\infty E [ c ( X , Y )] = ∞ for sufficiently large n n n (see Appendix § 9 ). The reason is simple: an exceptionally close pair in one sample, with no comparably close pair in the other, forces large distortion under every bijection. Such rare configurations produce a sufficiently heavy tail to make the expectation infinite. We therefore study the order of magnitude distortion encountered with high probability when comparing two random configurations.
How (not) to match points. The simplest bijection matches points in order. Writing X = { x 1 < ⋯ < x n } X=\{x_1<\cdots<x_n\} X = { x 1 < ⋯ < x n } and Y = { y 1 < ⋯ < y n } Y=\{y_1<\cdots<y_n\} Y = { y 1 < ⋯ < y n } , it sends x i x_i x i to y i y_i y i . The difficulty with this matching is that the smallest gaps in the two samples need not occur at corresponding positions. With high probability, the smallest gap in X X X has size n − 2 + o ( 1 ) n^{-2+o(1)} n − 2 + o ( 1 ) , while the gap at the same position in Y Y Y has size n − 1 + o ( 1 ) n^{-1+o(1)} n − 1 + o ( 1 ) . This pair alone requires expansion by a factor n 1 + o ( 1 ) n^{1+o(1)} n 1 + o ( 1 ) . The smallest gap in Y Y Y causes the same problem for the inverse map. Thus both Lipschitz constants are n 1 + o ( 1 ) n^{1+o(1)} n 1 + o ( 1 ) , giving distortion n 2 + o ( 1 ) n^{2+o(1)} n 2 + o ( 1 ) with high probability.
This suggests matching the closest pair in X X X to the closest pair in Y Y Y , with both gaps of length n − 2 + o ( 1 ) n^{-2+o(1)} n − 2 + o ( 1 ) whp, and extending this partial matching to a bijection with low distortion. This controls the distance ratio for the selected pair, but leaves a geometric constraint on all remaining points. The two pairs can lie in very different parts of the interval: for example, one may lie near an endpoint and the other near the middle. They then divide [ 0 , 1 ] [0,1] [ 0 , 1 ] into parts containing very different numbers of points, which need to be matched. We prove that, whp, every bijection matching the minimum gap pair in X X X to the minimum gap pair in Y Y Y has large distortion of order at least n 1 − o ( 1 ) n^{1-o(1)} n 1 − o ( 1 ) .
The main difficulty. A more flexible approach is to match each exceptionally short gap to a suitable gap within a limited distance, without insisting that its partner be the shortest gap in the other sample. The difficulty is then to choose these matches so that they can be completed to a bijection with low distortion. Although exceptionally short gaps involve only a small fraction of the points, matching them with low distortion requires a carefully coordinated mapping of the remaining points, as illustrated in the left panel of Figure 2 . In particular, many neighboring pairs at the typical distance of order 1 / n 1/n 1/ n , which initially posed no obstacle, are mapped much farther apart to accommodate the displacement required by the short-gap matches. The oscillations distribute this displacement across the remaining points while controlling expansion in both directions. These oscillations must be coordinated in both samples: nearby points of X X X must have nearby images in Y Y Y , and nearby points of Y Y Y must have nearby preimages in X X X , within the allowed expansion factor. Searching farther for partners makes short gaps easier to match, but also increases the displacement that these oscillations must accommodate. Balancing these two requirements yields the exponent 2 / 3 2/3 2/3 . This balance identifies the relevant scales, but realizing it requires matching nearby short gaps together, resolving conflicts between prescribed matches, and completing the resulting partial matching while controlling expansion in both directions. All these components of the mapping are delicate and have nontrivial interactions. We survey them in the technical overview, § 3 .
Figure 2. An enlarged view of the pink interval. Left: the restricted matching. Right: the matched pairs ( x , f ( x ) ) (x, f(x)) ( x , f ( x )) , joined in increasing order of x x x . Dots represent the actual matched pairs.
Related work
Geometric similarity of random point sets in R \mathbb{R} R . A closely related line of work asks whether two independent realizations of the same random process have the same geometry at large scales. Peled [17 ] studied quantitative versions of this question for random subsets of the line. Basu and Sly [1 ] subsequently developed a multiscale construction for embeddings of random sequences, resolving the question for Poisson processes on the line: two independent realizations are almost surely roughly isometric. Rough isometries allow both multiplicative and additive errors in distances. We highlight that additive error allows exceptionally short gaps to be mapped to substantially longer ones, whereas our distortion objective requires relative control even for these gaps. More recently, Li, Yu, and Zhen [14 ] established a contrasting result for independent Bernoulli subsets of a product of two regular trees, which are almost surely not quasi-isometric. These results show how the ambient geometry affects the existence of geometrically controlled correspondences between independent random configurations.
Minimum-distortion bijections in R \mathbb{R} R . The computational problem of finding a minimum-distortion bijection between two prescribed finite metric spaces was studied by Kenyon, Rabani, and Sinclair [13 ] . For point sets on the line, they gave a polynomial-time algorithm that finds an optimal bijection when the minimum distortion is below a fixed constant, with a subsequent work by Chandran et al. [4 ] improving this threshold. Nayeri and Raichel [16 ] subsequently developed exact and approximation algorithms with running-time bounds in terms of the optimum distortion and the spread of the input, namely its diameter after rescaling the minimum distance to be one. On the hardness side, Hall and Papadimitriou [10 ] showed that, even for point sets on the line, approximating the minimum distortion within factors growing polynomially with the number of points is NP-hard in the regime of polynomially large optimum distortion. We prove that when the two point sets are random, finding a mapping that achieves a near-optimal distortion can be done whp.
Random geometric matching. A related line of work studies matchings between random point sets that keep matched points close to one another. Goel, Rai, and Krishnamachari [9 ] studied bottleneck matching between independent uniform samples, minimizing the largest distance between matched points, and used it to establish sharp thresholds for random geometric graphs. Holroyd, Pemantle, Peres, and Schramm [11 ] studied invariant perfect matchings between independent Poisson processes, focusing on the tail of the distance from a typical point to its partner. More recently, Elboim, Spinka, and Yakir [8 ] studied invariant perfect matchings between randomly perturbed lattices and the lattice, obtaining optimal tail bounds under suitable assumptions on the perturbations.
Mappings to grids and lattices. The classical work of Burago and Kleiner [3 ] and McMullen [15 ] shows that, in dimensions at least two, even uniformly separated point sets with bounded holes need not admit a bi-Lipschitz bijection to a lattice. Therefore, obstructions to bounded distortion can arise even in the absence of exceptionally close pairs. For finite lattice subsets, Dymond, Kaluža, and Kopecká [7 ] showed that the minimum Lipschitz constant of a bijection onto a regular grid can be arbitrarily large. Dymond [6 ] subsequently obtained polylogarithmic upper bounds for suitable random lattice subsets. These finite grid-mapping problems concern one-sided Lipschitz control and a prescribed regular target, while our problem requires control in both directions between two random samples.
Organization of the paper
We begin with the probabilistic estimates and metric reductions used throughout the paper in Section 2. In Section 3, we explain the origin of the 2 / 3 2/3 2/3 exponent and outline the proofs of the lower and upper bounds. Next, in Section 4, we relate displacement to distortion in the equally spaced model (where we study permutations of [ n ] [n] [ n ] and their distortion), yielding a short proof of the lower bound in Section 5. We present the upper-bound construction in Section 6.
Use of AI. The proof strategy of the main theorem was developed by the authors, yet we incorporated a simplification found by AI. ChatGPT-6 suggested using the exponential representation in Proposition 2.3 to replace our previous balls and bins type argument which allowed to use the independence of the gaps in the exponential model, and to simplify the approach behind the proof of Proposition 6.5 . After this simplification, we realized that our previous n 2 / 3 + o ( 1 ) n^{2/3+o(1)} n 2/3 + o ( 1 ) upper bound, should yield an n 2 / 3 ( log n ) O ( 1 ) n^{2/3}(\log n)^{O(1)} n 2/3 ( log n ) O ( 1 ) upper bound, when choosing the parameters more delicately. We subsequently used ChatGPT-6 to choose the parameters for this refinement. We used ChatGPT-6 extensively to assist with writing the paper and generating the figures.
After completing the main results we conjectured that controlling displacement through oscillations, as we did in the upper bound, is necessary. We asked the model to prove the existence of oscillating permutation minors in any permutation with distortion and displacement nearly equal. ChatGPT-6 supplied the proof (and defined woven permutations) appearing in Appendix § 8 . One component of the AI’s proof, Lemma A.6 , provides a shorter proof to Theorem 4.1 . We decided to retain our original proof in the main body of the paper.
Acknowledgments We thank Uri Grupel for useful discussions.
Preliminaries
Throughout, let n ≥ 2 n \ge2 n ≥ 2 . We denote an n n n -point set Z ⊆ R Z \subseteq\mathbb{R} Z ⊆ R , by Z = { z 1 < ⋯ < z n } Z = \{z_1 < \cdots< z_n\} Z = { z 1 < ⋯ < z n } . Let O \mathcal{O} O denote the distribution of the random set Z = { U 1 , … , U n } Z = \{U_1,\ldots,U_n\} Z = { U 1 , … , U n } , where U 1 , … , U n U_1,\ldots,U_n U 1 , … , U n are iid uniform random variables on [ 0 , 1 ] [0,1] [ 0 , 1 ] . These variables are distinct almost surely, so Z Z Z has size n n n almost surely. For 1 ≤ i < n 1 \le i < n 1 ≤ i < n , define Δ i Z = z i + 1 − z i \Delta_i^Z = z_{i+1} - z_i Δ i Z = z i + 1 − z i , and put Δ min Z = min 1 ≤ i < n Δ i Z \Delta_{\min}^Z = \min_{1 \le i < n}\Delta_i^Z Δ m i n Z = min 1 ≤ i < n Δ i Z .
The following proposition is a well-known exponential representation of uniform spacings (see [18 Sec. 4.1] or [5 Ch. V, Thm. 2.2] ).
Proposition 2.1. Let E 0 , … , E n ∼ Exp ( 1 ) E_0,\ldots,E_n \sim\operatorname{Exp}(1) E 0 , … , E n ∼ Exp ( 1 ) be iid and let S n + 1 = ∑ i = 0 n E i S_{n+1} = \sum_{i=0}^{n} E_i S n + 1 = ∑ i = 0 n E i . Then,
( z 1 , Δ 1 Z , … , Δ n − 1 Z , 1 − z n ) = d 1 S n + 1 ( E 0 , E 1 , … , E n ) . \left(z_1,\Delta_1^Z,\ldots,\Delta_{n-1}^Z,1-z_n\right) \mathrel{\mathop{=}\limits^{d}} \frac{1}{S_{n+1}}(E_0,E_1,\ldots,E_n). ( z 1 , Δ 1 Z , … , Δ n − 1 Z , 1 − z n ) = d S n + 1 1 ( E 0 , E 1 , … , E n ) .
The following observation follows trivially from the definition of Lipschitz constant of a function.
Observation 2.2. Let M , N M,N M , N be finite metric spaces and let f : M → N f:M \to N f : M → N be a bijection. If the metrics of M M M and N N N are multiplied by constants λ > 0 \lambda> 0 λ > 0 and μ > 0 \mu> 0 μ > 0 , respectively, then Lip ( f ) \operatorname{Lip}(f) Lip ( f ) is multiplied by μ / λ \mu/\lambda μ / λ and Lip ( f − 1 ) \operatorname{Lip}(f^{-1}) Lip ( f − 1 ) by λ / μ \lambda/\mu λ / μ . Consequently,
dist ( f ) \operatorname{dist}(f) dist ( f ) and c ( M , N ) c(M,N) c ( M , N ) remain the same and
for M , N ⊆ R M,N \subseteq\mathbb{R} M , N ⊆ R , translating the sets does not change c ( M , N ) c(M,N) c ( M , N ) as well.
This observation combined with Proposition 2.1 allows us to reformulate our problem.
Proposition 2.3. Let X , Y ∼ O X,Y \sim\mathcal{O} X , Y ∼ O be iid. Let E 1 X , … , E n − 1 X E_1^X,\ldots,E_{n-1}^X E 1 X , … , E n − 1 X and E 1 Y , … , E n − 1 Y E_1^Y,\ldots,E_{n-1}^Y E 1 Y , … , E n − 1 Y be iid exponential random variables with mean 1 1 1 . Let X ~ = { x ~ 1 < ⋯ < x ~ n } \widetilde{X} = \{\widetilde{x}_1 < \cdots< \widetilde{x}_n\} X = { x 1 < ⋯ < x n } be defined as follows: x ~ 1 = 0 \widetilde{x}_1 = 0 x 1 = 0 and, for all i ∈ [ n − 1 ] i \in[n-1] i ∈ [ n − 1 ] , let x ~ i + 1 = x ~ i + E i X \widetilde{x}_{i+1} = \widetilde{x}_i + E_i^X x i + 1 = x i + E i X . Similarly, define Y ~ = { y ~ 1 < ⋯ < y ~ n } \widetilde{Y} = \{\widetilde{y}_1 < \cdots< \widetilde{y}_n\} Y = { y 1 < ⋯ < y n } . Then,
c ( X , Y ) = d c ( X ~ , Y ~ ) . c(X,Y) \mathrel{\mathop{=}\limits^{d}} c(\widetilde{X},\widetilde{Y}). c ( X , Y ) = d c ( X , Y ) .
In order to bound c ( X , Y ) c(X,Y) c ( X , Y ) from above we need to present some bijection f : X → Y f:X \to Y f : X → Y with small Lipschitz constants. The following claim shows that we only need bound the quantities ∣ f ( x 2 ) − f ( x 1 ) ∣ x 2 − x 1 \frac{|f(x_2)-f(x_1)|}{x_2-x_1} x 2 − x 1 ∣ f ( x 2 ) − f ( x 1 ) ∣ and ∣ f − 1 ( y 2 ) − f − 1 ( y 1 ) ∣ y 2 − y 1 \frac{|f^{-1}(y_2)-f^{-1}(y_1)|}{y_2-y_1} y 2 − y 1 ∣ f − 1 ( y 2 ) − f − 1 ( y 1 ) ∣ for the nearby pairs of points x 1 , x 2 ∈ X x_1,x_2 \in X x 1 , x 2 ∈ X and y 1 , y 2 ∈ Y y_1,y_2 \in Y y 1 , y 2 ∈ Y .
Claim 2.4. Let X = { x 1 < ⋯ < x n } ⊆ R X = \{x_1 < \cdots< x_n\} \subseteq\mathbb{R} X = { x 1 < ⋯ < x n } ⊆ R and Y = { y 1 < ⋯ < y n } ⊆ R Y = \{y_1 < \cdots< y_n\} \subseteq\mathbb{R} Y = { y 1 < ⋯ < y n } ⊆ R and let f : X → Y f:X \to Y f : X → Y be a bijection. Then,
Lip ( f ) = max 1 ≤ i < n ∣ f ( x i + 1 ) − f ( x i ) ∣ x i + 1 − x i , Lip ( f − 1 ) = max 1 ≤ i < n ∣ f − 1 ( y i + 1 ) − f − 1 ( y i ) ∣ y i + 1 − y i . \operatorname{Lip}(f) = \max_{1 \le i < n}\frac{|f(x_{i+1})-f(x_i)|}{x_{i+1}-x_i}, \qquad\operatorname{Lip}(f^{-1}) = \max_{1 \le i < n}\frac{|f^{-1}(y_{i+1})-f^{-1}(y_i)|}{y_{i+1}-y_i}. Lip ( f ) = 1 ≤ i < n max x i + 1 − x i ∣ f ( x i + 1 ) − f ( x i ) ∣ , Lip ( f − 1 ) = 1 ≤ i < n max y i + 1 − y i ∣ f − 1 ( y i + 1 ) − f − 1 ( y i ) ∣ .
Proof. Let
L = max i < n ∣ f ( x i + 1 ) − f ( x i ) ∣ x i + 1 − x i . L = \max_{i<n}\frac{|f(x_{i+1})-f(x_i)|}{x_{i+1}-x_i}. L = i < n max x i + 1 − x i ∣ f ( x i + 1 ) − f ( x i ) ∣ .
For $1 \le j < k \le n$, by the triangle inequality we get
∣ f ( x k ) − f ( x j ) ∣ ≤ ∑ i = j k − 1 ∣ f ( x i + 1 ) − f ( x i ) ∣ ≤ L ∑ i = j k − 1 ( x i + 1 − x i ) = L ( x k − x j ) . \begin{aligned}
|f(x_k)-f(x_j)| \le\sum_{i=j}^{k-1}|f(x_{i+1})-f(x_i)| \\
&\le L\sum_{i=j}^{k-1}(x_{i+1}-x_i) = L(x_k-x_j).
\end{aligned} ∣ f ( x k ) − f ( x j ) ∣ ≤ i = j ∑ k − 1 ∣ f ( x i + 1 ) − f ( x i ) ∣ ≤ L i = j ∑ k − 1 ( x i + 1 − x i ) = L ( x k − x j ) .
So, Lip ( f ) = L \operatorname{Lip}(f) = L Lip ( f ) = L . The formula for f − 1 f^{-1} f − 1 is proved similarly.
Corollary 2.5. Let X = { x 1 < ⋯ < x n } X=\{x_1<\cdots<x_n\} X = { x 1 < ⋯ < x n } and Y = { y 1 < ⋯ < y n } Y=\{y_1<\cdots<y_n\} Y = { y 1 < ⋯ < y n } . Let f : X → Y f:X\to Y f : X → Y be a bijection and let π : [ n ] → [ n ] \pi:[n]\to[n] π : [ n ] → [ n ] be the permutation satisfying f ( x i ) = y π ( i ) f(x_i)=y_{\pi(i)} f ( x i ) = y π ( i ) . Then,
Lip ( f ) = max 1 ≤ i < n ∣ y π ( i + 1 ) − y π ( i ) ∣ x i + 1 − x i , Lip ( f − 1 ) = max 1 ≤ i < n ∣ x π − 1 ( i + 1 ) − x π − 1 ( i ) ∣ y i + 1 − y i . \operatorname{Lip}(f)=\max_{1\leq i<n}\frac{\lvert y_{\pi(i+1)}-y_{\pi(i)}\rvert}{x_{i+1}-x_i},\qquad\operatorname{Lip}(f^{-1})=\max_{1\leq i<n}\frac{\lvert x_{\pi^{-1}(i+1)}-x_{\pi^{-1}(i)}\rvert}{y_{i+1}-y_i}. Lip ( f ) = 1 ≤ i < n max x i + 1 − x i ∣ y π ( i + 1 ) − y π ( i ) ∣ , Lip ( f − 1 ) = 1 ≤ i < n max y i + 1 − y i ∣ x π − 1 ( i + 1 ) − x π − 1 ( i ) ∣ .
Spacing estimates
First, we will need the following simple bound that can be found in [12 ] .
Claim 2.6. Let S n S_n S n denote the sum of n n n iid exponential variables with mean 1 1 1 . There exists a universal constant c > 0 c>0 c > 0 such that, for all large enough n n n , we have
P ( n / 2 ≤ S n + 1 ≤ 2 n ) ≥ 1 − exp ( − c n ) . \mathbb{P}\left(n/2\leq S_{n+1}\leq2n\right)\geq1-\exp(-cn). P ( n /2 ≤ S n + 1 ≤ 2 n ) ≥ 1 − exp ( − c n ) .
The following claim bounds x j − x i x_j-x_i x j − x i for all 1 ≤ i < j ≤ n 1\leq i<j\leq n 1 ≤ i < j ≤ n satisfying j ≥ i + C log n j\geq i+C\log n j ≥ i + C log n .
Claim 2.7. There exists C > 0 C>0 C > 0 such that for X ∼ O X\sim\mathcal{O} X ∼ O whp the following holds simultaneously for all 1 ≤ i < j ≤ n 1\leq i<j\leq n 1 ≤ i < j ≤ n with j − i ≥ C log n j-i\geq C\log n j − i ≥ C log n ,
j − i C n ≤ x j − x i ≤ C ( j − i ) n . \frac{j-i}{Cn}\leq x_j-x_i\leq\frac{C(j-i)}{n}. C n j − i ≤ x j − x i ≤ n C ( j − i ) .
Proof. Let E 0 , … , E n ∼ Exp ( 1 ) E_0,\ldots,E_n\sim\operatorname{Exp}(1) E 0 , … , E n ∼ Exp ( 1 ) be iid and let S n + 1 = ∑ i = 0 n E i S_{n+1}=\sum_{i=0}^{n}E_i S n + 1 = ∑ i = 0 n E i . By Proposition 2.1 ,
( x 1 , Δ 1 X , … , Δ n − 1 X , 1 − x n ) = d 1 S n + 1 ( E 0 , E 1 , … , E n ) . \left(x_1,\Delta_1^X,\ldots,\Delta_{n-1}^X,1-x_n\right)\mathrel{\mathop{=}^{d}}\frac{1}{S_{n+1}}(E_0,E_1,\ldots,E_n). ( x 1 , Δ 1 X , … , Δ n − 1 X , 1 − x n ) = d S n + 1 1 ( E 0 , E 1 , … , E n ) .
By Claim 2.6 , we whp have n / 2 ≤ S n + 1 ≤ 2 n n/2\leq S_{n+1}\leq2n n /2 ≤ S n + 1 ≤ 2 n . Also, for arbitrary indices 1 ≤ i < j ≤ n 1\leq i<j\leq n 1 ≤ i < j ≤ n satisfying j ≥ i + C log n j\geq i+C\log n j ≥ i + C log n it holds that
x j − x i = d 1 S n + 1 ∑ k = i j − 1 E k . (1) x_j-x_i\mathrel{\mathop{=}^{d}}\frac{1}{S_{n+1}}\sum_{k=i}^{j-1}E_k. \tag*{(1)} x j − x i = d S n + 1 1 k = i ∑ j − 1 E k . (1)
We first claim that uniformly over all pairs 1 ≤ i < j ≤ n 1\leq i<j\leq n 1 ≤ i < j ≤ n satisfying j ≥ i + C log n j\geq i+C\log n j ≥ i + C log n , we have
P ( j − i C / 2 ≤ ∑ k = i j − 1 E k ≤ ( C / 2 ) ( j − i ) ) = 1 − o ( n − 2 ) . \mathbb{P}\left(\frac{j-i}{C/2}\leq\sum_{k=i}^{j-1}E_k\leq(C/2)(j-i)\right)=1-o(n^{-2}). P ( C /2 j − i ≤ k = i ∑ j − 1 E k ≤ ( C /2 ) ( j − i ) ) = 1 − o ( n − 2 ) .
Indeed, by using the constant c > 0 c>0 c > 0 of Claim 2.6 , and setting C > max ( 4 , 10 c − 1 ) C>\max(4,10c^{-1}) C > max ( 4 , 10 c − 1 ) we have for large enough n n n :
P ( j − i C / 2 ≤ ∑ k = i j − 1 E k ≤ ( C / 2 ) ( j − i ) ) ≥ 1 − exp ( − c ⋅ ( j − i ) ) \mathbb{P}\left(\frac{j-i}{C/2}\leq\sum_{k=i}^{j-1}E_k\leq(C/2)(j-i)\right)\geq1-\exp(-c\cdot(j-i)) P ( C /2 j − i ≤ k = i ∑ j − 1 E k ≤ ( C /2 ) ( j − i ) ) ≥ 1 − exp ( − c ⋅ ( j − i ))
≥ 1 − exp ( − c ⋅ C ⋅ log n ) = 1 − o ( n − 2 ) . \geq1-\exp(-c\cdot C\cdot\log n)=1-o(n^{-2}). ≥ 1 − exp ( − c ⋅ C ⋅ log n ) = 1 − o ( n − 2 ) .
bounding the complement of the above event, over the O ( n 2 ) O(n^2) O ( n 2 ) pairs of indices ( i , j ) (i,j) ( i , j ) and combining that with n / 2 ≤ S n + 1 ≤ 2 n n/2\leq S_{n+1}\leq2n n /2 ≤ S n + 1 ≤ 2 n , which holds whp, and with (1) we conclude the proof. □
Notice that for all i ∈ [ n − 1 ] i\in[n-1] i ∈ [ n − 1 ] the quantity x i + 1 − x i x_{i+1}-x_i x i + 1 − x i does not exceed the quantity x k − x j x_k-x_j x k − x j for some j ≤ i < k j\leq i<k j ≤ i < k satisfying k − j = ⌈ C log n ⌉ k-j=\lceil C\log n\rceil k − j = ⌈ C log n ⌉ . Therefore, Claim 2.7 gives the following corollary.
Corollary 2.8. Let C > 0 C > 0 C > 0 be large. For X ∼ O X \sim\mathcal{O} X ∼ O and for all i ∈ [ n − 1 ] i \in[n-1] i ∈ [ n − 1 ] , whp
Δ i X ≤ C log n n . \Delta_i^X \le C\frac{\log n}{n}. Δ i X ≤ C n log n .
Claim 2.9. Let I ⊆ { 1 , … , n − 1 } I \subseteq\{1,\ldots,n-1\} I ⊆ { 1 , … , n − 1 } be an interval of size m ∈ [ n − 1 ] m \in[n-1] m ∈ [ n − 1 ] , and let X ∼ O X \sim\mathcal{O} X ∼ O . Then
P ( 1 2 m n log n ≤ min i ∈ I Δ i X ≤ 2 log n m n ) = 1 − O ( 1 log n ) . \mathbb{P}\left(\frac{1}{2mn\log n} \le\min_{i\in I}\Delta_i^X \le\frac{2\log n}{mn}\right)=1-\mathcal{O}\left(\frac{1}{\log n}\right). P ( 2 mn log n 1 ≤ i ∈ I min Δ i X ≤ mn 2 log n ) = 1 − O ( log n 1 ) .
Proof. Let E 0 , … , E n ∼ Exp ( 1 ) E_0,\ldots,E_n \sim\operatorname{Exp}(1) E 0 , … , E n ∼ Exp ( 1 ) be iid and let S n + 1 = ∑ i = 0 n E i S_{n+1}=\sum_{i=0}^{n}E_i S n + 1 = ∑ i = 0 n E i . By Proposition 2.1,
( x 1 , Δ 1 X , … , Δ n − 1 X , 1 − x n ) = d 1 S n + 1 ( E 0 , E 1 , … , E n ) . \left(x_1,\Delta_1^X,\ldots,\Delta_{n-1}^X,1-x_n\right)\mathrel{\mathop{=}^{d}}\frac{1}{S_{n+1}}(E_0,E_1,\ldots,E_n). ( x 1 , Δ 1 X , … , Δ n − 1 X , 1 − x n ) = d S n + 1 1 ( E 0 , E 1 , … , E n ) .
Recall that a minimum of m m m iid exponential variables with mean 1 1 1 is an exponential variable with mean 1 / m 1/m 1/ m and thus min i ∈ I E i ∼ Exp ( m ) \min_{i\in I}E_i\sim\operatorname{Exp}(m) min i ∈ I E i ∼ Exp ( m ) . Hence,
P ( min i ∈ I E i < 1 m log n ) ≤ 1 log n , P ( min i ∈ I E i > log n m ) = 1 n . \mathbb{P}\left(\min_{i\in I}E_i<\frac{1}{m\log n}\right)\le\frac{1}{\log n},\qquad
\mathbb{P}\left(\min_{i\in I}E_i>\frac{\log n}{m}\right)=\frac{1}{n}. P ( i ∈ I min E i < m log n 1 ) ≤ log n 1 , P ( i ∈ I min E i > m log n ) = n 1 .
Since min i ∈ I Δ i X = d min i ∈ I E i / S n + 1 \min_{i\in I}\Delta_i^X\mathrel{\mathop{=}^{d}}\min_{i\in I}E_i/S_{n+1} min i ∈ I Δ i X = d min i ∈ I E i / S n + 1 , combining the bounds above with Claim 2.6 we conclude the desired estimate.
Corollary 2.10. For X ∼ O X\sim\mathcal{O} X ∼ O , whp
1 3 n 2 log n ≤ Δ min X ≤ 3 log n n 2 . \frac{1}{3n^2\log n}\le\Delta_{\min}^X\le\frac{3\log n}{n^2}. 3 n 2 log n 1 ≤ Δ m i n X ≤ n 2 3 log n .
Technical Overview
Let X ∼ O X\sim\mathcal{O} X ∼ O and Y ∼ O Y\sim\mathcal{O} Y ∼ O be independent samples, and write X = { x 1 < ⋯ < x n } X=\{x_1<\cdots<x_n\} X = { x 1 < ⋯ < x n } and Y = { y 1 < ⋯ < y n } Y=\{y_1<\cdots<y_n\} Y = { y 1 < ⋯ < y n } . Observe that bijections f : X → Y f:X\to Y f : X → Y are in one-to-one correspondence with permutations π : [ n ] → [ n ] \pi:[n]\to[n] π : [ n ] → [ n ] , by setting f ( x i ) = y π ( i ) f(x_i)=y_{\pi(i)} f ( x i ) = y π ( i ) . We use this correspondence to analyze how well f f f maps points of X X X to points of Y Y Y . Throughout this overview, O ~ \widetilde{\mathcal{O}} O and Ω ~ \widetilde{\Omega} Ω suppress polylogarithmic factors.
The lower bound
For the lower bound, we classify the bijections f : X → Y f:X\to Y f : X → Y into two types: local or nonlocal, according to how the permutation π \pi π that induces f f f maps elements, and a parameter T ∈ { 2 , … , n } T\in\{2,\ldots,n\} T ∈ { 2 , … , n } to be chosen later. Informally, a T T T -local bijection maps every index to a T T T -neighborhood of itself, that is ∣ π ( i ) − i ∣ ≤ T |\pi(i)-i|\le T ∣ π ( i ) − i ∣ ≤ T for the permutation π \pi π associated with f f f . As the interval [ 0 , 1 ] [0,1] [ 0 , 1 ] has reflection as an isometry, we also need to take reflections into account in the definitions.
More formally, we measure the locality of a permutation by its ℓ ∞ \ell_\infty ℓ ∞ distance from a global isometry of [ n ] [n] [ n ] . The only such isometries are the identity and the reversal ρ ( i ) = n + 1 − i \rho(i)=n+1-i ρ ( i ) = n + 1 − i . We therefore define5
disp ( π ) = min τ ∈ { id , ρ } max i ∈ [ n ] ∣ π ( i ) − τ ( i ) ∣ . \operatorname{disp}(\pi)=\min_{\tau\in\{\operatorname{id},\rho\}}\max_{i\in[n]}|\pi(i)-\tau(i)|. disp ( π ) = τ ∈ { id , ρ } min i ∈ [ n ] max ∣ π ( i ) − τ ( i ) ∣.
Permutations with disp ( π ) ≤ T \operatorname{disp}(\pi) \le T disp ( π ) ≤ T . We show that, with high probability, all permutations π \pi π with disp ( π ) ≤ T \operatorname{disp}(\pi) \le T disp ( π ) ≤ T induce mappings f f f satisfying Lip ( f ) = Ω ~ ( n / T ) \operatorname{Lip}(f) = \widetilde{\Omega}(n/T) Lip ( f ) = Ω ( n / T ) . To see this, choose x i , x i + 1 x_i, x_{i+1} x i , x i + 1 achieving the minimum gap in X X X . By Corollary 2.10 , this gap is O ~ ( n − 2 ) \widetilde{O}(n^{-2}) O ( n − 2 ) with high probability.
Fix a choice of X X X , and let i i i be the index of its minimum gap. Since disp ( π ) ≤ T \operatorname{disp}(\pi) \le T disp ( π ) ≤ T , the points x i , x i + 1 x_i, x_{i+1} x i , x i + 1 are mapped to y j , y k y_j, y_k y j , y k , where either j , k ∈ [ i − 2 T , i + 2 T ] ∩ [ n ] j, k \in[i-2T,i+2T]\cap[n] j , k ∈ [ i − 2 T , i + 2 T ] ∩ [ n ] , or j , k ∈ [ ρ ( i ) − 2 T , ρ ( i ) + 2 T ] ∩ [ n ] j, k \in[\rho(i)-2T,\rho(i)+2T]\cap[n] j , k ∈ [ ρ ( i ) − 2 T , ρ ( i ) + 2 T ] ∩ [ n ] . Since Y Y Y is chosen independently of X X X , we can apply Claim 2.9 to the windows [ i − 2 T , i + 2 T ] ∩ [ n ] [i-2T,i+2T]\cap[n] [ i − 2 T , i + 2 T ] ∩ [ n ] , and [ ρ ( i ) − 2 T , ρ ( i ) + 2 T ] ∩ [ n ] [\rho(i)-2T,\rho(i)+2T]\cap[n] [ ρ ( i ) − 2 T , ρ ( i ) + 2 T ] ∩ [ n ] . As each window contains O ( T ) O(T) O ( T ) consecutive points, whp the minimum distance in each window is Ω ~ ( 1 / ( n T ) ) \widetilde{\Omega}(1/(nT)) Ω ( 1/ ( n T )) . Consequently, whp ∣ f ( x i ) − f ( x i + 1 ) ∣ = Ω ~ ( 1 / ( n T ) ) 2 |f(x_i)-f(x_{i+1})| = \widetilde{\Omega}(1/(nT))^2 ∣ f ( x i ) − f ( x i + 1 ) ∣ = Ω ( 1/ ( n T ) ) 2 . Comparing with the original gap gives
Lip ( f ) ≥ ∣ f ( x i + 1 ) − f ( x i ) ∣ x i + 1 − x i = Ω ~ ( n / T ) . \operatorname{Lip}(f) \ge\frac{|f(x_{i+1})-f(x_i)|}{x_{i+1}-x_i} = \widetilde{\Omega}(n/T). Lip ( f ) ≥ x i + 1 − x i ∣ f ( x i + 1 ) − f ( x i ) ∣ = Ω ( n / T ) .
Since disp ( π − 1 ) = disp ( π ) \operatorname{disp}(\pi^{-1}) = \operatorname{disp}(\pi) disp ( π − 1 ) = disp ( π ) , applying the same argument with X X X and Y Y Y interchanged gives Lip ( f − 1 ) = Ω ~ ( n / T ) \operatorname{Lip}(f^{-1}) = \widetilde{\Omega}(n/T) Lip ( f − 1 ) = Ω ( n / T ) . Thus, whp, all permutations π \pi π satisfying disp ( π ) ≤ T \operatorname{disp}(\pi) \le T disp ( π ) ≤ T give rise to mappings f f f with dist ( f ) = Ω ~ ( ( n / T ) 2 ) \operatorname{dist}(f) = \widetilde{\Omega}((n/T)^2) dist ( f ) = Ω (( n / T ) 2 ) .
Permutations with disp ( π ) ≥ T \operatorname{disp}(\pi) \ge T disp ( π ) ≥ T . For permutations of large displacement, we use a different obstruction. Our main ingredient is the following theorem, showing that displacement itself causes distortion. Concretely, let dist ( π ) \operatorname{dist}(\pi) dist ( π ) be the distortion of a permutation π \pi π , when [ n ] [n] [ n ] is considered with the induced metric from R \mathbb{R} R . We prove:
Theorem 3.1. There exists a universal constant c > 0 c > 0 c > 0 such that, for every n ≥ 2 n \ge2 n ≥ 2 and every permutation π : [ n ] → [ n ] \pi: [n] \to[n] π : [ n ] → [ n ] ,
dist ( π ) ≥ c disp ( π ) . \operatorname{dist}(\pi) \ge c\operatorname{disp}(\pi). dist ( π ) ≥ c disp ( π ) .
The above theorem can be seen as a finite (and quantitative) analog of a theorem by Benjamini and Shamov [2 ] which says that bi-Lipschitz bijections of Z \mathbb{Z} Z are at finite displacement from an isometry. Using the above theorem, the lower bound for the random setting of bijections f : X → Y f : X \to Y f : X → Y follows naturally from this theorem. Concretely, in Claim 5.2 we prove that, with high probability, Lip ( f ) ≥ c ′ Lip ( π ) / log n \operatorname{Lip}(f) \ge c'\operatorname{Lip}(\pi)/\log n Lip ( f ) ≥ c ′ Lip ( π ) / log n and Lip ( f − 1 ) ≥ c ′ Lip ( π − 1 ) / log n \operatorname{Lip}(f^{-1}) \ge c'\operatorname{Lip}(\pi^{-1})/\log n Lip ( f − 1 ) ≥ c ′ Lip ( π − 1 ) / log n simultaneously for every bijection f f f , where c ′ > 0 c' > 0 c ′ > 0 is an absolute constant. Together with Theorem 3.1 , this gives that whenever disp ( π ) ≥ T \operatorname{disp}(\pi) \ge T disp ( π ) ≥ T ,
dist ( f ) = Ω ~ ( dist ( π ) ) = Ω ~ ( disp ( π ) ) = Ω ~ ( T ) . \operatorname{dist}(f) = \widetilde{\Omega}(\operatorname{dist}(\pi)) = \widetilde{\Omega}(\operatorname{disp}(\pi)) = \widetilde{\Omega}(T). dist ( f ) = Ω ( dist ( π )) = Ω ( disp ( π )) = Ω ( T ) .
The 2/3 exponent. As explained, the two lower bounds above hold whp for all permutations π \pi π . On this event, every bijection f : X → Y f : X \to Y f : X → Y satisfies
dist ( f ) = Ω ~ ( min { n 2 T 2 , T } ) . \operatorname{dist}(f) = \widetilde{\Omega}\left(\min\left\{\frac{n^2}{T^2},T\right\}\right). dist ( f ) = Ω ( min { T 2 n 2 , T } ) .
The exponent 2 / 3 2/3 2/3 comes from balancing the terms, which happens for T ≈ n 2 / 3 T \approx n^{2/3} T ≈ n 2/3 . This proves that, with high probability, c ( X , Y ) = Ω ~ ( n 2 / 3 ) c(X,Y) = \widetilde{\Omega}(n^{2/3}) c ( X , Y ) = Ω ( n 2/3 ) as required.
The upper bound
We construct a bijection f : X → Y f : X \to Y f : X → Y satisfying max { Lip ( f ) , Lip ( f − 1 ) } = O ~ ( n 1 / 3 ) \max\{\operatorname{Lip}(f), \operatorname{Lip}(f^{-1})\} = \widetilde{O}(n^{1/3}) max { Lip ( f ) , Lip ( f − 1 )} = O ( n 1/3 ) . Suppose first that every pair of consecutive points in each sample were at distance at least n − 4 / 3 n^{-4/3} n − 4/3 . Since, with high probability, every consecutive gap in both X X X and Y Y Y is O ~ ( 1 / n ) \widetilde{O}(1/n) O ( 1/ n ) , the increasing matching f ( x i ) = y i f(x_i) = y_i f ( x i ) = y i and its inverse would have expansion O ~ ( n 1 / 3 ) \widetilde{O}(n^{1/3}) O ( n 1/3 ) , as required, by Claim 2.4. Thus the need to depart from the increasing bijection comes from exceptionally close pairs of consecutive points below the n − 4 / 3 n^{-4/3} n − 4/3 threshold. We call these pairs tiny ; the precise cutoff is slightly below n − 4 / 3 n^{-4/3} n − 4/3 by a polylogarithmic factor.
Handling the tiny pairs
We say that a pair y j , y j + 1 y_j,y_{j+1} y j , y j + 1 covers a pair x i , x i + 1 x_i,x_{i+1} x i , x i + 1 if
y j + 1 − y j x i + 1 − x i , x i + 1 − x i y j + 1 − y j ∈ O ~ ( n 1 / 3 ) . \frac{y_{j+1}-y_j}{x_{i+1}-x_i},\quad\frac{x_{i+1}-x_i}{y_{j+1}-y_j} \in\widetilde{O}(n^{1/3}). x i + 1 − x i y j + 1 − y j , y j + 1 − y j x i + 1 − x i ∈ O ( n 1/3 ) .
As explained above, when all gaps are at least n − 4 / 3 n^{-4/3} n − 4/3 , the increasing matching sends every consecutive pair to a pair covering it.
The lower-bound argument shows that a bijection of distortion O ~ ( n 2 / 3 ) \widetilde{O}(n^{2/3}) O ( n 2/3 ) must have displacement O ~ ( n 2 / 3 ) \widetilde{O}(n^{2/3}) O ( n 2/3 ) , measured relative to the identity or the reversal. We seek a mapping close to the identity and try to map each tiny pair x i , x i + 1 x_i,x_{i+1} x i , x i + 1 to a covering pair y j , y j + 1 y_j,y_{j+1} y j , y j + 1 with ∣ j − i ∣ = O ~ ( n 2 / 3 ) |j-i|=\widetilde{O}(n^{2/3}) ∣ j − i ∣ = O ( n 2/3 ) . Similarly, we require the mapping to control the tiny pairs in Y Y Y .
Observe that this approach does not work as is. Suppose that x i , x i + 1 x_i,x_{i+1} x i , x i + 1 and x i + d , x i + d + 1 x_{i+d},x_{i+d+1} x i + d , x i + d + 1 are tiny, where d ≪ n 1 / 3 d \ll n^{1/3} d ≪ n 1/3 . Choosing their covers independently within n 2 / 3 n^{2/3} n 2/3 index difference (for i i i and i + d i+d i + d ) may send them to pairs whose indices differ by Θ ( n 2 / 3 ) \Theta(n^{2/3}) Θ ( n 2/3 ) . The two pairs of images f ( x i ) , f ( x i + 1 ) f(x_i),f(x_{i+1}) f ( x i ) , f ( x i + 1 ) and f ( x i + d ) , f ( x i + d + 1 ) f(x_{i+d}),f(x_{i+d+1}) f ( x i + d ) , f ( x i + d + 1 ) will then be at distance of order n − 1 / 3 n^{-1/3} n − 1/3 , while x i x_i x i and x i + d x_{i+d} x i + d are at distance at most O ~ ( d / n ) \widetilde{O}(d/n) O ( d / n ) . Consequently, up to polylogs, the expansion can be as large as
n − 1 / 3 d / n = n 2 / 3 d . (2) \frac{n^{-1/3}}{d/n}=\frac{n^{2/3}}{d}. \tag*{(2)} d / n n − 1/3 = d n 2/3 . (2)
This exceeds our desired bounds on Lip ( f ) \operatorname{Lip}(f) Lip ( f ) and Lip ( f − 1 ) \operatorname{Lip}(f^{-1}) Lip ( f − 1 ) .
Therefore, tiny pairs whose indices are within O ~ ( n 1 / 3 ) \widetilde{O}(n^{1/3}) O ( n 1/3 ) of one another require extra care and need to be handled together. Concretely, we will force nearby tiny pairs (from either copy) to be mapped together using the same shift . Notice that a priori it is not clear that all these choices, mapping tiny pairs to covers, especially with the constraint that nearby tiny pairs are mapped together, can be made consistently. We show, however, that such a choice is possible, which is the main probabilistic argument in the upper bound (the rest of the argument is geometric in nature, and uses only basic properties of O O O ).
To handle both samples simultaneously, we call an index i i i tiny if either x i , x i + 1 x_i,x_{i+1} x i , x i + 1 or y i , y i + 1 y_i,y_{i+1} y i , y i + 1 is a tiny pair. Our main probabilistic argument shows that we can choose the partners consistently and obtain the following interval matches.
Proposition 3.2 (Proposition 6.5, informal). For suitable H = n 2 / 3 log O ( 1 ) n H=n^{2/3}\log^{O(1)}n H = n 2/3 log O ( 1 ) n and D = n 1 / 3 log O ( 1 ) n D=n^{1/3}\log^{O(1)}n D = n 1/3 log O ( 1 ) n , with high probability there exist a partition [ n ] = J 1 ⊔ ⋯ ⊔ J ℓ [n]=J_1\sqcup\cdots\sqcup J_\ell [ n ] = J 1 ⊔ ⋯ ⊔ J ℓ into consecutive parts, a family
I = { I 1 , I ‾ 1 , … , I m , I ‾ m } \mathcal{I}=\{I_1,\overline{I}_1,\ldots,I_m,\overline{I}_m\} I = { I 1 , I 1 , … , I m , I m } of pairwise disjoint intervals of indices, and a bijection g : X U → Y U g:X_U\to Y_U g : X U → Y U , where U = ⋃ I ∈ I I U=\bigcup_{I\in\mathcal{I}}I U = ⋃ I ∈ I I and Z A = { z i : i ∈ A } ∗ f o r ∗ Z_A=\{z_i:i\in A\} *for* Z A = { z i : i ∈ A } ∗ f or ∗ Z\in{X,Y}$, satisfying:
:::`
Each part satisfies H ≤ ∣ J j ∣ ≤ 3 H H \le|J_j| \le3H H ≤ ∣ J j ∣ ≤ 3 H , and each pair satisfies ∣ I k ∣ = ∣ I ‾ k ∣ = O ( D log n ) |I_k| = |\overline{I}_k| = O(D \log n) ∣ I k ∣ = ∣ I k ∣ = O ( D log n ) .
For every tiny index i i i , there is an interval I ∈ I I \in\mathcal{I} I ∈ I containing both i i i and i + 1 i+1 i + 1 .
For every k k k , the map g g g sends X I k X_{I_k} X I k increasingly onto Y I ‾ k Y_{\overline{I}_k} Y I k and X I ‾ k X_{\overline{I}_k} X I k increasingly onto Y I k Y_{I_k} Y I k . In each mapping, the first and last points are mapped to the first and last points, respectively.
For every I ∈ I I \in\mathcal{I} I ∈ I , max { Lip ( g ∣ X I ) , Lip ( g − 1 ∣ Y I ) } = O ~ ( n 1 / 3 ) \max\{\operatorname{Lip}(g|_{X_I}), \operatorname{Lip}(g^{-1}|_{Y_I})\} = \widetilde{O}(n^{1/3}) max { Lip ( g ∣ X I ) , Lip ( g − 1 ∣ Y I )} = O ( n 1/3 ) .
Each pair I k , I ‾ k I_k,\overline{I}_k I k , I k is contained in one part J j J_j J j , and both intervals are at distance at least D / 2 D/2 D /2 from its endpoints. Any two distinct intervals in I \mathcal{I} I are separated by at least D / 2 D/2 D /2 .
We omit from the overview a sketch of the proof and highlight what the proposition guarantees. By (2), every tiny pair in either sample has both endpoint indices in one of the intervals of I \mathcal{I} I . Property (3) pairs these intervals and matches their points increasingly, in both directions between X X X and Y Y Y . Property (4) gives the desired bound O ~ ( n 1 / 3 ) \widetilde{O}(n^{1/3}) O ( n 1/3 ) on the expansion in both directions of g g g when restricted to an interval.
Properties (1) and (5) impose additional size and separation requirements whose merit has not yet been motivated. They organize the construction at two scales. The parts J 1 , … , J ℓ J_1,\ldots,J_\ell J 1 , … , J ℓ have size Θ ( H ) \Theta(H) Θ ( H ) , where H = n 2 / 3 log O ( 1 ) n H = n^{2/3}\log^{O(1)} n H = n 2/3 log O ( 1 ) n , and each matched pair of intervals lies within a single part. The intervals in I \mathcal{I} I are separated from one another and from the endpoints of their containing parts by at least D / 2 D/2 D /2 indices. These requirements provide the additional geometric conditions needed for the rest of the proof, as we explain below.
An interlude: the role of oscillations
The partial matching g g g from the previous proposition was chosen with the aim of extending it to a bijection from X X X to Y Y Y with the desired Lipschitz bounds. A priori, however, it is unclear whether this can be done, and if so, how. We first illustrate this challenge through a toy example involving a permutation of equally spaced points. The example also shows that the bound in Theorem 4.1 is sharp up to constant factors in some cases.
Imagine completing the mapping from X X X to Y Y Y from left to right, mapping the points of X X X whose indices lie in J j J_j J j (whose length is ≈ n 2 / 3 \approx n^{2/3} ≈ n 2/3 ) to the points of Y Y Y with indices in the same part. For each J j J_j J j we will handle the displacements prescribed by tiny pairs inside it as we encounter them.
For our toy model, let m = k 2 m = k^2 m = k 2 , where k ≥ 3 k \ge3 k ≥ 3 is odd. We consider permutations of [ m ] [m] [ m ] , with m ≈ n 2 / 3 m \approx n^{2/3} m ≈ n 2/3 and k ≈ n 1 / 3 k \approx n^{1/3} k ≈ n 1/3 . Ignoring rounding and logarithmic factors, [ m ] [m] [ m ] represents the points of some J j J_j J j . We also require π ( 1 ) = 1 \pi(1) = 1 π ( 1 ) = 1 to model that the left boundary of J j J_j J j is mapped onto itself (together with fixing the right boundary of J j − 1 J_{j-1} J j − 1 , this controls the expansion across the boundary between J j − 1 J_{j-1} J j − 1 and J j J_j J j ).
Now suppose that we also require π ( k ) = m − k + 1 \pi(k) = m-k+1 π ( k ) = m − k + 1 . Think of this as modelling the requirement to map a tiny pair at index k k k to a covering pair at index m − k + 1 m-k+1 m − k + 1 . This forces k k k to move by m − 2 k + 1 = ( k − 1 ) 2 ≈ k 2 m-2k+1=(k-1)^2 \approx k^2 m − 2 k + 1 = ( k − 1 ) 2 ≈ k 2 indices. Can we find a permutation π : [ m ] → [ m ] \pi:[m]\to[m] π : [ m ] → [ m ] satisfying both π ( 1 ) = 1 \pi(1)=1 π ( 1 ) = 1 and π ( k ) = m − k + 1 \pi(k)=m-k+1 π ( k ) = m − k + 1 , while keeping Lip ( π ) \operatorname{Lip}(\pi) Lip ( π ) and Lip ( π − 1 ) \operatorname{Lip}(\pi^{-1}) Lip ( π − 1 ) bounded by O ( k ) O(k) O ( k ) ?
We now describe such a permutation. Moreover, the Lipschitz bounds in both directions strongly constrain how we can accommodate this displacement.
We will construct π \pi π with Lip ( π ) , Lip ( π − 1 ) ≤ 2 k \operatorname{Lip}(\pi),\operatorname{Lip}(\pi^{-1}) \le2k Lip ( π ) , Lip ( π − 1 ) ≤ 2 k , giving distortion at most 4 k 2 = 4 m 4k^2=4m 4 k 2 = 4 m . Since π ( 1 ) = 1 \pi(1)=1 π ( 1 ) = 1 and π ( k ) = m − k + 1 \pi(k)=m-k+1 π ( k ) = m − k + 1 , the first k k k images must travel a distance of order k 2 k^2 k 2 . We choose these images so that they generally move to the right, with consecutive images at most 2 k 2k 2 k indices apart, as shown by the black and blue arrows in Figure 3 .
Figure 3. An oscillating permutation for k = 5 k=5 k = 5 . (1) The black arrows represent the initial assumption on π \pi π , π ( 1 ) = 1 \pi(1)=1 π ( 1 ) = 1 , and π ( k ) = m − k + 1 \pi(k)=m-k+1 π ( k ) = m − k + 1 . (2) The blue arrows correspond to a walk from 1 to π ( 5 ) = 21 \pi(5)=21 π ( 5 ) = 21 with low expansion. (3) The orange arrows are neighbours of already assigned images whose preimages under π \pi π must lie near [ 1 , 5 ] [1,5] [ 1 , 5 ] . (4) The remaining arrows completing the permutation.
Now consider the unused neighbors of these images. By the inverse Lipschitz bound, their preimages must lie in [ k + 1 , 3 k ] [k+1,3k] [ k + 1 , 3 k ] . Thus the next 2 k 2k 2 k images must revisit points spread across the interval we have just traversed. For example, suppose we choose π ( 2 k ) = 2 \pi(2k)=2 π ( 2 k ) = 2 . Then π ( k ) = m − k + 1 \pi(k)=m-k+1 π ( k ) = m − k + 1 lies near the right endpoint, while π ( 2 k ) = 2 \pi(2k)=2 π ( 2 k ) = 2 lies near the left endpoint. Thus, as indices increase from k k k to 2 k 2k 2 k , the images must travel back across the interval, with consecutive images at most 2 k 2k 2 k indices apart. We choose them so that they generally move to the left, visiting the unused neighbors along the way, as shown by the orange arrows.
The example illustrates how oscillations reconcile large displacement with small expansion in both directions. Reaching the prescribed image requires moving across the interval, while controlling the inverse expansion requires returning to visit unused points near images already assigned. Thus, a single prescribed displacement constrains the mapping throughout the interval. The permutation described in this toy example is shown in Figure 3 . It achieves expansion O ( k ) O(k) O ( k ) in both directions and displacement Θ ( k 2 ) \Theta(k^2) Θ ( k 2 ) , so its distortion is comparable to its displacement. The full construction and its verification for general odd k k k appear in Appendix § 10 .
In the upper-bound proof, we face two additional difficulties not captured by the toy model:
The points left unmatched by the partial mapping g g g are not equally spaced and have varying gap lengths, so we need to be able to handle variable gap lengths.
The partial mapping g g g may prescribe many displacements (intervals of I \mathcal{I} I ), so we need to allow for more flexible oscillations that can handle more than a single displaced pair, in contrast to the toy example.
In the next subsection, we describe how we overcome these difficulties and conclude the overview.
Implementing the oscillations
We now explain how to adapt the above idea in the random setting. Our goal is to extend the partial mapping g g g on the points whose indices lie in the intervals of I \mathcal{I} I to a bijection from X X X to Y Y Y with small distortion. Let p = C ⌈ log n ⌉ p=C\lceil\log n\rceil p = C ⌈ log n ⌉ , where C C C is a sufficiently large constant, and assume for simplicity that p p p divides n n n . Partition X X X and Y Y Y into M = n / p M=n/p M = n / p blocks of p p p consecutive points each. We think of each block as a single object and seek a permutation of the blocks, mapping the points increasingly within each matched pair of blocks. All intervals in Proposition 3.2 can in fact be chosen as unions of whole blocks, including both the intervals in I \mathcal{I} I and the parts J 1 , … , J ℓ J_1,\ldots,J_\ell J 1 , … , J ℓ . Consequently, the partial mapping g g g already corresponds to a partial permutation of the blocks. We call the remaining blocks free .
Formally, the block mapping induced by a permutation π : [ M ] → [ M ] \pi:[M]\to[M] π : [ M ] → [ M ] sends
x ( i − 1 ) p + r ⟼ y ( π ( i ) − 1 ) p + r ( i ∈ [ M ] , r ∈ [ p ] ) . x_{(i-1)p+r}\longmapsto y_{(\pi(i)-1)p+r}\qquad(i\in[M],\ r\in[p]). x ( i − 1 ) p + r ⟼ y ( π ( i ) − 1 ) p + r ( i ∈ [ M ] , r ∈ [ p ]) .
As explained, we can ensure that the partial mapping g g g from Proposition 3.2 is induced by a partial permutation π ′ \pi' π ′ on blocks, hence we seek an extension of π ′ \pi' π ′ to a permutation whose induced block mapping has small expansion in both directions.
For any block mapping extending g g g , consecutive points within the same block already satisfy the desired expansion bound. Indeed, if the block is not free, the block mapping agrees with g g g , so property (4) of Proposition 3.2 will bound the expansion by O ~ ( n 1 / 3 ) \widetilde{O}(n^{1/3}) O ( n 1/3 ) . If the block is free, property (2) implies that it contains no tiny gaps, so consecutive points are separated by at least Ω ~ ( n − 4 / 3 ) \widetilde{\Omega}(n^{-4/3}) Ω ( n − 4/3 ) . Their images are consecutive points in another free block, and hence are at distance at most O ~ ( 1 / n ) \widetilde{O}(1/n) O ( 1/ n ) . Therefore, such pairs also satisfy the desired expansion bound O ~ ( n 1 / 3 ) \widetilde{O}(n^{1/3}) O ( n 1/3 ) . Thus, in our choice of π \pi π we only need to handle consecutive points lying in different blocks.
Unfortunately, oscillations cannot be applied directly in this setting. Ignoring polylogarithmic factors, a jump of approximately n 1 / 3 n^{1/3} n 1/3 indices may separate the images of two consecutive points by approximately n − 2 / 3 n^{-2/3} n − 2/3 . To allow such jumps at arbitrary boundaries between free blocks while keeping the expansion in each direction within our n 1 / 3 n^{1/3} n 1/3 budget, we would need every such gap to be at least approximately 1 / n 1/n 1/ n . However, gaps between adjacent free blocks can be substantially smaller. This is how difficulty (1) arises in the block formulation.
To resolve this, we apply a preparation step that enlarges the gaps between adjacent free blocks while preserving g g g . Formally, our rearrangement lemma, Lemma 6.26, shows that, with high probability, there exist configurations X ′ = { x 1 ′ < ⋯ < x n ′ } X'=\{x'_1<\cdots<x'_n\} X ′ = { x 1 ′ < ⋯ < x n ′ } and Y ′ = { y 1 ′ < ⋯ < y n ′ } Y'=\{y'_1<\cdots<y'_n\} Y ′ = { y 1 ′ < ⋯ < y n ′ } , and bijections R X : X → X ′ R_X:X\to X' R X : X → X ′ and R Y : Y → Y ′ R_Y:Y\to Y' R Y : Y → Y ′ , satisfying:
For every i ∈ U i\in U i ∈ U , R X ( x i ) = x i ′ = x i R_X(x_i)=x'_i=x_i R X ( x i ) = x i ′ = x i and R Y ( y i ) = y i ′ = y i R_Y(y_i)=y'_i=y_i R Y ( y i ) = y i ′ = y i . Thus the properties of g g g guaranteed by Proposition 3.2 also hold for X ′ X' X ′ and Y ′ Y' Y ′ .
Whenever blocks i i i and i + 1 i+1 i + 1 are free, their boundary gap satisfies x i p + 1 ′ − x i p ′ = Ω ~ ( 1 / n ) x'_{ip+1}-x'_{ip}=\widetilde{\Omega}(1/n) x i p + 1 ′ − x i p ′ = Ω ( 1/ n ) , and similarly in Y ′ Y' Y ′ .
The maps R X , R Y R_X,R_Y R X , R Y and their inverses have Lipschitz constants O ~ ( 1 ) \widetilde{O}(1) O ( 1 ) .
This preparation step resolves the difficulty (1) at only a polylogarithmic cost in distortion.
We now briefly explain how we address difficulty (2). To overcome the difficulty, we use a more flexible oscillation construction, Lemma 6.29, that extends the prescribed partial permutation π ′ \pi' π ′ to a permutation π \pi π of [ M ] [M] [ M ] with expansion O ~ ( n 1 / 3 ) \widetilde{O}(n^{1/3}) O ( n 1/3 ) in both directions.
Property (1) of Proposition 3.2 bounds the size of each part J j J_j J j by 3 H = O ~ ( n 2 / 3 ) 3H=\widetilde{O}(n^{2/3}) 3 H = O ( n 2/3 ) , limiting the distance the oscillations should travel. It also bounds the length of each prescribed interval by O ~ ( n 1 / 3 ) \widetilde{O}(n^{1/3}) O ( n 1/3 ) . This second bound controls the size of a jump when an oscillation passes across a prescribed interval.
Property (5) ensures that every prescribed interval and its partner lie in the same part J j J_j J j , allowing us to construct the extension separately within each part. Finally, consecutive intervals of I \mathcal{I} I are separated by at least D / 2 D/2 D /2 indices, and the same margin separates them from the endpoints of their containing part. This separation provides sufficiently long sequences of free blocks in which to carry out the oscillations. The full lemma appears in § 6 as Lemma 6.29.
The Equally Spaced Model
Throughout this section, we equip [ n ] [n] [ n ] with the metric d ( i , j ) = ∣ i − j ∣ d(i,j)=|i-j| d ( i , j ) = ∣ i − j ∣ . We call this the equally spaced model and refer to its elements as indices. Under the identification i ↔ i / n i \leftrightarrow i/n i ↔ i / n , a permutation π \pi π corresponds to the bijection i / n ↦ π ( i ) / n i/n \mapsto\pi(i)/n i / n ↦ π ( i ) / n of the equally spaced points { i / n : i ∈ [ n ] } ⊆ [ 0 , 1 ] \{i/n:i\in[n]\}\subseteq[0,1] { i / n : i ∈ [ n ]} ⊆ [ 0 , 1 ] . The distortion of the permutation (w.r.t. d d d ) and the map on { i / n : i ∈ [ n ] } ⊆ [ 0 , 1 ] \{i/n:i\in[n]\}\subseteq[0,1] { i / n : i ∈ [ n ]} ⊆ [ 0 , 1 ] associated with it, is the same by Observation 2.2 . Thus, the model captures the geometry at the typical spacing scale 1 / n 1/n 1/ n and above. There are three advantages to working in this model.
It is deterministic and isolates the permutation induced by a bijection f : X → Y f:X\to Y f : X → Y , where X , Y ∼ O X,Y\sim\mathcal{O} X , Y ∼ O , from irregularities in the consecutive gaps.
It makes the displacement obstruction transparent: if π ( i ) \pi(i) π ( i ) is far from both i i i and n + 1 − i n+1-i n + 1 − i (in which case we say that π \pi π displaces i i i ), then π \pi π must have large distortion. This explains why matching a microscopic gap of X X X to a distant microscopic gap of Y Y Y typically introduces large distortion.
It isolates the role of oscillations, which allow large displacement while controlling distortion. This mechanism is transparent when all adjacent gaps have the same length; its adaptation to random gaps requires a multiscale construction.
The only isometries of this space are the identity id ( i ) = i \operatorname{id}(i)=i id ( i ) = i and the reversal ρ ( i ) = n + 1 − i \rho(i)=n+1-i ρ ( i ) = n + 1 − i .
Distortion, displacement. For a permutation π ∈ S n \pi\in S_n π ∈ S n we define:
Lip ( π ) = max 1 ≤ i < n ∣ π ( i + 1 ) − π ( i ) ∣ , Lip ( π − 1 ) = max 1 ≤ j < n ∣ π − 1 ( j + 1 ) − π − 1 ( j ) ∣ . \operatorname{Lip}(\pi)=\max_{1\leq i<n}|\pi(i+1)-\pi(i)|,\qquad\operatorname{Lip}(\pi^{-1})=\max_{1\leq j<n}|\pi^{-1}(j+1)-\pi^{-1}(j)|. Lip ( π ) = 1 ≤ i < n max ∣ π ( i + 1 ) − π ( i ) ∣ , Lip ( π − 1 ) = 1 ≤ j < n max ∣ π − 1 ( j + 1 ) − π − 1 ( j ) ∣.
The distortion of a permutation π \pi π is defined by:
dist ( π ) = Lip ( π ) Lip ( π − 1 ) . \operatorname{dist}(\pi)=\operatorname{Lip}(\pi)\operatorname{Lip}(\pi^{-1}). dist ( π ) = Lip ( π ) Lip ( π − 1 ) .
The displacement of a permutation π \pi π is defined by:
disp ( π ) : = min τ ∈ { id , ρ } max i ∈ [ n ] ∣ π ( i ) − τ ( i ) ∣ . \operatorname{disp}(\pi):=\min_{\tau\in\{\operatorname{id},\rho\}}\max_{i\in[n]}|\pi(i)-\tau(i)|. disp ( π ) := τ ∈ { id , ρ } min i ∈ [ n ] max ∣ π ( i ) − τ ( i ) ∣.
The distortion measures how much π \pi π can shrink or expand distances between indices. The displacement measures (in ℓ ∞ \ell^\infty ℓ ∞ ) the distance of π \pi π to some global isometry. The following theorem is key to the proof of the lower bound in the random setting .
Theorem 4.1. There is an absolute constant c > 0 c>0 c > 0 such that every π ∈ S n \pi\in S_n π ∈ S n satisfies
dist ( π ) ≥ c disp ( π ) . \operatorname{dist}(\pi)\geq c\operatorname{disp}(\pi). dist ( π ) ≥ c disp ( π ) .
We first begin with the following claim.
Claim 4.2. Let π ∈ S n \pi\in S_n π ∈ S n and r > 0 r > 0 r > 0 . Suppose that Lip ( π ) < r \operatorname{Lip}(\pi) < r Lip ( π ) < r and disp ( π ) > 20 r \operatorname{disp}(\pi) > 20r disp ( π ) > 20 r . Then there exists i ∈ [ n ] i \in[n] i ∈ [ n ] such that
∣ π ( i ) − i ∣ ≥ r and ∣ π ( i ) − ρ ( i ) ∣ ≥ r . (3) |\pi(i)-i| \ge r \qquad\text{and} \qquad|\pi(i)-\rho(i)| \ge r. \tag*{(3)} ∣ π ( i ) − i ∣ ≥ r and ∣ π ( i ) − ρ ( i ) ∣ ≥ r . (3)
Proof. Suppose towards a contradiction that no index i ∈ [ n ] i \in[n] i ∈ [ n ] satisfies (3) . Label each i ∈ [ n ] i \in[n] i ∈ [ n ] by i d \mathrm{id} id if ∣ π ( i ) − i ∣ < r |\pi(i)-i| < r ∣ π ( i ) − i ∣ < r , and by ρ \rho ρ otherwise. Since no index satisfies (3) , every index i i i has label τ \tau τ then ∣ π ( i ) − τ ( i ) ∣ < r |\pi(i)-\tau(i)| < r ∣ π ( i ) − τ ( i ) ∣ < r .
Let us show that the assumptions force r > 1 r > 1 r > 1 and n > 20 r n > 20r n > 20 r . Indeed, the images of two consecutive points are distinct, so Lip ( π ) ≥ 1 \operatorname{Lip}(\pi) \ge1 Lip ( π ) ≥ 1 and thus r > Lip ( π ) ≥ 1 r > \operatorname{Lip}(\pi) \ge1 r > Lip ( π ) ≥ 1 . Also, every two points of [ n ] [n] [ n ] are at distance at most n − 1 n-1 n − 1 , so disp ( π ) ≤ n − 1 \operatorname{disp}(\pi) \le n-1 disp ( π ) ≤ n − 1 and hence n > disp ( π ) > 20 r n > \operatorname{disp}(\pi) > 20r n > disp ( π ) > 20 r .
Denote t = 2 r t=2r t = 2 r and divide [ n ] [n] [ n ] into
L = [ 1 , n 2 − t ] , M = ( n 2 − t , n 2 + t ) , R = [ n 2 + t , n ] , L=\left[1,\frac{n}{2}-t\right], \qquad M=\left(\frac{n}{2}-t,\frac{n}{2}+t\right), \qquad R=\left[\frac{n}{2}+t,n\right], L = [ 1 , 2 n − t ] , M = ( 2 n − t , 2 n + t ) , R = [ 2 n + t , n ] ,
where the intervals are understood as subsets of [ n ] [n] [ n ] .
Let us show that the label is constant on each of L L L and R R R . Otherwise, there exist consecutive indices i i i and i + 1 i+1 i + 1 in either L L L or R R R that have different labels. Thus, one of π ( i ) \pi(i) π ( i ) and π ( i + 1 ) \pi(i+1) π ( i + 1 ) is within distance r r r of an identity image and the other is within distance r r r of a reversal image. In either case, among the identity image and the reversal image one lies in L L L and the other lies in R R R . As the distance between L L L and R R R is at least 2 t 2t 2 t we have
∣ π ( i + 1 ) − π ( i ) ∣ ≥ 2 t − 2 r = 2 r , |\pi(i+1)-\pi(i)| \ge2t-2r=2r, ∣ π ( i + 1 ) − π ( i ) ∣ ≥ 2 t − 2 r = 2 r ,
which contradicts Lip ( π ) < r \operatorname{Lip}(\pi) < r Lip ( π ) < r .
Now, let us show that all indices of both L L L and R R R have the same label. Wlog, assume towards contradiction that L L L is labeled by i d \mathrm{id} id and R R R by ρ \rho ρ . Then π ( L ∪ R ) ⊆ { j ∈ [ n ] : j < n / 2 } \pi(L\cup R) \subseteq\{j \in[n]:j<n/2\} π ( L ∪ R ) ⊆ { j ∈ [ n ] : j < n /2 } . Indeed, for i ∈ L i \in L i ∈ L we have π ( i ) < i + r ≤ n / 2 − r \pi(i) < i+r \le n/2-r π ( i ) < i + r ≤ n /2 − r and for i ∈ R i \in R i ∈ R we have π ( i ) < ρ ( i ) + r ≤ n / 2 + 1 − r < n / 2 \pi(i) < \rho(i)+r \le n/2+1-r<n/2 π ( i ) < ρ ( i ) + r ≤ n /2 + 1 − r < n /2 . Thus, the size of the image π ( L ∪ R ) \pi(L\cup R) π ( L ∪ R ) is less than n / 2 n/2 n /2 . As π \pi π is bijection, we conclude ∣ L ∪ R ∣ < n / 2 |L\cup R|<n/2 ∣ L ∪ R ∣ < n /2 . On the other hand, from the definitions of L L L and R R R we have
∣ L ∪ R ∣ ≥ n − 4 r − 1 > ⌈ n 2 ⌉ , |L\cup R| \ge n-4r-1 > \left\lceil\frac{n}{2}\right\rceil, ∣ L ∪ R ∣ ≥ n − 4 r − 1 > ⌈ 2 n ⌉ ,
which is a contradiction.
Let τ ∈ { i d , ρ } \tau\in\{\mathrm{id},\rho\} τ ∈ { id , ρ } be the common global label of all indexes from L ∪ R L\cup R L ∪ R . For every i ∈ M i \in M i ∈ M ,
∣ i − ρ ( i ) ∣ ≤ 2 ∣ i − n 2 ∣ + 1 < 4 r + 1. |i-\rho(i)| \le2\left|i-\frac{n}{2}\right|+1<4r+1. ∣ i − ρ ( i ) ∣ ≤ 2 i − 2 n + 1 < 4 r + 1.
For i ∈ M i \in M i ∈ M , let τ i ∈ { i d , ρ } \tau_i \in\{\mathrm{id},\rho\} τ i ∈ { id , ρ } denote the map given in the label of i i i . Then, for all i ∈ M i \in M i ∈ M , we have
∣ π ( i ) − τ ( i ) ∣ ≤ ∣ π ( i ) − τ i ( i ) ∣ + ∣ τ i ( i ) − τ ( i ) ∣ ≤ ∣ π ( i ) − τ i ( i ) ∣ + ∣ i d ( i ) − ρ ( i ) ∣ < r + ( 4 r + 1 ) ≤ 5 r + 1. \begin{aligned}
|\pi(i)-\tau(i)| &\le|\pi(i)-\tau_i(i)|+|\tau_i(i)-\tau(i)|\\
&\le|\pi(i)-\tau_i(i)|+|\mathrm{id}(i)-\rho(i)|\\
&<r+(4r+1)\le5r+1.
\end{aligned} ∣ π ( i ) − τ ( i ) ∣ ≤ ∣ π ( i ) − τ i ( i ) ∣ + ∣ τ i ( i ) − τ ( i ) ∣ ≤ ∣ π ( i ) − τ i ( i ) ∣ + ∣ id ( i ) − ρ ( i ) ∣ < r + ( 4 r + 1 ) ≤ 5 r + 1.
Also, from the definition of τ \tau τ , we have ∣ π ( i ) − τ ( i ) ∣ ≤ r |\pi(i)-\tau(i)|\le r ∣ π ( i ) − τ ( i ) ∣ ≤ r for all i ∈ L ∪ R i \in L\cup R i ∈ L ∪ R . Thus, for all i i i we have ∣ π ( i ) − τ ( i ) ∣ < 6 r |\pi(i)-\tau(i)|<6r ∣ π ( i ) − τ ( i ) ∣ < 6 r and, hence, disp ( π ) < 6 r \operatorname{disp}(\pi)<6r disp ( π ) < 6 r — a contradiction with disp ( π ) > 20 r \operatorname{disp}(\pi)>20r disp ( π ) > 20 r .
Claim 4.3. Let π ∈ S n \pi\in S_n π ∈ S n , let i , j ∈ [ n ] i,j \in[n] i , j ∈ [ n ] , and let d > 0 d > 0 d > 0 . If π ( i ) = j \pi(i)=j π ( i ) = j and
∣ j − i ∣ ≥ d and ∣ j − ρ ( i ) ∣ ≥ d , |j-i| \ge d \qquad\text{and}\qquad|j-\rho(i)| \ge d, ∣ j − i ∣ ≥ d and ∣ j − ρ ( i ) ∣ ≥ d , then dist ( π ) ≥ d / 200 \operatorname{dist}(\pi) \ge d/200 dist ( π ) ≥ d /200 .
Proof. For d ≤ 200 d \le200 d ≤ 200 , the claim follows from dist ( π ) ≥ 1 \operatorname{dist}(\pi) \ge1 dist ( π ) ≥ 1 , so assume that d > 200 d > 200 d > 200 . At least one of i i i and j j j lies at distance at least d / 4 d/4 d /4 from both endpoints of [ n ] [n] [ n ] : if i i i is close to an endpoint, the two separation assumptions force j j j away from both endpoints. Replacing π \pi π by π − 1 \pi^{-1} π − 1 and interchanging i i i and j j j , if necessary, preserves the assumptions and the distortion. We may therefore assume that
i − 1 ≥ d 4 and n − i ≥ d 4 . i-1 \ge\frac{d}{4} \qquad\text{and}\qquad n-i \ge\frac{d}{4}. i − 1 ≥ 4 d and n − i ≥ 4 d . Put K = ⌊ d / 100 ⌋ K=\lfloor d/100\rfloor K = ⌊ d /100 ⌋ . For each integer 0 ≤ k ≤ K 0 \le k \le K 0 ≤ k ≤ K , let C k C_k C k denote the coloring of the following three intervals
[ 1 , i − 1 ] , [ i , i + k ] , [ i + k + 1 , n ] [1,i-1], \qquad[i,i+k], \qquad[i+k+1,n] [ 1 , i − 1 ] , [ i , i + k ] , [ i + k + 1 , n ] into red, yellow, and blue respectively. Transfer this coloring through π \pi π by setting
C k π ( x ) = C k ( π − 1 ( x ) ) . C_k^\pi(x)=C_k(\pi^{-1}(x)). C k π ( x ) = C k ( π − 1 ( x )) . We call two consecutive points colored red and blue, in either order, a collision. A collision in C k π C_k^\pi C k π implies Lip ( π − 1 ) ≥ k + 2 \operatorname{Lip}(\pi^{-1}) \ge k+2 Lip ( π − 1 ) ≥ k + 2 , since then there exist a red index r r r and a blue index b b b satisfying
∣ π ( r ) − π ( b ) ∣ = 1. |\pi(r)-\pi(b)|=1. ∣ π ( r ) − π ( b ) ∣ = 1. Let us show that the coloring C 0 π C_0^\pi C 0 π has a collision. Otherwise, j = π ( i ) j=\pi(i) j = π ( i ) is the unique yellow point, and the intervals on its two sides are monochromatic and have opposite colors. If the [ 1 , j − 1 ] [1,j-1] [ 1 , j − 1 ] interval is red then counting the number of red points gives j − 1 = i − 1 j-1=i-1 j − 1 = i − 1 , and hence j = i j=i j = i . If it is blue then counting the number of blue points gives j − 1 = n − i j-1=n-i j − 1 = n − i , and hence j = ρ ( i ) j=\rho(i) j = ρ ( i ) . Both conclusions contradict the assumptions of the claim.
If C K π C_K^\pi C K π has a collision, then Lip ( π − 1 ) ≥ K + 2 > d / 100 \operatorname{Lip}(\pi^{-1}) \ge K+2>d/100 Lip ( π − 1 ) ≥ K + 2 > d /100 , and the claim follows. Otherwise, let 1 ≤ k ≤ K 1 \le k \le K 1 ≤ k ≤ K be the minimal index for which C k π C_k^\pi C k π has no collision. Then C k − 1 π C_{k-1}^\pi C k − 1 π has a collision, and therefore
Lip ( π − 1 ) ≥ k + 1. \operatorname{Lip}(\pi^{-1}) \ge k+1. Lip ( π − 1 ) ≥ k + 1. We now show that Lip ( π ) ≥ d / 6 k \operatorname{Lip}(\pi) \ge d/6k Lip ( π ) ≥ d /6 k . The reader is instructed to see Figure 4 for illustration. Let [ a , b ] [a,b] [ a , b ] be the smallest interval containing all yellow points of C k π C_k^\pi C k π , and put w = b − a w=b-a w = b − a . Since j = π ( i ) j=\pi(i) j = π ( i ) is yellow, we have j ∈ [ a , b ] j \in[a,b] j ∈ [ a , b ] . Let us show that w > d / 6 w>d/6 w > d /6 . Suppose otherwise. The red and blue classes contain, respectively, i − 1 i-1 i − 1 and n − i − k n-i-k n − i − k points, and both quantities are larger than d / 5 d/5 d /5 . Since [ a , b ] [a,b] [ a , b ] contains fewer than d / 5 d/5 d /5 points, both red and blue colors of C k π C_k^\pi C k π occur outside of it. Since there are no collisions the two intervals outside of [ a , b ] [a,b] [ a , b ] are monochromatic and have opposite colors. If the [ 1 , a − 1 ] [1,a-1] [ 1 , a − 1 ] interval is red, counting red points gives a − 1 ≤ i − 1 ≤ b a-1 \le i-1 \le b a − 1 ≤ i − 1 ≤ b . Since j ∈ [ a , b ] j \in[a,b] j ∈ [ a , b ] , this yields ∣ j − i ∣ ≤ w + 1 < d |j-i| \le w+1<d ∣ j − i ∣ ≤ w + 1 < d , a contradiction. If the [ 1 , a − 1 ] [1,a-1] [ 1 , a − 1 ] interval is blue, then a − 1 ≤ n − i − k ≤ b a-1 \le n-i-k \le b a − 1 ≤ n − i − k ≤ b . Since ρ ( i ) = ( n − i − k ) + k + 1 \rho(i)=(n-i-k)+k+1 ρ ( i ) = ( n − i − k ) + k + 1 , we obtain
∣ j − ρ ( i ) ∣ ≤ w + k + 1 ≤ d 6 + d 100 + 1 < d , |j-\rho(i)| \le w+k+1 \le\frac{d}{6}+\frac{d}{100}+1<d, ∣ j − ρ ( i ) ∣ ≤ w + k + 1 ≤ 6 d + 100 d + 1 < d , again a contradiction. Thus w > d / 6 w>d/6 w > d /6 .
The yellow interval in C k C_k C k has diameter k k k , so Lip ( π ) ≥ w / k \operatorname{Lip}(\pi) \ge w/k Lip ( π ) ≥ w / k . Combining this with Lip ( π − 1 ) ≥ k + 1 \operatorname{Lip}(\pi^{-1}) \ge k+1 Lip ( π − 1 ) ≥ k + 1 gives
dist ( π ) ≥ w k ( k + 1 ) > w > d 6 . \operatorname{dist}(\pi) \ge\frac{w}{k}(k+1)>w>\frac{d}{6}. dist ( π ) ≥ k w ( k + 1 ) > w > 6 d . In particular, dist ( π ) ≥ d / 200 \operatorname{dist}(\pi) \ge d/200 dist ( π ) ≥ d /200 . ∎
Figure 4. The colorings C k C_k C k and C k π C^\pi_k C k π , where k k k is minimal such that C k π C^\pi_k C k π has no collision. Set j = π ( i ) j=\pi(i) j = π ( i ) , and let [ a , b ] [a,b] [ a , b ] be the smallest interval containing all yellow points of C k π C^\pi_k C k π , with width w = b − a w=b-a w = b − a . Since j ∈ [ a , b ] j\in[a,b] j ∈ [ a , b ] , the absence of collisions, together with the sizes of the red and blue classes, forces w = Ω ( d ) w=\Omega(d) w = Ω ( d ) ; Moreover, the yellow interval in C k C_k C k has diameter k k k , so Lip ( π ) ≥ w / k \operatorname{Lip}(\pi)\ge w/k Lip ( π ) ≥ w / k , while the collision in C k − 1 π C^\pi_{k-1} C k − 1 π gives Lip ( π − 1 ) ≥ k + 1 \operatorname{Lip}(\pi^{-1})\ge k+1 Lip ( π − 1 ) ≥ k + 1 . Hence dist ( π ) = Ω ( d ) \operatorname{dist}(\pi)=\Omega(d) dist ( π ) = Ω ( d ) .
Proof of Theorem 4.1. Put D = disp ( π ) D=\operatorname{disp}(\pi) D = disp ( π ) . If D = 0 D=0 D = 0 , there is nothing to prove. If Lip ( π ) ≥ D / 40 \operatorname{Lip}(\pi)\ge D/40 Lip ( π ) ≥ D /40 , then
dist ( π ) = Lip ( π ) Lip ( π − 1 ) ≥ D 40 . \operatorname{dist}(\pi)=\operatorname{Lip}(\pi)\operatorname{Lip}(\pi^{-1})\ge\frac{D}{40}. dist ( π ) = Lip ( π ) Lip ( π − 1 ) ≥ 40 D .
Suppose therefore that Lip ( π ) < D / 40 \operatorname{Lip}(\pi)<D/40 Lip ( π ) < D /40 , and put r = D / 40 r=D/40 r = D /40 . Since D > 20 r D>20r D > 20 r , Claim 4.2 gives an index i ∈ [ n ] i\in[n] i ∈ [ n ] satisfying
∣ π ( i ) − i ∣ ≥ r and ∣ π ( i ) − ρ ( i ) ∣ ≥ r . |\pi(i)-i|\ge r \qquad\text{and}\qquad|\pi(i)-\rho(i)|\ge r. ∣ π ( i ) − i ∣ ≥ r and ∣ π ( i ) − ρ ( i ) ∣ ≥ r .
Applying Claim 4.3 with j = π ( i ) j=\pi(i) j = π ( i ) and d = r d=r d = r gives
dist ( π ) ≥ r 200 = D 8000 . \operatorname{dist}(\pi)\ge\frac{r}{200}=\frac{D}{8000}. dist ( π ) ≥ 200 r = 8000 D .
Thus the theorem holds with c = 1 / 8000 c=1/8000 c = 1/8000 . □
Proof of the Lower Bound
Set T = ⌊ n 2 / 3 ⌋ T=\lfloor n^{2/3}\rfloor T = ⌊ n 2/3 ⌋ . Let X , Y ⊆ R X,Y\subseteq\mathbb{R} X , Y ⊆ R be sets of n n n points. For each Z ∈ { X , Y } Z\in\{X,Y\} Z ∈ { X , Y } , let λ Z \lambda_Z λ Z be an index satisfying Δ λ Z Z = Δ min Z \Delta_{\lambda_Z}^Z=\Delta_{\min}^Z Δ λ Z Z = Δ m i n Z , and define
I Z ρ = [ λ Z − T , λ Z + T + 1 ] ∩ [ n − 1 ] , I Z ρ = [ n − λ Z − T , n − λ Z + T + 1 ] ∩ [ n − 1 ] . I_Z^\rho=[\lambda_Z-T,\lambda_Z+T+1]\cap[n-1],\qquad I_Z^\rho=[n-\lambda_Z-T,n-\lambda_Z+T+1]\cap[n-1]. I Z ρ = [ λ Z − T , λ Z + T + 1 ] ∩ [ n − 1 ] , I Z ρ = [ n − λ Z − T , n − λ Z + T + 1 ] ∩ [ n − 1 ] .
The following claim summarizes several useful properties that hold whp for both X X X and Y Y Y .
Claim 5.1. Let X , Y ∼ O X,Y\sim\mathcal{O} X , Y ∼ O be iid. There exist universal constants c , C > 0 c,C>0 c , C > 0 such that, whp:
For each Z = { z 1 < ⋯ < z n } ∈ { X , Y } Z=\{z_1<\cdots<z_n\}\in\{X,Y\} Z = { z 1 < ⋯ < z n } ∈ { X , Y } ,
z j − z i ≥ j − i C n whenever j − i ≥ C log n , max i ∈ [ n − 1 ] Δ i Z ≤ C log n n . z_j-z_i\ge\frac{j-i}{Cn}\qquad\text{whenever }j-i\ge C\log n,\qquad\max_{i\in[n-1]}\Delta_i^Z\le\frac{C\log n}{n}. z j − z i ≥ C n j − i whenever j − i ≥ C log n , i ∈ [ n − 1 ] max Δ i Z ≤ n C log n .
For each τ ∈ { i d , ρ } \tau\in\{\mathrm{id},\rho\} τ ∈ { id , ρ } ,
Δ min X , Δ min Y ≤ C log n n 2 , min i ∈ I X τ Δ i Y ≥ c n T log n , min i ∈ I Y τ Δ i X ≥ c n T log n . \Delta_{\min}^{X},\Delta_{\min}^{Y}\leq\frac{C\log n}{n^{2}},
\qquad
\min_{i\in I_{X}^{\tau}}\Delta_{i}^{Y}\geq\frac{c}{nT\log n},
\qquad
\min_{i\in I_{Y}^{\tau}}\Delta_{i}^{X}\geq\frac{c}{nT\log n}. Δ m i n X , Δ m i n Y ≤ n 2 C log n , i ∈ I X τ min Δ i Y ≥ n T log n c , i ∈ I Y τ min Δ i X ≥ n T log n c .
Proof. The first property follows from Claim 2.7 and Corollary 2.8 . For the second property we apply Claim 2.9 and Corollary 2.10 , which says that in a window of length T T T , the minimum gap in Y Y Y is ≥ Ω ( 1 / n T log n ) \geq\Omega(1/nT\log n) ≥ Ω ( 1/ n T log n ) whp. The crucial point is that even though the interval of indices I X τ I_{X}^{\tau} I X τ is random – it is independent of the randomness of Y Y Y , hence by first fixing X X X we may apply Claim 2.9 and Corollary 2.10 for Y Y Y , and vice versa.
Let E \mathcal{E} E denote the event that both properties from Claim 5.1 hold. We will show that whenever E \mathcal{E} E holds, any mapping between X X X and Y Y Y has sufficiently large distortion. We begin with the following claim, which allows us to lower bound the Lipschitz constant of an arbitrary bijection f : X → Y f:X\to Y f : X → Y by the Lipschitz constant of its associated permutation.
Claim 5.2. Let X = { x 1 < ⋯ < x n ≤ 1 } X=\{x_{1}<\cdots<x_{n}\leq1\} X = { x 1 < ⋯ < x n ≤ 1 } and Y = { 0 ≤ y 1 < ⋯ < y n ≤ 1 } Y=\{0\leq y_{1}<\cdots<y_{n}\leq1\} Y = { 0 ≤ y 1 < ⋯ < y n ≤ 1 } be sets of points satisfying E \mathcal{E} E . Let f : X → Y f:X\to Y f : X → Y be a bijection and let π ∈ S n \pi\in S_{n} π ∈ S n be the permutation satisfying f ( x i ) = y π ( i ) f(x_{i})=y_{\pi(i)} f ( x i ) = y π ( i ) for all i ∈ [ n ] i\in[n] i ∈ [ n ] . Then, uniformly over f f f ,
Lip ( f ) ≳ Lip ( π ) log n . \operatorname{Lip}(f)\gtrsim\frac{\operatorname{Lip}(\pi)}{\log n}. Lip ( f ) ≳ log n Lip ( π ) .
Proof. Put r = Lip ( π ) r=\operatorname{Lip}(\pi) r = Lip ( π ) , and choose i ∈ [ n − 1 ] i\in[n-1] i ∈ [ n − 1 ] such that ∣ π ( i + 1 ) − π ( i ) ∣ = r |\pi(i+1)-\pi(i)|=r ∣ π ( i + 1 ) − π ( i ) ∣ = r . If r ≥ C log n r\geq C\log n r ≥ C log n , then Corollary 2.5 and the first property from E \mathcal{E} E give
Lip ( f ) ≥ ∣ f ( x i + 1 ) − f ( x i ) ∣ x i + 1 − x i = ∣ y π ( i + 1 ) − y π ( i ) ∣ x i + 1 − x i ≥ r / ( C n ) C log n / n = r C 2 log n . \operatorname{Lip}(f)\geq\frac{|f(x_{i+1})-f(x_i)|}{x_{i+1}-x_i}
=\frac{|y_{\pi(i+1)}-y_{\pi(i)}|}{x_{i+1}-x_i}
\geq\frac{r/(Cn)}{C\log n/n}
=\frac{r}{C^{2}\log n}. Lip ( f ) ≥ x i + 1 − x i ∣ f ( x i + 1 ) − f ( x i ) ∣ = x i + 1 − x i ∣ y π ( i + 1 ) − y π ( i ) ∣ ≥ C log n / n r / ( C n ) = C 2 log n r .
If r < C log n r<C\log n r < C log n , then a constant lower bound on Lip ( f ) \operatorname{Lip}(f) Lip ( f ) suffices. For all sufficiently large n n n , the first property of E \mathcal{E} E applied to i = 1 i=1 i = 1 and j = n j=n j = n gives
diam ( Y ) = y n − y 1 ≥ n − 1 C n ≥ 1 2 C . \operatorname{diam}(Y)=y_{n}-y_{1}\geq\frac{n-1}{Cn}\geq\frac{1}{2C}. diam ( Y ) = y n − y 1 ≥ C n n − 1 ≥ 2 C 1 .
Since f f f is onto and diam ( X ) ≤ 1 \operatorname{diam}(X)\leq1 diam ( X ) ≤ 1 , it follows that
Lip ( f ) ≥ diam ( Y ) diam ( X ) ≥ 1 2 C ≥ r 2 C 2 log n . \operatorname{Lip}(f)\geq\frac{\operatorname{diam}(Y)}{\operatorname{diam}(X)}
\geq\frac{1}{2C}\geq\frac{r}{2C^{2}\log n}. Lip ( f ) ≥ diam ( X ) diam ( Y ) ≥ 2 C 1 ≥ 2 C 2 log n r .
Proof of the lower bound in Theorem 1.1 . Due to Claim 5.1, we can assume that X X X and Y Y Y satisfy E \mathcal{E} E . Consider any bijection f : X → Y f:X\to Y f : X → Y and let π ∈ S n \pi\in S_{n} π ∈ S n be the permutation satisfying f ( x i ) = y π ( i ) f(x_{i})=y_{\pi(i)} f ( x i ) = y π ( i ) for all i ∈ [ n ] i\in[n] i ∈ [ n ] . Denote D = disp ( π ) D=\operatorname{disp}(\pi) D = disp ( π ) . Recall that T = ⌊ n 2 / 3 ⌋ T=\lfloor n^{2/3}\rfloor T = ⌊ n 2/3 ⌋ .
Suppose first that D ≤ T D\leq T D ≤ T . Choose τ ∈ { i d , ρ } \tau\in\{\mathrm{id},\rho\} τ ∈ { id , ρ } such that ∣ π ( i ) − τ ( i ) ∣ ≤ T |\pi(i)-\tau(i)|\leq T ∣ π ( i ) − τ ( i ) ∣ ≤ T for every i ∈ [ n ] i\in[n] i ∈ [ n ] . We have
Lip ( f ) ≥ ∣ f ( x λ X + 1 ) − f ( x λ X ) ∣ x λ X + 1 − x λ X = ∣ y π ( λ X + 1 ) − y π ( λ X ) ∣ x λ X + 1 − x λ X = ∣ y π ( λ X + 1 ) − y π ( λ X ) ∣ Δ min X ≥ min i ∈ I X τ Δ i Y Δ min X . (4) \operatorname{Lip}(f)\geq
\frac{|f(x_{\lambda_{X}+1})-f(x_{\lambda_{X}})|}{x_{\lambda_{X}+1}-x_{\lambda_{X}}}
=
\frac{|y_{\pi(\lambda_{X}+1)}-y_{\pi(\lambda_{X})}|}{x_{\lambda_{X}+1}-x_{\lambda_{X}}}
=
\frac{|y_{\pi(\lambda_{X}+1)}-y_{\pi(\lambda_{X})}|}{\Delta_{\min}^{X}}
\geq
\frac{\min_{i\in I_{X}^{\tau}}\Delta_{i}^{Y}}{\Delta_{\min}^{X}}. \tag*{(4)} Lip ( f ) ≥ x λ X + 1 − x λ X ∣ f ( x λ X + 1 ) − f ( x λ X ) ∣ = x λ X + 1 − x λ X ∣ y π ( λ X + 1 ) − y π ( λ X ) ∣ = Δ m i n X ∣ y π ( λ X + 1 ) − y π ( λ X ) ∣ ≥ Δ m i n X min i ∈ I X τ Δ i Y . (4)
where the last inequality holds, as both π ( λ X ) \pi(\lambda X) π ( λ X ) and π ( λ X + 1 ) \pi(\lambda X+1) π ( λ X + 1 ) belong to I X τ I_X^\tau I X τ . Due to the second property of E \mathcal{E} E we further have
≳ n T ( log n ) 2 . (5) \gtrsim\frac{n}{T(\log n)^2}. \tag*{(5)} ≳ T ( log n ) 2 n . (5)
Observe that the permutation π − 1 \pi^{-1} π − 1 is within distance T T T of the same τ \tau τ , hence disp ( π − 1 ) = disp ( π ) \operatorname{disp}(\pi^{-1})=\operatorname{disp}(\pi) disp ( π − 1 ) = disp ( π ) . Therefore, we may apply the same argument to f − 1 : Y → X f^{-1}:Y\to X f − 1 : Y → X . This gives Lip ( f − 1 ) ≳ n / ( T ( log n ) 2 ) \operatorname{Lip}(f^{-1})\gtrsim n/(T(\log n)^2) Lip ( f − 1 ) ≳ n / ( T ( log n ) 2 ) , and hence
dist ( f ) ≳ n 2 T 2 ( log n ) 4 . \operatorname{dist}(f)\gtrsim\frac{n^2}{T^2(\log n)^4}. dist ( f ) ≳ T 2 ( log n ) 4 n 2 .
Suppose now that D > T D>T D > T . Applying Claim 5.2 to f f f and f − 1 f^{-1} f − 1 , and then applying Theorem 4.1, gives
dist ( f ) ≳ dist ( π ) ( log n ) 2 ≳ D ( log n ) 2 > T ( log n ) 2 . \operatorname{dist}(f)\gtrsim\frac{\operatorname{dist}(\pi)}{(\log n)^2}\gtrsim\frac{D}{(\log n)^2}>\frac{T}{(\log n)^2}. dist ( f ) ≳ ( log n ) 2 dist ( π ) ≳ ( log n ) 2 D > ( log n ) 2 T .
Combining the two cases and using T = ⌊ n 2 / 3 ⌋ T=\lfloor n^{2/3}\rfloor T = ⌊ n 2/3 ⌋ , we obtain
dist ( f ) ≳ min { n 2 T 2 ( log n ) 4 , T ( log n ) 2 } ≳ n 2 / 3 ( log n ) 4 . \operatorname{dist}(f)\gtrsim\min\left\{\frac{n^2}{T^2(\log n)^4},\frac{T}{(\log n)^2}\right\}\gtrsim\frac{n^{2/3}}{(\log n)^4}. dist ( f ) ≳ min { T 2 ( log n ) 4 n 2 , ( log n ) 2 T } ≳ ( log n ) 4 n 2/3 .
Since the two properties hold simultaneously with high probability and f f f was arbitrary, taking the minimum over all bijections proves the theorem. □ \square □
The upper bound
The main goal of this section is to prove the following theorem.
Theorem 6.1 (Upper bound). Let X X X and Y Y Y be independent samples of n n n uniform points in [ 0 , 1 ] [0,1] [ 0 , 1 ] . With high probability,
c ( X , Y ) ≤ n 2 / 3 log O ( 1 ) n . c(X,Y)\le n^{2/3}\log^{O(1)}n. c ( X , Y ) ≤ n 2/3 log O ( 1 ) n .
Set-up
Assume throughout Section 6 that n n n is sufficiently large. Let X = { x 1 < ⋯ < x n } X=\{x_1<\cdots<x_n\} X = { x 1 < ⋯ < x n } and Y = { y 1 < ⋯ < y n } Y=\{y_1<\cdots<y_n\} Y = { y 1 < ⋯ < y n } be independent samples of n n n uniform points in [ 0 , 1 ] [0,1] [ 0 , 1 ] . For Z = { z 1 < ⋯ < z n } ∈ { X , Y } Z=\{z_1<\cdots<z_n\}\in\{X,Y\} Z = { z 1 < ⋯ < z n } ∈ { X , Y } and I ⊆ [ n ] I\subseteq[n] I ⊆ [ n ] , write Z I = { z i : i ∈ I } Z_I=\{z_i:i\in I\} Z I = { z i : i ∈ I } . By Proposition 2.1, in order to show Theorem 6.1 we may assume that X X X and Y Y Y are defined as follows. Let ( E i X ) i = 0 n (E_i^X)_{i=0}^n ( E i X ) i = 0 n and ( E i Y ) i = 0 n (E_i^Y)_{i=0}^n ( E i Y ) i = 0 n be independent Exp ( 1 ) \operatorname{Exp}(1) Exp ( 1 ) variables. Define the gaps of consecutive points of X X X and Y Y Y by: for Z ∈ { X , Y } Z\in\{X,Y\} Z ∈ { X , Y } , set z 1 = E 0 Z / Σ Z z_1=E_0^Z/\Sigma_Z z 1 = E 0 Z / Σ Z
Δ i Z : = z i + 1 − z i = E i Z Σ Z ( i ∈ [ n − 1 ] ) , Σ Z = ∑ j = 0 n E j Z . \Delta_i^Z:=z_{i+1}-z_i=\frac{E_i^Z}{\Sigma_Z}\quad(i\in[n-1]),\qquad\Sigma_Z=\sum_{j=0}^{n}E_j^Z. Δ i Z := z i + 1 − z i = Σ Z E i Z ( i ∈ [ n − 1 ]) , Σ Z = j = 0 ∑ n E j Z .
By Claim 2.6, n / 2 ≤ Σ X , Σ Y ≤ 2 n n/2\le\Sigma_X,\Sigma_Y\le2n n /2 ≤ Σ X , Σ Y ≤ 2 n with high probability. We now introduce the main parameters for the proof, and explain their role. Put
L = ⌈ log n ⌉ , H = ⌈ n 2 / 3 L 4 ⌉ , p = C 0 L , L=\lceil\log n\rceil,\qquad H=\lceil n^{2/3}L^4\rceil,\qquad p=C_0L, L = ⌈ log n ⌉ , H = ⌈ n 2/3 L 4 ⌉ , p = C 0 L ,
where C 0 C_0 C 0 is a sufficiently large even positive integer.
Between X X X and Y Y Y , we will construct a specific kind of mapping, called a block mapping . The parameter p = Θ ( log n ) p=\Theta(\log n) p = Θ ( log n ) is the block size: we partition each sample into blocks of p p p consecutive points, with a possibly shorter last block. A block mapping sends each full block of X X X increasingly onto a full block of Y Y Y , keeping its points together.
To construct the block mapping, we will also find a partition [ n ] = J 1 ∪ ˙ ⋯ ∪ ˙ J m [n] = J_1 \mathbin{\dot\cup} \cdots\mathbin{\dot\cup} J_m [ n ] = J 1 ∪ ˙ ⋯ ∪ ˙ J m into intervals of between H H H and 3 H 3H 3 H consecutive indices. Thus each part has Θ ( n 2 / 3 log 4 n ) \Theta(n^{2/3}\log^4 n) Θ ( n 2/3 log 4 n ) indices. For each J i J_i J i , we will construct a suitable matching between the corresponding blocks of X J i X_{J_i} X J i and Y J i Y_{J_i} Y J i separately.
We now introduce and explain a second set of parameters, used to design the block mapping within each J i J_i J i . Set
s = L 3 , D = A p ⌈ H ⌉ , t = 1 + ⌈ log s D ⌉ s = L^3,\qquad D = A p\lceil\sqrt{H}\rceil,\qquad t = 1 + \lceil\log_s D\rceil s = L 3 , D = A p ⌈ H ⌉ , t = 1 + ⌈ log s D ⌉
where A A A is a sufficiently large absolute integer.
Since we study distortion up to polylogarithmic factors, we record exponential gaps E i Z E_i^Z E i Z only up to a factor of s s s . Concretely, for x > 0 x > 0 x > 0 , define e s ( x ) = max { 0 , ⌊ log s ( 1 / x ) ⌋ } e_s(x) = \max\{0,\lfloor\log_s(1/x)\rfloor\} e s ( x ) = max { 0 , ⌊ log s ( 1/ x )⌋} . The integer e s ( x ) e_s(x) e s ( x ) records the scale of x x x , with larger exponents corresponding to smaller gaps. For the gaps of our samples, write e i Z = e s ( E i Z ) e_i^Z = e_s(E_i^Z) e i Z = e s ( E i Z ) for Z ∈ { X , Y } Z \in\{X,Y\} Z ∈ { X , Y } and i ∈ [ n − 1 ] i \in[n-1] i ∈ [ n − 1 ] . Then
e i Z = a ⟺ s − ( a + 1 ) < E i Z ≤ s − a ( a ≥ 1 ) . e_i^Z = a \quad\Longleftrightarrow\quad s^{-(a+1)} < E_i^Z \le s^{-a}\qquad(a \ge1). e i Z = a ⟺ s − ( a + 1 ) < E i Z ≤ s − a ( a ≥ 1 ) .
Thus each positive exponent a a a represents a class of gaps lying between two consecutive powers of 1 / s 1/s 1/ s . Within this class, replacing an exponential gap by the upper endpoint s − a s^{-a} s − a changes its value by at most a factor of s s s . In particular, two exponential gaps with the same positive exponent differ by at most this factor. All larger gaps, those with E i Z > s − 1 E_i^Z > s^{-1} E i Z > s − 1 , are grouped into the remaining class e i Z = 0 e_i^Z = 0 e i Z = 0 .
As explained in Section 3, consecutive points of X X X or Y Y Y whose gap is smaller than n − 4 / 3 n^{-4/3} n − 4/3 require extra care. We use t t t to define the precise cutoff in terms of the gap exponents.
Definition 6.2 (Tiny indices). An index i ∈ [ n − 1 ] i \in[n-1] i ∈ [ n − 1 ] is tiny if max { e i X , e i Y } ≥ t \max\{e_i^X,e_i^Y\} \ge t max { e i X , e i Y } ≥ t , or equivalently, if min { E i X , E i Y } ≤ s − t \min\{E_i^X,E_i^Y\} \le s^{-t} min { E i X , E i Y } ≤ s − t .
For Z ∈ { X , Y } Z \in\{X,Y\} Z ∈ { X , Y } , the condition E i Z ≤ s − t E_i^Z \le s^{-t} E i Z ≤ s − t is equivalent to Δ i Z ≤ 1 / ( Σ Z s t ) \Delta_i^Z \le1/(\Sigma_Zs^t) Δ i Z ≤ 1/ ( Σ Z s t ) . On the event n / 2 ≤ Σ X , Σ Y ≤ 2 n n/2 \le\Sigma_X,\Sigma_Y \le2n n /2 ≤ Σ X , Σ Y ≤ 2 n , this distance threshold is within a factor of 2 of 1 / ( n s t ) 1/(ns^t) 1/ ( n s t ) . The bounds 1 / ( n s 2 D ) ≤ 1 / ( n s t ) ≤ 1 / ( n s D ) 1/(ns^{2D}) \le1/(ns^t) \le1/(ns^D) 1/ ( n s 2 D ) ≤ 1/ ( n s t ) ≤ 1/ ( n s D ) , together with s = L 3 s = L^3 s = L 3 and D = Θ ( n 1 / 3 L 3 ) D = \Theta(n^{1/3}L^3) D = Θ ( n 1/3 L 3 ) , show that this threshold is smaller than n − 4 / 3 n^{-4/3} n − 4/3 by a polylogarithmic factor.
We partition tiny indices into components . Namely, we consider the partition of tiny indices into maximal subsets (which we call components ) such that every pair of tiny indices i < j i < j i < j with j − i ≤ D j-i \le D j − i ≤ D lies in the same component. The low-distortion map f : X → Y f : X \to Y f : X → Y we aim to construct will satisfy the following property: For every component C C C there exists an integer k ∈ [ − n , n ] k \in[-n,n] k ∈ [ − n , n ] such that, for all tiny indices i ∈ C i \in C i ∈ C the map f : X → Y f : X \to Y f : X → Y satisfies f ( x i ) = y i + k f(x_i)=y_{i+k} f ( x i ) = y i + k and f ( x i + 1 ) = y i + 1 + k f(x_{i+1})=y_{i+1+k} f ( x i + 1 ) = y i + 1 + k . In other words, f f f applies the same translation to all indices from C C C when transferring them from X X X to Y Y Y . Without this constraint, two nearby tiny gaps could be matched to distant gap in J i J_i J i , stretching the points between them too much. We aim to first build a partial mapping between X X X and Y Y Y that maps some intervals I = { I 1 , … , I 2 ℓ } \mathcal{I}=\{I_1,\ldots,I_{2\ell}\} I = { I 1 , … , I 2 ℓ } to each other in a way that all components of tiny indices are contained in these intervals. During this step, we will also require the intervals from I \mathcal{I} I to be separated by at least D / 2 D/2 D /2 points. Then, the choice D = A p ⌈ H ⌉ = Θ ( n 1 / 3 L 3 ) D = A p\lceil\sqrt{H}\rceil= \Theta(n^{1/3}L^3) D = A p ⌈ H ⌉ = Θ ( n 1/3 L 3 ) will give us enough space between the intervals to extend f f f to the whole set X X X while keeping its distortion small.
Matching the tiny gaps
Definition 6.3 (Blocks). Set M = ⌈ n / p ⌉ M=\lceil n/p\rceil M = ⌈ n / p ⌉ . For j ∈ [ M ] j \in[M] j ∈ [ M ] , define
B j = { ( j − 1 ) p + 1 , … , min { j p , n } } , B j Z = Z B j ( Z ∈ { X , Y } ) . B_j=\{(j-1)p+1,\ldots,\min\{jp,n\}\},\qquad B_j^Z=Z_{B_j}\quad(Z\in\{X,Y\}). B j = {( j − 1 ) p + 1 , … , min { j p , n }} , B j Z = Z B j ( Z ∈ { X , Y }) .
The blocks B 1 , … , B M B_1,\ldots,B_M B 1 , … , B M partition [ n ] [n] [ n ] into consecutive intervals, each of size p p p except possibly the last. We call B M X B_M^X B M X and B M Z B_M^Z B M Z the terminal blocks, even when they have p p p elements.
Definition 6.4 (Partial block mappings). Let S , T ⊆ [ M − 1 ] S,T\subseteq[M-1] S , T ⊆ [ M − 1 ] , and let σ : S → T \sigma:S\to T σ : S → T be a bijection. The partial block mapping induced by σ \sigma σ is the map f : ⋃ j ∈ S B j X → ⋃ j ∈ T B j Y f:\bigcup_{j\in S}B_j^X\to\bigcup_{j\in T}B_j^Y f : ⋃ j ∈ S B j X → ⋃ j ∈ T B j Y that sends each B j X B_j^X B j X , j ∈ S j\in S j ∈ S , increasingly onto B σ ( j ) Y B_{\sigma(j)}^Y B σ ( j ) Y . Explicitly,
f ( x ( j − 1 ) p + a ) = y ( σ ( j ) − 1 ) p + a ( j ∈ S , a ∈ [ p ] ) . f\left(x_{(j-1)p+a}\right)=y_{(\sigma(j)-1)p+a}\qquad(j\in S,\ a\in[p]). f ( x ( j − 1 ) p + a ) = y ( σ ( j ) − 1 ) p + a ( j ∈ S , a ∈ [ p ]) . Given a partial block mapping f f f induced by σ : S → T \sigma:S\to T σ : S → T , we say that B j X B_j^X B j X is free if j ∉ S j\notin S j ∈ / S , and that B j Y B_j^Y B j Y is free if j ∉ T j\notin T j ∈ / T . We call σ \sigma σ a partial involution if S = T S=T S = T and σ 2 = id S \sigma^2=\operatorname{id}_S σ 2 = id S .
Given a permutation π \pi π of [ M ] [M] [ M ] extending σ \sigma σ and satisfying π ( M ) = M \pi(M)=M π ( M ) = M , define F : X → Y F:X\to Y F : X → Y by mapping each B j X B_j^X B j X increasingly onto B π ( j ) Y B_{\pi(j)}^Y B π ( j ) Y . We call F F F the block extension of f f f induced by π \pi π .
Proposition 6.5 (Matching the tiny gaps). With high probability , there exist ℓ ∈ N 0 \ell\in\mathbb{N}_0 ℓ ∈ N 0 , a family I = { I 1 , I ‾ 1 , … , I ℓ , I ‾ ℓ } \mathcal{I}=\{I_1,\overline{I}_1,\ldots,I_\ell,\overline{I}_\ell\} I = { I 1 , I 1 , … , I ℓ , I ℓ } of 2 ℓ 2\ell 2 ℓ nonempty intervals in [ n ] [n] [ n ] , where each interval is a union of nonterminal blocks, and a partial block mapping f : X U → Y U f:X_U\to Y_U f : X U → Y U , where U = ⋃ I ∈ I I U=\bigcup_{I\in\mathcal{I}}I U = ⋃ I ∈ I I , with the following properties.
The mapping f f f is induced by a partial involution and maps X I i X_{I_i} X I i in increasing order onto Y I ‾ i Y_{\overline{I}_i} Y I i for every i ∈ [ ℓ ] i\in[\ell] i ∈ [ ℓ ] .
For every tiny index j j j , the pair { j , j + 1 } \{j,j+1\} { j , j + 1 } lies in some I ∈ I I\in\mathcal{I} I ∈ I .
For every I ∈ I I\in\mathcal{I} I ∈ I ,
max { Lip ( f ∣ X I ) , Lip ( f − 1 ∣ Y I ) } = O ( L s t + 1 ) . \max\left\{\operatorname{Lip}(f|_{X_I}),\operatorname{Lip}(f^{-1}|_{Y_I})\right\}=O(Ls^{t+1}). max { Lip ( f ∣ X I ) , Lip ( f − 1 ∣ Y I ) } = O ( L s t + 1 ) . The intervals in I \mathcal{I} I are pairwise separated by at least D / 2 D/2 D /2 , and each has size O ( L D ) O(LD) O ( L D ) .
There is a partition [ n ] = J 1 ∪ ˙ ⋯ ∪ ˙ J m [n]=J_1\mathbin{\dot\cup}\cdots\mathbin{\dot\cup}J_m [ n ] = J 1 ∪ ˙ ⋯ ∪ ˙ J m into consecutive intervals with H ≤ ∣ J j ∣ ≤ 3 H H\leq|J_j|\leq3H H ≤ ∣ J j ∣ ≤ 3 H for every j ∈ [ m ] j\in[m] j ∈ [ m ] . For every i ∈ [ ℓ ] i\in[\ell] i ∈ [ ℓ ] , there is j ∈ [ m ] j\in[m] j ∈ [ m ] such that both I i I_i I i and I ‾ i \overline{I}_i I i lie in J j J_j J j and are at distance at least D / 2 D/2 D /2 from each endpoint of J j J_j J j .
Patterns and covers
Recall that e i Z e_i^Z e i Z denotes the discretized exponent of the i i i -th exponential gap in Z Z Z . We introduce the definition of a rooted pattern. The rooted pattern is a formal tuple that we use to record some of the gap exponents e i X e_i^X e i X and e i Y e_i^Y e i Y and some distances between them. Namely, the rooted pattern consists of the root r ∈ [ n ] r\in[n] r ∈ [ n ] , and the pattern, which is a tuple of distances d 1 , … , d k − 1 ∈ [ n ] d_1,\ldots,d_{k-1}\in[n] d 1 , … , d k − 1 ∈ [ n ] and of integers α 1 X , α 1 Y , … , α k X , α k Y \alpha_1^X,\alpha_1^Y,\ldots,\alpha_k^X,\alpha_k^Y α 1 X , α 1 Y , … , α k X , α k Y for some k > 0 k>0 k > 0 . Roughly, the rooted pattern occurs if for all i ∈ { 0 , … , k − 1 } i\in\{0,\ldots,k-1\} i ∈ { 0 , … , k − 1 } the discretized exponential gaps e r + d 1 + ⋯ + d i X e_{r+d_1+\cdots+d_i}^X e r + d 1 + ⋯ + d i X and e r + d 1 + ⋯ + d i Y e_{r+d_1+\cdots+d_i}^Y e r + d 1 + ⋯ + d i Y coincide with α i + 1 X \alpha_{i+1}^X α i + 1 X and α i + 1 Y \alpha_{i+1}^Y α i + 1 Y . In that sense, a rooted pattern that occurs records several exponential gaps and the number of indices between them. Notice that the root of the pattern represents the index of the first gap of the pattern. Let us first define patterns without roots, then rooted patterns.
Definition 6.6 (Patterns). A pattern of size k ≥ 1 k\geq1 k ≥ 1 is a sequence
P = ( α 1 X , α 1 Y , d 1 , α 2 X , α 2 Y , … , d k − 1 , α k X , α k Y ) . P=\left(\alpha_1^X,\alpha_1^Y,d_1,\alpha_2^X,\alpha_2^Y,\ldots,d_{k-1},\alpha_k^X,\alpha_k^Y\right). P = ( α 1 X , α 1 Y , d 1 , α 2 X , α 2 Y , … , d k − 1 , α k X , α k Y ) . where α j X , α j Y ∈ N 0 \alpha_j^X,\alpha_j^Y\in\mathbb{N}_0 α j X , α j Y ∈ N 0 are the prescribed gap exponents at the j j j -th recorded position, and each d j d_j d j is a positive integer giving the difference between the j j j -th and ( j + 1 ) (j+1) ( j + 1 ) -st recorded indices. We write ∣ P ∣ = k |P|=k ∣ P ∣ = k .
We now introduce some terminology for patterns that will be used later in the proof. Let P P P be a pattern of size k k k , and set a 1 = 0 a_1=0 a 1 = 0 and a j = ∑ h < j d h a_j=\sum_{h<j}d_h a j = ∑ h < j d h for 2 ≤ j ≤ k 2\le j\le k 2 ≤ j ≤ k .
The shape of P P P is ( d 1 , … , d k − 1 ) (d_1,\ldots,d_{k-1}) ( d 1 , … , d k − 1 ) . For k = 1 k=1 k = 1 , the shape is empty.
The span of P P P is span ( P ) = a k = ∑ j = 1 k − 1 d j \operatorname{span}(P)=a_k=\sum_{j=1}^{k-1}d_j span ( P ) = a k = ∑ j = 1 k − 1 d j . For k = 1 k=1 k = 1 , the span is zero.
A rooted pattern is a pair P = ( r , P ) \mathbf{P}=(r,P) P = ( r , P ) , where r ∈ [ n − 1 ] r\in[n-1] r ∈ [ n − 1 ] and r + span ( P ) ≤ n − 1 r+\operatorname{span}(P)\le n-1 r + span ( P ) ≤ n − 1 . We call r r r its root.
The support of P = ( r , P ) \mathbf{P}=(r,P) P = ( r , P ) is supp ( P ) = { r + a 1 , r + a 2 , … , r + a k } \operatorname{supp}(\mathbf{P})=\{r+a_1,r+a_2,\ldots,r+a_k\} supp ( P ) = { r + a 1 , r + a 2 , … , r + a k } , the set of recorded gap indices.
The interval of P = ( r , P ) \mathbf{P}=(r,P) P = ( r , P ) is int ( r , P ) = [ r , r + span ( P ) ] ∩ Z \operatorname{int}(r,P)=[r,r+\operatorname{span}(P)]\cap\mathbb{Z} int ( r , P ) = [ r , r + span ( P )] ∩ Z . It contains all gap indices between the first and last recorded positions.
We say that ( r , P ) (r,P) ( r , P ) occurs if e r + a j Z = α j Z e^{Z}_{r+a_j}=\alpha^{Z}_j e r + a j Z = α j Z for every j ∈ [ k ] j\in[k] j ∈ [ k ] and Z ∈ { X , Y } Z\in\{X,Y\} Z ∈ { X , Y } . We say that P P P occurs in an interval I ⊆ [ n ] I\subseteq[n] I ⊆ [ n ] at root r r r if ( r , P ) (r,P) ( r , P ) occurs and [ r , r + span ( P ) + 1 ] ∩ Z ⊆ I [r,r+\operatorname{span}(P)+1]\cap\mathbb{Z}\subseteq I [ r , r + span ( P ) + 1 ] ∩ Z ⊆ I .
Definition 6.7 (Covers). Let P P P and P ‾ \overline{P} P be patterns of size k k k with the same shape, and denote their exponent pairs by ( α j X , α j Y ) (\alpha^X_j,\alpha^Y_j) ( α j X , α j Y ) and ( α ‾ j X , α ‾ j Y ) (\overline{\alpha}^X_j,\overline{\alpha}^Y_j) ( α j X , α j Y ) , respectively. We say that P ‾ \overline{P} P covers P P P if α ‾ j X ≤ α j X \overline{\alpha}^X_j\le\alpha^X_j α j X ≤ α j X and α ‾ j Y ≤ α j Y \overline{\alpha}^Y_j\le\alpha^Y_j α j Y ≤ α j Y for every j ∈ [ k ] j\in[k] j ∈ [ k ] . The cover has low expansion if
( α j Y − t ) + ≤ α ‾ j X ≤ α j Y , ( α j X − t ) + ≤ α ‾ j Y ≤ α j X ( j ∈ [ k ] ) , (\alpha^Y_j-t)_+\le\overline{\alpha}^X_j\le\alpha^Y_j,\qquad(\alpha^X_j-t)_+\le\overline{\alpha}^Y_j\le\alpha^X_j\qquad(j\in[k]), ( α j Y − t ) + ≤ α j X ≤ α j Y , ( α j X − t ) + ≤ α j Y ≤ α j X ( j ∈ [ k ]) , where ( a ) + = max { a , 0 } (a)_+=\max\{a,0\} ( a ) + = max { a , 0 } .
Before we justify the name low-expansion for covers, we record the following regularity bounds on the gap sizes Δ i Z = E i Z / Σ Z \Delta^Z_i=E^Z_i/\Sigma_Z Δ i Z = E i Z / Σ Z .
Observation 6.8. With high probability, simultaneously for Z ∈ { X , Y } Z\in\{X,Y\} Z ∈ { X , Y } , we have n / 2 ≤ Σ Z ≤ 2 n n/2\le\Sigma_Z\le2n n /2 ≤ Σ Z ≤ 2 n and E i Z ≤ 4 L E^Z_i\le4L E i Z ≤ 4 L for every 0 ≤ i ≤ n 0\le i\le n 0 ≤ i ≤ n . In particular, every gap is at most 8 L / n 8L/n 8 L / n .
Proof. By Claim 2.6, n / 2 ≤ Σ Z ≤ 2 n n/2\le\Sigma_Z\le2n n /2 ≤ Σ Z ≤ 2 n for both samples X X X and Y Y Y with high probability. Since P ( E i Z > 4 L ) = e − 4 L \mathbb{P}(E^Z_i>4L)=e^{-4L} P ( E i Z > 4 L ) = e − 4 L , a union bound over the 2 ( n + 1 ) 2(n+1) 2 ( n + 1 ) exponentials gives failure probability 2 ( n + 1 ) e − 4 L = o ( 1 ) 2(n+1)e^{-4L}=o(1) 2 ( n + 1 ) e − 4 L = o ( 1 ) for the remaining bound.
Consider a rooted pattern P = ( r , P ) \mathbf{P}=(r,P) P = ( r , P ) of size k k k . Recall, if P \mathbf{P} P occurs, it records the gap exponents in X X X and Y Y Y at k k k indices i 1 < ⋯ < i k i_1<\cdots<i_k i 1 < ⋯ < i k . In our strategy, we aim to match the indices i 1 , i 1 + 1 , i 2 , i 2 + 1 , … , i k + 1 i_1,i_1+1,i_2,i_2+1,\ldots,i_k+1 i 1 , i 1 + 1 , i 2 , i 2 + 1 , … , i k + 1 in increasing order to the corresponding indices of some other rooted pattern P ‾ = ( r ‾ , P ‾ ) \overline{\mathbf{P}}=(\overline{r},\overline{P}) P = ( r , P ) with the same size and shape as P \mathbf{P} P . We aim to choose P ‾ \overline{\mathbf{P}} P to be a low-expansion cover of P \mathbf{P} P located near P \mathbf{P} P , i.e. so that ∣ r ‾ − r ∣ |\overline{r}-r| ∣ r − r ∣ is small. As a consequence, the natural mapping from P \mathbf{P} P to P ‾ \overline{\mathbf{P}} P will have a small Lipschitz constant due to Observation 6.8. In addition, we will need to extend this natural mapping to some intervals I ⊇ int ( P ) I\supseteq\operatorname{int}(P) I ⊇ int ( P ) and I ‾ ⊇ int ( P ‾ ) \overline{I}\supseteq\operatorname{int}(\overline{P}) I ⊇ int ( P ) in a way that keeps the Lipschitz constants small. We aim to show that the increasing extension also has small Lipschitz constant as long as all tiny gaps of I I I and I ‾ \overline{I} I are recorded by the rooted patterns P \mathbf{P} P and P ‾ \overline{\mathbf{P}} P and some other natural conditions hold.
Claim 6.9. With high probability, the following holds. Let I , I ‾ ⊆ [ n ] I,\overline{I}\subseteq[n] I , I ⊆ [ n ] be intervals of consecutive point indices with ∣ I ∣ = ∣ I ‾ ∣ |I|=|\overline{I}| ∣ I ∣ = ∣ I ∣ . Suppose that P \mathbf{P} P occurs in I I I at root r r r , and a low-expansion cover P ‾ \overline{\mathbf{P}} P of P \mathbf{P} P occurs in I ‾ \overline{I} I at root r ‾ \overline{r} r , where
r − min I = r ‾ − min I ‾ . r-\min I=\overline{r}-\min\overline{I}. r − min I = r − min I . @page 21/47 starts_mid=0 ends_mid=1
Suppose that all tiny gap indices i i i with i , i + 1 ∈ I i, i+1 \in I i , i + 1 ∈ I lie in supp ( r , P ) \operatorname{supp}(r,P) supp ( r , P ) , and similarly for I ‾ \overline{I} I and ( r ‾ , P ‾ ) (\overline{r},\overline{P}) ( r , P ) . Then the increasing3 bijections X I → Y I ‾ X_I \to Y_{\overline{I}} X I → Y I and X I ‾ → Y I X_{\overline{I}} \to Y_I X I → Y I , together with their inverses, have Lipschitz constant O ( L s t + 1 ) = O ~ ( n 1 / 3 ) O(Ls^{t+1})=\widetilde{O}(n^{1/3}) O ( L s t + 1 ) = O ( n 1/3 ) .7
Proof. Assume that the event described in Observation 6.8 holds. The definition of e i Z e_i^Z e i Z and the bound E i Z ≤ 4 L E_i^Z \le4L E i Z ≤ 4 L give
s − ( e i Z + 1 ) < E i Z ≤ 4 L s − e i Z ( i ∈ [ n − 1 ] , Z ∈ { X , Y } ) , (6) s^{-(e_i^Z+1)} < E_i^Z \le4Ls^{-e_i^Z} \qquad(i \in[n-1],\ Z \in\{X,Y\}), \tag*{(6)} s − ( e i Z + 1 ) < E i Z ≤ 4 L s − e i Z ( i ∈ [ n − 1 ] , Z ∈ { X , Y }) , (6)
including when e i Z = 0 e_i^Z=0 e i Z = 0 .
Fix intervals and patterns satisfying the assumptions. For i , i + 1 ∈ I i,i+1 \in I i , i + 1 ∈ I , put i ‾ = i + r ‾ − r \overline{i}=i+\overline{r}-r i = i + r − r . The condition on the roots implies that the increasing bijection X I → Y I ‾ X_I \to Y_{\overline{I}} X I → Y I sends x i , x i + 1 x_i,x_{i+1} x i , x i + 1 to y i ‾ , y i ‾ + 1 y_{\overline{i}},y_{\overline{i}+1} y i , y i + 1 .
First, assume that i ∉ supp ( P ) i \notin\operatorname{supp}(P) i ∈ / supp ( P ) . Then i ‾ ∉ supp ( P ‾ ) \overline{i} \notin\operatorname{supp}(\overline{P}) i ∈ / supp ( P ) as well, since the patterns P P P and P ‾ \overline{P} P have the same shape. Thus, neither i i i nor i ‾ \overline{i} i are tiny, since all tiny indices inside the two intervals are recorded in supp ( P ) \operatorname{supp}(P) supp ( P ) and supp ( P ‾ ) \operatorname{supp}(\overline{P}) supp ( P ) , by assumption. Consequently, due to the event from Observation 6.8 , both gaps Δ i X , Δ i ‾ Y \Delta_i^X,\Delta_{\overline{i}}^Y Δ i X , Δ i Y lie between 1 / ( 2 n s t ) 1/(2ns^t) 1/ ( 2 n s t ) and 8 L / n 8L/n 8 L / n , so their ratio in either direction is at most 16 L s t 16Ls^t 16 L s t .
Now suppose that i ∈ supp P i \in\operatorname{supp} P i ∈ supp P . The low-expansion condition gives ∣ e i Y − e i X ∣ ≤ t |e_i^Y-e_i^X|\le t ∣ e i Y − e i X ∣ ≤ t . We have
Δ i ‾ Y Δ i X = E i ‾ Y / Σ Y E i X / Σ X = E i ‾ Y E i X Σ X Σ Y ≤ 16 L s t + 1 . \frac{\Delta_{\overline{i}}^Y}{\Delta_i^X}
=\frac{E_{\overline{i}}^Y/\Sigma_Y}{E_i^X/\Sigma_X}
=\frac{E_{\overline{i}}^Y}{E_i^X}\frac{\Sigma_X}{\Sigma_Y}
\le16Ls^{t+1}. Δ i X Δ i Y = E i X / Σ X E i Y / Σ Y = E i X E i Y Σ Y Σ X ≤ 16 L s t + 1 .
The inequality follows from (6) and the bounds on Σ X , Σ Y \Sigma_X,\Sigma_Y Σ X , Σ Y in Observation 6.8 . The same bound holds for the ratio in the other direction.
Since the bijection is increasing, it preserves the order of the points and sends consecutive points to consecutive points. By Claim 2.4 , the Lipschitz constants of the bijection and its inverse are each attained on a pair of consecutive points, so both are at most 16 L s t + 1 16Ls^{t+1} 16 L s t + 1 .
The same argument applies to both increasing bijections and their inverses, giving the claimed bounds, since L s t + 1 = O ~ ( n 1 / 3 ) Ls^{t+1}=\widetilde{O}(n^{1/3}) L s t + 1 = O ( n 1/3 ) .
We also record a uniform bound on the number of tiny indices in short intervals.
Claim 6.10. With high probability, every interval J ⊆ [ n − 1 ] J\subseteq[n-1] J ⊆ [ n − 1 ] of length at most 3 H 3H 3 H contains at most 12 H / s t 12H/s^t 12 H / s t tiny indices.
Proof. Each index is tiny with probability $1-e^{-2s^{-t}}\le 2/s^t$, independently of the other indices. Thus, for each fixed interval J J J of length at most 3 H 3H 3 H , the number N J N_J N J of tiny indices is binomial with mean at most 6 H / s t 6H/s^t 6 H / s t . A binomial tail bound and a union bound over the at most n 2 n^2 n 2 intervals give, for an absolute constant $c>0,
P ( ∃ J ⊆ [ n − 1 ] , ∣ J ∣ ≤ 3 H : N J > 12 H / s t ) ≤ n 2 e − c H / s t = o ( 1 ) , \mathbb{P}\left(\exists J\subseteq[n-1],\ |J|\le3H:\ N_J>12H/s^t\right)\le n^2e^{-cH/s^t}=o(1), P ( ∃ J ⊆ [ n − 1 ] , ∣ J ∣ ≤ 3 H : N J > 12 H / s t ) ≤ n 2 e − cH / s t = o ( 1 ) ,
where J J J ranges over intervals and we used H / s t ≫ log n H/s^t\gg\log n H / s t ≫ log n .
Properties of typical patterns
To prove Proposition 6.5, we will choose pairs of intervals to which Claim 6.9 applies. In particular, to apply the claim, the corresponding patterns must record all tiny gaps inside these intervals. We also require the intervals to be separated by at least D / 2 D/2 D /2 gap indices for later stages of the proof. Consequently, we have the following definition.
Definition 6.11 (Components). A component (of X X X and Y Y Y ) is a nonempty subset C = { c 1 < ⋯ < c k } C = \{c_1 < \cdots< c_k\} C = { c 1 < ⋯ < c k } of [ n − 1 ] [n-1] [ n − 1 ] such that:
The index c i c_i c i is tiny for every i ∈ [ k ] i \in[k] i ∈ [ k ] .
c i + 1 − c i ≤ D c_{i+1} - c_i \le D c i + 1 − c i ≤ D for every i ∈ [ k − 1 ] i \in[k-1] i ∈ [ k − 1 ] .
Every tiny index in [ min C − D , max C + D ] [\min C-D,\max C+D] [ min C − D , max C + D ] belongs to C C C .
For a component C = { c 1 < ⋯ < c k } C = \{c_1 < \cdots< c_k\} C = { c 1 < ⋯ < c k } , let P C P_C P C be the pattern with α j Z = e c j Z \alpha_j^Z=e_{c_j}^Z α j Z = e c j Z for j ∈ [ k ] j \in[k] j ∈ [ k ] , Z ∈ { X , Y } Z \in\{X,Y\} Z ∈ { X , Y } , and d j = c j + 1 − c j d_j=c_{j+1}-c_j d j = c j + 1 − c j for j ∈ [ k − 1 ] j \in[k-1] j ∈ [ k − 1 ] .
We call an occurrence ( r , P ) (r,P) ( r , P ) isolated if every tiny index within distance D D D of its gap-index interval int ( r , P ) \operatorname{int}(r,P) int ( r , P ) belongs to its support.
For each component C C C , the pattern P C P_C P C occurs at root min C \min C min C , with support C C C and gap-index interval [ min C , max C ] [\min C,\max C] [ min C , max C ] . By Item 3, this occurrence is isolated.
We first show that occurring patterns whose recorded indices are tiny and sufficiently close have at most L L L recorded positions.
Claim 6.12. Let P = ( α 1 X , α 1 Y , d 1 , α 2 X , α 2 Y , … , d k − 1 , α k X , α k Y ) P=(\alpha_1^X,\alpha_1^Y,d_1,\alpha_2^X,\alpha_2^Y,\ldots,d_{k-1},\alpha_k^X,\alpha_k^Y) P = ( α 1 X , α 1 Y , d 1 , α 2 X , α 2 Y , … , d k − 1 , α k X , α k Y ) be a pattern, and assume that
d j ∈ [ 12 D ] d_j \in[12D] d j ∈ [ 12 D ] for every j ∈ [ k − 1 ] j \in[k-1] j ∈ [ k − 1 ] ;
max { α j X , α j Y } ≥ t \max\{\alpha_j^X,\alpha_j^Y\} \ge t max { α j X , α j Y } ≥ t for every j ∈ [ k ] j \in[k] j ∈ [ k ] .
Then, with high probability, no such pattern P P P with k > L k>L k > L occurs at an admissible root.
Proof. For each i ∈ [ n − 1 ] i \in[n-1] i ∈ [ n − 1 ] , the independence of E i X E_i^X E i X and E i Y E_i^Y E i Y gives P ( i is tiny ) = 1 − e − 2 s − t ≤ 2 s − t \mathbb{P}(i\text{ is tiny})=1-e^{-2s^{-t}}\le2s^{-t} P ( i is tiny ) = 1 − e − 2 s − t ≤ 2 s − t .
It suffices to consider k = L + 1 k=L+1 k = L + 1 ; otherwise, take the subpattern consisting of the first L + 1 L+1 L + 1 positions of P P P .
We now take a union bound over all such patterns. There are 12 D 12D 12 D choices for each d j d_j d j , hence at most ( 12 D ) L (12D)^L ( 12 D ) L shapes. There are at most n n n choices for the root r r r . For a fixed shape and an admissible root, the probability that all L + 1 L+1 L + 1 recorded gap indices are tiny is at most ( 2 s − t ) L + 1 (2s^{-t})^{L+1} ( 2 s − t ) L + 1 , by independence across these indices. Therefore, by a union bound, the probability that one of these patterns occurs is at most
n ( 12 D ) L ( 2 s − t ) L + 1 ≤ 2 n s − t ( 24 / s ) L = o ( 1 ) , n(12D)^L(2s^{-t})^{L+1}\le2ns^{-t}(24/s)^L=o(1), n ( 12 D ) L ( 2 s − t ) L + 1 ≤ 2 n s − t ( 24/ s ) L = o ( 1 ) , where we used D / s t ≤ 1 / s D/s^t\le1/s D / s t ≤ 1/ s , s = L 3 s=L^3 s = L 3 , and L = ⌈ log n ⌉ L=\lceil\log n\rceil L = ⌈ log n ⌉ . Hence, with high probability, none of the aforementioned patterns occurs, completing the proof. □
Let P \mathcal{P} P be the family of patterns P P P of size 1 ≤ k ≤ L 1\le k\le L 1 ≤ k ≤ L with d j ≤ D d_j\le D d j ≤ D for every j ∈ [ k − 1 ] j\in[k-1] j ∈ [ k − 1 ] , and with 0 ≤ α j X , α j Y ≤ L 0\le\alpha_j^X,\alpha_j^Y\le L 0 ≤ α j X , α j Y ≤ L and max { α j X , α j Y } ≥ t \max\{\alpha_j^X,\alpha_j^Y\}\ge t max { α j X , α j Y } ≥ t for every j ∈ [ k ] j\in[k] j ∈ [ k ] .
Corollary 6.13. With high probability, P C ∈ P P_C\in\mathcal{P} P C ∈ P for every component C C C .
Proof. By Claim 6.12, with high probability every component C C C has size at most L L L : its pattern P C P_C P C occurs and satisfies the hypotheses of that claim. Moreover, for every i ∈ [ n − 1 ] i\in[n-1] i ∈ [ n − 1 ] and Z ∈ { X , Y } Z\in\{X,Y\} Z ∈ { X , Y } ,
P ( e i Z > L ) = P ( E i Z ≤ s − ( L + 1 ) ) ≤ s − ( L + 1 ) ≤ n − 3 , \mathbb{P}(e_i^Z>L)=\mathbb{P}(E_i^Z\le s^{-(L+1)})\le s^{-(L+1)}\le n^{-3}, P ( e i Z > L ) = P ( E i Z ≤ s − ( L + 1 ) ) ≤ s − ( L + 1 ) ≤ n − 3 , for sufficiently large n n n . A union bound over the gap indices in both samples shows that all exponents are at most L L L with probability at least 1 − 2 / n 2 1-2/n^2 1 − 2/ n 2 . The remaining conditions defining P \mathcal{P} P hold by the definition of a component, proving the corollary. □
We aim to bound the size of P \mathcal{P} P . Notice that every pattern P ∈ P P \in\mathcal{P} P ∈ P can be constructed by choosing its size, shape, and exponents. For each size k ∈ [ L ] k \in[L] k ∈ [ L ] , there are at most D k − 1 D^{k-1} D k − 1 choices of the distances and at most ( L + 1 ) 2 k (L+1)^{2k} ( L + 1 ) 2 k choices of the exponents. Hence
∣ P ∣ ≤ ∑ k = 1 L D k − 1 ( L + 1 ) 2 k = exp ( ( 1 / 3 + o ( 1 ) ) L 2 ) , (7) |\mathcal{P}| \le\sum_{k=1}^{L} D^{k-1}(L+1)^{2k} = \exp\left((1/3+o(1))L^2\right), \tag*{(7)} ∣ P ∣ ≤ k = 1 ∑ L D k − 1 ( L + 1 ) 2 k = exp ( ( 1/3 + o ( 1 )) L 2 ) , (7)
where we used log D = ( 1 / 3 + o ( 1 ) ) L \log D=(1/3+o(1))L log D = ( 1/3 + o ( 1 )) L and log ( L + 1 ) = o ( L ) \log(L+1)=o(L) log ( L + 1 ) = o ( L ) .
For each component pattern, we will seek a low-expansion cover with the largest probability of occurring at a fixed root. For a ∈ N 0 a \in\mathbb{N}_0 a ∈ N 0 , let γ a = P ( es ( E ) = a ) \gamma_a=\mathbb{P}(\operatorname{es}(E)=a) γ a = P ( es ( E ) = a ) , where E ∼ Exp ( 1 ) E\sim\operatorname{Exp}(1) E ∼ Exp ( 1 ) . Thus γ 0 = e − 1 / s \gamma_0=e^{-1/s} γ 0 = e − 1/ s and γ a = e − s − ( a + 1 ) − e − s − a \gamma_a=e^{-s^{-(a+1)}}-e^{-s^{-a}} γ a = e − s − ( a + 1 ) − e − s − a for a ≥ 1 a\ge1 a ≥ 1 . For a fixed pattern P P P and a fixed root r r r for which ( r , P ) (r,P) ( r , P ) is defined, the recorded exponential variables are independent, so let us denote
prob ( P ) : = P ( ( r , P ) occurs ) = ∏ j = 1 ∣ P ∣ γ α j X γ α j Y . \operatorname{prob}(P):=\mathbb{P}((r,P)\text{ occurs})=\prod_{j=1}^{|P|}\gamma_{\alpha_j^X}\gamma_{\alpha_j^Y}. prob ( P ) := P (( r , P ) occurs ) = j = 1 ∏ ∣ P ∣ γ α j X γ α j Y .
For sufficiently large n n n , γ a \gamma_a γ a is strictly decreasing in a a a . Thus, among the low-expansion covers of P P P , the occurrence probability is maximized by choosing each exponent as small as allowed. This motivates the following definition.
Definition 6.14 (Most probable cover). For a pattern P P P , we denote its most probable low-expansion cover by MPC ( P ) \operatorname{MPC}(P) MPC ( P ) ; it has the same size and shape as P P P and exponent pairs ( ( α j Y − t ) + , ( α j X − t ) + ) ((\alpha_j^Y-t)_+,(\alpha_j^X-t)_+) (( α j Y − t ) + , ( α j X − t ) + ) for j ∈ [ ∣ P ∣ ] j\in[|P|] j ∈ [ ∣ P ∣ ] . Put q P = prob ( MPC ( P ) ) q_P=\operatorname{prob}(\operatorname{MPC}(P)) q P = prob ( MPC ( P )) .
We next compare the occurrence probability of each pattern in P \mathcal{P} P with that of its most probable cover.
Claim 6.15. There is an absolute constant C > 0 C>0 C > 0 such that every pattern P ∈ P P\in\mathcal{P} P ∈ P of size k k k satisfies
prob ( P ) q P ≤ ( C s t ) k . \frac{\operatorname{prob}(P)}{q_P}\le\left(\frac{C}{s^t}\right)^k. q P prob ( P ) ≤ ( s t C ) k .
Proof. Recall that γ 0 = Θ ( 1 ) \gamma_0=\Theta(1) γ 0 = Θ ( 1 ) and γ a = Θ ( s − a ) \gamma_a=\Theta(s^{-a}) γ a = Θ ( s − a ) uniformly for a ≥ 1 a\ge1 a ≥ 1 . Since γ a \gamma_a γ a is decreasing, the ratio γ a / γ ( a − t ) + \gamma_a/\gamma_{(a-t)_+} γ a / γ ( a − t ) + is at most 1 1 1 for every a ≥ 0 a\ge0 a ≥ 0 . For a ≥ t a\ge t a ≥ t , the same ratio is at most C / s t C/s^t C / s t , for an absolute constant C C C .
By the definition of the most probable cover,
prob ( P ) q P = ∏ j = 1 k γ α j X γ ( α j X − t ) + γ α j Y γ ( α j Y − t ) + . \frac{\operatorname{prob}(P)}{q_P}
=\prod_{j=1}^{k}
\frac{\gamma_{\alpha_j^X}}{\gamma_{(\alpha_j^X-t)_+}}
\frac{\gamma_{\alpha_j^Y}}{\gamma_{(\alpha_j^Y-t)_+}}. q P prob ( P ) = j = 1 ∏ k γ ( α j X − t ) + γ α j X γ ( α j Y − t ) + γ α j Y .
At each recorded position, at least one of α j X , α j Y \alpha_j^X,\alpha_j^Y α j X , α j Y is at least t t t , since P ∈ P P\in\mathcal{P} P ∈ P . Thus each pair of factors is at most C / s t C/s^t C / s t , proving the claim. □
For a pattern P P P of size k k k with d j ≤ D d_j\le D d j ≤ D for every j ∈ [ k − 1 ] j\in[k-1] j ∈ [ k − 1 ] , put
w P = C 1 L 2 ( p k 2 q P + k D ) . w_P=C_1L^2\left(\frac{pk^2}{q_P}+kD\right). w P = C 1 L 2 ( q P p k 2 + k D ) .
where C 1 C_1 C 1 is a fixed, sufficiently large absolute constant. In Subsection 6.5, w P w_P w P will be the length of an interval in which we search for an isolated occurrence of MPC ( P ) \operatorname{MPC}(P) MPC ( P ) . The first term accounts for the occurrence probability of the cover, and the second provides separation for the matching intervals. The following lemma bounds the first contribution globally.
Lemma 6.16. With high probability,
C 1 L 2 p ∑ C ∣ C ∣ 2 q P C ≤ H / L , C_1L^2p\sum_C\frac{|C|^2}{q_{P_C}}\le H/L, C 1 L 2 p C ∑ q P C ∣ C ∣ 2 ≤ H / L , where the sum is over all components.
Proof. Before bounding the sum in the lemma, which is over components, we bound a related sum over all patterns in P \mathcal{P} P . This bound expresses that patterns with large 1 / q P 1/q_P 1/ q P are sufficiently unlikely to occur. Specifically, we show that
∑ P ∈ P ∣ P ∣ 2 prob ( P ) q P = O ( L 2 / s t ) . \sum_{P\in\mathcal{P}}|P|^2\frac{\operatorname{prob}(P)}{q_P}=O(L^2/s^t). P ∈ P ∑ ∣ P ∣ 2 q P prob ( P ) = O ( L 2 / s t ) . There are at most D k − 1 ( L + 1 ) 2 k D^{k-1}(L+1)^{2k} D k − 1 ( L + 1 ) 2 k patterns of size k k k in P \mathcal{P} P : each of the k − 1 k-1 k − 1 successive distances has at most D D D choices, and each of the 2 k 2k 2 k exponents has at most L + 1 L+1 L + 1 choices. By Claim 6.15,
∑ P ∈ P ∣ P ∣ 2 prob ( P ) q P ≤ ∑ k = 1 L k 2 D k − 1 ( L + 1 ) 2 k ( C s t ) k \sum_{P\in\mathcal{P}}|P|^2\frac{\operatorname{prob}(P)}{q_P}
\le\sum_{k=1}^{L}k^2D^{k-1}(L+1)^{2k}\left(\frac{C}{s^t}\right)^k P ∈ P ∑ ∣ P ∣ 2 q P prob ( P ) ≤ k = 1 ∑ L k 2 D k − 1 ( L + 1 ) 2 k ( s t C ) k = C ( L + 1 ) 2 s t ∑ k = 1 L k 2 ( C D ( L + 1 ) 2 s t ) k − 1 = O ( L 2 / s t ) . =\frac{C(L+1)^2}{s^t}\sum_{k=1}^{L}k^2\left(\frac{CD(L+1)^2}{s^t}\right)^{k-1}
=O(L^2/s^t). = s t C ( L + 1 ) 2 k = 1 ∑ L k 2 ( s t C D ( L + 1 ) 2 ) k − 1 = O ( L 2 / s t ) . We only need to explain the last equality. Since s t ≥ s D s^t\ge sD s t ≥ sD and s = L 3 s=L^3 s = L 3 , we have C D ( L + 1 ) 2 / s t = O ( 1 / L ) CD(L+1)^2/s^t=O(1/L) C D ( L + 1 ) 2 / s t = O ( 1/ L ) . For sufficiently large n n n , the last sum is at most ∑ k ≥ 1 k 2 2 − ( k − 1 ) = O ( 1 ) \sum_{k\ge1}k^2 2^{-(k-1)}=O(1) ∑ k ≥ 1 k 2 2 − ( k − 1 ) = O ( 1 ) .
To bound the sum over components, define the random variable S S S by:
S = C 1 L 2 p ∑ P ∈ P ∣ P ∣ 2 q P ∑ r = 1 n − 1 − span ( P ) 1 { ( r , P ) occurs } , S=C_1L^2p\sum_{P\in\mathcal{P}}\frac{|P|^2}{q_P}\sum_{r=1}^{n-1-\operatorname{span}(P)}\mathbf{1}_{\{(r,P)\text{ occurs}\}}, S = C 1 L 2 p P ∈ P ∑ q P ∣ P ∣ 2 r = 1 ∑ n − 1 − span ( P ) 1 {( r , P ) occurs } , where an inner sum is empty if there is no valid root. For each valid root r r r , the probability of ( r , P ) (r,P) ( r , P ) occurring is prob ( P ) \operatorname{prob}(P) prob ( P ) . By linearity of expectation and the preceding estimate,
E S ≤ C 1 L 2 p n ∑ P ∈ P ∣ P ∣ 2 prob ( P ) q P = O ( n L 4 p / s t ) = O ( n L 5 / s t ) . \mathbb{E}S\le C_1L^2pn\sum_{P\in\mathcal{P}}|P|^2\frac{\operatorname{prob}(P)}{q_P}=O(nL^4p/s^t)=O(nL^5/s^t). E S ≤ C 1 L 2 p n P ∈ P ∑ ∣ P ∣ 2 q P prob ( P ) = O ( n L 4 p / s t ) = O ( n L 5 / s t ) . Applying Markov’s inequality, we get
P ( S > H / L ) ≤ L E S H = O ( n L 6 H s t ) = O ( n L 3 H D ) = O ( L − 4 ) = o ( 1 ) , \mathbb{P}(S>H/L)\le\frac{L\mathbb{E}S}{H}=O\left(\frac{nL^6}{Hs^t}\right)=O\left(\frac{nL^3}{HD}\right)=O(L^{-4})=o(1), P ( S > H / L ) ≤ H L E S = O ( H s t n L 6 ) = O ( H D n L 3 ) = O ( L − 4 ) = o ( 1 ) , using s t ≥ s D s^t\ge sD s t ≥ sD , s = L 3 s=L^3 s = L 3 , and H D = Θ ( n L 7 ) HD=\Theta(nL^7) H D = Θ ( n L 7 ) .
By Corollary 6.13, with high probability every component pattern belongs to P \mathcal{P} P . On this event, each component C C C gives a distinct rooted occurrence ( min C , P C ) (\min C,P_C) ( min C , P C ) counted by S S S . Hence C 1 L 2 p ∑ C ∣ C ∣ 2 / q P C ≤ S C_1L^2p\sum_C|C|^2/q_{P_C}\le S C 1 L 2 p ∑ C ∣ C ∣ 2 / q P C ≤ S . Intersecting this event with S ≤ H / L S\le H/L S ≤ H / L proves the lemma. □
Proof of Proposition 6.5
We use the properties established in Subsection 6.4 to construct the matching. We first partition the indices [ n ] = J 1 ∪ ˙ ⋯ ∪ ˙ J m [n]=J_1\mathbin{\dot\cup}\cdots\mathbin{\dot\cup}J_m [ n ] = J 1 ∪ ˙ ⋯ ∪ ˙ J m while keeping components away from the boundaries of the partition { J i } i = 1 m \{J_i\}_{i=1}^m { J i } i = 1 m . We then find isolated covers, allocate disjoint search intervals, and select and align pairs of patterns.
Partition into intervals
Claim 6.17. With high probability , there is a partition [ n ] = J 1 ∪ ˙ ⋯ ∪ ˙ J m [n] = J_1 \mathbin{\dot\cup} \cdots\mathbin{\dot\cup} J_m [ n ] = J 1 ∪ ˙ ⋯ ∪ ˙ J m into consecutive intervals of sizes between H H H and 3 H 3H 3 H such that, for every component C C C , some J j J_j J j satisfies
[ min C , max C + 1 ] ⊆ [ min J j + 4 D , max J j − 4 D ] . [\min C, \max C + 1] \subseteq[\min J_j + 4D, \max J_j - 4D]. [ min C , max C + 1 ] ⊆ [ min J j + 4 D , max J j − 4 D ] . Proof. We start with a deterministic partition [ n ] = J ^ 1 ∪ ˙ ⋯ ∪ ˙ J ^ m [n] = \widehat{J}_1 \mathbin{\dot\cup} \cdots\mathbin{\dot\cup} \widehat{J}_m [ n ] = J 1 ∪ ˙ ⋯ ∪ ˙ J m into m = ⌊ n / ( 2 H ) ⌋ m = \lfloor n/(2H) \rfloor m = ⌊ n / ( 2 H )⌋ consecutive intervals whose sizes differ by at most one, each of size 2 H + o ( H ) 2H + o(H) 2 H + o ( H ) . It is easy to see that such a partition always exists. Write J ^ j = [ b j − 1 + 1 , b j ] \widehat{J}_j = [b_{j-1} + 1, b_j] J j = [ b j − 1 + 1 , b j ] , where 0 = b 0 < b 1 < ⋯ < b m = n 0 = b_0 < b_1 < \cdots< b_m = n 0 = b 0 < b 1 < ⋯ < b m = n . We now slightly perturb the boundaries b j b_j b j to new boundaries b j ′ b'_j b j ′ and define J j = [ b j − 1 ′ + 1 , b j ′ ] J_j = [b'_{j-1} + 1, b'_j] J j = [ b j − 1 ′ + 1 , b j ′ ] .
Condition on the event in Claim 6.12. Keep the two outer boundaries fixed: b 0 ′ = 0 b'_0 = 0 b 0 ′ = 0 and b m ′ = n b'_m = n b m ′ = n . For each j ∈ [ m − 1 ] j \in[m - 1] j ∈ [ m − 1 ] , if [ b j − 5 D , b j + 5 D ] [b_j - 5D, b_j + 5D] [ b j − 5 D , b j + 5 D ] contains no tiny index, set b j ′ = b j b'_j = b_j b j ′ = b j . Otherwise, all tiny indices in this interval lie within distance 10 D 10D 10 D of one another, so they belong to one maximal sequence of tiny indices with successive distances at most 12 D 12D 12 D . By Claim 6.12, this sequence has at most L L L indices. Let u j u_j u j be the last index of this sequence, and set b j ′ = u j + 5 D + 1 b'_j = u_j + 5D + 1 b j ′ = u j + 5 D + 1 . Then 0 ≤ b j ′ − b j ≤ 12 L D = o ( H ) 0 \le b'_j - b_j \le12LD = o(H) 0 ≤ b j ′ − b j ≤ 12 L D = o ( H ) . Moreover, every tiny index is either at most u j u_j u j or greater than u j + 12 D u_j + 12D u j + 12 D , so b j ′ b'_j b j ′ is more than 5 D 5D 5 D from every tiny index.
Since the original intervals have size 2 H + o ( H ) 2H + o(H) 2 H + o ( H ) and every boundary moves by o ( H ) o(H) o ( H ) , we have 0 = b 0 ′ < b 1 ′ < ⋯ < b m ′ = n 0 = b'_0 < b'_1 < \cdots< b'_m = n 0 = b 0 ′ < b 1 ′ < ⋯ < b m ′ = n , and the intervals J j = [ b j − 1 ′ + 1 , b j ′ ] J_j = [b'_{j-1} + 1, b'_j] J j = [ b j − 1 ′ + 1 , b j ′ ] form a partition with sizes between H H H and 3 H 3H 3 H . By construction, there are no tiny indices in the 5 D 5D 5 D -neighborhood of b j ′ b'_j b j ′ for any j ∈ [ m − 1 ] j \in[m - 1] j ∈ [ m − 1 ] .
It remains to verify that components in J 1 J_1 J 1 and J m J_m J m lie at the required distance from the outer endpoints of these intervals, corresponding to the boundaries b 0 ′ = 0 b'_0 = 0 b 0 ′ = 0 and b m ′ = n b'_m = n b m ′ = n . To handle these cases, observe that a simple union bound over the first and last 8 D 8D 8 D gap indices shows that none of them is tiny with probability 1 − O ( D / s t ) = 1 − o ( 1 ) 1 - O(D/s^t) = 1 - o(1) 1 − O ( D / s t ) = 1 − o ( 1 ) . This gives the remaining margins for J 1 J_1 J 1 and J m J_m J m , completing the proof.
Isolated covers
Recall the definition of the most probable cover and the search length w P w_P w P defined in Subsection 6.4. We show that intervals of length at least w P w_P w P contain suitable isolated occurrences.
The following simple claim shows that from an interval V V V of gap indices we can extract many pairwise disjoint sets with the same shape as a given pattern P P P . Their roots lie in a prescribed residue class modulo p p p , and the D D D -neighborhoods of their corresponding intervals lie in V V V . Concretely:
Claim 6.18 (Disjoint supports). Let P P P be a pattern of size k k k with d j ≤ D d_j \le D d j ≤ D for every j ∈ [ k − 1 ] j \in[k - 1] j ∈ [ k − 1 ] , let K ≥ 1 K \ge1 K ≥ 1 be an integer, and let V ⊆ [ n − 1 ] V \subseteq[n - 1] V ⊆ [ n − 1 ] be an interval of gap indices with
∣ V ∣ ≥ p k 2 K + ( k + 1 ) D . |V| \ge pk^2K + (k + 1)D. ∣ V ∣ ≥ p k 2 K + ( k + 1 ) D . For every r 0 ∈ { 0 , … , p − 1 } r_0 \in\{0, \ldots, p - 1\} r 0 ∈ { 0 , … , p − 1 } , there are K K K rooted patterns ( r 1 , P ) , … , ( r K , P ) (r_1, P), \ldots, (r_K, P) ( r 1 , P ) , … , ( r K , P ) such that:
Their supports are pairwise disjoint.
r i ≡ r 0 ( m o d p ) r_i \equiv r_0 \pmod p r i ≡ r 0 ( mod p ) for every i ∈ [ K ] i \in[K] i ∈ [ K ] .
The D D D -neighborhood of int ( r i , P ) \operatorname{int}(r_i, P) int ( r i , P ) is contained in V V V for every i ∈ [ K ] i \in[K] i ∈ [ K ] .
Proof. Fix r 0 ∈ { 0 , … , p − 1 } r_0 \in\{0,\ldots,p-1\} r 0 ∈ { 0 , … , p − 1 } . Let V ′ V' V ′ denote the set of roots r r r such that the D D D -neighborhood of int ( r , P ) \operatorname{int}(r,P) int ( r , P ) lies in V V V . Notice that V ′ V' V ′ is an interval of consecutive indices. We have
∣ V ′ ∣ = ∣ V ∣ − span ( P ) − 2 D ≥ ∣ V ∣ − ( k + 1 ) D ≥ p k 2 K . |V'|=|V|-\operatorname{span}(P)-2D\ge|V|-(k+1)D\ge pk^2K. ∣ V ′ ∣ = ∣ V ∣ − span ( P ) − 2 D ≥ ∣ V ∣ − ( k + 1 ) D ≥ p k 2 K .
where we used span ( P ) ≤ ( k − 1 ) D \operatorname{span}(P)\le(k-1)D span ( P ) ≤ ( k − 1 ) D , since each d j ≤ D d_j\le D d j ≤ D . Let V ′ ′ = { r ∈ V ′ : r ≡ r 0 ( m o d p ) } V''=\{r\in V':r\equiv r_0\pmod p\} V ′′ = { r ∈ V ′ : r ≡ r 0 ( mod p )} . Since V ′ V' V ′ is an interval, ∣ V ′ ′ ∣ ≥ k 2 K |V''|\ge k^2K ∣ V ′′ ∣ ≥ k 2 K .
We claim that there are K K K roots in V ′ ′ V'' V ′′ whose corresponding supports are pairwise disjoint. Indeed, define a graph G G G on V ′ ′ V'' V ′′ , joining distinct vertices u , v ∈ V ′ ′ u,v\in V'' u , v ∈ V ′′ if the supports of ( u , P ) (u,P) ( u , P ) and ( v , P ) (v,P) ( v , P ) intersect. Two such supports intersect only if u − v = a i − a j u-v=a_i-a_j u − v = a i − a j for some i , j ∈ [ k ] i,j\in[k] i , j ∈ [ k ] (recall that a i = ∑ h < i d h a_i=\sum_{h<i}d_h a i = ∑ h < i d h , as defined after Definition 6.6). There are at most k 2 k^2 k 2 such differences, including 0 0 0 , so the maximum degree of G G G satisfies Δ ( G ) < k 2 \Delta(G)<k^2 Δ ( G ) < k 2 .
Greedily choosing a vertex of G G G and deleting it and its neighbors gives an independent set of size at least ∣ V ′ ′ ∣ / ( Δ ( G ) + 1 ) ≥ ∣ V ′ ′ ∣ / k 2 ≥ K |V''|/(\Delta(G)+1)\ge|V''|/k^2\ge K ∣ V ′′ ∣/ ( Δ ( G ) + 1 ) ≥ ∣ V ′′ ∣/ k 2 ≥ K . Choose any K K K vertices r 1 , … , r K r_1,\ldots,r_K r 1 , … , r K in this independent set. It is easy to verify by construction that Items 1 to 3 hold for these roots, completing the proof. □ \square □
We now apply Claim 6.18 to search for isolated occurrences of most probable covers. We first fix a pattern in the family P \mathcal{P} P ; a union bound will then give the conclusion simultaneously for all component patterns.
Claim 6.19. Fix a pattern P ∈ P P\in\mathcal{P} P ∈ P , a residue class modulo p p p , and an interval W ⊆ [ n − 1 ] W\subseteq[n-1] W ⊆ [ n − 1 ] with ∣ W ∣ ≥ w P |W|\ge w_P ∣ W ∣ ≥ w P . Then, with probability at least 1 − 2 − 10 L 2 1-2^{-10L^2} 1 − 2 − 10 L 2 , there is an isolated occurrence of MPC ( P ) \operatorname{MPC}(P) MPC ( P ) whose root lies in the prescribed residue class and whose interval’s D D D -neighborhood is contained in W W W .
Proof. Put k = ∣ P ∣ k=|P| k = ∣ P ∣ , q = q P q=q_P q = q P , and K = ⌈ 4 / q ⌉ K=\lceil4/q\rceil K = ⌈ 4/ q ⌉ , so 4 ≤ K q ≤ 5 4\le Kq\le5 4 ≤ K q ≤ 5 . Split W W W into intervals of length
m = p k 2 K + ( k + 1 ) D ≤ 5 ( p k 2 / q + k D ) . m=pk^2K+(k+1)D\le5(pk^2/q+kD). m = p k 2 K + ( k + 1 ) D ≤ 5 ( p k 2 / q + k D ) .
discarding the remainder. By the definition of w P w_P w P , there are at least 10 L 2 10L^2 10 L 2 such intervals for sufficiently large C 1 C_1 C 1 . It suffices to show that each fixed subinterval contains a suitable isolated occurrence, with its entire D D D -neighborhood inside that subinterval, with probability at least 1 / 2 1/2 1/2 . Indeed, these events depend only on the exponential variables inside their respective subintervals and are therefore independent. The probability that all fail is then at most 2 − 10 L 2 2^{-10L^2} 2 − 10 L 2 .
Fix one such subinterval U U U . Apply Claim 6.18 to MPC ( P ) \operatorname{MPC}(P) MPC ( P ) to choose deterministic roots r 1 , … , r K r_1,\ldots,r_K r 1 , … , r K in the prescribed residue class. Their supports are pairwise disjoint, and the D D D -neighborhoods of their intervals lie in U U U . Let A i A_i A i be the event that ( r i , MPC ( P ) ) (r_i,\operatorname{MPC}(P)) ( r i , MPC ( P )) occurs. The events A i A_i A i are independent, each with probability q q q , so the probability that none occurs is at most e − K q ≤ e − 4 e^{-Kq}\le e^{-4} e − K q ≤ e − 4 .
Let T i T_i T i be the set of gap indices in the D D D -neighborhood of int ( r i , MPC ( P ) ) \operatorname{int}(r_i,\operatorname{MPC}(P)) int ( r i , MPC ( P )) that do not belong to its support. This neighborhood contains span ( P ) + 1 + 2 D \operatorname{span}(P)+1+2D span ( P ) + 1 + 2 D indices, of which k k k belong to the support. Thus
∣ T i ∣ = span ( P ) + 1 + 2 D − k ≤ ( k + 1 ) D , |T_i|=\operatorname{span}(P)+1+2D-k\le(k+1)D, ∣ T i ∣ = span ( P ) + 1 + 2 D − k ≤ ( k + 1 ) D ,
since span ( P ) ≤ ( k − 1 ) D \operatorname{span}(P)\le(k-1)D span ( P ) ≤ ( k − 1 ) D . Let B i B_i B i be the event that some index in T i T_i T i is tiny. For each j ∈ T i j\in T_i j ∈ T i , the variables E j x , E j y E_j^x,E_j^y E j x , E j y are independent of A i A_i A i . The index j j j is tiny if at least one of these variables is at most s − t s^{-t} s − t , so P ( j is tiny ∣ A i ) ≤ 2 s − t \mathbb{P}(j\text{ is tiny}\mid A_i)\le2s^{-t} P ( j is tiny ∣ A i ) ≤ 2 s − t . Recall that P ( A i ) = q \mathbb{P}(A_i)=q P ( A i ) = q by definition. Applying a union bound over T i T_i T i gives
P ( A i ∩ B i ) = P ( A i ) P ( B i ∣ A i ) = q P ( B i ∣ A i ) ≤ q ∑ j ∈ T i P ( j is tiny ∣ A i ) ≤ 2 q ∣ T i ∣ s t ≤ 2 q ( k + 1 ) D s t . \begin{aligned}
\mathbb{P}(A_i\cap B_i)=\mathbb{P}(A_i)\mathbb{P}(B_i\mid A_i)=q\mathbb{P}(B_i\mid A_i)\le q\sum_{j\in T_i}\mathbb{P}(j\text{ is tiny}\mid A_i) \\
&\le\frac{2q|T_i|}{s^t}\le\frac{2q(k+1)D}{s^t}.
\end{aligned} P ( A i ∩ B i ) = P ( A i ) P ( B i ∣ A i ) = q P ( B i ∣ A i ) ≤ q j ∈ T i ∑ P ( j is tiny ∣ A i ) ≤ s t 2 q ∣ T i ∣ ≤ s t 2 q ( k + 1 ) D .
By a union bound, the probability that no candidate gives an isolated occurrence is at most
e − 4 + ∑ i = 1 K P ( A i ∩ B i ) ≤ e − 4 + 10 ( k + 1 ) D s t = e − 4 + O ( L − 2 ) < 1 2 , e^{-4}+\sum_{i=1}^{K}\mathbb{P}(A_i\cap B_i)\le e^{-4}+\frac{10(k+1)D}{s^t}=e^{-4}+O(L^{-2})<\frac{1}{2}, e − 4 + i = 1 ∑ K P ( A i ∩ B i ) ≤ e − 4 + s t 10 ( k + 1 ) D = e − 4 + O ( L − 2 ) < 2 1 ,
where we used k ≤ L k\le L k ≤ L and D / s t ≤ 1 / s = L − 3 D/s^t\le1/s=L^{-3} D / s t ≤ 1/ s = L − 3 . This proves the claim.
Lemma 6.20 (Isolated covers). With high probability, for every component C C C , every residue class modulo p p p , and every interval V ⊆ [ n − 1 ] V\subseteq[n-1] V ⊆ [ n − 1 ] with ∣ V ∣ ≥ w P C |V|\ge w_{P_C} ∣ V ∣ ≥ w P C , there is an isolated occurrence of MPC ( P C ) \operatorname{MPC}(P_C) MPC ( P C ) with root in that residue class, whose interval and its D D D -neighborhood are contained in V V V .
Proof. By Claim 6.19, a union bound over P ∈ P P\in\mathcal{P} P ∈ P , the p p p residue classes, and the at most n 2 n^2 n 2 intervals V V V gives total failure probability at most
∣ P ∣ p n 2 2 − 10 L 2 ≤ exp ( ( 1 / 3 − 10 log 2 + o ( 1 ) ) L 2 ) = o ( 1 ) , |\mathcal{P}|pn^2 2^{-10L^2}\le\exp((1/3-10\log2+o(1))L^2)=o(1), ∣ P ∣ p n 2 2 − 10 L 2 ≤ exp (( 1/3 − 10 log 2 + o ( 1 )) L 2 ) = o ( 1 ) ,
by Equation (7) . By Corollary 6.13, with high probability every component pattern belongs to P \mathcal{P} P , so the lemma follows.
Selecting and aligning pairs
We now choose component–cover pairs and use them to define the partial block mapping. We first assign each component an isolated cover so that the cover intervals are mutually separated. A cover may still lie close to another component; we resolve these conflicts by selecting pairs in decreasing order of the components’ largest exponents.
Lemma 6.21. With high probability, the partition J 1 , … , J m J_1,\ldots,J_m J 1 , … , J m from Claim 6.17 admits pairs of occurring rooted patterns P i , P ‾ i P_i,\overline{P}_i P i , P i , for i ∈ [ ℓ ] i\in[\ell] i ∈ [ ℓ ] and some ℓ ∈ N 0 \ell\in\mathbb{N}_0 ℓ ∈ N 0 , such that:
Each P i P_i P i records a component, and P ‾ i \overline{P}_i P i is an isolated occurrence of its most probable cover, with root congruent to that of P i P_i P i modulo p p p .
The 2 ℓ 2\ell 2 ℓ gap-index intervals are pairwise separated by more than D D D , and every tiny index belongs to one of their supports.
For each i ∈ [ ℓ ] i\in[\ell] i ∈ [ ℓ ] , both corresponding gap-index intervals lie in [ min J j + 4 D , max J j − 4 D ] [\min J_j+4D,\max J_j-4D] [ min J j + 4 D , max J j − 4 D ] for some j ∈ [ m ] j\in[m] j ∈ [ m ] .
Proof. Assume that the events described in Claims 6.10 and 6.17 and Lemmas 6.16 and 6.20 hold. Fix the partition J 1 , … , J m J_1,\ldots,J_m J 1 , … , J m from Claim 6.17. We claim the following auxiliary property: for every j ∈ [ m ] j\in[m] j ∈ [ m ] it holds that
∑ C ⊆ J j ∣ C ∣ ≤ 12 H / s t . (8) \sum_{C\subseteq J_j}|C|\le12H/s^t. \tag*{(8)} C ⊆ J j ∑ ∣ C ∣ ≤ 12 H / s t . (8)
That is because the components C ⊆ J j C\subseteq J_j C ⊆ J j in the above sum are disjoint and consist of tiny indices. Therefore the sum on the left-hand side counts the number of tiny indices in J j J_j J j , which is at most 12 H / s t 12H/s^t 12 H / s t due to the event from Claim 6.10.
We allocate to every component C C C an interval of length ⌈ w P C ⌉ \lceil w_{P_C}\rceil ⌈ w P C ⌉ in which we want to allocate C C C ’s points to an isolated occurrence of MPC ( P C ) \operatorname{MPC}(P_C) MPC ( P C ) in the interval. For a fixed part J j J_j J j , the components are disjoint sets of tiny indices. Since w P C ≥ D ≥ 1 w_{P_C} \ge D \ge1 w P C ≥ D ≥ 1 we have
∑ C ⊆ J j ( ⌈ w P C ⌉ + D ) ≤ 3 ∑ C ⊆ J j w P C = 3 C 1 L 2 p ∑ C ⊆ J j ∣ C ∣ 2 q P C + 3 C 1 L 2 D ∑ C ⊆ J j ∣ C ∣ ≤ Lemma 6.16 3 H L + 3 C 1 L 2 D ∑ C ⊆ J j ∣ C ∣ ≤ 3 H L + 36 C 1 L 2 D H / s t = O ( H / L ) , (9) \begin{aligned}
\sum_{C\subseteq J_j} \left(\lceil w_{P_C}\rceil+ D\right) &\le3\sum_{C\subseteq J_j} w_{P_C}\\
&= 3\mathcal{C}_1L^2p\sum_{C\subseteq J_j}\frac{|C|^2}{q_{P_C}}+3\mathcal{C}_1L^2D\sum_{C\subseteq J_j}|C|\\
&\underset{\text{Lemma 6.16}}{\le} \frac{3H}{L}+3\mathcal{C}_1L^2D\sum_{C\subseteq J_j}|C|\\
&\le\frac{3H}{L}+36\mathcal{C}_1L^2DH/s^t=O(H/L), \tag*{(9)}
\end{aligned} C ⊆ J j ∑ ( ⌈ w P C ⌉ + D ) ≤ 3 C ⊆ J j ∑ w P C = 3 C 1 L 2 p C ⊆ J j ∑ q P C ∣ C ∣ 2 + 3 C 1 L 2 D C ⊆ J j ∑ ∣ C ∣ Lemma 6.16 ≤ L 3 H + 3 C 1 L 2 D C ⊆ J j ∑ ∣ C ∣ ≤ L 3 H + 36 C 1 L 2 D H / s t = O ( H / L ) , (9)
where the last equality uses D / s t ≤ L − 3 D/s^t\le L^{-3} D / s t ≤ L − 3 . Thus intervals V C V_C V C of lengths ⌈ w P C ⌉ \lceil w_{P_C}\rceil ⌈ w P C ⌉ , with D D D unused gap indices between consecutive intervals, fit inside [ min J j + 4 D , max J j − 4 D − 1 ] [\min J_j+4D,\max J_j-4D-1] [ min J j + 4 D , max J j − 4 D − 1 ] : this interval has at least H − 8 D − 1 H-8D-1 H − 8 D − 1 indices, and D = o ( H ) D=o(H) D = o ( H ) . The intervals V C V_C V C are separated by more than D D D , also across distinct parts because of the margins.
By Lemma 6.20, in each V C V_C V C choose an isolated occurrence Q C \mathbf{Q}_C Q C of MPC ( P C ) \operatorname{MPC}(P_C) MPC ( P C ) whose root is congruent to min C \min C min C modulo p p p and whose gap-index interval and its D D D -neighborhood lie in V C V_C V C . Observe that even when the intervals { V C } C \{V_C\}_C { V C } C are mutually separated, they need not be separated from the components. For example, if E i X ≈ 1 / n E_i^X\approx1/n E i X ≈ 1/ n , a low-expansion cover must match this gap to a gap of size at most O ~ ( n − 2 / 3 ) \widetilde{O}(n^{-2/3}) O ( n − 2/3 ) . The latter gap is still tiny, so its index belongs to a component. The following observation will allow us to resolve such conflicts by selecting pairs in decreasing order of the components’ largest exponents.
For every component C C C , put h ( C ) = max i ∈ C max { e i X , e i Y } h(C)=\max_{i\in C}\max\{e_i^X,e_i^Y\} h ( C ) = max i ∈ C max { e i X , e i Y } . Fix a component C C C , and let C ′ C' C ′ be any component whose gap-index interval is within distance D D D of the gap-index interval of Q C \mathbf{Q}_C Q C . We claim that
C ′ ⊆ supp ( Q C ) , h ( C ′ ) < h ( C ) . C'\subseteq\operatorname{supp}(\mathbf{Q}_C),\qquad h(C')<h(C). C ′ ⊆ supp ( Q C ) , h ( C ′ ) < h ( C ) .
Indeed, Q C \mathbf{Q}_C Q C is isolated by Lemma 6.20, so every tiny index within distance D D D of supp ( Q C ) \operatorname{supp}(\mathbf{Q}_C) supp ( Q C ) belongs to this support. Successive indices in both C ′ C' C ′ and supp ( Q C ) \operatorname{supp}(\mathbf{Q}_C) supp ( Q C ) are at distance at most D D D . Thus the assumed proximity of their gap-index intervals implies that some index of C ′ C' C ′ is within distance D D D of supp ( Q C ) \operatorname{supp}(\mathbf{Q}_C) supp ( Q C ) . Since all indices of C ′ C' C ′ are tiny, it follows that C ′ ∩ supp ( Q C ) ≠ ∅ C'\cap\operatorname{supp}(\mathbf{Q}_C)\ne\varnothing C ′ ∩ supp ( Q C ) = ∅ . Starting from an index in this intersection, each successive index of C ′ C' C ′ in either direction is tiny and within distance D D D of the previous one. Isolation therefore puts each of these indices in supp ( Q C ) \operatorname{supp}(\mathbf{Q}_C) supp ( Q C ) , proving C ′ ⊆ supp ( Q C ) C'\subseteq\operatorname{supp}(\mathbf{Q}_C) C ′ ⊆ supp ( Q C ) .
By the definition of MPC ( P C ) \operatorname{MPC}(P_C) MPC ( P C ) , each exponent recorded by Q C \mathbf{Q}_C Q C is of the form ( a − t ) + (a-t)_+ ( a − t ) + , where a a a is an exponent recorded by P C P_C P C . Since a ≤ h ( C ) a\le h(C) a ≤ h ( C ) and h ( C ) ≥ t h(C)\ge t h ( C ) ≥ t , every such exponent is at most h ( C ) − t h(C)-t h ( C ) − t . We have shown that every index of C ′ C' C ′ belongs to the support of Q C \mathbf{Q}_C Q C , so both exponents at every index of C ′ C' C ′ satisfy this bound. Thus
h ( C ′ ) ≤ h ( C ) − t < h ( C ) . h(C')\le h(C)-t<h(C). h ( C ′ ) ≤ h ( C ) − t < h ( C ) .
We now describe how to match components to low-expansion covers so that all components are covered. Process the components in decreasing order of h ( C ) h(C) h ( C ) , breaking ties arbitrarily. Let ( P C 1 , Q C 1 ) , … , ( P C j , Q C j ) (P_{C_1},Q_{C_1}),\ldots,(P_{C_j},Q_{C_j}) ( P C 1 , Q C 1 ) , … , ( P C j , Q C j ) be the pairs selected so far, where initially j = 0 j=0 j = 0 , and let C C C be the next component in this order. If the gap-index interval of C C C is within distance D D D of the gap-index interval of Q C i Q_{C_i} Q C i for some i ∈ [ j ] i\in[j] i ∈ [ j ] , skip C C C . Otherwise, set C j + 1 = C C_{j+1}=C C j + 1 = C and select the pair ( P C j + 1 , Q C j + 1 ) (P_{C_{j+1}},Q_{C_{j+1}}) ( P C j + 1 , Q C j + 1 ) , where P C = ( min C , P C ) P_C=(\min C,P_C) P C = ( min C , P C ) . We now verify the three properties in the lemma. Let ℓ \ell ℓ be the number of selected pairs, and relabel them as ( P i , P ‾ i ) (P_i,\overline{P}_i) ( P i , P i ) for i ∈ [ ℓ ] i \in[\ell] i ∈ [ ℓ ] .
Item 1 holds by construction: each selected P C \mathbf{P}_C P C records a component, and Q C \mathbf{Q}_C Q C is an isolated occurrence of MPC ( P C ) \operatorname{MPC}(\mathbf{P}_C) MPC ( P C ) with root congruent to that of P C \mathbf{P}_C P C modulo p p p .
Item 2 follows from the selection rule and the containment and strict decrease proved above: the selected intervals are pairwise separated by more than D D D , and every skipped component is contained in an already selected cover. Thus every tiny index belongs to a selected support.
For Item 3, consider a selected pair generated by C ⊆ J j C \subseteq J_j C ⊆ J j . The component’s gap-index interval has the required margins by Claim 6.17 . The cover’s gap-index interval has the same margins as V C ⊆ [ min J j + 4 D , max J j − 4 D ] V_C \subseteq[\min J_j + 4D, \max J_j - 4D] V C ⊆ [ min J j + 4 D , max J j − 4 D ] .
Proof of Proposition 6.5. Assume that the high-probability events of Lemma 6.21 , Corollary 6.13 , and Claim 6.9 hold. By Lemma 6.21 , we select pairs of rooted patterns P i , P ‾ i \mathbf{P}_i,\overline{\mathbf{P}}_i P i , P i such that P i \mathbf{P}_i P i records a component of tiny indices and P ‾ i \overline{\mathbf{P}}_i P i is a most probable cover of P i \mathbf{P}_i P i . For each selected pair of rooted patterns P i , P ‾ i \mathbf{P}_i,\overline{\mathbf{P}}_i P i , P i , let I i I_i I i be the smallest union of consecutive blocks containing j , j + 1 j,j+1 j , j + 1 for all indices j j j recorded by P i \mathbf{P}_i P i , and let I ‾ i \overline{I}_i I i be defined the same way with respect to Y Y Y and P ‾ i \overline{\mathbf{P}}_i P i .
By Corollary 6.13 , each component has at most L L L indices and span at most ( L − 1 ) D (L-1)D ( L − 1 ) D . Hence ∣ I i ∣ , ∣ I ‾ i ∣ ≤ L D + 2 p + 1 = O ( L D ) |I_i|,|\overline{I}_i| \le LD+2p+1=O(LD) ∣ I i ∣ , ∣ I i ∣ ≤ L D + 2 p + 1 = O ( L D ) . The original gap-index intervals of P i \mathbf{P}_i P i and P ‾ i \overline{\mathbf{P}}_i P i are separated by at least D D D and have margins at least 4 D 4D 4 D in their respective parts. Thus the enlarged intervals are separated by at least D − 2 p − 1 ≥ D / 2 D-2p-1 \ge D/2 D − 2 p − 1 ≥ D /2 and retain margins at least 4 D − p − 1 ≥ D / 2 4D-p-1 \ge D/2 4 D − p − 1 ≥ D /2 . They contain no terminal block, since that block lies within p p p indices of n n n , whereas these intervals remain more than p p p indices from n n n . This proves the interval requirements in Items 4 and 5.
Define f f f by the increasing bijections X I i → Y I ‾ i X_{I_i} \to Y_{\overline{I}_i} X I i → Y I i and X I ‾ i → Y I i X_{\overline{I}_i} \to Y_{I_i} X I i → Y I i for each i i i . The intervals I i I_i I i and I ‾ i \overline{I}_i I i are disjoint unions of nonterminal blocks, and corresponding blocks of I i I_i I i and I ‾ i \overline{I}_i I i are exchanged in pairs. Thus f f f is a partial block mapping induced by a partial involution, proving Item 1. Every tiny index j j j lies in a selected support, so j , j + 1 j,j+1 j , j + 1 lie together in its enlarged interval, proving Item 2.
Finally, both occurrences in each pair are isolated: one records a component, and the other was chosen to be isolated. Since the extensions are shorter than p < D p<D p < D , every tiny gap index inside either enlarged interval lies in the support of its pattern. The paired patterns are low-expansion covers, and their roots have the same relative positions in the enlarged intervals. Therefore Claim 6.9 applies and gives Item 3, completing the proof.
Rearrangement lemma
We now rearrange the free points (those whose indices lie in free blocks) so that gaps between consecutive free blocks have length Θ ~ ( 1 / n ) \widetilde{\Theta}(1/n) Θ ( 1/ n ) , while fixing every prescribed point.
The main idea is that we need not construct the matching directly between X X X and Y Y Y . We may first pass to perturbed copies X ′ X' X ′ and Y ′ Y' Y ′ that have essentially the same metric: each copy is related to the original sample by a bijection under which every distance is preserved up to a polylogarithmic factor. A matching between these copies can then be transferred back to X X X and Y Y Y with only a polylogarithmic increase in distortion. Concretely, we seek copies in the following canonical form .
Definition 6.22 (Canonical form). Let I \mathcal{I} I be a family of disjoint intervals in [ n ] [n] [ n ] , each a union of nonterminal blocks. Put U = ⋃ I ∈ I I U=\bigcup_{I\in\mathcal{I}} I U = ⋃ I ∈ I I , and suppose that there is at least one block outside U U U between consecutive intervals. Let f : X U → Y U f:X_U\to Y_U f : X U → Y U be a partial block mapping. We say that ( X , Y , f ; I ) (X,Y,f;\mathcal{I}) ( X , Y , f ; I ) is in canonical form if, for Z ∈ { X , Y } Z\in\{X,Y\} Z ∈ { X , Y } and i ∈ [ n − 1 ] i\in[n-1] i ∈ [ n − 1 ] ,
Δ i Z ≤ s 2 / n \Delta_i^Z \le s^2/n Δ i Z ≤ s 2 / n ;
if { i , i + 1 } ⊈ U \{i,i+1\}\not\subseteq U { i , i + 1 } ⊆ U , then Δ i Z ≥ 1 / ( 2 n s t ) \Delta_i^Z \ge1/(2ns^t) Δ i Z ≥ 1/ ( 2 n s t ) ;
if i , i + 1 ∈ U i,i+1\in U i , i + 1 ∈ U and p ∣ i p\mid i p ∣ i , then Δ i Z ≥ 1 / ( s n ) \Delta_i^Z \ge1/(sn) Δ i Z ≥ 1/ ( s n ) .
Notice that Item 1 requires every gap to have length O ~ ( 1 / n ) \widetilde{O}(1/n) O ( 1/ n ) . This property holds whp for the original samples by Corollary 2.8 and will remain true after rearrangement. Item 2 reflects that, in our construction, we pass to canonical form only after obtaining the partial mapping f f f from Proposition 6.5 . Every tiny gap then lies inside a prescribed interval. Thus every gap with at least one endpoint indexed outside U U U is nontiny, giving the lower bound in Item 2 on the event Σ X , Σ Y ≤ 2 n \Sigma_X,\Sigma_Y\le2n Σ X , Σ Y ≤ 2 n . Item 3 is the property we seek to enforce: every gap between consecutive free blocks has length Ω ~ ( 1 / n ) \widetilde{\Omega}(1/n) Ω ( 1/ n ) . In the original random samples, gaps at block boundaries can be much smaller. To achieve canonical form, we will use the following claim.
Claim 6.23 (Deterministic rearrangement). Let A = { a 1 < ⋯ < a ℓ } ⊆ R A=\{a_1<\cdots<a_\ell\}\subseteq\mathbb{R} A = { a 1 < ⋯ < a ℓ } ⊆ R , where ℓ ≥ 3 \ell\ge3 ℓ ≥ 3 , and let T > 0 T>0 T > 0 . Suppose at least a 0.9 0.9 0.9 -fraction of the gaps a i + 1 − a i a_{i+1}-a_i a i + 1 − a i , i ∈ [ ℓ − 1 ] i\in[\ell-1] i ∈ [ ℓ − 1 ] , are at least T T T , including the first and last gaps.
Then for every 1 ≤ k < ℓ 1\le k<\ell 1 ≤ k < ℓ , there exist an ordered set A ′ = { a 1 ′ < ⋯ < a ℓ ′ } A'=\{a'_1<\cdots<a'_\ell\} A ′ = { a 1 ′ < ⋯ < a ℓ ′ } and a bijection R : A → A ′ R:A\to A' R : A → A ′ such that:
R R R fixes the endpoints, which remain the endpoints of A ′ A' A ′ , i.e., R ( a 1 ) = a 1 ′ = a 1 R(a_1)=a'_1=a_1 R ( a 1 ) = a 1 ′ = a 1 and R ( a ℓ ) = a ℓ ′ = a ℓ R(a_\ell)=a'_\ell=a_\ell R ( a ℓ ) = a ℓ ′ = a ℓ .
a k + 1 ′ − a k ′ ≥ T a'_{k+1}-a'_k\ge T a k + 1 ′ − a k ′ ≥ T .
The multiset of gap lengths is preserved.
We have max { Lip ( R ) , Lip ( R − 1 ) } ≤ ( a ℓ − a 1 ) / T \max\{\operatorname{Lip}(R),\operatorname{Lip}(R^{-1})\}\le(a_\ell-a_1)/T max { Lip ( R ) , Lip ( R − 1 )} ≤ ( a ℓ − a 1 ) / T .
Proof. For 1 ≤ j ≤ ℓ − 2 1\le j\le\ell-2 1 ≤ j ≤ ℓ − 2 , put
c L = a ℓ − 1 − a j , c R = a j + 1 − a 2 . c_L=a_{\ell-1}-a_j,\qquad c_R=a_{j+1}-a_2. c L = a ℓ − 1 − a j , c R = a j + 1 − a 2 .
Define the j j j th rearrangement map R j R_j R j by R j ( a 1 ) = a 1 R_j(a_1)=a_1 R j ( a 1 ) = a 1 , R j ( a ℓ ) = a ℓ R_j(a_\ell)=a_\ell R j ( a ℓ ) = a ℓ , and
R j ( a i ) = { a i + c L , 2 ≤ i ≤ j , a i − c R , j + 1 ≤ i ≤ ℓ − 1. R_j(a_i)=
\begin{cases}
a_i+c_L, & 2\le i\le j,\\
a_i-c_R, & j+1\le i\le\ell-1.
\end{cases} R j ( a i ) = { a i + c L , a i − c R , 2 ≤ i ≤ j , j + 1 ≤ i ≤ ℓ − 1.
Set A j = R j ( A ) = { b 1 ( j ) < ⋯ < b ℓ ( j ) } A_j=R_j(A)=\{b^{(j)}_1<\cdots<b^{(j)}_\ell\} A j = R j ( A ) = { b 1 ( j ) < ⋯ < b ℓ ( j ) } . We claim that, for some j j j , A j A_j A j is the desired set A ′ A' A ′ and R : = R j R:=R_j R := R j is the desired map.
It is easy to check that Items 1 and 3 hold for every j j j by construction; see Figure 5 .
Figure 5. Eight points before (top) and after (bottom) applying R 3 R_3 R 3 . In the application, A A A is a set of consecutive points from one sample, X X X or Y Y Y ; the rows show its original and rearranged configurations. The endpoints a 1 , a 8 a_1,a_8 a 1 , a 8 are fixed. The blue interval translates right by c L = a 7 − a 3 c_L=a_7-a_3 c L = a 7 − a 3 , and the red interval translates left by c R = a 4 − a 2 c_R=a_4-a_2 c R = a 4 − a 2 , preserving all distances within each interval. The marked gap has length a 4 − a 3 a_4-a_3 a 4 − a 3 in both rows.
To verify Item 2, we choose j j j avoiding both of the following conditions:
a j + 1 − a j < T (short cut gap), a_{j+1}-a_j<T\qquad\text{(short cut gap),} a j + 1 − a j < T (short cut gap),
b k + 1 ( j ) − b k ( j ) < T (short target gap). b^{(j)}_{k+1}-b^{(j)}_k<T\qquad\text{(short target gap).} b k + 1 ( j ) − b k ( j ) < T (short target gap).
Let r r r be the number of gaps in A A A smaller than T T T ; then r ≤ ( ℓ − 1 ) / 10 r\le(\ell-1)/10 r ≤ ( ℓ − 1 ) /10 . At most r r r values of j j j satisfy the first condition. Among the remaining values of j j j , at most r r r satisfy the second: each original short gap occupies the target gap position for at most one value of j j j . Since 2 r < ℓ − 2 2r<\ell-2 2 r < ℓ − 2 , we may choose j j j satisfying Item 2 and a j + 1 − a j ≥ T a_{j+1}-a_j\ge T a j + 1 − a j ≥ T . Fix such a j j j and set A ′ = A j A'=A_j A ′ = A j and R = R j R=R_j R = R j .
It remains to verify Item 4. Distances within { a 2 , … , a j } \{a_2,\ldots,a_j\} { a 2 , … , a j } and within { a j + 1 , … , a ℓ − 1 } \{a_{j+1},\ldots,a_{\ell-1}\} { a j + 1 , … , a ℓ − 1 } are preserved by R R R . For every other pair of distinct points, both its distance in A A A and the distance between its images in A ′ A' A ′ lie between T T T and a ℓ − a 1 a_\ell-a_1 a ℓ − a 1 . Thus both Lipschitz constants are at most ( a ℓ − a 1 ) / T (a_\ell-a_1)/T ( a ℓ − a 1 ) / T . ∎
We will apply Claim 6.23 locally, and separately to X X X and Y Y Y . The following claim gives the required gap bounds.
Claim 6.24. Put T = 1 / ( s n ) T=1/(sn) T = 1/ ( s n ) . With high probability, the following hold simultaneously for Z ∈ { X , Y } Z\in\{X,Y\} Z ∈ { X , Y } :
Among the p − 1 p-1 p − 1 gaps between any p p p consecutive points, at least a 0.9-fraction have length at least T T T .
The last p p p gaps, Δ n − p Z , … , Δ n − 1 Z \Delta^Z_{n-p},\ldots,\Delta^Z_{n-1} Δ n − p Z , … , Δ n − 1 Z , are at least T T T .
Proof. By Observation 6.8, Σ X , Σ Y ≤ 2 n \Sigma_X,\Sigma_Y\le2n Σ X , Σ Y ≤ 2 n with high probability. When Σ Z ≤ 2 n \Sigma_Z\le2n Σ Z ≤ 2 n , a gap smaller than T T T must have E i Z < 2 / s E_i^Z<2/s E i Z < 2/ s . The events E i Z < 2 / s E_i^Z<2/s E i Z < 2/ s are independent as i i i and Z Z Z vary, each occurring with probability at most 2 / s 2/s 2/ s .
To prove Item 1, it suffices to take a union bound over the at most n n n windows of p − 1 p-1 p − 1 consecutive gaps in each sample. The probability that some window contains at least ( p − 1 ) / 10 (p-1)/10 ( p − 1 ) /10 gaps smaller than T T T , while Σ X , Σ Y ≤ 2 n \Sigma_X,\Sigma_Y\le2n Σ X , Σ Y ≤ 2 n , is at most
2 n 2 p − 1 ( 2 s ) ⌈ ( p − 1 ) / 10 ⌉ = n e − Ω ( p log s ) = o ( 1 ) . 2n\,2^{p-1}\binom{2}{s}^{\lceil(p-1)/10\rceil}=ne^{-\Omega(p\log s)}=o(1). 2 n 2 p − 1 ( s 2 ) ⌈( p − 1 ) /10 ⌉ = n e − Ω ( p l o g s ) = o ( 1 ) . For Item 2, a union bound over the last p p p gaps in both samples bounds the probability of failure while Σ X , Σ Y ≤ 2 n \Sigma_X,\Sigma_Y\le2n Σ X , Σ Y ≤ 2 n by 2 p ⋅ ( 2 / s ) = 4 p / s = o ( 1 ) 2p\cdot(2/s)=4p/s=o(1) 2 p ⋅ ( 2/ s ) = 4 p / s = o ( 1 ) . This proves the claim.
Claim 6.25. Put T = 1 / ( s n ) T=1/(sn) T = 1/ ( s n ) and fix Z ∈ { X , Y } Z\in\{X,Y\} Z ∈ { X , Y } . On the event of Claim 6.24, for each block boundary b = j p b=jp b = j p with Δ b Z < T \Delta_b^Z<T Δ b Z < T , there is an interval A b A_b A b of at most p p p consecutive points containing z b , z b + 1 z_b,z_{b+1} z b , z b + 1 . The intervals A b A_b A b are pairwise disjoint, and at least a 0.9-fraction of the gaps in each are at least T T T , including the first and last.
Proof. Fix Z ∈ { X , Y } Z\in\{X,Y\} Z ∈ { X , Y } . The gap preceding the terminal block, at b = ( M − 1 ) p b=(M-1)p b = ( M − 1 ) p , is among the last p p p gaps and hence is at least T T T by Item 2. Consequently, every boundary with gap smaller than T T T
lies between two full blocks. For each such boundary b = j p b=jp b = j p , let A ^ b \widehat{A}_b A b consist of the last p / 2 p/2 p /2 points of B j Z B_j^Z B j Z and the first p / 2 p/2 p /2 points of B j + 1 Z B_{j+1}^Z B j + 1 Z .
By Item 1, A ^ b \widehat{A}_b A b has at most ( p − 1 ) / 10 (p-1)/10 ( p − 1 ) /10 short gaps. Each half has p / 2 − 1 > ( p − 1 ) / 10 p/2-1>(p-1)/10 p /2 − 1 > ( p − 1 ) /10 gaps, so it contains a gap of length at least T T T . Trim A ^ b \widehat{A}_b A b at both ends until its first and last gaps are at least T T T , obtaining A b A_b A b . Since both the left and right parts of A ^ b \widehat{A}_b A b contain a gap of length at least T T T , A b A_b A b contains z b , z b + 1 z_b,z_{b+1} z b , z b + 1 . Since we removed only short gaps, at least a 0.9-fraction of the gaps of A b A_b A b are at least T T T . Finally, the A ^ b \widehat{A}_b A b are pairwise disjoint, and hence so are the A b A_b A b .
Lemma 6.26 (Rearrangement). With high probability, for every f , I f,\mathcal{I} f , I satisfying the conclusions of Proposition 6.5 , with U = ⋃ I ∈ I I U=\bigcup_{I\in\mathcal{I}} I U = ⋃ I ∈ I I , there exist ordered sets X ′ = { x 1 ′ < ⋯ < x n ′ } X'=\{x'_1<\cdots<x'_n\} X ′ = { x 1 ′ < ⋯ < x n ′ } , Y ′ = { y 1 ′ < ⋯ < y n ′ } Y'=\{y'_1<\cdots<y'_n\} Y ′ = { y 1 ′ < ⋯ < y n ′ } and bijections R Z : Z → Z ′ R_Z:Z\to Z' R Z : Z → Z ′ , for Z ∈ { X , Y } Z\in\{X,Y\} Z ∈ { X , Y } , such that
R Z ( z i ) = z i ′ = z i ( i ∈ U ) , R_Z(z_i)=z'_i=z_i \qquad(i\in U), R Z ( z i ) = z i ′ = z i ( i ∈ U ) , and max { Lip ( R Z ) , Lip ( R Z − 1 ) } = O ( L 5 ) \max\{\operatorname{Lip}(R_Z),\operatorname{Lip}(R_Z^{-1})\}=O(L^5) max { Lip ( R Z ) , Lip ( R Z − 1 )} = O ( L 5 ) . Moreover, f ′ = R Y ∘ f ∘ R X − 1 f'=R_Y\circ f\circ R_X^{-1} f ′ = R Y ∘ f ∘ R X − 1 is induced by the same partial involution as f f f , and ( X ′ , Y ′ , f ′ ; I ) (X',Y',f';\mathcal{I}) ( X ′ , Y ′ , f ′ ; I ) is in canonical form.
Proof. Put T = 1 / ( s n ) T=1/(sn) T = 1/ ( s n ) . Work on the events of Observation 6.8 and Claim 6.24 , and fix f , I f,\mathcal{I} f , I satisfying the conclusions of Proposition 6.5 . We construct R X R_X R X and R Y R_Y R Y separately; fix Z ∈ { X , Y } Z\in\{X,Y\} Z ∈ { X , Y } .
For each boundary b b b between free blocks with Δ b Z < T \Delta_b^Z<T Δ b Z < T , take A b A_b A b from Claim 6.25 . These intervals are pairwise disjoint and contain only free points, i.e., points in free blocks (recall Definition 6.4 ). Applying Claim 6.23 to each A b A_b A b , we obtain a map R b R_b R b such that the gap at index b b b after rearrangement is at least T T T , and
max { Lip ( R b ) , Lip ( R b − 1 ) } ≤ diam ( A b ) T ≤ 8 p s L = O ( L 5 ) , \max\{\operatorname{Lip}(R_b),\operatorname{Lip}(R_b^{-1})\}\le\frac{\operatorname{diam}(A_b)}{T}\le8psL=O(L^5), max { Lip ( R b ) , Lip ( R b − 1 )} ≤ T diam ( A b ) ≤ 8 p s L = O ( L 5 ) ,
where we used ∣ A b ∣ ≤ p |A_b|\le p ∣ A b ∣ ≤ p and Observation 6.8 .
Apply all these maps simultaneously, fixing every point outside the intervals A b A_b A b , to obtain R Z R_Z R Z , and put Z ′ = R Z ( Z ) Z'=R_Z(Z) Z ′ = R Z ( Z ) . Since the A b A_b A b are pairwise disjoint and each R b R_b R b fixes its endpoints, Claim 2.4 bounds Lip ( R Z ) \operatorname{Lip}(R_Z) Lip ( R Z ) by the maximum of 1 1 1 and the Lip ( R b ) \operatorname{Lip}(R_b) Lip ( R b ) ; the same holds for the inverses. Consequently,
max { Lip ( R Z ) , Lip ( R Z − 1 ) } ≤ 8 p s L = O ( L 5 ) . \max\{\operatorname{Lip}(R_Z),\operatorname{Lip}(R_Z^{-1})\}\le8psL=O(L^5). max { Lip ( R Z ) , Lip ( R Z − 1 )} ≤ 8 p s L = O ( L 5 ) .
Each R b R_b R b rearranges only free points within A b A_b A b , so the points with indices in U U U retain their positions and indices. Hence R Z ( z i ) = z i ′ = z i R_Z(z_i)=z'_i=z_i R Z ( z i ) = z i ′ = z i for i ∈ U i\in U i ∈ U , and f ′ f' f ′ is induced by the same partial block permutation as f f f .
We verify the three conditions of canonical form.
By Item 3, and Claim 6.23 the multiset of gap lengths is preserved, so every gap remains at most 8 L / n ≤ s 2 / n 8L/n\le s^2/n 8 L / n ≤ s 2 / n .
By Item 2 of Proposition 6.5 , E i Z > s − t E_i^Z>s^{-t} E i Z > s − t whenever { i , i + 1 } ⊈ U \{i,i+1\}\not\subseteq U { i , i + 1 } ⊆ U . Since Σ Z ≤ 2 n \Sigma_Z\le2n Σ Z ≤ 2 n , each such gap initially has length at least 1 / ( 2 n s t ) 1/(2nst) 1/ ( 2 n s t ) . These gaps are either unchanged or permuted within intervals of free points, so the lower bound persists.
Each boundary gap shorter than T T T between free blocks becomes at least T T T . Every other block-boundary gap is unchanged, since each R b R_b R b fixes its endpoints and acts within the two blocks adjacent to b b b . Thus every boundary between free blocks has gap at least T T T .
Hence ( X ′ , Y ′ , f ′ ; I ) (X',Y',f';\mathcal{I}) ( X ′ , Y ′ , f ′ ; I ) is in canonical form.
Reduction to permutations
We now use Lemma 6.26 to show that, after the pattern matching step of Proposition 6.5 , it remains to find a suitable block extension of the partial mapping f f f constructed there. We recall the relevant setup and conclusions in the following proposition.
For I ⊆ [ n ] I \subseteq[n] I ⊆ [ n ] , recall that B ( I ) = { j ∈ [ M ] : B j ⊆ I } B(I)=\{j\in[M]:B_j\subseteq I\} B ( I ) = { j ∈ [ M ] : B j ⊆ I } . For a family I \mathcal{I} I of prescribed intervals with union U U U , we have B ( U ) = ⋃ I ∈ I B ( I ) B(U)=\bigcup_{I\in\mathcal{I}}B(I) B ( U ) = ⋃ I ∈ I B ( I ) .
Proposition 6.27. Let X , Y ⊆ [ 0 , 1 ] X,Y\subseteq[0,1] X , Y ⊆ [ 0 , 1 ] have size n n n , and let the family I \mathcal{I} I and the partial block mapping f : X U → Y U f:X_U\to Y_U f : X U → Y U satisfy the conclusions of Proposition 6.5 , where U = ⋃ I ∈ I I U=\bigcup_{I\in\mathcal{I}}I U = ⋃ I ∈ I I . Assume that the conclusion of Lemma 6.26 holds for ( X , Y , f ; I ) (X,Y,f;\mathcal{I}) ( X , Y , f ; I ) . Let σ : B ( U ) → B ( U ) \sigma:B(U)\to B(U) σ : B ( U ) → B ( U ) be the partial involution inducing f f f as in Definition 6.4 . Suppose that there exists a permutation π \pi π of [ M ] [M] [ M ] such that
π \pi π extends σ \sigma σ and fixes M M M .
max { Lip ( π ) , Lip ( π − 1 ) } = O ( D ) \max\{\operatorname{Lip}(\pi),\operatorname{Lip}(\pi^{-1})\}=O(D) max { Lip ( π ) , Lip ( π − 1 )} = O ( D ) .
For every I ∈ I I\in\mathcal{I} I ∈ I , putting a = min B ( I ) a=\min B(I) a = min B ( I ) and b = max B ( I ) b=\max B(I) b = max B ( I ) , we have
π ( a − 1 ) = σ ( a ) − 1 , π ( b + 1 ) = σ ( b ) + 1. \pi(a-1)=\sigma(a)-1,\qquad\pi(b+1)=\sigma(b)+1. π ( a − 1 ) = σ ( a ) − 1 , π ( b + 1 ) = σ ( b ) + 1. Then there is a bijection F : X → Y F:X\to Y F : X → Y extending f f f with dist ( F ) = O ~ ( n 2 / 3 ) \operatorname{dist}(F)=\widetilde{O}(n^{2/3}) dist ( F ) = O ( n 2/3 ) .
Proof. Our plan is to use the block mapping F ′ F' F ′ induced by π \pi π between X ′ = R X ( X ) X'=R_X(X) X ′ = R X ( X ) and Y ′ = R Y ( Y ) Y'=R_Y(Y) Y ′ = R Y ( Y ) , where R X , R Y R_X,R_Y R X , R Y are provided by Lemma 6.26 . After bounding the Lipschitz constants of F ′ F' F ′ and its inverse, we transfer this map back by setting F = R Y − 1 ∘ F ′ ∘ R X F=R_Y^{-1}\circ F'\circ R_X F = R Y − 1 ∘ F ′ ∘ R X . The polylogarithmic Lipschitz bounds for the rearrangement maps and their inverses then give the desired distortion bound for F F F .
Apply Lemma 6.26 to obtain X ′ , Y ′ X',Y' X ′ , Y ′ , bijections R X , R Y R_X,R_Y R X , R Y , and f ′ = R Y ∘ f ∘ R X − 1 f'=R_Y\circ f\circ R_X^{-1} f ′ = R Y ∘ f ∘ R X − 1 such that ( X ′ , Y ′ , f ′ ; I ) (X',Y',f';\mathcal{I}) ( X ′ , Y ′ , f ′ ; I ) is in canonical form. The map f ′ f' f ′ is induced by the same σ \sigma σ . Since the rearrangement fixes every point whose index lies in U U U , Item 3 gives
max { Lip ( f ′ ∣ X I ′ ) , Lip ( ( f ′ ) − 1 ∣ Y I ′ ) } = O ( L s t + 1 ) ( I ∈ I ) . \max\{\operatorname{Lip}(f'|_{X'_I}),\operatorname{Lip}((f')^{-1}|_{Y'_I})\}=O(Ls^{t+1})\qquad(I\in\mathcal{I}). max { Lip ( f ′ ∣ X I ′ ) , Lip (( f ′ ) − 1 ∣ Y I ′ )} = O ( L s t + 1 ) ( I ∈ I ) .
Let F ′ : X ′ → Y ′ F':X'\to Y' F ′ : X ′ → Y ′ be the block mapping induced by π \pi π .
We bound the expansion of consecutive points x i ′ , x i + 1 ′ x'_i,x'_{i+1} x i ′ , x i + 1 ′ under F ′ F' F ′ . We distinguish four cases according to their locations.
Both indices lie in U U U . Since the prescribed intervals I \mathcal{I} I are separated, both indices belong to the same I ∈ I I\in\mathcal{I} I ∈ I . The map F ′ F' F ′ agrees with f ′ f' f ′ there, so their expansion is O ( L s t + 1 ) O(Ls^{t+1}) O ( L s t + 1 ) by Item 3 of Proposition 6.5 .
Both points lie in the same free block. Their images lie in the same block of Y ′ Y' Y ′ and are consecutive, since F ′ F' F ′ maps each block increasingly. Items 1 and 2 of Definition 6.22 (canonical form) give
x i + 1 ′ − x i ′ ≥ 1 2 n s t , ∣ F ′ ( x i + 1 ′ ) − F ′ ( x i ′ ) ∣ ≤ s 2 n . x'_{i+1}-x'_i\geq\frac{1}{2ns^t},\qquad\left|F'(x'_{i+1})-F'(x'_i)\right|\leq\frac{s^2}{n}. x i + 1 ′ − x i ′ ≥ 2 n s t 1 , F ′ ( x i + 1 ′ ) − F ′ ( x i ′ ) ≤ n s 2 .
Thus their expansion is at most ( s 2 / n ) / ( 1 / ( 2 n s t ) ) = 2 s t + 2 (s^2/n)/(1/(2ns^t))=2s^{t+2} ( s 2 / n ) / ( 1/ ( 2 n s t )) = 2 s t + 2 .
Exactly one index lies in U U U . The points lie across a boundary of some I ∈ I I \in\mathcal{I} I ∈ I . Put a = min B ( I ) a = \min B(I) a = min B ( I ) and b = max B ( I ) b = \max B(I) b = max B ( I ) . Here we use the crucial neighbor identities π ( a − 1 ) = σ ( a ) − 1 \pi(a-1)=\sigma(a)-1 π ( a − 1 ) = σ ( a ) − 1 and π ( b + 1 ) = σ ( b ) + 1 \pi(b+1)=\sigma(b)+1 π ( b + 1 ) = σ ( b ) + 1 . They ensure that F ′ F' F ′ sends the blocks immediately before and after X I ′ X'_I X I ′ to the blocks immediately before and after f ′ ( X I ′ ) f'(X'_I) f ′ ( X I ′ ) , respectively. Since each block is mapped increasingly, the two points across either boundary have consecutive images. Items 1 and 2 of Definition 6.22 gives x i + 1 ′ − x i ′ ≥ 1 / ( 2 n s t ) x'_{i+1}-x'_i \ge1/(2ns^t) x i + 1 ′ − x i ′ ≥ 1/ ( 2 n s t ) and ∣ F ′ ( x i + 1 ′ ) − F ′ ( x i ′ ) ∣ ≤ s 2 / n |F'(x'_{i+1})-F'(x'_i)| \le s^2/n ∣ F ′ ( x i + 1 ′ ) − F ′ ( x i ′ ) ∣ ≤ s 2 / n , so their expansion is at most 2 s t + 2 2s^{t+2} 2 s t + 2 .
The points lie in different free blocks. Here i = j p i=jp i = j p for some j ∈ [ M − 1 ] j\in[M-1] j ∈ [ M − 1 ] . Crucially, because we first applied the rearrangement, Item 3 of Definition 6.22 gives
x j p + 1 ′ − x j p ′ ≥ 1 s n . x'_{jp+1}-x'_{jp}\ge\frac{1}{sn}. x j p + 1 ′ − x j p ′ ≥ s n 1 .
The image indices are π ( j ) p \pi(j)p π ( j ) p and ( π ( j + 1 ) − 1 ) p + 1 (\pi(j+1)-1)p+1 ( π ( j + 1 ) − 1 ) p + 1 . This formula also holds when j + 1 = M j+1=M j + 1 = M , since π ( M ) = M \pi(M)=M π ( M ) = M . The upper gap bound in Item 1 therefore gives
∣ F ′ ( x j p + 1 ′ ) − F ′ ( x j p ′ ) ∣ x j p + 1 ′ − x j p ′ ≤ s 3 p ( ∣ π ( j + 1 ) − π ( j ) ∣ + 1 ) ≤ 2 s 3 p Lip ( π ) . \frac{\left|F'(x'_{jp+1})-F'(x'_{jp})\right|}{x'_{jp+1}-x'_{jp}}
\le s^3p\left(|\pi(j+1)-\pi(j)|+1\right)\le2s^3p\operatorname{Lip}(\pi). x j p + 1 ′ − x j p ′ F ′ ( x j p + 1 ′ ) − F ′ ( x j p ′ ) ≤ s 3 p ( ∣ π ( j + 1 ) − π ( j ) ∣ + 1 ) ≤ 2 s 3 p Lip ( π ) .
Combining the cases and applying Claim 2.4 yields
Lip ( F ′ ) ≤ max { O ( L s t + 1 ) , 2 s t + 2 , 2 s 3 p Lip ( π ) } . \operatorname{Lip}(F')\le\max\{O(Ls^{t+1}),2s^{t+2},2s^3p\operatorname{Lip}(\pi)\}. Lip ( F ′ ) ≤ max { O ( L s t + 1 ) , 2 s t + 2 , 2 s 3 p Lip ( π )} .
By symmetry, the same bound holds for Lip ( ( F ′ ) − 1 ) \operatorname{Lip}((F')^{-1}) Lip (( F ′ ) − 1 ) with π − 1 \pi^{-1} π − 1 in place of π \pi π . Substituting max { Lip ( π ) , Lip ( π − 1 ) } = O ( D ) \max\{\operatorname{Lip}(\pi),\operatorname{Lip}(\pi^{-1})\}=O(D) max { Lip ( π ) , Lip ( π − 1 )} = O ( D ) , s t ≤ s 2 D s^t\le s^2D s t ≤ s 2 D , s = L 3 s=L^3 s = L 3 , and p = C 0 L p=C_0L p = C 0 L into these bounds gives
max { Lip ( F ′ ) , Lip ( ( F ′ ) − 1 ) } = O ~ ( D ) . \max\{\operatorname{Lip}(F'),\operatorname{Lip}((F')^{-1})\}=\widetilde{O}(D). max { Lip ( F ′ ) , Lip (( F ′ ) − 1 )} = O ( D ) .
Finally, transfer the mapping back to the original samples by setting F = R Y − 1 ∘ F ′ ∘ R X F=R_Y^{-1}\circ F'\circ R_X F = R Y − 1 ∘ F ′ ∘ R X . Since F ′ F' F ′ extends f ′ f' f ′ , the map F F F extends f f f . The O ( L 5 ) O(L^5) O ( L 5 ) Lipschitz bounds for the rearrangement maps and their inverses imply
dist ( F ) = Lip ( F ) Lip ( F − 1 ) = O ~ ( D 2 ) = O ~ ( n 2 / 3 ) . \operatorname{dist}(F)=\operatorname{Lip}(F)\operatorname{Lip}(F^{-1})=\widetilde{O}(D^2)=\widetilde{O}(n^{2/3}). dist ( F ) = Lip ( F ) Lip ( F − 1 ) = O ( D 2 ) = O ( n 2/3 ) .
Oscillations
Lemma 6.28 (Oscillations on intervals). Let m ≥ 1 m\ge1 m ≥ 1 and let I 1 , … , I m I_1,\ldots,I_m I 1 , … , I m be nonempty disjoint integer intervals listed in increasing order, with ∣ I j ∣ > 10 m |I_j|>10m ∣ I j ∣ > 10 m . Write ℓ j = min I j \ell_j=\min I_j ℓ j = min I j , u j = max I j u_j=\max I_j u j = max I j , and let τ \tau τ be an involution of [ m − 1 ] [m-1] [ m − 1 ] . Then there exists an involution π \pi π of I 1 ∪ ⋯ ∪ I m I_1\cup\cdots\cup I_m I 1 ∪ ⋯ ∪ I m satisfying
π ( ℓ 1 ) = ℓ 1 , π ( ℓ j ) = ℓ τ ( j − 1 ) + 1 ( 2 ≤ j ≤ m ) , \pi(\ell_1)=\ell_1,\qquad\pi(\ell_j)=\ell_{\tau(j-1)+1}\qquad(2\le j\le m), π ( ℓ 1 ) = ℓ 1 , π ( ℓ j ) = ℓ τ ( j − 1 ) + 1 ( 2 ≤ j ≤ m ) , π ( u m ) = u m , π ( u j ) = u τ ( j ) ( 1 ≤ j < m ) . \pi(u_m)=u_m,\qquad\pi(u_j)=u_{\tau(j)}\qquad(1\le j<m). π ( u m ) = u m , π ( u j ) = u τ ( j ) ( 1 ≤ j < m ) . If i , i + 1 ∈ I j i,i+1\in I_j i , i + 1 ∈ I j and π ( i ) ∈ I k \pi(i)\in I_k π ( i ) ∈ I k , π ( i + 1 ) ∈ I k ′ \pi(i+1)\in I_{k'} π ( i + 1 ) ∈ I k ′ , then ∣ k − k ′ ∣ ≤ 1 |k-k'|\le1 ∣ k − k ′ ∣ ≤ 1 .
Proof. The reader may consult Figure 6 throughout the proof. Let s i , t i s_i,t_i s i , t i be the indices of the intervals to which the left and right endpoints of I i I_i I i must be mapped, respectively. Thus
Figure 6. An illustration of the construction in Lemma 6.28 for τ = ( 1 , 2 ) \tau=(1,2) τ = ( 1 , 2 ) , with nine points per interval and r i j = 3 r_{ij}=3 r ij = 3 for i , j = 1 , 2 , 3 i,j=1,2,3 i , j = 1 , 2 , 3 . The endpoint assignments (black) extend to an involution by pairing the point of C i j ′ C'_{ij} C ij ′ with those of C j i ′ C'_{ji} C j i ′ (colored arrows) and fixing C i i ′ C'_{ii} C ii ′ (gray arrows).
s 1 = 1 , s i = τ ( i − 1 ) + 1 ( 2 ≤ i ≤ m ) , s_1=1,\qquad s_i=\tau(i-1)+1\qquad(2\le i\le m), s 1 = 1 , s i = τ ( i − 1 ) + 1 ( 2 ≤ i ≤ m ) ,
t m = m , t i = τ ( i ) ( 1 ≤ i < m ) . t_m=m,\qquad t_i=\tau(i)\qquad(1\le i<m). t m = m , t i = τ ( i ) ( 1 ≤ i < m ) .
We construct π \pi π in two steps: first a labeling of I 1 ∪ ⋯ ∪ I m I_1 \cup\cdots\cup I_m I 1 ∪ ⋯ ∪ I m by [ m ] [m] [ m ] , then an involution respecting that labeling.
Constructing the labeling. Given a labeling of I 1 ∪ ⋯ ∪ I m I_1 \cup\cdots\cup I_m I 1 ∪ ⋯ ∪ I m by [ m ] [m] [ m ] , write
C i j = { x ∈ I i : x is labeled j } . C_{ij} = \{x \in I_i : x \text{ is labeled } j\}. C ij = { x ∈ I i : x is labeled j } .
Set
r i j = 3 ( i ≠ j ) , r i i = ∣ I i ∣ − 3 ( m − 1 ) . r_{ij}=3 \quad(i\ne j), \qquad r_{ii}=|I_i|-3(m-1). r ij = 3 ( i = j ) , r ii = ∣ I i ∣ − 3 ( m − 1 ) .
Then r i j = r j i ≥ 3 r_{ij}=r_{ji}\ge3 r ij = r j i ≥ 3 and ∑ j r i j = ∣ I i ∣ \sum_j r_{ij}=|I_i| ∑ j r ij = ∣ I i ∣ . We will arrange that ∣ C i j ∣ = r i j |C_{ij}|=r_{ij} ∣ C ij ∣ = r ij for every i , j ∈ [ m ] i,j\in[m] i , j ∈ [ m ] . For each i i i , concatenate the three walks
( s i , s i − 1 , … , 1 ) , ( 1 , 2 , … , m ) , ( m , m − 1 , … , t i ) . (s_i,s_i-1,\ldots,1), \qquad(1,2,\ldots,m), \qquad(m,m-1,\ldots,t_i). ( s i , s i − 1 , … , 1 ) , ( 1 , 2 , … , m ) , ( m , m − 1 , … , t i ) .
Each label occurs between one and three times. Since r i j ≥ 3 r_{ij}\ge3 r ij ≥ 3 , insert consecutive repetitions so that label j j j occurs exactly r i j r_{ij} r ij times. The resulting sequence has length ∣ I i ∣ |I_i| ∣ I i ∣ ; use it to label the points of I i I_i I i in increasing order. The resulting labeling has the following properties:
The points ℓ i \ell_i ℓ i and u i u_i u i have labels s i s_i s i and t i t_i t i , respectively.
Consecutive points of I i I_i I i have labels differing by at most one.
∣ C i j ∣ = r i j = r j i = ∣ C j i ∣ |C_{ij}|=r_{ij}=r_{ji}=|C_{ji}| ∣ C ij ∣ = r ij = r j i = ∣ C j i ∣ for every i , j ∈ [ m ] i,j\in[m] i , j ∈ [ m ] .
Defining the involution. We now use this labeling to construct π \pi π . First prescribe π ( ℓ i ) = ℓ s i \pi(\ell_i)=\ell_{s_i} π ( ℓ i ) = ℓ s i and π ( u i ) = u t i \pi(u_i)=u_{t_i} π ( u i ) = u t i , and put C i j ′ = C i j ∖ { ℓ i , u i } C'_{ij}=C_{ij}\setminus\{\ell_i,u_i\} C ij ′ = C ij ∖ { ℓ i , u i } . Since τ \tau τ is an involution, s j = i s_j=i s j = i if and only if s i = j s_i=j s i = j , and t j = i t_j=i t j = i if and only if t i = j t_i=j t i = j . Thus removing the endpoints preserves the equality in Item 3, so ∣ C i j ′ ∣ = ∣ C j i ′ ∣ |C'_{ij}|=|C'_{ji}| ∣ C ij ′ ∣ = ∣ C j i ′ ∣ .
For each i < j i<j i < j , match C i j ′ C'_{ij} C ij ′ with C j i ′ C'_{ji} C j i ′ arbitrarily and let π \pi π exchange each matched pair. Let π \pi π fix every point of C i i ′ C'_{ii} C ii ′ . Together with the endpoint assignments, this defines an involution with the required endpoint values. Every point labeled j j j is mapped into I j I_j I j , so Item 2 gives the last assertion of this lemma.
Lemma 6.29 (Oscillating completion). Let J J J be an interval of integers, and let A 1 , … , A h A_1,\ldots,A_h A 1 , … , A h be nonempty pairwise disjoint subintervals of J J J , where h ≥ 0 h\ge0 h ≥ 0 . Let σ \sigma σ be an involution on A 1 ∪ ⋯ ∪ A h A_1\cup\cdots\cup A_h A 1 ∪ ⋯ ∪ A h such that σ \sigma σ maps each A i A_i A i increasingly and bijectively onto some A j A_j A j .
Suppose that, for positive integers b , q b,q b , q with q 2 ≥ 16 ∣ J ∣ q^2\ge16|J| q 2 ≥ 16∣ J ∣ ,
For every i i i , ∣ A i ∣ ≤ b |A_i|\le b ∣ A i ∣ ≤ b .
J ∖ ( A 1 ∪ ⋯ ∪ A h ) = B 1 ∪ ⋯ ∪ B h + 1 J\setminus(A_1\cup\cdots\cup A_h)=B_1\cup\cdots\cup B_{h+1} J ∖ ( A 1 ∪ ⋯ ∪ A h ) = B 1 ∪ ⋯ ∪ B h + 1 , where the B j B_j B j are intervals interlacing the A i A_i A i in the order B 1 , A 1 , B 2 , … , A h , B h + 1 B_1,A_1,B_2,\ldots,A_h,B_{h+1} B 1 , A 1 , B 2 , … , A h , B h + 1 , and each B j B_j B j has size at least q q q .
Then σ \sigma σ extends to an involution π \pi π of J J J fixing both endpoints and satisfying
Lip ( π ) ≤ 4 q + b . \operatorname{Lip}(\pi)\le4q+b. Lip ( π ) ≤ 4 q + b . Moreover, whenever σ ( A i ) = A j \sigma(A_i)=A_j σ ( A i ) = A j , the map π \pi π sends the point immediately before A i A_i A i to the point immediately before A j A_j A j , and the point immediately after A i A_i A i to the point immediately after A j A_j A j .
Proof. Partition each B j B_j B j into pieces of sizes between q q q and 2 q − 1 2q-1 2 q − 1 : take pieces of size q q q and add the remainder to the last piece. List all pieces in increasing order as I 1 , … , I m I_1,\ldots,I_m I 1 , … , I m , and write ℓ j = min I j \ell_j=\min I_j ℓ j = min I j , u j = max I j u_j=\max I_j u j = max I j . Since m q ≤ ∣ J ∣ mq\le|J| m q ≤ ∣ J ∣ and q 2 ≥ 16 ∣ J ∣ q^2\ge16|J| q 2 ≥ 16∣ J ∣ , we have ∣ I j ∣ ≥ q ≥ 16 m > 10 m |I_j|\ge q\ge16m>10m ∣ I j ∣ ≥ q ≥ 16 m > 10 m .
We now define π \pi π . We first define an involution τ \tau τ on [ m − 1 ] [m-1] [ m − 1 ] , which we will use with Lemma 6.28 to construct π \pi π . We set τ ( j ) = k \tau(j)=k τ ( j ) = k if A i A_i A i lies between I j I_j I j and I j + 1 I_{j+1} I j + 1 and σ ( A i ) \sigma(A_i) σ ( A i ) lies between I k I_k I k and I k + 1 I_{k+1} I k + 1 . For every remaining gap between I j I_j I j and I j + 1 I_{j+1} I j + 1 , both pieces lie in the same B r B_r B r ; set τ ( j ) = j \tau(j)=j τ ( j ) = j . Since σ \sigma σ is an involution, so is τ \tau τ . Apply Lemma 6.28 to obtain an involution π \pi π on I 1 ∪ ⋯ ∪ I m I_1\cup\cdots\cup I_m I 1 ∪ ⋯ ∪ I m , and extend π \pi π by σ \sigma σ on A 1 ∪ ⋯ ∪ A h A_1\cup\cdots\cup A_h A 1 ∪ ⋯ ∪ A h . This defines an involution of J J J extending $\sigma. For the last assertion of the lemma, consider the images of the points immediately before and after each A i A_i A i . By the endpoint conditions in Lemma 6.28, π \pi π fixes both endpoints of J J J . By our choice of τ \tau τ , whenever σ ( A i ) = A j \sigma(A_i)=A_j σ ( A i ) = A j , these conditions also give
π ( min A i − 1 ) = min A j − 1 , π ( max A i + 1 ) = max A j + 1. (10) \begin{aligned}
\pi(\min A_i-1)&=\min A_j-1,\\
\pi(\max A_i+1)&=\max A_j+1. \tag*{(10)}
\end{aligned} π ( min A i − 1 ) π ( max A i + 1 ) = min A j − 1 , = max A j + 1. (10)
Finally, we bound the Lipschitz constant. Fix consecutive points x , x + 1 ∈ J x,x+1\in J x , x + 1 ∈ J . We distinguish three cases according to the partition of J J J into the intervals A i A_i A i and I j I_j I j .
Both points lie in one A i A_i A i . Since π \pi π agrees with the increasing bijection σ \sigma σ on A i A_i A i , their images are consecutive, so ∣ π ( x + 1 ) − π ( x ) ∣ = 1 |\pi(x+1)-\pi(x)|=1 ∣ π ( x + 1 ) − π ( x ) ∣ = 1 .
The points lie in different intervals of the partition. If one belongs to an A i A_i A i , Equation (10) and the fact that σ \sigma σ maps A i A_i A i increasingly onto its image show that their images are consecutive. Otherwise, the points lie across a subdivision cut within one B r B_r B r . At such a cut, τ ( j ) = j \tau(j)=j τ ( j ) = j , so the involution supplied by Lemma 6.28 fixes both u j = x u_j=x u j = x and ℓ j + 1 = x + 1 \ell_{j+1}=x+1 ℓ j + 1 = x + 1 . Thus the image distance is again 1.
Both points lie in one I j I_j I j . By Lemma 6.28, their images lie in the same interval or in two successive intervals among I 1 , … , I m I_1,\ldots,I_m I 1 , … , I m . Each has size less than 2 q 2q 2 q , and successive intervals are separated by at most one A i A_i A i , of size at most b b b . Hence ∣ π ( x + 1 ) − π ( x ) ∣ ≤ 4 q + b |\pi(x+1)-\pi(x)|\le4q+b ∣ π ( x + 1 ) − π ( x ) ∣ ≤ 4 q + b .
In all cases the image distance is at most 4 q + b 4q+b 4 q + b , so Claim 2.4 gives Lip ( π ) ≤ 4 q + b \operatorname{Lip}(\pi)\le4q+b Lip ( π ) ≤ 4 q + b .
Proof of the upper-bound theorem
Proof of Theorem 6.1. Work on the event where Proposition 6.5 and Lemma 6.26 both hold, which has probability 1 − o ( 1 ) 1-o(1) 1 − o ( 1 ) . Choose f , I f,\mathcal{I} f , I from Proposition 6.5. For I ⊆ [ n ] I\subseteq[n] I ⊆ [ n ] , recall that B ( I ) = { j ∈ [ M ] : B j ⊆ I } B(I)=\{j\in[M]:B_j\subseteq I\} B ( I ) = { j ∈ [ M ] : B j ⊆ I } . Denote U = ⋃ I ∈ I I U=\bigcup_{I\in\mathcal{I}}I U = ⋃ I ∈ I I , and let σ : B ( U ) → B ( U ) \sigma:B(U)\to B(U) σ : B ( U ) → B ( U ) be the partial involution inducing f f f . By Proposition 6.27, it suffices to extend σ \sigma σ to an involution π \pi π fixing M M M , satisfying the neighbor identities from Proposition 6.27, and having Lip ( π ) = O ( D ) \operatorname{Lip}(\pi)=O(D) Lip ( π ) = O ( D ) . Since π \pi π is an involution, the inverse bound Lip ( π − 1 ) = O ( D ) \operatorname{Lip}(\pi^{-1})=O(D) Lip ( π − 1 ) = O ( D ) holds as well.
Let J 1 , … , J m J_1,\ldots,J_m J 1 , … , J m be the partition from Item 5 of Proposition 6.5. Each prescribed pair I , I ‾ I,\overline{I} I , I is contained in a common part J j J_j J j . We will apply Lemma 6.29 to intervals of block indices. Assign each block B k B_k B k to the part containing its largest index, and let
K j = { k ∈ [ M ] : max B k ∈ J j } . K_j=\{k\in[M]:\max B_k\in J_j\}. K j = { k ∈ [ M ] : max B k ∈ J j } .
The sets K 1 , … , K m K_1,\ldots,K_m K 1 , … , K m partition [ M ] [M] [ M ] into intervals of block indices. Since ∣ J j ∣ ≤ 3 H |J_j|\le3H ∣ J j ∣ ≤ 3 H , each has size at most 3 H / p + 2 3H/p+2 3 H / p + 2 .
If a prescribed pair I , I ‾ I,\overline{I} I , I lies in J j J_j J j , every block forming either interval lies entirely in J j J_j J j . Thus B ( I ) , B ( I ‾ ) ⊆ K j B(I),B(\overline{I})\subseteq K_j B ( I ) , B ( I ) ⊆ K j , and σ \sigma σ maps B ( I ) B(I) B ( I ) increasingly onto B ( I ‾ ) B(\overline{I}) B ( I ) . Consequently, σ \sigma σ restricts to an involution on B ( U ) ∩ K j B(U)\cap K_j B ( U ) ∩ K j and, below, we aim to show that σ ∣ B ( U ) ∩ K j \sigma|_{B(U)\cap K_j} σ ∣ B ( U ) ∩ K j can be extended to an involution π j \pi_j π j fixing the end-points of K j K_j K j and satisfying Lip ( π j ) = O ( D ) \operatorname{Lip}(\pi_j)=O(D) Lip ( π j ) = O ( D ) .
Fix a part K j K_j K j . We verify the remaining hypotheses of Lemma 6.29. By Item 4, each prescribed interval I I I has size O ( L D ) O(LD) O ( L D ) . Since I I I is a union of blocks of size p p p , we have ∣ B ( I ) ∣ = ∣ I ∣ / p = O ( L D / p ) |B(I)|=|I|/p=O(LD/p) ∣ B ( I ) ∣ = ∣ I ∣/ p = O ( L D / p ) . Choose a positive integer b = O ( L D / p ) = O ( D ) b=O(LD/p)=O(D) b = O ( L D / p ) = O ( D ) bounding the size B ( I ) B(I) B ( I ) for all I ∈ I I\in\mathcal{I} I ∈ I from above.
Next, we bound the sizes of the free intervals in K j K_j K j . Successive prescribed intervals of I \mathcal{I} I contained in J j J_j J j are separated by at least D / 2 D/2 D /2 points by Item 4 of Proposition 6.5. Then, Item 5 of that proposition gives the same margin between the first prescribed interval and the left endpoint of J j J_j J j , and between the last prescribed interval and the right endpoint of J j J_j J j . In passing to block indices, we allow for a loss of at most two blocks from rounding at the boundaries. Thus each maximal interval of free indices in K j K_j K j has size at least D / ( 2 p ) − 2 D/(2p)-2 D / ( 2 p ) − 2 .
We choose q = ⌊ D / ( 4 p ) ⌋ q=\lfloor D/(4p)\rfloor q = ⌊ D / ( 4 p )⌋ , leaving room for this rounding loss. Since D / p = A ⌈ H ⌉ D/p=A\lceil\sqrt{H}\rceil D / p = A ⌈ H ⌉ tends to infinity, for sufficiently large n n n we have D / ( 2 p ) − 2 ≥ D / ( 4 p ) ≥ q D/(2p)-2\ge D/(4p)\ge q D / ( 2 p ) − 2 ≥ D / ( 4 p ) ≥ q . This verifies the lower bound on the sizes of the free intervals required by Lemma 6.29 .
Finally, we verify that q 2 ≥ 16 ∣ K j ∣ q^2\ge16|K_j| q 2 ≥ 16∣ K j ∣ . Recall that ∣ K j ∣ ≤ 3 H / p + 2 ≤ 4 H |K_j|\le3H/p+2\le4H ∣ K j ∣ ≤ 3 H / p + 2 ≤ 4 H for sufficiently large n n n , so it suffices to show that q ≥ 8 H q\ge8\sqrt{H} q ≥ 8 H . Substituting D = A p ⌈ H ⌉ D=Ap\lceil\sqrt{H}\rceil D = A p ⌈ H ⌉ into the definition of q q q gives
q = ⌊ A ⌈ H ⌉ 4 ⌋ ≥ A H 4 − 1. q=\left\lfloor\frac{A\lceil\sqrt{H}\rceil}{4}\right\rfloor\ge\frac{A\sqrt{H}}{4}-1. q = ⌊ 4 A ⌈ H ⌉ ⌋ ≥ 4 A H − 1.
Taking A ≥ 64 A\ge64 A ≥ 64 , the right-hand side is at least 8 H 8\sqrt{H} 8 H , as required.
Apply Lemma 6.29 to each part K j K_j K j , taking all prescribed intervals B ( I ) , B ( I ‾ ) ⊆ K j B(I),B(\overline{I})\subseteq K_j B ( I ) , B ( I ) ⊆ K j as the sets A 1 , … , A h A_1,\ldots,A_h A 1 , … , A h , to obtain π j : K j → K j \pi_j:K_j\to K_j π j : K j → K j with Lip ( π j ) ≤ 4 q + b = O ( D ) \operatorname{Lip}(\pi_j)\le4q+b=O(D) Lip ( π j ) ≤ 4 q + b = O ( D ) .
Define π \pi π by π ∣ K j = π j \left.\pi\right|_{K_j}=\pi_j π ∣ K j = π j for every j j j and let us bound Lip ( π ) \operatorname{Lip}(\pi) Lip ( π ) . As the local maps π ∣ K j \left.\pi\right|_{K_j} π ∣ K j are involutions, π \pi π is an involution of [ M ] [M] [ M ] extending σ \sigma σ . Also, the map π \pi π fixes all endpoints of all parts K j K_j K j , including M M M . Since each prescribed block interval B ( I ) B(I) B ( I ) and its neighbors lie in one part K j K_j K j , π \pi π also satisfies the neighbor identities from Item 3 of Proposition 6.27, as π j \pi_j π j satisfies them as well. Due to Claim 2.4 , in order to bound the Lipschitz constants of π \pi π we need to bound the quantity ∣ π ( x ) − π ( y ) ∣ ∣ x − y ∣ \frac{|\pi(x)-\pi(y)|}{|x-y|} ∣ x − y ∣ ∣ π ( x ) − π ( y ) ∣ for consecutive indices x , y ∈ [ M ] x,y\in[M] x , y ∈ [ M ] . If consecutive indices x , y ∈ [ M ] x,y\in[M] x , y ∈ [ M ] lie within one part K j K_j K j then the quantity is bounded from above by Lip ( π j ) = O ( D ) \operatorname{Lip}(\pi_j)=O(D) Lip ( π j ) = O ( D ) . Otherwise, x x x and y y y lie in different parts then they both lie at the boundaries of their parts and, as a consequence π ( x ) = x \pi(x)=x π ( x ) = x and π ( y ) = y \pi(y)=y π ( y ) = y , making the quantity equal 1 = O ( D ) 1=O(D) 1 = O ( D ) . Thus Claim 2.4 gives Lip ( π ) = O ( D ) \operatorname{Lip}(\pi)=O(D) Lip ( π ) = O ( D ) , and, as we noted at the beginning of the proof, the same bound holds for π − 1 = π \pi^{-1}=\pi π − 1 = π .
Given π \pi π , Proposition 6.27 gives a bijection F : X → Y F:X\to Y F : X → Y with
c ( X , Y ) ≤ dist ( F ) ≤ n 2 / 3 log O ( 1 ) n . c(X,Y)\le\operatorname{dist}(F)\le n^{2/3}\log^{O(1)}n. c ( X , Y ) ≤ dist ( F ) ≤ n 2/3 log O ( 1 ) n .
References
Woven permutation minors
The goal of this appendix is to show that an oscillating structure is necessary for low-distortion extensions of a given partial mapping with large displacement. For simplicity, we focus on permutations f ∈ S n f \in S_n f ∈ S n with a common Lipschitz bound k k k in both directions and displacement Ω ( k 2 ) \Omega(k^2) Ω ( k 2 ) . In this regime, the distortion Lip ( f ) Lip ( f − 1 ) \operatorname{Lip}(f)\operatorname{Lip}(f^{-1}) Lip ( f ) Lip ( f − 1 ) and the displacement disp ( f ) \operatorname{disp}(f) disp ( f ) are within a constant factor of each other.
Definition A.1 (Minors). Let f ∈ S n f \in S_n f ∈ S n and let S = { x 1 < ⋯ < x N } ⊆ [ n ] S = \{x_1 < \cdots< x_N\} \subseteq[n] S = { x 1 < ⋯ < x N } ⊆ [ n ] be nonempty. Write f ( S ) = { y 1 < ⋯ < y N } f(S) = \{y_1 < \cdots< y_N\} f ( S ) = { y 1 < ⋯ < y N } . The minor of f f f induced by S S S is the permutation θ ∈ S N \theta\in S_N θ ∈ S N determined by
f ( x i ) = y θ ( i ) ( i ∈ [ N ] ) . f(x_i) = y_{\theta(i)} \qquad(i \in[N]). f ( x i ) = y θ ( i ) ( i ∈ [ N ]) . Thus θ ( i ) = j \theta(i) = j θ ( i ) = j iff f f f maps the i i i th smallest element of S S S to the j j j th smallest element of f ( S ) f(S) f ( S ) .
A permutation θ ∈ S N \theta\in S_N θ ∈ S N is a box minor of f f f if it is induced by a nonempty set S = { x ∈ U : f ( x ) ∈ V } S = \{x \in U : f(x) \in V\} S = { x ∈ U : f ( x ) ∈ V } for integer intervals U , V ⊆ [ n ] U,V \subseteq[n] U , V ⊆ [ n ] . Equivalently, we retain exactly the points ( x , f ( x ) ) (x,f(x)) ( x , f ( x )) in the rectangle U × V U \times V U × V and replace each coordinate by its rank among the retained coordinates.
We now define a family of oscillating permutations, called woven permutations. We show that every permutation f ∈ S n f \in S_n f ∈ S n with Lip ( f ) , Lip ( f − 1 ) ≤ k \operatorname{Lip}(f),\operatorname{Lip}(f^{-1}) \leq k Lip ( f ) , Lip ( f − 1 ) ≤ k and displacement Ω ( k 2 ) \Omega(k^2) Ω ( k 2 ) contains a specific kind of box minor, of order Θ ( k 2 ) \Theta(k^2) Θ ( k 2 ) , which we call a woven permutation.
Definition A.2 (Coordinate orders). Let P ⊆ R 2 P \subseteq\mathbb{R}^2 P ⊆ R 2 be a finite nonempty set with distinct x x x -coordinates and distinct y y y -coordinates. Its x x x -order is the order on P P P induced by increasing x x x values. Similarly, define the y y y -order by increasing y y y values. The permutation induced by P P P sends the position of each point in x x x -order to its position in y y y -order.
For example, if P = { ( 1 , 30 ) , ( 4 , 10 ) , ( 7 , 20 ) } P = \{(1,30),(4,10),(7,20)\} P = {( 1 , 30 ) , ( 4 , 10 ) , ( 7 , 20 )} , then the points in x x x -order have y y y -ranks 3 , 1 , 2 3,1,2 3 , 1 , 2 , respectively. Thus P P P induces the permutation θ ∈ S 3 \theta\in S_3 θ ∈ S 3 given by θ ( 1 ) = 3 \theta(1) = 3 θ ( 1 ) = 3 , θ ( 2 ) = 1 \theta(2) = 1 θ ( 2 ) = 1 , and θ ( 3 ) = 2 \theta(3) = 2 θ ( 3 ) = 2 .
Definition A.3 (Woven permutations). Let P ⊆ R 2 P \subseteq\mathbb{R}^2 P ⊆ R 2 be a finite nonempty set with distinct x x x -coordinates and distinct y y y -coordinates, and let m ≥ 1 m \geq1 m ≥ 1 be an integer. Partition a rectangle containing P P P into m m m vertical x x x -strips and m m m horizontal y y y -strips, with no point on a boundary of a strip. The intersection of two strips is a cell of the resulting m × m m \times m m × m grid. We call P P P woven with respect to this grid if it satisfies the following two properties:
Every cell contains at least one point of P P P ;
Consecutive points in either the x x x -order or the y y y -order occupy either: (1) the same cell, (2) cells sharing a side, or (3) cells sharing a corner.
We call the permutation induced by P P P a woven permutation on this grid.
We say that the woven representation of P P P has strip bound b ≥ 1 b \geq1 b ≥ 1 if every x x x -strip and every y y y -strip contains at most b b b points of P P P .
Figure 7. An illustration of a set P P P which induces a woven permutation. The x x x -order is depicted on the left, and the y y y -order appears on the right.
Observation A.4. Every woven permutation θ ∈ S N \theta\in S_N θ ∈ S N on an m × m m \times m m × m grid with strip bound b b b satisfies
m 2 ≤ N ≤ m b , Lip ( θ ) , Lip ( θ − 1 ) ≤ 2 b − 1. m^2 \le N \le mb,\qquad\operatorname{Lip}(\theta),\operatorname{Lip}(\theta^{-1}) \le2b-1. m 2 ≤ N ≤ mb , Lip ( θ ) , Lip ( θ − 1 ) ≤ 2 b − 1. Proof. Since every cell is occupied, N ≥ m 2 N \ge m^2 N ≥ m 2 . Each of the m m m x x x -strips contains at most b b b points, so N ≤ m b N \le mb N ≤ mb .
Let p 1 , … , p N p_1,\ldots,p_N p 1 , … , p N be the points in x x x -order. For 1 ≤ i < N 1 \le i < N 1 ≤ i < N , the points p i p_i p i and p i + 1 p_{i+1} p i + 1 lie either in the same or in neighboring y y y -strips and these strips contain all the points that lie between p i p_i p i and p i + 1 p_{i+1} p i + 1 in y y y -order. As any set of at most two strips contains at most 2 b 2b 2 b points, we conclude ∣ θ ( i + 1 ) − θ ( i ) ∣ ≤ 2 b − 1 |\theta(i+1)-\theta(i)| \le2b-1 ∣ θ ( i + 1 ) − θ ( i ) ∣ ≤ 2 b − 1 and, thus, Lip ( θ ) ≤ 2 b − 1 \operatorname{Lip}(\theta) \le2b-1 Lip ( θ ) ≤ 2 b − 1 . By exchanging coordinates we get Lip ( θ − 1 ) ≤ 2 b − 1 \operatorname{Lip}(\theta^{-1}) \le2b-1 Lip ( θ − 1 ) ≤ 2 b − 1 .
The following theorem shows that large displacement, together with bounds on both directional Lipschitz constants, forces a large woven box minor.
Theorem A.5. For every ε > 0 \varepsilon> 0 ε > 0 , there exists c = c ( ε ) > 0 c=c(\varepsilon)>0 c = c ( ε ) > 0 such that, for all sufficiently large integers k k k , every f ∈ S n f \in S_n f ∈ S n satisfying
max { Lip ( f ) , Lip ( f − 1 ) } ≤ k , disp ( f ) ≥ ε k 2 \max\{\operatorname{Lip}(f),\operatorname{Lip}(f^{-1})\} \le k,\qquad\operatorname{disp}(f) \ge\varepsilon k^2 max { Lip ( f ) , Lip ( f − 1 )} ≤ k , disp ( f ) ≥ ε k 2 has a box minor θ ∈ S N \theta\in S_N θ ∈ S N that is a woven permutation on an m × m m \times m m × m grid with at most k k k points per strip, where
c k ≤ m ≤ k , m 2 ≤ N ≤ m k . ck \le m \le k,\qquad m^2 \le N \le mk. c k ≤ m ≤ k , m 2 ≤ N ≤ mk . In particular, the representing woven point set has Θ ε ( k 2 ) \Theta_\varepsilon(k^2) Θ ε ( k 2 ) points and O ε ( m ) O_\varepsilon(m) O ε ( m ) points per strip.
Proof of Theorem A.5
In this subsection, for integers a ≤ b a \le b a ≤ b let us denote [ a , b ] = { a , a + 1 , … , b } [a,b]=\{a,a+1,\ldots,b\} [ a , b ] = { a , a + 1 , … , b } . We begin by proving the following lemma.
Lemma A.6. Let f ∈ S n f \in S_n f ∈ S n , let 1 ≤ k < n 1 \le k < n 1 ≤ k < n be an integer, and suppose Lip ( f − 1 ) ≤ k \operatorname{Lip}(f^{-1}) \le k Lip ( f − 1 ) ≤ k . Then
disp ( f ) ≤ 2 max 1 ≤ p ≤ n − k diam f ( [ p , p + k ] ) . \operatorname{disp}(f) \le2\max_{1\le p\le n-k}\operatorname{diam} f([p,p+k]). disp ( f ) ≤ 2 1 ≤ p ≤ n − k max diam f ([ p , p + k ]) . Proof. Denote W = max 1 ≤ p ≤ n − k diam f ( [ p , p + k ] ) W=\max_{1\le p\le n-k}\operatorname{diam} f([p,p+k]) W = max 1 ≤ p ≤ n − k diam f ([ p , p + k ]) . We claim that for all a ≤ b ≤ c a \le b \le c a ≤ b ≤ c from [ 1 , n ] [1,n] [ 1 , n ] it holds that
min { f ( a ) , f ( c ) } − W ≤ f ( b ) ≤ max { f ( a ) , f ( c ) } + W . (11) \min\{f(a),f(c)\}-W \le f(b) \le\max\{f(a),f(c)\}+W. \tag*{(11)} min { f ( a ) , f ( c )} − W ≤ f ( b ) ≤ max { f ( a ) , f ( c )} + W . (11)
Wlog assume that f ( a ) ≤ f ( c ) f(a) \le f(c) f ( a ) ≤ f ( c ) and denote A = ( f ( a ) , f ( a ) + 1 , … , f ( c ) ) A = (f(a), f(a) + 1, \ldots, f(c)) A = ( f ( a ) , f ( a ) + 1 , … , f ( c )) . As Lip ( f − 1 ) ≤ k \operatorname{Lip}(f^{-1}) \le k Lip ( f − 1 ) ≤ k , the nearest coordinates in the tuple f − 1 ( A ) f^{-1}(A) f − 1 ( A ) differ by at most k k k . Since the first and the last elements of f − 1 ( A ) f^{-1}(A) f − 1 ( A ) are a a a and c c c respectively and a ≤ b ≤ c a \le b \le c a ≤ b ≤ c , there exists d ∈ f − 1 ( A ) d \in f^{-1}(A) d ∈ f − 1 ( A ) satisfying ∣ d − b ∣ ≤ k |d-b| \le k ∣ d − b ∣ ≤ k . By the definition of W W W we have ∣ f ( d ) − f ( b ) ∣ ≤ W |f(d)-f(b)| \le W ∣ f ( d ) − f ( b ) ∣ ≤ W . Also, by the definition of the set A A A we have f ( a ) ≤ f ( d ) ≤ f ( c ) f(a) \le f(d) \le f(c) f ( a ) ≤ f ( d ) ≤ f ( c ) . Combining the inequalities we conclude (9).
Wlog, we assume f ( 1 ) < f ( n ) f(1) < f(n) f ( 1 ) < f ( n ) , as we can substitute f ( i ) f(i) f ( i ) by f ( n + 1 − i ) f(n+1-i) f ( n + 1 − i ) for all i ∈ [ n ] i \in[n] i ∈ [ n ] . Applying (9) for a = 1 a=1 a = 1 and c = n c=n c = n we conclude that f ( i ) ≥ f ( 1 ) − W f(i) \ge f(1)-W f ( i ) ≥ f ( 1 ) − W for all i ∈ [ n ] i \in[n] i ∈ [ n ] . As f ( i ) = 1 f(i)=1 f ( i ) = 1 for some i ∈ [ n ] i \in[n] i ∈ [ n ] , we have f ( 1 ) ≤ W + 1 f(1) \le W+1 f ( 1 ) ≤ W + 1 . Now, applying (9) for a = 1 a=1 a = 1 and for all $1 \le b \le c \le n$ we get
f ( b ) ≤ max { f ( 1 ) , f ( c ) } + W ≤ f ( c ) + 2 W . (12) f(b) \le\max\{f(1),f(c)\}+W \le f(c)+2W. \tag*{(12)} f ( b ) ≤ max { f ( 1 ) , f ( c )} + W ≤ f ( c ) + 2 W . (12)
Consider arbitrary i ∈ [ n ] i \in[n] i ∈ [ n ] . Notice that f ( [ 1 , i ] ) f([1,i]) f ([ 1 , i ]) contains an element that is at least i i i . Due to (10) applied to c = i c=i c = i , all elements in f ( [ 1 , i ] ) f([1,i]) f ([ 1 , i ]) are at most f ( i ) + 2 W f(i)+2W f ( i ) + 2 W . Hence, f ( i ) ≥ i − 2 W f(i) \ge i-2W f ( i ) ≥ i − 2 W . Similarly, since f ( [ i , n ] ) f([i,n]) f ([ i , n ]) contains n − i + 1 n-i+1 n − i + 1 distinct integers, there exists c ∈ [ i , n ] c \in[i,n] c ∈ [ i , n ] with f ( c ) ≤ i f(c) \le i f ( c ) ≤ i . Applying (10) with b = i b=i b = i gives f ( i ) ≤ f ( c ) + 2 W ≤ i + 2 W f(i) \le f(c)+2W \le i+2W f ( i ) ≤ f ( c ) + 2 W ≤ i + 2 W . Together with the preceding lower bound, this yields ∣ f ( i ) − i ∣ ≤ 2 W |f(i)-i| \le2W ∣ f ( i ) − i ∣ ≤ 2 W for every i ∈ [ n ] i \in[n] i ∈ [ n ] , and hence disp ( f ) ≤ 2 W \operatorname{disp}(f) \le2W disp ( f ) ≤ 2 W , as required. □
Lemma A.7. Let f ∈ S n f \in S_n f ∈ S n , and let 1 ≤ k < n 1 \le k < n 1 ≤ k < n and m ≥ 2 m \ge2 m ≥ 2 be integers. Suppose that Lip ( f ) ≤ k \operatorname{Lip}(f) \le k Lip ( f ) ≤ k and Lip ( f − 1 ) ≤ k \operatorname{Lip}(f^{-1}) \le k Lip ( f − 1 ) ≤ k and disp ( f ) ≥ 128 m k \operatorname{disp}(f) \ge128mk disp ( f ) ≥ 128 mk . Then there exist intervals U , V ⊆ [ n ] U,V \subseteq[n] U , V ⊆ [ n ] , each of size m k mk mk , such that, writing U 1 , … , U m U_1,\ldots,U_m U 1 , … , U m and V 1 , … , V m V_1,\ldots,V_m V 1 , … , V m for their respective partitions into consecutive blocks of k k k integers, we have
f ( U r ) ∩ V s ≠ ∅ ( r , s ∈ [ m ] ) . f(U_r) \cap V_s \ne\varnothing\qquad(r,s \in[m]). f ( U r ) ∩ V s = ∅ ( r , s ∈ [ m ]) .
Proof. Set R = m k R=mk R = mk . By Lemma A.6, choose I = [ a , a + k ] I=[a,a+k] I = [ a , a + k ] with diam f ( I ) ≥ disp ( f ) / 2 ≥ 64 R \operatorname{diam} f(I) \ge\operatorname{disp}(f)/2 \ge64R diam f ( I ) ≥ disp ( f ) /2 ≥ 64 R , and set J = [ min f ( I ) , max f ( I ) ] J=[\min f(I),\max f(I)] J = [ min f ( I ) , max f ( I )] . We first construct the sets U U U and V V V and then verify that every intersection f ( U r ) ∩ V s f(U_r) \cap V_s f ( U r ) ∩ V s is non-empty.
Step 1: finding U U U and V V V . We claim that, after possibly substituting f ( i ) f(i) f ( i ) by f ( n + 1 − i ) f(n+1-i) f ( n + 1 − i ) for all i ∈ [ n ] i \in[n] i ∈ [ n ] , the set J − : = { y ∈ J : f − 1 ( y ) < a − R } J_- := \{y \in J : f^{-1}(y) < a-R\} J − := { y ∈ J : f − 1 ( y ) < a − R } satisfies ∣ J − ∣ ≥ 2 R + 1 |J_-| \ge2R+1 ∣ J − ∣ ≥ 2 R + 1 . Indeed, the number of elements in f − 1 ( J ) ⊆ [ n ] f^{-1}(J) \subseteq[n] f − 1 ( J ) ⊆ [ n ] that lie within distance R R R of I I I is at most ∣ I ∣ + 2 R ≤ k + 1 + 2 R ≤ 3 R |I|+2R \le k+1+2R \le3R ∣ I ∣ + 2 R ≤ k + 1 + 2 R ≤ 3 R . Of the remaining values f − 1 ( J ) f^{-1}(J) f − 1 ( J ) , at least ( ∣ J ∣ − 3 R ) / 2 ≥ 2 R + 1 (|J|-3R)/2 \ge2R+1 ( ∣ J ∣ − 3 R ) /2 ≥ 2 R + 1 lie on the same side of I I I . So, wlog, we can assume that ∣ J − ∣ ≥ 2 R + 1 |J_-| \ge2R+1 ∣ J − ∣ ≥ 2 R + 1 .
Let y 0 : = min J − y_0 := \min J_- y 0 := min J − and y 1 : = max J − y_1 := \max J_- y 1 := max J − , and set
U = [ a − R , a − 1 ] , V = [ y 0 + 1 , y 0 + R ] . U=[a-R,a-1], \qquad V=[y_0+1,y_0+R]. U = [ a − R , a − 1 ] , V = [ y 0 + 1 , y 0 + R ] .
Since J − ≠ ∅ J_- \ne\varnothing J − = ∅ , we have U ⊆ [ n ] U \subseteq[n] U ⊆ [ n ] . Since y 1 − y 0 ≥ ∣ J − ∣ − 1 ≥ 2 R y_1-y_0 \ge|J_-|-1 \ge2R y 1 − y 0 ≥ ∣ J − ∣ − 1 ≥ 2 R , we have V ⊆ J ⊆ [ n ] V \subseteq J \subseteq[n] V ⊆ J ⊆ [ n ] . Both U U U and V V V have order R R R , as desired. Denote their consecutive disjoint k k k -point blocks by U 1 , … , U m U_1,\ldots,U_m U 1 , … , U m and V 1 , … , V m V_1,\ldots,V_m V 1 , … , V m , respectively. Wlog, we can assume that for all $1 \le i < j \le m$ all the elements of U i U_i U i are smaller than all the elements of U j U_j U j and all the elements of V i V_i V i are smaller than all the elements of V j V_j V j .
Step 2: verifying that every cell is occupied. We claim that every U r U_r U r contains points x r , x r + x_r,x_r^+ x r , x r + satisfying
f ( x r − ) ≤ y 0 , f ( x r + ) ≥ y 1 − k . f(x_r^-) \le y_0, \qquad f(x_r^+) \ge y_1-k. f ( x r − ) ≤ y 0 , f ( x r + ) ≥ y 1 − k .
Fix i ∈ { 0 , 1 } i \in\{0,1\} i ∈ { 0 , 1 } . Let us first find z i ∈ f ( I ) ∩ [ y i − k , y i ] z_i \in f(I) \cap[y_i-k,y_i] z i ∈ f ( I ) ∩ [ y i − k , y i ] . Let r 1 , r 2 ∈ I r_1,r_2 \in I r 1 , r 2 ∈ I be the elements satisfying f ( r 1 ) = min f ( I ) f(r_1)=\min f(I) f ( r 1 ) = min f ( I ) and f ( r 2 ) = max f ( I ) f(r_2)=\max f(I) f ( r 2 ) = max f ( I ) . Wlog, assume that r 1 < r 2 r_1<r_2 r 1 < r 2 , in the case r 1 > r 2 r_1>r_2 r 1 > r 2 the proof is analogous. Consider the sequence f ( r 1 ) , f ( r 1 + 1 ) , … , f ( r 2 ) f(r_1),f(r_1+1),\ldots,f(r_2) f ( r 1 ) , f ( r 1 + 1 ) , … , f ( r 2 ) . As Lip ( f ) ≤ k \operatorname{Lip}(f) \le k Lip ( f ) ≤ k , the consecutive elements in this sequence differ by at most k k k . So, as y i ∈ [ f ( r 1 ) , f ( r 2 ) ] y_i \in[f(r_1), f(r_2)] y i ∈ [ f ( r 1 ) , f ( r 2 )] , the sequence contains an element z i ∈ [ y i − k , y i ] z_i \in[y_i-k,y_i] z i ∈ [ y i − k , y i ] and, therefore, f − 1 ( z i ) ∈ [ r 1 , r 2 ] ⊆ I f^{-1}(z_i) \in[r_1,r_2] \subseteq I f − 1 ( z i ) ∈ [ r 1 , r 2 ] ⊆ I , as needed.
Now, consider another sequence f − 1 ( z i ) , f − 1 ( z i + 1 ) , … , f − 1 ( y i ) f^{-1}(z_i), f^{-1}(z_i+1), \ldots, f^{-1}(y_i) f − 1 ( z i ) , f − 1 ( z i + 1 ) , … , f − 1 ( y i ) . Its first term belongs to I = [ a , a + k ] I=[a,a+k] I = [ a , a + k ] and therefore is larger than all elements of U U U . Its last term is smaller than a − R = min U a-R=\min U a − R = min U , since y i ∈ J − y_i\in J_- y i ∈ J − . Consecutive terms in the sequence differ by at most k k k , by Lip ( f − 1 ) ≤ k \operatorname{Lip}(f^{-1})\le k Lip ( f − 1 ) ≤ k . Hence the sequence has an element in every k k k -point block U r U_r U r for all r ∈ [ m ] r\in[m] r ∈ [ m ] as f − 1 ( z i ) ≥ max U r f^{-1}(z_i)\ge\max U_r f − 1 ( z i ) ≥ max U r and f − 1 ( y i ) ≤ min U r f^{-1}(y_i)\le\min U_r f − 1 ( y i ) ≤ min U r . Each such an element x x x satisfies z i ≤ f ( x ) ≤ y i z_i\le f(x)\le y_i z i ≤ f ( x ) ≤ y i . For i = 0 i=0 i = 0 , choose such a point as x r − ; x_r^-; x r − ; then f ( x r − ) ≤ y 0 f(x_r^-)\le y_0 f ( x r − ) ≤ y 0 . For i = 1 i=1 i = 1 , choose such a point as x r + ; x_r^+; x r + ; then f ( x r + ) ≥ z 1 ≥ y 1 − k f(x_r^+)\ge z_1\ge y_1-k f ( x r + ) ≥ z 1 ≥ y 1 − k .
Fix r ∈ [ m ] r\in[m] r ∈ [ m ] . Since y 1 − y 0 ≥ 2 R y_1-y_0\ge2R y 1 − y 0 ≥ 2 R and R ≥ k R\ge k R ≥ k , we have
f ( x r − ) ≤ y 0 < min V , f ( x r + ) ≥ y 1 − k ≥ y 0 + R = max V . f(x_r^-)\le y_0<\min V,\qquad f(x_r^+)\ge y_1-k\ge y_0+R=\max V. f ( x r − ) ≤ y 0 < min V , f ( x r + ) ≥ y 1 − k ≥ y 0 + R = max V .
Consider the sequence f ( x r − ) , f ( x r − + 1 ) , … , f ( x r + ) ⊆ f ( U r ) f(x_r^-),f(x_r^-+1),\ldots,f(x_r^+)\subseteq f(U_r) f ( x r − ) , f ( x r − + 1 ) , … , f ( x r + ) ⊆ f ( U r ) . It is sufficient for us to show that this sequence contains at least one element in every V s V_s V s for all s ∈ [ m ] s\in[m] s ∈ [ m ] . Every two consecutive elements in this sequence differ by at most k k k as Lip ( f ) ≤ k \operatorname{Lip}(f)\le k Lip ( f ) ≤ k . The first element of the sequence is smaller than min V \min V min V and the last element is larger than max V \max V max V . Therefore, as ∣ V s ∣ = k |V_s|=k ∣ V s ∣ = k for all s ∈ [ m ] s\in[m] s ∈ [ m ] , the sequence contains at least one element in every set V s V_s V s for all s ∈ [ m ] , s\in[m], s ∈ [ m ] , as needed. □
Lemma A.8. Let f ∈ S n f\in S_n f ∈ S n and let θ ∈ S N \theta\in S_N θ ∈ S N be its box minor induced by intervals U , V ⊆ [ n ] U,V\subseteq[n] U , V ⊆ [ n ] . Then
Lip ( θ ) ≤ Lip ( f ) , Lip ( θ − 1 ) ≤ Lip ( f − 1 ) . \operatorname{Lip}(\theta)\le\operatorname{Lip}(f),\qquad\operatorname{Lip}(\theta^{-1})\le\operatorname{Lip}(f^{-1}). Lip ( θ ) ≤ Lip ( f ) , Lip ( θ − 1 ) ≤ Lip ( f − 1 ) . Also, for integers k , m ≥ 1 k,m\ge1 k , m ≥ 1 , suppose that max { Lip ( f ) , Lip ( f − 1 ) } ≤ k \max\{\operatorname{Lip}(f),\operatorname{Lip}(f^{-1})\}\le k max { Lip ( f ) , Lip ( f − 1 )} ≤ k , suppose that U U U and V V V are partitioned into consecutive disjoint blocks U 1 , … , U m U_1,\ldots,U_m U 1 , … , U m and V 1 , … , V m V_1,\ldots,V_m V 1 , … , V m of size k k k respectively, and suppose that f ( U r ) ∩ V s ≠ ∅ f(U_r)\cap V_s\ne\varnothing f ( U r ) ∩ V s = ∅ for all r , s ∈ [ m ] r,s\in[m] r , s ∈ [ m ] . Then θ \theta θ is woven on an m × m m\times m m × m grid with strip bound k k k .
Proof. We first prove that Lip ( θ ) ≤ Lip ( f ) \operatorname{Lip}(\theta)\le\operatorname{Lip}(f) Lip ( θ ) ≤ Lip ( f ) and Lip ( θ − 1 ) ≤ Lip ( f − 1 ) \operatorname{Lip}(\theta^{-1})\le\operatorname{Lip}(f^{-1}) Lip ( θ − 1 ) ≤ Lip ( f − 1 ) . Let a < b a<b a < b be consecutive elements of U ∩ f − 1 ( V ) U\cap f^{-1}(V) U ∩ f − 1 ( V ) . As U U U is an interval, we have [ a , b ] ⊆ U [a,b]\subseteq U [ a , b ] ⊆ U . Since V V V is an interval containing f ( a ) f(a) f ( a ) and f ( b ) f(b) f ( b ) , it contains [ min { f ( a ) , f ( b ) } , max { f ( a ) , f ( b ) } ] [\min\{f(a),f(b)\},\max\{f(a),f(b)\}] [ min { f ( a ) , f ( b )} , max { f ( a ) , f ( b )}] . Since a a a and b b b are consecutive in U ∩ f − 1 ( V ) U\cap f^{-1}(V) U ∩ f − 1 ( V ) , we also have f ( [ a , b ] ) ∩ V = { f ( a ) , f ( b ) } f([a,b])\cap V=\{f(a),f(b)\} f ([ a , b ]) ∩ V = { f ( a ) , f ( b )} . In other words, no c c c with a < c < b a<c<b a < c < b has its image f ( c ) f(c) f ( c ) in V V V .
Assume f ( a ) < f ( b ) f(a)<f(b) f ( a ) < f ( b ) . Consider the smallest t ∈ [ a , b − 1 ] t\in[a,b-1] t ∈ [ a , b − 1 ] such that f ( t + 1 ) ≥ f ( b ) f(t+1)\ge f(b) f ( t + 1 ) ≥ f ( b ) . Then, f ( t ) ≤ f ( a ) < f ( b ) ≤ f ( t + 1 ) f(t)\le f(a)<f(b)\le f(t+1) f ( t ) ≤ f ( a ) < f ( b ) ≤ f ( t + 1 ) . Thus,
∣ f ( b ) − f ( a ) ∣ ≤ Lip ( f ) . (13) |f(b)-f(a)|\le\operatorname{Lip}(f). \tag*{(13)} ∣ f ( b ) − f ( a ) ∣ ≤ Lip ( f ) . (13)
This bound is proved the same way in the case f ( b ) < f ( a ) f(b)<f(a) f ( b ) < f ( a ) .
The points a a a and b b b correspond to consecutive indices i i i and i + 1 i+1 i + 1 in the domain of θ \theta θ . Replacing every element of U ∩ f − 1 ( V ) U\cap f^{-1}(V) U ∩ f − 1 ( V ) by the corresponding element in the domain of θ \theta θ does not increase the distance between elements. Hence,
∣ θ ( i + 1 ) − θ ( i ) ∣ ≤ ∣ f ( b ) − f ( a ) ∣ ≤ Lip ( f ) . |\theta(i+1)-\theta(i)|\le|f(b)-f(a)|\le\operatorname{Lip}(f). ∣ θ ( i + 1 ) − θ ( i ) ∣ ≤ ∣ f ( b ) − f ( a ) ∣ ≤ Lip ( f ) .
Since a < b a<b a < b were arbitrary consecutive elements of U ∩ f − 1 ( V ) U\cap f^{-1}(V) U ∩ f − 1 ( V ) , we conclude Lip ( θ ) ≤ Lip ( f ) \operatorname{Lip}(\theta)\le\operatorname{Lip}(f) Lip ( θ ) ≤ Lip ( f ) . Applying the same argument to f − 1 f^{-1} f − 1 and the rectangle V × U V\times U V × U gives Lip ( θ − 1 ) ≤ Lip ( f − 1 ) \operatorname{Lip}(\theta^{-1})\le\operatorname{Lip}(f^{-1}) Lip ( θ − 1 ) ≤ Lip ( f − 1 ) .
Now, assume that max { Lip ( f ) , Lip ( f − 1 ) } ≤ k \max\{\operatorname{Lip}(f),\operatorname{Lip}(f^{-1})\}\le k max { Lip ( f ) , Lip ( f − 1 )} ≤ k and that the consecutive k k k -point blocks of U U U and V V V satisfy f ( U i ) ∩ V j ≠ ∅ f(U_i)\cap V_j\ne\varnothing f ( U i ) ∩ V j = ∅ for all i , j ∈ [ m ] i,j\in[m] i , j ∈ [ m ] . Wlog, we can assume that for all 1 ≤ i < j ≤ m 1\le i<j\le m 1 ≤ i < j ≤ m all the elements of U i U_i U i are smaller than all the elements of U j U_j U j and all the elements of V i V_i V i are smaller than all the elements of V j V_j V j . We claim that, for all r , s , r ′ , s ′ ∈ [ m ] r,s,r',s' \in[m] r , s , r ′ , s ′ ∈ [ m ] , if there exist points ( a , f ( a ) ) ∈ U r × V s (a,f(a)) \in U_r \times V_s ( a , f ( a )) ∈ U r × V s and ( b , f ( b ) ) ∈ U r ′ × V s ′ (b,f(b)) \in U_{r'} \times V_{s'} ( b , f ( b )) ∈ U r ′ × V s ′ that are consecutive in either x x x -order or y y y -order then
∣ r − r ′ ∣ ≤ 1 , ∣ s − s ′ ∣ ≤ 1. |r-r'| \le1,\qquad|s-s'| \le1. ∣ r − r ′ ∣ ≤ 1 , ∣ s − s ′ ∣ ≤ 1.
Indeed, consider such consecutive points ( a , f ( a ) ) (a,f(a)) ( a , f ( a )) and ( b , f ( b ) ) (b,f(b)) ( b , f ( b )) in x x x -order, with a < b a < b a < b . If r ′ ≥ r + 2 r' \ge r+2 r ′ ≥ r + 2 , choose c ∈ U r + 1 ∩ f − 1 ( V ) c \in U_{r+1} \cap f^{-1}(V) c ∈ U r + 1 ∩ f − 1 ( V ) ; then a < c < b a < c < b a < c < b , contradicting consecutiveness. Hence 0 ≤ r ′ − r ≤ 1 0 \le r'-r \le1 0 ≤ r ′ − r ≤ 1 . If ∣ s − s ′ ∣ ≥ 2 |s-s'| \ge2 ∣ s − s ′ ∣ ≥ 2 , the k k k -point blocks V s V_s V s and V s ′ V_{s'} V s ′ are separated by at least k k k integers, giving
k + 1 ≤ ∣ f ( b ) − f ( a ) ∣ ≤ ( 11 ) Lip ( f ) ≤ k , k+1 \le|f(b)-f(a)| \overset{(11)}{\le} \operatorname{Lip}(f) \le k, k + 1 ≤ ∣ f ( b ) − f ( a ) ∣ ≤ ( 11 ) Lip ( f ) ≤ k ,
a contradiction. Thus ∣ s − s ′ ∣ ≤ 1 |s-s'| \le1 ∣ s − s ′ ∣ ≤ 1 as well, so the cells coincide or share a side or corner. For the y y y -order the claim is proved the same way.
Finally, we verify that every cell is occupied and every strip contains at most k k k points. For all r , s ∈ [ m ] r,s \in[m] r , s ∈ [ m ] the set f ( U r ) ∩ V s f(U_r) \cap V_s f ( U r ) ∩ V s is non-empty and thus θ \theta θ is a woven permutation. Each U r U_r U r and each V s V_s V s have size k k k . Hence every strip contains at most k k k points, so the woven representation of θ \theta θ has strip bound k k k .
Proof of Theorem A.5. Set m = min { k , ⌊ ε k / 128 ⌋ } m=\min\{k,\lfloor\varepsilon k/128\rfloor\} m = min { k , ⌊ ε k /128 ⌋} and c = min { 1 , ε / 256 } c=\min\{1,\varepsilon/256\} c = min { 1 , ε /256 } . For sufficiently large k k k , we have k < n k<n k < n , 2 ≤ m ≤ k 2\le m\le k 2 ≤ m ≤ k , and m ≥ c k m\ge ck m ≥ c k . Since disp ( f ) ≥ ε k 2 ≥ 128 m k \operatorname{disp}(f)\ge\varepsilon k^2\ge128mk disp ( f ) ≥ ε k 2 ≥ 128 mk , Lemmas A.7 and A.8 give the required woven box minor. Finally, Observation A.4 gives m 2 ≤ N ≤ m k m^2\le N\le mk m 2 ≤ N ≤ mk .
Infinite expectation
Claim B.1 (Infinite expectation). For every n ≥ 3 n \ge3 n ≥ 3 and iid X , Y ∼ O X,Y \sim\mathcal{O} X , Y ∼ O , we have
E [ c ( X , Y ) ] = ∞ . \mathbb{E}[c(X,Y)] = \infty. E [ c ( X , Y )] = ∞. Proof. By Proposition 2.1, we may write Δ i Z = E i Z / Σ Z \Delta_i^Z=E_i^Z/\Sigma^Z Δ i Z = E i Z / Σ Z for Z ∈ { X , Y } Z\in\{X,Y\} Z ∈ { X , Y } and every i ∈ [ n − 1 ] i\in[n-1] i ∈ [ n − 1 ] , where E 0 X , … , E n X , E 0 Y , … , E n Y E_0^X,\ldots,E_n^X,E_0^Y,\ldots,E_n^Y E 0 X , … , E n X , E 0 Y , … , E n Y are iid exponential with mean 1 and Σ Z = ∑ j = 0 n E j Z \Sigma^Z=\sum_{j=0}^n E_j^Z Σ Z = ∑ j = 0 n E j Z . In particular,
z n − z 1 = ∑ i = 1 n − 1 E i Z / Σ Z . z_n-z_1=\sum_{i=1}^{n-1} E_i^Z/\Sigma^Z. z n − z 1 = i = 1 ∑ n − 1 E i Z / Σ Z . Let f : X → Y f:X\to Y f : X → Y be a bijection. The images of a closest pair in X X X are distinct points of Y Y Y , so Lip ( f ) ≥ Δ min Y / Δ min X \operatorname{Lip}(f)\ge\Delta_{\min}^Y/\Delta_{\min}^X Lip ( f ) ≥ Δ m i n Y / Δ m i n X . Since ∣ f ( x n ) − f ( x 1 ) ∣ ≤ y n − y 1 |f(x_n)-f(x_1)|\le y_n-y_1 ∣ f ( x n ) − f ( x 1 ) ∣ ≤ y n − y 1 , we also have Lip ( f − 1 ) ≥ ( x n − x 1 ) / ( y n − y 1 ) \operatorname{Lip}(f^{-1})\ge(x_n-x_1)/(y_n-y_1) Lip ( f − 1 ) ≥ ( x n − x 1 ) / ( y n − y 1 ) . Since f f f is arbitrary, and the normalization factors Σ X , Σ Y \Sigma^X,\Sigma^Y Σ X , Σ Y cancel in the expression below, we get,
c ( X , Y ) ≥ Δ min Y y n − y 1 ⋅ x n − x 1 Δ min X = min i ∈ [ n − 1 ] E i Y ∑ i = 1 n − 1 E i Y ⋅ ∑ i = 1 n − 1 E i X min i ∈ [ n − 1 ] E i X . c(X,Y) \ge\frac{\Delta_{\min}^Y}{y_n-y_1}\cdot\frac{x_n-x_1}{\Delta_{\min}^X}
=\frac{\min_{i\in[n-1]}E_i^Y}{\sum_{i=1}^{n-1}E_i^Y}\cdot\frac{\sum_{i=1}^{n-1}E_i^X}{\min_{i\in[n-1]}E_i^X}. c ( X , Y ) ≥ y n − y 1 Δ m i n Y ⋅ Δ m i n X x n − x 1 = ∑ i = 1 n − 1 E i Y min i ∈ [ n − 1 ] E i Y ⋅ min i ∈ [ n − 1 ] E i X ∑ i = 1 n − 1 E i X . For s ≥ 1 s\ge1 s ≥ 1 , consider the two events
A s = { E 1 X ≤ 1 / s , E 2 X ≥ 1 } , B = { 1 ≤ E i Y ≤ 2 for all i ∈ [ n − 1 ] } , A_s=\{E_1^X\le1/s,\ E_2^X\ge1\},\qquad B=\{1\le E_i^Y\le2\text{ for all }i\in[n-1]\}, A s = { E 1 X ≤ 1/ s , E 2 X ≥ 1 } , B = { 1 ≤ E i Y ≤ 2 for all i ∈ [ n − 1 ]} , i.e., the first internal exponential variable of X X X is at most 1 / s 1/s 1/ s and the second is at least 1, while all internal exponential variables of Y Y Y lie in [ 1 , 2 ] [1,2] [ 1 , 2 ] . Whenever both events hold, we have min i E i X ≤ 1 / s \min_i E_i^X\le1/s min i E i X ≤ 1/ s , ∑ i E i X ≥ 1 \sum_i E_i^X\ge1 ∑ i E i X ≥ 1 , min i E i Y ≥ 1 \min_i E_i^Y\ge1 min i E i Y ≥ 1 and ∑ i E i Y ≤ 2 n \sum_i E_i^Y\le2n ∑ i E i Y ≤ 2 n , and hence c ( X , Y ) ≥ s / ( 2 n ) c(X,Y)\ge s/(2n) c ( X , Y ) ≥ s / ( 2 n ) .
By independence,
P ( A s ∩ B ) = ( 1 − e − 1 / s ) e − 1 ( e − 1 − e − 2 ) n − 1 ≥ a n s \mathbb{P}(A_s\cap B)=(1-e^{-1/s})e^{-1}(e^{-1}-e^{-2})^{n-1}\ge\frac{a_n}{s} P ( A s ∩ B ) = ( 1 − e − 1/ s ) e − 1 ( e − 1 − e − 2 ) n − 1 ≥ s a n for some constant a n > 0 a_n>0 a n > 0 . Thus P ( c ( X , Y ) ≥ u ) ≥ a n / ( 2 n u ) \mathbb{P}(c(X,Y)\ge u)\ge a_n/(2nu) P ( c ( X , Y ) ≥ u ) ≥ a n / ( 2 n u ) for all u ≥ 1 u\ge1 u ≥ 1 , and
E [ c ( X , Y ) ] ≥ ∫ 1 ∞ P ( c ( X , Y ) ≥ u ) d u = ∞ . \mathbb{E}[c(X,Y)]\ge\int_1^\infty\mathbb{P}(c(X,Y)\ge u)\,du=\infty. E [ c ( X , Y )] ≥ ∫ 1 ∞ P ( c ( X , Y ) ≥ u ) d u = ∞.
Oscillating permutations
We now describe the oscillation construction appearing in the technical overview in more detail, and give some more intuition on how to define it for general k k k . Let S i → j S_{i\to j} S i → j be the set of permutations sending i i i to j j j .
Proposition C.1. Let k > 0 k>0 k > 0 be odd and let k ≤ i < j ≤ n − k k\leq i<j\leq n-k k ≤ i < j ≤ n − k satisfy j − i = ( k − 1 ) 2 j-i=(k-1)^2 j − i = ( k − 1 ) 2 . Then, there exists π ∈ S i → j \pi\in S_{i\to j} π ∈ S i → j satisfying
dist ( π ) = ( 2 k − 1 ) 2 , disp ( π ) = ( k − 1 ) 2 . \operatorname{dist}(\pi)=(2k-1)^2,\qquad\operatorname{disp}(\pi)=(k-1)^2. dist ( π ) = ( 2 k − 1 ) 2 , disp ( π ) = ( k − 1 ) 2 .
Sketch of the proof. Let us construct π ∈ S i → j \pi\in S_{i\to j} π ∈ S i → j . For all r ∈ [ n ] r\in[n] r ∈ [ n ] outside of [ i − k + 1 , j + k − 1 ] [i-k+1,j+k-1] [ i − k + 1 , j + k − 1 ] we let π \pi π to be the identity, i.e. π ( r ) = r \pi(r)=r π ( r ) = r . Now we need to define the remaining permutation π ∣ [ i − k + 1 , j + k − 1 ] \left.\pi\right|_{[i-k+1,j+k-1]} π ∣ [ i − k + 1 , j + k − 1 ] . In order to do that, consider the following specific permutation σ \sigma σ of the set [ 0 , k 2 − 1 ] [0,k^2-1] [ 0 , k 2 − 1 ] that satisfies
dist ( σ ) = ( 2 k − 1 ) 2 \operatorname{dist}(\sigma)=(2k-1)^2 dist ( σ ) = ( 2 k − 1 ) 2 ;
σ ( k − 1 ) = k 2 − k \sigma(k-1)=k^2-k σ ( k − 1 ) = k 2 − k ;
σ ( 0 ) = 0 \sigma(0)=0 σ ( 0 ) = 0 and σ ( k 2 − 1 ) = k 2 − 1 \sigma(k^2-1)=k^2-1 σ ( k 2 − 1 ) = k 2 − 1 ;
disp ( σ ) = ( k − 1 ) 2 \operatorname{disp}(\sigma)=(k-1)^2 disp ( σ ) = ( k − 1 ) 2 .
If σ \sigma σ exists then it is easy to verify that the following definition of π \pi π suffices: for every r ∈ [ 0 , k 2 − 1 ] r\in[0,k^2-1] r ∈ [ 0 , k 2 − 1 ] , set
π ( ( i − k + 1 ) + r ) = ( i − k + 1 ) + σ ( r ) . \pi\left((i-k+1)+r\right)=(i-k+1)+\sigma(r). π ( ( i − k + 1 ) + r ) = ( i − k + 1 ) + σ ( r ) .
It remains to verify the existence of such σ \sigma σ and this will yield the idea of oscillations.
Claim C.2. There exists a permutation σ \sigma σ of the set [ 0 , k 2 − 1 ] [0,k^2-1] [ 0 , k 2 − 1 ] satisfying P1, P2, P3, and P4.
Sketch of the proof. Let M = [ 0 , k − 1 ] 2 M=[0,k-1]^2 M = [ 0 , k − 1 ] 2 denote the set of pairs of size k 2 k^2 k 2 . Let the “snake” bijection s : M → [ 0 , k 2 − 1 ] s:M\to[0,k^2-1] s : M → [ 0 , k 2 − 1 ] be defined as follows: for a , b ∈ [ 0 , k − 1 ] a,b\in[0,k-1] a , b ∈ [ 0 , k − 1 ] set s ( a , b ) = a k + b s(a,b)=ak+b s ( a , b ) = ak + b for a a a being even and set s ( a , b ) = ( a + 1 ) k − b − 1 s(a,b)=(a+1)k-b-1 s ( a , b ) = ( a + 1 ) k − b − 1 for a a a being odd. It is easy to verify that s s s is indeed a bijection. Let inv : M → M \operatorname{inv}:M\to M inv : M → M be the symmetry bijection, i.e. inv ( a , b ) = ( b , a ) \operatorname{inv}(a,b)=(b,a) inv ( a , b ) = ( b , a ) . The left-hand side of Figure 8 represents the horizontal “snake” bijection, i.e. the set M M M is represented as a square and every cell ( a , b ) (a,b) ( a , b ) of the square contains the value of s ( a , b ) s(a,b) s ( a , b ) inside. At the same time the right-hand side of Figure 8 represents the vertical snake, i.e. that is the depiction of map s ∘ inv s\circ\operatorname{inv} s ∘ inv . Now, every cell of the square has two values written inside, given by s s s and s ∘ inv s\circ\operatorname{inv} s ∘ inv respectively. Consider the map σ \sigma σ so that for every ( a , b ) ∈ M (a,b)\in M ( a , b ) ∈ M it maps s ( a , b ) s(a,b) s ( a , b ) to ( s ∘ inv ) ( a , b ) (s\circ\operatorname{inv})(a,b) ( s ∘ inv ) ( a , b ) (see, Figure 9 ), i.e. for all r ∈ [ 0 , k 2 − 1 ] r\in[0,k^2-1] r ∈ [ 0 , k 2 − 1 ]
σ ( r ) = ( s ∘ inv ∘ s − 1 ) ( r ) . \sigma(r)=(s\circ\operatorname{inv}\circ s^{-1})(r). σ ( r ) = ( s ∘ inv ∘ s − 1 ) ( r ) . It is easy to verify that σ \sigma σ possess properties P1-P4 by picture, see Figure 8 . We say that σ \sigma σ oscillates (see, Figure 9 ) because if we follow the motion of σ ( r ) \sigma(r) σ ( r ) as r r r increases then it makes oscillating movements — it first monotonically increases then decreases then increases again and so on. In addition, all phases of subsequent increments/decrements take about the same amount of time of length Θ ( k ) \Theta(k) Θ ( k ) and, also, every step ∣ σ ( r + 1 ) − σ ( r ) ∣ |\sigma(r+1)-\sigma(r)| ∣ σ ( r + 1 ) − σ ( r ) ∣ is at most Θ ( k ) \Theta(k) Θ ( k ) .
Figure 8. The row-wise and column-wise snake orderings for k = 5 k = 5 k = 5 .
Figure 9. The permutation induced by matching the LR and UD snakes according to their common grid positions. Its distortion is at most ( 2 k − 1 ) 2 (2k - 1)^2 ( 2 k − 1 ) 2 . Colors interpolate from red to blue.