The Negative Spherical Perceptron: Capacity, Isostatic Jamming, and Full Replica Symmetry Breaking

Capacity, Isostatic Jamming, and Full Replica Symmetry Breaking

Authors’ note

This manuscript is the result of a methodological experiment in using large language models to develop the rigorous mathematical theory of the negative perceptron, which has received significant attention in the physics literature as an accessible model for studying jamming and related phenomena.

With the exception of this note, the entire document is machine written. We have taken care to supervise this writing to ensure that the proofs meet a minimum standard of readability and that previous literature is appropriately cited. However, the resulting document still leaves much to be desired in terms of exposition and overall coherence. We are nonetheless releasing this manuscript on Hexagon to disseminate the results as quickly as possible, because we believe they may be of interest to the mathematics and physics communities.

We plan to take some time to write the results in this manuscript carefully for publication. While we focus on writing, we do not intend to produce further results for the negative perceptron. However, we invite readers to do so, because we believe there is much more to be done. Notably, it remains to establish the scaling solution of Franz, Parisi, Sevelev, Urbani, and Zamponi [ref-59] for the Parisi minimizers as $\kappa\uparrow\kappa_{\mathrm{c}}$ and the predictions that follow from it. We welcome emails from anyone who obtains such results and will update the references accordingly. We also welcome suggestions concerning mathematical corrections, omitted citations, and attribution of ideas.

P.L. was partially supported by NSF grant DMS-2450004.

Notation

Shared notation. Both Parts use the following. $G$ is a standard Gaussian variable, $\phi$ and $\Phi$ are its density and distribution function, $R=\phi/\Phi$, $\bar{R}(z)=z+R(z)$, $V=R\bar{R}$, $m(k)=\mathbb{E}(k-G)_{+}$, and $m_{1}(k)=\mathbb{E}(k-G)_{+}$. An order parameter is a right-continuous nondecreasing $\gamma:[0,1]\to[0,1]$ with $\gamma=1$ near one; $\mu_{\gamma}$ is the measure with distribution function $\gamma$, $Q_{\gamma}$ (often written $Q$) is the top of its support, $\delta=1-Q_{\gamma}$, and $\lambda_{\gamma}(q)=\int_{q}^{1}\gamma$; $\mathbf{1}_{[q,1]}$ is a replica-symmetric order parameter. $\mathcal{U}$ and $\mathcal{U}_{r}$ are the classes of order parameters, $\mathcal{P}_{\kappa}$ is the Parisi functional, $\mathcal{P}_{*}(\kappa)$ its infimum (the Parisi value in Part I), $\mathcal{P}_{\kappa}^{RS}(q)=\mathcal{P}_{\kappa}(\mathbf{1}_{[q,1]})$ Gardner’s replica-symmetric functional, $u$ the solution of the Parisi equation, $X$ the Parisi diffusion, $D$, $S$, and $\mathcal{H}_{\gamma}$ the functions of the first-order conditions, $T_{m,s}$ one Gaussian step of the Parisi recursion, $m_{j}$ the values of a step order parameter, $K=(1-r)^{-1}$ the constant of the class $\mathcal{U}_{r}$, and $b=\partial_{x}u$. The margins are the critical margin $\kappa_{c}$, Gardner’s prediction $\kappa_{\mathrm{RS}}$, and the de Almeida–Thouless margin $\kappa_{\mathrm{d}}$; near $\alpha=2$ we write $\alpha=2+e$. The free energy is $N^{-1}\log Z_{N}(U)$ for a soft activation $U$ (Part I) and $N^{-1}\log V_{N}(\kappa)$ at a hard margin $\kappa$. In Part II the rescaled profile of $\gamma$ is $\zeta(s)=\gamma(1-e^{2}s)$.

The table lists the letters whose meaning differs between the two Parts. Many letters, for example $\tau$, $\vartheta$, $\rho$, $Y$, $\varepsilon$, $\eta$, and $c_{0}, c_{1}, \ldots$, also carry local meanings that are fixed where they appear.

Symbol	Part I	Part II
$\ell$	level of the soft penalty $-\beta(\ell-x)^{2}$; width of the top layer near $\alpha=2$	$\ell(s)$, $\ell_{0}$: rescaled masses $\lambda_{\gamma}/e^{2}$
$\beta$	leaves of the cascade tree; strength of the soft penalty and of the source term	$\beta=\sqrt{(1-q)/q}$ for replica-symmetric order parameters; also $\delta/\lambda$
$J$	$J_{\ell}(k)=J^{k}(\ell^{-1}\mathbf{1}_{[1-\ell,\ell_{1})})$; the level at which two replicas branch	$J^{k}$: the zero-temperature functional; $\mathcal{J}$: the jamming coefficient
$C_{0}$	$-\log\Phi(-1)$; the growth constant $2L$ of an activation	a growth constant for terminal data; the constant in the a priori bound $s_{0}\le C_{0}e^{-2/3}$
$U$	the activation	$U=U_{\zeta}$, the solution in rescaled variables
$A$	a constant bounding an activation; the pattern matrix	$A(\zeta)$, the entropy term of the flat limit; a function $A(k)$ of the margin
$K,\mathcal{K}$	$\lVert U''\rVert_{\infty}$; the number of atoms of a cascade; an operator-norm constant, $C_{\mathrm{op}}$; $\mathcal{K}$ a cone or convex set; $K_{N}$ a polyhedron	$K_{e}$, a constant; $K=\gamma/\lambda$; $\mathcal{K}(d_{1},S)$, the compact class of profiles
$S,s$	$S(q)=\alpha\lambda\gamma(q)^{2}\mathbb{E}\,\partial_{xx}u_{\gamma}(q,X_{q})^{2}$; $s$ a sparsity level	$S_{\gamma}$, the same; the depth parameter $S$ in $\mathcal{K}(d_{1},S)$ and the bound $S_{*}$; $s=(1-q)/e^{2}$, the rescaled overlap
$m$	$m_{N}(\sigma)$, the minimal margin of $\sigma$	$m_{*}$, a bound on $\gamma$ below the top of the support
$\zeta$	$\zeta_{l}$, conditional means of the response overlap; the limiting overlap law	the rescaled profile
$b$	also the rows $b_{a}=g_{a}/\sqrt{N}$ and the top precision $b$ of a cascade	only $b=\partial_{x}u$

I Parisi formula, capacity, and isostaticity for the negative spherical perceptron

We study the spherical perceptron with Gaussian patterns at negative margins, a mean-field model of jamming in which the set of solutions is not convex. We prove the Parisi formula for its free energy for bounded-above $C^{2}$ activations with bounded second derivative and either bounded first derivative or concavity, without any monotonicity assumption on the activation. For every constraint density $\alpha>2$, we show that the normalized logarithm of the volume of solutions converges in probability to the Parisi value at every margin below a critical margin $\kappa_{\mathrm{c}}(\alpha)<0$, that this value tends to $-\infty$ continuously as the margin increases to $\kappa_{\mathrm{c}}(\alpha)$, and that the largest achievable margin converges in probability to $\kappa_{\mathrm{c}}(\alpha)$. At sufficiently negative margins, Gardner’s replica-symmetric formula for the free energy is exact. For $\alpha$ slightly above two, the limit $\kappa_{\mathrm{c}}(\alpha)$ lies below Gardner’s prediction by at least a constant times $(\alpha-2)^{3}$; with Part II, this order is exact. At the optimal margin, with probability exponentially close to one, every optimal configuration has exactly $N$ contacts: the jamming point is isostatic. The empirical gap measures of optimal configurations, normalized by $N$, converge weakly in probability to a deterministic measure with an atom of mass exactly one at zero. At a fixed margin below $\kappa_{\mathrm{c}}(\alpha)$, the gap statistics of all but an exponentially small fraction of the feasible set are given by the Parisi diffusion.

Introduction

The perceptron is the simplest model of a neural network and a basic example of a random constraint satisfaction problem. Given $M$ random patterns $g_{1},\ldots,g_{M}\in\mathbb{R}^{N}$ and a margin $\kappa\in\mathbb{R}$, the spherical perceptron asks for a vector $\sigma$ on the sphere of radius $\sqrt{N}$ such that

Two questions are classical: how large the set of such vectors is, and what the largest achievable margin is when $M/N\to\alpha$. Gardner and Derrida computed the answers with the replica method, under the assumption of replica symmetry [ref-62, ref-63]. In particular, Gardner’s formula predicts that the largest achievable margin converges to the solution $\kappa_{\mathrm{RS}}(\alpha)$ of

where $G$ is a standard Gaussian variable.

For $\kappa\ge0$ each constraint cuts out a geodesically convex spherical cap, and replica symmetry is expected to hold throughout the satisfiable phase. In this regime Gardner’s formula for the free energy was proved by [ref-115] and by [ref-128], Chapter 8, and Stojnic gave another proof of the formula for the capacity [ref-118]. For $\kappa<0$, which is the relevant regime when $\alpha>2$, each constraint removes a cap smaller than a hemisphere, and the feasible set is a nonconvex intersection of the complements of such caps. Franz and Parisi proposed this negative spherical perceptron as the simplest model of jamming [ref-58]. Franz, Parisi, Sevelev, Urbani, and Zamponi argued that its satisfiability threshold belongs to the same universality class as the jamming transition of hard spheres in infinite dimension [ref-59]. The same model describes linear classification with a negative margin, a basic example of an overparametrized learning problem [ref-60, ref-90].

In the physical picture, replica symmetry is lost at negative margins before the satisfiability threshold is reached. The stability of the replica-symmetric solution of the perceptron was studied in [ref-23, ref-63]. Close to $\kappa=0$ replica symmetry is lost at a de Almeida–Thouless line [ref-45], through a continuous transition to full replica symmetry breaking in the sense of Parisi [ref-104, ref-105], and the whole jamming line lies in the broken phase [ref-58, ref-59]. The physics literature therefore predicts that Gardner’s formula for the critical margin is exact only for $\kappa\ge0$. At the critical margin, jammed configurations are predicted to be isostatic: they are in contact with as many constraints as there are degrees of freedom. Franz and Parisi predicted that jamming is hypostatic in the convex regime $\kappa> 0$ and isostatic in the nonconvex regime $\kappa< 0$ [ref-58]; for the isostatic counting in jammed packings see [ref-2, ref-91]. The gaps and contact forces of jammed configurations are predicted to have power-law distributions with universal exponents [ref-29, ref-30, ref-58, ref-59, ref-107], related to marginal stability [ref-106, ref-132] and to the jamming of soft spheres [ref-95]. Annesi, Malatesta, and Zamponi computed the satisfiability threshold numerically from the equations of full replica symmetry breaking, and located the region of the phase diagram in which the overlap distribution of typical solutions has connected support [ref-3].

Rigorous results for negative margins have so far been partial. In [ref-118], Stojnic used Gordon’s comparison inequality [ref-66] to show that Gardner’s formula gives an upper bound on the capacity at every margin. He also proved a sharper bound by applying Gordon’s inequality to an exponential of the cost; its improvement over the replica-symmetric bound was shown numerically [ref-117]. El Alaoui and Sellke conjectured that the Parisi variational problem for the hard-margin perceptron, in the form used below, gives the free energy and the capacity [ref-54], Conjecture 1; they trace this conjecture to [ref-59]. Assuming that some Parisi minimizer $\gamma$ has no overlap gap, and given oracle access to it, their algorithm constructs vectors of norm close to $\sqrt{Q_{\gamma}N}$ with a small vector of constraint violations, where $Q_{\gamma} < 1$ is the top of the support of $\gamma$ [ref-54], Theorem 1; see also [ref-55]. Montanari, Zhong, and Zhou proved upper and lower bounds on the capacity that agree to leading order as $\kappa\to-\infty$, and studied its algorithmic tractability [ref-90]. Their upper bound [ref-90], Theorem 4.2 shows that Gardner’s prediction overestimates the capacity when $\alpha$ is large. Near $\alpha= 2$, where $\kappa_{\mathrm{RS}}(\alpha)$ is close to zero, the cited bounds do not quantify this separation. Stojnic later proposed a characterization of the capacity through fully lifted random duality theory, and computed the resulting values numerically at finite levels of a stationarized lifting [ref-119]. Huang, Sellke, and Sun characterized the empirical distributions of the margins that algorithms with dimension-free Lipschitz dependence on the Gaussian data can achieve [ref-73].

For the Ising perceptron, where $\sigma\in\{-1,1\}^{N}$, the capacity predicted by Krauth and Mézard [ref-83] was studied by Ding and Sun and by Huang, who proved lower and upper bounds matching the prediction, subject in part to numerical conditions [ref-48, ref-72]. Xu proved a sharp threshold for the Ising perceptron with Bernoulli disorder [ref-133]. At small constraint densities, Talagrand proved that the free energy of the Ising perceptron is given by the replica-symmetric formula, for a class of activations that includes the one-sided threshold [ref-123], [ref-127], Chapter 2, [ref-128], Chapter 9; his treatment of the Shcherbina–Tirozzi model, whose spins are continuous, is in [ref-127], Chapter 3. Bolthausen, Nakajima, Sun, and Xu gave another proof of this replica-symmetric formula, for a larger class of activations [ref-20]. In the Bayes-optimal (teacher–student) setting, replica-symmetric formulas for perceptron-type generalized linear models were established by Barbier, Krzakala, Macris, Miolane, and Zdeborová [ref-13].

The Parisi formula for mean-field spin glasses was predicted in [ref-104, ref-105] (see also [ref-31, ref-88]) and proved by Guerra and Talagrand for the Sherrington–Kirkpatrick model [ref-68, ref-124]. Talagrand and W.-K. Chen proved the Crisanti–Sommers formula [ref-44] for spherical models, the latter through the Aizenman–Sims–Starr scheme [ref-1, ref-37, ref-125], and Panchenko proved the formula for general mixed $p$-spin models, building on the ultrametricity theorem [ref-97, ref-98, ref-99]. The perceptron differs from these models in a basic way. Its Hamiltonian $\sum_{a} U(g_{a}\cdot\sigma/\sqrt{N})$ is a nonlinear function of Gaussian fields, and it is not itself a Gaussian process in $\sigma$. Structurally the model is closer to multi-species or bipartite spin glasses, with one species formed by the coordinates and the other by the constraints. For such models Guerra’s interpolation gives an upper bound when the interaction is convex [ref-14, ref-16, ref-100]. The matching lower bound by the Aizenman–Sims–Starr scheme requires synchronization of the overlaps of the species [ref-100, ref-101], and it needs no convexity, for Ising spins [ref-100] and for spherical spins [ref-16]; see also [ref-15, ref-17]. The interaction of the bipartite model, which the perceptron resembles, is not convex. For spherical bipartite models see [ref-6, ref-82, ref-122]. Mourrat proved that the solution of an infinite-dimensional Hamilton–Jacobi equation bounds the free energy of bipartite models from above, and conjectured that it is the limit [ref-92]; see also [ref-49]. Our upper bound follows the proof of the upper bound in [ref-92]. That work also contains a quantitative synchronization estimate for pairs of overlap arrays, which is an input to both bounds below. Mourrat later proved an upper bound of the same kind for vector spin glasses whose energy is a Gaussian field [ref-94]; the energy $\sum_{a} U(g_{a}\cdot\sigma/\sqrt{N})$ is not of this form. For nonconvex vector models, H.-B. Chen and Mourrat showed that the limit of the free energy, if it exists, is a critical value of a Parisi-type functional [ref-34], and H.-B. Chen proved the same for nonconvex multi-species models [ref-33]. H.-B. Chen, Issa, and Mourrat identified the free energy of nonconvex multi-species models with centered Ising spins [ref-36]. H.-B. Chen and Mourrat then identified the limit of the free energy of all multi-species spherical spin glasses, with no convexity assumption [ref-35]. Ho proved that the enriched free energy of vector spin glasses converges without convexity, so the critical-point representation of H.-B. Chen and Mourrat holds without assuming convergence [ref-69].

Our contribution is a Parisi formula for the free energy, an identification of the limit of the largest achievable margin, and a description of the gap statistics and contacts of optimal configurations for the negative spherical perceptron. We prove the Parisi formula for smooth activations, including activations that are neither monotone nor concave (Theorem 1.1). For $\alpha>2$ we identify the exponential volume of the feasible set at every margin below the critical margin $\kappa_{\mathrm{c}}(\alpha)$ of the Parisi functional, and we prove that the largest achievable margin converges to $\kappa_{\mathrm{c}}(\alpha)$ (Theorem 1.2). For $\alpha$ slightly above two, we show that $\kappa_{\mathrm{c}}(\alpha)$ lies below Gardner’s prediction by at least a constant times $(\alpha-2)^{3}$ (Theorem 1.4). Combined with the matching upper bound on $\kappa_{\mathrm{RS}}-\kappa_{\mathrm{c}}$ (Part II, Theorem 1.1), this order is exact (Corollary 1.6).

Our main result on the geometry of solutions concerns the configurations of maximal margin. With probability exponentially close to one, every optimal configuration has exactly $N$ contacts. All configurations whose margin is close to optimal share a deterministic limiting distribution of gaps, and this distribution has an atom of mass exactly one at zero (Theorem 1.8). Thus, in the limit, no macroscopic set of constraints beyond the $N$ contacts accumulates near contact. This is a rigorous form of the predicted isostaticity of the jamming point recalled above. The link between the random model and the limiting gap law is a collapse of Parisi minimizers. As the margin increases to $\kappa_{c}(\alpha)$, the minimal value of the Parisi functional tends to $-\infty$ continuously, and the mass of every minimizer (defined in §1.1) tends to zero at a power rate (Theorem 1.3). This is a rigorous counterpart of the physical picture of jamming, in which the overlap of solutions tends to one and the weight of the overlaps below any fixed level tends to zero [ref-58].

Model and the Parisi functional

Fix $\alpha>0$ and integers $M=M_{N}$ with $M_{N}/N\to\alpha$. Let $\mathbf{G}=(g_{ai})_{a\leq M,i\leq N}$ have independent standard Gaussian entries, write $g_{a}$ for its $a$-th row, and put $b_{a}=g_{a}/\sqrt{N}$. Let $\mu_{N}$ be the uniform probability measure on

For $\sigma\in S_{N}$, the number $b_{a}\cdot\sigma$ is the margin of the constraint $a$ at $\sigma$; for each fixed $\sigma\in S_{N}$ it is a standard Gaussian variable. For an activation $U:\mathbb{R}\to[-\infty,\infty)$ that is bounded above, define

For a margin $\kappa\in\mathbb{R}$, the feasible set, its volume, and the largest achievable margin are

We use the convention $\log0=-\infty$. Thus $V_{N}(\kappa)=Z_{N}(H_{\kappa})$ for the hard constraint $H_{\kappa}(x)=\log\mathbf{1}_{\{x\geq\kappa\}}$. For $\sigma\in S_{N}$ let $m_{N}(\sigma)=\min_{a\leq M}b_{a}\cdot\sigma$, so that $\kappa_{N}=\max_{S_{N}}m_{N}$. The gaps of $\sigma$ are the numbers $b_{a}\cdot\sigma-m_{N}(\sigma)\geq0$, and the empirical gap measure of $\sigma$ is

a measure on $[0,\infty)$ of total mass $M/N$. A constraint with gap zero is called a contact of $\sigma$.

An order parameter is a nondecreasing, right-continuous function $\gamma:[0,1]\to[0,1]$ that is identically one on $[\bar q,1]$ for some $\bar q<1$. We write $\mathcal U$ for the set of order parameters and

Thus $\gamma$ is the distribution function of a probability measure $\mu_{\gamma}$ on $[0,Q_{\gamma}]$, and $Q_{\gamma}$ is the top of its support. We call $\lambda_{\gamma}(0)$ the mass of $\gamma$. Given $\gamma\in\mathcal U$ and an activation $U$, let $u_{\gamma}=u_{\gamma}(q,x;U)$ solve the Parisi equation

When $\gamma$ is a step function, (3) is solved by an explicit backward recursion of Gaussian integrals, the Cole–Hopf transformation [ref-42, ref-71] on each step; for general $\gamma$, and for the terminal data used in this part, the solution is defined by approximation. Both are recalled in Chapter 2. For the Sherrington–Kirkpatrick model these constructions are in [ref-52, ref-67, ref-68, ref-105] and [ref-76]. The Crisanti–Sommers entropy [ref-44, ref-125] and the Parisi functional are

where $\bar q$ is any number in $[Q_{\gamma},1)$. The value of $\operatorname{CS}(\gamma)$ does not depend on this choice, since $\lambda_{\gamma}(q)=1-q$ on $[Q_{\gamma},1]$. The first term in $\mathcal P_{U}(\gamma)$ accounts for the constraints and the second for the spherical entropy.

For the hard constraint, let $\phi$ and $\Phi$ denote the standard Gaussian density and distribution function. For $\gamma\in\mathcal U$, $\bar q\in[Q_{\gamma},1)$ and $\kappa\in\mathbb R$, let $u_{\gamma}(q,x;\kappa)$ solve (3) on $[0,\bar q]\times\mathbb R$ with the terminal datum

where $G$ is a standard Gaussian variable, and set

Again the value does not depend on $\bar q\in[Q_{\gamma},1)$. The terminal datum (5) is what one obtains by running the equation with $U=H_{\kappa}$ from time 1 to time $\bar q$ with $\gamma=1$, which avoids a singular terminal problem. This variational problem appears in the physics literature [ref-58, ref-59], and it was stated in the form (5)–(6) by El Alaoui and Sellke [ref-54]. For the terminal datum (5), they constructed a classical solution of (3), with bounds on its derivatives, and proved that its derivatives depend continuously on $\gamma$ [ref-54]. We call $\mathcal{P}_{*}(\kappa)$ the Parisi value and $\kappa_{\mathrm{c}}(\alpha)$ the critical margin.

The replica-symmetric order parameters $\mathbf{1}_{[q,1]}$, $q\in[0,1)$, give Gardner’s replica-symmetric functional:

Recall that $\kappa_{\mathrm{RS}}(\alpha)$ is the unique solution of $\alpha\mathbb{E}(\kappa_{\mathrm{RS}}-G)_{+}^{2}=1$. It is Gardner’s prediction for the limit of $\kappa_{N}$ [ref-62], and $\kappa_{\mathrm{RS}}(\alpha)<0$ if and only if $\alpha>2$, since $\mathbb{E}G_{-}^{2}=\frac{1}{2}$. The value $\alpha=2$ at $\kappa=0$ is Cover’s capacity of the perceptron [ref-43].

Finally, for $\gamma\in\mathcal{U}$ and $\kappa\in\mathbb{R}$ we define the Parisi diffusion $X=X^{\gamma,\kappa}$ as follows. On $[0,Q_{\gamma}]$ it is the unique strong solution of

for a standard Brownian motion $B$. Conditionally on $(X_{q})_{q\leq Q_{\gamma}}$, the endpoint $X_{1}$ has the law of $X_{Q_{\gamma}}+\sqrt{1-Q_{\gamma}}G$ conditioned on the event $\{X_{Q_{\gamma}}+\sqrt{1-Q_{\gamma}}G\geq\kappa\}$, where $G$ is a standard Gaussian variable independent of $B$. The equation (8) is well posed by Lemmas 2.6 and 2.7. For mixed $p$-spin models this diffusion appears in the work of Auffinger and W.-K. Chen [ref-7], and for the terminal datum (5) it appears in [ref-54]. For step order parameters, $X$ observed at the breakpoints of $\gamma$ and at time 1 is a Markov chain with explicit tilted Gaussian transitions (Lemma 9.3). The variable $X_{1}-\kappa\geq0$ plays the role of the gap of a typical constraint.

Main results

Our first result is the Parisi formula for smooth activations. We consider two classes of activations:

(A) $U\in C^{2}(\mathbb{R})$, $\sup U<\infty$, and $\lVert U'\rVert_{\infty}+\lVert U''\rVert_{\infty}<\infty$;

(B) $U\in C^{2}(\mathbb{R})$ is concave, $\sup U<\infty$, and $\lVert U''\rVert_{\infty}<\infty$.

No sign condition is imposed on $U'$ in class (A), and the activations in class (B) may tend to $-\infty$ quadratically.

Theorem 1.1. Let $\alpha>0$ and let $U$ belong to class (A) or class (B). Then

Moreover, the infimum defining $\mathcal{P}_{*}(U)$ is attained, and every minimizer $\gamma$ satisfies $Q_{\gamma}\leq r_{U}$, where $r_{U}<1$ is an explicit number depending only on $\alpha$ and $U$, defined in (5.1). If $U$ belongs to class (A), then $r_{U}\leq\alpha\lVert U'\rVert_{\infty}^{2}/(1+\alpha\lVert U'\rVert_{\infty}^{2})$.

Class (A) contains activations that are neither monotone nor concave. This generality is used for the gaps: adding a bounded source term to a smooth approximation of the hard wall destroys monotonicity and concavity. Class (B) contains smooth concave penalties comparable to $-\beta(\ell-x)_{+}$, which we use to pass to hard constraints. The bound on $Q_{\gamma}$ says that the minimizers for a smooth activation stay a fixed distance away from overlap one. For the hard wall this fails as the margin approaches the critical margin; see Theorem 1.3(iv).

For $\alpha>2$ the critical margin is negative, and the feasible sets $S_{N}(\kappa)$ with $\kappa<0$ are not convex. Our second result identifies their exponential volume and the largest achievable margin.

Theorem 1.2. Let $\alpha>2$. Then $-\infty<\kappa_{\mathrm{c}}(\alpha)<0$, and the following statements hold.

(i) For every $\kappa<\kappa_{\mathrm{c}}(\alpha)$, $N^{-1}\log V_{N}(\kappa)\to\mathcal{P}_{*}(\kappa)$ in probability.

(ii) $\kappa_{N}\to\kappa_{\mathrm{c}}(\alpha)$ in probability.

(iii) For every $\kappa\geq\kappa_{\mathrm{c}}(\alpha)$ and every $C<\infty$, $\mathbb{P}\{N^{-1}\log V_{N}(\kappa)>-C\}\to0$.

Part (iii) states that the normalized log volume tends to $-\infty$ at the critical margin itself; it does not assert that $S_{N}(\kappa_{\mathrm{c}})$ is empty. The assumption $\alpha>2$ enters only through the inequality $\kappa_{\mathrm{c}}(\alpha)<0$; for every $\alpha>0$, part (i) holds at the margins $\kappa<\min\{\kappa_{\mathrm{c}}(\alpha),0\}$ (see the end of §8.4). For $\alpha>2$, parts (i) and (iii) confirm the free-energy part of a conjecture of El Alaoui and Sellke [ref-54] (Conjecture 1), and part (ii) confirms its capacity part in the form $\kappa_{N}\to\kappa_{\mathrm{c}}(\alpha)$.

The proofs of Theorem 1.2 and of the gap theorems below rest on the following properties of the variational problem, which hold for every $\alpha>0$. For $\kappa\in\mathbb{R}$ put $C_{\kappa}=\frac{1}{2}(\kappa+\sqrt{\kappa^{2}+4})$, and let $C_{0}=-\log\Phi(-1)$.

Theorem 1.3. Let $\alpha>0$.

(i) If $\mathcal{P}_{*}(\kappa)>-\infty$, then the infimum defining $\mathcal{P}_{*}(\kappa)$ is attained, every minimizer $\gamma$ satisfies $1-Q_{\gamma}>e^{2\mathcal{P}_{*}(\kappa)}$, and the set of minimizers is compact in $L^{1}([0,1])$.

(ii) $-\infty<\kappa_{\mathrm{c}}(\alpha)\leq\kappa_{\mathrm{RS}}(\alpha)<\infty$. If $\alpha>2$, then $\kappa_{\mathrm{c}}(\alpha)<0$.

(iii) $\mathcal{P}_{*}$ is finite, nonincreasing, concave, and continuous on $(-\infty,\kappa_{\mathrm{c}})$, and $\mathcal{P}_{*}(\kappa)=-\infty$ for $\kappa\geq\kappa_{\mathrm{c}}$. More precisely, with $C_{*}=C_{\kappa_{\mathrm{c}}}$, for all sufficiently small $\eta>0$,

(iv) If $\mathcal{P}_{*}(\kappa)>-\infty$ and $\gamma$ is a minimizer, then

Consequently there are $C,\eta_{0}>0$, depending only on $\alpha$, such that for $0<\eta\leq\eta_{0}$ every minimizer $\gamma$ of $\mathcal{P}_{\kappa_{\mathrm{c}}-\eta}$ satisfies

(v) There is $\kappa_{0}(\alpha)<\kappa_{\mathrm{c}}(\alpha)$ such that, for every $\kappa\leq\kappa_{0}(\alpha)$, every minimizer of $\mathcal{P}_{\kappa}$ equals $\mathbf{1}_{[q,1]}$ for some $q\in(0,1)$. In particular $\mathcal{P}_{*}(\kappa)=\min_{q\in[0,1)}\mathcal{P}_{\kappa}^{\mathrm{RS}}(q)$. If $\alpha\leq1$, the same holds for every $\kappa<\kappa_{\mathrm{c}}(\alpha)$.

Part (iii) says that the jamming transition is continuous on the exponential scale: the Parisi value tends to $-\infty$ as the margin increases to $\kappa_{\mathrm{c}}$, and it does so at least logarithmically fast. Part (iv) shows that a very negative value pins the top of the support of every minimizer close to one. Together, (iii) and (iv) give the collapse of minimizers at the critical margin: their tops tend to one and their masses tend to zero, at power rates. This collapse is the input for the gap theorems. By part (v) and Theorem 1.2(i), for $\alpha>2$ and $\kappa\leq\kappa_{0}(\alpha)$,

which is Gardner’s replica-symmetric formula for the free energy [ref-62]. Thus Gardner’s formula holds at sufficiently negative margins; the margin $\kappa_{0}(\alpha)$ is not explicit. Theorem 1.3 is proved in §6.6, and part (v) is Corollary 6.9. Two structural facts behind it hold for every order parameter: an exact value identity (Lemma 6.2) and the universal slope bound $0<\lambda_{\gamma}(0)\,\partial_{x}u_{\gamma}(0,0;\kappa)<C_{\kappa}$ (Lemma 6.4).

Our next result concerns constraint densities slightly above two. There $\kappa_{\mathrm{RS}}(\alpha)$ is close to zero, and Gardner’s prediction is closest to being correct. We write $\alpha=2+e$.

Theorem 1.4. There are numerical constants $e_{0}>0$ and $c>0$ such that, for $0<e<e_{0}$,

Theorem 1.4 is a statement about the variational problem alone, and it is proved in Chapter 7. Combined with Theorem 1.2(ii), it gives the following.

Corollary 1.5. Let $e_{0}$ and $c$ be as in Theorem 1.4, let $0<e<e_{0}$, and put $\alpha=2+e$. Then $\kappa_{N}\to\kappa_{\mathrm{c}}(\alpha)$ in probability, and $\kappa_{\mathrm{c}}(\alpha)\leq\kappa_{\mathrm{RS}}(\alpha)-ce^{3}$. In particular,

For large $\alpha$, comparing the asymptotics as $\kappa\to-\infty$ of the upper bound of Montanari, Zhong, and Zhou, recalled above, with the relation $\alpha\mathbb{E}(\kappa_{\mathrm{RS}}-G)_{+}^{2}=1$ shows that, for all sufficiently large $\alpha$, with probability tending to one no configuration has margin at least $\kappa_{\mathrm{RS}}(\alpha)-1/|\kappa_{\mathrm{RS}}(\alpha)|$. Together with Theorem 1.2(ii), this gives $\kappa_{\mathrm{c}}(\alpha)<\kappa_{\mathrm{RS}}(\alpha)$ for all sufficiently large $\alpha$. Corollary 1.5 shows that the failure of Gardner’s formula extends to $\alpha$ close to two, where $\kappa_{\mathrm{RS}}(2+e)=-\frac{\sqrt{2\pi}}{8}e+O(e^{2})$ is itself close to zero (see eq:7.3). The proof of Theorem 1.4 evaluates the functional at an explicit order parameter with two support points near one.

In Part II we study the minimizers of $\mathcal{P}_{\kappa}$ for $\alpha$ slightly above two. We show that the replica-symmetric solution loses its stability at a de Almeida–Thouless margin $\kappa_{\mathrm{d}}(\alpha)<\kappa_{\mathrm{c}}(\alpha)$, and that immediately beyond $\kappa_{\mathrm{d}}(\alpha)$ every minimizer exhibits full replica symmetry breaking: its Parisi measure is supported on a nondegenerate interval, with a positive continuous density in the interior and atoms at both endpoints (Part II, Theorem 1.2). We also determine the distances between the three margins $\kappa_{\mathrm{d}}<\kappa_{\mathrm{c}}<\kappa_{\mathrm{RS}}$ (Part II, Theorem 1.1). We now restate the results of that part that we use.

Let $R=\phi/\Phi$ and $V(z)=R(z)(z+R(z))$, so that $0<V<1$, and put

By Part II, Lemma 3.5 and Lemma 3.3(c), for $e>0$ sufficiently small and every $\kappa<\kappa_{\mathrm{RS}}(\alpha)$ the function $\mathcal{P}_{\kappa}^{\mathrm{RS}}$ of (7) has a unique minimizer $q^{*}(\kappa)$ on $[0,1)$, and $q^{*}(\kappa)\in(0,1)$. By Part II, Proposition 3.8, there is also a numerical constant $c_{2}>0$ such that exactly one margin $\kappa\in(\kappa_{\mathrm{RS}}(\alpha)-c_{2},\kappa_{\mathrm{RS}}(\alpha))$ satisfies

This margin is the de Almeida–Thouless margin $\kappa_{\mathrm{d}}(\alpha)$. The displayed condition says that the replicon eigenvalue of the replica-symmetric solution vanishes (see [ref-23, ref-59]); on $(\kappa_{\mathrm{d}},\kappa_{\mathrm{RS}})$ this eigenvalue is negative, and the replica-symmetric solution is unstable (Part II, Proposition 3.8). The ordering of thresholds (Part II, Theorem 1.1) states that there are numerical constants $e_{0},c,C>0$ such that, for $0<e<e_{0}$ and $\alpha=2+e$,

The lower bound on $\kappa_{\mathrm{RS}}-\kappa_{\mathrm{c}}$ is Theorem 1.4, which Part II proves again through the zero-temperature functional (Part II, Theorem 3.13). The upper bound rests on a dual certificate for this zero-temperature version of the variational problem (Part II, Theorems 3.10 and 3.11). Zero-temperature Parisi functionals were introduced for the ground state energy of mixed $p$-spin models by Auffinger and W.-K. Chen [ref-9], and for spherical models by [ref-40] and by [ref-78]; see also [ref-75]. Franz and Parisi described the jamming limit of the perceptron through the boundary condition that the replica equations take as the overlap tends to one [ref-58] (15). Part II also shows that for $\kappa\in(\kappa_{\mathrm{d}}(\alpha),\kappa_{\mathrm{c}}(\alpha))$ no minimizer of $\mathcal{P}_{\kappa}$ is replica symmetric (Part II, Corollary 3.14). Combining Theorems 1.2 and 1.3 with Part II, Theorem 1.1, and with this absence of replica-symmetric minimizers gives the following.

Corollary 1.6. Let $e_{0},c,C$ be as in (10), let $0<e<e_{0}$, and put $\alpha=2+e$. Then the following hold.

(i) $\kappa_{N}\to\kappa_{\mathrm{c}}(\alpha)$ in probability, $c e^{3}\leq\kappa_{\mathrm{RS}}(\alpha)-\kappa_{\mathrm{c}}(\alpha)\leq C e^{3}$, and

(ii) For every $\kappa\in(\kappa_{\mathrm{d}}(\alpha),\kappa_{\mathrm{RS}}(\alpha))$, Gardner’s replica-symmetric formula $\min_{q\in[0,1)}\mathcal{P}_{\kappa}^{\mathrm{RS}}(q)$ is finite, and there is $\varepsilon>0$ such that

Part (i) sharpens Corollary 1.5 to exact order: the largest achievable margin converges to a limit that lies below Gardner’s prediction by an amount of exact order $(\alpha-2)^{3}$, and above the de Almeida–Thouless margin by an amount of order $(\alpha-2)^{2}$. The thermodynamic limit is taken at fixed $e$, and part (i) describes the subsequent behavior of the limit $\kappa_{\mathrm{c}}(2+e)$ as $e\downarrow0$. Part (ii) shows that Gardner’s formula for the free energy fails on the whole interval $(\kappa_{\mathrm{d}}(\alpha),\kappa_{\mathrm{RS}}(\alpha))$. By Theorem 1.3(v) it holds at sufficiently negative margins, and by Theorem 1.2(i) and Part II, Theorem 1.2(i), it holds at the margins in $[\kappa_{\mathrm{d}}(\alpha)-c'e^{2},\kappa_{\mathrm{d}}(\alpha)]$ for a numerical constant $c'>0$. The corollary follows directly from the results above. Part (i) is Theorem 1.2(ii) together with (10), since $\kappa_{\mathrm{c}}-\kappa_{\mathrm{d}}=(\kappa_{\mathrm{RS}}-\kappa_{\mathrm{d}})-(\kappa_{\mathrm{RS}}-\kappa_{\mathrm{c}})$. For part (ii), the minimum of $\mathcal{P}_{\kappa}^{\mathrm{RS}}$ is attained at $q^{*}(\kappa)$, so it is finite. Let $\kappa\in(\kappa_{\mathrm{d}},\kappa_{\mathrm{c}})$. By Theorem 1.3(i),(iii), minimizers of $\mathcal{P}_{\kappa}$ exist, and by Part II, Corollary 3.14, none of them is replica symmetric. Hence $\mathcal{P}_{*}(\kappa)<\mathcal{P}_{\kappa}^{\mathrm{RS}}(q^{*}(\kappa))$, and Theorem 1.2(i) gives the claim. For $\kappa\in[\kappa_{\mathrm{c}},\kappa_{\mathrm{RS}})$ the claim follows from Theorem 1.2(iii).

Part II also proves a lower bound on the Parisi value that matches Theorem 1.3(iii) up to the constant in front of the logarithm, through a comparison with a zero-temperature functional (Part II, Proposition 2.24): for every $\alpha>2$ there are $C,\eta_{0}>0$, depending only on $\alpha$, such that

Together with Theorem 1.2(i), this logarithmic divergence gives the following.

Corollary 1.7. Let $\alpha>2$. There are $C,\eta_{0}>0$, depending only on $\alpha$, such that for every $0<\eta\leq\eta_{0}$,

Equivalently, with probability tending to one, $e^{-CN}\eta^{\alpha N}\leq V_{N}(\kappa_{\mathrm{c}}-\eta)\leq e^{CN}\eta^{N/2}$. On the exponential scale the feasible volume thus vanishes polynomially in the distance $\eta$ to the critical margin. The upper bound uses only Theorems 1.2 and 1.3, and we do not claim that either exponent is optimal.

We next describe the configurations that nearly achieve the maximal margin, through their empirical gap measures $\nu_{N,\sigma}$.

Theorem 1.8. Let $\alpha>2$. There is a deterministic measure $\nu_{\alpha}$ on $[0,\infty)$ with $\nu_{\alpha}([0,\infty))=\alpha$ and $\nu_{\alpha}(\{0\})=1$, whose positive part satisfies

such that the following statements hold.

(i) For every bounded Lipschitz $\psi:[0,\infty)\to\mathbb{R}$ and every $\varepsilon>0$ there is $\rho>0$ such that

(ii) For every $c<(\alpha-2)^{2}/(2\alpha)$ and all sufficiently large $N$, with probability at least $1-e^{-cN}$ every maximizer $\sigma_{*}$ of $m_{N}$ has exactly $N$ contacts. For any choice of maximizers, $\nu_{N,\sigma_{*}}\to\nu_{\alpha}$ weakly in probability, and

at every continuity point $t>0$ of $\nu_{\alpha}$.

(iii) For every choice of margins $k\uparrow\kappa_{\mathrm{c}}(\alpha)$ and minimizers $\gamma_{k}$ of $P_{k}$, the measures $\alpha\,\operatorname{Law}(X_{1}^{\gamma_{k},k}-k)$ converge weakly to $\nu_{\alpha}$.

Here $\nu_{N,\sigma_{*}}\to\nu_{\alpha}$ weakly in probability means that $\int\psi\,\mathrm{d}\nu_{N,\sigma_{*}}\to\int\psi\,\mathrm{d}\nu_{\alpha}$ in probability for every bounded continuous $\psi$, uniformly over the maximizers. The count $N$ in part (ii) is isostatic in the sense recalled above: it equals the $N-1$ degrees of freedom on the sphere plus one for the optimized margin. The identity $\nu_{\alpha}(\{0\})=1$ and the bound (13) show that, in the limit, no additional macroscopic set of constraints has small positive gaps: the mass $\nu_{\alpha}((0,t])$ of the small positive gaps tends to zero as $t\downarrow0$, at least as fast as a constant times $1/\log(1/t)$. Nearly optimal configurations need not have exactly $N$ contacts, but by part (i) their gap statistics are close to $\nu_{\alpha}$, and by part (iii) the limiting gap law is identified through Parisi minimizers approaching the critical margin. The theorem does not identify the behavior of $\nu_{\alpha}$ near zero, where physics predicts a power law.

The last theorem identifies the gap statistics of typical feasible configurations at a fixed margin below the critical one.

Theorem 1.9. Let $\alpha> 2$ and $\kappa< \kappa_{\mathrm{c}}(\alpha)$, let $\gamma$ be a minimizer of $\mathcal{P}_{\kappa}$, and let $\psi: [0,\infty) \to\mathbb{R}$ be bounded and $L$-Lipschitz with $L > 0$. For every $\varepsilon> 0$, with probability tending to one, $V_{N}(\kappa) > 0$ and

Consequently, writing $\langle\cdot\rangle_{N,\kappa}$ for expectation under the uniform probability measure on $S_{N}(\kappa)$,

and all minimizers of $\mathcal{P}_{\kappa}$ induce the same law of $X_{1}^{\gamma,\kappa}$.

Applied to finitely many test functions at once, the theorem shows that all but an exponentially small fraction of the feasible set has the gap statistics of the Parisi diffusion. The rate in (14) depends on the minimizer only through its mass $\lambda_{\gamma}(0)$, and any minimizer may be used. We do not prove that the minimizers of $\mathcal{P}_{\kappa}$ are unique, and none of our arguments requires it; the last assertion of the theorem shows that they all induce the same terminal law. For mixed $p$-spin models, uniqueness follows from strict convexity of the Parisi functional [ref-7, ref-76], which is not known for $\mathcal{P}_{\kappa}$. For generic mixed $p$-spin models, Auffinger and Jagannath expressed spin statistics through solutions of partial differential equations [ref-10].

Throughout, $C$ and $c$ denote positive constants. A numerical constant depends on nothing; otherwise the dependence of a constant is indicated in the statement where it appears, as in “depending only on $\alpha$”. Their values may change from one occurrence to the next. Limits in probability refer to the joint law of the patterns and of any auxiliary randomness introduced in the proofs.

Ideas of the proof

The proof has three parts. Steps 1 and 2 below prove the Parisi formula for smooth activations. Steps 3 and 4 analyze the deterministic variational problem for the hard wall.

Steps 5 and 6 transfer these results to the hard constraints of the random model and to the gaps. Figure 1.1 shows how the steps depend on one another.

Step 1: The upper bound. The argument follows the proof of [ref-92], Theorem 4.1, a step in Mourrat’s upper bound for bipartite models; the opening of Chapter 3 compares our argument with his. We use a Guerra-type interpolation [ref-68] between the perceptron and a decoupled system on a Ruelle probability cascade [ref-19, ref-112]. Since $P_U$ is continuous in the order parameter, it suffices to treat order parameters $\gamma_h$ with finitely many atoms $h_0 \le\cdots\le h_k$ of equal mass. As the interpolation time $t$ runs from 0 to 1, the argument of each activation moves from a Gaussian field on the leaves of the cascade, with covariance $h_l$ between two leaves that branch at level $l$, to $b_a \cdot\sigma$. The coordinates are coupled to a second cascade field, whose levels $p=(p_0,\ldots,p_k)$ are free parameters. The derivative in $t$ of the interpolating free energy $u_N$ is expressed through two overlaps: the spin overlap $R_{12}=\sigma^1\cdot\sigma^2/N$, and the response overlap $X_{12}=M^{-1}\sum_a U'(S_a^1)U'(S_a^2)$, where $S_a^l$ is the interpolated field of constraint $a$ for replica $l$. Overlaps of the derivatives of the activation enter in a similar way in Talagrand’s treatment of the perceptron [ref-127], Chapters 2 and 3, [ref-128], Chapter 8. Let $J$ be the level at which the leaves of two replicas branch, $w_l$ the probability that $J=l$, and $r_l$ and $\zeta_l$ the conditional means of $R_{12}$ and of $\alpha_N X_{12}$ given $J=l$, where $\alpha_N=M/N$. Then

Neither term has a sign. We control the products through the choice of $p$. At a maximizer over $p$ of $u_N+\frac{1}{2}\sum_l w_l h_l p_l$, the first-order conditions, together with the monotonicity $0\leq\zeta_0\leq\cdots\leq\zeta_k$, make the total contribution of the products nonpositive. This monotonicity holds for every activation in class (A); we derive it from an exact change of measure on a cascade with marks. For Gaussian fields the corresponding monotonicity is [ref-128], Proposition 14.3.2 and [ref-92], Lemma 2.4. A maximum principle in $t$ alone then bounds the free energy by the maximum over $p$ at $t=0$, up to the covariance terms. To control these, small Gaussian perturbations of the Hamiltonian, of the form used in [ref-92], Section 4, enforce approximate Ghirlanda–Guerra identities [ref-64] for the pair of overlap arrays at the points where the maximum principle is applied. Mourrat’s quantitative synchronization [ref-92], which builds on Panchenko’s synchronization mechanism [ref-100, ref-101], then makes the conditional variance of $R_{12}$ small, and the Cauchy–Schwarz inequality makes the covariances small whatever the sign of $U'$. This is why no sign condition on $U'$ is needed. At $t=0$ the system splits into the row term $\alpha_N u_{\gamma_h}(0,0;U)$ and a spherical term. An exact computation with Gaussian cascades bounds the spherical term plus $\frac{1}{2}\sum_l w_l h_l p_l$, maximized over $p$, by $\operatorname{CS}(\gamma_h)+o(1)$; a computation of this kind for spherical spin glasses is in [ref-125]. This is carried out in Chapter 3.

Step 2: The lower bound. We use the Aizenman–Sims–Starr scheme [ref-1] in the spherical form of [ref-37]. For multi-species spherical models, Bates and Sohn combined W.-K. Chen’s cavity computation with Panchenko’s synchronization [ref-100, ref-101] to prove the lower bound [ref-16]. We follow the same route, with the response overlap in the role of the overlap of the second species. The scheme bounds the free energy from below by the increment of the expected log partition function when $n$ coordinates, and about $\alpha n$ constraints, are added to a system with $N$ coordinates. We write the sphere of dimension $N+n$ exactly as a product of the sphere of dimension $N$ and a density for the $n$ new coordinates (compare [ref-47]). The increment then splits into a row term and a coordinate term. In the limit the row term is at least $\alpha n\,u_\zeta(0,0;U)$, where $\zeta$ is the limiting overlap law. After a Gaussian interpolation, the coordinate term becomes a Gaussian integral over the new coordinates $\varepsilon\in\mathbb{R}^n$. After a perturbation (see [ref-98] and [ref-100], (24)–(28)), the joint law of the spin overlap and the response overlap satisfies the Ghirlanda–Guerra identities. By synchronization and the characterization of such arrays [ref-49, ref-92, ref-97, ref-100, ref-101], it is a limit of Ruelle probability cascades with paired nondecreasing levels. On such a cascade the coordinate term is an explicit Gaussian integral, which for general paired levels is not the entropy of any order parameter.

Two identities of the system determine the result. First, a Gaussian integration by parts in the rows expresses the mean effective precision of the new coordinates through the two overlaps. In the limit, this precision identity shows that the smallest precision is at least one, so that the Gaussian integral is finite, and it cancels the constant terms. Second, the law of the patterns is invariant under rotations. Under a uniformly random rotation, the overlap $\varepsilon^{1}\cdot\varepsilon^{2}/n$ of the new coordinates of two replicas is close to their full overlap, with mean squared error at most $3/n$ in the limit. Further, $\mathbb{E}\lVert\varepsilon\rVert^{2}/n=1$, and this norm identity turns the coordinate term into $n\operatorname{CS}(\gamma)$, where $\gamma$ is the law of the mean overlap of the new coordinates at the branching level, and the overlap estimate makes $\gamma$ close to $\zeta$. Hence, as $N\to\infty$, the increment per added coordinate is at least

Rotation invariance thus forces the cavity term to equal the Crisanti–Sommers entropy, and no minimizer or critical point of a cavity functional has to be identified. The main technical issue is the unbounded Gaussian integral over $\varepsilon$. We compute it on balls, pass to the limit $N\to\infty$ there, evaluate the limit on finite cascades, and remove the cutoff on these cascades, where the precision identity gives uniform control. This is carried out in Chapter 4. Chapter 5 combines the two bounds. There a truncation argument, which moves the mass of an order parameter above the level $r_{U}$ down to $r_{U}$ and so decreases $\mathcal{P}_{U}$, gives the attainment of the infimum and the bound $Q_{\gamma}\leq r_{U}$ for minimizers.

Step 3: The variational problem at hard margins. For the hard wall we prove an exact value identity, valid for every order parameter:

Here $Q=Q_{\gamma}$, $\delta=1-Q$, $Z=(X_{Q}-\kappa)/\sqrt{\delta}$ is the normalized top position of the Parisi diffusion, $h(z)=\log\Phi(z)+\frac{1}{2}R(z)^{2}<0$, and $D(q)=\alpha\mathbb{E}\,\partial_{x}u_{\gamma}(q,X_{q};\kappa)^{2}-\int_{0}^{q}\lambda_{\gamma}^{-2}$ is twice the density of the first variation of $\mathcal{P}_{\kappa}$. El Alaoui and Sellke computed this first variation at a minimizer whose support is an interval, in directions supported on that interval [ref-54] (Proposition 16) (see the opening of Chapter 2). Here it is used at every order parameter and in every direction that keeps the top of the support below a fixed level; Chapter 6 recalls the first-order conditions known for mixed $p$-spin models. At a minimizer over the order parameters whose top is at most a fixed level $r<1$, the last two terms are nonpositive, so $\mathcal{P}_{\kappa}(\gamma)<\frac{1}{2}\log\delta$. A finite value therefore keeps the top of every minimizer away from one, which gives existence and compactness of minimizers. The second tool is the universal slope bound

valid for every order parameter and proved by a maximum principle for an explicit combination of $\partial_x u_\gamma$ and $\partial_{xx}u_\gamma$. The map $\kappa\mapsto\mathcal{P}_\kappa(\gamma)$ is concave, by Prékopa’s theorem [ref-108], with derivative $-\alpha\,\partial_xu_\gamma(0,0;\kappa)$, and $\lambda_\gamma(0)\geq\delta$, so the two tools transport values between margins. Take a constrained minimizer $\gamma$ with a prescribed value $w$ at a margin $k>\kappa_c$, where the unconstrained value is $-\infty$. The value identity gives $1-Q_\gamma>e^{2w}$, so its slope in the margin is at most $\alpha C_k e^{-2w}$. Concavity and the limit $k\downarrow\kappa_c$ give $\mathcal{P}_*(\kappa_c-\eta)\leq w+\alpha\eta C_{\kappa_c}e^{-2w}$, and the choice $w=\frac{1}{2}\log(2\alpha C_{\kappa_c}\eta)$ gives Theorem 1.3(iii).

For a minimizer over all order parameters, the value identity holds without the last two terms. Since $h(z)$ is bounded below by $-\log(1+z_-)$ up to an additive constant, and the slope bound controls the negative part of $Z$, Jensen’s inequality gives the pinning bound of Theorem 1.3(iv). Combined with (iii) at the margin $\kappa_c-\eta$, it forces $1-Q_\gamma\leq C\eta^{1/(1+\alpha)}$. Finally, weak top stability, a lower bound on the quantity $S(Q)=\alpha\delta^2\mathbb{E}\,\partial_{xx}u_\gamma(Q,X_Q;\kappa)^2$ for constrained minimizers with more than one support point, gives $\lambda_\gamma(0)\leq C(1-Q_\gamma)^{1/2}$. At very negative margins the same lower bound cannot hold, so every minimizer is replica symmetric; this gives $\kappa_c>-\infty$ and Theorem 1.3(v). No zero-temperature functional is needed. This is carried out in Chapter 6.

Step 4: The critical margin near $\alpha=2$. To go below Gardner’s prediction we evaluate $\mathcal{P}_\kappa$ at an explicit order parameter with two support points near one,

which splits the atom of a replica-symmetric order parameter and places a thin layer of relative width $\ell$ below the top. The homogeneity and the monotonicity of the Cole–Hopf step and a Gaussian tail bound give $\delta\mathcal{P}_\kappa(\gamma_\delta)\leq QJ_\ell(\kappa/\sqrt{Q})+\frac{1}{2}\delta\log\delta$ for an explicit Gaussian integral $J_\ell$, and

If $J_\ell(k)<0$, then $\mathcal{P}_\kappa(\gamma_\delta)\to-\infty$ as $\delta\downarrow0$ for every $\kappa>k$, so that $\kappa_c\leq k$. At $k=\kappa_{\mathrm{RS}}$ the first term vanishes, while $\alpha\Phi(\kappa_{\mathrm{RS}})-1$ is of exact order $e=\alpha-2$. The layer therefore gains an amount of order $e\ell$ at a cost of order $\ell^{3/2}$, and the choice $\ell\asymp e^2$ makes $J_\ell(\kappa_{\mathrm{RS}})$ negative, of order $e^3$. Lowering the margin by a small multiple of $e^3$ keeps it negative. The argument uses no minimizer and no regularity theory beyond the recursion for this single order parameter. Unlike the bounds of Stojnic and of Montanari, Zhong, and Zhou recalled at the beginning of this chapter, the bound comes from the Parisi functional at an explicit order parameter, and the thin layer of width $\ell\asymp e^2$ gives the order $e^3$ near $\alpha=2$. An explicit trial order parameter was used in the same way by Toninelli to show that the Sherrington–Kirkpatrick model is not replica symmetric beyond the de Almeida–Thouless line [ref-129]. This is carried out in Chapter 7.

Step 5: From soft free energy to hard feasible volume. The upper bounds on the volume and on $\kappa_N$ come from Step 1, applied to smooth approximations of the hard wall from above, together with a lemma showing that a single feasible configuration forces an exponentially nonnegligible feasible volume after a fixed decrease of the margin. For the lower bound, the Parisi formula for a concave penalty comparable to $-\beta(\ell-x)_{+}^{2}$ bounds from below the volume of the configurations whose total squared violation $\sum_a(k_0-b_a\cdot\sigma)_{+}^{2}$, at a level $k_0$ slightly below $\ell$, is at most $tN$, for small $t>0$. Two geometric facts turn this into feasible volume. First, a small violation can be repaired by a small displacement. Using the smallest singular value of the pattern matrix over sparse sets of rows (see [ref-130]), a separation argument moves such a configuration by at most $4\sqrt{tN}$ to a point that satisfies every constraint at a slightly smaller margin. The opening of Chapter 8 compares this repair with that of El Alaoui and Sellke [ref-54]. Second, the repair map may collapse volume, so we do not use its image. Instead we add to the configuration a Gaussian vector tangent to the sphere, with variance $\rho^{2}$ in each direction, and project back to $S_N$. This preserves $\mu_N$. Translating the Gaussian vector by a displacement $d$ lowers the probability of a symmetric set by at most the factor $e^{-\lVert d\rVert^{2}/(2\rho^{2})}$, and Šidák’s inequality [ref-116] shows that, with probability at least $e^{-N\epsilon}$ for a small $\epsilon>0$, the Gaussian vector moves no constraint by more than a small amount. Hence the perturbed point is feasible at a slightly smaller margin with probability at least $e^{-N\epsilon'}$, where $\epsilon'$ is small when $t$ is small compared with $\rho^{2}$. The upper bounds are proved in §§3.8 and 3.9 and the lower bound in Chapter 8.

Step 6: Gap statistics. Gap averages enter through the free energy of the feasible set with a source term $\beta\sum_a\psi(b_a\cdot\sigma-k)$, for a bounded Lipschitz test function $\psi$. Adding the source to a smooth approximation of the hard wall destroys monotonicity, and this is where the generality of class (A) is used. We prove a quadratic source estimate: the Parisi functional at $\gamma$ with the source exceeds its linearization in $\beta$, whose slope is $\alpha\mathbb{E}\psi(X_1^{\gamma,k}-k)$, by at most $\frac{1}{2}\alpha\operatorname{Lip}(\psi)^{2}\beta^{2}\lambda_\gamma(0)$. For a step order parameter the effect of the source is an iterated logarithmic moment along the Parisi diffusion observed at the breakpoints of $\gamma$. The transitions of this Markov chain are Gaussian laws tilted by log-concave weights. Like Gaussian laws, they have sub-Gaussian linear statistics and map Lipschitz functions to Lipschitz functions with the same constant (for log-concave tilts of Gaussian laws see [ref-25, ref-27]), and a backward induction with a second-order expansion at each step gives the estimate. Combined with Jensen’s inequality and the upper bound of Step 1, it yields one deviation inequality. With probability tending to one, simultaneously for every set $T\subseteq S_N(k)$ of positive volume, the average over $T$ of $N^{-1}\sum_a\psi(b_a\cdot\sigma-k)$ differs from $\alpha\mathbb{E}\psi(X_1^{\gamma,k}-k)$ by at most

For Theorem 1.9 we apply it at a fixed margin $\kappa< \kappa_{\mathrm{c}}$ to the set of feasible configurations whose gap average deviates by more than $\varepsilon$, and Theorem 1.2(i) shows that this set is an exponentially small fraction of $S_N(\kappa)$. For Theorem 1.8, a single configuration of nearly maximal margin is surrounded by a set of configurations that are feasible at a margin smaller by $\eta$, have almost the same gaps, and have volume at least $\mathrm{e}^{-N(\log(1/\eta)+C)}$. Optimizing over $\beta$ leaves an error of order $\eta+\{\lambda\gamma(0)(1+\log(1/\eta))\}^{1/2}$, which tends to zero by the collapse of minimizers in Theorem 1.3(iv).

The exact count of contacts does not use the variational problem. By Wendel’s theorem [ref-131], $\kappa_N<0$ with probability exponentially close to one when $\alpha>2$. The maximizers of $m_N$ then correspond to the points of the polytope $\{y\in\mathbb{R}^N:g_a\cdot y\geq-1\text{ for all }a\}$ farthest from the origin. These are vertices, and Gaussian general position gives exactly $N$ contacts. A union bound over sets of $N$ rows gives a finite-$N$ form of (13) (Proposition 9.2). This is carried out in Chapter 9.

Chapter 2 collects the analytic properties of the Parisi functional used throughout: the Cole–Hopf recursion for step order parameters, Lipschitz continuity in the order parameter, and, for the hard wall, a priori bounds, signs of derivatives, and the first variation. Theorem 1.1 is proved in Chapter 5, Theorem 1.3 in Chapter 6, Theorem 1.4 in Chapter 7, Theorem 1.2 in Chapter 8, and Theorems 1.8 and 1.9 in Chapter 9. The corollaries follow from these theorems and, for Corollaries 1.6 and 1.7, from Part II, as explained next to their statements. The lemmas of §2.3, parts (i)–(iv) of Lemma 6.1, one Gaussian bound in the proof of Lemma 6.6 and the expansions of Chapter 7 are derived from Part II, §§2.1–2.5, Lemma 3.1 and Chapter A, which do not use Part I. Through them, Theorems 1.2–1.4, 1.8, and 1.9 also rest on Part II; Theorem 1.1 does not.

The Parisi functional

This chapter collects the deterministic facts about the Parisi functional that are used in the rest of this part. §2.1 introduces the Cole–Hopf step $T_{m,s}$ and the backward recursion that solves the Parisi equation (3) when the order parameter is a step function. It records the monotonicity of the recursion in its terminal datum, which is used in Chapters 3, 5, 8, and 9, and the homogeneity of the Cole–Hopf step, which is used in Chapter 7. It also shows that neither the Crisanti–Sommers entropy nor the hard-wall value depends on the cutoff used to define it. §2.2 proves that, for activations in class (A) or class (B), the value $u_{\gamma}(0,0;U)$ is Lipschitz in $\gamma$ for the $L^{1}$ distance, with an explicit constant (Lemmas 2.3 and 2.4). This defines $u_{\gamma}$ for general order parameters, and it lets the upper bound (Chapter 3), the lower bound (Chapter 4), the truncation argument of Chapter 5 and the passage to hard constraints in Chapter 8 move between step order parameters and general ones. These continuity estimates rest on one identity, (30) in Lemma 2.2, which writes the difference of two solutions with the same terminal datum as an integral along a diffusion. Its only input is a formula for the derivatives of one Cole–Hopf step (Lemma 2.1). §2.3 treats the hard-wall functional $\mathcal{P}_{\kappa}$ on order parameters that equal one above a fixed level $r<1$. There we state a priori bounds on the solution and on the associated diffusion (Lemma 2.6), Lipschitz continuity in the order parameter (Lemma 2.7), signs and a martingale (Lemma 2.8), and the first variation (Lemma 2.9). These are the inputs for the analysis of the variational problem in Chapter 6 and for the gap statistics in Chapter 9. Most of them are derived from Part II, as explained at the beginning of §2.3.

The hard-wall functional $\mathcal{P}_{\kappa}$ was studied by El Alaoui and Sellke in [ref-54], Section 4. For step order parameters they solved the Parisi equation with the terminal datum (5) by the Cole–Hopf recursion [ref-54], (4.16) and proved a priori bounds on the solution [ref-54], Proposition 12. They proved Lipschitz continuity in the order parameter [ref-54], Proposition 13, extended the solution to general order parameters [ref-54], (4.12), and proved that it has one-sided derivatives in time [ref-54], Lemma 14. They also computed the first variation of $\mathcal{P}_{\kappa}$ in terms of the Parisi diffusion [ref-54], (2.8), (4.13), and (4.15), at a minimizer whose support is an interval, in directions supported on that interval, and deduced the first-order conditions $D=0$ and $S=1$ on that interval [ref-54], Proposition 16, with $D$ and $S$ as in (44). Their arguments build on work on the Sherrington–Kirkpatrick and mixed $p$-spin models: Guerra’s Lipschitz bound in the order parameter [ref-67], the Feynman–Kac proof of it by Jagannath and Tobasco [ref-76] (Lemma 14 and Remark 15), the diffusion and the regularity estimates of Auffinger and W.-K. Chen [ref-7, ref-8], and the first-variation formulas of W.-K. Chen [ref-38] and of El Alaoui, Montanari, and Sellke [ref-55] (Proposition 6.8). This chapter and Part II, Chapter 2 follow the same route, with the following differences. Concavity of the solution is obtained from Prékopa’s theorem (Lemma 2.1) rather than from an explicit computation along the recursion, which uses that the order parameter is nondecreasing [ref-54] (proof of (4.6)). The bound $-\partial_{xx}u_{\gamma}\le(1-\bar q)^{-1}$ of [ref-54] ((4.6)) is sharpened to the strict bound $-\partial_{xx}u_{\gamma}<1/\lambda_{\gamma}$ (Lemma 2.8). The constants in §2.3 are uniform for $\kappa$ in a compact interval. The first variation is computed at every order parameter in the class $\mathcal{U}_{r}$ of order parameters equal to one on $[r,1]$, in every direction that stays in $\mathcal{U}_{r}$ (Lemma 2.9), and not only at a minimizer. Finally, the Lipschitz estimates of §2.2 cover activations in class (B), which are concave and may tend to $-\infty$ quadratically.

Throughout this chapter, $G$ denotes a standard Gaussian variable and $G'$ an independent copy of it, and $\phi$ and $\Phi$ denote the standard Gaussian density and distribution function. All functions are Borel measurable, and $\lVert\cdot\rVert_{L^{1}}$ is the norm of $L^{1}([0,1])$.

Order parameters and the Cole–Hopf recursion

Recall from §1.1 that an order parameter is a nondecreasing, right-continuous function $\gamma:[0,1]\to[0,1]$ that is identically one on $[\bar q,1]$ for some $\bar q<1$, and that $\mathcal{U}$ is the set of order parameters. For $\gamma\in\mathcal{U}$ we write

Thus $\gamma$ is the distribution function of a probability measure on $[0,Q_{\gamma}]$. More generally, we identify every probability measure on $[0,1]$ with its distribution function, a nondecreasing right-continuous function $\gamma:[0,1]\to[0,1]$ with $\gamma(1)=1$.

The Cole–Hopf step. For $m\ge0$, $s\ge0$, and a function $f:\mathbb{R}\to[-\infty,\infty)$ that is bounded above, define

with the conventions $e^{-\infty}=0$ and $\log0=-\infty$. The expectations exist because $f$ is bounded above, $T_{m,s}f$ takes values in $[-\infty,\sup f]$, and $T_{m,0}f=f$. For functions $f,g$ that are bounded above, $a\in\mathbb{R}$, $m\ge0$, and $s,s'\ge0$, the following hold.

(T1) If $f\le g+a$ on $\mathbb{R}$, then $T_{m,s}f\le T_{m,s}g+a$ on $\mathbb{R}$.

(T2) $T_{m,s}T_{m,s'}f=T_{m,s+s'}f$.

(T3) $t\,T_{m,s}f=T_{m/t,s}(tf)$ for every $t>0$.

(T4) $\mathcal{T}_{m,s}f \ge\mathcal{T}_{0,s}f$.

If $f \equiv-\infty$, then $\mathcal{T}_{m,s}f \equiv-\infty$ for all $m,s$, and (T2)–(T4) for $f$ are immediate; below we assume $f \not\equiv-\infty$, so that $\sup f$ is finite. Property (T1) holds because $y \mapsto e^{my}$ is nondecreasing and $\mathcal{T}_{m,s}(g+a)=\mathcal{T}_{m,s}g+a$. For (T2), write $x+\sqrt{s+s'}G$ as $x+\sqrt{s}G+\sqrt{s'}G'$. If $m>0$, Tonelli’s theorem applied to $e^{mf}\ge0$ gives

and if $m=0$ the same argument applies to $\sup f-f\ge0$. Property (T3) follows from (16), and (T4) is Jensen’s inequality $\mathbb{E}e^{mf}\ge e^{m\mathbb{E}f}$ for $m>0$ and an equality for $m=0$.

The operator $\mathcal{T}_{m,s}$ solves the Parisi equation over an interval on which the order parameter equals a constant $m$; this is the Cole–Hopf transformation. Indeed, if $u$ solves (3) with $\gamma=m>0$, then $h=e^{mu}$ satisfies

the backward heat equation, whose solutions are Gaussian averages of their later values. When $m=0$ the equation is itself the backward heat equation. The recursion below turns this computation into a definition.

The step recursion. Let $T\in[0,1]$. A step function on $[0,T)$ is a function $\gamma=\sum_{j=0}^{n}m_j\mathbf{1}_{[t_j,t_{j+1})}$ with $0=t_0<t_1<\cdots<t_{n+1}=T$ and $m_j\ge0$. For such $\gamma$ and a function $F:\mathbb{R}\to[-\infty,\infty)$ that is bounded above, define $u_\gamma(\mathord{\cdot},\mathord{\cdot};F)$ on $[0,T]\times\mathbb{R}$ by the backward recursion

We call $F$ the terminal datum and $T$ the terminal time; the terminal time is always clear from the context. By (T2), two consecutive steps with the same value compose into one step, so inserting a breakpoint does not change $u_\gamma$. Since two representations of $\gamma$ have a common refinement, $u_\gamma(\mathord{\cdot},\mathord{\cdot};F)$ depends only on $\gamma$ and $F$. For $T=0$ there are no steps and $u_\gamma(0,\mathord{\cdot};F)=F$. The recursion goes back to Parisi [ref-105]; see also [ref-68, ref-128], Chapter 14, and [ref-98], Chapter 3.

A step order parameter is an order parameter of the form

with $0\le m_0\le\cdots\le m_n=1$ and $\gamma(1)=1$. For such $\gamma$ and an activation $U$ that is bounded above, $u_\gamma(\mathord{\cdot},\mathord{\cdot};U)$ is given by (17) with $T=1$ and $F=U$. The same definition applies to every step function $\gamma$ that is the distribution function of a probability measure on $[0,1]$; its value $m_n$ on the last interval $[t_n,1)$ need not be one.

The properties of $\mathcal{T}_{m,s}$ pass to the recursion. By (T1), for functions $F,F'$ bounded above and $a \in\mathbb{R}$,

In particular $F \mapsto u_{\gamma}(q,x;F)$ is nondecreasing and commutes with the addition of constants. For real-valued $F,F'$ whose recursions are finite on $[0,T] \times\mathbb{R}$ (in particular, when both terminal data are bounded below by a quadratic),

By (T4), (T1) and the case $m=0$ of (T2), $u_{\gamma}(q,x;F) \ge\mathbb{E}F(x+\sqrt{T-q}\,G)$. Hence, if $F(y) \ge-C(1+y^{2})$ for all $y$, then

If $\gamma=1$ on $[\bar{q},T)$ for some $\bar{q} \in[0,T)$, then (T2) gives

We next record the regularity of the recursion. Suppose that $F$ is real-valued and continuous, and that $-C(1+y^{2}) \le F(y) \le C$ for a constant $C$. Then $u=u_{\gamma}(\mathord{\cdot},\mathord{\cdot};F)$ is real-valued and continuous on $[0,T] \times\mathbb{R}$, and on each open interval $(t_{j},t_{j+1})$ it is $C^{\infty}$ and solves

By (21) and induction over the steps, it suffices to prove the following for a continuous function $f$ with $-C'(1+y^{2}) \le f(y) \le C'$, a number $m \ge0$, and $g(s,x)=\mathcal{T}_{m,s}f(x)$: the function $g$ is $C^{\infty}$ on $(0,\infty) \times\mathbb{R}$ with $\partial_{s}g=\frac{1}{2}\partial_{xx}g+\frac{m}{2}(\partial_{x}g)^{2}$, and $g(s,x) \to f(x_{0})$ as $(s,x) \to(0,x_{0})$. For $m>0$ and $s>0$,

Differentiation under the integral shows that the right side is positive, $C^{\infty}$ in $(s,x)$, and solves $\partial_{s}h=\frac{1}{2}\partial_{xx}h$; reading the Cole–Hopf computation backwards gives the equation for $g$. For $m=0$ the same argument applies to $g$ itself, since $|f(y)| \le C'(1+y^{2})$. For the limit, fix $\rho>0$. For $m>0$,

and for $m=0$ the last term is replaced by $\mathbb{E}[|f(x+\sqrt{s}\,G)-f(x)|;\sqrt{s}|G|>\rho]$, which is at most $C''(1+x^{2})\mathbb{P}\{\sqrt{s}|G|>\rho\}^{1/2}$ for $s \le1$, with $C''$ depending only on $C'$, by the Cauchy–Schwarz inequality. Letting $s \downarrow0$ and then $\rho\downarrow0$, the continuity of $f$ shows that $\mathcal{T}_{m,s}f \to f$ locally uniformly as $s \downarrow0$ (for $m>0$ because $e^{mf}$ is continuous and positive), and hence $g(s,x) \to f(x_{0})$.

General order parameters. For $\gamma\in\mathcal{U}$ choose $\bar{q} \in[Q_{\gamma},1)$. There are step order parameters $\gamma_{n}$ with $\gamma_{n}=1$ on $[\bar{q},1]$ and $\lVert\gamma_{n}-\gamma\rVert_{L^{1}}\le1/n$. Indeed, let $0=t_{0}<\cdots<t_{\ell}=\bar{q}$ have mesh at most $1/n$, and put $\gamma_{n}(q)=\gamma(t_{j})$ on $[t_{j},t_{j+1})$ for $j<\ell$ and $\gamma_{n}=1$ on $[\bar{q},1]$. Since $\gamma$ is nondecreasing,

Further breakpoints may be added to the partition without affecting this bound. The same construction with $\bar{q}=1$ (and $\gamma_{n}(1)=1$) approximates the distribution function of any probability measure on $[0,1]$ by nondecreasing step functions with values in $[0,1]$. For general $\gamma$, and for the terminal data treated below (activations in class (A) or class (B), and the hard wall), the solution $u_{\gamma}$ is defined as the limit of $u_{\gamma_{n}}$. That the limit exists and does not depend on the approximating sequence follows from the Lipschitz estimates in $\gamma$: Lemma 2.3 for class (A), Lemma 2.4 for class (B), and Lemma 2.7 for the hard wall. The properties (19), (20) and (21) pass to these limits. For mixed $p$-spin models, Auffinger and W.-K. Chen define the solution for general order parameters in the same way [ref-8].

The entropy and the cutoff. The definitions (4) and (5) involve a number $\bar{q}\in[Q_{\gamma},1)$, and neither $\mathrm{CS}(\gamma)$ nor $u_{\gamma}(\cdot,\cdot;\kappa)$ depends on this choice. We prove this here for the entropy and for step order parameters; the extension to general $\gamma\in\mathcal{U}$ is given after Lemma 2.7. For $\mathrm{CS}$, let $Q_{\gamma}\le\bar{q}<\bar{q}'<1$. Since $\lambda_{\gamma}(q)=1-q$ on $[Q_{\gamma},1]$,

Equivalently, since the integrand below vanishes on $[Q_{\gamma},1]$,

For the hard wall, put

the terminal datum (5) at time $\bar{q}$; its dependence on $\kappa$ is suppressed. Let $\gamma$ be a step order parameter and $Q_{\gamma}\le\bar{q}<\bar{q}'<1$. Since $\gamma=1$ on $[\bar{q},\bar{q}']$, (22) with terminal time $\bar{q}'$ gives

Thus the recursion from $f_{\bar{q}'}$ at time $\bar{q}'$ produces $f_{\bar{q}}$ at time $\bar{q}$, and the two definitions of $u_{\gamma}(q,x;\kappa)$ agree on $[0,\bar{q}]$. The same computation with the hard constraint $H_{\kappa}=\log1_{[\kappa,\infty)}$ of §1.1 as terminal datum at time one gives $T_{1,1-\bar q}H_{\kappa}=f_{\bar q}$, so for every step order parameter $\gamma$ and $\bar q\in[Q_{\gamma},1)$,

Taking $\bar q=q$ in the definition, we obtain for every step order parameter

Classes with a common cutoff. For $0\leq r<1$ let

The entropy is Lipschitz on each $\mathcal{U}_{r}$. For $\gamma_{1},\gamma_{2}\in\mathcal{U}_{r}$ use the common cutoff $\bar q=r$. On $[0,r]$ we have $\lambda_{\gamma_i}\geq\lambda_{\gamma_i}(r)=1-r$, and $|\lambda_{\gamma_1}(q)-\lambda_{\gamma_2}(q)|\leq\|\gamma_1-\gamma_2\|_{L^1}$. Hence

Each $\mathcal{U}_{r}$ is a compact metric space for the $L^1$ distance. The $L^1$ distance is a metric on $\mathcal{U}_{r}$, because two right-continuous functions that agree almost everywhere agree on $[0,1)$. By Helly’s selection theorem, a sequence in $\mathcal{U}_{r}$ has a subsequence converging at every point of $[0,1]$ to a nondecreasing function $g$ with values in $[0,1]$ and $g=1$ on $[r,1]$. The right-continuous version of $g$ lies in $\mathcal{U}_{r}$ and differs from $g$ at countably many points only, so the subsequence converges to it in $L^1$ by dominated convergence.

Continuity in the order parameter for smooth activations

We now show that $\gamma\mapsto u_{\gamma}(0,0;U)$ is Lipschitz for the $L^1$ distance when $U$ belongs to class (A) or class (B). Three arguments need this. The upper bound of Chapter 3 is proved for step order parameters and is transferred to all of $\mathcal{U}$ within a class $\mathcal{U}_{r}$. The lower bound of Chapter 4 compares two overlap laws in the Wasserstein distance, and these laws need not equal one near $q=1$. The truncation argument of Chapter 5 needs an explicit Lipschitz constant. We therefore allow distribution functions $\gamma,\nu$ of arbitrary probability measures on $[0,1]$, for which

We use one pathwise bound for diffusions whose drift has linear growth. Let $B$ be a standard Brownian motion on $[0,1]$ and $B^{*}=\sup_{t\leq1}|B_t|$, so that $\mathbb{E}(B^{*})^2\leq4$ by Doob’s inequality. If $0\leq q\leq T\leq1$ and $\mathrm{d}Y_s=\beta(s,Y_s)\,\mathrm{d}s+\mathrm{d}B_s$ on $[q,T]$ with $Y_q=x$ and $|\beta(s,y)| \le a + K'|y|$, then $|Y_s| \le|x| + 2B^* + a + K'\int_q^s |Y_t|\,\mathrm{d}t$, since $T-q \le1$, and Gronwall’s inequality gives

The next lemma computes the first two derivatives of one Cole–Hopf step. It is the only place where the form of $\mathcal{T}_{m,s}$ enters the continuity estimates.

Lemma 2.1. Let $f \in\mathrm{C}^2(\mathbb{R})$ be bounded above with $f''$ bounded, let $m \ge0$ and $s > 0$, and put $g = \mathcal{T}_{m,s}f$. For $x \in\mathbb{R}$ let $P_x$ be the law of $Y = x + \sqrt{s}G$ tilted by $e^{mf(Y)}$, that is, $\mathrm{d}P_x/\mathrm{d}\operatorname{Law}(Y) = e^{mf(Y)}/\mathbb{E}e^{mf(Y)}$. Then $g \in\mathrm{C}^2(\mathbb{R})$, $g \le\sup f$, and

Consequently the following hold.

(a) If $|f'| \le L$ on $\mathbb{R}$, then $|g'| \le L$ and $|g''| \le\sup|f''| + mL^2$.

(b) If $f$ is concave and $f'' \ge-K$ on $\mathbb{R}$, then $g$ is concave and $g'' \ge-K$.

Proof. Since $f'$ has at most linear growth, $f''$ is bounded, and $e^{mf} \le e^{m\sup f}$, the function $e^{mf}$ and its first two derivatives $mf'e^{mf}$ and $(mf'' + m^2f'^2)e^{mf}$, evaluated at $x + \sqrt{s}G$, are dominated by $C(1+|G|)^2$ for $x$ in a compact set, where $C$ depends on the compact set. Hence we may differentiate $\mathbb{E}e^{mf(x+\sqrt{s}G)}$ twice under the expectation, and dividing by $\mathbb{E}e^{mf(Y)}$ gives (29) for $m > 0$. For $m = 0$ we have $P_x = \operatorname{Law}(Y)$, and the same argument applied to $\mathbb{E}f(x+\sqrt{s}G)$ gives $g' = \mathbb{E}f'(Y)$ and $g'' = \mathbb{E}f''(Y)$. Part (a) follows from (29) and $\operatorname{Var}_{P_x}f'(Y) \le L^2$. In part (b), $g'' \ge-K$ by (29). If $m = 0$, then $g$ is an average of concave functions. If $m > 0$, put $h(x,y) = \exp\{mf(y) - (y-x)^2/(2s)\}$. Its logarithm is concave in $(x,y)$, and $\int_{\mathbb{R}} h(x,y)\,\mathrm{d}y = \sqrt{2\pi s}\,e^{mg(x)}$. By Prékopa’s theorem [ref-108] (Theorem 6), the marginal of a log-concave function on $\mathbb{R}^2$ is log-concave, so $mg$ is concave. $\square$

For the hard-wall terminal datum, the formula for $g''$ in (29) appears in [ref-54] (4.17). Part II, Lemma A.1 proves the same formulas for concave nondecreasing $f$; here part (a) also covers data that are neither concave nor monotone, as in class (A). As in Part II, Lemma A.1, part (b) gives concavity one step at a time. Hence Lemma 2.2(i) holds for all step functions with values in $[0,1]$, whereas Part II, Lemma A.2 and [ref-54] proof of (4.6) assume the coefficient nondecreasing.

The next lemma compares the solutions for two step coefficients with the same terminal datum, at a general terminal time $T$. Part II, Lemma A.2 proves the identity (2.15) for a class of concave nondecreasing terminal data that contains the hard-wall datum.

Lemma 2.2. Let $T \in(0,1]$ and let $F \in C^{2}(\mathbb{R})$ be bounded above with $F''$ bounded. Let $\gamma,\nu:[0,T)\to[0,1]$ be step functions, let $u_{\gamma}=u_{\gamma}(\cdot,\cdot;F)$ and $u_{\nu}=u_{\nu}(\cdot,\cdot;F)$ be given by (17) with terminal datum $F$ at time $T$, and put $w=u_{\gamma}-u_{\nu}$. Assume one of the following.

(A) $|F'|\leq L$ on $\mathbb{R}$.

(B) $F$ is concave and $F''\geq-K$ on $\mathbb{R}$ for a constant $K\geq0$. In this case put $A=\sup F-F(0)+|F'(0)|+K$.

Then the following hold for all $(q,x)\in[0,T]\times\mathbb{R}$.

(i) In case (A), $u_{\gamma}(q,\cdot)\in C^{2}(\mathbb{R})$ and $|\partial_{x}u_{\gamma}|\leq L$. In case (B), $u_{\gamma}(q,\cdot)\in C^{2}(\mathbb{R})$ and

The same holds for $u_{\nu}$.

(ii) Let $B$ be a standard Brownian motion and let $Y$ solve $\mathrm{d}Y_{s}=\beta(s,Y_{s})\,\mathrm{d}s+\mathrm{d}B_{s}$ on $[q,T]$ with $Y_{q}=x$, where $\beta=\frac{\gamma}{2}(\partial_{x}u_{\gamma}+\partial_{x}u_{\nu})$. This equation has a unique strong solution, and

(iii) In case (A), $|w(q,x)|\leq\frac{L^{2}}{2}\int_{q}^{T}|\gamma-\nu|\,\mathrm{d}s$. In case (B),

Proof. Refining the partitions, we may assume that $\gamma$ and $\nu$ are constant on the intervals $[t_{j},t_{j+1})$ of a common partition $0=t_{0}<\cdots<t_{n+1}=T$. We divide the proof into four steps.

Step 1. Proof of (i). For $q\in[t_{j},t_{j+1})$, $u_{\gamma}(q,\cdot)$ is $\mathcal{T}_{\gamma(t_{j}),t_{j+1}-q}$ applied to $u_{\gamma}(t_{j+1},\cdot)$. Induction over the intervals, starting from the last, and Lemma 2.1 give the following: in case (A), $u_{\gamma}(q,\cdot)\in C^{2}$, $|\partial_{x}u_{\gamma}|\leq L$, and $|\partial_{xx}u_{\gamma}|\leq\sup|F''|+(n+1)L^{2}$ on $[0,T]\times\mathbb{R}$; in case (B), $u_{\gamma}(q,\cdot)\in C^{2}$ is concave with $\partial_{xx}u_{\gamma}\geq-K$. In case (B), $u_{\gamma}\leq\sup F$, and (21) together with $F(y)\geq F(0)+F'(0)y-\frac{K}{2}y^{2}$ gives

which is the asserted lower bound, since $T-q\leq1$. For $x\in\{-1,0,1\}$ the right side is at least $F(0)-|F'(0)|-K$. By concavity,

so $|\partial_x u_\gamma(q,0)| \le\sup F-F(0)+|F'(0)|+K=A$, and $|\partial_{xx}u_\gamma| \le K$ gives $|\partial_xu_\gamma(q,x)| \le A+K|x|$.

Step 2. The equation for $w$. By §2.1, $u_\gamma$ and $u_\nu$ are continuous on $[0,T]\times\mathbb{R}$, and on each open interval $(t_j,t_{j+1})$ they are smooth and solve (2.8) with the constant coefficients $\gamma$ and $\nu$. Subtracting the two equations and using

we obtain on each $(t_j,t_{j+1})\times\mathbb{R}$

By Step 1, $\beta$ is Lipschitz in $y$ uniformly in $s$, and $|\beta(s,y)|\le a+K'|y|$ with $(a,K')=(L,0)$ in case (A) and $(a,K')=(A,K)$ in case (B). It is Borel, since $\partial_xu_\gamma$ and $\partial_xu_\nu$ are pointwise limits of difference quotients of continuous functions. Hence the equation for $Y$ has a unique strong solution [ref-79], and (2.13) gives $\mathbb{E}\sup_{s\in[q,T]}Y_s^2<\infty$. Moreover $|\partial_xw(s,y)|\le2(a+K'|y|)$ by (i), and $|w(s,y)|\le C(1+y^2)$ for a constant $C$, because $u_\gamma$ and $u_\nu$ lie between the lower bound in (2.6) and $\sup F$ (in case (A), $F(y)\ge F(0)-L|y|$).

Step 3. Itô’s formula and the passage through the breakpoints. Fix $j$ with $t_{j+1}>q$, put $t'_j=\max\{q,t_j\}$, and let $t'_j<s_1<s_2<t_{j+1}$. On $[s_1,s_2]\times\mathbb{R}$ the function $w$ is smooth, so Itô’s formula applies to $w(s,Y_s)$ on $[s_1,s_2]$. The stochastic integral $\int_{s_1}^{s_2}\partial_xw(s,Y_s)\,\mathrm{d}B_s$ has mean zero, because $\mathbb{E}\int_{s_1}^{s_2}(\partial_xw)^2(s,Y_s)\,\mathrm{d}s\le4\mathbb{E}(a+K'\sup_s|Y_s|)^2<\infty$. Taking expectations and using (31),

Now let $s_1\downarrow t'_j$ and $s_2\uparrow t_{j+1}$. The function $w$ is continuous on $[0,T]\times\mathbb{R}$ and $Y$ has continuous paths. Since $|w|\le C(1+y^2)$ and $(\partial_xu_\nu)^2(s,y)\le(a+K'|y|)^2$ are dominated along $Y$ by $C'(1+\sup_sY_s^2)$, which is integrable, dominated convergence gives the same identity with $s_1=t'_j$ and $s_2=t_{j+1}$. Summing over $j$, and using $Y_q=x$ and $w(T,\cdot)=0$, gives (2.15).

Step 4. Proof of (iii). In case (A), $(\partial_xu_\nu)^2\le L^2$ in (2.15). In case (B), $(\partial_xu_\nu)^2(s,y)\le2A^2+2K^2y^2$ by (i). By (2.13) with $(a,K')=(A,K)$ and $\mathbb{E}(B^*)^2\le4$,

so

and (2.15) gives the bound.

The identity (30) is the Feynman–Kac formula of Jagannath and Tobasco [ref-76], Lemma 14, and part (iii) is the analogue of Guerra’s bound (see the beginning of this chapter). Case (B) covers concave terminal data whose derivative grows linearly, such as the hard-wall datum and the activations in class (B).

For activations in class (A), the Lipschitz estimate holds in the uniform norm and at every time; this is the form used in the upper bound.

Lemma 2.3. Let $U$ belong to class (A) and put $L=\lVert U'\rVert_{\infty}$. Then for all $q\in[0,1]$ and all distribution functions $\gamma_{1},\gamma_{2}$ of probability measures on $[0,1]$, in particular for all $\gamma_{1},\gamma_{2}\in\mathcal{U}$,

If moreover $\gamma_{1},\gamma_{2}\in\mathcal{U}_{\bar q}$ for some $\bar q<1$, then

Proof. For step functions, (32) is Lemma 2.2(iii) in case (A), with $T=1$ and $F=U$. For general $\gamma_{1},\gamma_{2}$, let $\gamma_{i,n}$ be step approximations as in §2.1. By (32) for step functions, $(u_{\gamma_{i,n}}(q,\cdot;U))_{n}$ is Cauchy in the uniform norm, and two approximating sequences of $\gamma_{i}$ have the same limit, since they can be interleaved. This proves that $u_{\gamma_{i}}$ is well defined, and (32) passes to the limit. Finally, (33) is $\alpha$ times (32) at $(q,x)=(0,0)$ plus (27) with $r=\bar q$. $\square$

The next lemma gives the Lipschitz estimate for both classes at the origin, with an explicit constant.

Lemma 2.4. Let $U$ belong to class (A) or class (B). If $U$ belongs to class (A), put $C_{U}^{\mathrm{A}}=L^{2}/2$ with $L=\lVert U'\rVert_{\infty}$. If $U$ belongs to class (B), put $C_{U}^{\mathrm{B}}=C_{A,K}$, the constant of Lemma 2.2(iii) with

Let $C_{U}$ be the smallest of the numbers $C_{U}^{\mathrm{A}},C_{U}^{\mathrm{B}}$ that are defined. Then, for any two probability measures $\gamma,\nu$ on $[0,1]$,

Consequently, for $r\in[0,1)$ and $\gamma,\nu\in\mathcal{U}_{r}$,

Proof. If $U$ belongs to class (A), (34) with the constant $C_U^{\mathrm{A}}$ is (32) at $(q,x)=(0,0)$. Let $U$ belong to class (B). Then $F=U$ satisfies case (B) of Lemma 2.2 with $T=1$ and the constants $K$ and $A$ above. For step functions $\gamma,\nu$ with values in $[0,1]$, Lemma 2.2(iii) gives

For general $\gamma$, approximate it by step functions as in §2.1. By this estimate the values at $(q,x)$ form a Cauchy sequence, locally uniformly in $(q,x)$, and the limit does not depend on the approximating sequence. This defines $u_{\gamma}(q,x;U)$ (for $U$ in both classes it agrees with the limit in Lemma 2.3), and the estimate passes to the limit. At $(q,x)=(0,0)$ it gives (34) with the constant $C_U^{\mathrm{B}}$. Finally, (35) follows from (34), the identity $W_1(\gamma,\nu)=\|\gamma-\nu\|_{L^1}$ and (27). $\square$

The hard-wall functional at a finite cutoff

We now study $\mathcal{P}_{\kappa}(\gamma)$ as a deterministic functional of the order parameter. The top $Q_{\gamma}$ of the support is not continuous for the $L^1$ distance, since adding a small mass near one moves it. We therefore work on the classes $\mathcal{U}_r$, $0\le r<1$, and evaluate every $\gamma\in\mathcal{U}_r$ with the common terminal time $r$. All constants below depend on $r$ and on a compact interval $I$ containing $\kappa$, but not on the number of steps of a step order parameter. None of the results concerns the limit $r\to1$. Part II, §§2.1–2.5 and Appendix A, which do not use Part I, prove the results of this section other than the martingale identity (43), in a stronger form with constants that are uniform for $|\kappa|\le K_0$, and we derive them from there. Part II defines the hard-wall solution for step order parameters by the same recursion, with the terminal datum $f_r$ defined below, and for general order parameters by the same approximation within $\mathcal{U}_r$ (Part II, Definition 2.2(iii)), so the two definitions agree.

Gaussian quantities. We use the functions

Lemma 2.5. The following statements hold.

(i) For $t>0$,

(ii) $R'=-V$ and $\bar{R}'=1-V$.

(iii) For $z\in\mathbb{R}$ let $T_z$ have the law of $z+G$ conditioned on the event $\{z+G\ge0\}$, that is, the law on $[0,\infty)$ with density proportional to $e^{zt-t^{2}/2}$. Then $\mathbb{E}T_z=\bar{R}(z)$ and $\operatorname{Var}(T_z)=1-V(z)$. Consequently $\bar{R}>0$ and $0<V<1$.

Proof. The functions $R$, $\bar R$, $V$, and the variable $T_Z$ are those of Part II, §2.1. Part (i) is Part II, Lemma 2.1(b), and parts (ii) and (iii) are contained in Part II, Lemma 2.1(a). $\square$

Part (i) consists of Gordon’s bounds for the Mills ratio [ref-65], and the bound $V<1$ in part (iii) is due to Sampford [ref-113]. El Alaoui and Sellke collect these facts, in terms of the inverse Mills ratio, in [ref-54].

The terminal datum. Fix $r\in[0,1)$. For $\gamma\in\mathcal U_r$, the hard-wall solution $u_\gamma(\cdot,\cdot;\kappa)$ on $[0,r]\times\mathbb R$ is given by the recursion (17) with terminal time $r$ and terminal datum

by (5) with $\bar q=r$; for step order parameters this is the definition, and for general $\gamma\in\mathcal U_r$ it is completed in Lemma 2.7. Using the same terminal time for all of $\mathcal U_r$ avoids the discontinuity of $\gamma\mapsto Q_\gamma$. With $z=(x-\kappa)/\sqrt{1-r}$, Lemma 2.5 gives

Every derivative of $f_r$ has polynomial growth, uniformly for $\kappa$ in a compact interval; this gives the case $r=0$ of Lemma 2.6(v). Indeed, $R'=-zR-R^2$, so every derivative of $R$ is a polynomial in $z$ and $R$, and $0<R(z)\le R(0)+|z|$ because $-1<R'<0$ by Lemma 2.5(ii),(iii). Throughout the rest of this section, $I$ is a compact interval, $\kappa\in I$, $K=(1-r)^{-1}$, and constants depend only on $r$ and $I$ unless stated otherwise.

Notation. The following abbreviations are used in this section and in Chapter 6. We fix $\kappa\in\mathbb R$ and $\gamma\in\mathcal U$, and write

Thus $\lambda(q)\ge\lambda(Q)=\delta>0$ for $q\in[0,Q]$, and $u$ is defined on $[0,1)\times\mathbb R$ by (26). We write

for the generator of the diffusion

where $B$ is a standard Brownian motion. For $\gamma\in\mathcal U_r$ we consider $X$ on $[0,r]$, and we write $X^{s,x}$ for the solution of the equation in (37) on $[s,r]$ with $X_s^{s,x}=x$; it exists and is unique by Lemma 2.6(iii) for step order parameters and by Lemma 2.7 in general. On $[0,Q]$, $X$ is the Parisi diffusion (8) of §1.1, and the processes defined with two admissible values of $r$ agree on their common interval. It is also the diffusion of Part II, (2.14). When two order parameters $\gamma,\widetilde{\gamma}$ are compared, we indicate the order parameter as a subscript, as in $u_{\widetilde{\gamma}},b_{\gamma}$, or $X^{\widetilde{\gamma}}$. With this notation, the hard-wall functional (1.6) with cutoff $\bar q=Q$ reads

the second form by (2.9), and $\mathcal{P}_{*}(\kappa)=\inf_{\gamma\in\mathcal{U}}\mathcal{P}_{\kappa}(\gamma)$.

A priori bounds and continuity. The next lemma collects the bounds on the solution and the diffusion that justify Itô’s formula and the Feynman–Kac formula in the rest of this part.

Lemma 2.6. Fix $r\in[0,1)$ and a compact interval $I$. There is a constant $C$, and for each $p\geq1$ and $k\geq1$ there are constants $C_p$ and $C'_k$, depending only on $r$, $I$, and $p$ or $k$, such that for every $\kappa\in I$ and every step order parameter $\gamma\in\mathcal{U}_r$ the following hold on $[0,r]\times\mathbb{R}$.

(i) $-C(1+x^2)\leq u(q,x)\leq0$.

(ii) On each open interval where $\gamma$ is constant, $u$ is smooth and

(iii) $0\leq c\leq(1-r)^{-1}$ and $|b(q,x)|\leq C(1+|x|)$. In particular the drift $\gamma b$ in (2.22) is Lipschitz in $x$ with constant $(1-r)^{-1}$ and has linear growth, so the equation in (2.22) has a unique strong solution $X^{s,x}$ from every starting point $(s,x)\in[0,r]\times\mathbb{R}$.

(iv) $\mathbb{E}\sup_{t\in[s,r]}|X_t^{s,x}|^p\leq C_p(1+|x|)^p$; in particular $\mathbb{E}\sup_{q\leq r}|X_q|^p\leq C_p$.

(v) $|\partial_x^k u(q,x)|\leq C'_k(1+|x|)^{C'_k}$, and $\partial_x^k u$ is continuous on $[0,r]\times\mathbb{R}$.

Proof. For $r=0$ the statements concern $u=f_0$ only and follow from (2.21) and the polynomial growth of the derivatives of $f_0$ noted after it; let $r>0$. Every statement is contained in Part II, Chapter 2, applied with $K_0=\max_{\kappa\in I}|\kappa|$; the constants there depend only on $r,K_0$, and $p$ or $k$. Part (i) and the bound on $b$ in (iii) are Part II, Lemma 2.4(b). By Part II, Lemma 2.4(c), $0<c<1/\lambda\leq(1-r)^{-1}$ on $[0,r]\times\mathbb{R}$, since $\lambda\geq\lambda(r)=1-r$ there. Hence $|\partial_x(\gamma b)|=\gamma c\leq(1-r)^{-1}$, and the drift $\gamma b$ has linear growth. Strong existence and uniqueness for (2.22) are stated in Part II after (2.14). Part (ii) is Part II, Lemma 2.4(a) and Lemma 2.9(a). Part (iv) is Part II, Lemma 2.6(a), since $1+|x|^p\leq(1+|x|)^p$ for $p\geq1$. Part (v) is Part II, Lemma 2.4(a),(b): the bound there is $C_k(1+|x|^{p_k})\leq2C_k(1+|x|)^{p_k}$, so one may take $C'_k=\max\{2C_k,p_k\}$.

Parts (i) and (iii) correspond to the bounds [ref-54], (4.4)–(4.6), and part (v) is a weaker form of [ref-54], (4.7), with polynomial growth in place of boundedness.

The next lemma is the hard-wall counterpart of Lemma 2.4. It extends the solution, its derivatives, and the diffusion to general order parameters, and it is the source of the compactness and continuity used in Chapter 6. For mixed $p$-spin models, the convergence of all spatial derivatives of the solution as the order parameter converges is [ref-8], Proposition 1.

Lemma 2.7. Fix $r \in[0,1)$ and a compact interval $I$. There is a constant $C$ depending only on $r$ and $I$, and for each $p \ge1$ a constant $C_p$ depending only on $r$, $I$, and $p$, such that for all $\kappa\in I$ and all step order parameters $\gamma,\widetilde{\gamma} \in\mathcal{U}_r$, with $\varepsilon=\lVert\widetilde{\gamma}-\gamma\rVert_{L^1}$;

on $[0,r]\times\mathbb{R}$. For every $k \ge2$ and $A_0>0$,

uniformly in $\kappa\in I$ and in the step order parameters. When $X^{\widetilde{\gamma}}$ and $X^\gamma$ start from the same point $(s,x)\in[0,r]\times\mathbb{R}$ and are driven by the same Brownian motion,

Consequently the solution $u$, its spatial derivatives, and the diffusion $X$ extend from step order parameters to every $\gamma\in\mathcal{U}_r$ by $L^1$ approximation within $\mathcal{U}_r$, and the bounds of Lemma 2.6(i), (iii)–(v) and of this lemma hold for all $\gamma,\widetilde{\gamma}\in\mathcal{U}_r$. If $\gamma\in\mathcal{U}_r$ equals a constant $m$ on an interval $[a,a')\subset[0,r]$, then $u(q,\cdot)=T_{m,a'-q}u(a',\cdot)$ for $q\in[a,a')$, and $u$ is smooth and satisfies $\mathcal{L}b=0$ and $\mathcal{L}c=\gamma c^2$ on $(a,a')\times\mathbb{R}$. Finally,

In particular $\mathcal{P}_\kappa$ is continuous on the compact set $\mathcal{U}_r$, uniformly for $\kappa\in I$, and attains its minimum over $\mathcal{U}_r$.

Proof. For $r=0$ the class $\mathcal{U}_0$ has one element; let $r>0$. We apply the results of Part II, Chapter 2 with $K_0=\max_{\kappa\in I}|\kappa|$; they hold for all $\gamma,\widetilde{\gamma}\in\mathcal{U}_r$, and their constants depend only on $r$, $K_0$, and $p$ or $k$. The solution for general $\gamma\in\mathcal{U}_r$ is the limit of the solutions for step approximations in $\mathcal{U}_r$ (Part II, Definition 2.2(iii)). By Part II, Lemma 2.4(f), for every $k\ge0$,

with $p_0=2$ and $p_1=1$. The cases $k=0,1$ give (39), the cases $k\ge2$ give the convergence of the higher derivatives, and the derivatives of $u_\gamma$ are the limits of those of its step approximations. The bound (40) is Part II, Lemma 2.6(b); in particular the diffusions of step approximations converge to $X^\gamma$. Part II, Lemma 2.4(a)–(c) and Lemma 2.6(a) hold for general $\gamma\in\mathcal{U}_r$, so the proof of Lemma 2.6 gives its parts (i) and (iii)–(v) for all $\gamma\in\mathcal{U}_{r}$. If $\gamma=m$ on $[a,a')$, then $u(q,\cdot)=\mathcal{T}_{m,a'-q}u(a',\cdot)$ for $q\in[a,a')$ by Part II, Lemma 2.5(a), and $u$ is smooth and satisfies $\mathcal{L}b=0$ and $\mathcal{L}c=\gamma c^{2}$ on $(a,a')\times\mathbb{R}$ by Part II, Lemma 2.4(a) and Lemma 2.9(a). Finally, the first bound in (39) at $(0,0)$ controls the energy term of $\mathcal{P}_{\kappa}$, and (27) controls the entropy term; this proves (41). Compactness of $\mathcal{U}_{r}$ was shown in §2.1. $\square$

We record a consequence of Lemma 2.7. If $\gamma\in\mathcal{U}_{r}\subset\mathcal{U}_{r'}$ with $r<r'<1$, then the step approximations of $\gamma$ in $\mathcal{U}_{r}$ also lie in $\mathcal{U}_{r'}$, and for them the solutions computed with the cutoffs $r$ and $r'$ agree on $[0,r]$ by the discussion before (25). Passing to the limit, $u_{\gamma}(\cdot,\cdot\,;\kappa)$ and $\mathcal{P}_{\kappa}(\gamma)$ do not depend on the cutoff for any $\gamma\in\mathcal{U}$, and (26) holds for every $\gamma\in\mathcal{U}$. This is also Part II, Lemma 2.3.

We next record the signs of $b$, $c$, and $1-v$, which sharpen Lemma 2.6(iii) to strict inequalities, and the martingale structure along the diffusion. They are the starting point of the analysis in Chapter 6.

Lemma 2.8. Let $r\in[0,1)$, $\kappa\in\mathbb{R}$, and $\gamma\in\mathcal{U}_{r}$. Then, on $[0,r]\times\mathbb{R}$,

Moreover, for $q\in[0,r]$,

in particular $b(q,X_{q})$ is a square-integrable martingale on $[0,r]$.

Proof. The signs (42) are Part II, Lemma 2.4(b),(c), applied with $K_{0}=|\kappa|$; the last inequality holds because $\lambda\geq\lambda(r)=1-r$ on $[0,r]$. It remains to prove (43). For $r=0$ there is nothing to prove; let $r>0$ and take $I=\{\kappa\}$.

Step order parameters. Let $\gamma$ be a step order parameter and fix $(s,x)\in[0,r]\times\mathbb{R}$. By Itô’s formula on each open interval of constancy and $\mathcal{L}b=0$, $\mathrm{d}b(t,X_{t}^{s,x})=\partial_{x}b(t,X_{t}^{s,x})\,\mathrm{d}B_{t}=-c(t,X_{t}^{s,x})\,\mathrm{d}B_{t}$ there. Since $b$ is continuous on $[0,r]\times\mathbb{R}$ with linear growth and $0\leq c\leq K$ by Lemma 2.6(iii),(v), the identity extends to the endpoints of the intervals as in Step 3 of the proof of Lemma 2.2, and

With $(s,x)=(0,0)$ this is (43), and since $0\leq c\leq K$ the stochastic integral is a square-integrable martingale on $[0,r]$.

General order parameters. Let $\gamma\in\mathcal{U}_{r}$, let $\gamma_{n}\in\mathcal{U}_{r}$ be step approximations, and write $X^{n}=X^{\gamma_{n}}$, $b_{n}=b_{\gamma_{n}}$, and $c_{n}=c_{\gamma_{n}}$. By Lemma 2.7, $c_{n}\to c$ uniformly on compact subsets of $[0,r] \times\mathbb{R}$, and $X^{n} \to X$ in every moment, uniformly on $[0,r]$. By (39) and the Lipschitz bound $|\partial_{x}b| \le K$,

with $\varepsilon_{n}=\|\gamma_{n}-\gamma\|_{L^{1}}$, so $b_{n}(q,X_{q}^{n}) \to b(q,X_{q})$ in $L^{2}$. Moreover $c_{n}(s,X_{s}^{n}) \to c(s,X_{s})$ in probability for each $s$, by the locally uniform convergence of $c_{n}$, the continuity of $c$, and the convergence of $X^{n}$; since $0 \le c_{n},c \le K$, dominated convergence and Itô’s isometry give $\int_{0}^{q}c_{n}(s,X_{s}^{n})\,\mathrm{d}B_{s} \to\int_{0}^{q}c(s,X_{s})\,\mathrm{d}B_{s}$ in $L^{2}$. The identity (43) for $\gamma_{n}$ therefore passes to the limit, almost surely for each fixed $q \in[0,r]$. Both sides of (43) have continuous paths, since $b$ and $X$ are continuous and the stochastic integral has a continuous version, so the identity holds simultaneously for all rational $q$ on one event of probability one, and then for all $q \in[0,r]$ by continuity. $\square$

The martingale identity (43) is the hard-wall form of [ref-7], Lemma 2. Part II, Lemma 2.9(b) states the martingale property of $b(q,X_{q})$ without the stochastic integral.

The first variation. For $\gamma\in\mathcal{U}_{r}$ and $q \in[0,r]$ set

The first term of $D$ comes from the energy term of $\mathcal{P}_{\kappa}$ and the second from the entropy term, as the proof of Part II, Proposition 2.11(a) shows. Since $v=\lambda c$, these are the functions $D$ and $S$ of Part II, (37). The function $D$ and the first variation below are the basic tools of Chapter 6.

Lemma 2.9. Let $r \in[0,1)$, $\kappa\in\mathbb{R}$, and $\gamma,\widetilde{\gamma} \in\mathcal{U}_{r}$. The function $D$ is continuously differentiable on $[0,r]$ with

Moreover, with $\gamma_{\theta}=\gamma+\theta(\widetilde{\gamma}-\gamma) \in\mathcal{U}_{r}$ for $\theta\in[0,1]$,

Proof. The formula for $D'$ is among the generator identities of Part II, Lemma 2.9(c)(i), and (45) is the first equality in Part II, (43) (Proposition 2.11(a)); both hold for all $\gamma,\widetilde{\gamma} \in\mathcal{U}_{r}$ and $\kappa\in\mathbb{R}$. $\square$

Formula (45) is the hard-wall analogue of the directional derivative of the Parisi functional of mixed $p$-spin models computed by W.-K. Chen [ref-38], Theorem 2; see also [ref-55], Proposition 6.8. The first variation of El Alaoui and Sellke [ref-54], (4.15) involves the same function $D$; the beginning of this chapter compares the two. The formula for $D'$ is the analogue of [ref-8] (Proposition 3), where Auffinger and W.-K. Chen differentiate the corresponding function for mixed $p$-spin models. Talagrand computed the derivative of the Parisi functional when the order parameter is transported by a map close to the identity [ref-126].

The upper bound

This chapter proves that $\mathcal{P}_{U}(\gamma)$ bounds the free energy from above for every order parameter $\gamma$ and every activation $U$ in class (A) (Theorem 3.1), and deduces upper bounds for the feasible volume and for the maximal margin. Lemma 3.2 is a change of measure on a Ruelle probability cascade with marks, which turns Gibbs averages over the leaves into expectations along a Markov chain, and Corollary 3.3 deduces that conditional overlaps are nondecreasing in the branching level. Lemma 3.4 computes a Gaussian cascade exactly. Lemma 3.5 gives the derivatives of the interpolating free energy. Lemma 3.6 proves concentration of the free energy and computes its derivatives in the perturbation parameters, and Lemmas 3.7 and 3.8 use it to control the conditional covariance of the two overlaps. Lemma 3.9 bounds the spherical term by the Crisanti–Sommers entropy. Proposition 3.10 combines these facts through a maximum principle in the interpolation time. Corollary 3.11 passes to the hard wall, with a bounded source term, and Lemma 3.13 and Corollary 3.14, with Lemma 3.12, bound the maximal margin.

Step 1 of §1.3 outlines the argument and introduces the notation used below. The proof follows that of [ref-92], Theorem 4.1, a step in Mourrat’s proof that the Hamilton–Jacobi limit bounds the free energy of bipartite models from above [ref-92], Theorem 1.1. As there, the atoms of the order parameter have equal mass, the derivative in $t$ is a sum of products of conditional overlap means and a conditional covariance (see [ref-92], (2.23)), and Mourrat’s perturbation enforces the Ghirlanda–Guerra identities only at contact points (§§3.3 and 3.4). Four things differ. The row side carries a non-Gaussian activation $U$, with no sign condition on $U'$, in place of Mourrat’s bilinear Gaussian energy. Only the spin side has free cascade levels, so the viscosity argument becomes a maximum principle in $t$ for the maximum over $p$ (§3.7). The monotonicity of the response overlaps $\zeta_{l}$ follows from a change of measure on a marked cascade; its Gaussian analogue is the monotonicity of the normalized derivatives of the enriched free energy in the cascade parameters [ref-92], Lemma 2.4 (see also [ref-128], Proposition 14.3.2). The bound closes on the Parisi functional $\mathcal{P}_{U}(\gamma_{h})$ instead of the solution of a Hamilton–Jacobi equation.

Throughout this chapter, $b_{a}=g_{a}/\sqrt{N}$ for the rows $g_{a}$ of $\mathbf{G}$, $\alpha_{N}=M/N\to\alpha$, and $\alpha'=\sup_{N}\alpha_{N}<\infty$. For $U$ in class (A) we write $U_{\max}=\sup U$ and $L=\lVert U'\rVert_{\infty}$.

Theorem 3.1. Let $U$ belong to class (A). For every $\gamma\in\mathcal{U}$,

Moreover, $\mathbb{P}\{f_N(U)>\mathcal{P}_U(\gamma)+\varepsilon\}\to0$ for every $\varepsilon>0$.

Chapter 5 combines Theorem 3.1 with the lower bound of Chapter 4.

A change of measure on a marked cascade

The interpolation places Gaussian fields on the nodes of a Ruelle probability cascade. Its free energy is an iterated logarithmic moment over the levels of the tree, and to differentiate it we need to express averages over the cascade weights as expectations along a Markov chain on the marks. The following change-of-measure identity does this. It also gives the law of the branching level of two leaves sampled from the weights, and it shows that conditional overlaps are nondecreasing in the level.

The cascade. Fix $k\ge0$ and $0<\theta_1<\cdots<\theta_k<1$, and set $\theta_0=0$, $\theta_{k+1}=1$, and $w_l=\theta_{l+1}-\theta_l$ for $0\le l\le k$. The nodes of depth $l$ are the strings $\beta\in\mathbb{N}_*^l$, with the empty string as the root, and the leaves are the nodes of depth $k$. For a leaf $\beta$ we write $\beta|l$ for its ancestor at depth $l$, and for two leaves we put $\beta\wedge\beta'=\max\{l:\beta|l=\beta'|l\}$. At every node of depth $l-1$, independently, place a Poisson point process on $(0,\infty)$ with intensity measure $\theta_lx^{-1-\theta_l}\,\mathrm{d}x$. Its points, in decreasing order, are attached to the edges from that node to its children; $\pi_\beta$ denotes the point attached to the edge into the node $\beta$. For a leaf $\beta$ put

The weights $(v_\beta)$ are the Ruelle probability cascade [ref-112]. For $k=0$ the root is the only leaf, and $\Pi=v=1$.

Independently of the cascade, attach to each node $\beta$ of depth $l\ge1$ a random mark $e_\beta$ with law $P_l$ on a standard Borel space; marks at different nodes are independent. The root mark $e_0$ is fixed for now; below it contains the disorder shared by all leaves. For a leaf $\beta$ write $e_{[\beta]}=(e_0,e_{\beta|1},\ldots,e_{\beta|k})$ for the marks along its path. Let $X_k(e_0,\ldots,e_k)$ be a measurable function of a path of marks, and define backwards, for $l=k,\ldots,1$,

We assume that every $X_l$ is finite on every path. Then each $K_l$ is a probability kernel. We write $\mathbb{E}_K$ for expectation over the Markov chain $(e_0,e_1,\ldots,e_k)$ started from $e_0$ with the kernels $K_1,\ldots,K_k$, and, for $0\le l\le k$, $\mathbb{E}_{K,l}$ for expectation over two paths $(e,e')$ that coincide up to depth $l$, follow the chain up to depth $l$, and then continue independently with the kernels $K_{l+1},\ldots,K_k$.

Lemma 3.2. The quantities $\Pi$ and $W=\sum_{\beta}v_{\beta}e^{X_{k}(e_{[\beta]})}$ are positive and finite almost surely. Let $\omega_{\beta}=v_{\beta}e^{X_{k}(e_{[\beta]})}/W$ be the associated Gibbs weights on the leaves. Then, for every measurable $\varphi$ that is nonnegative, or integrable under the law on the right side,

Moreover, conditionally on every value of the root mark $e_{0}$,

In words, (48) says that the path of marks of a leaf sampled from the Gibbs weights is distributed as the chain with the kernels $K_{l}$, and (49) says that two independently sampled leaves branch at depth $l$ with probability $w_{l}$, whatever the function $X_{k}$, and that their paths then share the first $l$ marks and continue independently.

These are known properties of Ruelle probability cascades, and they rest on the invariance property of Bolthausen and Sznitman [ref-19]; see [ref-98], Chapter 2. With marks and the kernels $K_{l}$, (48) and (49) correspond to [ref-103], Theorems 6 and 7, and (50) to [ref-103], (3.12)–(3.13); Panchenko and Talagrand use them to write Guerra’s interpolation on a cascade. See also [ref-4]. Steps 1 and 3 of the proof follow [ref-103], Lemmas 2 and 3, and Step 4 follows Mourrat’s derivation of the law of $\beta\wedge\beta'$ by Gaussian integration by parts [ref-93], which adapts the derivation of [ref-98], (2.82).

Proof. For $k=0$ we have $W=e^{X_{0}(e_{0})}$, $\omega=1$, and $w_{0}=1$, and all assertions hold with $\Pi=\Pi'=1$. Let $k\ge1$. We divide the proof into four steps.

Step 1. A Poisson computation. Let $(x_{n})$ be a Poisson process with intensity $\theta x^{-1-\theta}\,\mathrm{d}x$, let $(e_{n})$ be independent marks with law $P$, and let $f$ be measurable with $c=\int e^{\theta f}\,\mathrm{d}P<\infty$. By the mapping and marking theorems for Poisson processes [ref-81], Chapters 2 and 5, the pairs $(x_{n}e^{f(e_{n})},e_{n})$ form a Poisson process with intensity

Indeed, for a test function $\chi$ the substitution $y=xe^{f(e)}$ gives

After the first coordinate is multiplied by $c^{-1/\theta}$, the first factor becomes $\theta y^{-1-\theta}\,\mathrm{d}y$ again. Since the intensity is a product, after ranking the first coordinates the marks are independent with law $c^{-1}e^{\theta f}P$ and independent of the ranked points. If each mark also carries an independent subtree and $f$ depends only on the first component of the mark, the conditional law of the subtree given the first component is unchanged.

Step 2. The totals. The sum $S_{\theta}$ of the points of one process satisfies $\mathbb{E}e^{-uS_{\theta}}=\exp\{-\Gamma(1-\theta)u^{\theta}\}$ for $u\geq0$, so it is positive and finite almost surely. The identities

applied to $x=S_{\theta}$ and integrated, show that $S_{\theta}$ has negative moments of all orders and positive moments of all orders $a<\theta$; the second integral converges at $u=0$ because $1-e^{-\Gamma(1-\theta)u^{\theta}}$ is of order $u^{\theta}$. In particular $\mathbb{E}|\log S_{\theta}|^{2}<\infty$. Next, $\Pi$ is a deterministic multiple of a copy of $S_{\theta_{1}}$. If $S_{n}$ denotes the total of the subtree below the $n$-th child of the root, then $\Pi=\sum_{n}x_{n}S_{n}$, and by Step 1 with $f=\log S_{n}$ the points $x_{n}S_{n}$ form a Poisson process with intensity $\mathbb{E}[S^{\theta_{1}}]\,\theta_{1}x^{-1-\theta_{1}}\,\mathrm{d}x$. By induction on $k$, the subtree total $S$ is a multiple of a copy of $S_{\theta_{2}}$, so $\mathbb{E}S^{\theta_{1}}<\infty$ because $\theta_{1}<\theta_{2}$. This proves the assertions about $\Pi$.

Step 3. The change of measure. For a node $\beta$ of depth $l\geq1$ with parent $\beta^{-}$, replace the edge weight by

where $e_{[\beta]}$ denotes the marks along the path from the root to $\beta$. Given the ancestors of $\beta$, the multiplier depends only on the mark $e_{\beta}$, and $\int e^{\theta_{l}(X_{l}-X_{l-1})}\,\mathrm{d}P_{l}=1$ by (47). Apply Step 1 at the root with $c=1$, carrying the independent subtree of each child as part of its mark, and then apply it recursively below each child. By induction on the depth, after ranking at each node the modified weights $\widetilde{\pi}$ form a cascade with the same law as the original one and independent of the marks, while the mark of a child of depth $l$ has conditional law $K_{l}$ given its ancestors. The relabeling by ranks preserves the ancestry. The multipliers telescope along a path:

Summing over $\beta$ gives $W=e^{X_{0}(e_{0})}\Pi'/\Pi$ with $\Pi'=\sum_{\beta}\prod_{l}\widetilde{\pi}_{\beta|l}$, which has the law of $\Pi$. This is (50), including the positivity and finiteness of $W$. Moreover, $\omega_{\beta}=\prod_{l}\widetilde{\pi}_{\beta|l}/\Pi'$ are the weights of the relabeled cascade, which is independent of the marks. This proves (48). It also proves (49) with the coefficient $\bar{w}_{l}=\mathbb{E}\sum_{\beta\wedge\beta'=l}v_{\beta}v_{\beta'}$ in place of $w_{l}$, because, given the relabeled cascade, the marks along the paths to two leaves with $\beta\wedge\beta'=l$ share the first $l$ marks and then continue independently.

Step 4. The branching probabilities. The coefficient $\bar{w}_{l}$ does not depend on the marks, so we may compute it with a convenient choice. Fix $1 \le l \le k$ and $s>0$, let the mark of each node $\nu$ of depth $l$ be a standard Gaussian variable $G_{\nu}$ (the other marks play no role), and take $X_{k}(e_{[\beta]})=sG_{\beta|_{l}}$. Then $X_{m}=sG_{\beta|_{l}}$ for $m \ge l$ and $X_{l-1}=\theta_{l}s^{2}/2$, so $K_{l}$ is the law $N(\theta_{l}s,1)$, and (3.3) gives

The sum over nodes is absolutely integrable, by (3.3) applied to the path function $|G_{\beta|_{l}}|$. Conditionally on all other variables, $\omega_{\nu}=Ae^{sG_{\nu}}/(Ae^{sG_{\nu}}+B)$ with $A,B$ independent of $G_{\nu}$, so Gaussian integration by parts gives $\mathbb{E}[G_{\nu}\omega_{\nu}]=s\,\mathbb{E}[\omega_{\nu}(1-\omega_{\nu})]$. Summing over $\nu$ and using $\sum_{\nu}\omega_{\nu}=1$ gives $\mathbb{E}\sum_{\nu}\omega_{\nu}^{2}=1-\theta_{l}$. By (3.4) with the coefficients $\bar{w}_{m}$ and $\varphi=1$, the left side equals $\sum_{m\ge l}\bar{w}_{m}$. Hence $\sum_{m\ge l}\bar{w}_{m}=1-\theta_{l}$ for $1\le l\le k$, and also for $l=0$ since $\sum_{m}\bar{w}_{m}=1$. Taking differences gives $\bar{w}_{l}=\theta_{l+1}-\theta_{l}=w_{l}$. $\square$

We write $\langle\cdot\rangle$ for averages under the Gibbs weights and their independent replicas, that is, leaves $\beta^{1},\beta^{2},\ldots$ sampled independently from $(\omega_{\beta})$. Conditional averages given $\beta^{1}\wedge\beta^{2}=l$ are taken with respect to the probability $\mathbb{E}\langle\cdot\,\mathbf{1}_{\{\beta^{1}\wedge\beta^{2}=l\}}\rangle/w_{l}$.

Corollary 3.3. Let $f$ be a bounded function of a path of marks with values in a Euclidean space, and let $F_{l}=\mathbb{E}_{K}[f\mid e_{0},\ldots,e_{l}]$. Then

and the right side is nonnegative and nondecreasing in $l$. Moreover,

Proof. In (3.4), condition on the common part of the two paths. Their continuations are independent, so the conditional expectation of $f(e)\cdot f(e')$ is $\lVert F_{l}\rVert^{2}$. The sequence $(F_{l})$ is a martingale for the chain, so $\mathbb{E}_{K}\lVert F_{l}\rVert^{2}$ is nondecreasing by conditional Jensen. For (51), take conditional expectations in (3.5), using $\Pi'\stackrel{d}{=}\Pi$ given $e_{0}$. For the variance, center both logarithms at $\mathbb{E}\log\Pi$ and use $(x-y)^{2}\le2x^{2}+2y^{2}$; independence of $\Pi$ and $\Pi'$ is not needed. $\square$

In Lemma 3.5(iv) we apply the monotonicity in $l$ to the response overlap of the activation.

Gaussian cascades

Both bounds on the free energy use one explicit computation: a Gaussian field on a cascade, integrated against a Gaussian weight at each leaf. In this chapter it bounds the spherical term (Lemma 3.9); in Chapter 4 it evaluates the cavity term. The useful observation is that the field along a sampled path, divided by its precision, is a Gaussian martingale. Its variances give every overlap of the added coordinates, and the normalization of their squared norm identifies the value with a Crisanti–Sommers entropy.

Let $\mathcal{K}=\{p\in\mathbb{R}^{k+1}:0\leq p_{0}\leq\cdots\leq p_{k}\}$, and put $\Delta p_{0}=p_{0}$ and $\Delta p_{l}=p_{l}-p_{l-1}$ for $1\leq l\leq k$. For $x\in\mathcal{K}$ with $x_{k}<1$ let

$\gamma_{x}\in\mathcal{U}$ is the order parameter with atoms of mass $w_{l}$ at $x_{l}$.

Fix $n\geq1$ and $p\in\mathcal{K}$. Let $(y_{\nu})$ be independent standard Gaussian vectors in $\mathbb{R}^{n}$ indexed by all nodes $\nu$ of the tree, including the root, independent of the cascade, and put

for a leaf $\beta$. The coordinates of $Y$ are independent, and each has covariance $p_{\beta\wedge\beta'}$ between the leaves $\beta$ and $\beta'$. We call $Y$ the Gaussian cascade field with levels $p$. Summation by parts gives

For $b>\sum_{l=1}^{k}\theta_{l}\Delta p_{l}$ define, for $0\leq l\leq k$,

and put $\varrho=m_{k}+1/b$. Since $d_{l}-d_{l-1}=\theta_{l}\Delta p_{l}\geq0$ and $d_{0}=b-p_{k}+\sum_{l}w_{l}p_{l}>0$, we have $0<d_{0}\leq\cdots\leq d_{k}=b$ and $0\leq m_{0}\leq\cdots\leq m_{k}$. We call $d_{l}$ the precision at depth $l$. The top precision $b$ is a scalar and is unrelated to the rows $b_{a}$. Let $\nu_{n}$ be the standard Gaussian measure on $\mathbb{R}^{n}$, and let $\mathcal{M}$ be the random probability measure on pairs $(\beta,\varepsilon)$ of a leaf and a vector of $\mathbb{R}^{n}$ given by

We write $\mathbb{E}_{\mathcal{M}}$ and $\mathbb{P}_{\mathcal{M}}$ for expectation and probability over the cascade, the field $Y$, and independent samples $(\beta^{1},\varepsilon^{1}),(\beta^{2},\varepsilon^{2}),\ldots$ from $\mathcal{M}$, and we put $J=\beta^{1}\wedge\beta^{2}$. Conditional expectations given $J=l$ are taken with respect to $\mathbb{E}_{\mathcal{M}}[\cdot\,\mathbf{1}_{\{J=l\}}]/\mathbb{P}_{\mathcal{M}}(J=l)$.

Lemma 3.4. In this setting the following hold. (i) The normalization of $\mathcal{M}$ is finite, and

(ii) Under $\mathbb{P}_{\mathcal{M}}$, the process $M_{l}=Y_{\beta^{1},l}/d_{l}$, $0\leq l\leq k$, is a centered Gaussian martingale with independent increments and

and conditionally on $\beta^{1}$ and $Y$ the vector $\varepsilon^{1}$ has the law $N(M_{k},b^{-1}I_{n})$. In particular,

(iii) The levels satisfy the duality identity

(iv) The order parameter $\gamma_{m/\varrho}$, with atoms of mass $w_{l}$ at $m_{l}/\varrho$, belongs to $\mathcal{U}$, its largest atom is $m_{k}/\varrho=1-1/(\varrho b)$, and

If $\varrho=1$, then moreover, for $0\leq l\leq k$,

and $d_{l}\geq1$, $1\leq b\leq1+p_{k}$, and $m_{k}\leq1-(1+p_{k})^{-1}$.

(v) If $Y^{c}$ is a real Gaussian cascade field with levels $c\in\mathcal{K}$ (the case $n=1$), then

By (49), the branching probabilities $w_l$ do not depend on the function integrated at the leaves. In particular, they remain $w_l$ when the $\varepsilon$-integral in the definition of $\mathcal{M}$ is restricted to a ball. The restricted measure, used in §4.5, is defined for every $b$, including the case $d_0 \le0$ in which $\mathcal{M}$ is not defined. In (60), $\varrho$ is the mean of $\lVert\varepsilon^1 \rVert^2/n$ under $\mathbb{P}_{\mathcal{M}}$, by (57); when it equals one, the value (55) is exactly the Crisanti–Sommers entropy of the law of the conditional overlaps $m_J$.

The value $\mathcal{G}(b,p)$, together with the term $(b-1-\log b)/2$, is the spherical part of the expression minimized over $b$ in the Parisi formula for spherical models of Talagrand [ref-125] and W.-K. Chen [ref-37]. Parts (iii) and (iv) identify it with a Crisanti–Sommers entropy, in line with Talagrand’s theorem that the spherical Parisi formula agrees with the Crisanti–Sommers formula [ref-44, ref-125]. Part (v) is the linear case computed by Arguin [ref-4]; see also [ref-100] (22).

Proof. We divide the proof into five steps. Steps 1, 2, and 5 apply Lemma 3.2 to the Gaussian increments; Steps 3 and 4 are algebraic.

Step 1. The recursion. We use the marks $e_0=\sqrt{p_0}\,y_{\varnothing}$ at the root and $e_\nu=\sqrt{\Delta p_l}\,y_\nu$ at a node $\nu$ of depth $l\ge1$, with laws $P_l=N(0,\Delta p_l I_n)$, and write $x_l=e_0+e_1+\cdots+e_l$ for the partial sums along a path, so that $Y_{\beta,l}=x_l$ along the path of $\beta$. Since $\int\nu_n(\mathrm{d}\varepsilon)e^{\varepsilon\cdot x-(b-1)\lVert\varepsilon\rVert^2/2}=b^{-n/2}e^{\lVert x\rVert^2/(2b)}$ for $b>0$, the leaf function of (55) is $X_k=\lVert x_k\rVert^2/(2b)-\frac{n}{2}\log b$, and that of (54) is $\lVert x_k\rVert^2/(2b)$. For one coordinate, $\theta\in(0,1)$, $\Delta\ge0$, $d>\theta\Delta$, and a standard Gaussian variable $G$, completing the square gives

Since $d_l-\theta_l\Delta p_l=d_{l-1}>0$, backward induction in (47) shows that $X_l(e_0,\ldots,e_l)=\lVert x_l\rVert^2/(2d_l)+a_l$ with $a_{l-1}=a_l+\frac{n}{2\theta_l}\log(d_l/d_{l-1})$, where $a_k=-\frac{n}{2}\log b$ for (55) and $a_k=0$ for (54). All values are finite, so Lemma 3.2 applies. By (50), $n$ times the left side of (55) equals $\mathbb{E}X_0(e_0)=np_0/(2d_0)+a_0=n[\mathcal{G}(b,p)-\frac{1}{2}\log b]$, and the same computation with $a_k=0$ gives (54). This proves (i).

Step 2. The law of a sampled path. Under $\mathcal{M}$ the leaf $\beta^1$ is sampled from the Gibbs weights of Lemma 3.2 with the leaf function $X_k$ of (55), and, given the leaf and $Y$, the vector $\varepsilon^1$ has density proportional to $e^{\varepsilon\cdot Y_{\beta^1}-b\lVert\varepsilon\rVert^2/2}$ with respect to Lebesgue measure, that is, the law $N(Y_{\beta^1}/b,b^{-1}I_n)$. By (47), the kernel $K_l(\mathrm{d}e_l\mid e_0,\ldots,e_{l-1})$ has density proportional to $\exp\{\theta_l\lVert x_{l-1}+e_l\rVert^2/(2d_l)-\lVert e_l\rVert^2/(2\Delta p_l)\}$. The coefficient of $-\lVert e_l\rVert^2/2$ in the exponent is $1/\Delta p_l-\theta_l/d_l=d_{l-1}/(d_l\Delta p_l)$, so $K_l$ is the Gaussian law with mean $\theta_l\Delta p_l x_{l-1}/d_{l-1}$ and covariance $(\Delta p_l d_l/d_{l-1})I_n$ (the point mass at $0$ if $\Delta p_l=0$). Since $d_{l-1}+\theta_l\Delta p_l=d_l$, under the chain

with independent standard Gaussian vectors $\xi_l$ that are independent of the root mark. The root mark keeps its law $N(0,p_0I_n)$, so $x_0/d_0\sim N(0,(p_0/d_0^2)I_n)$. By (48), averaged over the root mark, the process $M_l=x_l/d_l$ along the path of $\beta^1$ has this law, which is (56) by the definition of $m_l$. Since $Y_{\beta^1}/b=M_k$, the vector $\varepsilon^1$ is $M_k+b^{-1/2}\xi'$ with an independent standard Gaussian vector $\xi'$, which gives (57). By (49), $J=l$ with probability $w_l$, and the two paths then share $x_0,\ldots,x_l$ and continue independently with the same kernels. Given $x_l$, the vectors $M_k^1$ and $M_k^2$ of the two paths are therefore independent, each with conditional mean $M_l=x_l/d_l$, and $\mathbb{E}_M[\varepsilon^1\cdot\varepsilon^2\mid J=l]=\mathbb{E}\lVert M_l\rVert^2=nm_l$. All functions involved are polynomials in Gaussian vectors, so the integrability required in Lemma 3.2 holds. This proves (ii).

Step 3. The duality identity. Let $\lambda_\varrho(s)=\varrho-\sum_l w_l\max(m_l,s)$ for $s\in[0,m_k]$. This function is nonincreasing, with $\lambda_\varrho'(s)=-\sum_{l:m_l<s}w_l$ for $s\notin\{m_l\}$; it is constant on $[0,m_0]$ and affine with slope $-\theta_j$ on $[m_{j-1},m_j]$. Moreover $\lambda_\varrho(m_k)=\varrho-m_k=1/b=1/d_k$, and

since $d_j-d_{j-1}=\theta_j\Delta p_j$. By downward induction, $\lambda_\varrho(m_j)=1/d_j$ for all $j$; in particular $\lambda_\varrho\ge1/b>0$ on $[0,m_k]$. On $[0,m_0]$ we have $\lambda_\varrho=1/d_0$, and on $[m_{j-1},m_j]$ we have $(1/\lambda_\varrho)'=\theta_j/\lambda_\varrho^2$ and $(\log\lambda_\varrho)'=-\theta_j/\lambda_\varrho$. Hence the constant piece contributes $m_0d_0^2=p_0$ and $m_0d_0=p_0/d_0$, and the $j$-th affine piece contributes $(d_j-d_{j-1})/\theta_j=\Delta p_j$ and $\theta_j^{-1}\log(d_j/d_{j-1})$, to the two integrals

(If $m_{j-1}=m_j$, then $\Delta p_j=0$ and $d_j=d_{j-1}$, and both contributions vanish.) Insert the first identity of (64) into the left side of (59) and use Fubini’s theorem:

The numerator equals $\lambda_\varrho(s)+s\sum_{l:m_l\le s}w_l=\lambda_\varrho(s)-s\lambda_\varrho'(s)$ for almost every $s$, so the integrand is $(s/\lambda_\varrho(s))'$, and the integral is $m_k/\lambda_\varrho(m_k)=bm_k=b\varrho-1$. This proves (59) and (iii).

Step 4. The entropy. Suppose first that $\varrho=1$. Then $m_k=1-1/b<1$, $\gamma_m\in\mathcal{U}$, and $\lambda_\varrho=\lambda_{\gamma_m}$ on $[0,m_k]$. Step 3 gives the first two identities in (61), and (59) becomes (62). By (64) and the definition of $\operatorname{CS}$ with the cutoff $m_k\in[Q_{\gamma_m},1)$,

Moreover $1/d_l=\lambda_{\gamma_m}(m_l)\leq\lambda_{\gamma_m}(0)\leq1$, so $b=d_k\geq d_l\geq1$; since $m_l,p_l\geq0$, (62) gives $b-1\leq p_k$, and then $m_k=1-1/b\leq1-(1+p_k)^{-1}$. For general $\varrho>0$, multiply the levels and the top precision by $\varrho$. The parameters $(\varrho b,\varrho p)$ satisfy the standing condition, and by (53) they have precisions $\varrho d_l$, conditional levels $m_l/\varrho$, and the same value $\mathcal{G}(\varrho b,\varrho p)=\mathcal{G}(b,p)$, because $p_0/d_0$ and the ratios $d_j/d_{j-1}$ do not change. Their top level satisfies $m_k/\varrho+1/(\varrho b)=1$. The case $\varrho=1$, applied to them, shows that $\gamma_m/\varrho\in\mathcal{U}$ with largest atom $m_k/\varrho=1-1/(\varrho b)$, and

This proves (iv).

Step 5. The linear field. Apply Lemma 3.2 with the marks of Step 1 for the levels $c$ (with $n=1$) and the leaf function $X_k=x_k$. For $\theta\in(0,1)$ and $\Delta\geq0$, $\theta^{-1}\log\mathbb{E}e^{\theta(x+\sqrt{\Delta}G)}=x+\theta\Delta/2$, so $X_l=x_l+\frac{1}{2}\sum_{j>l}\theta_j\Delta c_j$ and $\mathbb{E}X_0(e_0)=\frac{1}{2}\sum_{j=1}^{k}\theta_j\Delta c_j$. By (50) and (52) for $c$, this is (63). For $k=0$ both sides of (63) vanish. $\square$

The interpolation and its derivatives

We now define the interpolating free energy and compute its derivatives. Throughout this section the activation $U$ is in class (A). From now on we fix an integer $K\geq2$ and use the cascade of §3.1 with

We keep writing $w_l$, which makes the role of each factor visible. We also fix the levels of the row field,

so that $\gamma_h$ is the order parameter with atoms of mass $w_l$ at $h_l$. The spin field has levels $p\in\mathcal{K}$. At every node $\beta$ of the cascade, including the root, take independent standard Gaussian vectors $y_\beta\in\mathbb{R}^N$ and $z_\beta\in\mathbb{R}^M$, independent of $G$, and set, for a leaf $\beta$,

where $\Delta h_0=h_0$ and $\Delta h_l=h_l-h_{l-1}$. Thus $Y$ and $\Xi$ are the Gaussian cascade fields with levels $p$ and $h$ in dimensions $N$ and $M$. The remaining variance $1-h_k$ of the row field is carried by an auxiliary standard Gaussian vector $\xi\in\mathbb{R}^M$ inside each leaf, so that each coordinate of $\Xi_\beta+\sqrt{1-h_k}\,\xi$ has variance one, as does $b_a\cdot\sigma$ for fixed $\sigma\in S_N$. This equality of variances is what makes the diagonal terms cancel when the activation is differentiated in $t$. Without the cascade, interpolations of this form were used for the perceptron by Talagrand [ref-127].

The perturbation. To obtain approximate Ghirlanda–Guerra identities we add a small Gaussian field whose covariance is a polynomial in the spin overlap and in the branching level. Fix an enumeration $(\vartheta_n)_{n \ge1}$ of $\mathbb{Q} \cap[0,1]$ and an integer $j_{+} \ge1$, and let $\mathcal{I} = \{1,\ldots,j_{+}\}^{3}$. For $j=(j_1,j_2,j_3) \in\mathcal{I}$ put

We use centered Gaussian fields $H^j(\sigma,\beta)$, $j \in\mathcal{I}$, indexed by $\sigma\in S_N$ and leaves $\beta$, independent of each other and of all previous variables, with covariance

This perturbation, together with the prefactor $s_N=N^{-1/16}$ used below, is the one of Mourrat [ref-92]. It has the form of Panchenko’s perturbation for multi-species models [ref-100] (see [ref-98] for one species), with the branching level $J/k$ in place of the overlap of a second species. Panchenko averages over random coefficients in the perturbation; here, as in [ref-92], the perturbation coefficients are chosen at a contact point (§3.4). Mourrat proves that such a field exists in [ref-92]; the construction below places its Gaussian tensors on the nodes of the cascade, so that they become part of the marks. For integrability we use the following explicit construction. For $m \ge1$ let $a_{m,0}=0$ and $a_{m,l}=(l/k)^m-((l-1)/k)^m$ for $1 \le l \le k$, and let $a_{0,0}=1$ and $a_{0,l}=0$ for $l \ge1$. Thus $a_{m,l} \ge0$ and $\sum_{l \le J}a_{m,l}=(J/k)^m$ for all $m \ge0$ and $0 \le J \le k$. Attach to each node $\beta$ independent standard Gaussian tensors $g_{\beta}^{j,i} \in(\mathbb{R}^N)^{\otimes i}$, $0 \le i \le j_3$, and set

Only the nodes $\beta|l$ with $l \le\beta\wedge\beta'$ are shared by two leaves, so the covariance is

which is (65) by the binomial theorem. Since $a_{m,0}=0$ unless $m=0$, only the tensor of order $i=j_3$ has a root component, with $c_{j,j_3,0}^{2}=N\vartheta_{j_1}^{j_3} \le N$.

The interpolating free energy. Let $s_N=N^{-1/16}$ and $x=(x_j)_{j \in\mathcal{I}} \in\mathbb{R}^{\mathcal{I}}$. A configuration is a triple $(\sigma,\xi,\beta)$ with $\sigma\in S_N$, $\xi\in\mathbb{R}^M$, and $\beta$ a leaf, with prior $\mu_N(\mathrm{d}\sigma)\,\mathrm{N}(0,I_M)(\mathrm{d}\xi)\,v_\beta$. For $t \in[0,1]$, $p \in K$, and $x \in\mathbb{R}^{\mathcal{I}}$ the Hamiltonian is

Let $W=\sum_{\beta}v_{\beta}\iint e^{\mathscr{H}(\sigma,\xi,\beta)}\,\mu_{N}(\mathrm{d}\sigma)\,\mathrm{N}(0,I_{M})(\mathrm{d}\xi)$ be the partition function, and put

The function $\psi_{\sigma,N}$ is the spherical part of $u_{N}$ at $t=0$ (see (70)), and the subtraction of $p_{k}/2$ normalizes it: $\psi_{\sigma,N}(0)=0$. This is a marked cascade in the sense of §3.1. The root mark consists of $G$, $y_{\varnothing}$, $z_{\varnothing}$, and the root tensors; the mark of a node $\beta$ of depth $l\geq1$ consists of $y_{\beta}$, $z_{\beta}$, and the tensors $g_{\beta}^{j,i}$; and $X_{k}=\log\iint e^{\mathscr{H}}\,\mathrm{d}\mu_{N}\,\mathrm{d}\mathrm{N}(0,I_{M})$ is the logarithm of the leaf integral. From now on $\langle\cdot\rangle$ denotes the Gibbs measure on configurations, proportional to $v_{\beta}e^{\mathscr{H}}$ times the product prior, together with its independent replicas $(\sigma^{l},\xi^{l},\beta^{l})$. We write

where $S_{a}^{l}$ is $S_{a}$ evaluated at the $l$-th replica, and

By Lemma 3.2, $\mathbb{E}\langle\mathbf{1}_{\{J=l\}}\rangle=w_{l}$ whatever the parameters, and conditional averages given $J=l$ are as in §3.1. For random variables $A,B$ of two replicas we write $\operatorname{Cov}(A,B\mid J=l)=\mathbb{E}\langle AB\mid J=l\rangle-\mathbb{E}\langle A\mid J=l\rangle\mathbb{E}\langle B\mid J=l\rangle$.

The next lemma collects what the comparison uses: continuity, Guerra’s formula for the derivative in $t$, the right derivatives in $p$ in the directions $\mathbf{1}_{\geq m}$, and the monotonicity of $\zeta_{l}$. For $0\leq m\leq k$ let $\mathbf{1}_{\geq m}\in\mathcal{K}$ be the vector with coordinates $\mathbf{1}_{\{l\geq m\}}$; moving $p$ in this direction increases $\Delta p_{m}$ and leaves the other increments unchanged.

Lemma 3.5. The following hold.

(i) The function $u_{N}$ is continuous on $[0,1]\times\mathcal{K}\times\mathbb{R}^{I}$, and so is $\mathbb{E}\langle f(R_{12},J_{12})\rangle$ for every bounded measurable function $f$ on $[-1,1]\times\{0,\ldots,k\}$.

(ii) For $0<t<1$, $u_{N}$ is differentiable in $t$, and

(iii) For every $(t,p,x)$ and $0\leq m\leq k$, the function $\epsilon\mapsto u_{N}(t,p+\epsilon\mathbf{1}_{\geq m},x)$ on $[0,\infty)$ has right derivative at $\epsilon=0$ equal to

(iv) The conditional response overlaps are ordered:

Proof. Given its leaf $\beta$, a replica $(\sigma,\xi)$ has the leaf measure, proportional to $\mathrm{e}^{\mathcal{H}(\sigma,\xi,\beta)}$ times the prior $\mu_{N}\otimes N(0,I_{M})$. It depends only on the path of marks $e_{[\beta]}$, and we write $\langle\cdot\rangle_{\beta}$ for averages under it. Given the leaves, replicas are independent. We divide the proof into six steps.

Step 1. Integrability. For a node $\beta$ of depth $m$ let $\Theta_{m}(e_{\beta})$ be the sum of the Euclidean norms of $y_{\beta}$, $z_{\beta}$, and all tensors $g_{\beta}^{j,i}$; at the root add the Frobenius norm $\lVert\mathbf{G}\rVert_{F}$. On any compact set of parameters there is $C$, depending also on $N$, such that

For $l=k$, the upper bound follows from $U\le U_{\max}$, $|Y_{\beta}\cdot\sigma|\le\sqrt{N}|Y_{\beta}|$, and $|\langle g,\sigma^{\otimes i}\rangle|\le N^{i/2}\lVert g\rVert$, which hold uniformly in $(\sigma,\xi)$. For the lower bound, apply Jensen’s inequality to the leaf integral and use $U(y)\ge U(0)-L|y|$, $\int|b_{a}\cdot\sigma|\,\mathrm{d}\mu_{N}\le|b_{a}|$ (since $\int\sigma\sigma^{\mathsf{T}}\,\mathrm{d}\mu_{N}=I_{N}$), $\int|\xi_{a}|\,\mathrm{d}N(0,I_{M})\le1$, $\int Y_{\beta}\cdot\sigma\,\mathrm{d}\mu_{N}=0$, and the bound on the tensors. Backward induction in (47) gives (69) at every level: Jensen’s inequality gives the lower bound, and the upper bound uses that each $\Theta_{m}$ has finite exponential moments of all orders. In particular the hypotheses of Lemma 3.2 hold, and each kernel density $\mathrm{e}^{\theta_{l}(X_{l}-X_{l-1})}$ is at most $\exp\{C(1+\sum_{m\le l}\Theta_{m})\}$.

Step 2. Continuity. Fix the marks. The leaf integral is continuous in $(t,p,x)$ by dominated convergence, since its integrand is continuous in the parameters and bounded, uniformly in $(\sigma,\xi)$ and on compact parameter sets, by $\exp\{C(1+\sum_{m}\Theta_{m})\}$. Backward induction in (47), with dominated convergence justified by (69), shows that every $X_{l}$ and every kernel density is continuous in the parameters. Since $\mathbb{E}\log W=\mathbb{E}X_{0}(e_{0})$ by (50) and $|X_{0}|\le C(1+\Theta_{0})$, $u_{N}$ is continuous. For the second claim in (i), let $\varphi(e,e')$ be the average of $f(\sigma\cdot\sigma'/N,l)$ under the product of the leaf measures of two paths $e,e'$. It is bounded by $\sup|f|$ and continuous in the parameters. By (49), applied for a fixed root mark, $\mathbb{E}\langle f(R_{12},J)\mathbf{1}_{\{J=l\}}\rangle=w_{l}\mathbb{E}\mathbb{E}_{K,l}\varphi$, and $\mathbb{E}_{K,l}$ integrates $\varphi$ against a product of kernel densities, each continuous and bounded by $\exp\{C(1+\sum_{m}\Theta_{m})\}$. Dominated convergence proves (i).

Step 3. Differentiation under the recursion. Let $\partial$ denote the derivative in one of the parameters $t\in(0,1)$ and $\Delta p_{m}$, the latter in the range $\Delta p_{m}>0$. Locally uniformly in the parameters, $|\partial\mathcal{H}|\le C(1+|\xi|+\sum_{m}\Theta_{m})$. Hence $\partial X_{k}=\langle\partial\mathcal{H}\rangle_{\beta}$, and, bounding $\langle|\xi|\rangle_{\beta}\le\mathrm{e}^{\sup\mathcal{H}-X_{k}}\int|\xi|\,\mathrm{d}N(0,I_{M})$ with (69), we get $|\partial X_{k}|\le\exp\{C(1+\sum_{m}\Theta_{m})\}$. Differentiating (47) gives $\partial X_{l-1}=\int\partial X_{l}\,\mathrm{d}K_{l}$, and the same kind of bound propagates to every level, which justifies each differentiation under the integral. Hence $\partial X_{0}=\mathbb{E}_{K}\partial X_{k}$, and by (50) and (48)

Step 4. Gaussian integration by parts. Each Gaussian coordinate at a node of depth $m$ enters the leaf integrals of all leaves below that node, and only those. Fix one such coordinate and let it vary in a compact set. Since $|U'|\leq L$ and $|\sigma|=\sqrt{N}$, the derivative of each descendant Hamiltonian in this coordinate is bounded, uniformly over the spin and auxiliary variables. Thus the perturbed descendant leaf integrals are bounded above and below by common finite multiples of their original values, their first derivatives satisfy the same domination, and so do their second derivatives, by the bound on $U''$. The series over leaves defining the Gibbs averages therefore converge locally uniformly together with their derivatives, and by (48) and (69) the sum over nodes of the expected absolute contributions is finite. This justifies Gaussian integration by parts node by node. Summing over nodes and coordinates gives, for every row $a$ and every $0\leq m\leq k$,

Here $\beta$ is the leaf of the sampled configuration. In the first line, the derivative of $\langle\sigma_{i}\mathbf{1}_{\{\beta|m=\nu\}}\rangle$ in the coordinate $y_{\nu,i}$ produces the one-replica term $\sqrt{\Delta p_{m}}\langle\sigma_{i}^{2}\mathbf{1}_{\{\beta|m=\nu\}}\rangle$ and the two-replica term $-\sqrt{\Delta p_{m}}\langle\sigma_{i}^{1}\sigma_{i}^{2}\mathbf{1}_{\{\beta^{1}|m=\beta^{2}|m=\nu\}}\rangle$; summing over $i$ and over the nodes $\nu$ of depth $m$ gives the formula, because $|\sigma|^{2}=N$ and $\mathbf{1}_{\{\beta^{1}|m=\beta^{2}|m\}}=\mathbf{1}_{\{J\geq m\}}$. The second and fourth lines are obtained in the same way; there the factor $U''+(U')^{2}$ comes from differentiating $U'(S_{a})e^{U(S_{a})}$. The variable $\zeta$ is integrated inside the leaf measure, so in the third line only the one-replica term appears.

Step 5. The derivative in $t$. For $0<t<1$, $\partial_{t}\mathcal{H}=\sum_{a}U'(S_{a})\partial_{t}S_{a}$ with

The last three identities of Step 4, together with $\sum_{m}\Delta h_{m}=h_{k}$ and $\sum_{m}\Delta h_{m}\mathbf{1}_{\{J\geq m\}}=h_{J}$, give for each row $a$

with $U''+(U')^{2}$ evaluated at $S_{a}$. The one-replica terms cancel because $b_{a}\cdot\sigma$ and $\Xi_{\beta,a}+\sqrt{1-h_{k}}\,\xi_{a}$ both have variance one. Summing over $a$ and dividing by $N$ gives the first identity in (66), by Step 3; the term $-p_{k}/2$ of $u_{N}$ does not depend on $t$. For the second, split the average according to the value of $J$ and use $\mathbb{E}\langle R_{12}\alpha_{N}X_{12}\mid J=l\rangle=r_{l}\zeta_{l}+\operatorname{Cov}(R_{12},\alpha_{N}X_{12}\mid J=l)$. This proves (ii).

Step 6. The derivatives in $p$ and the monotonicity. Moving $p$ along $\mathbf{1}_{\ge m}$ increases $\Delta p_{m}$ and $p_{k}$ at unit rate and leaves the other increments fixed. If $\Delta p_{m}>0$, then $\partial_{\Delta p_{m}}\mathcal{H}=y_{\beta\mid m}\cdot\sigma/(2\sqrt{\Delta p_{m}})$, and the first identity of Step 4 and Step 3 show that $\frac{1}{N}\mathbb{E}\log W$ has derivative $\frac{1}{2}(1-\mathbb{E}\langle R_{12}\mathbf{1}_{\{J\ge m\}}\rangle)$; subtracting $\frac{1}{2}$ for the term $-p_{k}/2$ gives (67). If $\Delta p_{m}=0$, this formula holds at $p+\epsilon\mathbf{1}_{\ge m}$ for every $\epsilon>0$, and by (i), applied with $f(R,l)=R\mathbf{1}_{\{l\ge m\}}$, both $u_{N}$ and the right side of (67) are continuous as $\epsilon\downarrow0$. The mean value theorem then gives the right derivative at $\epsilon=0$. Finally, $\mathbb{E}\langle R_{12}\mathbf{1}_{\{J\ge m\}}\rangle=\sum_{l\ge m}w_{l}r_{l}$ by the definition of $r_{l}$. This proves (iii).

For (iv), fix the root mark and apply Corollary 3.3 to the path function with coordinates $\langle U'(S_{a})\rangle_{\beta}/\sqrt{M}$, $a\le M$, whose norm is at most $L$. Since replicas are independent given their leaves, $\langle U'(S_{a}^{1})U'(S_{a}^{2})\rangle$ given the two leaves equals $\langle U'(S_{a})\rangle_{\beta^{1}}\langle U'(S_{a})\rangle_{\beta^{2}}$, so for fixed root mark $\mathbb{E}\langle X_{12}\mid J=l\rangle$ is the quantity $\mathbb{E}_{\mathsf{K}}\lVert F_{l}\rVert^{2}$ of that corollary. It is nonnegative, nondecreasing in $l$, and at most $L^{2}$. These properties survive averaging over the root mark, because $w_{l}$ does not depend on it. This proves (68). $\square$

The endpoints. At $t=1$, $p=0$, and $x=0$ the Hamiltonian is $\sum_{a}U(b_{a}\cdot\sigma)$, which depends neither on $\widetilde{\xi}$ nor on the leaf, so $W=Z_{N}(U)$. At $t=0$ and $x=0$ the leaf integral is the product of $\int e^{Y_{\beta}\cdot\sigma}\mu_{N}(\mathrm{d}\sigma)$, which depends only on the vectors $y$, and of

which depends only on the vectors $z$; these are independent components of the marks. For independent $F,F'$ we have $\theta^{-1}\log\mathbb{E}e^{\theta(F+F')}=\theta^{-1}\log\mathbb{E}e^{\theta F}+\theta^{-1}\log\mathbb{E}e^{\theta F'}$, so the recursion (47) factorizes into a spin part and a row part, and by (50) so does $\mathbb{E}\log W$. The spin part is $N\psi_{\sigma,N}(p)+Np_{k}/2$. The row part factorizes in the same way over the rows, whose coordinates $z_{\cdot,a}$ of the node vectors are independent. For one row it is the recursion (17) for the order parameter $\gamma_{h}$, evaluated at $(0,0)$: the auxiliary variable gives the step $\mathcal{T}_{1,1-h_{k}}$ over $[h_{k},1]$; the level $l\ge1$ gives the step $\mathcal{T}_{\theta_{l},\Delta h_{l}}$ over $[h_{l-1},h_{l})$, where $\gamma_{h}=\theta_{l}$; and the root gives the heat step $\mathcal{T}_{0,h_{0}}$ over $[0,h_{0})$, where $\gamma_{h}=0$. Levels with $\Delta h_{l}=0$ give empty intervals and the identity step $\mathcal{T}_{\theta_{l},0}$, so they can be omitted. Hence

The identification of the row part with the recursion (17) is the representation of the Parisi functional through Ruelle probability cascades (see [ref-98] Chapter 2 and [ref-4]).

Concentration and identities at a contact

The comparison argument evaluates the interpolation only at maximum points of a deterministic function of the parameters. At such points we need the Ghirlanda–Guerra identities for the perturbed Gibbs measure to hold approximately. They follow from concentration of the derivatives of $\log W$ in the perturbation parameters $x$. Since the points are deterministic, concentration at each fixed parameter suffices.

This section follows Steps 1–3 of the proof of [ref-92], Theorem 4.1. There a quadratic penalty around the point $(1,\ldots,1)$ [ref-92], (4.18)–(4.19) keeps the perturbation coefficients at the contact point close to one; the bound on the Hessian in $x$ at a contact point and the concentration of the free energy [ref-92], Proposition 4.2 control the fluctuations of the perturbation Hamiltonians; and Gaussian integration by parts gives approximate Ghirlanda–Guerra identities [ref-92], (4.28). Lemmas 3.6 and 3.7 carry out these steps for the present interpolation, and the penalty appears in the proof of Proposition 3.10.

Lemma 3.6. Fix $K$, $h$, and $\mathcal{I}$, and a compact subset of $[0,1]\times K\times\mathbb{R}^{\mathcal{I}}$. There is a constant $C$ such that

on this set. Moreover, almost surely $x\mapsto\log W$ is convex and $C^{2}$ on $\mathbb{R}^{\mathcal{I}}$, with

The function $u_N$ is convex and differentiable in $x$, with $N\partial_{x_j}u_N=s_N\mathbb{E}\langle H^j\rangle$, $|\partial_{x_j}u_N|\leq2^{j_++1}N^{-1/8}|x_j|$, and, with $C_{j_+}=2^{j_+}$,

Proof. We divide the proof into three steps.

Step 1. The variance. By the law of total variance, $\operatorname{Var}(\log W)=\mathbb{E}\operatorname{Var}(\log W\mid e_0)+\operatorname{Var}(\mathbb{E}[\log W\mid e_0])$. By (51) the first term is at most $4\operatorname{Var}(\log\Pi)$, a constant depending only on the cascade, and $\mathbb{E}[\log W\mid e_0]=X_0(e_0)$ is a function of the root Gaussian variables $\boldsymbol{G}$, $y_{\varnothing}$, $z_{\varnothing}$, and the root tensors. As in Steps 2 and 3 of the proof of Lemma 3.5, applied to these variables (the derivatives of $\mathcal{H}$ in them are bounded), $X_0$ is continuously differentiable in them and its gradient is the $\mathbb{E}_K$-average of the gradient of the leaf logarithm, so by Jensen’s inequality its squared norm is at most the $\mathbb{E}_K$-average of the squared norm of the leaf gradient. The derivative of the leaf logarithm in $g_{ai}$ is $\sqrt{t/N}\langle U'(S_a)\sigma_i\rangle_\beta$; by Jensen’s inequality and $|\sigma|^2=N$ the sum of its squares over $a,i$ is at most $(t/N)MNL^2=Nt\alpha_NL^2$. The derivative in $y_{\varnothing}$ is $\sqrt{p_0}\langle\sigma\rangle_\beta$, with squared norm at most $Np_0$. The derivative in $z_{\varnothing,a}$ is $\sqrt{(1-t)h_0}\langle U'(S_a)\rangle_\beta$, with squared norm summed over $a$ at most $N\alpha_{N}h_{0}L^{2}$. The root tensor of $H^{j}$ has coefficient $c_{j,j_{3},0}^{2}\leq N$, and the derivative of the leaf logarithm in it has squared norm at most $s_{N}^{2}x_{j}^{2}c_{j,j_{3},0}^{2}N^{-j_{3}}|\sigma|^{2j_{3}}\leq s_{N}^{2}Nx_{j}^{2}$. Altogether

and the Gaussian Poincaré inequality [ref-21] gives $\operatorname{Var}(X_{0})\leq\mathbb{E}|\nabla X_{0}|^{2}\leq CN$ on the compact set. Hence $\operatorname{Var}(N^{-1}\log W)\leq N^{-2}(4\operatorname{Var}\log\Pi+CN)\leq C/N$.

Step 2. The derivatives in $x$. Let $\bar{\Theta}_{\beta}=\sum_{j\in\mathcal{I}}\sum_{i,l}|c_{j,i,l}|\|g_{\beta|l}^{j,i}\|$, so that $\sum_{j}\sup_{\sigma}|H^{j}(\sigma,\beta)|\leq\bar{\Theta}_{\beta}$, and let $A_{\beta}(x)$ be the leaf integral. For each $x_{0}\in\mathbb{Q}^{\mathcal{I}}$ and each integer $n\geq1$,

Indeed, the sum equals $W(x_{0})$ times a Gibbs average whose expectation, by (48), is an $\mathbb{E}_{K}$-expectation of $(1+\bar{\Theta}+\bar{\Theta}^{2})e^{s_{N}n\bar{\Theta}}$, which is finite by the kernel density bound of Step 1 in the proof of Lemma 3.5. On the intersection of these countably many events, since $A_{\beta}(x)\leq A_{\beta}(x_{0})e^{s_{N}|x-x_{0}|_{1}\bar{\Theta}_{\beta}}$, the series defining $W(x)$ and the series of its first two derivatives converge locally uniformly in $x$. Termwise differentiation gives (71) and (72). More generally, the Hessian of $\log W$ in $x$ is $s_{N}^{2}$ times the covariance matrix of $(H^{j})_{j\in\mathcal{I}}$ under $\langle\cdot\rangle$, which is positive semidefinite, so $\log W$ is convex in $x$. For a convex function the difference quotients $(\log W(x)-\log W(x-\epsilon\mathbf{e}_{j}))/\epsilon$ and $(\log W(x+\epsilon\mathbf{e}_{j})-\log W(x))/\epsilon$, where $\mathbf{e}_{j}$ is the $j$-th coordinate vector of $\mathbb{R}^{\mathcal{I}}$, bound $\partial_{x_{j}}\log W$ from below and above, are integrable, and converge monotonically as $\epsilon\downarrow0$. Hence $\partial_{x_{j}}\mathbb{E}\log W=\mathbb{E}\partial_{x_{j}}\log W$, and $u_{N}$ is convex in $x$. A convex function on $\mathbb{R}^{\mathcal{I}}$ whose partial derivatives exist is differentiable, so $u_{N}$ is differentiable in $x$.

Step 3. Replica integration by parts. For a bounded measurable function $f$ of the overlap arrays $(R_{ll'},J_{ll'})_{l,l'\leq n}$ of $n$ replicas, Gaussian integration by parts in the tensors gives

where $H^{j}(1)=H^{j}(\sigma^{1},\beta^{1})$. To justify the exchange of the sum over nodes with the expectation, differentiate with respect to one tensor $g_{\nu}^{j,i}$ at one node $\nu$ of depth $m$. Each replica $l$ contributes the indicator that $\beta^{l}$ descends from $\nu$, and contracting the two tensors gives $N^{i}R_{1l}^{i}$. Summing the indicators over the nodes of depth $m$ gives $\mathbf{1}_{\{J_{1l}\geq m\}}$, and summing the squared coefficients over $i$ and $m$ gives the covariance (65). The expected absolute sum over nodes is bounded by a constant times $\mathbb{E}\langle\|g_{\beta|m}^{j,i}\|\rangle$, which is finite by (48) and (69). Taking $n=1$ and $f=1$ in (74), and using $|c_{j}|\leq2^{j_{3}}$, gives $|\mathbb{E}\langle H^{j}\rangle|\leq2^{j_{3}+1}s_{N}|x_{j}|N$, hence $|\partial_{x_{j}}u_{N}|=s_{N}N^{-1}|\mathbb{E}\langle H^{j}\rangle|\leq2^{j_{3}+1}N^{-1/8}|x_{j}|$. Integrating along the segment from $0$ to $x$ gives (73), since $j_{3}\leq j_{+}$. $\square$

For two families of overlap arrays $A^{ll'}, B^{ll'}$, a test function $f$ of the arrays of the first $n$ replicas, and a function $c$ of two real variables, the Ghirlanda–Guerra discrepancy is

The Ghirlanda–Guerra identities [ref-64] state that $\mathcal{D}_{n}(f,c)=0$. For $c=c_{j}$ this is the form used in [ref-92], (4.28) and (5.6), and it is the analogue, for two overlap arrays, of the multi-species discrepancy in [ref-100], (31).

Lemma 3.7. Fix a compact set of parameters $(t,p)\in[0,1]\times K$. There is a constant $C$ with the following property. Suppose that $(t,p)$ lies in this set, that $|x-\mathbf{1}|\leq1/2$, where $\mathbf{1}\in\mathbb{R}^{\mathcal{I}}$ has all coordinates equal to one, and that for every $y\in\mathbb{R}^{\mathcal{I}}$

where $u_{N}(x)$ abbreviates $u_{N}(t,p,x)$. Then for $A=R$ and $B=J/k$, every $j\in\mathcal{I}$, every $n\geq1$, and every measurable function $f$ of the overlap arrays of $n$ replicas with $|f|\leq1$,

Proof. We show that $H^{j}$ has small fluctuations, both under the Gibbs measure and over the disorder, at the scale relevant to (3.29). We divide the proof into three steps.

Step 1. Thermal fluctuation. Adding (75) at $y=\pm\epsilon e_{j}$ gives $u_{N}(x+\epsilon e_{j})+u_{N}(x-\epsilon e_{j})-2u_{N}(x)\leq2\epsilon^{2}$, and the expectation of the corresponding second difference of $\log W$ is $N$ times the left side. This second difference is nonnegative by convexity, and, divided by $\epsilon^{2}$, it converges to $s_{N}^{2}\operatorname{Var}_{\langle\cdot\rangle}(H^{j})$ by (3.27). Fatou’s lemma gives $\mathbb{E}\operatorname{Var}_{\langle\cdot\rangle}(H^{j})\leq2Ns_{N}^{-2}=2N^{9/8}$.

Step 2. Disorder fluctuation. Put $\widehat{u}_{N}=N^{-1}\log W-pk/2$, so that $u_{N}=\mathbb{E}\widehat{u}_{N}$, and fix $\epsilon\in(0,1]$. Let $\Delta=\max|\widehat{u}_{N}(z)-u_{N}(z)|$ over the three points $z\in\{x,x+\epsilon e_{j},x-\epsilon e_{j}\}$. By convexity of $\widehat{u}_{N}$ in $x_{j}$ and the upper bound in (75),

and in the same way, with the secant through $x-\epsilon e_{j}$, $\partial_{j}\widehat{u}_{N}(x)\geq\partial_{j}u_{N}(x)-\epsilon-2\Delta/\epsilon$. The three points are deterministic and lie in a fixed compact set, so Lemma 3.6 gives $\mathbb{E}\Delta^{2}\leq C/N$. With $\epsilon=N^{-1/4}$,

By (3.26), $\partial_{j}\widehat{u}_{N}=N^{-1}s_{N}\langle H^{j}\rangle$ and $\partial_{j}u_{N}=N^{-1}s_{N}\mathbb{E}\langle H^{j}\rangle$, so $\mathbb{E}(\langle H^{j}\rangle-\mathbb{E}\langle H^{j}\rangle)^{2}\leq N^{2}s_{N}^{-2}CN^{-1/2}=CN^{13/8}$. Adding the thermal fluctuation, $\mathbb{E}\langle(H^{j}-\mathbb{E}\langle H^{j}\rangle)^{2}\rangle\leq CN^{13/8}$.

Step 3. Conclusion. Subtract from (3.29) for $n$ replicas the same identity with $n=1$ and $f=1$, multiplied by $\mathbb{E}\langle f\rangle$. The terms with $l=1$ cancel, because $c_j(R_{11},J_{11}/k)=c_j(1,1)$ is a constant, and what remains is

By the Cauchy–Schwarz inequality the left side is at most $CN^{13/16}$. Dividing by $ns_Nx_jN\ge\frac{1}{2}N^{15/16}$, which uses $x_j\ge1/2$, gives $\lvert\mathcal{D}_n(f,c_j)\rvert\le CN^{-1/8}$. □

Synchronization

We use Panchenko’s synchronization of overlaps, in the quantitative form proved by Mourrat. Panchenko proved that, under the Ghirlanda–Guerra identities, the overlap of each species in a multi-species model is a nondecreasing Lipschitz function of the total overlap [ref-100]. The proof rests on his theorem that the Ghirlanda–Guerra identities imply ultrametricity [ref-97]. He proved the analogue for vector spins in [ref-101]. Mourrat’s synchronization theorem [ref-92] is a variant for two overlap arrays under approximate identities. The following form of it bounds the conditional variance of one overlap given the other, once approximate Ghirlanda–Guerra identities hold for a sufficiently rich finite family of test functions. Chapter 4 uses the same synchronization in the lower bound.

Lemma 3.8. Fix an enumeration $(\vartheta_n)$ of $\mathbb{Q}\cap[0,1]$. For every $\varepsilon>0$ there is an integer $n_\varepsilon\ge1$ with the following property. Let a random probability measure be supported on the product of the unit balls of two Hilbert spaces, and let $A^{ll'}$ and $B^{ll'}$ be the two inner-product arrays of independent samples from it. Suppose that

and every continuous test function $f$ of the arrays of $n$ samples with $\lvert f\rvert\le1$, where $c_j(u,v)=(\vartheta_{j_1}u+\vartheta_{j_2}v)^{j_3}$. If the averaged law of $B^{12}$ is $K^{-1}\sum_{l=1}^{K}\delta_{q_l}$, where $-1=q_0<q_1<\cdots<q_K\le1$, then

Proof. This is [ref-92], stated there with a number $\delta>0$ in place of $1/n_\varepsilon$ and with the conditions required for $n,j_1,j_2,j_3\le\lfloor\delta^{-1}\rfloor$; its hypothesis on the test functions and on the law of $B^{12}$ is the one above, and $\mathbb{E}\langle A^{12}\mid B^{12}\rangle$ is the conditional expectation under the averaged measure. Our form follows by taking $n_\varepsilon=\lceil\delta^{-1}\rceil$, because then $1/n_\varepsilon\le\delta$ and $\lfloor\delta^{-1}\rfloor\le n_\varepsilon$. □

We apply Lemma 3.8 with the following choices. Recall that $K=k+1$ and $w_l=1/K$ for $0\le l\le k$. The maps $\sigma\mapsto\sigma/\sqrt{N}$ and $\beta\mapsto k^{-1/2}\sum_{l=1}^{k}\mathbf{u}_{\beta|l}$, where $(\mathbf{u}_\nu)$ is an orthonormal family indexed by the nodes of depth at least one, represent $R$ and $J/k$ as inner products of unit vectors, since $k^{-1}\#\{1\le l\le k:\beta|l=\beta'|l\}=(\beta\wedge\beta')/k$. Up to the factor $k^{-1/2}$, this is the embedding of [ref-92]. The image of the Gibbs measure under the product of these maps is a random probability measure on the product of two unit balls. By Lemma 3.2, $J/k$ has averaged law exactly uniform on $\{0,1/k,\ldots,1\}$, for every value of the parameters, including the perturbation. In the notation of Lemma 3.8 this is $q_l=(l-1)/k$ for $1\le l\le K$, so the gaps $q_{l+1}-q_l$ are $1$ (for $l=0$) and $1/k$, and the right side of (76) is at most $12/K+\varepsilon K^3$.

We choose the parameters in the following order. Given $K$, let $\varepsilon=K^{-4}$, let $n_K=n_\varepsilon$ be given by Lemma 3.8, and take $j_+=n_K$, so that the functions $c_j$, $j\in\mathcal{I}$, are exactly those of Lemma 3.8. At any point satisfying the hypotheses of Lemma 3.7, and for $N$ so large that $CN^{-1/8}\le1/n_K$, the hypotheses of Lemma 3.8 hold. The left side of (76) equals $\sum_l w_l\operatorname{Var}(R_{12}\mid J=l)$, since $\mathbb{E}\langle R_{12}\mid J\rangle=r_J$, and $12/K+K^{-4}K^3=13/K$. Hence, with

we have $\sum_l w_l\operatorname{Var}(R_{12}\mid J=l)\le13/K$ and $|\mathcal{E}_N|\le\eta_N$ at such a point. The second bound follows from the first: since $|X_{12}|\le L^2$, the conditional Cauchy–Schwarz inequality gives $|\operatorname{Cov}(R_{12},\alpha_NX_{12}\mid J=l)|\le\alpha_NL^2\operatorname{Var}(R_{12}\mid J=l)^{1/2}$, and the Cauchy–Schwarz inequality over $l$, with $\sum_l w_l=1$, gives $\sum_l w_l\operatorname{Var}(R_{12}\mid J=l)^{1/2}\le(13/K)^{1/2}$. By (66),

The tolerance $\varepsilon=K^{-4}$ and the bound $13/K$ are those of Mourrat [ref-92]. There the conditional variances of both overlaps enter. Here only the spin overlap is synchronized with $J$, and the response overlap enters through the bound $|X_{12}|\le L^2$, which gives the square root in $\eta_N$.

The spherical term

At $t=0$ the spins interact only with the cascade field $Y$, through the term $\psi_{\sigma,N}(p)$ of (70). This is the only geometric term in the comparison. It enters through the maximum over $p$ of $\psi_{\sigma,N}(p)+\frac{1}{2}\sum_l w_lh_lp_l$, and we show that this maximum is at most $\mathrm{CS}(\gamma_h)+o_N(1)$. The idea is to replace the uniform measure on the sphere by a Gaussian measure with a well-chosen variance $1/b$, for which the cascade recursion is computed exactly by Lemma 3.4. This replacement, at the cost $(b-1-\log b)/2$, is the large deviation argument behind the Parisi formula for spherical models [ref-125]; see also [ref-37, ref-93]. Here only an upper bound is needed, and it holds for each $N$ with an error of order $\log N/N$.

Lemma 3.9. The following hold.

(i) For every $N$ and every $p \in K$, $\psi_{\sigma,N}(p) \le\sqrt{p_k}-\frac{1}{2}\sum_l w_l p_l$.

(ii) For $h \in K$ with $h_k<1$,

where $o_N(1)$ depends only on $K$ and $h$.

Proof. Part (ii) rests on part (i).

Proof of (i). The leaf logarithm $\log\int e^{Y_\beta\cdot\sigma}\,\mathrm{d}\mu_N$ is $\sqrt{N\Delta p_l}$-Lipschitz in the Gaussian vector $y_{\beta|l}$, since its gradient is $\sqrt{\Delta p_l}\langle\sigma\rangle_\beta$. Differentiating (47) shows that each $X_{l-1}$ keeps these Lipschitz bounds in the earlier vectors, since its gradient is a $K_l$-average. Gaussian concentration in the form $\mathbb{E}e^{\theta(F-\mathbb{E}F)}\le e^{\theta^2\ell^2/2}$ for an $\ell$-Lipschitz function $F$ of a standard Gaussian vector [ref-21], Theorem 5.5, applied at each level $l=k,\ldots,1$, shows that $X_{l-1}$ exceeds the plain average of $X_l$ over $y_{\beta|l}$ by at most $N\theta_l\Delta p_l/2$. The root is averaged without a logarithm. Therefore, by (50),

where $Y\sim N(0,p_kI_N)$, and we used $Y\cdot\sigma\le\sqrt{N}|Y|$ and $\mathbb{E}|Y|\le\sqrt{Np_k}$. By (52) this proves (i).

Proof of (ii). We divide the proof into three steps.

Step 1. Localization. By (i), since $h_l\le h_k$ and $p_l\ge0$,

The value at $p=0$ is zero, and the right side is negative when $p_k>\bar p\coloneqq4/[w_k(1-h_k)]^2$. So the supremum may be restricted to the set $\{p\in K:p_k\le\bar p\}$, and we prove the bound uniformly over this set. Fix such a $p$.

Step 2. Choice of the variance. In the notation of (53) with $n=N$, as $b$ increases every $d_l$ increases, so $m_k+1/b$ decreases continuously. It tends to zero as $b\to\infty$, and to infinity as $b\downarrow\sum_{j\ge1}\theta_j\Delta p_j$: if $p_0>0$, the term $p_0/d_0^2$ diverges; if $p_0=0$ and $p\ne0$, then for the first index $j_*$ with $\Delta p_{j_*}>0$ we have $d_{j_*-1}=b-\sum_{j\ge1}\theta_j\Delta p_j$, and the $j_*$-th term of $m_k$ diverges; if $p=0$, the term $1/b$ diverges. We choose $b$ as the unique solution of $m_k+1/b=1$, so that Lemma 3.4(iv) with $\varrho=1$ applies. In particular $b\geq1$. Also $b<2(\bar p+1)$: since $d_l\geq d_0\geq b-p_k\geq b-\bar p$, we have $m_k\leq p_k/(b-\bar p)^2\leq\bar p/(b-\bar p)^2$ for $b>\bar p$, and at $b=2(\bar p+1)$ this plus $1/b$ is at most $\frac{1}{8}+\frac{1}{2}<1$, because $(\bar p+2)^2\geq8\bar p$.

Step 3. Comparison with the sphere. For fixed $Y$, the average of $e^{\rho Y\cdot\sigma}$ over $\sigma\in S_N$ is nondecreasing in $\rho\geq0$: by the symmetry $\sigma\mapsto-\sigma$ it equals the average of $\cosh(\rho Y\cdot\sigma)$, whose derivative in $\rho$ is nonnegative. If $\varepsilon\sim N(0,b^{-1}I_N)$, then $\varepsilon=|\varepsilon|N^{-1/2}\sigma$ with $\sigma$ uniform on $S_N$ and independent of $|\varepsilon|$, and $b|\varepsilon|^2$ has the chi-square law $\chi_N^2$ with $N$ degrees of freedom. Restricting $\mathbb{E}e^{Y\cdot\varepsilon}=e^{|Y|^2/(2b)}$ to $\{|\varepsilon|^2\geq N\}$ and using the monotonicity gives, pathwise,

For $N\geq2$ and $b\geq1$,

Indeed, the chi-square density $x^{N/2-1}e^{-x/2}/(2^{N/2}\Gamma(N/2))$ is nonincreasing on $[N-2,\infty)$, so $\mathbb{P}\{\chi_N^2\geq bN\}$ is at least twice its value at $bN+2$. Then (80) follows from $(bN+2)^{N/2-1}\geq(bN)^{N/2-1}$ and Stirling’s bound $\Gamma(z)\leq\sqrt{2\pi}z^{z-1/2}e^{-z+1/(12z)}$ with $z=N/2$, after collecting terms. Take expectations, divide by $N$, and use (54) (with $n=N$), (61) and (62) in the form $\frac{b-1}{2}-\frac{p_k}{2}=-\frac{1}{2}\sum_l w_lm_lp_l$. Since $1\leq b<2(\bar p+1)$, this gives

It remains to show that $\operatorname{CS}(\gamma_h)\geq\operatorname{CS}(\gamma_m)+\frac{1}{2}\sum_l w_lp_l(h_l-m_l)$. The map $x\mapsto\operatorname{CS}(\gamma_x)$ is convex on the convex set $\{x\in K:x_k<1\}$. Indeed, compute the entropies at two points $x,x'$ with a common cutoff $q'\in[\max(x_k,x'_k),1)$: for each $s<q'$, $\lambda_{\gamma_x}(s)=1-\sum_lw_l\max(x_l,s)$ is positive and concave in $x$, so its reciprocal is convex. Hence $f(\epsilon)=\operatorname{CS}(\gamma_{m+\epsilon(h-m)})$ is convex on $[0,1]$, and $\operatorname{CS}(\gamma_h)\geq\operatorname{CS}(\gamma_m)+f'(0+)$; this one-sided derivative is all we need, also when $m$ lies on the boundary of $K$. With the common cutoff $q'=\max(h_k,m_k)$, all the functions $\lambda$ involved are at least $1-q'>0$ on $[0,q']$, and $|\max(m_l+\epsilon(h_l-m_l),s)-\max(m_l,s)|\leq\epsilon|h_l-m_l|$. For almost every $s$, the right derivative of $\max(m_l+\epsilon(h_l-m_l),s)$ at $\epsilon=0$ is $(h_l-m_l)\mathbf{1}_{\{m_l>s\}}$. Dominated convergence and (61) give

This bounds the left side of the previous display by $\operatorname{CS}(\gamma_h)+C_{\bar p}\log N/N$, uniformly over $p\in K$ with $p_k\leq\bar p$, and proves (ii). $\square$

The comparison

We now show that, up to the synchronization error, the free energy at $t=1$ is at most the maximum over $p$ of the interpolation at $t=0$. The reason is the following. At a point where $p$ maximizes $u_N(t,\cdot,x)+\frac{1}{2}\sum_l w_lh_lp_l$, the product term in (78) has a sign, so $\partial_tu_N$ is at most the synchronization error. A maximum principle in $t$ turns this into a bound on the growth of the maximum over $p$. Maximizing also over $x$, with a quadratic penalty, selects a perturbation at which Lemma 3.7 applies.

This is a finite-$N$ form of Step 5 of the proof of [ref-92] (Theorem 4.1), where the first-order conditions [ref-92] ((4.39)–(4.40)) at a contact point and the monotonicity of [ref-92] (Lemma 2.4) give the sign [ref-92] ((4.32)) needed for the supersolution property. Here the maximum principle in $t$ for the maximum over $p$ takes the place of viscosity solutions and of the comparison principle [ref-92] (Proposition 3.2). The use of first-order conditions in the levels to control the derivative in $t$ also parallels the stationarization of Stojnic’s fully lifted interpolation for bilinearly indexed random processes [ref-120, ref-121]. There the levels on both sides are chosen at stationary points along the interpolation path, and the derivative in $t$ vanishes when the overlaps concentrate.

Proposition 3.10. Let $U$ belong to class (A), let $K\geq2$ and $h\in\mathcal K$ with $h_k<1$, let $n_K$ and $j_+=n_K$ be as in §3.5, and let $\eta_N$ and $\bar\psi_N$ be given by (77) and (79). There is $N_0$, depending only on $U$, $K$, $h$, and $\alpha'$, such that for $N\geq N_0$

Consequently,

Proof. Fix $t_0\in(0,1)$ and $\varsigma\in(0,1]$, and consider on $[0,t_0]\times\mathcal K\times\mathbb R^{\mathcal I}$ the function

We divide the proof into four steps.

Step 1. Localization. We show that $\Psi$ attains its maximum and that all maximum points lie in a compact set that does not depend on $N$, $t_0$, or $\varsigma$. We take $N\geq N_0$, with $N_0$ fixed at the end of this step; in particular

where $C_{j_+}$ is the constant of (73). Then $C_{j_+}N^{-1/8}|x|^2\leq2C_{j_+}N^{-1/8}(|x-\mathbf1|^2+|\mathcal I|)\leq\frac{1}{2}|x-\mathbf1|^2+1$. Since $U\leq U_{\max}$, the leaf integral is at most $e^{MU_{\max}}\int e^{Y_\beta\cdot\sigma}\mathrm d\mu_N$ when $x=0$, so $u_N(t,p,0)\leq\alpha_NU_{\max}+\psi_{\sigma,N}(p)$. With (73), Lemma 3.9(i) and $h_l\leq h_k$,

On the other hand, by (73), (70), $\psi_{\sigma,N}(0)=0$, and the bound $u_{\gamma h}(0,0;U)\geq\mathbb{E}U(G)\geq U(0)-L$ from (2.6),

Since $w_k(1-h_k)>0$ and $0\leq p_l\leq p_k$ on $K$, we have $\Psi<\Psi(0,0,1)$ outside $[0,t_0]\times K_0$, for a compact set $K_0\subset K\times\mathbb{R}^{\mathcal{I}}$ that depends only on $U,K,h,\mathcal{I}$, and $\alpha'$. As $\Psi$ is continuous by Lemma 3.5(i), it attains its maximum, and every maximum point lies in $[0,t_0]\times K_0$. Let $N_0$ be the least integer such that, for all $N\geq N_0$, condition (a) holds and also

(b) $2^{j_+}N^{-1/8}|x|\leq\frac{1}{2}$ for all $(p,x)\in K_0$, and

(c) $CN^{-1/8}\leq1/n_K$, where $C$ is the constant of Lemma 3.7 for the compact set $[0,1]\times\{p:(p,x)\in K_0\text{ for some }x\}$.

Since $K_0,\mathcal{I}$, and $n_K$ are determined by $U,K,h$, and $\alpha'$, so is $N_0$; it does not depend on $t_0$ or $\varsigma$.

Step 2. Conditions at a maximum point. Let $(t_*,p_*,x_*)$ be a maximum point. First, $x_*$ maximizes $x\mapsto u_N(t_*,p_*,x)-|x-1|^2$, so $\nabla_xu_N=2(x_*-1)$ there. The derivative bound of Lemma 3.6 gives $|x_*-1|\leq2^{j_+}N^{-1/8}|x_*|$, so $|x_*-1|\leq\frac{1}{2}$ by (b). Maximality in $x$ gives $u_N(x_*+y)-u_N(x_*)\leq|x_*+y-1|^2-|x_*-1|^2=\nabla_xu_N(x_*)\cdot y+|y|^2$ for all $y\in\mathbb{R}^{\mathcal{I}}$, and convexity of $u_N$ in $x$ gives the lower bound in (75). Thus Lemma 3.7 applies, and by (c) and §3.5, $|\mathcal{E}_N|\leq\eta_N$ at $(t_*,p_*,x_*)$.

Second, $p_*$ maximizes $p\mapsto u_N(t_*,p,x_*)+\frac{1}{2}\sum_l w_lh_lp_l$ over $K$. Since $p_*+\epsilon\mathbf{1}_{\geq m}\in K$ for $\epsilon\geq0$, the right derivative in this direction is nonpositive, and by (67)

Third, $t_*$ maximizes $t\mapsto\Psi(t,p_*,x_*)$ over $[0,t_0]$.

Step 3. Every maximum point has $t_*=0$. Suppose that $t_*>0$. Since $t_*\leq t_0<1$, $u_N$ is differentiable in $t$ at $(t_*,p_*,x_*)$ by Lemma 3.5(ii), and maximality in $t$ gives $\partial_tu_N(t_*,p_*,x_*)\geq\eta_N+\varsigma$. We now show the opposite inequality. Put $\zeta_{-1}=0$, so that $\zeta_l=\sum_{m\leq l}(\zeta_m-\zeta_{m-1})$. Summation by parts, (68) and (83) give

By (78) and Step 2, $\partial_tu_N=-\frac{1}{2}\sum_lw_l(r_l-h_l)\zeta_l-\mathcal{E}_N\leq|\mathcal{E}_N|\leq\eta_N$, a contradiction. Hence $t_*=0$.

Step 4. Conclusion. Compare $\Psi$ at $(t_{0},0,1)$ with its value at a maximum point, which has $t_{*}=0$. By (73) and (70),

By (a), the last two terms are at most $2C_{j_{+}}N^{-1/8}|\mathcal{I}|$, and the first two are at most $\bar{\psi}_{N}$. Letting $t_{0}\uparrow1$, by continuity of $u_{N}$, and then $\varsigma\downarrow0$,

Finally, $F_{N}(U)=u_{N}(1,0,0)\leq u_{N}(1,0,1)+C_{j_{+}}N^{-1/8}|\mathcal{I}|$ by (70) and (73). This proves (81). By Lemma 3.9(ii), $\bar{\psi}_{N}\leq\mathrm{CS}(\gamma_{h})+o_{N}(1)$, and $\alpha_{N}\to\alpha$, so (81) gives (82). $\square$

Proof of Theorem 3.1. Let $\gamma\in\mathcal{U}$ and $K\geq2$. Approximate $\gamma$ by the order parameter $\gamma_{K}=\lfloor K\gamma\rfloor/K$, which has $K$ atoms of mass $1/K$ at the quantiles $h_{l}=\inf\{q:\gamma(q)\geq(l+1)/K\}$, $0\leq l\leq k=K-1$. Indeed, by right-continuity $h_{l}\leq q$ if and only if $\gamma(q)\geq(l+1)/K$, so $\gamma_{h}(q)=K^{-1}\#\{l:h_{l}\leq q\}=\lfloor K\gamma(q)\rfloor/K$. Thus $h\in\mathcal{K}$, $h_{k}=Q_{\gamma}<1$, $\gamma-1/K<\gamma_{K}\leq\gamma$, so that $\|\gamma_{K}-\gamma\|_{L^{1}}\leq1/K$, and $\gamma_{K}=\gamma=1$ on $[Q_{\gamma},1]$. By (82) for this $h$, and by (33) with $\bar{q}=Q_{\gamma}$, since $\gamma,\gamma_{K}\in\mathcal{U}_{Q_{\gamma}}$,

Letting $K\to\infty$ proves (46).

If $L=0$, the activation is constant and $f_{N}(U)=F_{N}(U)$, so the probability bound follows from (46). Let $L>0$. The map $G\mapsto f_{N}(U)$ is Lipschitz: its derivative in $g_{ai}$ is $N^{-3/2}\langle U'(b_{a}\cdot\sigma)\sigma_{i}\rangle$, where now $\langle\cdot\rangle$ is the Gibbs measure of $Z_{N}(U)$, and by Jensen’s inequality and $|\sigma|^{2}=N$ the sum of the squares is at most $\alpha_{N}L^{2}/N$. Gaussian concentration [ref-21] (Theorem 5.6) gives $\mathbb{P}\{f_{N}(U)-F_{N}(U)>\varepsilon/2\}\leq\exp(-\varepsilon^{2}N/(8\alpha_{N}L^{2}))$, and $F_{N}(U)\leq\mathcal{P}_{U}(\gamma)+\varepsilon/2$ for large $N$ by (46).

The parameters and limits are chosen, with $U$ and $\gamma$ fixed, in the following order: first $K$ (and with it $h$), then the synchronization tolerance $\varepsilon=K^{-4}$, the integer $n_{K}$, and the perturbation family $\mathcal{I}$; then, for each $N\geq N_{0}$, the auxiliary $t_{0}\uparrow1$ and $\varsigma\downarrow0$ inside the proof of Proposition 3.10; then $N\to\infty$; and finally $K\to\infty$. $\square$

Hard constraints

We now pass to the hard wall. Recall that $V_{N}(\kappa)=Z_{N}(H_{\kappa})$ with $H_{\kappa}=\log\mathbf{1}_{[\kappa,\infty)}$ and $\log0=-\infty$. Chapter 9 also needs the feasible volume weighted by a bounded function of the fields, so we allow a source term from the start.

Terminal data with a hard wall. Let $\gamma$ be a step order parameter as in (18), and let $\varphi:\mathbb{R}\to\mathbb{R}$ be bounded and measurable. The terminal datum $H_{\kappa}+\varphi$ is bounded above by $\lVert\varphi\rVert_{\infty}$, so the recursion (17) with $T=1$ defines $u_{\gamma}(\cdot,\cdot;H_{\kappa}+\varphi)$, independently of the representation of $\gamma$, and we put

The function $u_{\gamma}(q,x;H_{\kappa})$ is finite for $q<1$: for $q\leq t_{n}$, the last breakpoint of $\gamma$, it equals $u_{\gamma}(q,x;\kappa)$ by (25), and for $t_{n}\leq q<1$ it equals $\log\mathbb{P}\{x+\sqrt{1-q}\,G\geq\kappa\}$ by (22). By (19), applied in both directions, for bounded measurable $\varphi,\varphi'$ the functions $u_{\gamma}(\cdot,\cdot;H_{\kappa}+\varphi)$ are finite on $[0,1)\times\mathbb{R}$, and

By (25), $\mathcal{P}_{H_{\kappa}}(\gamma)=\mathcal{P}_{\kappa}(\gamma)$.

The wall approximation. We approximate the hard wall from above by activations in class (A) that decrease to it. Let

The function $\Omega$ vanishes on $(-\infty,0]$, is positive on $(0,\infty)$, and is $\mathrm{C}^{2}$ with $\Omega'(y)=(y^{+})^{2}/(1+(y^{+})^{2})\in[0,1]$ and $\Omega''(y)=2y^{+}/(1+(y^{+})^{2})^{2}\in[0,1]$. Hence $U_{\ell}(\cdot;\kappa)$ belongs to class (A), with $\sup U_{\ell}=0$ and $|U'_{\ell}|,|U''_{\ell}|\leq\ell$. Moreover, $e^{U_{\ell}}=1$ on $[\kappa,\infty)$, and $e^{U_{\ell}(y)}$ decreases to zero as $\ell\to\infty$ for $y<\kappa$. Thus $e^{U_{\ell}}\downarrow\mathbf{1}_{[\kappa,\infty)}$ pointwise.

Corollary 3.11. Let $\kappa\in\mathbb{R}$ and $\varepsilon>0$.

(i) For every $\gamma\in\mathcal{U}$,

(ii) Let $\gamma$ be a step order parameter and let $\varphi:\mathbb{R}\to\mathbb{R}$ be bounded and Lipschitz. Then

Part (ii) with $\varphi=0$ is part (i) for step order parameters. Chapter 9 uses part (ii) with $\varphi$ a real multiple of a bounded Lipschitz test function translated by $\kappa$, and Chapter 8 uses part (i).

Proof. We prove (ii) first, since (i) is deduced from it.

Proof of (ii). Let $\varphi$ be $L_{\varphi}$-Lipschitz. The activation must be $\mathrm{C}^{2}$, so we first smooth $\varphi$. Let $\chi$ be a smooth probability density supported in $[-1,1]$, and for $\tau\in(0,1]$ let $\varphi_{\tau}=\varphi\ast\chi_{\tau}$ with $\chi_{\tau}(y)=\tau^{-1}\chi(y/\tau)$. Then $\varphi_{\tau}$ is smooth, $\lVert\varphi_{\tau}\rVert_{\infty}\leq\lVert\varphi\rVert_{\infty}$, and $\lvert\varphi_{\tau}-\varphi\rvert\leq L_{\varphi}\tau$. Moreover $\varphi_{\tau}'=\varphi'*\chi_{\tau}$ and $\varphi_{\tau}''=\varphi'*\chi_{\tau}'$, where $\varphi'$ is the almost everywhere derivative, so $\lvert\varphi_{\tau}'\rvert\leq L_{\varphi}$ and $\lvert\varphi_{\tau}''\rvert\leq L_{\varphi}\lVert\chi'\rVert_{L^{1}}/\tau$. Hence $\widetilde U_{\ell}=U_{\ell}(\mathord{\cdot};\kappa)+\varphi_{\tau}$, with $U_{\ell}$ from (85), belongs to class (A) for every $\ell$.

Since $U_{\ell}(\mathord{\cdot};\kappa)=0$ on $[\kappa,\infty)$ and the integrand of $Z_{N}(\widetilde U_{\ell})$ is positive,

So the normalized logarithm in (ii) is at most $f_{N}(\widetilde U_{\ell})+\alpha' L_{\varphi}\tau$.

We show that $u_{\gamma}(0,0;\widetilde U_{\ell})$ decreases to $u_{\gamma}(0,0;H_{\kappa}+\varphi_{\tau})$ as $\ell\to\infty$. The terminal data satisfy $\mathrm{e}^{\widetilde U_{\ell}}\downarrow\mathbf{1}_{[\kappa,\infty)}\mathrm{e}^{\varphi_{\tau}}=\mathrm{e}^{H_{\kappa}+\varphi_{\tau}}$ pointwise, with values in $(0,\mathrm{e}^{\lVert\varphi\rVert_{\infty}}]$. The recursion (2.2) applies the same operators $T_{m,s}$ to both terminal data. Each is nondecreasing, so by induction over the steps the functions obtained from $\widetilde U_{\ell}$ decrease in $\ell$; they are bounded above by $\lVert\varphi\rVert_{\infty}$, and their limits are the functions obtained from $H_{\kappa}+\varphi_{\tau}$, which are finite by (84). Indeed, at a step with $m>0$ the functions $\mathrm{e}^{m f_{\ell}}\in(0,\mathrm{e}^{m\lVert\varphi\rVert_{\infty}}]$ decrease to $\mathrm{e}^{m f}$, and at a step with $m=0$ the functions $\lVert\varphi\rVert_{\infty}-f_{\ell}\geq0$ increase to $\lVert\varphi\rVert_{\infty}-f$; bounded and monotone convergence apply. Hence $\mathcal{P}_{\widetilde U_{\ell}}(\gamma)\downarrow\mathcal{P}_{H_{\kappa}+\varphi_{\tau}}(\gamma)$, and $\mathcal{P}_{H_{\kappa}+\varphi_{\tau}}(\gamma)\leq\mathcal{P}_{H_{\kappa}+\varphi}(\gamma)+\alpha L_{\varphi}\tau$ by (84).

Now choose $\tau$ with $(\alpha'+\alpha)L_{\varphi}\tau\leq\varepsilon/3$, and then $\ell$ with $\mathcal{P}_{\widetilde U_{\ell}}(\gamma)\leq\mathcal{P}_{H_{\kappa}+\varphi_{\tau}}(\gamma)+\varepsilon/3$. Then $\mathcal{P}_{\widetilde U_{\ell}}(\gamma)+\alpha' L_{\varphi}\tau\leq\mathcal{P}_{H_{\kappa}+\varphi}(\gamma)+2\varepsilon/3$, and Theorem 3.1, applied to $\widetilde U_{\ell}$ and $\gamma$, gives

Proof of (i). Let $\gamma\in\mathcal{U}$ and $r=Q_{\gamma}<1$. The step order parameters $\gamma_{n}\in\mathcal{U}_{r}$ of §2.1 converge to $\gamma$ in $L^{1}([0,1])$, so $u_{\gamma_{n}}(0,0;\kappa)\to u_{\gamma}(0,0;\kappa)$ by the definition of $u_{\gamma}$ through Lemma 2.7, and $\mathrm{CS}(\gamma_{n})\to\mathrm{CS}(\gamma)$ by (2.12). Hence there is a step order parameter $\gamma'\in\mathcal{U}_{r}$ with $\mathcal{P}_{\kappa}(\gamma')\leq\mathcal{P}_{\kappa}(\gamma)+\varepsilon/2$. Part (ii) with $\varphi=0$, the order parameter $\gamma'$, and $\varepsilon/2$ gives the claim, since $\mathcal{P}_{H_{\kappa}}(\gamma')=\mathcal{P}_{\kappa}(\gamma')$. $\square$

Volume from one feasible point

A bound on the volume does not by itself bound the largest feasible margin, because a feasible set may have zero spherical volume. Lemma 3.13 shows that one feasible point at margin $\kappa$ produces exponentially small, but not superexponentially small, volume at margin $\kappa-\eta$, on a set where the fields $b_{a}\cdot\sigma$ change little. The set it constructs is used again in Chapter 9. The proof uses Šidák’s inequality for a Gaussian vector whose covariance may be singular; Chapter 8 uses the same extension.

Lemma 3.12. Let $(\xi_{1},\ldots,\xi_{M})$ be a centered Gaussian vector with an arbitrary, possibly singular, covariance matrix, and let $c_{1},\ldots,c_{M}\ge0$. Then

Proof. For a nonsingular covariance matrix this is Šidák’s inequality [ref-116], Corollary 1, proved independently by Khatri [ref-80]. If $c_{a}=0$ and $\operatorname{Var}\xi_{a}>0$ for some $a$, the right side vanishes, and a coordinate with $c_{a}=0$ and $\xi_{a}=0$ contributes the factor one to both sides and may be removed. So we may assume that all $c_{a}>0$. Let $\xi'\sim N(0,I_{M})$ be independent of $\xi$, and for $\epsilon>0$ put $\xi^{\epsilon}=\xi+\epsilon\xi'$, whose covariance matrix is nonsingular. Then $\mathbb{P}\{|\xi_{a}^{\epsilon}|\le c_{a}\ \forall a\}\ge\prod_{a}\mathbb{P}\{|\xi_{a}^{\epsilon}|\le c_{a}\}$. Let $\epsilon\downarrow0$ along a sequence. Each factor converges to $\mathbb{P}\{|\xi_{a}|\le c_{a}\}$: if $\operatorname{Var}\xi_{a}>0$, because $\xi_{a}^{\epsilon}$ and $\xi_{a}$ are centered Gaussian with variances converging and a continuous limiting law; if $\xi_{a}=0$, because $\mathbb{P}\{|\epsilon\xi_{a}'|\le c_{a}\}\to1$. Since $\xi^{\epsilon}\to\xi$ pointwise and the intervals are closed, the indicator of $\{|\xi_{a}^{\epsilon}|\le c_{a}\ \forall a\}$ has $\limsup$ at most the indicator of $\{|\xi_{a}|\le c_{a}\ \forall a\}$. Fatou’s lemma, applied to one minus the indicators, proves the lemma. □

Lemma 3.13. Fix $\alpha'<\infty$. There are constants $K_{\mathrm{cap}}\ge1$, $C_{\mathrm{cap}}<\infty$, and $N_{0}$, depending only on $\alpha'$, with the following property. Let $N\ge N_{0}$, $M\le\alpha'N$, and let $b_{1},\ldots,b_{M}\in\mathbb{R}^{N}$ be deterministic vectors with $|b_{a}|^{2}\le2$. Let $\kappa\in\mathbb{R}$ and $\eta\in(0,1]$, suppose that some $\sigma_{0}\in S_{N}$ satisfies $b_{a}\cdot\sigma_{0}\ge\kappa$ for all $a$, and put

There is a Borel set $T\subset\{\sigma\in S_{N}:b_{a}\cdot\sigma\ge\kappa-\eta\ \text{for all }a\}$ with

If $\kappa$ ranges over a compact interval $I$, then $\log(2/s_{0})\le\log(1/\eta)+C_{I}$ with $C_{I}$ depending only on $\alpha'$ and $I$.

Proof. We divide the proof into four steps.

Step 1. Constants. Let $c_{0}=\frac{1}{2}(\log2-\frac{1}{2})>0$. For $X\sim\chi_{d}^{2}$ and $0<y<1$, Markov’s inequality applied to $e^{-uX}$ with $u=(1-y)/(2y)$ gives the Chernoff bound $\mathbb{P}\{X\le yd\}\le e^{uyd}(1+2u)^{-d/2}=\exp\{-\frac{d}{2}(y-1-\log y)\}$; with $y=\frac{1}{2}$ this is $\mathbb{P}\{X\le d/2\}\le e^{-c_{0}d}$. Let $\pi(z)=\mathbb{P}\{|\sqrt{2}G|\le z/2\}$, and choose $K_{\mathrm{cap}}\ge1$ with $\alpha'\log(1/\pi(K_{\mathrm{cap}}))\le c_{0}/4$, which is possible because $\pi(z)\to1$ as $z\to\infty$. Put $C_{\mathrm{cap}}=c_{0}/4+\log2$, and let $N_{0}\ge3$ be such that $e^{-c_{0}(N-1)}\le\frac{1}{2}e^{-c_{0}N/4}$ for $N\ge N_{0}$.

Step 2. Coordinates. Let $P$ be the orthogonal projection onto $\sigma_{0}^{\perp}$. Every $\sigma\in S_{N}$ with $|\sigma\cdot\sigma_{0}| < N$ can be written uniquely as

Under $\mu_{N}$, $\rho$ and $v$ are independent, $v$ is uniform on the unit sphere of $\sigma_{0}^{\perp}$, and $\rho$ has density $c_{N}'(1-\rho^{2})^{(N-3)/2}$ on $(-1,1)$, with $c_{N}'=\Gamma(N/2)/(\sqrt{\pi}\Gamma((N-1)/2))$. This constant is nondecreasing in $N$ by log-convexity of $\Gamma$, and equals $1/2$ at $N=3$. Let

It is a Borel set, and on it $s\in[s_{0}/2,s_{0}]$ and $\rho\ge0$. Also $s_{0}\le\eta/(2K_{\mathrm{cap}})\le1/2$.

Step 3. Fields. For $\sigma\in T$, $b_{a}\cdot\sigma-b_{a}\cdot\sigma_{0}=(\rho-1)b_{a}\cdot\sigma_{0}+s\sqrt{N}\,b_{a}\cdot v$, and $0\le1-\rho\le s^{2}\le s_{0}^{2}$ because $\rho=\sqrt{1-s^{2}}\ge1-s^{2}$. This gives the second bound in (86). For feasibility, $b_{a}\cdot\sigma_{0}\ge\kappa$ and $\rho\in[1-s_{0}^{2},1]$ imply $\rho b_{a}\cdot\sigma_{0}\ge\kappa-s_{0}^{2}|\kappa|$: if $\kappa>0$, then $\rho b_{a}\cdot\sigma_{0}\ge(1-s_{0}^{2})\kappa$; if $b_{a}\cdot\sigma_{0}\ge0\ge\kappa$, the left side is nonnegative; and if $\kappa\le b_{a}\cdot\sigma_{0}<0$, then $\rho b_{a}\cdot\sigma_{0}\ge b_{a}\cdot\sigma_{0}$. Therefore

using $s_{0}^{2}\le\eta/(2|\kappa|+1)$ and $s_{0}\le\eta/(2K_{\mathrm{cap}})$. Thus every $\sigma\in T$ is feasible at margin $\kappa-\eta$.

Step 4. Volume. On the interval of $\rho$ defining $T$ we have $1-\rho^{2}\ge s_{0}^{2}/4$, and the interval has length $\frac{3}{4}s_{0}^{2}/(\sqrt{1-s_{0}^{2}/4}+\sqrt{1-s_{0}^{2}})\ge\frac{3}{8}s_{0}^{2}$. Hence its probability is at least $\frac{1}{2}(s_{0}/2)^{N-3}\cdot\frac{3}{8}s_{0}^{2}=\frac{3}{2s_{0}}(s_{0}/2)^{N}\ge(s_{0}/2)^{N}$. For the direction, represent $v=P\widetilde{\zeta}/|P\widetilde{\zeta}|$ with $\widetilde{\zeta}\sim N(0,I_{N})$, so that $b_{a}\cdot v=(Pb_{a})\cdot\widetilde{\zeta}/|P\widetilde{\zeta}|$. On the event $\{|P\widetilde{\zeta}|^{2}\ge N/4\}\cap\{|(Pb_{a})\cdot\widetilde{\zeta}|\le K_{\mathrm{cap}}/2\text{ for all }a\}$ we have $\sqrt{N}|b_{a}\cdot v|\le K_{\mathrm{cap}}$ for all $a$. The variables $(Pb_{a})\cdot\widetilde{\zeta}$ are centered and jointly Gaussian with variances $|Pb_{a}|^{2}\le2$, and $\mathbb{P}\{|N(0,v)|\le K_{\mathrm{cap}}/2\}$ is nonincreasing in $v$. By Lemma 3.12, the choice of $K_{\mathrm{cap}}$, and $M\le\alpha' N$,

Since $|P\widetilde{\zeta}|^{2}$ has the law $\chi_{N-1}^{2}$ and $N/4\le(N-1)/2$, Step 1 gives $\mathbb{P}\{|P\widetilde{\zeta}|^{2}<N/4\}\le e^{-c_{0}(N-1)}\le\frac{1}{2}e^{-c_{0}N/4}$. The directional event therefore has probability at least $\frac{1}{2}e^{-c_{0}N/4}$. By independence of $\rho$ and $v$,

Finally, for $\kappa\in I$ and $\eta\le1$ we have $\sqrt{\eta/(2|\kappa|+1)}\ge\eta/\sqrt{2|\kappa|+1}$, so $s_{0}\ge c_{I}\eta$ with $c_{I}=\min\{1/(2K_{\mathrm{cap}}),(2\sup_{\kappa\in I}|\kappa|+1)^{-1/2}\}$, and $\log(2/s_{0})\le\log(1/\eta)+\log(2/c_{I})$. $\square$

Corollary 3.14. Let $\alpha>0$. For every real $\kappa>\kappa_{\mathrm{c}}(\alpha)$, $\mathbb{P}\{\kappa_{N}\ge\kappa\}\to0$.

Proof. Choose $\eta\in(0,1]$ with $\kappa-\eta>\kappa_{c}$, and let $N_{0}$ and $C_{V}=\log(2/s_{0})+C_{\mathrm{cap}}$ be given by Lemma 3.13 for $\alpha'=\sup_{N}\alpha_{N}$, the margin $\kappa$, and this $\eta$. Since $\kappa-\eta>\kappa_{c}$, the definition of $\kappa_{c}$ gives $\mathcal{P}_{*}(\kappa-\eta)=-\infty$, so there is $\gamma\in\mathcal{U}$ with $\mathcal{P}_{\kappa-\eta}(\gamma)<-C_{V}-1$. By Corollary 3.11(i) with $\varepsilon=1$,

On the other hand, suppose that $N\geq N_{0}$, $\kappa_{N}\geq\kappa$, and $\max_{a}|b_{a}|^{2}\leq2$. A maximizer $\sigma_{0}$ of $\min_{a}b_{a}\cdot\sigma$ over the compact set $S_{N}$ satisfies $b_{a}\cdot\sigma_{0}\geq\kappa$ for all $a$, so Lemma 3.13 gives $V_{N}(\kappa-\eta)\geq\mu_{N}(T)\geq e^{-NC_{V}}$. Finally, $N|b_{a}|^{2}$ has the law $\chi_{N}^{2}$, and the Chernoff bound $\mathbb{P}\{\chi_{N}^{2}\geq2N\}\leq e^{-N(1-\log2)/2}$ (the upper-tail analogue of Step 1 in the proof of Lemma 3.13) gives

Hence $\mathbb{P}\{\kappa_{N}\geq\kappa\}\leq\mathbb{P}\{N^{-1}\log V_{N}(\kappa-\eta)\geq-C_{V}\}+\mathbb{P}\{\max_{a}|b_{a}|^{2}>2\}\to0$. $\square$

The lower bound

In this chapter we prove the lower bound $\liminf_{N} F_{N}(U) \ge\mathcal{P}_{*}(U)$ for smooth activations $U$ (Theorem 4.1). Together with the upper bound of Chapter 3, it gives the Parisi formula in Chapter 5. The argument is outlined in Step 2 of §1.3.

§4.1 introduces the spherical coordinates, the Poisson row count, and the three systems that are compared, and §4.2 proves the overlap representation theorem (Theorem 4.6) from the external results behind it. §4.3 defines the perturbation and states the perturbation, continuity, and precision lemmas, which are proved in §4.8. §4.4 reduces the increment to the row term and the coordinate term. §4.5 evaluates them on cascades with Lemma 3.4 and removes the cutoff on $\lVert\varepsilon\rVert$ there (Lemma 4.10). §4.6 proves the rotation estimate, and §4.7 proves Theorem 4.1.

The two cavity steps, adding constraints and adding coordinates, follow the cavity method for the perceptron in the replica-symmetric regime [ref-115], [ref-127], Chapters 2 and 3, [ref-128], Chapter 8. As in [ref-37, ref-99], the trial order parameter comes from the limiting overlap law of the system. Compared with W.-K. Chen’s spherical form [ref-37] of the Aizenman–Sims–Starr scheme and with its multi-species version by Bates and Sohn [ref-16], two points differ. First, in both works the cavity functional is identified with the spherical Parisi functional, as the number of cavity coordinates grows, through Talagrand’s computation [ref-125]; here the coordinate term is an explicit Gaussian integral on cascades, and the rotation invariance of the patterns identifies it with the Crisanti–Sommers entropy. Second, the other species in the Ghirlanda–Guerra identities is not a group of coordinates, as in [ref-16, ref-100], but the response vector.

Activation class. Throughout this chapter, $U$ satisfies, for some $U_{\max} < \infty$ and $L < \infty$ (enlarged, if necessary, so that $L \ge1 + |U(0)|$),

and in addition either $U'$ is bounded or $U$ is concave. In the first case $U$ belongs to class (A), and in the second to class (B). The derivatives of orders three and four are used only in the perturbation of §4.3, in the error estimates of the cavity interpolation, and in Lemma 4.9. They control the first and second derivatives of $xU'(x)$ and $U'(x)^{2}$. In Chapter 5 this extra smoothness is removed by mollification. By (87) and Taylor’s formula,

Conventions. Throughout this chapter, $C$ and $c$ denote positive constants that may depend on $\alpha$, $U_{\max}$, and $L$, but not on $N$; their values may change from one occurrence to the next. A dependence on the cavity dimension $n$ or on the cutoff $\Lambda$ is indicated by a subscript, as in $C_{n,\Lambda}$ or $O_{n,\Lambda}(\cdot)$. The cavity dimension $n$ is fixed until the last step of the proof. We write $\nu_{n}$ for the standard Gaussian measure on $\mathbb{R}^{n}$ and $G$ for a standard Gaussian variable. The letter $a$ is a row index, and $\mathbf{G},\mathbf{G}',\mathbf{G}''$ denote Gaussian matrices with rows $g_{a},g'_{a},g''_{a}$. The letter $b$, with or without subscripts, denotes a precision parameter of a Gaussian integral, as in Lemma 3.4. The rows $b_{a}=g_{a}/\sqrt{N}$ of Chapter 1 and the derivative $b=\partial_{x}u$ of §2.3 are not used in this chapter.

The goal of the chapter is the following theorem.

Theorem 4.1. For every $\alpha>0$ and every activation $U$ satisfying (87), with either bounded $U'$ or concave $U$,

Spherical coordinates, the row count, and the three systems

We first separate the added coordinates without changing the surface measure. Throughout this chapter

For $\sigma\in S_{N}$ and $\varepsilon\in\mathbb{R}^{n}$ with $\lVert\varepsilon\rVert^{2}<N'$, define

The uniform probability measure $\mu_{N'}$ on $S_{N'}$ is the image of $\mu_{N}(\mathrm{d}\sigma)\otimes f_{N,n}(\varepsilon)\,\mathrm{d}\varepsilon$ under $(\sigma,\varepsilon)\mapsto\rho(\sigma,\varepsilon)$, where

Indeed, write a uniform point of $S_{N'}$ as $\sqrt{N'}\,\eta/\lVert\eta\rVert$ for a standard Gaussian vector $\eta=(\eta_{1},\eta_{2})\in\mathbb{R}^{N}\times\mathbb{R}^{n}$. Its last $n$ coordinates are $\varepsilon=\sqrt{N'}\eta_{2}/\lVert\eta\rVert$, and its first $N$ coordinates are $\chi(\varepsilon)\sigma$ with $\sigma=\sqrt{N}\eta_{1}/\lVert\eta_{1}\rVert$. The vector $\sigma$ is uniform on $S_{N}$ and independent of $(\lVert\eta_{1}\rVert,\eta_{2})$, hence of $\varepsilon$. The variable $\lVert\varepsilon\rVert^{2}/N'=\lVert\eta_{2}\rVert^{2}/\lVert\eta\rVert^{2}$ has the beta distribution with parameters $n/2$ and $N/2$, and the direction of $\varepsilon$ is uniform and independent of its norm. Writing the beta density in polar coordinates on $\mathbb{R}^{n}$ gives (89). For $\Lambda\ge1$ let

Stirling’s formula, $\Gamma(N'/2)/\Gamma(N/2)=(N'/2)^{n/2}(1+O_n(N^{-1}))$, and the expansion $(N/2-1)\log(1-\lVert\varepsilon\rVert^2/N')=-\lVert\varepsilon\rVert^2/2+O_{n,\Lambda}(N^{-1})$ give, uniformly on $\mathbb{B}_{\Lambda}$,

[ref-37] separates the cavity coordinates of spherical models by a decomposition of the uniform measure of the same kind. The expansion of $f_{N,n}$ reflects the classical fact that a fixed number of coordinates of a uniform point on a high-dimensional sphere are asymptotically independent standard Gaussian variables; see [ref-47].

Gaussian matrices. We use the following standard bound on the operator norm of a Gaussian matrix [ref-130], Corollary 5.35, here and in Chapters 5 and 8.

Lemma 4.2. Let $A$ be an $M\times N$ matrix with independent standard Gaussian entries. For every $t\ge0$,

Consequently $\mathbb{E}\lVert A\rVert_{\mathrm{op}}^2\le4(M+N)+8$, and $\mathbb{E}\exp(c\lVert A\rVert_{\mathrm{op}}^2/N)\le3\exp(4c(M+N)/N)$ for $0<c\le N/8$.

Proof. The tail bound is the upper half of [ref-130], Corollary 5.35. Put $T=(\lVert A\rVert_{\mathrm{op}}-\sqrt{M}-\sqrt{N})_+$, so that $\mathbb{P}(T>t)\le2e^{-t^2/2}$ and $\lVert A\rVert_{\mathrm{op}}^2\le4(M+N)+2T^2$. Then $\mathbb{E}T^2=\int_0^\infty2t\mathbb{P}(T>t)\,\mathrm{d}t\le4$, which gives the second bound. For $0<s\le1/4$,

and the choice $s=2c/N$ gives the third bound. $\square$

Poissonization. We replace the number $M_N$ of rows by a Poisson variable $M\sim\operatorname{Poi}(\alpha N)$, and when passing to dimension $N'$ we add independently $\pi\sim\operatorname{Poi}(\alpha n)$ rows, so that the $N'$-dimensional system has $M+\pi\sim\operatorname{Poi}(\alpha N')$ rows; from now on the letter $\pi$ denotes this count. This changes the free energy by $o(1)$. To check this, let $h_N(j)$ be the expected log partition function in dimension $N$ with $j$ rows. Adding a fresh row $g$ changes $h_N$ by $\mathbb{E}\log\langle e^{U(g\cdot\sigma/\sqrt{N})}\rangle$, where $\langle\cdot\rangle$ is the Gibbs measure before the row is added. By Jensen’s inequality and $U\le U_{\max}$, this lies between $\mathbb{E}U(G)$ and $U_{\max}$, because $g\cdot\sigma/\sqrt{N}$ is standard Gaussian for every fixed $\sigma$ and $g$ is independent of the Gibbs measure. By (88), $\lvert\mathbb{E}U(G)\rvert<\infty$. Hence $h_N$ is Lipschitz with a constant independent of $N$, and

Poisson numbers of constraints are standard in the study of diluted spin glasses [ref-57, ref-102].

Conditioning on the number of rows. The concentration estimates below are proved at a fixed number of rows, so we carry out the cavity computation conditionally on the value of $M$, for values in the window

The Poisson tail bound $\mathbb{P}(|M-\alpha N|>t)\leq2\exp(-t^2/(2(\alpha N+t)))$ gives $\mathbb{P}(M\notin W_N)\leq2e^{-cN^{1/3}}$, and for large $N$ every $j\in W_N$ satisfies $j\leq2\alpha N$. Step 2 of §4.7 reduces the proof to one value of $M$ in $W_N$. From §4.3 on, $M$ denotes a fixed integer in $W_N$, expectations are conditional on this value, and all bounds and all terms $o(1)$ are uniform in $M\in W_N$; the exceptions are Lemma 4.7(i) and the quantities $\Xi_N$ of Step 1 of §4.7, which concern the Poissonized systems. The number $\pi\sim\operatorname{Poi}(\alpha n)$ of rows added in dimension $N'$ remains Poisson throughout.

The three systems. We compare three systems. Each has $M$ rows, and the $N'$-dimensional system has $\pi$ further rows.

(a) The $N$-dimensional system has configuration space $S_N$, rows $g_a^{(N)}\in\mathbb{R}^N$ for $a\leq M$, and Hamiltonian $\sum_{a\leq M}U(g_a^{(N)}\cdot\sigma/\sqrt{N})$.

(b) The $N'$-dimensional system has configuration space $S_{N'}$ and rows $(g_a,g'_a)\in\mathbb{R}^N\times\mathbb{R}^n$ for $a\leq M$, together with $\pi$ further rows $\widetilde{g}_i\in\mathbb{R}^{N'}$. We write $\langle\cdot\rangle_{N'}$ for its Gibbs measure with the first $M$ rows only.

(c) The reference system has configuration space $S_N$, the rows $g_a\in\mathbb{R}^N$ for $a\leq M$ (the first $N$ coordinates of the rows of the $N'$-dimensional system), and the fields

All rows are independent standard Gaussian vectors. We write $\mathbf{G},\mathbf{G}'$ for the $M\times N$ and $M\times n$ matrices with rows $g_a$ and $g'_a$. In the coordinates $\rho=\rho(\sigma,\varepsilon)$ the fields of the $N'$-dimensional system are

The fields of the $N$-dimensional system can be realized from the same rows. Let $g''_a\in\mathbb{R}^N$ be independent standard Gaussian vectors, forming the matrix $\mathbf{G}''$, and put

As processes indexed by $\sigma\in S_N$, the centered Gaussian families $(S_a+\eta_a)_{a\leq M}$ and $(g_a^{(N)}\cdot\sigma/\sqrt{N})_{a\leq M}$ have the same covariance $\sigma\cdot\sigma'/N$, hence the same law. Both the $N'$-dimensional and the $N$-dimensional partition functions are therefore integrals against the reference system with modified fields.

Observables of the reference system. The following quantities determine the coordinate term:

The spin overlap is $R_N$. We call $X_N$ the response overlap; it is a Gram matrix, hence positive semidefinite, and its diagonal is $X_N(\sigma,\sigma)=N'^{-1}\sum_a U'(S_a(\sigma))^2$. The terms $A_N$ and $B_N$ arise below from the radial factor $\chi(\varepsilon)$ and from the second-order term of a Gaussian integration by parts, respectively, and $b_N-1$ will be the coefficient of $-(\lVert\varepsilon\rVert^2-n)/2$ in the exponent of the Gaussian integral over $\varepsilon$.

Let $\Omega_N$ be the event on which the operator norms of $\mathbf{G}$, $\mathbf{G}''$, and $\mathbf{G}'$ are at most $C_1\sqrt{N}$. Since $M \le2\alpha N$, Lemma 4.2 with $t=\sqrt{N}$ shows that $\mathbb{P}(\Omega_N^c)\le6e^{-N/2}$ for $N \ge n$, with the constant $C_1=\sqrt{2\alpha}+4$. Since $\sum_a S_a(\sigma)^2=\lVert\mathbf{G}\sigma\rVert^2/N' \le\lVert\mathbf{G}\rVert_{\mathrm{op}}^2$, the growth bounds in (4.1) give, for every $\sigma\in S_N$,

On $\Omega_N$ the right sides are at most $CN$ and $C$. By Lemma 4.2, for every $c>0$ and $N \ge N_0(c)$,

uniformly in $M \le2\alpha N$. In particular, all polynomial moments of these quantities and of $\lVert\mathbf{G}\rVert_{\mathrm{op}}^2/N$ are bounded.

The overlap representation theorem

The cavity computation requires a description of the limiting joint law of the spin overlap $R_N$ and the response overlap $X_N$. We use a general result about random probability measures on Hilbert spaces, which does not involve the perceptron Hamiltonian. In this section we state it, recall the three external results on which it rests, and derive it from them.

Setting. For each $N \ge1$, let $\mathfrak{G}_N$ be a random probability measure on the product of the unit balls of two separable Hilbert spaces, which may depend on $N$. For independent samples $(x_N^\ell,y_N^\ell)_{\ell\ge1}$ from $\mathfrak{G}_N$ put

We write $\mathbb{E}\langle\cdot\rangle_{N}$ for expectation over the random measure and its samples. The arrays take values in $[-1,1]$, they are positive semidefinite, and they are invariant in law under finite permutations of the sample labels. Suppose that the pair of arrays converges in finite-dimensional distribution to a pair $(R^{1},R^{2})$, and write $\mathbb{E}\langle\cdot\rangle$ for expectations of the limiting arrays. The limiting arrays have the same three properties. For $r\geq1$, rational $v_{1},v_{2}\geq0$ and an integer $m\geq1$ put $Q(x,y)=(v_{1}x+v_{2}y)^{m}$. For a bounded measurable function $f$ of the arrays of the first $r$ samples, the Ghirlanda–Guerra discrepancy of §3.4, written with $r$ replicas since $n$ is the cavity dimension here, is

Here $f$ may depend on the diagonal entries of the arrays. For a single random measure $\mathfrak{G}$ we drop the index $N$ and write $\mathscr{D}_{r}(f,Q)$.

Ruelle probability cascade arrays with paired levels. We use the Ruelle probability cascade of §3.1: a depth $k\geq0$, parameters $0=\theta_{0}<\theta_{1}<\cdots<\theta_{k}<\theta_{k+1}=1$, masses $w_{l}=\theta_{l+1}-\theta_{l}$ for $0\leq l\leq k$, weights $(v_{\beta})$ indexed by the leaves $\beta$ of the infinitely branching tree of depth $k$, and the depth $\beta\wedge\beta'$ of the last common ancestor of two leaves. If $\beta^{1}$ and $\beta^{2}$ are sampled independently from the weights $(v_{\beta})$, then the averaged probability that $\beta^{1}\wedge\beta^{2}=l$ is $w_{l}$, by (3.4) with the leaf function zero. Given levels $0\leq q_{0}\leq\cdots\leq q_{k}$ and $0\leq p_{0}\leq\cdots\leq p_{k}$, the Ruelle probability cascade array with paired levels $(q_{l},p_{l})$ is the pair of arrays with off-diagonal entries $(q_{\beta^{\ell}\wedge\beta^{\ell'}},p_{\beta^{\ell}\wedge\beta^{\ell'}})$, $\ell\neq\ell'$, where $\beta^{1},\beta^{2},\ldots$ are independent samples from $(v_{\beta})$. The law of its pair of $(1,2)$ entries is $\sum_{l=0}^{k}w_{l}\delta_{(q_{l},p_{l})}$. A single sequence of levels $0\leq s_{0}<\cdots<s_{k}$ gives the Ruelle probability cascade array with levels $(s_{l})$, whose overlap law is $\sum_{l}w_{l}\delta_{s_{l}}$; every probability measure with finite support in $[0,\infty)$ is of this form.

External results. We use three results from the literature. The first is the Dovbysh–Sudakov representation [ref-51] of positive semidefinite exchangeable arrays, in the form of [ref-96]. A random array $(R_{\ell\ell'})_{\ell,\ell'\geq1}$ is called a Gram–de Finetti array if it is symmetric, every finite principal submatrix is positive semidefinite, and its law is invariant under finite permutations of the indices.

Lemma 4.3. Let $(R_{\ell\ell'})$ be a Gram–de Finetti array with $R_{\ell\ell}\leq1$ almost surely. There is a random probability measure $\mathfrak{G}$ on the unit ball of a separable Hilbert space such that, for independent samples $\sigma^{1},\sigma^{2},\ldots$ from $\mathfrak{G}$, the off-diagonal array $(R_{\ell\ell'})_{\ell\neq\ell'}$ has the law of $(\sigma^{\ell}\cdot\sigma^{\ell'})_{\ell\neq\ell'}$.

Proof. By [ref-96], the law of the array is a mixture over a random parameter $\omega$ of the laws of $(h_{\ell}\cdot h_{\ell'}+a_{\ell}\delta_{\ell\ell'})_{\ell,\ell'}$, where $(h_{\ell},a_{\ell})$ are independent samples from a probability measure $\eta_{\omega}$ on $\ell^{2} \times[0,\infty)$. Since $\lVert h_{\ell}\rVert^{2} \le\lVert h_{\ell}\rVert^{2}+a_{\ell}=R_{\ell\ell}\le1$ almost surely, $\eta_{\omega}$ is concentrated on pairs with $\lVert h\rVert\le1$ for almost every $\omega$. The law of the first coordinate under $\eta_{\omega}$ is the required random measure. $\square$

The second result, from [ref-49], Sections 5.6–5.7, characterizes the scalar arrays that satisfy the Ghirlanda–Guerra identities. Let $\mathcal{G}$ be a random probability measure on the unit ball of a separable Hilbert space, with overlap array $R=(\sigma^{\ell}\cdot\sigma^{\ell'})_{\ell\ne\ell'}$ and overlap law $\zeta(A)=\mathbb{E}\langle\mathbf{1}_{\{R_{12}\in A\}}\rangle$. We say that $\mathcal{G}$ satisfies the Ghirlanda–Guerra identities [ref-64] if for every $r\ge1$, every bounded measurable function $f$ of $(R_{\ell\ell'})_{\ell\ne\ell'\le r}$, and every bounded measurable $\psi:\mathbb{R}\to\mathbb{R}$,

Part (a) of the following lemma is Panchenko’s characterization of such measures, which rests on his proof of ultrametricity [ref-97] (see also [ref-98], Theorem 2.14); the positivity of the overlap is Talagrand’s positivity principle (see [ref-98], Theorem 2.16). The identities for Ruelle probability cascades used in part (b), [ref-49], Theorem 5.28, are due to Bovier and Kurkova [ref-24].

Lemma 4.4. Let $\mathcal{G}$ satisfy the Ghirlanda–Guerra identities, with overlap array $R$ and overlap law $\zeta$.

(a) The measure $\zeta$ is supported in $[0,1]$, and the law of $R$ is determined by $\zeta$.

(b) If $\zeta_{h}$, $h\ge1$, are probability measures with finite support in $[0,1]$ that converge weakly to $\zeta$, then the Ruelle probability cascade arrays whose overlap laws are $\zeta_{h}$ converge in finite-dimensional distribution to $R$.

Proof. Part (a) is [ref-49], Theorem 5.29, together with $|R_{12}|\le1$. For (b), [ref-49], Corollary 5.32 shows that the cascade arrays converge in finite-dimensional distribution (the Poisson–Dirichlet cascades of [ref-49] are the Ruelle probability cascades of §3.1). The limit has overlap law $\zeta$, and it satisfies the identities (97): each cascade array satisfies them by [ref-49], Theorem 5.28, so the limit satisfies them for continuous $f$ and $\psi$, and hence for bounded measurable $f$ and $\psi$, because for fixed $\psi$ (respectively $f$) both sides are integrals against finite signed measures. The cascade array with levels $s_{0}<\cdots<s_{k}$ in $[0,1]$ is the off-diagonal part of the Gram array of independent samples from the random measure $\sum_{\beta}v_{\beta}\delta_{h_{\beta}}$, where $h_{\beta}=\sum_{l=0}^{k}(s_{l}-s_{l-1})^{1/2}e_{\beta|l}$ with $s_{-1}=0$ and an orthonormal family $(e_{\nu})$ indexed by the nodes; its diagonal entries equal $s_{k}\le1$. Along a subsequence these Gram arrays converge in finite-dimensional distribution, and the limit is a Gram–de Finetti array with diagonal entries at most one whose off-diagonal part is the limit of the cascade arrays. By Lemma 4.3 this off-diagonal part is the overlap array of a random measure on a unit ball, and this measure satisfies the Ghirlanda–Guerra identities, since these are statements about the law of the off-diagonal array. By (a), the limit has the law of $R$. $\square$

The third result is Mourrat’s quantitative synchronization theorem [ref-92], Theorem 5.3, a finitary variant of Panchenko’s synchronization theorem; see §3.5. For a real random variable $Y$ let $F_Y^{-1}(v)=\inf\{s\in\mathbb{R}:\mathbb{P}(Y\le s)\ge v\}$ for $v\in[0,1]$, and let $V$ be uniform on $[0,1]$. The pair $(F_{Y_1}^{-1}(V),F_{Y_2}^{-1}(V))$ is the monotone coupling of the laws of $Y_1$ and $Y_2$.

Lemma 4.5. Fix an enumeration $(\vartheta_i)_{i\ge1}$ of $\mathbb{Q}\cap[0,1]$, and put $c_j(x,y)=(\vartheta_{j_1}x+\vartheta_{j_2}y)^{j_3}$ for $j=(j_1,j_2,j_3)$, as in Lemma 3.8.

(a) For every $\epsilon>0$ there is $\delta>0$ with the following property. Let $\mathfrak{G}$ be a random probability measure on the product of the unit balls of two Hilbert spaces, with arrays $R^1,R^2$ as above. Suppose that $|\mathcal{D}_r(f,c_j)|\le\delta$ for all $r,j_1,j_2,j_3\le\lfloor\delta^{-1}\rfloor$ and all continuous functions $f$ of the arrays of $r$ samples with $\lVert f\rVert_\infty\le1$, where $\mathcal{D}_r$ is (4.10) for $\mathfrak{G}$. Then, for every $\varphi\in C^\infty(\mathbb{R}^2)$,

where $F_a^{-1}$ is the function $F_Y^{-1}$ for the law of $R_{12}^a$ under $\mathbb{E}\langle\cdot\rangle$.

(b) If $Y_{1,N}\to Y_1$ and $Y_{2,N}\to Y_2$ in law, then the monotone couplings converge in law:

Proof. Part (a) is [ref-92], Theorem 5.3, whose hypothesis is formulated with the same defects, for continuous test functions of the arrays of $r$ samples including their diagonal entries. Part (b) is [ref-92], Lemma 5.4. □

Theorem 4.6. In the setting above, assume that for every $r\ge1$, every integer $m\ge1$, and all rational $v_1,v_2\ge0$,

where the supremum is over bounded measurable functions of the arrays of the first $r$ samples. Then the limiting pair satisfies the Ghirlanda–Guerra identities

for every bounded measurable $f$ of the arrays of its first $r$ samples. Moreover:

(i) The off-diagonal entries of $R^1$ and $R^2$ are nonnegative almost surely.

(ii) The off-diagonal pair of arrays is a limit in finite-dimensional distribution of Ruelle probability cascade arrays with paired nondecreasing levels $(q_l,p_l)\in[0,1]^2$.

Part (ii) is the form, for a pair of Gram arrays, of Panchenko’s synchronization combined with his approximation by Ruelle probability cascades [ref-100]. Step 1 of the proof follows the proof of [ref-100], Theorem 3 in Section 3 there, Step 2 the positivity argument of [ref-100], Section 4, and Step 4 the approximation by cascades in [ref-100], Section 5. In Step 3 we use Mourrat’s quantitative form, Lemma 4.5(a). It applies to the Gram pairs at finite $N$, before passing to the limit, so no representing measure of the limiting arrays needs to preserve their diagonal values.

Proof. We divide the proof into four steps.

Step 1. The identities in the limit. For continuous $f$, finite-dimensional convergence and (98) give (99), since all arrays take values in $[-1,1]$. Both sides of (99) are integrals of $f$ against finite signed measures on the arrays of $r$ samples, and they agree on continuous functions, so they agree for bounded measurable $f$. Choosing $(v_{1},v_{2})=(1,0)$, $(0,1)$ and $(1/2,1/2)$ gives (97) with $\psi(x)=x^{m}$, $m\geq1$, for the scalar arrays $R^{1}$, $R^{2}$, and $S/2$, where $S=R^{1}+R^{2}$, and for test functions $f$ of their off-diagonal entries. It also holds for constant $\psi$, since the sum of its three coefficients is one. By Weierstrass approximation it holds for continuous $\psi$ on $[-1,1]$, and, again by equality of finite measures, for bounded measurable $\psi$.

Step 2. Positivity. Each of the arrays $R^{1}$, $R^{2}$, and $S/2$ is a Gram–de Finetti array with diagonal entries at most one, as a limit of such arrays. By Lemma 4.3, its off-diagonal part is the overlap array of a random measure on a unit ball, and this measure satisfies the Ghirlanda–Guerra identities by Step 1. Lemma 4.4(a) gives (i), and it shows that the law of the off-diagonal array $S/2$ is determined by the law of $S_{12}/2$.

Step 3. Monotone coupling. Let $\epsilon>0$, and let $\delta>0$ be given by Lemma 4.5(a). The hypothesis of Lemma 4.5(a) involves finitely many tuples $(r,j_{1},j_{2},j_{3})$, and by (98) it holds for $\mathfrak{G}_{N}$ for all sufficiently large $N$. Hence, for $\varphi\in C^{\infty}(\mathbb{R}^{2})$,

where $F_{a,N}^{-1}$ is defined from the law of $R_{12}^{a,N}$. As $N\to\infty$, the first term converges to $\mathbb{E}\langle\varphi(R_{12}^{1},R_{12}^{2})\rangle$, and by Lemma 4.5(b) the second converges to $\mathbb{E}\varphi(F_{1}^{-1}(V),F_{2}^{-1}(V))$. Since $\epsilon$ is arbitrary, the law of $(R_{12}^{1},R_{12}^{2})$ is the monotone coupling of its marginals. The maps $F_{1}^{-1}$ and $F_{2}^{-1}$ are nondecreasing, so the image of $(0,1)$ under $v\mapsto(F_{1}^{-1}(v),F_{2}^{-1}(v))$ is totally ordered for the coordinatewise order, and so is its closure. Thus the support $C$ of the law of $(R_{12}^{1},R_{12}^{2})$, a compact subset of $[0,1]^{2}$ by (i), is totally ordered. Two points $(x,y)$ and $(x',y')$ of $C$ with $x+y\leq x'+y'$ therefore satisfy

In particular the sum determines both coordinates: on the compact set $\{x+y:(x,y)\in C\}$, which is the support of the law of $S_{12}$, there are nondecreasing 1-Lipschitz functions $Q_{1},Q_{2}$ with $(x,y)=(Q_{1}(x+y),Q_{2}(x+y))$ on $C$. We extend them to nondecreasing 1-Lipschitz functions on $\mathbb{R}$, by linear interpolation across the gaps of this compact set and by constants outside its hull. Almost surely $R_{12}^{1}=Q_{1}(S_{12})$ and $R_{12}^{2}=Q_{2}(S_{12})$. By exchangeability and a countable intersection, the same holds for every off-diagonal pair of indices.

Step 4. Approximation by cascades. Let $F^{-1}$ be the function $F_{\gamma}^{-1}$ for the law $\zeta_{S}$ of $S_{12}/2$, and let $\zeta_{h}$ be the law of $F^{-1}(\lceil hV\rceil/h)$. It has finite support contained in the support of $\zeta_{S}$, and it converges weakly to $\zeta_{S}$ as $h\to\infty$, since $0\leq\lceil hV\rceil/h-V<1/h$ almost surely and $F^{-1}$ is continuous at almost every point of $(0,1)$. By Steps 1 and 2 and Lemma 4.4(b), the cascade arrays with overlap laws $\zeta_{h}$, with levels $s_{0}^{h}<\cdots<s_{k}^{h}$ in $[0,1]$, converge in finite-dimensional distribution to $(S_{\ell\ell'}/2)_{\ell\ne\ell'}$. Apply the maps $s\mapsto(Q_{1}(2s),Q_{2}(2s))$ to their entries. The resulting arrays are the cascade arrays with the paired levels $(q_{l},p_{l})=(Q_{1}(2s_{l}^{h}),Q_{2}(2s_{l}^{h}))$, which are nondecreasing in $l$ and lie in $[0,1]^{2}$, because $2s_{l}^{h}$ lies in the support of the law of $S_{12}$. By continuity of $Q_{1}$ and $Q_{2}$ they converge in finite-dimensional distribution to $(Q_{1}(S_{\ell\ell'}),Q_{2}(S_{\ell\ell'}))_{\ell\ne\ell'}=(R_{\ell\ell'}^{1},R_{\ell\ell'}^{2})_{\ell\ne\ell'}$. This proves (ii). $\square$

Perturbations, continuity, and the precision identity

The cavity computation needs three properties of the Gibbs measures, which we obtain by adding a small perturbation to each Hamiltonian. First, the Ghirlanda–Guerra identities (99) for the spin and response overlaps must hold in the limit, so that Theorem 4.6 applies. Second, the per-configuration observables $b_{N}$ and $X_{N}(\sigma,\sigma)$ must concentrate, so that they can be replaced by constants in the limit. Third, the perturbations must be compatible with adding coordinates and rows. In this section we define the perturbation and state its properties (Lemma 4.7), a continuity lemma for the functionals that appear in the cavity computation (Lemma 4.8), and an identity for the mean precision (Lemma 4.9). We then explain how these lemmas are combined with Theorem 4.6. The three lemmas are proved in §4.8. Every assertion in this section concerns a fixed number $n$ of added coordinates and a fixed cutoff $\Lambda$ for their squared norm; removing the cutoff is a separate step.

The perturbation. Consider a $d$-dimensional system, $d\in\{N,N'\}$, with configurations $\rho\in S_{d}$ and rows $g_{a}\in\mathbb{R}^{d}$ indexed by a finite set $\mathcal{A}$ (in the $N'$-dimensional system, $\mathcal{A}=\{1,\ldots,M\}$ for the Gibbs measure $\langle\cdot\rangle_{N'}$, and $\mathcal{A}$ also contains the $\pi$ added rows when these are present), and write $x_{a}(\rho)=g_{a}\cdot\rho/\sqrt{d}$. The perturbation has two parts.

Scalar part. Put

and add $\zeta_{d}\sum_{a\in\mathcal{A}}(u_{1}\tau_{1}+u_{2}\tau_{2})(x_{a}(\rho))$ to the Hamiltonian, where $u_{1},u_{2}\in[1,2]$ are parameters. Equivalently, the activation $U$ is replaced by

By (87), $|\tau_i(x)| \le C_\tau(1+x^2)$ for a constant $C_\tau$ depending only on $L$, and the first and second derivatives of $\tau_i$ grow at most linearly. The derivative of the Hamiltonian in $u_i$ is $\widetilde{\xi}_d \sum_a \tau_i(x_a(\rho))$. For the reference fields these sums are exactly

so concentration of these derivatives gives concentration of $X_N(\sigma,\sigma)$ and $b_N$.

Gaussian part. We first bound the response vectors without changing them on a typical event. Fix a smooth function $\psi_0:[0,\infty)\to(0,1]$ with $\psi_0=1$ on $[0,3/4]$, $\psi_0(t)\le1/t$ for $t\ge1$, and $\psi_0(t)=1/t$ for $t\ge2$, and define $\varpi(y)=\psi_0(\lVert y\rVert)y$ on every Euclidean space. Then $\varpi$ equals the identity on the ball of radius $3/4$, takes values in the closed unit ball, and its first and second derivatives are bounded by a constant that does not depend on the dimension. Put

where $C_X$ is a constant depending only on $\alpha$ and $L$. We choose $C_X$ so large that, on the event $\Omega_N$, the vector $\Psi_d(\rho)$ with $\mathcal A=\{1,\ldots,M\}$ has norm at most $1/\sqrt{2}$ for every $\rho\in S_d$ in each of the three systems. We also require the same bound in the $N'$-dimensional system when $\mathcal A$ contains the $\pi$ added rows as well, on the event

where $\widetilde{\mathbf G}$ is the $\pi\times N'$ matrix with rows $\widetilde{g}_i$. This is possible because $\lVert\Psi_d(\rho)\rVert^2\le2L^2(\#\mathcal A+\sum_a x_a(\rho)^2)/(C_Xd)$ by (87), on $\Omega_N$ the fields of the three systems ((a)–(c) of §4.1, with the fields $S_a+\eta_a$ for the $N$-dimensional system) satisfy $\sum_{a\le M}x_a(\rho)^2\le CN$, and the fields of the added rows satisfy $\sum_{i\le\pi}(\widetilde{g}_i\cdot\rho)^2/N'\le\lVert\widetilde{\mathbf G}\rVert_{\mathrm{op}}^2$. By Lemma 4.2 and the Poisson tail of $\pi$, $\mathbb P(\Omega_N'{}^{c})\le C_ne^{-N/2}$. Set

Since $1/\sqrt{2}<3/4$, we have $\widehat{\Psi}_d=\Psi_d$ on $\Omega_N$ when $\mathcal A=\{1,\ldots,M\}$. Hence, on $\Omega_N$, $\widehat{X}_d$ is the response overlap $d^{-1}\sum_a U'(x_a(\rho))U'(x_a(\rho'))$ of the $d$-dimensional system divided by $C_X$.

Enumerate the pairs $v=(v_1,v_2)$ of nonnegative rationals with $v_1+v_2\le1$ as $v^{(1)},v^{(2)},\ldots$, write $j(v)$ for the index of $v$, and put $c_{v,m}=2^{-j(v)-m}$ for integers $m\ge1$. Let $h_{v,m}$ be independent (conditionally on the rows) centered Gaussian processes on $S_d$ with covariance

These processes exist. Indeed, by the binomial theorem

and $R(\rho,\rho')^{i}\widehat{X}_{d}(\rho,\rho')^{m-i}$ is the inner product of the tensors $(\rho/\sqrt{d})^{\otimes i}\otimes\widehat{\Psi}_{d}(\rho)^{\otimes(m-i)}$ and $(\rho'/\sqrt{d})^{\otimes i}\otimes\widehat{\Psi}_{d}(\rho')^{\otimes(m-i)}$. Hence

where $T_{v,m}=(T_{v,m,i})_{0\le i\le m}$ is a family of independent standard Gaussian tensors of the matching shapes, has covariance $F_{v,m}(\rho)\cdot F_{v,m}(\rho')=Q_{v,m}(\rho,\rho')$. Every diagonal covariance satisfies $\lVert F_{v,m}(\rho)\rVert^{2}=Q_{v,m}(\rho,\rho)\le1$, since $R(\rho,\rho)=1$, $\widehat{X}_{d}(\rho,\rho)\le1$ and $v_{1}+v_{2}\le1$. With $s_{d}=d^{1/3}$ and parameters $u_{v,m}\in[1,2]$ we add

to the Hamiltonian. The series converges almost surely, uniformly in $\rho$ (see eq:4.41 below). The Gaussian part is modeled on Panchenko’s perturbation for the multi-species Sherrington–Kirkpatrick model [ref-100], which has weights of the same form $2^{-j-m}$, parameters uniform on $[1,2]$, and a strength $N^{\gamma}$ with $1/4<\gamma<1/2$ (here $s_{d}=d^{1/3}$). In [ref-100] the covariance is a power of a weighted sum of the overlaps within the species. Here it is a power of a weighted sum of the spin overlap and the overlap of the response vectors, which the cutoff $\varpi$ keeps in the unit ball.

Parameters and the three systems. The parameters $u_{1},u_{2}$ and $u_{v,m}$ are independent and uniform on $[1,2]$, and they take the same values in the $N$-dimensional, the $N'$-dimensional, and the reference system. The first two systems use the construction above with $d=N$ and $d=N'$. The reference system uses the scalar part $\widetilde{\xi}_{N'}\sum_{a\le M}(u_{1}\tau_{1}+u_{2}\tau_{2})(S_{a}(\sigma))$, and as its Gaussian part it uses, by definition, the Gaussian part of the $N'$-dimensional system with the rows $a\le M$, evaluated at $\rho(\sigma,0)=(\sqrt{N'/N}\,\sigma,0)$. This common restriction is what makes the comparison in Lemma 4.7(iv) possible. We have $R(\rho(\sigma,0),\rho(\sigma',0))=R_{N}(\sigma,\sigma')$ and $x_{a}(\rho(\sigma,0))=\sqrt{N'/N}\,S_{a}(\sigma)$. Thus the response vector of the reference perturbation is built from $U'(\sqrt{N'/N}\,S_{a})$ rather than $U'(S_{a})$. Since $\lvert U'(\sqrt{N'/N}\,S)-U'(S)\rvert\le L\lvert S\rvert n/N$, the Cauchy–Schwarz inequality shows that on $\Omega_{N}$

Finally, the reference system is tilted by $\exp(nB_{N}(\sigma)/2)$. We write $\langle\cdot\rangle_{\mathrm{ref}}$ for the Gibbs measure of the perturbed and tilted reference system. The tilt will cancel between the two cavity terms in §4.4.

Invariance. The Gaussian tensors are invariant in law under orthogonal transformations of their spin slots, and $\Psi_d(O\rho)$ computed with the rows $g_a$ equals $\Psi_d(\rho)$ computed with the rows $O^{\mathsf T}g_a$. Since the rows are standard Gaussian vectors, the perturbed $N'$-dimensional system is invariant in law under a simultaneous orthogonal transformation of the configurations and of the rows. In particular, for every orthogonal $O$ of $\mathbb{R}^{N'}$ and every bounded measurable $F$,

Properties of the perturbation. For a system with Gibbs measure $\langle\cdot\rangle$ and a Gaussian component with covariance $Q=Q_{v,m}$, write $Q_{\ell\ell'}=Q(\rho^\ell,\rho^{\ell'})$. For a bounded measurable function $f$ of the arrays $(R(\rho^\ell,\rho^{\ell'}),\widehat{X}_d(\rho^\ell,\rho^{\ell'}))_{\ell,\ell'\le r}$ of $r$ replicas, the defect in the identity (99) is $|\widehat{\mathscr{D}}_r(f,Q)|$, where $\widehat{\mathscr{D}}_r$ is the discrepancy (96) of the pair $(R,\widehat{X}_d)$ with $(v_1,v_2)=v$; its terms $Q(R_{\ell\ell'}^1,R_{\ell\ell'}^2)$ are then the values $Q_{\ell\ell'}$. For the reference system the arrays are those of the points $\rho(\sigma^\ell,0)$.

Lemma 4.7. Fix $n$ and $\Lambda$. In the following statements $o(1)$ denotes a quantity that tends to zero as $N\to\infty$, uniformly in the perturbation parameters and in $M\in W_N$, except where a statement specifies otherwise: in (ii) the defects tend to zero after averaging over the parameters, and in (iii) uniformly on the sets $\mathcal{G}_N(M)$.

(i) For the $N$-dimensional system with $M\sim\operatorname{Poi}(\alpha N)$ rows, the perturbation changes the expected log partition function by $o(N)$.

(ii) For the reference system let $A_i(\sigma)=N'^{-1}\sum_{a\le M}\tau_i(S_a(\sigma))$, $i=1,2$. After averaging over the perturbation parameters, $\mathbb{E}\langle|A_i-\mathbb{E}\langle A_i\rangle_{\mathrm{ref}}|\rangle_{\mathrm{ref}}\to0$. By (101), the observables $b_N$ and $X_N(\sigma,\sigma)$ therefore concentrate under $\langle\cdot\rangle_{\mathrm{ref}}$ about their expectations, and so does the diagonal entry $\widehat{X}_{N'}(\rho(\sigma,0),\rho(\sigma,0))$ of its perturbation.

(iii) There are sets $\mathcal{G}_N=\mathcal{G}_N(M)$ of perturbation parameters with $\inf_{M\in W_N}\mathbb{P}(\mathcal{G}_N(M))\to1$ such that

for the reference system and every fixed $r$ and $(v,m)$, and for the $N'$-dimensional system with the Gibbs measure $\langle\cdot\rangle_{N'}$ and every fixed $r$ and $(v,m)$ with $v_2=0$, and such that the concentration defects of (ii), without the average over the parameters, also tend to zero uniformly on $\mathcal{G}_N(M)$ and in $M\in W_N$. The sets $\mathcal{G}_N(M)$ do not depend on $\Lambda$.

(iv) Comparison with the reference perturbation. (a) In the partition function of the $N'$-dimensional system with $M$ rows, restricted to $\varepsilon\in\mathbb{B}_\Lambda$, replacing the Gaussian part of the perturbation at $\rho(\sigma,\varepsilon)$ by its value at $\rho(\sigma,0)$ changes the expected logarithm by $o(1)$, and changes the Gibbs expectations of bounded measurable functions of finitely many spin overlaps and cavity coordinates by $o(1)$ times their supremum norm. (b) In the representation of the $N$-dimensional system through the fields $S_a+\eta_a$ of (92), replacing its Gaussian perturbation by that of the reference system, and the strengths $s_N,\zeta_N$ by $s_{N'},\zeta_{N'}$, changes the expected log partition function by $o(1)$.

(v) Rows. Adding the $\pi$ further rows to the $N'$-dimensional system, including the corresponding change of its perturbation, increases the expected log partition function by at least

where $y_i(\rho)=\widetilde{g}_i\cdot\rho/\sqrt{N'}$ are the fields of the added rows. Moreover, for $N\geq N_0(n)$ the difference between the expected log partition functions of the $N'$-dimensional system with $M+\pi$ rows and of the $N$-dimensional system with $M$ rows is at least $-C_n$, uniformly in the perturbation parameters and in $M\in W_N$, and for every $N$ and every number $M$ of rows it is at most $C_n(N+M)$ in absolute value.

The second lemma provides continuity of the quantities in the cavity computation under convergence of overlap arrays. We apply it both to the Gibbs measures of §4.4 and to finite Ruelle probability cascades, so we state it for a random probability measure $\mathfrak{G}$ on a measurable space $\mathrm{E}$. We write $\langle\cdot\rangle$ for the average over independent samples $\rho^1,\rho^2,\ldots$ from $\mathfrak{G}$ (replicas) and $\mathbb{E}\langle\cdot\rangle$ for the expectation over $\mathfrak{G}$ and the replicas. Let $R$ and $X$ be random positive semidefinite kernels on $\mathrm{E}$ with $R(\rho,\rho)=1$, let $b$ be a random function on $\mathrm{E}$, and let $z:\mathrm{E}\to\mathbb{R}^n$ and $y,y_1,y_2,\ldots:\mathrm{E}\to\mathbb{R}$ be random fields, all defined on one probability space with $\mathfrak{G}$ and jointly measurable. For $r$ replicas write $A_r=(R(\rho^\ell,\rho^{\ell'}),X(\rho^\ell,\rho^{\ell'}),b(\rho^\ell))_{\ell,\ell'\leq r}$ for their arrays. For such an array $a$ let $\mathrm{N}_a$ be the centered Gaussian law of vectors $(z^\ell,y^\ell,y_1^\ell,y_2^\ell,\ldots)_{\ell\leq r}$ under which the families $z,y,y_1,y_2,\ldots$ are independent and

these matrices are positive semidefinite by the Schur product theorem. We assume:

(G) For every $r$ and every bounded measurable function $\Phi$ of $A_r$ and of the values at $\rho^1,\ldots,\rho^r$ of $z,y$, and finitely many $y_i$, $\mathbb{E}\langle\Phi\rangle=\mathbb{E}\langle\overline{\Phi}(A_r)\rangle$, where $\overline{\Phi}(a)$ is the integral of $\Phi(a,\cdot)$ against $\mathrm{N}_a$.

This holds, for instance, if $\mathrm{E}=S_d$ and, conditionally on $(\mathfrak{G},R,X,b)$, the fields are independent centered Gaussian processes with covariances $\delta_{ij}X$, $nRX$, and $R$, and are independent of the replicas. This is the situation in §4.4. The numerator, or cavity partition function on $\mathbb{B}_\Lambda$, and the denominator are

and the cavity measure is the random probability measure on $\mathrm{E}\times\mathbb{B}_\Lambda$ with density proportional to the integrand of $\mathcal{N}_\Lambda$ with respect to $\mathfrak{G}\otimes\nu_n$.

Lemma 4.8. Consider a sequence of such random measures satisfying (G) for which, for every $c>0$, the quantities

are bounded along the sequence from some index on, and suppose that the finite-dimensional distributions of the arrays converge. Then the following quantities converge, and their limits depend only on the limiting law of the arrays:

1. $\mathbb{E}\log\mathcal{N}_{\Lambda}$ and $\mathbb{E}\log\mathcal{D}$;

2. expectations under the cavity measure of bounded continuous functions of the arrays and the cavity coordinates of finitely many replicas;

3. the expected logarithmic increment $\mathbb{E}\log\left\langle\exp\left\{\sum_{i\leq\pi}U(y_i(\rho))\right\}\right\rangle$, for $\pi\sim\operatorname{Poi}(\alpha n)$ independent of the other variables, whenever $U$ is continuous and satisfies (88).

In particular, if $\mathbb{E}\langle|b-\bar b_N|\rangle\to0$ for constants $\bar b_N\to b_\infty$, then $b$ may be replaced by the constant $b_\infty$ in these limits.

Lemma 4.8 is the analogue for the perceptron of the continuity of the cavity functionals in the law of the overlap array, proved for mixed $p$-spin models in [ref-99] and for spherical models in [ref-37]. The difference is that the functionals here also depend on the response overlap and on the precision $b$, and that part (iii) treats the row term, whose activation may be unbounded below.

The third lemma is an identity for the mean of $b_N$ under the reference system, obtained by Gaussian integration by parts in the rows $g_a$; it turns $S_aU'(S_a)$ into $U''(S_a)+U'(S_a)^2$ minus a two-replica term. In the limit it shows that the smallest precision of the Gaussian integral over the added coordinates is at least one (Step 3 of §4.7).

Lemma 4.9. Fix $n$. For two replicas $\sigma^1,\sigma^2$ of the reference system, uniformly in the perturbation parameters and in $M\in W_N$,

Finite cascades as random measures. Lemma 4.8 is also applied to finite Ruelle probability cascades carrying prescribed diagonal values. Take a cascade as in §4.2, with depth $k$, weights $(v_\beta)$, and paired levels $0\leq q_0\leq\cdots\leq q_k\leq1$ and $0\leq p_0\leq\cdots\leq p_k$, and let $\bar p\geq p_k$ and $\bar b\in\mathbb{R}$. Independently of $(v_\beta)$, let $Y^p$, $Y^c$, and $Y^{q,1},Y^{q,2},\ldots$ be independent Gaussian cascade fields (§3.2), with values in $\mathbb{R}^n$, $\mathbb{R}$, and $\mathbb{R}$, and with levels $p$, $c=(nq_lp_l)_{l\leq k}$, and $q$, respectively; the levels $c_l$ are nonnegative and nondecreasing. The cascade realization with diagonal values $(1,\bar p,\bar b)$ is the random measure $\mathfrak{G}=\sum_\beta v_\beta\delta_\beta\otimes\nu_n\otimes\nu_1\otimes\nu_1^{\otimes\mathbb{N}}$ on $E=\mathbb{N}_*^k\times\mathbb{R}^n\times\mathbb{R}\times\mathbb{R}^{\mathbb{N}}$, with $b\equiv\bar b$, with kernels $R$ and $X$ equal to $q_{\beta\wedge\beta'}$ and $p_{\beta\wedge\beta'}$ at distinct points with leaves $\beta,\beta'$ and to $1$ and $\bar p$ at equal points, and with the fields

at $\rho=(\beta,x,x',x'')$. The within-state coordinates $x,x',x''$ carry the nonnegative differences between the diagonal values and the top levels. Two independent samples from $\mathfrak{G}$ are distinct points almost surely, so the arrays of the replicas are cascade arrays with paired levels $(q_l,p_l)$ off the diagonal, with diagonal values $1$, $\bar{p}$, and $\bar{b}$. Given the cascade weights and the leaves of the replicas, the values of the fields at the replicas are centered Gaussian with the covariances in (G), because the cascade fields are independent of the weights and the within-state coordinates are independent standard Gaussian. This gives (G), and the moment condition of Lemma 4.8 holds because $X(\rho,\rho)=\bar{p}$ and $\mathsf{b}=\bar{b}$ are constant.

How the lemmas are combined with the overlap theorem. Fix $M\in W_N$ and parameters in the set $\mathcal{G}_N(M)$ of Lemma 4.7(iii) (in §4.7, $M=M_N$ depends on $N$), and pass to a subsequence along which the finite-dimensional distributions of the arrays $(R_N,X_N,b_N)$ under $\mathbb{E}\langle\cdot\rangle_{\mathrm{ref}}$ converge; they are tight by (94) and (95). The reference system satisfies (G) with $\mathfrak{G}=\langle\cdot\rangle_{\mathrm{ref}}$, $R=R_N$, $X=X_N$, and $\mathsf{b}=b_N$, for fields that are, conditionally on the disorder, independent Gaussian processes with the covariances in (G); for instance

with independent standard Gaussian variables $g_{ai},g'_{ai},g''_{ji}$, independent of the reference system. By (95), it satisfies the moment condition of Lemma 4.8. We apply Theorem 4.6 to the Gram pairs generated by $\sigma/\sqrt{N}$ and $\widehat{\Psi}_{N'}(\rho(\sigma,0))$. Their limiting arrays have $R^1=R$ and $R^2=X/C_X$, where $(R,X)$ is the limit of $(R_N,X_N)$, for three reasons.

(a) Both feature vectors lie in unit balls, and on $\Omega_N$ the response Gram array differs from $X_N/C_X$ by $O(n/N)$, by (103); moreover $\mathbb{P}(\Omega_N^c)\to0$.

(b) The diagonals concentrate by Lemma 4.7(ii)–(iii), so the limiting diagonal values are $1$ and $p_\infty/C_X$, where

along a further subsequence.

(c) Lemma 4.7(iii) gives vanishing discrepancies $\mathfrak{D}_N^N(f,Q)$ of (96), uniformly over bounded tests, for each fixed number of replicas and each $Q=Q_{v,m}$, that is, for $v_1+v_2\leq1$. Homogeneity of degree $m$ extends the conclusion to every nonnegative rational pair $(v_1,v_2)$, so (98) holds.

By Theorem 4.6(ii), after multiplying the second levels of its approximating cascades by $C_X$, the off-diagonal limit of $(R_N,X_N)$ is a limit in finite-dimensional distribution of Ruelle probability cascade arrays with paired nondecreasing levels $(q_l,p_l)$. The observable $b_N$ concentrates about $\mathbb{E}\langle b_{N}\rangle_{\mathrm{ref}}$, which converges along a further subsequence to a constant $b_{\infty}$; by the last assertion of Lemma 4.8, $b_{N}$ may be replaced by $b_{\infty}$ in all limits.

In the limit, $0 \le R_{12} \le1$ and $0 \le X_{12} \le p_{\infty}$ almost surely. Nonnegativity is Theorem 4.6(i), and $R_{N} \le1$. For the last bound, the Cauchy–Schwarz inequality gives $X_{N}(\sigma^{1},\sigma^{2}) \le(X_{N}(\sigma^{1},\sigma^{1})+X_{N}(\sigma^{2},\sigma^{2}))/2$, so $\mathbb{E}\langle(X_{N}(\sigma^{1},\sigma^{2})-p_{\infty})_{+}\rangle_{\mathrm{ref}} \le\mathbb{E}\langle|X_{N}(\sigma,\sigma)-p_{\infty}|\rangle_{\mathrm{ref}} \to0$, and the left side converges to the corresponding expectation in the limit by uniform integrability. Projecting the levels of the approximating cascades onto $[0,1]\times[0,p_{\infty}]$, which is monotone, continuous, and the identity on the values of the limiting arrays, we may assume $0 \le q_{l} \le1$ and $0 \le p_{l} \le p_{\infty}$. The cascade realizations of these cascades with diagonal values $(1,p_{\infty},b_{\infty})$ then have arrays converging to the limiting arrays of $(R_{N},X_{N},b_{N})$. We apply Lemma 4.8 to the sequence that alternates the reference systems along our subsequence (with $b_{N}$ replaced by $b_{\infty}$) and these cascade realizations, whose arrays converge to the same limit. We conclude that the limits along our subsequence of the quantities in Lemma 4.8(i)–(ii) for the reference system are the limits of their values on the cascade realizations.

For the spin overlaps of the $N'$-dimensional system we apply Theorem 4.6 to the Gram pairs generated by $\rho/\sqrt{N'}$ and the zero vector. Then every $Q$ in its hypothesis is $(v_{1}x)^{m}=v_{1}^{m}x^{m}$, so the hypothesis reduces to the defects with $v=(1,0)$ of Lemma 4.7(iii). By parts (i)–(ii) of the theorem, the limiting spin overlap array is a limit in finite-dimensional distribution of Ruelle probability cascade arrays with nondecreasing levels in $[0,1]$, and the overlap laws of these cascades converge to the limiting overlap law $\widetilde{\zeta}$ of the $N'$-dimensional system.

The two cavity terms

We now compute the increment of the expected log partition function from dimension $N$ to dimension $N'$ at fixed perturbation parameters and a fixed number $M\in W_{N}$ of rows. Write $\log Z_{N'}^{M+\pi}$, $\log Z_{N'}^{M}$, and $\log Z_{N}^{M}$ for the log partition functions of the perturbed $N'$-dimensional system with all $M+\pi$ rows, of the same system with its first $M$ rows only, and of the perturbed $N$-dimensional system. The increment splits as

The row term. By Lemma 4.7(v), the row term is at least $\mathbb{E}\log\langle\exp\{\sum_{i\le\pi}U(y_{i}(\rho))\}\rangle_{N'}-o(1)$, where the fields $y_{i}$ of the added rows are centered Gaussian with covariance $\rho\cdot\rho'/N'$ and are independent of $\langle\cdot\rangle_{N'}$. Thus $\langle\cdot\rangle_{N'}$, with $R(\rho,\rho')=\rho\cdot\rho'/N'$, $X\equiv0$, $b\equiv0$, and the fields $y_{i}$ of the added rows, satisfies (G) and the moment condition of Lemma 4.8. By Lemma 4.8(iii), which applies by (88), the limit of this quantity along a subsequence along which the spin overlap array converges depends only on the limiting law of that array. For parameters in $\mathcal{G}_{N}(M)$ this array is a limit of Ruelle probability cascade arrays with nondecreasing levels in $[0,1]$ whose overlap laws $\widetilde{\zeta}_{h}$ converge to the limiting overlap law $\widetilde{\zeta}$ (§4.3). Consider the $h$-th of these cascades, with levels $0 \le q_{0} \le\cdots\le q_{k} \le1$ (we drop the index $h$ from its levels and masses), and its cascade realization with diagonal values $(1,0,0)$ and $p \equiv0$. Given $\pi$, the fields $y_{i}(\rho)=Y_{\beta}^{q,i}+\sqrt{1-q_{k}}x_{i}^{\prime\prime}$ of different rows are independent, so

Apply Lemma 3.2 with the marks $(e_{\nu}^{(i)})_{i\le\pi}$, where $e_{\nu}^{(i)}$ are the Gaussian increments of $Y^{q,i}$ at the node $\nu$ (with variance $q_{0}$ at the root and $q_{l}-q_{l-1}$ at depth $l$), and with the leaf function $\sum_{i\le\pi}\mathcal{T}_{1,1-q_k}U(Y_{\beta}^{q,i})$. The leaf function is bounded above and at least $-C\sum_i(1+(Y_{\beta}^{q,i})^2)$ by (4.2) and (2.6), so all quantities in (3.2) are finite. Since the marks of different rows are independent and the leaf function is a sum over the rows, the recursion (3.2) factorizes over the rows. For one row, its steps are $\mathcal{T}_{\theta_l,q_l-q_{l-1}}$ at depth $l$, and the average over the root mark is $\mathcal{T}_{0,q_0}$. By (3.5) and the recursion (2.2), the expected logarithm of the display equals

where $\zeta_h=\sum_l w_l\delta_{q_l}$ is the overlap law of the cascade, a probability measure on $[0,1]$ whose distribution function equals $0$ on $[0,q_0)$ and $\theta_{l+1}$ on $[q_l,q_{l+1})$ for $0\le l\le k$, with $q_{k+1}=1$ (some of these intervals may be empty); its steps are exactly the operators in the display. Averaging over $\pi\sim\operatorname{Poi}(\alpha n)$ gives $\alpha n\,u_{\zeta_h}(0,0;U)$. As $h\to\infty$, these values converge, by Lemma 4.8(iii) applied to the sequence that alternates the $N'$-dimensional systems and the cascade realizations, to the limit of the lower bound for the row term. Moreover $u_{\zeta_h}(0,0;U)\to u_{\widetilde{\zeta}}(0,0;U)$ by Lemma 2.4, since the laws $\zeta_h$ converge weakly on $[0,1]$ and hence in the Wasserstein distance. Thus, along the subsequences used below,

In §4.7 we show that $\widetilde{\zeta}$ equals the limiting overlap law $\zeta$ of the reference system.

Reduction of the coordinate term to the reference system. By (4.3) and (4.5), $Z_{N'}^{M}$ is an integral over $(\sigma,\varepsilon)$ against $\mu_N(\mathrm{d}\sigma)f_{N,n}(\varepsilon)\,\mathrm{d}\varepsilon$. Restricting $\varepsilon$ to the ball $\mathbb{B}_{\Lambda}$ decreases it, so the coordinate term is bounded below by the same expression with $Z_{N'}^{M}$ replaced by its restriction to $\mathbb{B}_{\Lambda}$. By Lemma 4.7(iv)(a) we may replace the Gaussian part of the perturbation at $\rho(\sigma,\varepsilon)$ by its value at $\rho(\sigma,0)$, at a cost $o(1)$. After this replacement the Gaussian part depends only on $\sigma$. We add and subtract the activation terms $\sum_a U^{\widetilde{\zeta}}(S_a)$ at the reference fields, with $\widetilde{\zeta}=\widetilde{\zeta}_{N'}$, which include the scalar part, and factor out the partition function of the tilted reference system. Similarly, by Lemma 4.7(iv)(b) the $N$-dimensional system may be written, at a cost $o(1)$, with the fields $S_a+\eta_a$ of (4.6), the reference perturbation, and the strength $\tilde{\xi}_{N'}$. Since the reference partition function is common to both terms, it cancels, and the coordinate term is at least

The factors $e^{-nB_N/2}$ undo the tilt of $\langle\cdot\rangle_{\mathrm{ref}}$; they appear in both terms. Here $S_a=S_a(\sigma)$, $B_N=B_N(\sigma)$, and $\langle\cdot\rangle_{\mathrm{ref}}$ acts on $\sigma$.

Gaussian interpolation for the numerator. We replace the activation increment in the first term of (107) by a Gaussian field. Let $z(\sigma)=(z_1(\sigma),\ldots,z_n(\sigma))$ be, conditionally on the disorder, a centered Gaussian process with $\mathbb{E}z_i(\sigma)z_j(\sigma')=\delta_{ij}X_N(\sigma,\sigma')$, independent of everything else (for instance the process $z$ of §4.3). For $0\le t\le1$ put $S_a^t=\chi(\varepsilon)S_a+\sqrt{t}\,w_a(\varepsilon)$ and

and let $\varphi(t)$ be the first term of (107) with the exponent replaced by $\Delta_t$; thus $\varphi(1)$ is that term. Write $\langle\cdot\rangle_t$ for the associated Gibbs measure on $S_N\times\mathbb{B}_{\Lambda}$. We first discuss the terms produced by the unperturbed activation $U$ and then the terms that carry a factor $\tilde{\xi}$ or $\tilde{\xi}^2$.

The field $w_a(\varepsilon)$ is centered Gaussian with $\mathbb{E}w_a(\varepsilon)w_a(\varepsilon')=\varepsilon\cdot\varepsilon'/N'$, independently over $a$, and it is independent of the reference system. Gaussian integration by parts in the fresh columns $g'_a$ produces the diagonal terms $\lVert\varepsilon\rVert^2(U''+U'^2)(S_a^t)$ (the $U''$ part from differentiating $U'$, the $U'^2$ part from differentiating the Gibbs density of the same replica) and a two-replica term. Integration by parts in $z$ produces the $U'^2$ part and the two-replica term at the reference fields $S_a$, with the opposite sign, and the derivative of the explicit term $(1-t)\lVert\varepsilon\rVert^2B_N/2$ supplies the $U''$ part at the reference fields, also with the opposite sign. The result, for the activation $U$, is

where the superscripts $1,2$ refer to two replicas $(\sigma^1,\varepsilon^1)$, $(\sigma^2,\varepsilon^2)$. The first line collects the diagonal contributions and the second the two-replica covariance contributions.

We bound (108) on $\Omega_N$. Uniformly on $S_N\times\mathbb{B}_{\Lambda}$,

The second bound follows from $S_a^t-S_a=(\chi(\varepsilon)-1)S_a+\sqrt{t}\,w_a(\varepsilon)$, from $\chi(\varepsilon)-1=O_{n,\Lambda}(N^{-1})$, and from $\sum_a w_a(\varepsilon)^2\leq\lVert\mathbf{G}'\rVert_{\mathrm{op}}^2\lVert\varepsilon\rVert^2/N'\leq C\lVert\varepsilon\rVert^2$. Since $\lvert U'\rvert$ grows at most linearly and $U'',U'''$ are bounded, the function $\omega_U=U''+U'^2$ satisfies $\lvert\omega_U(x)-\omega_U(x')\rvert\leq C(1+\lvert x\rvert+\lvert x-x'\rvert)\lvert x-x'\rvert$, and the Cauchy–Schwarz inequality gives

The product difference in the second line obeys the same estimate, after writing it as $[U'(S_a^{t,1})-U'(S_a^1)]U'(S_a^{t,2})+U'(S_a^1)[U'(S_a^{t,2})-U'(S_a^2)]$. Since $\lVert\varepsilon\rVert^2\leq\Lambda n$ and $\lvert\varepsilon^1\cdot\varepsilon^2\rvert\leq\Lambda n$ on $\mathrm{B}_\Lambda$, the right side of (108) is $O_{n,\Lambda}(N^{-1/2})$ on $\Omega_N$. On $\Omega_N^c$ the integrand is bounded by a polynomial in the operator norms, whose moments are bounded, so its contribution is at most $C_{n,\Lambda}\mathbb{P}(\Omega_N^c)^{1/2}$.

The terms carrying $\xi$ or $\xi^2$ arise because the interpolation uses the activation $U^\xi$ of (100) while $B_N$ and $X_N$ are defined with $U$. In (108) they replace $\omega_U(S_a^t)$ by $\omega_{U^\xi}(S_a^t)$ and $U'(S_a^{t,1})U'(S_a^{t,2})$ by $(U^\xi)'(S_a^{t,1})(U^\xi)'(S_a^{t,2})$. By (87) the first and second derivatives of $\tau_1$ and $\tau_2$ grow at most linearly, and the first derivatives of $U$ as well; hence $\lvert\omega_{U^\xi}(x)-\omega_U(x)\rvert\leq C(\xi+\xi^2)(1+x^2)$ and $\lvert(U^\xi)'(x)(U^\xi)'(x')-U'(x)U'(x')\rvert\leq C(\xi+\xi^2)(1+x^2+x'^2)$. Since $N'^{-1}\sum_a(1+(S_a^t)^2)\leq C$ on $\Omega_N$, these terms contribute $O_{n,\Lambda}(\xi_{N'}+\xi_{N'}^2)=o(1)$ on $\Omega_N$; this is sufficient, although it is weaker than the rate $N^{-1/2}$ available for the activation $U$.

We conclude that $\varphi(1)-\varphi(0)=o(1)$ for fixed $n$ and $\Lambda$. The same holds for Gibbs expectations: differentiating $\mathbb{E}\langle F\rangle_t$ for a bounded measurable function $F$ of the spin overlaps and cavity coordinates of $r$ replicas, which does not depend on the fresh Gaussian variables, inserts at most $r+2$ replicas and the same covariance differences, so $\lvert\partial_t\mathbb{E}\langle F\rangle_t\rvert\leq C_{r,n,\Lambda}\lVert F\rVert_\infty(N^{-1/2}+\xi_{N'}+\xi_{N'}^2)$ on $\Omega_N$, up to a contribution $C_{r,n,\Lambda}\lVert F\rVert_\infty\mathbb{P}(\Omega_N^c)^{1/2}$.

The numerator at $t=0$. By Taylor’s formula and (90), uniformly on $\mathrm{B}_\Lambda$ and on $\Omega_N$,

and the corresponding terms carrying $\xi$ are $O_{n,\Lambda}(\xi_{N'})$. Adding $\lVert\varepsilon\rVert^2B_N/2$ and using $b_N-1=A_N-B_N$, the exponent $\Delta_0$ equals

The term $nB_N/2$ cancels the factor $\mathrm{e}^{-nB_N/2}$ in (107), and $f_{N,n}(\varepsilon)\,\mathrm{d}\varepsilon$ may be replaced by $\nu_n(\mathrm{d}\varepsilon)$ by (90). This explains the definitions of $A_N$, $B_N$, and $b_N$.

Gaussian interpolation for the denominator. The second term of (107) is treated by the same interpolation. The field $\eta_a(\sigma)=c_N g_N^{\prime\prime a}\cdot\sigma$ has covariance $c_N^2\sigma\cdot\sigma'=(n/N')R_N(\sigma,\sigma')$, and on $\Omega_N$ we have $\sum_a\eta_a^2\leq c_N^2\lVert G^{\prime\prime}\rVert_{\mathrm{op}}^2N\leq Cn$. Let $y(\sigma)$ be, conditionally on the disorder, a centered Gaussian process with covariance $nR_N(\sigma,\sigma')X_N(\sigma,\sigma')$, and interpolate with

Integration by parts gives (108) with $\lVert\varepsilon\rVert^2$ replaced by $n$ and $\varepsilon^1\cdot\varepsilon^2$ replaced by $nR_N(\sigma^1,\sigma^2)$, and the same estimates show that the two endpoints differ by $o(1)$. At $t=0$ the activation increment is replaced by $y(\sigma)+nB_N(\sigma)/2$, and $nB_N/2$ again cancels the factor $e^{-nB_N/2}$.

The coordinate term. Combining the preceding paragraphs, for fixed $n$ and $\Lambda$ the coordinate term is at least

The same comparison holds for Gibbs expectations of bounded continuous functions of finitely many replicas, including their cavity coordinates: under the Gibbs measure of the $N'$-dimensional system with $M$ rows, conditioned on $\varepsilon\in\mathbb{B}_{\Lambda}$ for each replica, such expectations differ by $o(1)$ from those under the cavity measure of the first term of (109), where the spin overlaps are $R_N(\sigma^\ell,\sigma^{\ell'})$. Indeed, the chain of comparisons above consists of Lemma 4.7(iv)(a), the interpolation, whose effect on Gibbs expectations was bounded above, and the replacement of an exponent and a density by quantities that differ from them by $o(1)$ uniformly on $\Omega_N\times\mathbb{B}_{\Lambda}$. Moreover, on $\mathbb{B}_{\Lambda}$ the spin overlap $\rho^\ell\cdot\rho^{\ell'}/N'=[\chi(\varepsilon^\ell)\chi(\varepsilon^{\ell'})NR_N(\sigma^\ell,\sigma^{\ell'})+\varepsilon^\ell\cdot\varepsilon^{\ell'}]/N'$ of the $N'$-dimensional system differs from $R_N(\sigma^\ell,\sigma^{\ell'})$ by $O_{n,\Lambda}(N^{-1})$, and a continuous function is uniformly continuous on the compact set of overlaps in $[-1,1]$ and cavity coordinates in $\mathbb{B}_{\Lambda}$.

For parameters in $\mathcal{G}_N(M)$ and along the subsequences of §4.3, Lemma 4.8 allows us to replace $b_N$ by its limit $b_{\infty}$. Thus, for fixed $n$ and $\Lambda$, the coordinate term is bounded below, up to $o(1)$, by

where, conditionally on the reference system, $z$ and $y$ are centered Gaussian with

The first term of (110) is the expected logarithm of the numerator $\mathcal{N}_{\Lambda}$ of Lemma 4.8 for the reference system with $b=b_{\infty}$, and the second is minus that of the denominator $\mathcal{D}$; we call the two expected logarithms the numerator and the denominator. By Lemma 4.8(i), both converge along the subsequence, and by §4.3 their limits are the limits of their values on cascade realizations. We compute these values next.

The cavity quantities on cascades

We evaluate the numerator and the denominator of (110) on a cascade realization, using Lemma 3.4, and we show that the cutoff $\Lambda$ can be removed when the smallest precision is bounded below. Throughout this section we use the notation of §3.2: a Ruelle probability cascade of depth $k$ with parameters $\theta_l$ and masses $w_l$, the branching level $J=\beta^1\wedge\beta^2$ of two replicas, and, for levels $p\in\mathcal{K}$ and a precision $b$, the precisions $d_l$, the mean levels $m_l$, the function $G(b,p)$, the mean squared norm $\varrho=m_k+1/b$, and the tilted measure $\mathcal{M}$ defined in (53) and below it.

Fix a cascade with paired levels $0\le q_0\le\cdots\le q_k\le1$ and $0\le p_0\le\cdots\le p_k$, and its cascade realization with diagonal values $(1,\bar p,\bar b)$ (§4.3). The overlap law of the cascade is $\sum_l w_l\delta_{q_l}$, and the law of the response overlap is $\sum_l w_l\delta_{p_l}$. On the realization, the response field $z$ is the cascade field $Y_\beta^p$ plus the within-state variable $\sqrt{\bar p-p_k}\,x$. Averaging $x$ inside the Gibbs average multiplies the integrand of the numerator by $e^{(\bar p-p_k)\lVert\varepsilon\rVert^2/2}$, which changes the coefficient of $-\lVert\varepsilon\rVert^2/2$ from $\bar b-1$ to $b-1$, where

By (52), the smallest precision $d_0=b-\sum_{l\ge1}\theta_l\Delta p_l$ of (53) is expressed through the mean of the response overlap:

When $d_0>0$, all precisions are positive and Lemma 3.4 applies.

The cavity quantities on a cascade realization. Averaging the within-state coordinates inside the Gibbs average gives the following values. The numerator of (110), with $b_\infty$ replaced by $\bar b$, equals $\Theta_\Lambda+n(\bar b-1)/2$, where

and we write $\Theta$ for the same quantity with $\varepsilon$ integrated over $\mathbb{R}^n$. When $d_0>0$, (55) gives $\Theta=n[G(b,p)-\frac12\log b]$. The denominator equals

by (63) with the levels $c_l=nq_lp_l$; here $\bar p\ge p_k\ge q_kp_k$. Hence, when $d_0>0$, the unrestricted numerator minus the denominator, divided by $n$, is

The last term vanishes in the limit by Lemma 4.9 (Step 3 of §4.7).

The cutoff is removed with the following lemma. It concerns the unrestricted tilted measure $\mathcal{M}$ of §3.2 with the field $Y=Y^{\rho}$, defined when $d_{0}>0$, and the random probability measure $\mathcal{M}_{\Lambda}$ on pairs $(\beta,\varepsilon)$ with density proportional to $v_{\beta}\nu_{n}(\mathrm{d}\varepsilon)e^{\varepsilon\cdot Y_{\beta}-(b-1)\lVert\varepsilon\rVert^{2}/2}\mathbf{1}_{B_{\Lambda}}(\varepsilon)$, which is defined for every $b$; when $d_{0}>0$, $\mathcal{M}_{\Lambda}$ is $\mathcal{M}$ conditioned on $\{\varepsilon\in B_{\Lambda}\}$.

Lemma 4.10. Fix the cavity dimension $n$ and constants $P_{0},B_{0},c_{0}>0$, and consider finite cascades with

There are constants $C,c^{\prime}>0$, depending only on $n,P_{0},B_{0},c_{0}$, such that for all $\Lambda\ge1$

Moreover $\varrho=m_{k}+1/b\le P_{0}/c_{0}^{2}+1/c_{0}$.

Proof. Every denominator $d_{l-1}d_{l}$ and $d_{0}^{2}$ in (53) is at least $c_{0}^{2}$, and the increments of $p$ sum to $p_{k}\le P_{0}$, so $m_{k}\le P_{0}/c_{0}^{2}$; also $1/b\le1/d_{0}\le1/c_{0}$. By (57), one replica $\varepsilon^{1}$ has the law $N(0,\varrho I_{n})$ under $\mathbb{P}_{\mathcal{M}}$, and the Gaussian tail bound for $\lVert\varepsilon^{1}\rVert^{2}$ gives the second estimate in (114), for $\Lambda$ larger than a constant; enlarging $C$ covers the remaining $\Lambda\ge1$.

A small averaged probability of $\{\varepsilon\notin B_{\Lambda}\}$ alone would not bound $\Theta-\Theta_{\Lambda}$, so we use convexity. Define the leaf functions

Then $f_{\Lambda}\le f$, so $\Theta_{\Lambda}\le\Theta$. The map $\varphi\mapsto\mathbb{E}\log\sum_{\beta}v_{\beta}e^{\varphi(Y_{\beta})}$ is convex on leaf functions, and for a bounded direction $\psi$ its derivative at $f$ is $\mathbb{E}_{\mathcal{M}}\psi(Y_{\beta_{1}})$, by dominated convergence. Applying convexity with the bounded direction $\psi_{t}=\max\{f_{\Lambda},f-t\}-f\in[-t,0]$ gives

and letting $t\to\infty$, by monotone convergence, $\Theta-\Theta_{\Lambda}\le\mathbb{E}_{\mathcal{M}}[f(Y_{\beta_{1}})-f_{\Lambda}(Y_{\beta_{1}})]$. Now $\exp(f_{\Lambda}-f)(y)$ is the probability that a vector with law $N(y/b,b^{-1}I_{n})$ lies in $B_{\Lambda}$, and by Lemma 3.4(ii), $Y_{\beta_{1}}/b=M_{k}\sim N(0,m_{k}I_{n})$ under $\mathbb{P}_{\mathcal{M}}$. Hence

where $G^{\prime}$ is a standard Gaussian vector in $\mathbb{R}^{n}$ independent of $M_{k}\sim N(0,m_{k}I_{n})$. We split the last expectation according to the size of $M_k$. On $\{\lVert M_k\rVert\le\sqrt{\Lambda n}/2\}$, the conditional probability of leaving $\mathbb{B}_{\Lambda}$ is at most $\mathbb{P}(\lVert G'\rVert^{2}>b\Lambda n/4)\le\mathbb{P}(\lVert G'\rVert^{2}>c_{0}\Lambda n/4)\le Ce^{-c'\Lambda}$, and for $\Lambda$ larger than a constant the negative logarithm is at most twice this bound, since $-\log(1-x)\le2x$ for $0\le x\le1/2$. For all $M_k$ and $\Lambda\ge1$, integrating the Gaussian density of $M_k+b^{-1/2}G'$ over the fixed ball $\mathbb{B}_{1}$ and using $\lVert x-M_k\rVert^{2}\le2\lVert x\rVert^{2}+2\lVert M_k\rVert^{2}$ gives

with $c_{2},C_{2}$ depending only on $n,c_{0},B_{0}$, since $c_{0}\le b\le B_{0}$. The contribution of $\{\lVert M_k\rVert>\sqrt{\Lambda n}/2\}$ is therefore bounded by a Gaussian tail expectation of $\log(1/c_{2})+C_{2}(1+\lVert M_k\rVert^{2})$, which is also at most $Ce^{-c'\Lambda}$ uniformly for $m_k\le P_{0}/c_{0}^{2}$. Enlarging $C$ covers the remaining $\Lambda\ge1$. $\square$

The measure $\mathcal{M}_{\Lambda}$ is the law of the leaves and cavity coordinates under the cavity measure of the cascade realization on $\mathbb{B}_{\Lambda}$, after the within-state coordinates are integrated out. For $r$ replicas, $\mathcal{M}_{\Lambda}$ is $\mathcal{M}$ conditioned on the event that $\varepsilon^{\ell}\in\mathbb{B}_{\Lambda}$ for $\ell\le r$, realization by realization. If a probability measure gives mass $1-\delta$ to an event, then conditioning each of $r$ replicas on this event changes the expectation of a bounded function $F$ of the $r$ replicas by at most $2r\lVert F\rVert_{\infty}\delta$, because the conditioning event for the $r$ replicas has mass at least $1-r\delta$. Since this bound is linear in $\delta$, it survives averaging over the disorder:

Rotation invariance

The two identities that determine the entropy come from the following lemma. It uses only the invariance (104) of the $N'$-dimensional system under simultaneous rotations of the configurations and the rows. In the coordinates $\rho=\rho(\sigma,\varepsilon)$, the quantity $R^{\mathrm{cav}}$ below is the overlap $\varepsilon^{1}\cdot\varepsilon^{2}/n$ of the added coordinates.

Lemma 4.11. Suppose that the disorder-averaged law of a random Gibbs measure on $S_{N'}$ is invariant under simultaneous orthogonal transformations of its replicas. For two replicas $\rho^{1},\rho^{2}$ define

Then

Moreover, the averaged one-replica law is uniform on $S_{N'}$, and the law of its last $n$ coordinates converges to $\nu_n$ as $N\to\infty$.

Proof. By the invariance, we may replace two given replicas by their images under an independent Haar-distributed rotation without changing the averaged law. Condition on two unit vectors $\rho^{1}/\sqrt{N'}$ and $\rho^{2}/\sqrt{N'}$ with inner product $R$. Their images can be written as $x$ and $Rx+\sqrt{1-R^{2}}x^{\perp}$, where $x$ is uniform on the unit sphere of $\mathbb{R}^{N'}$ and, given $x$, $x^{\perp}$ is uniform on the unit sphere of the orthogonal complement of $x$. For the $n$ coordinate indices $i\in\{N+1,\ldots,N'\}$ put $A=\sum_{i}x_{i}^{2}$ and $B=\sum_{i}x_{i}x_{i}^{\perp}$, so that $R^{\mathrm{cav}}=(N'/n)(RA+\sqrt{1-R^{2}}B)$. The variable $A$ has the beta distribution with parameters $n/2$ and $N/2$, so

Given $x$, $\mathbb{E}[x_{i}^{\perp}x_{j}^{\perp}\mid x]=(\delta_{ij}-x_{i}x_{j})/(N'-1)$, hence $\mathbb{E}[B\mid x]=0$ and $\mathbb{E}[B^{2}\mid x]=(A-A^{2})/(N'-1)$. Therefore

Consequently

and multiplying by $(N'/n)^{2}$ gives (116). The averaged one-replica law is a rotation-invariant probability measure on $S_{N'}$, hence uniform. By (89) its last $n$ coordinates have the density $f_{N,n}$, which converges pointwise to the standard Gaussian density; by Scheffé’s lemma the laws converge to $\nu_{n}$ in total variation. $\square$

Proof of the lower bound

The order of the limits matters, because the Gaussian integral over all of $\mathbb{R}^{n}$ in the numerator is not a bounded function of the overlaps. For fixed $n$ we first let $N\to\infty$ along a subsequence at a fixed cutoff $\Lambda$, then let the index $h$ of the approximating cascades tend to infinity, then let $\Lambda\to\infty$; the cavity dimension $n$ is sent to infinity last.

Proof of Theorem 4.1. We divide the proof into eight steps.

Step 1. Increments. Let $\Xi_{N}$ be the expected log partition function of the perturbed $N$-dimensional system with $M\sim\operatorname{Poi}(\alpha N)$ rows, averaged also over the perturbation parameters. By the Poissonization estimate of §4.1 and Lemma 4.7(i),

For every fixed $n$,

To verify this, let $\ell'$ be smaller than $n$ times the right side. Then $\Xi_{N+n}-\Xi_N \ge\ell'$ for all $N \ge N_0$. Every $N \ge N_0+n$ can be written as $N=N_1+jn$ with $N_0 \le N_1<N_0+n$ and $j \ge1$, and telescoping along the residue class of $N_1$ gives $\Xi_N \ge\min_{N_0 \le N_1<N_0+n}\Xi_{N_1}+j\ell'$. Since $j/N \to1/n$, the left side of (118) is at least $\ell'/n$. This is the inequality on which the Aizenman–Sims–Starr scheme rests; see [ref-100].

Let $\ell_n=\liminf_N(\Xi_{N+n}-\Xi_N)$. Then $\ell_n<\infty$, since $\Xi_N/N$ is bounded above by $\alpha U_{\max}+o(1)$ and (118) holds; and $\ell_n \ge-C_n$ by Step 2.

Step 2. Choice of parameters and row count. For fixed perturbation parameters and a number $M$ of rows, let $\Delta_N(M)$ be the increment on the left of (4.19), computed conditionally on the number of rows; thus $\Xi_{N+n}-\Xi_N$ is the average of $\Delta_N(M)$ over the parameters and over $M \sim\operatorname{Poi}(\alpha N)$, because $M+\pi\sim\operatorname{Poi}(\alpha N')$. By Lemma 4.7(v), $\Delta_N(M) \ge-C_n$ uniformly in the parameters and in $M \in W_N$, and $|\Delta_N(M)| \le C_n(N+M)$ for every $M$. Since $\mathbb{P}(M \notin W_N) \le2e^{-cN^{1/3}}$ and $\mathbb{E}(N+M)^2 \le CN^2$, the Cauchy–Schwarz inequality shows that the event $\{M \notin W_N\}$ contributes $o(1)$ to the average; in particular $\ell_n \ge-C_n$. Choose a subsequence along which $\Xi_{N+n}-\Xi_N \to\ell_n$. Let $\mathcal{G}'_N$ be the event that $M \in W_N$ and that the parameters lie in the set $\mathcal{G}_N(M)$ of Lemma 4.7(iii). Its probability is at least $\mathbb{P}(M \in W_N)\inf_{M \in W_N}\mathbb{P}(\mathcal{G}_N(M)) \to1$. Since $\Delta_N \ge-C_n$ on $\{M \in W_N\}$, the average of $\Delta_N(M)$ over $\mathcal{G}'_N$ is at most

along the subsequence. Hence we can choose $M_N \in W_N$ and parameters in $\mathcal{G}_N(M_N)$ whose increment satisfies $\Delta_N(M_N) \le\ell_n+o(1)$. We fix these parameters and this row count from now on.

Step 3. One limit of the overlap arrays, and positive precision. Along a further subsequence, the finite-dimensional distributions of the arrays $(R_N,X_N,b_N)$ under $\mathbb{E}\langle\cdot\rangle_{\mathrm{ref}}$ converge, $\mathbb{E}\langle b_N\rangle_{\mathrm{ref}} \to b_\infty$, $\mathbb{E}\langle X_N(\sigma,\sigma)\rangle_{\mathrm{ref}} \to p_\infty$, and the spin overlap array of the $N'$-dimensional system under $\mathbb{E}\langle\cdot\rangle_{N'}$ converges. Write $\mathrm{L}$ for the limiting law of the arrays of the reference system, with diagonal values $(1,p_\infty,b_\infty)$, and $\mathbb{E}_{\mathrm{L}}$ for expectations under it; let $\zeta$ be the law of $R_{12}$ under $\mathrm{L}$, and $\widetilde{\zeta}$ the limiting overlap law of the $N'$-dimensional system. These limits do not depend on $\Lambda$. The error terms $o(1)$ of §4.4 depend on $\Lambda$, but for every fixed $\Lambda$ they tend to zero along the whole sequence.

By §4.3, there are finite cascades, indexed by $h \ge1$, with depths $k_h$ and paired levels $0 \le q_l^h \le1$ and $0 \le p_l^h \le p_\infty$, whose cascade realizations with diagonal values $(1,p_\infty,b_\infty)$ have arrays converging to $\mathrm{L}$. We attach the index $h$ to the quantities