WorksPhysics and Astronomy
Constructive Cavity Method for Ising Restricted and Deep Boltzmann Machines with Gaussian Couplings
Abstract
We apply the Constructive Cavity Method (CCM) to the dense, Ising type, restricted and deep Boltzmann machine with Gaussian couplings between adjacent layers. After identifying the core and interface contributions, CCM identifies the blanket decomposition of the cavity variables with the Aizenman–Sims–Starr random overlap structure. We apply the Random Energy Model (REM) universality ansatz to represent the cavity free energies by those of a generalized random energy model (GREM) and evaluate them recursively through Poisson point processes (PPPs). In the layer-wise gauge each layer is cut with its own cumulative fraction , with , where . Here is the cumulative blanket union up to blanket . The index labels architectural layers, whereas labels the blankets. The dependence on the multilayer geometry is carried by the two matrices and . The constructive cavity field in layer has blanket variances , while the correction field has blanket variances . The nested REM–PPP average gives the finite-level constructive cavity functional . The balanced scalar gauge is recovered by imposing . Selecting the physical value in the non-convex multilayer regime is a separate problem.
Introduction
The Constructive Cavity Method (CCM) [1] is a procedure for evaluating scalar cavity functionals by constructing the cavity variables explicitly. It originates from the Kernel/Blanket framework of disordered systems [2, 3, 4] where replica-symmetry-breaking structures are represented through blanket decompositions, REM universality [2, 5, 6, 7], and constructive cavity variables. In the Sherrington–Kirkpatrick and Parisi spin-glass setting [8, 9, 10, 11], the method starts from the Aizenman–Sims–Starr cavity functional [12], identifies the blanket decomposition of its cavity variables with the random overlap structure (ROSt), and evaluates the averages recursively through Poisson point processes (PPPs).
The aim of this paper is to apply the same practical construction to the dense Ising restricted and deep Boltzmann machines with Gaussian couplings. The model has layers, interactions only between adjacent layers, and layer proportions
The case is the dense bipartite Ising RBM with Gaussian couplings, while the case is the corresponding DBM architecture. Boltzmann machines and their restricted and deep variants originate as layered energy-based models [13, 14, 15]. Their statistical-mechanics and spin-glass formulations are connected with bipartite, multi-species, and multilayer mean-field models [16, 17, 18, 19, 20]. In non-convex multilayer regimes, the selection of the physical value is not expected to be governed automatically by a standard minimization principle [21, 22]. We do not address that selection problem here. We derive the constructive cavity functional.
The central constructive coordinate is a layer-wise gauge. For each blanket , each layer is cut by its own cumulative fraction , weighted by the layer size:
Thus the finite-level construction uses one layer-wise coordinate
per blanket, with vector valued parameter and
componentwise. The layer dependence is carried by the layer sizes, by the coupling strengths, and by the different components of . In particular, if
where is the weighted adjacency matrix of the layer chain and , then the cavity field acting on a spin in layer has blanket variances
while the correction field has blanket variances
The balanced scalar gauge is obtained as the diagonal specialization
In that case the general matrix quantities reduce to
We first identify cores and interfaces in the layer-wise partition and compute the Gaussian blanket amplitudes. The REM universality ansatz of Ref. [1], applied in § 6, represents jointly the cavity free-energy shifts and the correction shift by their generalized random energy model (GREM) counterparts. The blanket components are identified with the Aizenman–Sims–Starr ROSt variables, and nested Poisson averaging yields the finite-level functional. The final finite-level functional is
The quantities are obtained by the nested REM–PPP Gaussian recursion with layer-dependent scale at blanket level . The layer-wise cumulative fractions determine the amplitudes and supply the levels of the resulting Parisi-type functional.
The paper is organized as follows. § 2 recalls the CCM in the operational form used here. § 3 defines the Ising RBM/DBM with Gaussian couplings and the matrices and . § 4 separates cores and interfaces and introduces the layer-wise gauge. The following sections construct the Gaussian variables, write the Aizenman–Sims–Starr functional, perform the nested REM–PPP average, and record basic checks. § 11 relates the result to the bipartite and multi-species full-RSB formulations, and § 12 explains the construction.
The CCM in practice
We use the minimal construction of Ref. [1 Section 4.7]; § 12 discusses its theoretical background.
The CCM [1] computes cavity free energies by decomposing the cavity variables into blanket components and evaluating their normalized averages. In a single-population setting the scalar cavity functional has the schematic form
where is the cavity field, is the correction or fugacity field, and is a reservoir. This is the Aizenman–Sims–Starr type of scalar cavity functional [12], evaluated in the constructive form of the CCM [1]. For Ising spins,
For a multilayer system there is one cavity mark for each layer. Any spin added to layer has a cavity field denoted by . The multilayer scalar functional will therefore have the form
The coefficient appears because the pressure is normalized per total spin, while the layer contains a fraction of all spins. In its minimal form, CCM starts from the Aizenman–Sims–Starr cavity functional, applies the blanket decomposition to its cavity and correction variables, and evaluates the resulting averages by nested Poisson averaging. Equivalently, CCM identifies the blanket decomposition of the cavity variables with the Aizenman–Sims–Starr random overlap structure (ROSt). Ref. [1], § 3 and § 4 provides a physical justification of this identification through the GREM representation of the states and the joint construction of their masses and cavity fields.
For RBM/DBM, we first identify the cores and interfaces of the layer-wise partition. The adjacent-layer vertex counts determine the cavity amplitudes, and the complete edge-shell counts determine the correction amplitudes. We then use these amplitudes in the thermodynamic representation stated in § 6 and perform the nested average of § 7.
The Ising RBM/DBM model with Gaussian couplings
Fix a finite number of layers . Let and
At each site there is an Ising spin
The Hamiltonian of the RBM/DBM is
where the variables are independent standard Gaussians and is the coupling strength between layers and . The case is a dense bipartite Ising RBM with Gaussian couplings. The case is the corresponding DBM with interactions only between adjacent layers. Introduce the diagonal matrix
We write for the -dimensional vector of ones. Let be the symmetric tridiagonal matrix defined by the following formula:
with all other entries equal to zero. We use
Equivalently, for a vector ,
where the terms with indices outside are omitted. This convention also applies to the boundary layers in the formulas below. In the balanced specialization used for comparison later, the two scalar constants are
The factor in (2) is chosen so that these expressions can have this simple symmetric form. External fields are not written in the Hamiltonian. Throughout the derivation, denotes an optional deterministic field for layer entering only the one-spin cavity mark. Setting recovers exactly the Hamiltonian in (2).
Interface-only structure and layer-wise gauge
We first separate the core and interface contributions of the layer-wise partition, and use interface dominance under refinement to obtain the interface-only (IO) representative. Here and below, labels the architectural layers , while labels the blankets of the constructive partition; is the Poisson exponent used to average blanket , and is the fraction of layer included up to blanket . We fix ordered disjoint blankets , independently of the couplings, and set
Thus is the cumulative blanket up to , and is the -th blanket increment. This is the ordered blanket decomposition used in the CCM [1]. Each blanket is a global CCM block crossing all architectural layers, with
We choose a layer-wise nondecreasing profile
where the inequalities are componentwise. The layer-wise constructive gauge cuts each architectural layer by the cumulative fraction , weighted by the layer size: therefore the cumulative blocks have size
The corresponding blanket block is
and therefore its cardinality is
The variables are layer-wise cumulative fractions of the constructive gauge. The balanced scalar gauge is recovered by imposing .
The adjacent-layer structure of (2) determines the cavity components as follows. A cavity spin inserted in layer interacts only with spins in the neighboring layers and . Therefore the variance of its blanket- cavity field is determined by the vertex counts in
The correction field has the same adjacent-layer interaction support and is decomposed into complete edge shells [2, 3]. For adjacent layers and , define the cumulative rectangle
The blanket- edge shell is
Using (5), its cardinality is
Figure 1. CCM blanket geometry for a multilayer RBM/DBM. Architectural layers are indexed by , while CCM blankets are indexed by . Each blanket is a global partition block crossing all architectural layers, and the cell has size .
At finite level, the shell decomposes into an interface part and a core part. This decomposition introduces no additional datum. If
then we have
The first two terms are the interface contribution, while the last is the core contribution. Refining the layer-wise partition makes the total core fraction vanish. The IO pressure reduction follows from Eq. (49). The interface-dominance argument and the order of limits are discussed in Ref. [1 Section 2.1]. We retain the complete finite edge-shell counts when constructing the correction field below.
The counts (6) and (7) determine the Gaussian amplitudes of the constructive cavity variables.
Constructive cavity variables
The blanket decomposition of Section § 4 supplies the two sets of counts required by CCM: vertices in neighboring-layer blocks for the cavity fields, and edges in adjacent-layer shells for the correction field. The Gaussian amplitudes depend only on the layer-wise profile , the layer proportions , and the coupling strengths . We use these amplitudes to realize the cavity variables on the REM–GREM reservoir of Sections § 6–§ 7.
The cavity field
Consider a new spin inserted in layer . It interacts only with the two neighboring layers and . The random part of its cavity field is
with the missing boundary term omitted. The Gaussian variables in (8) are fresh and independent of the reservoir.
The blanket- contribution is obtained by restricting the sums to the blanket blocks in the neighboring layers:
Conditionally on the reservoir configuration, the cross terms vanish because the fresh couplings are independent and centered. Since the spins are Ising variables, the conditional variance is exactly computed as follows:
At the boundary layers, the absent term is omitted. Using the layer-wise block sizes (6), this becomes
Write for a path of the REM–GREM representation. Its depth- prefix is . With the amplitude in (11), the constructive cavity field in layer is
where distinct nodes, levels and architectural layers carry independent standard Gaussians, independent of the reservoir weights. Paths through the same node share its Gaussian variable, as in Ref. [1 Section 4.5]. Equivalently, if
Thus the layer-wise profile and the matrix fix the amplitudes of the layer- cavity field.
The correction field
The correction, or fugacity, mark accounts for the internal adjacent-layer interactions compensated in the cavity normalization. For adjacent layers and , the blanket- edge shell is from (7). The corresponding Gaussian correction mark is
The normalization is , rather than , because the correction mark is an internal edge sum over terms and must have variance of order one. Again, conditionally on the reservoir, the cross terms vanish by independence and centering of the fresh Gaussians, while each diagonal term contributes one because the spins are Ising. Hence, the expression for the variance is
Using (7), we get
The edge-shell contributions use independent fresh couplings, so the blanket variances add over . Summing over the adjacent layer pairs gives
Equivalently, we can write it in a compact matrix notation
Since the profile is componentwise nondecreasing and has nonnegative off-diagonal entries along the layer chain, the increments are nonnegative. The constructive correction field is therefore
with standard Gaussian variables , independent between distinct nodes and of all cavity-field Gaussians and reservoir weights. Paths through the same prefix share the same correction variable.
Aizenman–Sims–Starr cavity functional and constructive substitution
We now write the cavity functional using the blanket components constructed above. Let be a trial reservoir on configurations of the existing -layer system. For the Gibbs reservoir, the Aizenman–Sims–Starr construction compares the partition functions before and after adding one cavity spin; the same expression defines the functional on trial reservoirs. If the added spin belongs to architectural layer , summing its Ising value gives
Since the architectural layer contributes a fraction of all spins, the cavity terms are averaged with weights , and the factors give the total entropy contribution . The fugacity mark compensates the internal adjacent-layer interactions of the reservoir and therefore appears with a minus sign. Thus the Aizenman–Sims–Starr-type scalar cavity functional associated with the multilayer model has the form [12, 1]
Here is the one-spin cavity field associated with adding a spin to layer , and is the Aizenman–Sims–Starr fugacity or correction mark. The coefficient appears because the pressure is normalized per total spin and layer contains a fraction of all spins. The single entropy term is because
The outer expectation is over the fresh Gaussian variables used in the cavity marks and over any randomness of the reservoir. The brackets denote averaging with respect to the reservoir.
For the physical calculation, we apply the REM universality ansatz of Ref. [1 Sections 3 and 4.5]: the logarithmic cosh averages and the logarithmic correction average in (20) are represented asymptotically in joint distribution by the corresponding GREM free-energy shifts, with the cavity variables below. On a physical fibre, which fixes the magnetizations of the nonempty cells , the blanket average uses complete Gibbs conditional kernels, defined as ratios of full marginals. Contributions of retained fibres are summed in each partition function before taking its logarithm [1 Section 2.4]. The disorder expectation follows these operations. This thermodynamic replacement uses the constructive fields
Here denotes this replacement in the cavity free-energy calculation. The realization by prefix-indexed variables follows Ref. [1 Section 4.5].
Thus, for a generic constructive reservoir , the finite-level constructive functional associated with the layer-wise profile is
The matrices , and the layer-wise profile determine the amplitudes in this functional. We now evaluate it on the REM–GREM reservoir.
REM–PPP evaluation
We evaluate (21) on the hierarchical REM–PPP reservoir of Ref. [1 Sections 3.5–3.6], with parameters
First take . At every parent , draw a PPP with intensity on . Different parents and levels carry independent PPPs. The normalized path weights and their average are
The increasing exponents make the total path mass finite almost surely. Each parent mass includes all its descendants. These are the Ruelle cascade weights [23, 24, 25], with the Gaussian innovations of § 5 independent of the weights.
The marked Poisson identity acts at each parent on positive insertions and their complete descendant sums. For exponent , masses , and i.i.d. positive pairs independent of the masses, it gives
The rescaled totals have the same marginal law, so their expected logarithms cancel. The recursion with descendant totals is given in Ref. [1 Section 4.6]. In the multilayer functional the common normalization has coefficient . The expectation is over the reservoir and its Gaussian variables, after the state sums and logarithms.
Because the cavity part of (21) is a sum of layer-wise log-brackets, the Gaussian recursion is applied separately to each layer , with the same hierarchy . Define the terminal value
where the are independent standard Gaussians. For , set
where integrates the Gaussian increment at blanket for the fixed layer , with exponent . The formulas extend continuously to the boundary parameters. If , the limiting step is
Any initial zero-exponent blocks have fixed prefix labels, so their Gaussian increments in both fields are shared by all states. Together they represent the common overlap component. For , the step is , the thermal average. Consecutive equal exponents merge by Gaussian convolution; records the terminal convention. The nested REM–PPP average of the layer- cavity mark is then
The correction mark is evaluated by the same stability identity in its linear Gaussian form. Since is centered and has blanket- variance increment
we get the fugacity contribution
Indeed, at one PPP level the identity is
with the value when by continuity. Subdivision at a fixed averaging exponent preserves this correction, since the differences of the quadratic form telescope. Substituting (24) and (25) into (21) gives the finite-level CCM functional
This is the layer-wise constructive cavity functional of the multilayer RBM/DBM model. The variables entered the derivation as layer-wise cumulative fractions of the constructive gauge. The balanced scalar gauge is recovered by imposing . An equivalent representation by independent PPPs, with parent indices carrying the shared cavity variables, gives the same random cavity functional in distribution [1 Sections 3.6 and 4.6].
Checks
We present consistency checks of the layer-wise functional (26). The balanced scalar formulas are recovered by imposing . These checks test different parts of the construction: the RS reduction tests the REM–PPP evaluation, the annealed value tests the normalization of the correction field, the and reductions test the multilayer geometry, and the splitting-instability surface tests the local curvature of the functional.
Replica-symmetric reduction
The vector replica-symmetric reduction corresponds to two effective blankets,
and terminal . For layer , the variance increments in the terminal condition are
Thus we get the formula
Using, for , the exact identity
the step with gives
Since , the first step is an ordinary Gaussian expectation:
Substituting this into (26), the deterministic part is
By applying the following identity
with , this becomes
Hence the vector replica-symmetric expression is
This is the multilayer RS reduction associated with the layer-wise CCM gauge. The balanced scalar reduction is recovered by imposing the condition
Then follow the relations
and (27) becomes
This is the balanced-gauge specialization of the multilayer replica-symmetric expression for Gaussian-coupling DBM models [19].
As an annealed normalization check, set and in (27). Then the Gaussian cavity term vanishes and
This is the annealed high-temperature value of the adjacent-layer Gaussian RBM/DBM model, and therefore checks the normalization of the correction term in (26).
RS splitting-instability surface
The replica-symmetric vector reduction (27) also gives a local splitting-instability check for the CCM functional. Let be a self-consistent RS solution, satisfying
Define the diagonal matrix
A first splitting of an RS blanket is governed, at linear order, by the multilayer stability matrix . Thus the RS splitting-instability surface is
where denotes the spectral radius. Equivalently, is similar to the symmetric tridiagonal matrix with and
Therefore (28) can also be written simply as
The derivation of this condition from the one-split perturbation of the CCM functional is given in Appendix § 16. For the two-layer RBM, write and . Then
and the instability condition becomes
In zero external field, on the paramagnetic RS branch , hence , and (30) reduces to the following closed expression:
where . This reproduces the boundary of the annealed/high-temperature region of the bipartite spin glass of Barra–Genovese–Guerra [26]. Thus the splitting-instability computation checks not only the RS value but also the local curvature of the CCM functional in the bipartite direction.
Two-layer RBM and symmetric three-layer DBM
For , write and the parameter is specialized as
Then we can simplify the expressions for the cavity component
and then for the fugacity component
The finite-level CCM functional becomes
This check is non-trivial because the two architectural layers carry distinct coordinates and ; the scalar diagonal specialization would not recover this bipartite structure in asymmetric cases. The balanced RBM formula is recovered by setting .
Consider now with parameters
The size parameter is reduced to this setting as follows:
If the two outer layers are treated symmetrically, we impose
while the middle layer keeps its own coordinate . Then for the cavity component
and for the fugacity component we have
If the two outer layers have the same external field, the functional reduces to
Here and are obtained from the same REM–PPP recursion with outer and middle variance scales displayed above. This check verifies that the middle layer, which is adjacent to two layers, receives twice the cavity scale of an outer layer in the symmetric three-layer model. The balanced three-layer formula is recovered by setting . At zero external field this gives
Nishimori-line deformation as an exact benchmark
The model studied in the previous sections has centered Gaussian couplings but the CCM method is not limited to that. As an exact benchmark for the layer-wise normalization, we can apply the same CCM procedure to the exactly solvable Nishimori deformation of the adjacent-layer DBM. In the ordered-pair convention of Ref. [27], the two orientations of each adjacent-layer edge carry independent couplings with tied mean and variance,
and, when the random fields are included, the layer field is on the same Nishimori line. We remark that this is different from the centered model of the paper; the Nishimori deformation is used only as an exact functional benchmark for the CCM procedure.
The one-body Gaussian mark on the Nishimori line has equal mean and variance. Thus a mark with scale has the form , and its one-body contribution is
In the layer-wise CCM variables, the scale in architectural layer is . The corresponding Nishimori deformation of the CCM functional is
The first term is the Nishimori one-body cavity contribution. The last term is the deterministic correction produced by the non-centered part of the Nishimori couplings. The two changes with respect to the centered CCM functional—the one-body mark and the deterministic edge correction—are derived in Appendix § 17. With the identification
the Nishimori interaction matrix has entries
and the edge coefficient satisfies
Consequently, equation (37) coincides with the finite-dimensional variational integrand for the DBM on the Nishimori line [27]:
This is an important benchmark. It checks the vector structure of the CCM functional, including the matrices , , and the layer-wise coordinate . It does not give a new proof of the Nishimori min–max selection rule: once the two integrands are identified, the known selection rule is inherited.
On the local selection problem
The procedure given above is to obtain the finite-level CCM functional. The problem of selecting the physical value of this functional is a separate variational and stability problem. We do not prove here the global theorem selecting the thermodynamic value. This will be explored in a separate paper. We only recall the usual local criterion naturally associated with the functional.
For a fixed number of blankets, a candidate state is a point
in the admissible domain of (26). If all the relevant coordinates are internal, the local selection equations are the stationarity conditions
When a rate is on the boundary, or two consecutive blankets merge, these equations are replaced by the corresponding Karush–Kuhn–Tucker (KKT) conditions. These conditions select stationary candidates in the blanket coordinates and Poisson exponents.
Stationarity alone is not sufficient. A locally selected state must also be stable with respect to further splitting of any active blanket. For a self-consistent RS solution, this stability test produces the matrix condition already displayed in (28),
If the spectral radius is larger than one, then the corresponding blanket is locally unstable and has to be refined. The same logic applies after one splitting: the stability matrix is obtained by replacing the RS susceptibility by the susceptibility computed in the tilted one-split state.
We remark that this is not a global selection theorem. It is the CCM version of the standard finite-step stationarity and stability analysis used in spin-glass variational functionals, including the SK cavity/Parisi setting and its minimization theory [12, 10, 11, 28], the de Almeida–Thouless stability analysis [29], and bipartite or multilayer mean-field variational structures [26, 30, 27, 19]. In particular, whenever the CCM functional reduces exactly to a known variational integrand, the known selection rule is inherited automatically. The equality of the functionals is the actual consistency check; the selection rule is then a consequence of the known theory.
The usefulness of the local criterion is therefore diagnostic. Away from regimes where a global selection theorem is available, one can study candidate physical states by solving the finite-level stationarity equations, imposing KKT conditions on boundary rates or merged blankets, and recursively refining any blanket whose splitting matrix is unstable. This procedure does not change the derivation of the functional; it is a local method for exploring its possible physical selection.
Relation with other full-RSB formulations
We compare the functional (26) with the bipartite replica calculation of Hartnett, Parker and Geist [37] and the multi-species formulation of Chen, Issa and Mourrat [38]. Both comparisons concern centered Gaussian couplings and zero external fields.
Write the finite hierarchy as the nondecreasing step function
The endpoint conventions are those of § 7. Summation by parts rewrites the quadratic correction as
Bipartite model
For the specialization (33), choose unit coupling scale in Ref. [37] and denote its inverse temperature by . The parameter correspondence is
This matches their coupling normalization, given before Eq. (2.7), with the Hamiltonian (2). Their variance parameters, defined in Eq. (2.43), then satisfy
These are precisely the two cavity variances of our bipartite construction. Their Parisi equation (2.44), written in dimensionless field coordinates, therefore gives the Gaussian recursion (23). The unit-exponent Gaussian step completes each terminal variance to its diagonal value. Combining this contribution with (41) reproduces the endpoint terms and the integral of the product of the two profiles in their Eq. (2.42). Thus (33) coincides with their full-RSB expression in pressure units: the pressure is minus their inverse temperature times their free-energy density. The identification holds for each trial hierarchy, before imposing stationarity.
General multilayer model
Chen, Issa and Mourrat [38] normalize the species overlaps by the total number of spins. The corresponding path and covariance polynomial for our model are
Write for their species-weighted, compensated one-site cascade functional, defined in Eqs. (2.1)–(2.7) and (3.7). At time one half and zero auxiliary field, their Parisi functional (3.14) specializes to
The factor one half matches their Gaussian normalization with the Hamiltonian (2). On the step hierarchy, their one-site calculation uses the same recursion (23). Including its deterministic compensation and the unit-exponent terminal step gives
Substituting (41) and (46) into (45) yields
Thus the complete functional (26) is also obtained by specializing the multi-species Parisi functional. The identity holds at every finite depth and extends to bounded monotone paths by the continuity established in Lemma 3.6 of Ref. [38]. Also their sup–inf characterization of the thermodynamic value at zero deterministic field given in Theorem 1.1 is consistent with our local selection approach given before. These identifications connect the blanket calculation with the replica and Hamilton–Jacobi formulations.
Why CCM works
This section is independent from the derivation of the RBM/DBM functional above and may be skipped by readers interested only in the explicit formula and its checks. The CCM [1] originates in the Kernel/Blanket theory of disordered systems [2, 3, 4]. The blanket-theoretic viewpoint is related to the Free Energy Principle and the Markov-blanket formulation of variational free energy [31, 32, 33] and it can be interpreted as a purely kinetic LFT of the kind described in [34, 35, 36] by identifying the potential energy with the vanishing core contribution. Here we connect the calculations of § 4–§ 7 with this viewpoint [1] in the low temperature limit.
The blanket decomposition separates interactions within each blanket from those connecting it to preceding blankets. Since each blanket crosses the architectural layers, its core contains couplings between cells in adjacent layers. Writing the sum of the interface energies as , the full and IO partition functions are
Gaussian interpolation of the independent core couplings gives
The cell sizes in Eq. (6) make this bound vanish as the largest blanket fraction in each architectural layer tends to zero. Taking the thermodynamic limit at fixed partition and then refining it therefore reduces the pressure to that of the IO model [1 Section 2.1], while the correction field retains the complete edge-shell counts of Eq. (17). The resulting interface is linear in the spins of the new blanket:
Terms involving absent architectural layers are omitted.
This IO model provides the reference for the low-temperature description. We rectify configurations with respect to a global ground state, which maximizes the Hamiltonian in our convention. Energies and Gibbs weights are transported together, and the overlaps are preserved [1 Sections 2.2–2.4]. Excitations then form a dilute gas of flipped spins whose complete energy costs include interactions with preceding and subsequent blankets and between defects.
For finitely many nodes with finitely many defects, independently sampled supports become disjoint with probability tending to one as the cells grow; defect interactions vanish, and the new energy contributions are jointly approximated at fixed disorder by independent sums of complete local costs. Descendants retain their ancestors’ defects and energies, while new nodes receive new arrangements [1 Sections 3.1–3.2]. Two states thus share both spins and energy contributions up to their last common ancestor, which is the origin of the hierarchical structure. Because each node specifies all architectural cells of its blanket, this genealogy determines the overlaps in every species.
To obtain thermodynamic weights, we extend the finite construction to exponential populations within the magnetization shells. At fixed small positive defect densities, these populations can be chosen with small relative intersections throughout every kernel strip, including between nodes with different parents; the thermodynamic limit is taken at fixed partition and followed by dilution [1 Section 3.3]. Along each branch, the complete centered energy is built from centered increments that are linear in the new defects once the prefix is fixed. Their coefficients contain the preceding configuration and the mean fields of subsequent blankets, so sampling the defects leads to the local problem of REM universality [5, 6, 7]. For the coefficient populations selected by the ground state, the regularity needed in this step is physically justified in Ref. [1 Section 3.3]. We then adopt the extensive GREM approximation, representing the conditional increment laws by a common law at each level, independently realized at distinct nodes. Comparing the rarity of energies with the number of available states determines which consecutive levels condense together and fixes their Poisson exponents [1 Section 3.4]. In assembling the resulting Gibbs weights, we retain the energy centers and shell entropies and include all descendant continuations in the mass of each prefix [1 Section 2.4].
This construction supplies the Gibbs masses and species overlaps on the same states. The overlaps in turn determine the complete law of the fresh cavity fields: conditioned on the reservoir, these are jointly Gaussian, and rectification preserves their law because their couplings are independent of the disorder selecting the reference [1 Section 4.2]. For two configurations indexed by the paths and , averaging over the fresh noise gives
Normalizing the spin sums by the respective layer sizes identifies the species overlaps. In the thermodynamic limit, the two covariances give the action of and the quadratic form defined by ; together with the independence of the fresh-noise families, they fix the joint Gaussian law. The physical overlaps include the magnetization contributions from cells beyond the shared prefix, whose spins remain aligned on average with the ground state [1 Section 4.3]. These overlaps increase componentwise to one.
Following the operational correspondence established in Ref. [1 Section 4.7], we choose the constructive cell fractions in each architectural layer to represent their common part and successive increments, retaining the same state masses, ancestry, and averaging exponents. With these fractions, the amplitudes in Eqs. (12) and (19) reproduce the full covariance increments, so the masses and complete cavity fields have the same joint law as in the physical representation. The common linear fields remain inside the hyperbolic cosines throughout the recursive averages and are integrated last at zero exponent. The common correction field remains in the random logarithmic contribution and has zero mean over the fresh noise [1 Sections 4.4–4.6].
Using these complete fields with the same Gibbs masses and ancestors identifies all cavity contributions jointly with the Rϑst. The cascade can be reconstructed from one independent PPP per level by assigning a parent to each node and normalizing the terminal masses globally. Descendants inherit the assignments and Gaussian increments of their ancestors. Transporting the complete fields with these same assignments preserves the random cavity functional in joint distribution [1 Sections 3.5–3.6 and 4.5]. If several physical fibres contribute, their weighted contributions are combined before taking the logarithms, as in § 6. The mean logarithms are then evaluated by Eqs. (23) and (25), yielding Eq. (26) [1 Section 4.6]. The parameters remain free during this calculation and are selected as discussed in § 10 and § 11. An extension of this low temperature argument to arbitrary temperature is possible through the REM universality approach developed in Refs. [6, 7].
Discussion
The calculation above applies the CCM of [1] to dense Ising RBM/DBM models with Gaussian couplings in a layer-wise constructive gauge. The blanket vector of parameter variables
is the vector of cumulative layer fractions in the constructive partition,
and supplies the levels of the nested REM–PPP evaluation of the cavity functional. The model-specific content of the computation is contained in two matrix quantities. The cavity field in layer has scale
while the correction field has scale
These amplitudes enter the blanket components (12) and (19). CCM identifies the blanket decomposition of the cavity variables with the Aizenman–Sims–Starr ROSt, and nested REM–PPP averaging gives the functional [1]. Notice that the balanced gauge is the diagonal specialization
In this specialization one recovers
with the parameter and as follows:
Thus the scalar balanced formula is the diagonal case of the layer-wise functional. The final result is the finite-level functional of (26). For , it gives the dense bipartite Ising RBM with Gaussian couplings and for it gives the corresponding adjacent-layer DBM. Its vector replica-symmetric reduction agrees with the expected multilayer RS structure, while the balanced RS expression is recovered by imposing . Finally, the RS splitting-instability check tests the local curvature of the functional: in the zero-field bipartite specialization it yields , matching the known annealed boundary of the bipartite spin glass [26].
The Nishimori-line deformation gives another exact benchmark: under and , the CCM functional reduces to the known Nishimori variational integrand, confirming the vector layer-wise normalization of and [27].
Section 11 connects the full functional with two independent formulations. For centered Gaussian couplings at zero external field, the bipartite specialization coincides with the replica expression of Hartnett, Parker and Geist [37], while (47) identifies the general multilayer functional with that of Chen, Issa and Mourrat [38]. The CCM thus provides a constructive derivation of these full-RSB expressions through blanket amplitudes and REM universality. In this setting, the thermodynamic value is characterized by the sup–inf theorem of Ref. [38], while § 10 studies local stationarity and splitting stability at finite depth.
The same counting principle suggests natural extensions. If the layer graph is not a path, is replaced by the weighted adjacency matrix of that graph, and , are defined as before. For higher-order interactions, rectangular edge shells would be replaced by the corresponding hyperedge shells, with the appropriate combinatorial factors. The practical CCM rule remains the same: decompose the cavity variables into blanket components, identify their ROSt realization, and perform the nested REM–PPP average with the model-specific amplitudes.
Technical details of the layer-wise CCM computation
This appendix records the normalizations and finite-level identities used in the layer-wise scalar cavity functional. It is not needed for the flow of the main text, but it makes explicit where the factors in § 5 and § 7 come from. We use the boundary convention
With this convention, the balanced specialization gives
for all layers . Also,
Indeed, since and is symmetric with off-diagonal entries ,
The factor 2 appears because the symmetric quadratic form counts both ordered pairs and . For the cavity field, the blanket- contribution in layer is
Conditioning on the reservoir configuration, the cross terms vanish because the fresh couplings are independent and centered. Since the spins are Ising variables,
Using the layer-wise estimate
gives the formula for the variance:
At a fixed blanket, the neighboring-layer sums are independent Gaussians, so their variances add. This fixes the amplitude of the prefix-indexed cavity variable of § 5:
For the correction field, the blanket- shell between two adjacent layers is
Thus and . The layer-wise gauge gives
Let us introduce a symbol for the difference
then the increment decomposes as
The last term is the finite-level core contribution controlled on refinement in the IO reduction. The complete finite shell gives the correction variance below. The correction mark associated with this shell is
The normalization keeps the variance of this edge sum of order one. Again, cross terms vanish by independence and centering, and each diagonal term contributes one because the spins are Ising. Hence we get
Therefore, the variance is rewritten as follows
The contributions of distinct edge-shells use independent fresh couplings, so the variances add:
The and errors in the block and shell cardinalities produce only errors in the corresponding variances.
The Gaussian step of the nested REM–PPP average in § 7 is elementary. If is a standard Gaussian and , then the identity holds
for , the value is 0 for by continuity. Applying this identity blanket by blanket to
gives the expression for the fugacity correction
This is the finite-level form of the CCM interface reduction used in the main text. The balanced scalar expression is recovered by setting .
Finite-size identities for the symmetric three-layer model
This appendix records the finite-size identities for the symmetric three-layer model. They are diagnostic identities only; they are not used in the derivation of the layer-wise CCM functional. The point is that, for moderate layer size, the partition function can be summed exactly over the middle layer. Take
Let and have independent standard Gaussian entries. Writing the middle-layer spin as , the Hamiltonian is
For fixed , the two outer layers can be summed independently. With
one obtains the formula
Thus the formula for the average pressure
can be estimated by exact spin summation for each disorder sample, followed by an empirical average over the disorder.
The same reduction gives an exact finite-size ground-state identity. For fixed , the optimal outer spins align with their local fields, hence
Therefore,
Ground-state values quoted per unit of correspond to setting in this expression. Pair observables can also be sampled without Markov-chain Monte Carlo once the middle layer has been enumerated. The exact Gibbs weight of the middle configuration is proportional to
Conditioned on , the outer spins are independent with means
These identities are useful for checking numerical implementations of the layer-wise functional and of its balanced scalar specialization. They do not determine the physical selection rule in the non-convex regime.
RS splitting-instability calculation
This appendix shows the linear calculation behind the RS splitting-instability surface used in § 8. The calculation is local around a vector RS solution of the CCM functional and tests whether an RS blanket can be split into two nearby blankets. Let be a self-consistent RS solution,
Consider a one-split perturbation of the RS blanket. The two nearby blanket coordinates are denoted by and , and their difference is
The corresponding infinitesimal increment of the cavity variance in layer is
The one-split self-consistency equations say that the difference between the two nearby blanket coordinates is, to first order, the local variance of the magnetization response produced by the additional Gaussian increment. Thus
Define the function
Then the linearized splitting equation is
The boundary of linear stability of the RS solution is therefore
This gives the RS splitting-instability surface. It is useful to rewrite the same condition with a symmetric matrix. Assume first that on the layers under consideration and put
Then, the matrix is defined as follows:
it is symmetric and tridiagonal. Its diagonal entries vanish, and its adjacent-layer entries are
Since is similar to , we have
and the instability condition is . For two-layer RBM, let and . Then
Therefore by doing the algebra we find
and the critical condition is therefore the following:
At zero external field, on the paramagnetic RS branch , one has , and the condition becomes the known identity [26]
Equivalently, since , this is
For a three-layer chain, the symmetric matrix has two non-zero adjacent entries,
Since the spectral radius of a three-node chain satisfies
the splitting-instability surface is
This is the three-layer version of the same RS blanket-splitting criterion.
Nishimori-line deformation of the CCM functional
This appendix records the calculation behind the Nishimori benchmark used in § 9. The model studied in the main text has centered Gaussian couplings. In the ordered-pair convention of § 9, the Nishimori coupling between layers and , in either orientation, is Gaussian with mean and variance tied,
If random fields are included, the layer- field is taken on the same Nishimori line, with mean and variance both equal to . Therefore the one-body Gaussian mark is not centered. A scalar mark of scale has the form
and contributes to the functional as
For the layer-wise CCM coordinate , the scale seen by a spin in the architectural layer is
Thus the Nishimori one-body contribution is
The correction term is modified as well. In the centered model the RS/vector correction is
The non-centered part of the Nishimori couplings produces an additional deterministic contribution on the activated adjacent-layer edge mass:
Therefore the Nishimori correction is
Since , this can be written edge by edge as follows. First,
whereas the additional contribution
Hence the Nishimori correction is
Combining (62) and (63), the Nishimori deformation of the CCM functional is
Equivalently, by (64),
We now perform the comparison with the Nishimori variational integrand in coordinates. In the notation of the exactly solved Nishimori DBM, the layer proportions are denoted by , the adjacent-layer Nishimori parameters by , and
The corresponding interaction matrix acts as
with the missing boundary terms omitted. The finite-dimensional variational integrand on the Nishimori line is
We now identify the Nishimori parameters with the parameters of the centered model by setting
Then the one-body argument in equation (67) becomes exactly the one-body argument in the CCM expression. Indeed,
Therefore we can write
The edge coefficient also matches exactly:
Consequently the edge part of (67) becomes
which is precisely the edge expansion (64) of the CCM Nishimori correction. Substituting these two identities into (67) gives, term by term,
Thus the appendix proves the matching used in Section 9: the one-body Nishimori mark fixes the first term, the deterministic part of the non-centered couplings gives the extra contribution , and the identification (68) matches every adjacent-layer edge coefficient.
Acknowledgments
We thank Nicola Kistler (Goethe University Frankfurt) and ChatGPT (OpenAI) for important discussions and suggestions.
Declaration of Generative AI
Research, writing, and proofreading of the present manuscript were performed with AI assistance (GPT-6 Astra) under strict human guidance and scrutiny. We independently checked all mathematical arguments and take full responsibility for the content of the publication.
References
- [1]S. Franchini, Simplified Parisi Ansatz III: Constructive Cavity Method and Ultrametricity, arXiv:2610.34710 (2026). doi.org/10.48550/arXiv.2610.34710arxiv.org/abs/2610.34710
- [2]S. Franchini, Replica Symmetry Breaking without Replicas, Annals of Physics 450 (2023), 169220.DOI
- [3]S. Franchini, Universal scaling limits for spin networks via martingale methods, Proc. R. Soc. A 481 (2025), 20240774. doi:10.1098/rspa.2024.0774.DOI
- [4]S. Franchini, A simplified Parisi Ansatz, Communications in Theoretical Physics 73 (5) (2021), 055601.DOI
- [5]S. Franchini, A simplified Parisi Ansatz II: REM universality, Chaos, Solitons & Fractals 191 (2025), 115821. Corrigendum: Chaos, Solitons & Fractals 201 (2025), 117395.DOI
- [6]F. Concetti and S. Franchini, REM universality for linear random energy, arXiv:2604.06122 (2026). doi.org/10.48550/arXiv.2604.06122arxiv.org/abs/2604.06122
- [7]F. Concetti and S. Franchini, REM universality and Poisson-Dirichlet Gibbs weights for linear random energy, arXiv:2606.07757 (2026). doi.org/10.48550/arXiv.2606.07757arxiv.org/abs/2606.07757
- [8]D. Sherrington and S. Kirkpatrick, Solvable model of a spin-glass, Phys. Rev. Lett. 35 (1975), 1792–1796. doi:10.1103/PhysRevLett.35.1792.
- [9]G. Parisi, Infinite number of order parameters for spin-glasses, Phys. Rev. Lett. 43 (1979), 1754–1756. doi:10.1103/PhysRevLett.43.1754.
- [10]F. Guerra, Broken replica symmetry bounds in the mean field spin glass model, Commun. Math. Phys. 233 (2003), 1–12.DOI
- [11]M. Talagrand, The Parisi formula, Ann. of Math. 163 (2006), 221–263.
- [12]M. Aizenman, R. Sims, and S. L. Starr, An extended variational principle for the SK spin-glass model, Phys. Rev. B 68 (2003), 214403. doi:10.1103/PhysRevB.68.214403.DOI
- [13]D. H. Ackley, G. E. Hinton, and T. J. Sejnowski, A learning algorithm for Boltzmann machines, Cognitive Science 9 (1985), 147–169.
- [14]P. Smolensky, Information processing in dynamical systems: Foundations of harmony theory, in Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Vol. 1, D. E. Rumelhart, J. L. McClelland, and the PDP Research Group (eds.), MIT Press, 1986, pp. 194–281.
- [15]R. Salakhutdinov and G. E. Hinton, Deep Boltzmann machines, Proceedings of the Twelfth International Conference on Artificial Intelligence and Statistics (AISTATS), PMLR 5 (2009), 448–455.
- [16]A. Barra, G. Genovese, P. Sollich, and D. Tantari, Phase transitions in restricted Boltzmann machines with generic priors, Phys. Rev. E 96 (2017), 042156. doi:10.1103/PhysRevE.96.042156.DOI
- [17]A. Barra, P. Contucci, E. Mingione, and D. Tantari, Multi-species mean field spin glasses. Rigorous results, Ann. Henri Poincaré 16 (2015), 691–708.arxiv.org/abs/1307.5154
- [18]D. Panchenko, The free energy in a multi-species Sherrington–Kirkpatrick model, Ann. Probab. 43 (2015), 3494–3513.DOI
- [19]D. Alberici, P. Contucci, and E. Mingione, Deep Boltzmann machines: Rigorous results at arbitrary depth, Ann. Henri Poincaré 22 (2021), 2619–2642. doi:10.1007/s00023-021-01027-2.DOI
- [20]E. Agliari, L. Albanese, F. Alemanno, and A. Fachechi, Pattern recognition in Deep Boltzmann machines, arXiv:2106.08978, 2021.arxiv.org/abs/2106.08978
- [21]H.-B. Chen and J.-C. Mourrat, On the free energy of vector spin glasses with non-convex interactions, Probab. Math. Phys. 6 (1) (2025), 1–80. doi:10.2140/pmp.2025.6.1. Preprint version: arXiv:2311.08980.DOI
- [22]H.-B. Chen, On free energy of non-convex multi-species spin glasses, ALEA Lat. Am. J. Probab. Math. Stat. 23 (2026), 429–473. doi:10.30757/ALEA.v23-17.DOI
- [23]D. Ruelle, A mathematical reformulation of Derrida’s REM and GREM, Commun. Math. Phys. 108 (1987), 225–239.DOI
- [24]E. Bolthausen and A.-S. Sznitman, On Ruelle’s probability cascades and an abstract cavity method, Commun. Math. Phys. 197 (1998), 247–276. doi:10.1007/s002200050450.DOI
- [25]L.-P. Arguin, Spin glass computations and Ruelle’s probability cascades, arXiv:math-ph/0608045 (2006).DOI
- [26]A. Barra, G. Genovese, and F. Guerra, Equilibrium statistical mechanics of bipartite spin systems, J. Phys. A: Math. Theor. 44 (2011), 245002. arXiv:1012.1261.DOI
- [27]D. Alberici, F. Camilli, P. Contucci, and E. Mingione, The solution of the deep Boltzmann machine on the Nishimori line, Comm. Math. Phys. 387 (2021), 1191–1214. doi:10.1007/s00220-021-04165-0.DOI
- [28]A. Auffinger and W.-K. Chen, The Parisi formula has a unique minimizer, Comm. Math. Phys. 335 (2015), 1429–1444.DOI
- [29]J. R. L. de Almeida and D. J. Thouless, Stability of the Sherrington–Kirkpatrick solution of a spin glass model, J. Phys. A: Math. Gen. 11 (1978), 983–990.
- [30]G. Genovese, Minimax formula for the replica symmetric free energy of deep restricted Boltzmann machines, Ann. Appl. Probab. 33 (2023), 2324–2341. doi:10.1214/22-AAP1868.DOI
- [31]K. Friston, The free-energy principle: a unified brain theory?, Nature Reviews Neuroscience 11 (2010), 127–138.DOI
- [32]K. Friston, Life as we know it, Journal of The Royal Society Interface 10 (2013), 20130475.DOI
- [33]M. D. Kirchhoff, T. Parr, E. Palacios, K. Friston, and J. Kiverstein, The Markov blankets of life: autonomy, active inference and the free energy principle, Journal of The Royal Society Interface 15 (2018), 20170792.DOI
- [34]S. Franchini, G. Bardella, Lattice Field Theory for a network of real neurons, 42nd International Symposium on Lattice Field Theory (LATTICE2025), arXiv:2604.05251 (2026).DOI
- [35]G. Bardella, S. Franchini, L. Pan, R. Balzan, E. Brunamonti, P. Pani, S. Ferraina, Neural activity in quarks language: Lattice Field Theory for a network of real neurons, Entropy 26 (2024), 495.DOI
- [36]G. Bardella, S. Franchini, P. Pani, S. Ferraina, Lattice physics approaches for neural networks, iScience 27 (2024), 111390.DOI
- [37]G. S. Hartnett, E. Parker, and E. Geist, Replica symmetry breaking in bipartite spin glasses and neural networks, Phys. Rev. E 98 (2018), 022116. doi:10.1103/PhysRevE.98.022116. Cited preprint version: arXiv:1803.06442v4.DOI
- [38]H.-B. Chen, V. Issa, and J.-C. Mourrat, Free energy of non-convex multi-species spin glasses with centered Ising spins, arXiv:2606.16636v2 (2026).arxiv.org/abs/2606.16636
