Introduction
Knowing every parameter value of a neural network does not yet determine the network. One must still decide which values become weights and which become biases, which neurons each weight connects, and which computed values later neurons share. This paper asks how hard these decisions are when the required behavior of the network is given by a compact specification.
An instance consists of a multiset W W W of rational numbers, called the inventory, a cap on the number of computational neurons, and a target. A repeated value supplies separate occurrences: three copies of 1 permit three uses. A candidate is a network built from the inventory: each of its edges consumes one occurrence, and so does each neuron’s bias, which may also be absent. We study two rules for the inventory: at-most-once use, which may leave occurrences unused, and exactly-once use, which must consume every occurrence; every problem below has a version for each rule.
Within the cap, the graph may have any depth, width, skip connections and fan-out, and its scalar output is affine. The cap counts the output neuron, but it does not bound the input dimension or the size of the target. When the target is itself a network, its parameters need not belong to W W W and do not consume occurrences.
The model separates which numbers are available from how the network is built. Related formulations constrain the structure more. In permutation training, Cai, Chen, and Qiao [4 ] rearrange the output coefficients over a prescribed univariate hinge basis. Arithmetic expression construction [1 ] assigns supplied numbers to the positions of an expression tree. Here one inventory supplies every weight and bias, and computed values may be shared. The simplest specification, a list of labeled examples, is already hard: two explicit samples give an NP-complete feasibility problem with positive integer parameters (Theorem 3.3 ). Our main question is how specifications that constrain the behavior on all inputs, or on average over a probability distribution, change this baseline.
The specifications we study ask for different kinds of agreement. Exact reproduction of a Boolean circuit requires agreement on every Boolean input, and exact reproduction of a supplied ReLU network requires agreement on every real input. Uniform approximation requires agreement within a tolerance on a box. A population-risk specification instead compares the mean squared error with a rational threshold, under the uniform distribution on the Boolean cube, the uniform distribution on the unit cube, or the standard Gaussian distribution. In each case the candidate’s graph is part of the witness. Hardness for a prescribed graph therefore does not give hardness here, because another legal placement of the same numbers may meet the same specification. Excluding every such alternative graph and placement is the central difficulty in our reductions.
Even an elementary change of realization can violate the inventory constraint. Write ρ ( t ) = max { 0 , t } \rho(t)=\max\{0,t\} ρ ( t ) = max { 0 , t } . The identity ρ ( x ) = 1 2 ρ ( 2 x ) \rho(x)=\frac{1}{2}\rho(2x) ρ ( x ) = 2 1 ρ ( 2 x ) rescales one hidden neuron without changing its function. The left side uses two occurrences of 1, one on the input edge and one on the output edge. The right side uses 2 and 1 / 2 1/2 1/2 instead, which the inventory W = ( 1 , 1 ) W=(1,1) W = ( 1 , 1 ) does not supply. A functional identity therefore need not preserve a legal assignment of occurrences. Our reductions must control every realization that the actual inventory permits, including graphs different from the one used to encode a source witness.
The existential-universal verification pattern behind our second-level upper bounds is established. Existential choice followed by universal verification already arises in succinct circuit minimization [2 ] , and it underlies the second-level completeness results for formula minimization [3 Jaa26] . Equivalence of ReLU networks has a standard coNP verification procedure based on activation patterns and linear constraints [Wur24]. Our contribution is to determine how a consumable numerical inventory interacts with these verification requirements, and with the counting that population risk involves. The hardness results distinguish these sources of complexity at small caps: every synthesis hardness result holds with at most four computational neurons, except the exactly-once versions of Boolean risk, whose cap is the size of the inventory.
Two findings organize the classifications below. The first concerns exact agreement with a ReLU network on all real inputs. For a fixed cap, a polynomial-size set of test inputs separates all candidates that compute different functions, which gives a DP upper bound, tight at cap three, instead of only the general existential-universal bound. Uniform approximation on a compact box behaves differently: even one affine candidate with identical positive inventory values gives second-level completeness. In both comparisons the input dimension and the target size are unrestricted. The second finding concerns population risk, where the main issue is to preserve a numerical choice through an average. Exact continuous synthesis must retain the sign of a rational weighted sum even when that sum is zero. The construction achieves this by canceling a source-dependent second moment, without assuming a positive accuracy gap.
Our Results
Table 1 summarizes the classifications, with all numbers encoded in binary and running times measured in the binary input length.
Table 1. Synthesis from a parameter multiset. Caps count the affine output. Both inventory rules are included unless stated otherwise.
Table 2.
The first row is the numerical baseline. Its two-sample construction excludes every hidden graph, even when a larger cap is permitted. Our main results concern the implicit specifications in the remaining rows. We first treat the universal requirements and then ask how averaging changes their verification.
Universal specifications. The Boolean target in the next theorem is a monotone threshold function with positive coefficients, which is simple to realize when real weights may be chosen freely. Its difficulty arises from the prescribed occurrences and their possible placements.
Theorem 1.1 (Informal; restated as Theorem 4.12). Boolean-circuit exact synthesis is Σ 2 P \Sigma^P_2 Σ 2 P -complete under either inventory rule. Hardness holds with four computational neurons, including the affine output, even when the target is a monotone threshold with positive coefficients.
A target defined on all real inputs changes the classification at a fixed cap. In the next theorem the candidate cap is constant, while the rational ReLU target and the input dimension remain part of the input.
Theorem 1.2 (Informal; restated as Theorem 5.4; see also Theorems 5.3 and 5.9). Full-space ReLU-target synthesis lies in DP for every fixed computational cap under either inventory rule. Cap three is DP-complete under both rules. Under at-most-once use, cap two is coNP-complete.
A problem belongs to DP when it is the conjunction of an NP condition and a coNP condition. The upper bound comes from finitely many separating evaluations, not from listing all candidate functions in polynomial time.
The two conditions have separate sources in the hardness proof. An explicit affine target already gives NP-completeness at cap three (Theorem 5.8), and requiring equality with an arbitrary ReLU target adds the universal condition. When the cap grows with the input, the full-space problem lies between the DP lower bound and the second-level upper bound, and its exact classification is open. The Boolean theorem does not resolve it, because agreement at the vertices of the cube need not extend to agreement on all real inputs.
Uniform approximation on a compact domain. Changing the domain and the error requirement yields a different fixed-cap classification.
Theorem 1.3 (Informal; restated as Theorem 6.2). Uniform approximation of a supplied rational ReLU target on the unit box is Σ 2 P \Sigma^P_2 Σ 2 P -complete, under either inventory rule, already at tolerance one with one affine computational neuron and an inventory of identical positive rational occurrences.
The construction first establishes completeness for explicitly listed multilinear polynomial targets of degree at most five and affine candidates, and a Lovász extension then turns these targets into ReLU targets. The domain and the error requirement therefore matter even when the candidate has no hidden neuron.
Population-risk specifications. The risk results keep the same occurrence constraint, but replace universal agreement by a threshold on an average. For Boolean inputs, verifying a candidate means summing over a finite cube. For continuous inputs, the risk becomes a weighted count whose terms may have different rational denominators. Exact rational weighted counting, the counting model used for continuous inputs, sums a rational weight over all binary strings of a polynomially bounded length. Each weight is a short rational computed exactly in polynomial time, and different weights may have different denominators. The associated decision class asks whether this sum is at least a rational threshold, also computed exactly in polynomial time. Its discrete existential closure allows one polynomial-length binary choice before a single such comparison; it does not allow arbitrary adaptive oracle access. The formal definitions appear in Sections 8 and 8.3.
Theorem 1.4 (Informal; restated as Theorem 7.1; see also Theorems 8.2 and 8.8). *Under uniform Boolean inputs, exact squared-risk synthesis is N P P P \mathrm{NP}^{\mathrm{PP}} NP PP -complete under either inventory rule. Hardness
uses an integer inventory and target labels in { − 1 , + 1 } \{-1,+1\} { − 1 , + 1 } , with cap two for at-most-once use and cap ∣ W ∣ |W| ∣ W ∣ for exactly-once use. Under continuous uniform inputs on the unit cube, comparing the squared risk between two supplied networks with a threshold is complete for exact rational weighted counting, even in one dimension when one of the two networks is zero. Synthesis is complete for its discrete existential closure, with integer-inventory cap-two hardness under either inventory rule.
For Boolean inputs, target labels in { 0 , 1 } \{0,1\} { 0 , 1 } also give N P P P \mathrm{NP}^{\mathrm{PP}} NP PP -completeness when the inventory consists of integers and exactly one occurrence of 1 / 2 1/2 1/2 (Corollary 7.4).
Exact continuous risk also has an obstruction of a different kind, concerning the size of the risk value itself. Some bounded one-dimensional dyadic targets have descriptions of length O ( n log n ) O(n\log n) O ( n log n ) , yet their exact risk denominators require Ω ( 2 n ) \Omega(2^n) Ω ( 2 n ) bits (Theorem 8.4). This shows only that the exact value can be too long to print; it does not separate classes of threshold decision problems. Neither exact-risk classification asserts hardness at a fixed bounded-loss accuracy.
Accuracy gaps and Gaussian inputs. The remaining risk results relax the exact threshold by a gap. In the binary-gap version, in YES instances some legal candidate has risk at most τ \tau τ , and in NO instances every legal candidate has risk at least τ + δ \tau+\delta τ + δ , where δ > 0 \delta>0 δ > 0 is supplied in binary and may be exponentially small. For such a promise problem, membership means that some problem in the stated class, a total separator, accepts every YES instance and rejects every NO instance. Hardness is under deterministic polynomial-time many-one promise reductions, which map YES and NO instances of the source to YES and NO instances, respectively.
Theorem 1.5 (Informal; restated as Theorem 8.6; see also Theorem 9.5). For continuous uniform inputs and for standard Gaussian inputs, binary-gap squared-risk synthesis is N P P P \mathrm{NP}^{\mathrm{PP}} NP PP -complete under either inventory rule, with integer inventory and cap two.
For Gaussian inputs, the exact positive-threshold problem has the corresponding lower bound, but Theorem 9.5 supplies no upper bound for it.
Scaling the inventory and the target keeps the Gaussian node cap and enlarges the absolute gap to the fixed value 1 / 4 1/4 1/4 , while retaining N P P P \mathrm{NP}^{\mathrm{PP}} NP PP -completeness under both rules (Corollary 9.6). This consequence uses unbounded squared loss and a supplied positive threshold that may be numerically large. It gives no classification for a fixed threshold, a relative gap, or a clipped loss.
Our Techniques
Because the graph is chosen together with the placement of the numbers, each reduction must control every network that the inventory can build, not only the intended one. The paragraphs below describe the device each reduction uses for this purpose.
Controlling every occurrence placement. The finite-sample reduction adds a large common positive offset to every source number. An all-ones sample then excludes every path that multiplies two parameters, so all occurrences must lie in a single affine output. A second sample, through its zero and one coordinates, selects a subset of prescribed cardinality. This establishes the numerical baseline without prescribing the graph.
The Boolean reduction needs a stronger converse. Its source problem asks for a small set of forbidden occurrences that intersects every subset with a specified sum. We prove this restricted blocker problem complete by a quantified-CNF construction, together with congruence and nonpower promises that the network proof uses. Two shifted ReLUs implement a threshold ramp, an established representation [35 ] . With threshold P P P , width w > 0 w>0 w > 0 , and shared affine score S S S , the intended output is
ρ ( S − ( P − w ) ) − ρ ( S − P ) w . \frac{\rho(S-(P-w))-\rho(S-P)}{w}. w ρ ( S − ( P − w )) − ρ ( S − P ) .
It is zero when S ≤ P − w S\le P-w S ≤ P − w and one when S ≥ P S\ge P S ≥ P . Small changes to the score encode forbidden source occurrences. The challenge is to force every successful candidate with four computational neurons into this common-score form, with the two copies of each coefficient paid for by distinct supplied occurrences. An alternative placement of a gain as a bias, or an extra path through a shared vertex, would otherwise invalidate extraction of the source choice.
The target’s inputs include several query blocks, groups of coordinates that are repeated so that some block stays free of unintended parameters. The proof begins with an obstruction visible on one two-coordinate face of the target restricted to a query block. On this face the target is an OR: its four values are 0 , 1 , 1 , 1 0,1,1,1 0 , 1 , 1 , 1 , so the mixed difference is 1 − 1 − 1 + 0 = − 1 1 - 1 - 1 + 0 = -1 1 − 1 − 1 + 0 = − 1 . Helmbold and Long’s supermodularity property [21 Lemma 12] implies that a ReLU network with nonnegative edge weights has nonnegative mixed differences on Boolean inputs, even with arbitrary biases (Lemma A.4 ). It therefore cannot realize this face.
This obstruction alone does not identify the candidate, and two separated numerical scales give stronger restrictions: Boolean output equalities become formal identities under each witness’s actual activation masks. Replication leaves an uncontaminated query block, on which these identities exclude all unintended gain and negative-parameter positions. The full target then identifies the shared score, and anchor inputs recover a source blocker. Section A gives a structural map and the complete case analysis.
Separating finitely many candidate behaviors. For a fixed cap, a disagreement between two candidates has a witness supported on a constant number of input coordinates. The bound counts their hidden preactivations and one output difference. Enumerating local graphs on those subspaces yields a polynomial set of separating inputs. All legal candidates consistent with the target on the separating inputs compute the same function on the full space. Synthesis therefore requires a consistent candidate and the absence of any consistent candidate that disagrees with the target elsewhere. Existence of a consistent candidate is one NP condition; existence of a consistent disagreeing candidate is a second NP condition. Taking the first condition and the complement of the second gives the DP upper bound.
For the matching lower bound at cap three, shifted numerical matching forces the two output gains to be unit occurrences. The remaining required positions exhaust the inventory and exclude an internal hidden edge. A separate target slice encodes unsatisfiability, supplying the second conjunct of DP hardness.
Uniform error through covering. For compact approximation, a one-node candidate chooses feature coordinates for a succinct set cover. Two selector coordinates calibrate all possible bias and nonfeature placements away. Local circuit constraints form a multilinear penalty, whose consistent Boolean vertices require the chosen sets to cover every input. Multilinearity extends the inequalities to the entire box. The standard Lovász extension [19 ] produces a rational ReLU target with the same vertex values and the same uniform-error test against each affine candidate.
Preserving choice in an average. For Boolean risk, parity blocks hide source random bits from every candidate reading only the number of coordinates its inventory permits. We use the parity device of Koriche, Lagniez, and Tran [25 ] with existential-majority satisfiability [28 ] . An integer-valued pointwise quadratic inequality excludes all unintended candidates.
For the upper bound on continuous risk, activation masks and complete flags of polytope faces write the risk as a sum of simplex integrals, each a short rational summand of a counting problem. For the lower bound, tent folding [33 ] decodes words into dyadic intervals, and narrow triangular payloads realize reciprocal denominators and numerator multiplicities. Balanced occurrence pairs restrict low-risk candidates with two computational neurons to corner functions encoding existential strings. A continuous modular-sharing construction, based on the uniform sharing of Chor and Kushilevitz [6 Section 4] , keeps a virtual uniform coordinate independent of every affordable raw-coordinate observation. The exact construction must do more than the binary-gap reduction. Smoothing an encoded bit perturbs the risk, and such a perturbation can reverse a comparison in which the source sum equals zero. Writing T T T for the target and N N N for a legal candidate, expand the loss as
E [ ( T − N ) 2 ] = E [ T 2 ] + E [ N 2 ] − 2 E [ N T ] . \mathbb{E}\left[(T-N)^2\right] = \mathbb{E}\left[T^2\right] + \mathbb{E}\left[N^2\right] - 2\mathbb{E}\left[NT\right]. E [ ( T − N ) 2 ] = E [ T 2 ] + E [ N 2 ] − 2 E [ N T ] .
The aim is to keep the source sum in the cross term while the target energy E [ T 2 ] \mathbb{E}[T^2] E [ T 2 ] stays independent of that sum. For a fixed nonnegative envelope height U U U , the payload B B B , integrated over its hidden uniform coordinate, satisfies
3 ∫ B 2 = 2 U ∫ B . 3\int B^2 = 2U\int B. 3 ∫ B 2 = 2 U ∫ B .
Two positive copies on zero background and one negative copy on background U U U , each occupying an interval of length λ \lambda λ , change the energy by λ ( 3 ∫ B 2 − 2 U ∫ B ) = 0 \lambda(3\int B^2 - 2U\int B) = 0 λ ( 3 ∫ B 2 − 2 U ∫ B ) = 0 . Their first-moment change is still λ ∫ B \lambda\int B λ ∫ B . Conditional independence makes this retained mean available against every permitted candidate; the numerical separation argument then excludes all candidates except those encoding source choices.
Thus the threshold comparison preserves the sign of the source sum, including equality, rather than replacing it by a promised gap (Theorem 8.8 ).
For Gaussian inputs, a sign decoder and an independent homogeneous amplitude give the forced-zero counting lower bound. The use of Boolean-to-Gaussian lifting follows earlier learning-hardness constructions [12 , 5 ] . To encode existential choice, opposite unit occurrences on paired coordinates define rare hinges. Their small cross moments preserve the source signal, while a second Hermite coefficient excludes every competing cap-two placement. Uniform growth, tail and Lipschitz bounds turn continuous risk with a binary gap into a polynomial-time fair-bit counting experiment. Finally, scaling all inventory occurrences and the target enlarges the Gaussian absolute gap; the proof separately controls the candidates’ affine skip terms.
Other Related Work
We compare our results with three further lines of work: circuit minimization and neural verification, numerical resource constraints and training, and integration and counting.
Minimization and neural verification. Circuit minimization from truth tables [31 ] differs from a succinctly supplied target; the latter already leads to existential-universal verification [2 ] . Buchfuhrer and Umans [3 ] and Jaakkola [Jaa26] establish second-level completeness for Boolean formula minimization. Their formula-tree occurrence constraints do not identify the possible sharing and numerical placements of a ReLU graph. Williams’s exact Boolean representation results [35 ] concern freely chosen real coefficients and therefore do not impose the supplied numerical resource constraint.
Wurm’s neural verification framework [Wur24] includes equivalence via activation patterns and linear constraints. His finite-alphabet universal-trigger problem is Σ 2 P \Sigma^{P}_{2} Σ 2 P -complete [Wur26, Theorem 5.2], so second-level neural decision problems precede this work. Those choices modify inputs of a fixed network; ours consume weight and bias occurrences while choosing a graph with a constant computational cap. Lee [27 ] studies minimizing the hidden-neuron count under positive-tolerance L p L^{p} L p approximation with freely chosen real parameters and two hidden layers. Its approximation guarantee and numerical freedom differ from full-space exact synthesis with a fixed inventory.
Parameter symmetries give a different tractable equivalence regime. Gegenfurtner, Grillo, and Montúfar [15 Proposition 14] give a polynomial-time equivalence test for two generic parameter choices in the same three-layer bottleneck architecture. In their architecture, the first hidden layer is no wider than the input. Our fixed-cap synthesis theorem searches over occurrence placements and arbitrary permitted graphs, allows degenerate parameters, and leaves the target size unrestricted.
Numerical resources and training. Permutation training [4 ] assigns output coefficients over a prescribed hinge basis, whereas our single inventory governs every parameter position. Arithmetic expression construction [1 ] uses expression trees and supplied occurrences, rather than shared ReLU graphs. Doron-Arad and Mossel [DAM26] prove NP-completeness for finite-sample ReLU training with polynomial-bit rational parameters on a supplied architecture. Our finite-sample theorem restricts all parameters to an input multiset and permits alternative architectures; the higher classifications concern implicit specifications. Subset-sum interdiction fits the general blocking framework of Grüne and Wulf [20 ] ; our source lemma proves the additional arithmetic promises directly. The distinct-number numerical matching premise for cap-three hardness is due to Hulett, Will, and Woeginger [22 Appendix, Corollary 8] .
Integration and counting. We use the exact polynomial-time rational-weight model of de Campos, Stamatopoulos, and Weyland [10 ] , and distinguish it from their broader approximation-defined model. The general arithmetic upper bound we quote is Meiburg’s recent preprint [29 ] . Feischl, Gräßle, and Zehetgruber already use folding to establish fixed-dimensional ReLU quadrature hardness [14 ] . Morettin, Passerini, and Sebastiani formulate probabilistic neural verification through weighted model integration [30 ] . Our exact continuous classification gives reciprocal rational-weight realization and then preserves existential numerical choice in synthesis. Lawrence’s long polytope-volume denominators [26 ] precede the output-size phenomenon; our example uses one real coordinate and dyadic network parameters.
The Gaussian counting reduction concerns threshold comparison, whereas Rannen-Triki et al. [32 ] study whether a supplied ReLU realization has zero norm. Gaussian agnostic learning of a freely chosen ReLU [16 ] has sample access and a different output requirement. Neither that learning problem nor smooth perturbations of Boolean data replace the single standard Gaussian probability distribution used here.
Discussion
The same finite-inventory model accommodates four different requirements: universal Boolean agreement, agreement as piecewise-affine functions on all real inputs, uniform approximation on a compact box, and an average squared loss. In each case the proof must combine the verification mechanism with a numerical choice that survives every alternative architecture. The resulting classes are not ordered by neuron count alone. The input domain, the target representation and the promise guarantee change the class to which our results assign the problem, and the inventory rule changes the caps at which our hardness proofs apply. For example, the Boolean-risk hardness uses cap two only under at-most-once use, and cap ∣ W ∣ |W| ∣ W ∣ under exactly-once use. Two-neuron hardness under both inventory rules holds for the continuous uniform and Gaussian results.
The exact continuous classification and the binary-gap classifications answer different questions. A binary gap may be exponentially small, and the fixed absolute Gaussian gap allows both unbounded squared loss and growing thresholds. Neither result establishes hardness of bounded-loss approximation at a fixed accuracy. Nor can one decide whether a Gaussian risk equals a rational threshold by enumerating candidates and approximating each risk, since approximations do not certify equality.
Open Problems
We close with four open questions. The first concerns exact synthesis on the full space when the cap grows with the input; the other three concern risk synthesis and arise from the distinctions just drawn between the exact and the gap classifications.
Unrestricted full-space exact synthesis. Fixed-cap membership in DP and cap-three hardness leave open how the complexity changes when the cap grows with the input. With the cap part of the input, the problem is DP-hard by Theorem 5.9 and belongs to Σ 2 P \Sigma_{2}^{\mathrm{P}} Σ 2 P by Theorem 5.2 ; the paper does not determine where it lies between these bounds.
Open Problem 1. Determine the exact deterministic polynomial-time many-one reduction complexity of full-space synthesis for a supplied rational ReLU target, a rational inventory used at most once, arbitrary permitted graphs, and an affine output. In particular, determine whether it is Σ 2 P \Sigma_{2}^{\mathrm{P}} Σ 2 P -complete.
Clipped risk with an inverse-polynomial gap. For uniform Boolean inputs, define the clipped risk by
R ‾ T ( N ) = E [ min { 1 , ( N ( X ) − T ( X ) ) 2 } ] . \overline{R}_{T}(N)=\mathbb{E}[\min\{1,(N(X)-T(X))^{2}\}]. R T ( N ) = E [ min { 1 , ( N ( X ) − T ( X ) ) 2 }] .
The input supplies 0 < δ ≤ 1 0<\delta\le1 0 < δ ≤ 1 , 0 ≤ τ ≤ 1 − δ 0\le\tau\le1-\delta 0 ≤ τ ≤ 1 − δ , and a unary string of length ⌈ 1 / δ ⌉ \lceil1/\delta\rceil ⌈ 1/ δ ⌉ , so the gap is at least inverse-polynomial in the input length. In YES instances some legal candidate has clipped risk at most τ \tau τ ; in NO instances every candidate has clipped risk at least τ + δ \tau+\delta τ + δ .
Open Problem 2. Determine the deterministic polynomial-time many-one promise-reduction complexity of this clipped-risk synthesis problem for a rational inventory used at most once, an independent rational ReLU target and an affine output. In particular, determine whether it is complete for MA.
Here MA permits a polynomial certificate followed by a randomized polynomial-time check. The exact-risk reductions may distinguish only an exponentially rare assignment fiber; clipping does not amplify its probability.
Exact standard-Gaussian comparison. The binary-gap theorem gives no algorithm for deciding exact equality with a positive threshold.
Open Problem 3. Determine whether standard-Gaussian squared-risk synthesis is decidable for an independent rational ReLU target, a rational inventory used at most once, a node cap, an explicitly listed dimension and a positive rational threshold. If it is, determine its deterministic polynomial-time many-one reduction complexity.
Theorem 9.5 already gives an N P P P \mathrm{NP}^{\mathrm{PP}} NP PP lower bound with integer inventory and cap two.
Fixed Gaussian thresholds. Scaling to a fixed absolute gap changes the threshold; it does not settle the following normalized question.
Open Problem 4. Determine the exact deterministic polynomial-time many-one promise-reduction complexity of standard-Gaussian squared-risk synthesis, with YES risk at most one and NO risk at least 5 / 4 5/4 5/4 , an independent rational ReLU target, a rational inventory, an explicitly listed dimension and an affine output. Treat both inventory rules. In particular, determine whether it is Σ 2 P \Sigma_{2}^{\mathrm{P}} Σ 2 P -complete.
AI Disclosure
This work was developed with substantial assistance from Codex and Claude Code. These tools were used to discover almost all of the proofs in this work, find relevant literature, and write parts of the prose; the author wrote the rest. The author posed the problem, guided the development, and is responsible for any errors.
Paper Organization
Section 2 defines the network and inventory conventions, and Section 3 proves the finite-sample baseline. Section 4 proves the Boolean exact classification, with its complete placement converse in Section A. Section 5 treats exact ReLU targets on all real inputs and the fixed-cap classifications. Section 6 treats uniform approximation on a compact box. Sections 7 to 9 treat population squared risk under the three input probability distributions.
Preliminaries
We write R , Q , Z \mathbb{R},\mathbb{Q},\mathbb{Z} R , Q , Z for the real numbers, rational numbers and integers. Unless another encoding is explicitly specified, integer inputs are encoded in binary, and a rational input is encoded by a signed integer numerator and a positive integer denominator. The binary input length is the total number of bits in these encodings and the explicitly supplied graphs and coordinate lists. For a nonnegative integer r r r , write [ r ] = { 1 , … , r } [r]=\{1,\ldots,r\} [ r ] = { 1 , … , r } , with [ 0 ] = ∅ [0]=\varnothing [ 0 ] = ∅ . The symbols ⌊ t ⌋ , ⌈ t ⌉ \lfloor t\rfloor,\lceil t\rceil ⌊ t ⌋ , ⌈ t ⌉ denote the greatest integer at most t t t and the least integer at least t t t , respectively. For an integer r ≥ 0 r\ge0 r ≥ 0 , r ! = ∏ i = 1 r i r!=\prod_{i=1}^{r}i r ! = ∏ i = 1 r i , with 0 ! = 1 0!=1 0 ! = 1 ; empty sums and products have values zero and one, respectively. An expression poly ( L ) \operatorname{poly}(L) poly ( L ) denotes a fixed polynomial bound in the input length L L L . For nonnegative functions of an integer variable, f = O ( g ) f=O(g) f = O ( g ) means f ≤ C g f\le Cg f ≤ C g eventually for a constant C > 0 C>0 C > 0 , while f = Ω ( g ) f=\Omega(g) f = Ω ( g ) means f ≥ c g f\ge cg f ≥ c g eventually for a constant c > 0 c>0 c > 0 . Logarithms are natural unless a base is displayed.
Let d ≥ 1 d\ge1 d ≥ 1 be an integer, the input dimension, and write x = ( x 1 , … , x d ) ∈ R d x=(x_{1},\ldots,x_{d})\in\mathbb{R}^{d} x = ( x 1 , … , x d ) ∈ R d for an input. For vectors a , x ∈ R d a,x\in\mathbb{R}^{d} a , x ∈ R d , their dot product is a ⋅ x = ∑ j = 1 d a j x j a\cdot x=\sum_{j=1}^{d}a_{j}x_{j} a ⋅ x = ∑ j = 1 d a j x j . The vector e j e_{j} e j has coordinate one at j j j and zero elsewhere; 1 \mathbf{1} 1 denotes the all-ones vector in the stated dimension. The support of a vector is the set of coordinates where it is nonzero. Write ∥ x ∥ 2 = ( ∑ j x j 2 ) 1 / 2 \lVert x\rVert_{2}=(\sum_{j}x_{j}^{2})^{1/2} ∥ x ∥ 2 = ( ∑ j x j 2 ) 1/2 for its Euclidean length and ∥ x ∥ ∞ = max j ∣ x j ∣ \lVert x\rVert_{\infty}=\max_{j}|x_{j}| ∥ x ∥ ∞ = max j ∣ x j ∣ for its infinity norm.
An affine form is b + a ⋅ x b+a\cdot x b + a ⋅ x , where a ∈ R d a\in\mathbb{R}^{d} a ∈ R d and b ∈ R b\in\mathbb{R} b ∈ R ; it is linear when b = 0 b=0 b = 0 . An affine hyperplane is { x : a ⋅ x = b } \{x:a\cdot x=b\} { x : a ⋅ x = b } with a ≠ 0 a\ne0 a = 0 . The vector a a a is a normal vector to this hyperplane. The corresponding closed and open affine halfspaces are { x : a ⋅ x ≤ b } \{x:a\cdot x\le b\} { x : a ⋅ x ≤ b } and { x : a ⋅ x < b } \{x:a\cdot x<b\} { x : a ⋅ x < b } . An open ball with center x x x and radius ε > 0 \varepsilon>0 ε > 0 is { y : ∥ y − x ∥ 2 < ε } \{y:\lVert y-x\rVert_{2}<\varepsilon\} { y : ∥ y − x ∥ 2 < ε } . A set is open if each of its points has an open ball contained in the set. The interior of any set consists of all points with such a ball contained in the set.
Write ρ ( t ) = max { 0 , t } \rho(t)=\max\{0,t\} ρ ( t ) = max { 0 , t } for the ReLU function. A function f : R → R f:\mathbb{R}\to\mathbb{R} f : R → R is convex if
f ( λ u + ( 1 − λ ) v ) ≤ λ f ( u ) + ( 1 − λ ) f ( v ) ( u , v ∈ R , 0 ≤ λ ≤ 1 ) . f(\lambda u+(1-\lambda)v)\le\lambda f(u)+(1-\lambda)f(v)\qquad(u,v\in\mathbb{R},0\le\lambda\le1). f ( λ u + ( 1 − λ ) v ) ≤ λ f ( u ) + ( 1 − λ ) f ( v ) ( u , v ∈ R , 0 ≤ λ ≤ 1 ) .
For f = ρ f=\rho f = ρ , the right side is both nonnegative and at least λ u + ( 1 − λ ) v \lambda u+(1-\lambda)v λ u + ( 1 − λ ) v , so it is at least their maximum ρ ( λ u + ( 1 − λ ) v ) \rho(\lambda u+(1-\lambda)v) ρ ( λ u + ( 1 − λ ) v ) , the left side; hence ρ \rho ρ is convex.
Write W = ( w 1 , … , w m ) W=(w_{1},\ldots,w_{m}) W = ( w 1 , … , w m ) for the inventory, an explicitly listed multiset of m ≥ 1 m\ge1 m ≥ 1 rational numbers, where m m m is an integer, and let n max ≥ 1 n_{\max}\ge1 n m a x ≥ 1 be an integer, the cap on the number of computational vertices (also called the computational-node budget). A multiset permits repeated values, and its indices distinguish their occurrences. The operation ⊎ \uplus ⊎ joins multisets while retaining every occurrence.
A directed graph consists of vertices and edges ( u , v ) (u,v) ( u , v ) pointing from u u u to v v v . A directed path follows a sequence of such edges; a directed cycle is a nonempty path returning to its starting vertex. A graph is acyclic when it has no directed cycle. A topological order lists its vertices so every edge points forward. The empty graph has the empty order. Every nonempty finite acyclic graph has a vertex with no incoming edge, since otherwise repeatedly following incoming edges would revisit a vertex and give a cycle. Removing that vertex and repeating lists the vertices of the graph in a topological order. A predecessor of v v v is a vertex u u u with an edge ( u , v ) (u,v) ( u , v ) .
Definition 2.1 (Permitted network). A permitted network is a finite directed acyclic graph with d d d input vertices and between one and n max n_{\max} n m a x computational vertices. Input vertices have no incoming edges; every edge enters a computational vertex. One computational vertex is the scalar output. The other computational vertices are called hidden vertices. Every computational vertex lies on a directed path to the output and has at least one incoming edge or a present bias. For each ordered pair of vertices, there is at most one edge. A hidden vertex v v v computes
z v = max { 0 , b v + ∑ u : ( u , v ) ∈ E a u v z u } . z_v = \max\left\{0,b_v+\sum_{u:(u,v)\in E}a_{uv}z_u\right\}. z v = max ⎩ ⎨ ⎧ 0 , b v + u : ( u , v ) ∈ E ∑ a uv z u ⎭ ⎬ ⎫ . Here E E E is the edge set, a u v a_{uv} a uv is the weight of edge ( u , v ) (u,v) ( u , v ) , and b v b_v b v is the bias at v v v , with b v = 0 b_v=0 b v = 0 when that bias is absent. Input vertex j j j has value z j = x j z_j=x_j z j = x j . The output computes an affine form in its predecessors’ values. The sample-based results and the bounds of Theorem 4.2 also hold if the output applies ReLU to that form. The completeness result, Theorem 4.12 , uses the affine-output convention.
Each edge and each present bias is assigned a distinct index from { 1 , … , m } \{1,\ldots,m\} { 1 , … , m } and takes the corresponding value w i w_i w i . Unassigned occurrences are unused. No other numerical weights or present biases are allowed. In the exactly-once variant, every index must be assigned.
Throughout, a network or an assignment of inventory occurrences is called legal when it is permitted in the sense of Definition 2.1, under whichever inventory rule is in force. A computational vertex is called legal when it satisfies the incoming-edge-or-present-bias and path-to-output conditions of Definition 2.1. In Section 4 and Section A, legal refers to the at-most-once rule unless the exactly-once rule is stated explicitly; for the exactly-once case, the proof of Theorem 4.12 enlarges the blocker in the forward direction and, in the converse, uses that every exactly-once witness is an at-most-once witness.
The zero implicit in an absent bias is not an extra supplied parameter. A present bias equal to zero does consume a zero occurrence. Hidden activations and summation are operations; they do not provide free weighted edges. A vertex may have several outgoing edges, which is called fan-out, but each outgoing edge consumes its own weight occurrence. A direct input-to-output edge is called a skip edge. The convention on vertices excludes disconnected storage of unused parameters. It still permits an assigned edge whose contribution vanishes at a particular sample.
If input vertices are included in the requested total node limit, replace n max n_{\max} n m a x here by that limit minus d d d . A budget below one computational vertex is infeasible in this model. An optional output ReLU has no separate numerical parameter. All classifications below use the affine output unless explicitly stated otherwise.
The affine expression supplied to a hidden ReLU is its preactivation. At a fixed input, the actual activation indicator η v ∈ { 0 , 1 } \eta_v\in\{0,1\} η v ∈ { 0 , 1 } is one when the preactivation of hidden vertex v v v is positive, and zero otherwise. The list of these indicators over all hidden vertices is the activation pattern at that input. Freezing a pattern replaces each active ReLU by the identity and each inactive ReLU by zero. For fixed numerical parameters, the vertex values of the frozen network are affine forms in the input, by induction in topological order. The closed activation region of a pattern is the set of inputs at which every vertex labeled active has nonnegative preactivation and every vertex labeled inactive has nonpositive preactivation. When a verification argument guesses a pattern and imposes only these weak inequalities, as in the proof of Theorem 5.2 , a zero preactivation may receive either label: both substitutions give the correct value zero.
Let q ≥ 1 q\ge1 q ≥ 1 be an integer, the sample count, and let
D q = ( ( x 1 , y 1 ) , … , ( x q , y q ) ) D_q=((x_1,y_1),\ldots,(x_q,y_q)) D q = (( x 1 , y 1 ) , … , ( x q , y q ))
be an explicitly listed sample, with x i ∈ Q d x_i\in\mathbb{Q}^d x i ∈ Q d and y i ∈ Q y_i\in\mathbb{Q} y i ∈ Q . For a network N, its total squared empirical loss is
L D q ( N ) = ∑ i = 1 q ( N ( x i ) − y i ) 2 . L_{D_q}(N)=\sum_{i=1}^{q}(N(x_i)-y_i)^2. L D q ( N ) = i = 1 ∑ q ( N ( x i ) − y i ) 2 .
The decision problem asks whether a permitted N N N satisfies L D q ( N ) ≤ τ L_{D_q}(N) \le\tau L D q ( N ) ≤ τ , for a rational threshold τ ≥ 0 \tau\ge0 τ ≥ 0 . Dividing by q q q gives mean squared error and only rescales the threshold.
We next bound the search space. For a fixed inventory and budget, let H W , n max \mathcal{H}_{W,n_{\max}} H W , n m a x be the class of functions computed by permitted networks. Distinct graphs can compute the same function; a count of graph descriptions is therefore an upper bound on the class size. Put
n = min { m , n max } , K = n ( d + n + 1 ) , N H = ( n + 1 ) 2 min { ( K + 1 ) m , ( m + 1 ) K } . n = \min\{m,n_{\max}\}, \qquad K = n(d+n+1), \qquad N_{\mathcal{H}} = (n+1)^2 \min\{(K+1)^m,(m+1)^K\}. n = min { m , n m a x } , K = n ( d + n + 1 ) , N H = ( n + 1 ) 2 min {( K + 1 ) m , ( m + 1 ) K } .
Here K K K bounds the possible edge and bias positions after labeling the computational vertices.
Proposition 2.2 (Finite class size). Every permitted network has at most m m m computational vertices, and
∣ H W , n max ∣ ≤ N H . |\mathcal{H}_{W,n_{\max}}| \le N_{\mathcal{H}}. ∣ H W , n m a x ∣ ≤ N H . The same bound holds for the exactly-once variant.
Proof. For each computational vertex choose one incoming edge or its present bias. The positions chosen for different vertices are distinct because they have different target vertices. Each consumes a different occurrence of W W W . Thus the number of computational vertices is at most m m m , even if n max n_{\max} n m a x is encoded in binary.
Label at most n n n computational vertices. There are at most n ( d + n ) n(d+n) n ( d + n ) directed edge positions and n n n bias positions, hence at most K K K positions. This count includes self-loops and other forbidden edges; retaining them only increases the upper bound. There are at most n n n choices of the number of computational vertices and at most n n n choices of the output label.
One encoding sends each of the m m m occurrence indices to a position or declares it unused. There are at most ( K + 1 ) m (K+1)^m ( K + 1 ) m such assignments. A second encoding assigns to each position either an occurrence index or the declaration that the position is absent. There are at most ( m + 1 ) K (m+1)^K ( m + 1 ) K such assignments. In each encoding, reject assignments with repeated use, collisions, cycles, invalid vertices, or the wrong number of vertices. Every permitted labeled network has an encoding of both kinds. Multiplying the smaller bound by n 2 ≤ ( n + 1 ) 2 n^2 \le(n+1)^2 n 2 ≤ ( n + 1 ) 2 proves the assertion. Repeated numerical values and functionally identical descriptions decrease the actual class size. Requiring every occurrence to be assigned restricts the class further. □
For the finite-sample problem, both encodings also give exhaustive search algorithms: enumerate assignments, discard invalid ones, and evaluate the survivors. If L L L denotes the total binary input length, the running time is at most
N H poly ( L ) . N_{\mathcal{H}} \operatorname{poly}(L). N H poly ( L ) .
The counting bound may be loose. Taking logarithms in the definition of N H N_{\mathcal{H}} N H gives
log N H = O ( min { m log ( K + 1 ) , K log ( m + 1 ) } + log ( n + 1 ) ) . \log N_{\mathcal{H}} = O\bigl(\min\{m\log(K+1),K\log(m+1)\}+\log(n+1)\bigr). log N H = O ( min { m log ( K + 1 ) , K log ( m + 1 )} + log ( n + 1 ) ) .
For fixed d d d and n max n_{\max} n m a x , K K K is constant and the search is polynomial in the input length. This does not conflict with hardness below, where d d d grows.
The linear span of finitely many vectors consists of their linear combinations, with arbitrary real coefficients. They are linearly independent when a linear combination equal to zero has all coefficients zero. A basis is a linearly independent spanning family; the dimension of the span is the size of a basis. The rank of a finite vector family is the dimension of its span, and the rank of a matrix is the rank of its columns. The affine span consists of combinations whose coefficients sum to one. A convex combination has nonnegative coefficients summing to one. For a nonzero vector a a a , write ker a = { z : a ⋅ z = 0 } \ker a = \{z : a \cdot z = 0\} ker a = { z : a ⋅ z = 0 } . Two vectors are proportional when one is a scalar multiple of the other.
For an integer r ≥ 0 r \ge0 r ≥ 0 , a permutation of [ r ] [r] [ r ] is a rearrangement in which each element appears once. For an r × r r \times r r × r matrix B B B , its determinant is
det B = ∑ π ( − 1 ) inv ( π ) ∏ i = 1 r B i , π ( i ) , \det B = \sum_{\pi}(-1)^{\operatorname{inv}(\pi)}\prod_{i=1}^{r} B_{i,\pi(i)}, det B = π ∑ ( − 1 ) inv ( π ) i = 1 ∏ r B i , π ( i ) ,
where the sum ranges over permutations π \pi π of [ r ] [r] [ r ] , and inv ( π ) \operatorname{inv}(\pi) inv ( π ) counts pairs i < j i<j i < j with π ( i ) > π ( j ) \pi(i)>\pi(j) π ( i ) > π ( j ) . The empty determinant is one. A square matrix is called nonsingular, or invertible, when its determinant is nonzero. The formula is linear in each row and column and changes sign when two rows or two columns are exchanged; pairing the permuted terms proves these properties. It therefore vanishes when two rows or two columns coincide. Subtracting a multiple of one row from another leaves the determinant unchanged.
Elimination chooses a nonzero entry, called a pivot, in the column currently being processed, among the rows not yet used. Exchange its row into the next diagonal position, then subtract suitable multiples of that row from the rows below it to make their entries in that column zero. These row operations are invertible, meaning that they can be reversed, and therefore preserve every linear dependence or independence relation among the columns.
Elimination shows that an r × r r \times r r × r matrix B B B with linearly independent columns has det B ≠ 0 \det B \ne0 det B = 0 . If r = 0 r = 0 r = 0 , the determinant is already one. For r ≥ 1 r \ge1 r ≥ 1 and independent columns, the first column is nonzero, so a first pivot exists. After this step the lower-right square block also has independent columns: a dependence there would give a nontrivial combination of the remaining full columns supported only in the first row, which could be canceled using the first column. That would contradict the independence of the full columns. Repeating the argument on each remaining block gives a triangular matrix, with every entry below the diagonal zero and every diagonal pivot nonzero. Only the identity permutation can contribute a nonzero term to its determinant, so that determinant is the product of these pivots. Row exchanges change only the determinant’s sign and the other elimination operations leave it unchanged, proving det B ≠ 0 \det B \ne0 det B = 0 .
For such a matrix B B B , if B z = y Bz = y B z = y , replace column j j j of B B B by y = ∑ i z i B i y = \sum_i z_i B_i y = ∑ i z i B i , where B i B_i B i is column i i i , and denote the resulting matrix by B ( j ) B^{(j)} B ( j ) . Column linearity and vanishing for repeated columns give det B ( j ) = z j det B \det B^{(j)} = z_j \det B det B ( j ) = z j det B , so
z j = det B ( j ) det B . z_j = \frac{\det B^{(j)}}{\det B}. z j = det B det B ( j ) .
This determinant-ratio formula is Cramer’s rule.
The elimination argument also shows that a rectangular matrix with k k k independent columns contains a nonsingular k × k k \times k k × k submatrix formed by selecting k k k original rows. At each stage the residual columns remain independent by the same lifting argument used for the square block above, so the next column has a nonzero entry among the unused rows. Choose that row as the next pivot row. After k k k stages, these selected rows form a square triangular system with nonzero diagonal pivots. Each transformed selected row is its original row minus a linear combination of earlier selected rows. Thus those same row operations, applied just to the selected original square submatrix, give the nonsingular triangular matrix. Their invertibility proves that the original submatrix is nonsingular as well. For k = 0 k = 0 k = 0 , choose the empty submatrix, whose determinant is one.
Later verification and integration arguments use rational linear programming. A rational linear program consists of finitely many non-strict affine inequalities and a linear objective, all with binary-encoded rational coefficients. A point is feasible if it satisfies every inequality; maximizing the objective asks for its largest value on the feasible points. The following standard result does not require the feasible region to have an interior or to be bounded.
Theorem 2.3 (Rational linear programming [17 , 18 ] ). Feasibility of a rational linear program is decidable in deterministic polynomial time in its binary input length. If the program is feasible and its objective has a finite optimum, an optimal rational solution and the optimum value can be computed exactly in polynomial time.
The integer-coefficient formulation applies to rational coefficients by clearing denominators in each row and in the objective. Unrestricted variables can be written as differences of nonnegative variables. These transformations increase the binary encoding length by at most a polynomial factor.
Computational Complexity
A decision problem, also called a language, is a set of finite binary strings, its yes instances. Its complement contains precisely the binary strings outside that set. A polynomial-time many-one reduction maps inputs of one decision problem to inputs of another, preserving the yes or no answer. The class P consists of problems decided by deterministic algorithms in time polynomial in the input length. The class PSPACE consists of problems decided by deterministic algorithms using polynomially many working-memory bits, with no prescribed time bound. For a binary string a a a , write ∣ a ∣ |a| ∣ a ∣ for its length. A decision problem A A A is in NP when there are a nonnegative integer-valued polynomial p p p and a polynomial-time predicate V V V such that, for every binary input a a a ,
a ∈ A ⟺ ∃ z ∈ { 0 , 1 } p ( ∣ a ∣ ) V ( a , z ) = 1. a \in A \quad\Longleftrightarrow\quad\exists z \in\{0,1\}^{p(|a|)} V(a,z)=1. a ∈ A ⟺ ∃ z ∈ { 0 , 1 } p ( ∣ a ∣ ) V ( a , z ) = 1.
The string z z z is a certificate, padded to the stated length if necessary. For a class C \mathcal{C} C of decision problems, a decision problem A A A is C \mathcal{C} C -hard if every problem in C \mathcal{C} C has a polynomial-time many-one reduction to A A A . The problem A A A is C \mathcal{C} C -complete if it is both C \mathcal{C} C -hard and in C \mathcal{C} C .
An integer c c c divides an integer b b b when b = c t b=ct b = c t for some integer t t t . For integers a , b a,b a , b and a positive integer q q q , the congruence a ≡ b ( m o d q ) a \equiv b \pmod{q} a ≡ b ( mod q ) means that q q q divides a − b a-b a − b . The residue of an integer a a a modulo q > 0 q>0 q > 0 is its remainder in { 0 , … , q − 1 } \{0,\ldots,q-1\} { 0 , … , q − 1 } upon division by q q q . Two integers are coprime when their only common positive divisor is one. A fraction with positive denominator is reduced when its numerator and denominator are coprime; that denominator is called its reduced denominator.
The exact linear-algebra computations used below also have polynomial bit complexity. To test whether a rational vector is in the span of supplied rational columns, solve for their linear-combination coefficients by rational linear-program feasibility, writing each equality as two inequalities. Process the columns in order and retain precisely those that are not in the span of the columns already retained; the retained columns form an independent spanning family. This computes rank and a column basis with polynomially many calls to Theorem 2.3 ; the same procedure applies to rows viewed as vectors. A matrix with no rows or no columns has rank zero.
For a square matrix, rank below its size gives determinant zero. Otherwise clear denominators row by row and apply elimination to the resulting integer matrix. A minor is the determinant of a square submatrix obtained by selecting equally many rows and columns. After each pivot stage, every remaining entry is a signed ratio of two minors of that integer matrix. Indeed, take an invertible pivot block P P P , a column u u u , a row v v v , a scalar a a a , and the vector z z z with P z = u Pz=u P z = u . Clearing the lower-left row by the first rows gives
det ( P u v a ) = det ( P ) ( a − v z ) . \det\begin{pmatrix} P & u \\ v & a \end{pmatrix}=\det(P)(a-vz). det ( P v u a ) = det ( P ) ( a − v z ) .
Thus the residual entry is the bordered minor divided by the pivot minor, with a possible sign from restoring the original row order.
If the integer entries have absolute value at most 2 B 2^B 2 B , every minor of size h h h has absolute value at most h ! 2 h B h!2^{hB} h ! 2 h B , by its determinant expansion. Its binary length, and hence the lengths of the residual numerators and denominators after cancellation, are polynomial in the original matrix encoding length. Elimination uses polynomially many rational operations; its final diagonal product, the row-swap sign, and the known row-denominator factors recover the original determinant. For an empty square matrix return one.
Common factors can be canceled by the Euclidean algorithm: on nonnegative integers not both zero, replace a pair ( u , v ) (u,v) ( u , v ) , with v > 0 v>0 v > 0 , by ( v , u − ⌊ u / v ⌋ v ) (v,u-\lfloor u/v\rfloor v) ( v , u − ⌊ u / v ⌋ v ) until the second entry is zero. The common divisors are unchanged at every step, so the last nonzero entry is the greatest common divisor, meaning the largest common positive divisor. The positive remainders fall by at least a factor of two within two divisions: if a remainder exceeds half of the remainder before it, the next remainder is their difference and is smaller than that half. The number of binary divisions is therefore polynomial in the bit length. This makes the preceding exact rational elimination a polynomial-time bit algorithm.
We use the following imported starting point, with positive integers and binary encoding.
Theorem 3.1 (Subset sum; Karp [Karp72]). Given positive integers c 1 , … , c r c_1,\ldots,c_r c 1 , … , c r and a positive target B B B , deciding whether some subset sums to B B B is NP-complete.
The following elementary conversion makes the number of selected entries explicit. It is convenient that W W W is a multiset: equal padding entries present no difficulty.
Lemma 3.2 (Prescribed cardinality). Given nonnegative integers a 1 , … , a m a_1,\ldots,a_m a 1 , … , a m , an integer k k k with 1 ≤ k ≤ m 1\leq k\leq m 1 ≤ k ≤ m , and a target B B B with 0 ≤ B ≤ ∑ j a j 0\leq B\leq\sum_j a_j 0 ≤ B ≤ ∑ j a j , deciding whether exactly k k k entries sum to B B B is NP-complete.
Proof. Membership in NP follows by listing the selected indices and checking their count and sum. For hardness start from Theorem 3.1 . Instances with target larger than the sum are immediate no instances and can be mapped to a fixed no instance of the present problem, for example a = ( 2 , 2 ) a=(2,2) a = ( 2 , 2 ) , k = 1 k=1 k = 1 , B = 1 B=1 B = 1 . For all other instances append r r r zero entries to c 1 , … , c r c_1,\ldots,c_r c 1 , … , c r , set m = 2 r m=2r m = 2 r , and require k = r k=r k = r selected entries. A subset of the original entries of size t ≤ r t\le r t ≤ r and sum B B B can be supplemented by r − t r-t r − t of the new zeros. Conversely, removing all selected zeros leaves an original subset with the same sum. The construction is polynomial and satisfies the stated ranges. □
We now give a reduction in which deeper networks cannot circumvent the intended numerical choice.
Theorem 3.3 (Finite samples). Squared-error feasibility for permitted networks is NP-complete. The NP-hardness holds with zero threshold, two samples, cap n max = 1 n_{\max}=1 n m a x = 1 , positive integer inventory, and input coordinates in { 0 , 1 } \{0,1\} { 0 , 1 } . It holds both for at-most-once use and for exactly-once use. Moreover, the same reduction remains valid for any larger node budget.
Proof. First consider membership. A certificate describes at most m m m computational vertices, at most m m m parameter positions, the output, and the assigned occurrence indices. It has polynomial length. The graph and inventory rules are checkable in polynomial time.
Evaluation uses exact rational arithmetic of polynomial bit length. To justify this point, fix a sample. Inactive ReLUs have value zero; replace active ReLUs by the identity. Each vertex value then expands as a sum over directed paths beginning at an input or a bias. Each term is a product of an input coordinate, if any, and distinct parameter occurrences. No path repeats an edge or bias because the graph is acyclic.
The product of all denominators in the inventory and this sample’s coordinates is a common denominator for all terms. Its bit length is at most the input length. A term’s numerator after scaling to this common denominator also has polynomial bit length, since it uses a subset of the supplied numerators and denominators. There are at most exponentially many directed paths in a graph with at most m m m edges, so the logarithm of the number of summands is polynomial. All vertex values thus have polynomial bit length. Evaluation in topological order determines the active ReLUs without enumerating paths, and exact comparison of the squared loss with τ \tau τ is polynomial.
For hardness take an instance from Lemma 3.2, and put
A = ∑ j = 1 m a j , M = A + B + m + 3 , w j = M + a j . A=\sum_{j=1}^{m}a_j,\qquad M=A+B+m+3,\qquad w_j=M+a_j. A = j = 1 ∑ m a j , M = A + B + m + 3 , w j = M + a j .
Supply the multiset W = ( w 1 , … , w m ) W=(w_1,\ldots,w_m) W = ( w 1 , … , w m ) . Define the two requested labels by
T = k M + B , U = m M + A . T=kM+B,\qquad U=mM+A. T = k M + B , U = m M + A .
Use dimension d = 2 m d=2m d = 2 m . The first input u u u has its first m m m coordinates equal to one and its last m m m equal to zero. The second input v v v has all 2 m 2m 2 m coordinates equal to one. The samples are ( u , T ) (u,T) ( u , T ) and ( v , U ) (v,U) ( v , U ) . All these integers have polynomial bit length.
The inequalities needed to control hidden paths are
A < M , B < M , 0 ≤ T ≤ U < M 2 . A<M,\qquad B<M,\qquad0\le T\le U<M^2. A < M , B < M , 0 ≤ T ≤ U < M 2 .
The first two follow from the definition of M M M . Since k ≤ m k\le m k ≤ m and B ≤ A B\le A B ≤ A , we have T ≤ U T\le U T ≤ U . Finally, M 2 − U = M ( M − m ) − A > 0 M^2-U=M(M-m)-A>0 M 2 − U = M ( M − m ) − A > 0 , because M − m = A + B + 3 M-m=A+B+3 M − m = A + B + 3 .
Suppose a permitted network fits both samples. Every assigned parameter is positive. On v v v , every computational vertex has a positive value: an induction in topological order applies because it has either a positive bias or an incoming edge from a positive predecessor. Thus hidden ReLUs are identities on v v v . A directed input-to-output path containing at least two edges contributes at least M 2 M^2 M 2 to the output. A contribution from a hidden bias followed by an edge likewise contributes at least M 2 M^2 M 2 . All contributions are nonnegative. Either contribution would make the output at least M 2 M^2 M 2 , contradicting the required output U < M 2 U<M^2 U < M 2 .
Every hidden vertex has a path to the output and ultimately has an input predecessor or a present bias. Such a vertex would create one of these forbidden contributions. Consequently there are no hidden vertices in any fitting network, even when a larger node budget is permitted. On v v v , each assigned parameter now contributes exactly its value, either through an input edge or through the output bias. For the at-most-once problem, omitting any occurrence gives a sum strictly smaller than U = ∑ j w j U = \sum_j w_j U = ∑ j w j . Hence all occurrences must be used. In the exactly-once problem this is already required.
On u u u , the output is the sum of a subset I I I of the occurrences: those assigned to edges from coordinates equal to one, together with the output bias if present. Writing s = ∣ I ∣ s = |I| s = ∣ I ∣ and C = ∑ j ∈ I a j C = \sum_{j \in I} a_j C = ∑ j ∈ I a j , exact fitting gives
s M + C = k M + B . sM + C = kM + B. s M + C = k M + B .
Both C C C and B B B lie in { 0 , … , M − 1 } \{0,\ldots,M-1\} { 0 , … , M − 1 } . Reduction modulo M M M gives C = B C = B C = B , and then s = k s = k s = k . A fitting network therefore supplies a yes answer to the cardinality instance.
Conversely, given a subset I I I of size k k k and sum B B B , use one affine output neuron without a bias. Assign the k k k selected occurrences to distinct edges from the first m m m input vertices, and the other m − k m-k m − k occurrences to distinct edges from the last m m m input vertices. There are enough vertices in both groups. This uses every occurrence once and gives outputs T T T and U U U . It respects n max = 1 n_{\max} = 1 n m a x = 1 , and an output ReLU would leave both positive labels unchanged. Every assigned parameter contributes positively at v v v .
This proves both reductions and NP-completeness. The proof also shows that a mandatory output bias would not change hardness: put one of the k ≥ 1 k \ge1 k ≥ 1 selected occurrences in that bias and assign the others to edges. □
Optimization asks for a permitted network of minimum loss, or a report that no permitted network exists. When some permitted network exists, a minimum is attained because the class of functions computed by permitted networks is finite by Proposition 2.2. The construction yields a quantitative gap as well as a feasibility result.
Corollary 3.4 (Optimization). Finding a global minimum of total squared empirical loss is NP-hard. On the integer instances in Theorem 3.3 , computing a permitted network with loss at most the optimum plus any fixed 0 ≤ η < 1 0 \le\eta< 1 0 ≤ η < 1 is NP-hard.
Proof. The outputs of every permitted network on integer samples are integers, by induction over additions, multiplications by integer parameters, and ReLUs. Therefore its total squared loss is a nonnegative integer. A yes instance has optimum zero, whereas a no instance has optimum at least one. A feasible output whose loss is less than one must have loss zero, so either optimization guarantee decides the NP-complete problem. For mean squared loss on the two samples the corresponding additive gap is 1 / 2 1/2 1/2 . □
A Boolean Target Supplied by a Circuit
In this section the target is supplied by a Boolean circuit rather than by an explicit sample list. A Boolean bit is an element of { 0 , 1 } \{0,1\} { 0 , 1 } , with one meaning true and zero meaning false. AND outputs one precisely when both inputs are one; OR does so when at least one is one, and NOT exchanges zero and one. A Boolean formula combines variables with these operations; a literal is a variable or its negation. A clause is an OR of literals, and a formula in conjunctive normal form (CNF) is an AND of clauses. The width of a clause is its number of literals. An assignment satisfies a formula when the formula takes the value one at that assignment. A tautological clause contains both a variable and its negation and is true on every assignment.
Let C C C be an explicitly encoded Boolean circuit with d d d input bits and one output bit. Its gates are AND, OR, and NOT, with the meanings just given. Such a circuit is a directed acyclic graph computing C : { 0 , 1 } d → { 0 , 1 } C : \{0,1\}^{d} \to\{0,1\} C : { 0 , 1 } d → { 0 , 1 } ; evaluation on one input takes polynomial time. The synthesis input is ( C , W , n max ) (C,W,n_{\max}) ( C , W , n m a x ) , and the requirement is
N ( x ) = C ( x ) for every x ∈ { 0 , 1 } d . N(x) = C(x) \qquad\text{for every } x \in\{0,1\}^{d}. N ( x ) = C ( x ) for every x ∈ { 0 , 1 } d .
Equality means that the numerical output is exactly zero or one. There is no free rounding or threshold gate. A thresholded-output formulation is a different convention. The network convention remains Definition 2.1. The preliminary bounds below also hold with a ReLU output; the completeness theorem uses an affine output.
A problem belongs to coNP if its complement belongs to NP. The class DP consists of languages expressible as the intersection of a language in NP and a language in coNP. The class Σ 2 P \Sigma_{2}^{\mathrm{P}} Σ 2 P consists of problems expressible by an existentially quantified polynomial-length string followed by a universally quantified polynomial-length string, with a polynomial-time predicate. We recall the satisfiability theorem before using it as a hardness source.
Theorem 4.1 (Satisfiability; Cook [7 ] ). Deciding whether a Boolean formula in conjunctive normal form has a satisfying assignment is NP-complete under polynomial-time many-one reductions.
The NP-completeness of Boolean circuit satisfiability follows from Theorem 4.1 : formulas are circuits, and a satisfying circuit input is a polynomially checkable witness. Circuit unsatisfiability is therefore coNP-complete.
Theorem 4.2 (Circuit-specified targets). Exact synthesis for a Boolean target supplied by a circuit is coNP-hard under polynomial-time many-one reductions and belongs to Σ 2 P \Sigma_{2}^{\mathrm{P}} Σ 2 P . The lower bound holds with W = ( 0 ) W=(0) W = ( 0 ) and n max = 1 n_{\max}=1 n m a x = 1 . Both bounds hold for at-most-once use and exactly-once use.
Proof. By the parameter-count argument in Theorem 3.3 , a candidate network has a polynomial-length description. Its validity and its equality with C C C on one Boolean input are polynomial-time checkable. Let z z z range over binary strings of the polynomial length bounding a candidate description, padded as necessary. The answer is yes precisely when
∃ z ∀ x ∈ { 0 , 1 } d ( z describes a permitted network N z and N z ( x ) = C ( x ) ) . \exists z \forall x \in\{0,1\}^{d}\ \left(z\text{ describes a permitted network }N_z\text{ and }N_z(x)=C(x)\right). ∃ z ∀ x ∈ { 0 , 1 } d ( z describes a permitted network N z and N z ( x ) = C ( x ) ) .
This proves the upper bound. For a fixed candidate, a disagreeing Boolean input certifies failure in polynomial time, so candidate equivalence belongs to coNP.
For hardness map a circuit C C C to ( C , ( 0 ) , 1 ) (C,(0),1) ( C , ( 0 ) , 1 ) . Every permitted network for this inventory has only zero parameters and computes the constant zero function. Such a network exists: use the single zero as the output bias. It uses the occurrence exactly once. Thus synthesis succeeds if and only if C C C is unsatisfiable. This proves both lower bounds. □
Enumerating networks and all Boolean inputs gives a deterministic running time of at most
N H 2 2 d poly ( L ) , N_{\mathcal{H}}2^{2^{d}}\operatorname{poly}(L), N H 2 2 d poly ( L ) ,
where L L L includes the circuit, inventory, and budget encodings. Checking a fixed candidate is already coNP-hard, by taking that candidate to be the fixed zero network and allowing C C C to vary.
Exact reproduction is stronger than agreement with high probability under random inputs. A Boolean function can differ from zero at one cube vertex, with disagreement probability only 2 − d 2^{-d} 2 − d under the uniform distribution on { 0 , 1 } d \{0,1\}^{d} { 0 , 1 } d . The completeness theorem below concerns all vertices, without a promise that a disagreement has substantial probability.
Blocking Exact Subset Sums
The numerical problem used below asks for a small set of occurrences meeting every exact subset sum. This is a form of subset-sum interdiction, studied by Grüne and Wulf [20 ] . We need additional arithmetic restrictions on the numbers. The following proof establishes those restrictions directly and treats equal-valued occurrences as distinct choices.
Definition 4.3 (Exact-sum blocker). For integers n ≥ 1 n \ge1 n ≥ 1 , p ≥ 0 p \ge0 p ≥ 0 and positive integers β 1 , … , β n , T \beta_1,\ldots,\beta_n,T β 1 , … , β n , T , an exact-sum blocker of size at most p p p is a set H ⊆ { 1 , … , n } H \subseteq\{1,\ldots,n\} H ⊆ { 1 , … , n } such that ∣ H ∣ ≤ p |H| \le p ∣ H ∣ ≤ p and
∑ j = 1 n β j z j = T ⇒ ∑ j ∈ H z j ≥ 1 for every z ∈ { 0 , 1 } n . \sum_{j=1}^{n}\beta_j z_j=T\quad\Rightarrow\quad\sum_{j\in H}z_j\ge1\qquad\text{for every }z\in\{0,1\}^{n}. j = 1 ∑ n β j z j = T ⇒ j ∈ H ∑ z j ≥ 1 for every z ∈ { 0 , 1 } n .
The condition is vacuous when no subset sums to T T T . An index j j j with z j = 1 z_j=1 z j = 1 , and the occurrence it labels, is said to be selected by z z z .
A prime is an integer q ≥ 2 q\ge2 q ≥ 2 whose only positive divisors are one and q q q . We will cancel nonzero residues modulo a prime q q q . A map is injective when equal outputs require equal inputs. For an integer a a a not divisible by q q q , choose the least positive integer g g g of the form u a + v q ua+vq u a + v q , with u , v ∈ Z u,v\in\mathbb{Z} u , v ∈ Z ; such positive integers exist because q q q is one of them. Dividing a a a and q q q by g g g with remainders in { 0 , … , g − 1 } \{0,\ldots,g-1\} { 0 , … , g − 1 } shows that each remainder is also such an integer combination. Minimality of g g g forces both remainders to be zero, so g g g divides both a a a and q q q . Primality and q ∤ a q \nmid a q ∤ a then force g = 1 g = 1 g = 1 , giving integers u , v u,v u , v with u a + v q = 1 ua+vq = 1 u a + v q = 1 . If q q q divides a b ab ab , multiplying this identity by b b b shows that q q q divides b b b . Thus multiplication by a nonzero residue is injective on residues modulo q q q , and a product of nonzero residues remains nonzero.
Trial division tests whether an integer q ≥ 2 q \ge2 q ≥ 2 is prime by checking divisors from two through ⌊ q ⌋ \lfloor\sqrt{q}\rfloor ⌊ q ⌋ . Indeed, any factorization q = c 1 c 2 q = c_1c_2 q = c 1 c 2 with c 1 , c 2 ≥ 2 c_1,c_2 \ge2 c 1 , c 2 ≥ 2 has at least one factor at most q \sqrt{q} q . For integers bounded by a polynomial in the input length, this uses polynomially many binary divisions of polynomial bit length. The numerical calibration uses the following classical prime bound.
Theorem 4.4 (Bertrand’s bound; Erdős [13 ] ). For every integer h ≥ 2 h \ge2 h ≥ 2 , there is a prime q q q with h < q < 2 h h < q < 2h h < q < 2 h .
The restrictions in the next lemma separate the numerical values that will play different roles in a synthesis instance. They leave the universal exact-sum condition computationally difficult.
Lemma 4.5 (Restricted exact-sum blockers). Deciding whether an exact-sum blocker exists is Σ 2 P \Sigma^P_2 Σ 2 P -complete under deterministic polynomial-time many-one reductions, even under the following restrictions. An odd prime q ≥ 5 q \ge5 q ≥ 5 is supplied, p = q − 1 p = q - 1 p = q − 1 , every β j \beta_j β j satisfies 0 < β j < T 0 < \beta_j < T 0 < β j < T and is not a power of two, and
β j ≡ 3 ( m o d q ) , T ≡ 0 ( m o d q ) . \beta_j \equiv3 \pmod{q}, \qquad T \equiv0 \pmod{q}. β j ≡ 3 ( mod q ) , T ≡ 0 ( mod q ) . The reduction produces p p p bounded by a polynomial in its input length. Every produced instance has n > p n > p n > p , where n n n is the number of its source numbers β j \beta_j β j .
Proof. Membership follows by choosing H H H existentially and quantifying z z z universally in Definition 4.3. The size constraint and the implication are checkable by polynomial-time binary arithmetic. We give the hardness construction in three steps: an encoding of counterexamples, ordinary subset sums with designated occurrences, and protected choices with the required congruences.
Counterexamples and calibration assignments.
Let ℓ , s ≥ 0 \ell,s \ge0 ℓ , s ≥ 0 be integers and let ψ \psi ψ be a CNF formula on bit vectors u ∈ { 0 , 1 } ℓ u \in\{0,1\}^{\ell} u ∈ { 0 , 1 } ℓ , v ∈ { 0 , 1 } s v \in\{0,1\}^{s} v ∈ { 0 , 1 } s , with clause width at most three. We start from the quantified formula
∃ u ∈ { 0 , 1 } ℓ ∀ v ∈ { 0 , 1 } s ¬ ψ ( u , v ) . \exists u \in\{0,1\}^{\ell}\ \forall v \in\{0,1\}^{s}\ \neg\psi(u,v). ∃ u ∈ { 0 , 1 } ℓ ∀ v ∈ { 0 , 1 } s ¬ ψ ( u , v ) .
This source is complete for Σ 2 P \Sigma^P_2 Σ 2 P by the following encoding of polynomial-time computation as a circuit and then as a formula. A polynomial-time deterministic computation has polynomially many configuration bits and steps. The configuration records its state, head positions, and symbols in the polynomially many tape cells it can visit. One transition is computed by a polynomial-size Boolean circuit: test the head positions, update the state and the scanned cells, and copy every other cell. Unrolling polynomially many copies of this transition circuit yields a polynomial-size circuit for the predicate, constructible in polynomial time.
To see the required polarity directly, start with any polynomial-time predicate defining a Σ 2 P \Sigma^P_2 Σ 2 P problem, and apply the circuit construction just described to its failure predicate, the negation of that predicate. Introduce one bit for every circuit gate, impose its truth-table constraints as clauses of size at most three, and assert a failure output of one. These new bits belong to the universally quantified counterexample block. We call the predicate’s own input bits, existential and universal alike, the original input bits, as opposed to the new gate bits. For fixed original input bits, a satisfying extension of the gate bits exists precisely when the failure circuit outputs one. Thus forbidding every satisfying counterexample gives the displayed quantified formula. This is the usual circuit computation encoding underlying satisfiability [7 ] .
Choose an odd prime q ≥ max { 5 , ℓ + 1 } q \ge\max\{5,\ell+1\} q ≥ max { 5 , ℓ + 1 } of size polynomial in ℓ + 1 \ell+1 ℓ + 1 , and put p = q − 1 p = q - 1 p = q − 1 . Such a prime can be found by deterministic search in a polynomial-sized interval: Theorem 4.4 supplies one between an integer h ≥ 2 h \ge2 h ≥ 2 and 2 h 2h 2 h , and trial division is polynomial for these polynomially bounded integers. Add unused existential variables until there are exactly p p p of them. Their unused values do not change the quantified formula.
For each i ∈ { 1 , … , p } i \in\{1,\ldots,p\} i ∈ { 1 , … , p } , introduce two marker bits m i 0 , m i 1 m_i^0,m_i^1 m i 0 , m i 1 . Construct a Boolean circuit for a relation R R R with the following accepting assignments. A main assignment has exactly one true marker in each pair and satisfies ψ ( u ′ , v ) \psi(u',v) ψ ( u ′ , v ) , where u i ′ = m i 1 u'_i = m_i^1 u i ′ = m i 1 . A calibration assignment for index i i i has both m i 0 , m i 1 m_i^0,m_i^1 m i 0 , m i 1 true and every other marker false; its v v v -bits are unrestricted. The relation R R R is the disjunction of these cases. It has a polynomial-size circuit. Encode that circuit by gate constraints and an asserted true output, obtaining a formula Φ \Phi Φ with clauses of size at most three. The projections of satisfying assignments of Φ \Phi Φ onto the markers and v v v are exactly the assignments accepted by R R R . Gate bits are auxiliary variables, distinct from the markers.
Subset sums and designated choices.
Use one base-ten digit for every variable of Φ \Phi Φ and one for every clause. The target has digit one in each variable position and digit four in each clause position. For every variable create a true item and a false item. Each item has digit one in its own variable position, and digit one in every clause satisfied by that literal. For each clause add two slack items, with digits one and two in that clause position and zero elsewhere. All these item values are positive integers. Only the true items of the 2 p 2p 2 p marker variables are designated : each will receive a single supplied take occurrence below, so that a blocker can attack it.
Remove repeated literals from clauses and discard tautological clauses first. Every clause then has at most three distinct literal occurrences. Even an arbitrary subset of all items contributes at most three literal units and three slack units to a clause digit, and at most two units to a variable digit. Consequently no base-ten carry occurs in any subset sum. The target variable digits force exactly one of the two items for each variable. If t ∈ { 0 , 1 , 2 , 3 } t \in\{0,1,2,3\} t ∈ { 0 , 1 , 2 , 3 } literals satisfy a clause, its target digit four can be obtained using slack of total at most three if and only if t ≥ 1 t \ge1 t ≥ 1 . Thus exact subsets correspond precisely to satisfying assignments of Φ \Phi Φ , with appropriate slack choices. Write their positive item values as a 1 ′ , … , a N 0 ′ a'_1,\ldots,a'_{N_0} a 1 ′ , … , a N 0 ′ , and their target as A ′ A' A ′ .
Multiply every item and the target by q q q , giving a j = q a j ′ a_j = qa'_j a j = q a j ′ and A = q A ′ A = qA' A = q A ′ . Append fewer than q q q additional items of value q ( A ′ + 1 ) > A q(A' + 1) > A q ( A ′ + 1 ) > A until the number N N N of items is a multiple of q q q . The appended items cannot occur in an exact A A A -sum. They are not designated. There are at least 2 p 2p 2 p marker variables, so N ≥ 2 N \ge2 N ≥ 2 . The scale and padding preserve every original solution and its designated choices.
Protected classes.
Choose positive multiples b , L b,L b , L of q − 1 q-1 q − 1 so that
B = 2 b > 2 p + 2 , K = 3 ⋅ 2 L > 4 ( A + ( p + 1 ) ∑ j = 1 N a j + 1 ) . B = 2^b > 2p+2,\qquad K = 3 \cdot2^L > 4\left(A+(p+1)\sum_{j=1}^{N} a_j+1\right). B = 2 b > 2 p + 2 , K = 3 ⋅ 2 L > 4 ( A + ( p + 1 ) j = 1 ∑ N a j + 1 ) .
For every j ∈ { 0 , … , N − 1 } j \in\{0,\ldots,N-1\} j ∈ { 0 , … , N − 1 } , create a take value and a skip value
τ j = K B j + a j + 1 , σ j = K B j . \tau_j = KB^j+a_{j+1},\qquad\sigma_j = KB^j. τ j = K B j + a j + 1 , σ j = K B j .
If a j + 1 a_{j+1} a j + 1 is designated, supply one occurrence of τ j \tau_j τ j . Otherwise supply p + 1 p+1 p + 1 occurrences of τ j \tau_j τ j . Supply p + 1 p+1 p + 1 occurrences of every σ j \sigma_j σ j . The occurrences of τ j \tau_j τ j and σ j \sigma_j σ j form class j j j , and its two class options are take, an occurrence of τ j \tau_j τ j , and skip, an occurrence of σ j \sigma_j σ j . An option supplied with p + 1 p+1 p + 1 copies is protected , and so are its occurrences; since every skip option is protected, we call these classes protected classes . A take class of a subset is a class from which it selects a take occurrence. These take and skip occurrences are the numbers β 1 , … , β n \beta_1,\ldots,\beta_n β 1 , … , β n produced by the reduction. Their target is
T = A + K ∑ j = 0 N − 1 B j . T = A + K\sum_{j=0}^{N-1} B^j. T = A + K j = 0 ∑ N − 1 B j .
The total possible contribution of all low parts a j a_j a j , even when all copies are selected, is at most ( p + 1 ) ∑ j a j < K / 4 (p+1)\sum_j a_j < K/4 ( p + 1 ) ∑ j a j < K /4 . Also A < K / 4 A < K/4 A < K /4 . Reducing an exact T T T -sum modulo K K K therefore forces its selected low parts to sum exactly to A A A . After subtracting those parts and dividing by K K K , the remaining equality has target ∑ j B j \sum_j B^j ∑ j B j . At class j j j , at most 2 p + 2 < B 2p+2 < B 2 p + 2 < B supplied occurrences can contribute B j B^j B j . There is no carry in base B B B , so exactly one occurrence is selected from each class. The selected take classes give an exact A A A -sum of original items. Conversely, every such original sum lifts to a T T T -sum by taking its chosen classes and skipping the others.
An occurrence that lies in a candidate blocker H H H is said to be attacked by H H H , and a set of attacked occurrences is an attack. A class option with p + 1 p+1 p + 1 copies cannot be removed completely by attacking at most p p p occurrences. For every fixed original solution, any attack of that size on its protected options can therefore be bypassed by choosing surviving copies independently in each class. Only designated take occurrences can be removed without such a replacement.
Consider a blocker H H H of size at most p p p . For each i i i , a calibration assignment exists whose only designated true items are m i 0 , m i 1 m_i^0,m_i^1 m i 0 , m i 1 . If H H H contained neither of these two designated take occurrences, lift that calibration solution while avoiding every attacked protected occurrence. This would produce a T T T -sum disjoint from H H H , contradicting its blocker property. Hence H H H contains at least one designated take occurrence from every marker pair. There are p p p pairs and the budget is p p p , so it contains exactly one from each pair and no other occurrence.
Let θ i ∈ { 0 , 1 } \theta_i \in\{0,1\} θ i ∈ { 0 , 1 } be the label it attacks in pair i i i . A main assignment has exactly one true marker in each pair, and it avoids H H H exactly when, for every i i i , that single true marker has label 1 − θ i 1-\theta_i 1 − θ i . Its encoded existential assignment is therefore u i ′ = 1 − θ i u_i' = 1-\theta_i u i ′ = 1 − θ i . No main assignment avoids H H H if and only if ψ ( u ′ , v ) \psi(u',v) ψ ( u ′ , v ) is false for every v v v . Conversely, any existential witness u ′ u' u ′ to that universal condition gives a blocker by attacking marker 1 − u i ′ 1-u_i' 1 − u i ′ in every pair. Each calibration is hit, and no main solution avoids these attacks. This proves both directions of the reduction.
Arithmetic restrictions and encoding length.
For a nonzero residue α \alpha α modulo the prime q q q , multiplication by α \alpha α permutes the q − 1 q-1 q − 1 nonzero residues. Multiplying those residues and canceling their nonzero product gives α q − 1 ≡ 1 ( m o d q ) \alpha^{q-1} \equiv1 \pmod q α q − 1 ≡ 1 ( mod q ) . Apply this with α = 2 \alpha=2 α = 2 . Since b , L b,L b , L are multiples of q − 1 q-1 q − 1 , it gives B ≡ 1 ( m o d q ) B \equiv1 \pmod q B ≡ 1 ( mod q ) and K ≡ 3 ( m o d q ) K \equiv3 \pmod q K ≡ 3 ( mod q ) . Every low part a j a_j a j and A A A is divisible by q q q , and N ≡ 0 ( m o d q ) N \equiv0 \pmod q N ≡ 0 ( mod q ) . Thus every take or skip value is congruent to three modulo q q q , whereas T ≡ 0 ( m o d q ) T \equiv0 \pmod q T ≡ 0 ( mod q ) .
Put E = L + b j E=L+bj E = L + bj . The skip value is 3 ⋅ 2 E 3\cdot2^E 3 ⋅ 2 E , strictly between 2 E + 1 2^{E+1} 2 E + 1 and 2 E + 2 2^{E+2} 2 E + 2 . The take value adds a j + 1 < K / 4 ≤ 3 ⋅ 2 E / 4 a_{j+1}<K/4\leq3\cdot2^E/4 a j + 1 < K /4 ≤ 3 ⋅ 2 E /4 , so it remains strictly below 4 ⋅ 2 E = 2 E + 2 4\cdot2^E=2^{E+2} 4 ⋅ 2 E = 2 E + 2 . Both values lie strictly between consecutive powers of two. For a take value,
T − τ j = A − a j + 1 + K ∑ h ≠ j B h > − K / 4 + K > 0 , T-\tau_j=A-a_{j+1}+K\sum_{h\ne j}B^h>-K/4+K>0, T − τ j = A − a j + 1 + K h = j ∑ B h > − K /4 + K > 0 ,
since N ≥ 2 N\geq2 N ≥ 2 . A skip value is smaller than its corresponding take value. All output values are consequently positive and strictly below T T T . Every class supplies p + 1 p+1 p + 1 skip occurrences and at least one take occurrence, and there are N ≥ 2 N\geq2 N ≥ 2 classes, so the number of output values is n ≥ 2 ( p + 2 ) > p n\geq2(p+2)>p n ≥ 2 ( p + 2 ) > p .
Finally, q , p , N q,p,N q , p , N are polynomially bounded in the original formula length. The exponents b , L b,L b , L can be chosen polynomially bounded by rounding the relevant binary-length bounds up to multiples of q − 1 q-1 q − 1 . Every exponent L + b j L+bj L + bj and every output binary length is polynomial. There are polynomially many supplied copies. The construction is therefore an explicit deterministic polynomial-time reduction, including when the original existential block is empty.
A Calibrated Threshold Construction
Fix an instance in the hardness family produced by Lemma 4.5, with source numbers β 1 , … , β n \beta_1,\ldots,\beta_n β 1 , … , β n , target T T T , prime q q q , and budget p = q − 1 p=q-1 p = q − 1 . In that family the number n n n of source numbers satisfies n > p n>p n > p , by Lemma 4.5. Thus q = p + 1 ≤ n q=p+1\leq n q = p + 1 ≤ n is polynomially bounded in the source encoding. We construct a Boolean threshold target and a rational inventory. The construction has two purposes. A small perturbation of the source coefficients will encode a blocker, while additional input coordinates will constrain every alternative use of the inventory.
Choose an odd prime r > q r>q r > q large enough that
Q = 2 r − 1 > 8 ( T + ∑ j = 1 n β j + 2 ) 2 . Q=2^r-1>8\left(T+\sum_{j=1}^{n}\beta_j+2\right)^2. Q = 2 r − 1 > 8 ( T + j = 1 ∑ n β j + 2 ) 2 .
The multiplicative order of two modulo q q q is the least positive integer t t t with 2 t ≡ 1 ( m o d q ) 2^t\equiv1\pmod q 2 t ≡ 1 ( mod q ) . It exists with t ≤ q − 1 t\leq q-1 t ≤ q − 1 . Indeed, q ∤ 2 q\nmid2 q ∤ 2 because q q q is an odd prime, so by the cancellation of nonzero residues, q ∣ 2 k q\mid2^k q ∣ 2 k with k ≥ 1 k\geq1 k ≥ 1 would give q ∣ 2 k − 1 q\mid2^{k-1} q ∣ 2 k − 1 , and repeating would give q ∣ 1 q\mid1 q ∣ 1 ; thus no power of two is divisible by q q q . Hence the q q q numbers 2 0 , … , 2 q − 1 2^0,\ldots,2^{q-1} 2 0 , … , 2 q − 1 have residues among the q − 1 q-1 q − 1 nonzero residues, and two of them, 2 k 2^k 2 k and 2 k ′ 2^{k'} 2 k ′ with 0 ≤ k < k ′ ≤ q − 1 0\leq k<k'\leq q-1 0 ≤ k < k ′ ≤ q − 1 , have the same residue. Canceling the nonzero residue of 2 k 2^k 2 k gives 2 k ′ − k ≡ 1 ( m o d q ) 2^{k'-k}\equiv1\pmod q 2 k ′ − k ≡ 1 ( mod q ) with 1 ≤ k ′ − k ≤ q − 1 1\leq k'-k\leq q-1 1 ≤ k ′ − k ≤ q − 1 . If q q q divided Q Q Q , then 2 r ≡ 1 ( m o d q ) 2^r\equiv1\pmod q 2 r ≡ 1 ( mod q ) . Write r = h t + k 0 r=ht+k_0 r = h t + k 0 with integers h ≥ 0 h\geq0 h ≥ 0 and 0 ≤ k 0 < t 0\leq k_0<t 0 ≤ k 0 < t . Then 1 = 2 r = ( 2 t ) h 2 k 0 ≡ 2 k 0 ( m o d q ) 1=2^r=(2^t)^h2^{k_0}\equiv2^{k_0}\pmod q 1 = 2 r = ( 2 t ) h 2 k 0 ≡ 2 k 0 ( mod q ) , so the minimality of t t t forces k 0 = 0 k_0=0 k 0 = 0 , and t t t divides r r r . Since r r r is prime, t ∈ { 1 , r } t\in\{1,r\} t ∈ { 1 , r } , and t ≤ q − 1 < r t\leq q-1<r t ≤ q − 1 < r excludes t = r t=r t = r ; hence t = 1 t=1 t = 1 . Then 2 ≡ 1 ( m o d q ) 2\equiv1\pmod q 2 ≡ 1 ( mod q ) , that is, q q q divides one, which is impossible for a prime, so Q ≢ 0 ( m o d q ) Q\not\equiv0\pmod q Q ≡ 0 ( mod q ) . Since T ≡ 0 ( m o d q ) T\equiv0\pmod q T ≡ 0 ( mod q ) by Lemma 4.5, it follows that Q − T ≡ Q ≢ 0 ( m o d q ) Q-T\equiv Q\not\equiv0\pmod q Q − T ≡ Q ≡ 0 ( mod q ) . These two nonzero residues are in Lemma A.7 and Lemma A.15 of Section A, where Q − T Q-T Q − T and Q Q Q , the multiples of F F F in the anchor baseline B B B and the control baseline C C C defined below, must be nonzero modulo q q q ; up to sign, Q Q Q is also the multiple of F F F in each of the two finite negative parameters.
Let
n a = p + 3 , K = 2 n + 3 p + 14 , J = K + 1 , R = 3 J r , d = n + n a + R + 1. n_a = p+3,\qquad K = 2n+3p+14,\qquad J = K+1,\qquad R = 3Jr,\qquad d = n+n_a+R+1. n a = p + 3 , K = 2 n + 3 p + 14 , J = K + 1 , R = 3 J r , d = n + n a + R + 1.
The input coordinates consist of the n n n source bits z j z_j z j , n a n_a n a anchor bits a h a_h a h , J J J query blocks, and one control bit c c c . Each query block has three query coordinates, called copies, at every level i ∈ { 0 , … , r − 1 } i \in\{0,\ldots,r-1\} i ∈ { 0 , … , r − 1 } . The level of a query coordinate is a label fixed by the construction, and it determines the baseline that the coordinate receives, defined below. There are R R R query coordinates in total. Each query coordinate keeps its specified level even if a candidate assigns parameter occurrences of other values to its incident edges.
Choose a power of two F F F satisfying
F > max { 8 Q 4 , 4 d + 64 q 2 + 8 } , P = F Q . F > \max\{8Q^4,4d+64q^2+8\},\qquad P = FQ. F > max { 8 Q 4 , 4 d + 64 q 2 + 8 } , P = F Q .
Put m = K + 2 R m = K+2R m = K + 2 R and
H 0 = 16 q 4 ( 64 ( m + 1 ) 4 P 4 + 1 ) . H_0 = 16q^4\left(64(m+1)^4P^4+1\right). H 0 = 16 q 4 ( 64 ( m + 1 ) 4 P 4 + 1 ) .
Choose a power of two M > max { 2 H 0 , q ( 2 R + 3 ) } M > \max\{2H_0,q(2R+3)\} M > max { 2 H 0 , q ( 2 R + 3 )} , and set
w = M − 1 , Γ = M 9 , s 0 = 1 q Γ . w = M^{-1},\qquad\Gamma= M^9,\qquad s_0 = \frac{1}{q\Gamma}. w = M − 1 , Γ = M 9 , s 0 = q Γ 1 .
The four baseline families are
F β j , B = F ( Q − T ) − w , F 2 i + 2 w , C = P − 3 2 w . F\beta_j,\qquad B = F(Q-T)-w,\qquad F2^i+2w,\qquad C = P-\frac{3}{2}w. F β j , B = F ( Q − T ) − w , F 2 i + 2 w , C = P − 2 3 w .
Assign these to source, anchor, level-i i i query, and control coordinates, respectively. Write λ t \lambda_t λ t for the baseline of input coordinate t t t , and define
Λ ( x ) = ∑ t = 1 d λ t x t , f ( x ) = { 1 , Λ ( x ) ≥ P − w , 0 , Λ ( x ) < P − w . \Lambda(x)=\sum_{t=1}^{d}\lambda_t x_t,\qquad
f(x)=
\begin{cases}
1, & \Lambda(x)\ge P-w,\\
0, & \Lambda(x)<P-w.
\end{cases} Λ ( x ) = t = 1 ∑ d λ t x t , f ( x ) = { 1 , 0 , Λ ( x ) ≥ P − w , Λ ( x ) < P − w .
We call Λ ( x ) \Lambda(x) Λ ( x ) the baseline score of x x x . Every baseline is positive. The inventory contains two occurrences of every coordinate’s baseline, p p p occurrences of s 0 s_0 s 0 , two of Γ \Gamma Γ , and the four signed parameters
+ M , − M , − ( P − w ) , − P . +M,\qquad-M,\qquad-(P-w),\qquad-P. + M , − M , − ( P − w ) , − P .
Its size is m m m , because the number of occurrences outside the query-baseline family is
2 n + 2 n a + 2 + p + 2 + 2 + 2 = K . 2n+2n_a+2+p+2+2+2=K. 2 n + 2 n a + 2 + p + 2 + 2 + 2 = K .
The computational-node cap is four, including the affine output. We call + M , − M +M,-M + M , − M the gain parameters and the remaining parameters nongain parameters . A primitive means a single inventory value, before any path multiplication. The nongain values of the form F ν + δ w F\nu+\delta w F ν + δ w , with nonzero integer ν \nu ν and rational δ \delta δ , are called finite primitives ; they are exactly the baseline families and the two negative finite values. Here δ \delta δ is 0 for a source baseline and for − P -P − P , − 1 -1 − 1 for B B B , 2 for a query baseline, − 3 / 2 -3/2 − 3/2 for C C C , and 1 for − ( P − w ) -(P-w) − ( P − w ) . The other nongain types are Γ \Gamma Γ and s 0 s_0 s 0 . We call occurrences of s 0 s_0 s 0 selectors .
All these numbers have polynomial binary length. The blocker instances produced by Lemma 4.5 have polynomially bounded p p p . Theorem 4.4 permits a polynomially bounded numerical choice of r r r , larger than q q q and a constant times the binary length of T + ∑ j β j + 2 T+\sum_j\beta_j+2 T + ∑ j β j + 2 . The quantities K , J , R , d , m K,J,R,d,m K , J , R , d , m are polynomially bounded integers, and the logarithms of the parameters defining F , M F,M F , M are polynomial. Clearing the common denominator 2 M 2M 2 M turns the condition defining f f f into an integer threshold comparison. Binary addition and comparison therefore give a polynomial-size Boolean circuit for f f f . The reduction supplies this circuit, rather than a table of its values.
Formal Identities and Query Restrictions
A Laurent polynomial in formal variables w , Γ w,\Gamma w , Γ is a finite sum of terms c i j w i Γ j c_{ij}w^i\Gamma^j c ij w i Γ j , where c i j ∈ Q c_{ij} \in\mathbb{Q} c ij ∈ Q and i , j ∈ Z i,j \in\mathbb{Z} i , j ∈ Z . A power product w i Γ j w^i\Gamma^j w i Γ j is a monomial; after collecting equal monomials, c i j c_{ij} c ij is its coefficient. Formal equality means equality of all these coefficients, before evaluating the variables at numerical values. The formal expansion of an inventory value is the following Laurent polynomial, in which F F F , P P P , Q Q Q and q q q are numerical coefficients. A finite primitive F ν + δ w F\nu+\delta w F ν + δ w expands as itself, with coefficients F ν F\nu F ν and δ \delta δ ; Γ \Gamma Γ expands as the monomial Γ \Gamma Γ ; s 0 s_0 s 0 expands as q − 1 Γ − 1 q^{-1}\Gamma^{-1} q − 1 Γ − 1 ; and the gains ± M \pm M ± M expand as ± w − 1 \pm w^{-1} ± w − 1 . The specialization w = M − 1 w=M^{-1} w = M − 1 , Γ = M 9 \Gamma=M^9 Γ = M 9 returns each inventory value from its formal expansion. The choices of F F F and M M M allow two different separations. The factor F F F distinguishes products from individual finite parameters. The much larger specialization Γ = M 9 \Gamma=M^9 Γ = M 9 , w = M − 1 w=M^{-1} w = M − 1 makes every exact Boolean output equation a formal polynomial identity. The argument for the second separation freezes the actual activation pattern at the tested input before treating the expression formally; it does not assert that the same pattern persists under changes to the numerical parameters.
Lemma 4.6 (Uniform specialization). Fix any candidate with at most four computational vertices and any Boolean input. Replace each hidden ReLU by either its affine preactivation or zero according to its actual activation at that input. Expand the resulting output as a Laurent polynomial in w , Γ w,\Gamma w , Γ , replacing every parameter occurrence by the formal expansion of its value. If the numerical output at that input is b ∈ { 0 , 1 } b \in\{0,1\} b ∈ { 0 , 1 } , the expanded Laurent polynomial is identically b b b . This assertion holds uniformly for every legal assignment of inventory occurrences and every Boolean input.
Proof. Under a fixed activation pattern, the output is the sum of the products along the active input-to-output and bias-to-output paths. A directed path visits distinct computational vertices, and there are at most four of them. An input-to-output path therefore has at most four edges, and a bias-to-output path consists of one bias and at most three edges, so every path contains at most four parameter occurrences. The occurrences along a path determine it, since its first occurrence fixes its starting input or bias and each edge fixes its head; hence there are at most ∑ k = 1 4 m k ≤ ( m + 1 ) 4 \sum_{k=1}^{4}m^k \le(m+1)^4 ∑ k = 1 4 m k ≤ ( m + 1 ) 4 paths.
Each nongain finite parameter F ν + δ w F\nu+\delta w F ν + δ w has at most two formal terms; the other parameters are monomials. Every formal term of a single parameter has the form c w i Γ j cw^i\Gamma^j c w i Γ j with i , j ∈ { − 1 , 0 , 1 } i,j \in\{-1,0,1\} i , j ∈ { − 1 , 0 , 1 } . The finite primitives give F ν F\nu F ν and δ w \delta w δ w , Γ \Gamma Γ gives Γ \Gamma Γ , s 0 s_0 s 0 gives q − 1 Γ − 1 q^{-1}\Gamma^{-1} q − 1 Γ − 1 , and the gains give ± w − 1 \pm w^{-1} ± w − 1 . Expanding the product along one path, with its input coordinate in { 0 , 1 } \{0,1\} { 0 , 1 } as a further factor, gives at most 2 4 = 16 2^4=16 2 4 = 16 terms, and the exponent of either formal variable in each term lies between minus four and four. The single-parameter coefficients F ν F\nu F ν , δ \delta δ , 1 1 1 , q − 1 q^{-1} q − 1 and ± 1 \pm1 ± 1 have denominators dividing 2 q 2q 2 q , because F ν F\nu F ν is an integer and δ ∈ { 0 , − 1 , 2 , − 3 / 2 , 1 } \delta\in\{0,-1,2,-3/2,1\} δ ∈ { 0 , − 1 , 2 , − 3/2 , 1 } . A product of at most four of them therefore has a denominator dividing ( 2 q ) 4 = 16 q 4 (2q)^4=16q^4 ( 2 q ) 4 = 16 q 4 , which is a common denominator of all coefficients. Each single-parameter coefficient has absolute value at most P P P : every finite primitive has ∣ ν ∣ ≤ Q |\nu| \le Q ∣ ν ∣ ≤ Q , so ∣ F ν ∣ ≤ F Q = P |F\nu| \le FQ=P ∣ F ν ∣ ≤ F Q = P , while ∣ δ ∣ ≤ 2 |\delta| \le2 ∣ δ ∣ ≤ 2 , q − 1 q^{-1} q − 1 and 1 1 1 are at most P P P . Hence each term coefficient has absolute value at most P 4 P^4 P 4 .
After collecting equal monomials, each coefficient of the output expansion is a sum of at most 16 ( m + 1 ) 4 ≤ 64 ( m + 1 ) 4 16(m+1)^4 \le64(m+1)^4 16 ( m + 1 ) 4 ≤ 64 ( m + 1 ) 4 term coefficients, so it has absolute value at most 64 ( m + 1 ) 4 P 4 64(m+1)^4P^4 64 ( m + 1 ) 4 P 4 . Subtracting b ∈ { 0 , 1 } b \in\{0,1\} b ∈ { 0 , 1 } changes only the constant coefficient, by at most one. Multiplying by 16 q 4 16q^4 16 q 4 clears all denominators, so the resulting integer coefficients have absolute value at most 16 q 4 ( 64 ( m + 1 ) 4 P 4 + 1 ) = H 0 16q^4\left(64(m+1)^4P^4+1\right)=H_0 16 q 4 ( 64 ( m + 1 ) 4 P 4 + 1 ) = H 0 .
Under the numerical substitution, a monomial w u Γ v w^u\Gamma^v w u Γ v becomes M − u + 9 v M^{-u+9v} M − u + 9 v . The map ( u , v ) ↦ − u + 9 v (u,v)\mapsto-u+9v ( u , v ) ↦ − u + 9 v is injective on { − 4 , … , 4 } 2 \{-4,\ldots,4\}^2 { − 4 , … , 4 } 2 : equality would give u − u ′ = 9 ( v − v ′ ) u-u'=9(v-v') u − u ′ = 9 ( v − v ′ ) , whose left side has absolute value at most eight. The resulting exponents lie between minus forty and forty. Multiplying by M 40 M^{40} M 40 gives an integer polynomial of degree at most eighty. For any nonzero such polynomial, the highest nonzero term, of degree k k k , has absolute value at least M k M^k M k , while the sum of the absolute values of the lower terms is less than
H 0 M k M − 1 < M k . H_0\frac{M^k}{M-1}<M^k. H 0 M − 1 M k < M k .
Because M > 2 H 0 M>2H_0 M > 2 H 0 , the displayed inequality holds and the polynomial cannot vanish. Consequently the numerical output equation forces every formal coefficient to vanish. The same calculation applies to a preactivation. Its actual sign, when nonzero, is the sign of the highest specialized term. Nothing here asserts that the activation pattern remains fixed when w w w and Γ \Gamma Γ vary independently. □
The next restriction resolves a possible mismatch between input labels and parameter values. An input called a query coordinate need not receive a query-valued parameter in a candidate. Replication ensures that, for every candidate, one complete block receives only query-valued parameters on its input edges.
A coordinate face of a Boolean cube is obtained by fixing some coordinates to bits and leaving the others free. Such a face is also called a subcube. For a Boolean-valued function, a vertex wins if its value is one and loses if its value is zero. A coordinate is selected at a vertex of the cube when its value there is one. A coordinate is essential if changing only that coordinate changes the value at some fixed setting of the other coordinates. Such a setting is called a background, and the coordinate is pivotal at it. A background has (binary) weight W W W , and is then a query background of weight W W W , when the coordinates it sets to one are query coordinates whose values 2 i 2^i 2 i , for their levels i i i , sum to W W W , and it sets every other coordinate to zero. The function is coordinatewise monotone if changing any coordinate from zero to one never decreases its value. For two free coordinates of a function h h h , the mixed difference is h ( 1 , 1 ) − h ( 1 , 0 ) − h ( 0 , 1 ) + h ( 0 , 0 ) h(1,1)-h(1,0)-h(0,1)+h(0,0) h ( 1 , 1 ) − h ( 1 , 0 ) − h ( 0 , 1 ) + h ( 0 , 0 ) , with every other coordinate fixed.
Lemma 4.7 (A block with only query-valued input edges). For every candidate, some query block has no input edge from its coordinates that carries an occurrence outside the query-baseline family. Fix every coordinate outside that block to zero. Then each nonzero input edge from a coordinate of the block carries an actual value F 2 i + 2 w F2^i+2w F 2 i + 2 w for some level i i i ; this is the level of the occurrence, which need not be the level of the coordinate. Write y i , γ y_{i,\gamma} y i , γ for the copy-γ \gamma γ coordinate of level i i i in the block, where i i i is now the level of the coordinate. The target restriction is
g ( y ) = 1 if and only if ∑ i = 0 r − 1 ∑ γ = 1 3 2 i y i , γ ≥ Q . g(y)=1 \quad\text{if and only if}\quad\sum_{i=0}^{r-1}\sum_{\gamma=1}^{3}2^i y_{i,\gamma}\ge Q. g ( y ) = 1 if and only if i = 0 ∑ r − 1 γ = 1 ∑ 3 2 i y i , γ ≥ Q . The origin and all unit vectors lose; the all-ones vector and every one- or two-coordinate deletion win. Every query coordinate is essential. The function g g g cannot be expressed as an affine function plus a scalar multiple of one ReLU of an affine function, and neither can any restriction of g g g obtained by fixing up to two coordinates to zero or any one coordinate to one.
Proof. An occurrence outside the query-baseline family can contaminate at most one block when used on an external-input edge , an edge leaving an input vertex. There are K K K such occurrences and J = K + 1 J=K+1 J = K + 1 blocks. The asserted block follows by counting. All input edges from the other coordinates contribute zero on its restriction. The target value there depends on the baseline score, which there equals F I + 2 w κ F_I+2w\kappa F I + 2 w κ , where I I I is the displayed binary weighted sum and K ≤ 3 r K\leq3r K ≤ 3 r is the number of selected query coordinates. Since 2 R w < 1 2R w<1 2 R w < 1 , inputs with I ≥ Q I\geq Q I ≥ Q win and inputs with I ≤ Q − 1 I\leq Q-1 I ≤ Q − 1 lose.
A single query weight is at most ( Q + 1 ) / 2 < Q (Q+1)/2<Q ( Q + 1 ) /2 < Q . The all-ones sum is 3 Q 3Q 3 Q , and deleting two coordinates leaves at least 3 Q − ( Q + 1 ) = 2 Q − 1 > Q 3Q-(Q+1)=2Q-1>Q 3 Q − ( Q + 1 ) = 2 Q − 1 > Q . A level-i i i coordinate is pivotal against a background of one copy of every other level, whose sum is Q − 2 i Q-2^i Q − 2 i . Thus every coordinate is essential.
For two tested level-zero coordinates, backgrounds of weight Q − 2 Q-2 Q − 2 and Q − 1 Q-1 Q − 1 give respectively AND and OR on that same pair. Their pairwise mixed differences are one and minus one. The other coordinates are fixed to the stated background in each difference.
Let ϕ ( y ) = ℓ ( y ) + a ρ ( t + v ⋅ y ) \phi(y)=\ell(y)+a\rho(t+v\cdot y) ϕ ( y ) = ℓ ( y ) + a ρ ( t + v ⋅ y ) , where ℓ \ell ℓ is affine. We show that every mixed difference of ϕ \phi ϕ in coordinates i i i and j j j has the fixed sign of a v i v j av_i v_j a v i v j , or is zero. The affine contribution to this mixed difference is zero. For an interval length u > 0 u>0 u > 0 , the ReLU definition gives
ρ ( t + u ) − ρ ( t ) = { 0 , t ≤ − u , t + u , − u < t < 0 , u , t ≥ 0. \rho(t+u)-\rho(t)=
\begin{cases}
0, & t\leq-u,\\
t+u, & -u<t<0,\\
u, & t\geq0.
\end{cases} ρ ( t + u ) − ρ ( t ) = ⎩ ⎨ ⎧ 0 , t + u , u , t ≤ − u , − u < t < 0 , t ≥ 0.
This increment is nondecreasing in the starting point t t t . Let u ′ > 0 u'>0 u ′ > 0 be a second length. The increment from the starting point t + u ′ t+u' t + u ′ minus the increment from t t t is the mixed difference of ρ \rho ρ for the lengths u u u and u ′ u' u ′ . By the monotonicity just noted, this mixed difference is nonnegative. For arbitrary real lengths write Δ ( t ; u , u ′ ) = ρ ( t + u + u ′ ) − ρ ( t + u ) − ρ ( t + u ′ ) + ρ ( t ) \Delta(t;u,u')=\rho(t+u+u')-\rho(t+u)-\rho(t+u')+\rho(t) Δ ( t ; u , u ′ ) = ρ ( t + u + u ′ ) − ρ ( t + u ) − ρ ( t + u ′ ) + ρ ( t ) for this mixed difference, which is symmetric in u u u and u ′ u' u ′ . If u < 0 < u ′ u<0<u' u < 0 < u ′ , put t ′ = t + u t'=t+u t ′ = t + u and u ˉ = − u > 0 \bar{u}=-u>0 u ˉ = − u > 0 ; since t ′ + u ˉ = t t'+\bar{u}=t t ′ + u ˉ = t , t ′ + u ′ = t + u + u ′ t'+u'=t+u+u' t ′ + u ′ = t + u + u ′ and t ′ + u ˉ + u ′ = t + u ′ t'+\bar{u}+u'=t+u' t ′ + u ˉ + u ′ = t + u ′ , substitution gives
Δ ( t ; u , u ′ ) = − Δ ( t ′ ; u ˉ , u ′ ) ≤ 0. \Delta(t;u,u')=-\Delta(t';\bar{u},u')\leq0. Δ ( t ; u , u ′ ) = − Δ ( t ′ ; u ˉ , u ′ ) ≤ 0.
The case u > 0 > u ′ u > 0 > u' u > 0 > u ′ follows by exchanging the two lengths. If u , u ′ < 0 u,u' < 0 u , u ′ < 0 , put t ′ ′ = t + u + u ′ t'' = t + u + u' t ′′ = t + u + u ′ ; the points t ′ ′ , t ′ ′ − u , t ′ ′ − u ′ , t ′ ′ − u − u ′ t'',t'' - u,t'' - u',t'' - u - u' t ′′ , t ′′ − u , t ′′ − u ′ , t ′′ − u − u ′ are t + u + u ′ , t + u ′ , t + u , t t + u + u',t + u',t + u,t t + u + u ′ , t + u ′ , t + u , t , so Δ ( t ; u , u ′ ) = Δ ( t ′ ′ ; − u , − u ′ ) ≥ 0 \Delta(t;u,u') = \Delta(t'';-u,-u') \ge0 Δ ( t ; u , u ′ ) = Δ ( t ′′ ; − u , − u ′ ) ≥ 0 . A zero length gives zero. At a background y y y with y i = y j = 0 y_i = y_j = 0 y i = y j = 0 , the mixed difference of ϕ \phi ϕ in coordinates i i i and j j j equals a Δ ( t + v ⋅ y ; v i , v j ) a\Delta(t + v \cdot y;v_i,v_j) a Δ ( t + v ⋅ y ; v i , v j ) . Taking the two lengths to be v i v_i v i and v j v_j v j and multiplying by a a a therefore gives the stated sign of a v i v j av_iv_j a v i v j , or zero. Consequently, one ReLU plus an affine function cannot produce the two opposite nonzero differences.
After at most two zero fixings, choose a level h < r − 1 h < r - 1 h < r − 1 with all three copies still free. Such a level exists because r ≥ 7 r \ge7 r ≥ 7 , since r r r is an odd prime larger than q ≥ 5 q \ge5 q ≥ 5 . Reserve two copies for testing. The binary expansions of Q − 2 h + 1 Q - 2^{h+1} Q − 2 h + 1 and Q − 2 h Q - 2^h Q − 2 h give AND and OR backgrounds, using at most one surviving copy of every other level and a third copy of level h h h for the AND background. Every level still has at least one surviving copy. If instead a coordinate of level k k k is fixed to one, choose h < r − 1 h < r - 1 h < r − 1 with k ∉ { h , h + 1 } k \notin\{h,h+1\} k ∈ / { h , h + 1 } . Both backgrounds then include the prescribed fixed coordinate at level k k k ; the two tested coordinates and the third level-h h h copy are free. The same mixed-difference argument applies on each of these faces. □
Section A uses the last assertion of Lemma 4.7 to exclude placements that leave the restricted output an affine function plus a multiple of one ReLU.
The full target also has the geometric properties needed after the graph has been constrained. Those properties are independent of the candidate’s assignment of parameters.
Lemma 4.8 (Full-target tests). The function f f f is monotone, every coordinate is essential, and f ( 0 ) = f ( e j ) = 0 f(0) = f(e_j) = 0 f ( 0 ) = f ( e j ) = 0 for every coordinate j j j . Call a winning vertex nonminimal if deleting one of its selected coordinates gives a winning vertex. The nonminimal winning vertices affinely span R d \mathbb{R}^d R d . The function has a pairwise mixed difference of minus one on a Boolean face.
Proof. All baselines are positive, so f f f is monotone. A source coordinate z j z_j z j is pivotal at a query background of weight Q − β j Q - \beta_j Q − β j , an anchor at one of weight T T T , and a level-i i i query at one of weight Q − 2 i Q - 2^i Q − 2 i . These backgrounds lie between zero and Q Q Q and are realized by binary expansions using one available copy per level. They can avoid the tested query coordinate. The losing side is separated by at least one integer unit in the F F F -scaled sum; its total small correction is less than one. On the winning side the source and query tests reach at least P P P , and the anchor test reaches P − w P - w P − w plus a nonnegative correction. The control coordinate is pivotal at the background in which one level-zero query coordinate is one and every other coordinate is zero. Its baseline score without the control is F + 2 w < P − w F + 2w < P - w F + 2 w < P − w , while adding C C C gives more than P P P . All singleton baseline scores, including C = P − 3 w / 2 C = P - 3w/2 C = P − 3 w /2 , are below P − w P - w P − w .
On the slice c = 1 c = 1 c = 1 , every vertex with another selected coordinate wins. Every vertex of that slice with at least two selected coordinates other than c c c is a nonminimal winner. The all-ones vertex of that slice and its single deletions affinely span the hyperplane c = 1 c = 1 c = 1 . There is also a nonminimal winner with c = 0 c = 0 c = 0 and three anchors selected: two anchors already have baseline score greater than P P P , because Q > 2 T Q > 2T Q > 2 T . This gives a point outside that hyperplane and proves the full affine span. The OR background in Lemma 4.7 gives the claimed negative mixed difference. □
Write D = M = w − 1 D = M = w^{-1} D = M = w − 1 for the gain magnitude. A positive wire is a hidden ReLU whose incoming coefficients and present bias are positive; on Boolean inputs it equals its nonnegative preactivation. The following placement statement is the part of the reduction that addresses arbitrary graph choices. Its proof, including every internal, output, bias and unused-parameter case, is given in Section A. The argument first restricts the two large gains, then forces the finite negative parameters into hidden biases and identifies the source vertices of the two gain edges. It next proves that the two biased hinges , the hidden ReLUs that receive the finite negative parameters as biases, are not coupled : there is no directed path from one to the other, neither a direct edge nor a path through the remaining hidden vertex. A coupling edge is an edge on such a path. Only after these actual edges are excluded does the full-cube argument identify their common score , the preactivation of each biased hinge without its finite negative bias.
Lemma 4.9 (Placement for the calibrated construction). Let W , f W,f W , f be the calibrated inventory and target above, with computational-node cap four and affine output. If any permitted network represents f f f on the
Boolean cube, then some permitted network using a submultiset of the same occurrences has the form
D ( ρ ( S ( x ) − ( P − w ) ) − ρ ( S ( x ) − P ) ) , S ( x ) = a u ( x ) + ∑ j = 1 d b j x j . D\left(\rho\left(S(x)-(P-w)\right)-\rho\left(S(x)-P\right)\right), \qquad S(x)=au(x)+\sum_{j=1}^{d}b_jx_j. D ( ρ ( S ( x ) − ( P − w ) ) − ρ ( S ( x ) − P ) ) , S ( x ) = a u ( x ) + j = 1 ∑ d b j x j .
The coefficients a , b 1 , … , b d , v b , v 1 , … , v d a,b_1,\ldots,b_d,v_b,v_1,\ldots,v_d a , b 1 , … , b d , v b , v 1 , … , v d are nonnegative reals; a zero denotes an absent parameter, and each nonzero coefficient is a single positive actual inventory primitive. The optional wire is u ( x ) = v b + ∑ j = 1 d v j x j u(x)=v_b+\sum_{j=1}^{d}v_jx_j u ( x ) = v b + ∑ j = 1 d v j x j , also called the sharing wire. It enters each hinge through one sharing edge of weight a a a , and each input x j x_j x j enters each hinge through one direct input edge of weight b j b_j b j . Every nonzero a a a uses two identical-value occurrences for its two sharing edges, and every nonzero b j b_j b j uses two identical-value occurrences for its two direct input edges. The two displayed biases and gains use their specified supplied occurrences. The hinges carry no other bias, so the score bias, the constant term of S S S , is a v b av_b a v b . The output has no ordinary output contribution, meaning no output bias, no skip edge and no edge from the wire to the output. A wire without incoming parameters, or with no sharing edge, is omitted.
We call the displayed representation a common ramp : its two hinges share one affine score and their thresholds differ by w w w .
Recovering a Blocker from a Common Ramp
The placement argument ensures a common score without assigning every remaining primitive to its intended numerical role. We now recover an actual source blocker from any legal common-ramp representation. A binary threshold identity will first determine every query coefficient.
Lemma 4.10 (Three-copy binary reference). Let Q = 2 r − 1 Q=2^r-1 Q = 2 r − 1 , with r ≥ 2 r\ge2 r ≥ 2 . Assign positive integer weights to three copies of every level i ∈ { 0 , … , r − 1 } i\in\{0,\ldots,r-1\} i ∈ { 0 , … , r − 1 } . If the assigned weights with threshold Q Q Q give exactly the same Boolean function as the weights 2 i 2^i 2 i with threshold Q Q Q , then every assigned weight at level i i i equals 2 i 2^i 2 i . The conclusion also holds with any larger number of copies per level.
Proof. Let l i , u i l_i,u_i l i , u i be the minimum and maximum assigned weights at level i i i , and put L = ∑ i l i L=\sum_i l_i L = ∑ i l i . Call a selection of copies an original winner if its sum under the weights 2 i 2^i 2 i is at least Q Q Q , and an original loser otherwise. Selecting one minimum at every level is an original winner, so L ≥ Q L\ge Q L ≥ Q . Selecting one maximum at every positive level and no level-zero coordinate is an original loser, giving ∑ i ≥ 1 u i ≤ Q − 1 \sum_{i\ge1}u_i\le Q-1 ∑ i ≥ 1 u i ≤ Q − 1 .
For i ≥ 1 i\ge1 i ≥ 1 , start instead with the losing pattern of minima at every positive level. Remove its level-i i i coordinate and add two unused level-( i − 1 ) (i-1) ( i − 1 ) coordinates. Its original binary sum remains Q − 1 Q-1 Q − 1 , so it is an original loser and its assigned sum is at most Q − 1 Q-1 Q − 1 . Its assigned sum is also at least L − l 0 − l i + 2 l i − 1 L-l_0-l_i+2l_{i-1} L − l 0 − l i + 2 l i − 1 , because the pattern of minima has assigned sum L − l 0 L-l_0 L − l 0 , the removed coordinate has weight l i l_i l i , and each of the two added coordinates has weight at least l i − 1 l_{i-1} l i − 1 . Combining the two bounds with L ≥ Q L\ge Q L ≥ Q gives
L − l 0 − l i + 2 l i − 1 ≤ Q − 1 ≤ L − 1 , L-l_0-l_i+2l_{i-1}\le Q-1\le L-1, L − l 0 − l i + 2 l i − 1 ≤ Q − 1 ≤ L − 1 ,
that is,
l i ≥ 2 l i − 1 + 1 − l 0 . l_i\ge2l_{i-1}+1-l_0. l i ≥ 2 l i − 1 + 1 − l 0 .
This uses two additional copies when the lower-level minimum was already selected, which explains the three-copy hypothesis. Induction on i i i gives l i ≥ l 0 + 2 i − 1 l_i\ge l_0+2^i-1 l i ≥ l 0 + 2 i − 1 : equality holds at i = 0 i=0 i = 0 , and for i ≥ 1 i\ge1 i ≥ 1 the inductive hypothesis gives l i ≥ 2 ( l 0 + 2 i − 1 − 1 ) + 1 − l 0 = l 0 + 2 i − 1 l_i\ge2(l_0+2^{i-1}-1)+1-l_0=l_0+2^i-1 l i ≥ 2 ( l 0 + 2 i − 1 − 1 ) + 1 − l 0 = l 0 + 2 i − 1 . Summing over the positive levels, using u i ≥ l i u_i\ge l_i u i ≥ l i , the maxima bound and ∑ i = 1 r − 1 2 i = 2 r − 2 = Q − 1 \sum_{i=1}^{r-1}2^i=2^r-2=Q-1 ∑ i = 1 r − 1 2 i = 2 r − 2 = Q − 1 , gives
Q − 1 ≥ ∑ i = 1 r − 1 u i ≥ ∑ i = 1 r − 1 l i ≥ ∑ i = 1 r − 1 ( l 0 − 1 + 2 i ) = ( r − 1 ) ( l 0 − 1 ) + Q − 1. Q-1\ge\sum_{i=1}^{r-1}u_i\ge\sum_{i=1}^{r-1}l_i\ge\sum_{i=1}^{r-1}(l_0-1+2^i)=(r-1)(l_0-1)+Q-1. Q − 1 ≥ i = 1 ∑ r − 1 u i ≥ i = 1 ∑ r − 1 l i ≥ i = 1 ∑ r − 1 ( l 0 − 1 + 2 i ) = ( r − 1 ) ( l 0 − 1 ) + Q − 1.
The weights are positive integers, so l 0 ≥ 1 l_0\ge1 l 0 ≥ 1 , and r ≥ 2 r\ge2 r ≥ 2 ; the display therefore forces l 0 = 1 l_0=1 l 0 = 1 . Every inequality in the display is then an equality, and since u i ≥ l i ≥ 2 i u_i\ge l_i\ge2^i u i ≥ l i ≥ 2 i for each i ≥ 1 i\ge1 i ≥ 1 , it follows that l i = u i = 2 i l_i=u_i=2^i l i = u i = 2 i for all i ≥ 1 i\ge1 i ≥ 1 .
It remains to identify the weight of any chosen level-zero copy. If that copy has the minimum weight l 0 l_0 l 0 , its weight is already one. Otherwise let its weight be v v v , remove the level-one coordinate from the losing positive-level pattern, and add that copy and a distinct level-zero copy of weight l 0 l_0 l 0 . The new binary sum is still Q − 1 Q-1 Q − 1 , while its assigned sum is Q − 1 − 2 + 1 + v Q-1-2+1+v Q − 1 − 2 + 1 + v . Hence v ≤ 1 v\le1 v ≤ 1 , and positivity gives v = 1 v=1 v = 1 . This proves the three-copy case. With more copies, restrict to any three per level, including a chosen copy whose weight is to be identified. ▫
The next statement permits finite sharing values, selector sharing, direct selectors, a positive wire bias, and rearrangements of all baseline occurrences. Here the sharing value is the weight a a a of the sharing edges, and sharing by a value means that a a a takes it, so selector sharing, or sharing by s 0 s_0 s 0 , means a = s 0 a=s_0 a = s 0 . Only the common-ramp form and the actual occurrence budget are assumed.
Lemma 4.11 (Common-ramp witness extraction). There exists a legal common-ramp network
N ( x ) = D ( ρ ( S ( x ) − ( P − w ) ) − ρ ( S ( x ) − P ) ) , S ( x ) = a u ( x ) + ∑ j b j x j , N(x)=D\left(\rho\left(S(x)-(P-w)\right)-\rho\left(S(x)-P\right)\right),\qquad S(x)=au(x)+\sum_j b_jx_j, N ( x ) = D ( ρ ( S ( x ) − ( P − w ) ) − ρ ( S ( x ) − P ) ) , S ( x ) = a u ( x ) + j ∑ b j x j , that equals f f f on all Boolean inputs if and only if the fixed source instance β 1 , … , β n , T \beta_1,\ldots,\beta_n,T β 1 , … , β n , T has an exact-sum blocker of size at most p p p . Here the network follows the positive primitive and paired-occurrence conventions of Lemma 4.9, u u u is the optional sharing wire, and a a a and the b j b_j b j are the hinges’ sharing and direct input weights, each zero or a single positive inventory primitive. The statement includes the case in which no source subset sums to T T T , and allows an omitted sharing wire.
Proof. We first prove extraction, allowing all legal primitive roles. The ramp is zero exactly for S ≤ P − w S\le P-w S ≤ P − w and one exactly for S ≥ P S\ge P S ≥ P . The target is zero at the origin and at every unit vector by Lemma 4.8, so at these points the score, and hence each of its nonnegative contributions, is at most P − w P-w P − w . A direct Γ \Gamma Γ primitive exceeds this bound at the unit vector of its coordinate. By the form of S S S and of the wire u u u , the only products in S S S are the terms a v b av_b a v b and a v j av_j a v j . A product containing two finite positive primitives is at least F 2 > P F^2>P F 2 > P , because every finite positive primitive is at least F F F and F > Q F>Q F > Q ; a finite-Γ \Gamma Γ product and Γ 2 \Gamma^2 Γ 2 are also too large, being at least Γ = M 9 > P \Gamma=M^9>P Γ = M 9 > P . Such products cannot occur at an origin or unit score. Call a nonfinite term of the score tiny when it is at most P s 0 Ps_0 P s 0 . The only remaining nonfinite products that are not tiny are Γ s 0 = 1 / q \Gamma s_0=1/q Γ s 0 = 1/ q . Every other remaining nonfinite contribution is tiny: this includes direct selectors, finite-selector products and s 0 2 s_0^2 s 0 2 .
Consequently the score has the form
S ( x ) = I b / q + τ 0 + ∑ j ( μ j + I j / q + τ j ) x j . S(x)=I_b/q+\tau_0+\sum_j\left(\mu_j+I_j/q+\tau_j\right)x_j. S ( x ) = I b / q + τ 0 + j ∑ ( μ j + I j / q + τ j ) x j .
where μ j \mu_j μ j is an actual finite direct primitive or zero, I b , I j ∈ { 0 , 1 } I_b,I_j\in\{0,1\} I b , I j ∈ { 0 , 1 } , and τ 0 , τ j ≥ 0 \tau_0,\tau_j\ge0 τ 0 , τ j ≥ 0 . In this proof, an increment is a product Γ s 0 \Gamma s_0 Γ s 0 . The total number of increments satisfies I b + ∑ j I j ≤ p I_b+\sum_j I_j\le p I b + ∑ j I j ≤ p . Recall that the products in S S S are the terms a v b av_b a v b and a v j av_j a v j . We therefore consider the sharing values a = Γ a=\Gamma a = Γ and a = s 0 a=s_0 a = s 0 in turn. If a = Γ a=\Gamma a = Γ , the two sharing edges consume both occurrences of Γ \Gamma Γ , and the wire uses at most p p p selector inputs or a selector bias. If a = s 0 a=s_0 a = s 0 , the two sharing edges consume two identical occurrences of s 0 s_0 s 0 , by the paired-occurrence convention of Lemma 4.9. The wire can then use at most two Γ \Gamma Γ inputs or biases, and 2 ≤ p 2\le p 2 ≤ p . No other primitive sharing produces an increment 1 / q 1/q 1/ q .
There are at most d d d tiny direct terms, d d d tiny shared coordinate terms and one tiny shared bias. Since
2 d + 1 = 2 n + 2 p + 2 R + 9 < m = 2 n + 3 p + 2 R + 14 , 2d+1=2n+2p+2R+9<m=2n+3p+2R+14, 2 d + 1 = 2 n + 2 p + 2 R + 9 < m = 2 n + 3 p + 2 R + 14 ,
we have
τ 0 + ∑ j τ j ≤ m P s 0 < w / 4. \tau_0+\sum_j\tau_j\le mPs_0<w/4. τ 0 + j ∑ τ j ≤ m P s 0 < w /4.
The last inequality is equivalent to M 8 > 4 m P / q M^8>4mP/q M 8 > 4 m P / q , which follows from M 8 ≥ M > 2 H 0 > 4 m P ≥ 4 m P / q M^8\ge M>2H_0>4mP\ge4mP/q M 8 ≥ M > 2 H 0 > 4 m P ≥ 4 m P / q . Tiny bias terms remain allowed throughout the argument. Write s j = μ j + I j / q + τ j s_j=\mu_j+I_j/q+\tau_j s j = μ j + I j / q + τ j for the effective score coefficient of coordinate j j j .
Excluding anchor and control baselines from queries. Fix any anchor and any level-i i i query coordinate. Choose a query background of binary weight T + 1 T+1 T + 1 , avoiding the compared copy; spare copies allow this. Adding the anchor wins, whereas adding the compared query loses, because T + 1 + 2 i < Q T+1+2^i<Q T + 1 + 2 i < Q . In the candidate, the score at the winning point is at least P P P and at the losing point at most P − w P-w P − w . Subtracting their common background and score bias gives
s a n c h o r − s q u e r y ≥ w . s_{\mathrm{anchor}}-s_{\mathrm{query}}\ge w. s anchor − s query ≥ w .
Each nonzero μ j \mu_j μ j is a finite primitive F v + δ w Fv+\delta w F v + δ w . We call μ j \mu_j μ j the finite part of coordinate j j j , the integer v v v its leading integer , F v Fv F v its leading part , and δ w \delta w δ w its finite correction . Suppose some query has finite part B B B or C C C . Every finite family below B B B , namely the source baselines F β j F\beta_j F β j and the query baselines F 2 i + 2 w , F2^i+2w, F 2 i + 2 w , has leading integer smaller than Q − T Q-T Q − T , the leading integer of B B B , by more than one. Indeed, β j ≤ T − 1 \beta_j \le T-1 β j ≤ T − 1 and 2 i ≤ ( Q + 1 ) / 2 2^i \le(Q+1)/2 2 i ≤ ( Q + 1 ) /2 , while Q − T ≥ T + 2 Q-T \ge T+2 Q − T ≥ T + 2 and Q − T ≥ ( Q + 1 ) / 2 + 2 Q-T \ge(Q+1)/2+2 Q − T ≥ ( Q + 1 ) /2 + 2 because Q > 8 ( T + 2 ) 2 Q > 8(T+2)^2 Q > 8 ( T + 2 ) 2 . The query’s effective score coefficient is at least its finite part, hence at least B = F ( Q − T ) − w B=F(Q-T)-w B = F ( Q − T ) − w , since C > B C>B C > B and its other contributions are nonnegative. An anchor whose finite part lies in such a family, or is zero, has effective score coefficient at most
F ( Q − T − 2 ) + 2 w + 1 q + w 4 < F ( Q − T ) − w , F(Q-T-2)+2w+\frac{1}{q}+\frac{w}{4}<F(Q-T)-w, F ( Q − T − 2 ) + 2 w + q 1 + 4 w < F ( Q − T ) − w ,
because its finite part is at most F ( Q − T − 2 ) + 2 w F(Q-T-2)+2w F ( Q − T − 2 ) + 2 w , its increment at most 1 / q 1/q 1/ q , its tiny terms less than w / 4 w/4 w /4 , and 2 F > 1 / q + 13 w / 4 2F>1/q+13w/4 2 F > 1/ q + 13 w /4 . Such an anchor would have effective score coefficient below that of the query, contrary to the displayed gap, so every anchor must itself have finite part B B B or C C C . An anchor with finite part B B B but no increment cannot satisfy the displayed gap against a query with finite part at least B B B : the anchor’s effective score coefficient exceeds the query’s by at most τ j \tau_j τ j , the sum of its tiny terms, which is smaller than w / 4 w/4 w /4 . Thus at most p p p anchors can have finite part B B B . At most two anchors can use C C C , since only two actual occurrences exist; the paired direct convention only strengthens that bound. This accounts for at most p + 2 < n a = p + 3 p+2<n_a=p+3 p + 2 < n a = p + 3 anchors, a contradiction. Therefore no query has finite part B B B or C C C .
Identifying all query primitives.
All remaining query finite corrections are nonnegative, being zero for source primitives and 2 w 2w 2 w for query primitives. On the query-only cube, the nonleading score is the score minus the sum of the leading parts of the selected coordinates. It consists of the finite corrections, the increments and the tiny terms, including those in the score bias, and it lies in
0 ≤ nonleading score ≤ p / q + 2 R w + w / 4 < 1. 0\le\text{nonleading score}\le p/q+2Rw+w/4<1. 0 ≤ nonleading score ≤ p / q + 2 R w + w /4 < 1.
The sum of the leading parts is F F F times an integer weighted sum. Thus a leading integer sum at least Q Q Q gives score at least P P P , while a sum at most Q − 1 Q-1 Q − 1 gives score below P − w P-w P − w . The assigned integer query coefficients therefore represent precisely the original threshold at quota Q Q Q . None is zero, by query essentiality. Lemma 4.10, applied to any three copies per level, identifies each integer coefficient as 2 i 2^i 2 i . A source primitive cannot have that value by the nonpower promise of Lemma 4.5, that no β j \beta_j β j is a power of two, so each full finite query part is exactly F 2 i + 2 w F2^i+2w F 2 i + 2 w .
Identifying sources, anchors and control.
For source coordinate j j j , use query backgrounds of weights Q − β j Q-\beta_j Q − β j and Q − β j − 1 Q-\beta_j-1 Q − β j − 1 . The first wins when that source is selected, and the second loses. All query-background perturbations, including score bias, are nonnegative and total less than one. Hence
F β j − 1 < s j < F ( β j + 1 ) . F\beta_j-1<s_j<F(\beta_j+1). F β j − 1 < s j < F ( β j + 1 ) .
A smaller finite leading integer cannot bridge the gap of at least F F F . A larger source or query leading integer has nonnegative correction and violates the upper bound. The negative-correction families B , C B,C B , C have leading integers larger than T T T , by a gap exceeding one, and cannot lie in this interval either. Thus μ j = F β j \mu_j=F\beta_j μ j = F β j in full; the nonpower promise excludes a query primitive at the same leading value.
For an anchor, backgrounds T , T − 1 T,T-1 T , T − 1 give the interval
F ( Q − T ) − 1 < s a n c h o r < F ( Q − T + 1 ) . F(Q-T)-1<s_{\mathrm{anchor}}<F(Q-T+1). F ( Q − T ) − 1 < s anchor < F ( Q − T + 1 ) .
Only B B B lies in it: every source or query leading value is much smaller, and C C C ’s is larger by at least T ≥ 3 T\ge3 T ≥ 3 . For the control coordinate, its singleton loss and its win with one level-zero query give
F ( Q − 1 ) − 1 < s c < P . F(Q-1)-1<s_c<P. F ( Q − 1 ) − 1 < s c < P .
Only C C C lies in this interval. These comparisons use actual finite baselines and do not assign physical selector roles in advance.
The control singleton now gives
( I b + I c ) / q + τ 0 + τ c ≤ w / 2. (I_b+I_c)/q+\tau_0+\tau_c\le w/2. ( I b + I c ) / q + τ 0 + τ c ≤ w /2.
Since 1 / q > w / 2 1/q>w/2 1/ q > w /2 , it forces I b = I c = 0 I_b=I_c=0 I b = I c = 0 . A tiny score bias remains within the already proved bound.
Choose an anchor with I h = 0 I_h=0 I h = 0 , possible because there are p + 3 p+3 p + 3 anchors and at most p p p increments in all. Let H = { j ≤ n : I j = 1 } H=\{j\le n:I_j=1\} H = { j ≤ n : I j = 1 } be the incremented source coordinates. Its size is at most p p p . At that anchor together with any source solution z z z satisfying ∑ j β j z j = T \sum_j \beta_j z_j = T ∑ j β j z j = T , the sum Λ \Lambda Λ of the selected baselines is P − w P-w P − w , so the target is one. If that solution avoided H H H , the complete candidate score would be less than
P − w + w / 4 < P , P-w+w/4<P, P − w + w /4 < P ,
contradicting its required ramp value one. Thus H H H hits every source solution and is an exact-sum blocker.
The forward network.
Conversely, fix an exact-sum blocker H H H of size at most p p p . The output uses the standard difference of two shifted ReLUs [35 ] . For each input coordinate, assign one of its two baseline occurrences to each hinge. Use the prescribed negative biases and opposite output gains. If H ≠ ∅ H\ne\varnothing H = ∅ , add
u = ρ ( s 0 ∑ j ∈ H z j ) u=\rho\left(s_0\sum_{j\in H}z_j\right) u = ρ s 0 j ∈ H ∑ z j
and use the two Γ \Gamma Γ sharing occurrences. On the Boolean cube the common score of this network is S = Λ + ∑ j ∈ H z j / q S=\Lambda+\sum_{j\in H}z_j/q S = Λ + ∑ j ∈ H z j / q . This uses at most four computational vertices and at most p p p selectors, with no reused occurrence or parallel edge. If H H H is empty, omit u u u and both sharing edges, retaining a legal three-vertex network.
To verify all inputs, we use letters local to this verification, unrelated to the symbols of the extraction above: let k k k count selected anchors, t t t count selected queries, v v v be their binary weighted sum, b = ∑ j β j z j b=\sum_j\beta_jz_j b = ∑ j β j z j , and h = ∑ j ∈ H z j h=\sum_{j\in H}z_j h = ∑ j ∈ H z j . Then
Λ = F I + w δ , I = b + k ( Q − T ) + v + Q c , δ = − k + 2 t − 3 c / 2 , S = F I + w δ + h / q . \Lambda=FI+w\delta,\quad I=b+k(Q-T)+v+Qc,\quad\delta=-k+2t-3c/2,\quad S=FI+w\delta+h/q. Λ = F I + w δ , I = b + k ( Q − T ) + v + Q c , δ = − k + 2 t − 3 c /2 , S = F I + w δ + h / q .
The perturbations of Λ \Lambda Λ and S S S , namely w δ w\delta w δ and w δ + h / q w\delta+h/q w δ + h / q , have absolute value less than one, using p / q + 2 R w < 1 p/q+2Rw<1 p / q + 2 R w < 1 and 2 R > n a + 3 / 2 2R>n_a+3/2 2 R > n a + 3/2 . If I < Q I<Q I < Q , both target and ramp are zero; if I > Q I>Q I > Q , both are one, since the F F F -gap exceeds the perturbation. At I = Q I=Q I = Q , a selected control forces every other coordinate zero, giving the losing control singleton. With c = 0 c=0 c = 0 , two or more anchors are impossible because Q > 2 T Q>2T Q > 2 T . With no anchor, some query must be selected since ∑ j β j < Q \sum_j\beta_j<Q ∑ j β j < Q , and δ = 2 t ≥ 2 \delta=2t\ge2 δ = 2 t ≥ 2 gives baseline score above P P P , hence also S > P S>P S > P . With one anchor and at least one query, δ = − 1 + 2 t ≥ 1 \delta=-1+2t\ge1 δ = − 1 + 2 t ≥ 1 again gives baseline score above P P P , hence also S > P S>P S > P . With one anchor and no query, b = T b=T b = T and Λ = P − w \Lambda=P-w Λ = P − w . The blocker condition gives h ≥ 1 h\ge1 h ≥ 1 , so S ≥ P − w + 1 / q > P S\ge P-w+1/q>P S ≥ P − w + 1/ q > P . These cases exhaust every Boolean input, including the case in which no source subset sums to T T T .
Completeness for Boolean Targets
The calibrated construction separates numerical selection from verification of every Boolean input. Its graph restrictions are necessary: a network may otherwise assign any occurrence to a bias, an input edge, an internal edge or an output edge, and may leave occurrences unused. The placement and extraction lemmas address all those choices.
Theorem 4.12 (restatement of Theorem 1.1). Under either the at-most-once or the exactly-once rule, with an affine output, exact synthesis for a Boolean target supplied by a circuit is Σ 2 P \Sigma_2^P Σ 2 P -complete under deterministic polynomial-time many-one reductions. Hardness holds with a computational-node budget of four, including the output, even when the circuit computes a monotone threshold with positive rational coefficients.
Proof. Membership is the existential candidate followed by universal Boolean-input verification in Theorem 4.2. For hardness, compose the quantified-CNF reduction in Lemma 4.5 with the calibrated network construction above. The blocker instances that reduction produces satisfy p < n p<n p < n by Lemma 4.5, as required for the polynomial numerical-size bounds. Form the calibrated inventory and its polynomial-size threshold circuit. Every explicit number has polynomial binary length, every coordinate and occurrence is listed in polynomial time, and the budget is the fixed value four. If the source has a blocker, the forward construction in Lemma 4.11 gives a legal permitted network for the target. Conversely, any permitted network for that target has a legal common-ramp representation by Lemma 4.9. Lemma 4.11 then returns an actual source-coordinate blocker of size at most p p p . The source and synthesis answers are therefore equivalent under this deterministic polynomial-time transformation. All target weights are positive, so the target function is monotone. This proves the restricted hardness and hence the full completeness claim.
For exactly-once use, enlarge a source blocker to cardinality exactly p p p . This is possible because p < n p<n p < n , and enlarging a blocker preserves the property of meeting every source solution. The source construction has p ≥ 2 p\ge2 p ≥ 2 , so the enlarged blocker is nonempty and the selector neuron keeps an incoming edge, as Definition 2.1 requires. Use all p p p selector occurrences on the enlarged blocker in the same forward network. Both baseline copies for every coordinate, both sharing occurrences, the two hinge biases and the two output gains are then used, so every inventory occurrence is assigned exactly once. The forward proof needs only the blocker property and the bound ∣ H ∣ / q ≤ p / q |H|/q\le p/q ∣ H ∣/ q ≤ p / q , which remain true after enlargement. Conversely, every exactly-once witness is an at-most-once witness, so the same placement and extraction lemmas apply. Membership still follows by guessing a legal occurrence-complete graph and verifying its Boolean outputs universally. □
In the forward network of Lemma 4.11, the budget of four counts three hidden ReLUs, namely the two hinges and the sharing wire u u u , and one affine output. The theorem does not assume those vertices or any of their parameter roles are prescribed to the synthesis algorithm. The inventory in Theorem 4.12 contains three negative occurrences, − D , − ( P − w ) , − P -D,-(P-w),-P − D , − ( P − w ) , − P .
A ReLU Network as the Target
Here the target is an explicitly given rational ReLU network F : R d → R F:\mathbb{R}^{d}\to\mathbb{R} F : R d → R , with an affine output. Its graph and all parameter positions are listed explicitly; its parameters need not belong to W W W . The synthesis input is ( F , W , n max ) (F,W,n_{\max}) ( F , W , n m a x ) , and the requirement is
N ( x ) = F ( x ) for every x ∈ R d . N(x)=F(x)\qquad\text{for every }x\in\mathbb{R}^{d}. N ( x ) = F ( x ) for every x ∈ R d .
This formulation asks whether the supplied computation can be represented using the available inventory and node budget. Unlike a nonlinear polynomial target, a ReLU target has the same piecewise-affine form as the candidate networks. This shared form does not ensure that the target is representable with a particular inventory.
Lemma 5.1 (A network encoding of unsatisfiability). From a CNF formula Φ \Phi Φ with d ≥ 1 d\ge1 d ≥ 1 variables and at least one clause, every clause nonempty, one can construct in polynomial time a rational ReLU network F Φ F_{\Phi} F Φ with the following two properties. If Φ \Phi Φ is unsatisfiable, then F Φ = 0 F_{\Phi}=0 F Φ = 0 on all of R d \mathbb{R}^{d} R d . If Φ \Phi Φ is satisfiable, then some Boolean input a a a has F Φ ( a ) = 1 F_{\Phi}(a)=1 F Φ ( a ) = 1 .
Proof. For a positive literal on variable i i i , use the affine function 2 x i − 1 2x_i-1 2 x i − 1 ; for a negative literal, use 1 − 2 x i 1-2x_i 1 − 2 x i . Denote the function corresponding to literal l l l by s l ( x ) s_l(x) s l ( x ) , and put
F Φ ( x ) = max { 0 , min C a clause of Φ max l ∈ C s l ( x ) } . F_{\Phi}(x)=\max\left\{0,\ \min_{C\text{ a clause of }\Phi}\max_{l\in C}s_l(x)\right\}. F Φ ( x ) = max { 0 , C a clause of Φ min l ∈ C max s l ( x ) } .
At a satisfying Boolean input every clause has a literal of value one, and every literal value is either one or minus one. Every clause maximum is therefore one, and the displayed expression is one.
Suppose instead that the expression is positive at some real input x x x . Each clause then has a literal whose affine value is positive. Assign variable i i i to true when x i > 1 / 2 x_i>1/2 x i > 1/2 , and to false otherwise. A positive literal of positive affine value has x i > 1 / 2 x_i>1/2 x i > 1/2 and is true. A negative literal of positive affine value has x i < 1 / 2 x_i<1/2 x i < 1/2 and is true. Thus this assignment satisfies every clause. Consequently an unsatisfiable formula gives the zero function everywhere.
It remains to realize the expression by an explicitly listed ReLU network of polynomial size. Delete repeated literals within each clause and repeated clauses; the displayed expression takes a maximum over the set of literals of a clause and a minimum over the set of clauses, so it does not change. Write the clauses as C 1 , … , C k C_1,\ldots,C_k C 1 , … , C k , with k ≥ 1 k\ge1 k ≥ 1 , and the literals of C j C_j C j as l j , 1 , … , l j , w j l_{j,1},\ldots,l_{j,w_j} l j , 1 , … , l j , w j , with w j ≥ 1 w_j\ge1 w j ≥ 1 . For all real a , b a,b a , b ,
max { a , b } = ρ ( a − b ) + b , min { a , b } = a − ρ ( a − b ) , \max\{a,b\}=\rho(a-b)+b,\qquad\min\{a,b\}=a-\rho(a-b), max { a , b } = ρ ( a − b ) + b , min { a , b } = a − ρ ( a − b ) ,
because ρ ( a − b ) = a − b \rho(a-b)=a-b ρ ( a − b ) = a − b when a ≥ b a\ge b a ≥ b and ρ ( a − b ) = 0 \rho(a-b)=0 ρ ( a − b ) = 0 otherwise.
For every clause C j C_j C j and every t t t with $2\le t\le w_j$, introduce a hidden vertex g j , t g_{j,t} g j , t . Put G j , 1 = s l j , 1 ( x ) G_{j,1}=s_{l_{j,1}}(x) G j , 1 = s l j , 1 ( x ) and, for t ≥ 2 t\ge2 t ≥ 2 , G j , t = g j , t + s l j , t ( x ) G_{j,t}=g_{j,t}+s_{l_{j,t}}(x) G j , t = g j , t + s l j , t ( x ) , and let the preactivation of g j , t g_{j,t} g j , t be G j , t − 1 − s l j , t ( x ) G_{j,t-1}-s_{l_{j,t}}(x) G j , t − 1 − s l j , t ( x ) . Then G j , t = ρ ( G j , t − 1 − s l j , t ( x ) ) + s l j , t ( x ) = max { G j , t − 1 , s l j , t ( x ) } G_{j,t}=\rho(G_{j,t-1}-s_{l_{j,t}}(x))+s_{l_{j,t}}(x)=\max\{G_{j,t-1},s_{l_{j,t}}(x)\} G j , t = ρ ( G j , t − 1 − s l j , t ( x )) + s l j , t ( x ) = max { G j , t − 1 , s l j , t ( x )} , and induction on t t t shows that c j = G j , w j c_j=G_{j,w_j} c j = G j , w j equals max l ∈ C j s l ( x ) \max_{l\in C_j}s_l(x) max l ∈ C j s l ( x ) . Put R 1 = c 1 R_1=c_1 R 1 = c 1 , and for 2 ≤ j ≤ k 2\le j\le k 2 ≤ j ≤ k introduce a hidden vertex h j h_j h j with preactivation $R_{j-1}-c_j, and put R j = R j − 1 − h j R_j=R_{j-1}-h_j R j = R j − 1 − h j . Then R j = R j − 1 − ρ ( R j − 1 − c j ) = min { R j − 1 , c j } R_j=R_{j-1}-\rho(R_{j-1}-c_j)=\min\{R_{j-1},c_j\} R j = R j − 1 − ρ ( R j − 1 − c j ) = min { R j − 1 , c j } , and induction on j j j gives R k = min j c j R_k=\min_j c_j R k = min j c j . Finally introduce a hidden vertex o o o with preactivation R k R_k R k , and let the affine output be the value of o o o , through one edge of weight one and with no bias. The output equals ρ ( R k ) = F Φ ( x ) \rho(R_k)=F_\Phi(x) ρ ( R k ) = F Φ ( x ) at every real input x x x , since every identity used holds for all real numbers.
Each preactivation above is an affine combination of the input coordinates and of the values of earlier hidden vertices, because G j , t G_{j,t} G j , t involves only g j , t g_{j,t} g j , t and the input, and R j = c 1 − ∑ i = 2 j h i R_j=c_1-\sum_{i=2}^j h_i R j = c 1 − ∑ i = 2 j h i . For each preactivation, collect the coefficient of every input coordinate and of every hidden vertex, and the constant term. A nonzero collected coefficient becomes the weight of one edge, a nonzero constant term becomes a present bias, and zero coefficients give absent edges and biases, so there is at most one edge per ordered pair. Every preactivation involves at most two literal functions and hidden coefficients ± 1 \pm1 ± 1 , so all weights and biases are integers of absolute value at most four. In the order g 1 , 2 , … , g k , w k , h 2 , … , h k , o g_{1,2},\ldots,g_{k,w_k},h_2,\ldots,h_k,o g 1 , 2 , … , g k , w k , h 2 , … , h k , o , followed by the output, every edge points forward, so the graph is acyclic.
Every hidden vertex reaches the output: g j , t g_{j,t} g j , t with t < w j t<w_j t < w j enters g j , t + 1 g_{j,t+1} g j , t + 1 with weight one, g 1 , w 1 g_{1,w_1} g 1 , w 1 enters o o o with weight one, g j , w j g_{j,w_j} g j , w j with j ≥ 2 j\ge2 j ≥ 2 enters h j h_j h j with weight minus one, each h j h_j h j enters o o o with weight minus one, and o o o enters the output. Each hidden vertex also has an incoming edge. For g j , t g_{j,t} g j , t with t ≥ 3 t\ge3 t ≥ 3 it is the edge from g j , t − 1 g_{j,t-1} g j , t − 1 , and for h j h_j h j with j ≥ 3 j\ge3 j ≥ 3 it is the edge from h j − 1 h_{j-1} h j − 1 . The preactivation of g j , 2 g_{j,2} g j , 2 is s l j , 1 − s l j , 2 s_{l_{j,1}}-s_{l_{j,2}} s l j , 1 − s l j , 2 for two distinct literals, whose input coefficients are ± 2 \pm2 ± 2 on two variables, or ± 4 \pm4 ± 4 on one variable when the literals are opposite, so it has an input edge. The preactivation of h 2 h_2 h 2 is c 1 − c 2 c_1-c_2 c 1 − c 2 ; it has an edge from g 1 , w 1 g_{1,w_1} g 1 , w 1 or from g 2 , w 2 g_{2,w_2} g 2 , w 2 when w 1 ≥ 2 w_1\ge2 w 1 ≥ 2 or w 2 ≥ 2 w_2\ge2 w 2 ≥ 2 , and otherwise it is s l − s l ′ s_l-s_{l'} s l − s l ′ for the single literals l ≠ l ′ l\ne l' l = l ′ of two distinct clauses, which has an input edge as before. The preactivation of o o o has the edge from h k h_k h k when k ≥ 2 k\ge2 k ≥ 2 ; when k = 1 k=1 k = 1 it is c 1 c_1 c 1 , which has the edge from g 1 , w 1 g_{1,w_1} g 1 , w 1 when w 1 ≥ 2 w_1\ge2 w 1 ≥ 2 and is otherwise a literal function with input coefficient ± 2 \pm2 ± 2 . The network has ∑ j ( w j − 1 ) + ( k − 1 ) + 1 = ∑ j w j \sum_j(w_j-1)+(k-1)+1=\sum_j w_j ∑ j ( w j − 1 ) + ( k − 1 ) + 1 = ∑ j w j hidden vertices and one output, and it is computed in polynomial time. □
This construction reduces unsatisfiability to exact reproduction with the zero network as the only permitted candidate function. It gives the lower bound in the next theorem.
Theorem 5.2 (ReLU-network targets). Exact synthesis on all real inputs for a target supplied as a rational ReLU network is coNP-hard under deterministic polynomial-time many-one reductions and belongs to Σ 2 P \Sigma^P_2 Σ 2 P . Both bounds hold for at-most-once and exactly-once use. The lower bound holds with W = ( 0 ) W=(0) W = ( 0 ) and n max = 1 n_{\max}=1 n m a x = 1 . Equality of a supplied candidate and a supplied target belongs to coNP.
Proof. For candidate verification, guess the activation patterns of both networks and a sign for their nonzero output difference. Introduce variables for the input and all computational values in both networks. For an active hidden vertex, set its value equal to its affine preactivation and require that preactivation to be nonnegative. For an inactive hidden vertex, set its value to zero and require its preactivation to be nonpositive. The output equations are affine. These constraints form a rational linear system of polynomial encoding length. Add the strict inequality N ( x ) − F ( x ) > 0 N(x)-F(x)>0 N ( x ) − F ( x ) > 0 , or its negative according to the guessed sign. If the networks disagree, their actual states at a disagreeing point satisfy one of these systems. Conversely a feasible system gives the actual network values and hence a disagreement.
Such a system has a polynomial-bit rational solution whenever it is feasible. Indeed, introduce a slack s s s , constrain 0 ≤ s ≤ 1 0\le s\le1 0 ≤ s ≤ 1 , and replace the strict signed gap by a gap at least s s s . The original strict system is feasible exactly when this linear program has a positive maximum of s s s .
The objective of this linear program is bounded, so a feasible program attains its optimum. Split unrestricted variables into differences of nonnegative variables. A slack variable for an inequality is an additional nonnegative variable equal to its right side minus its left side, turning that inequality into an equality. An inequality is tight at a point when equality holds. In this standard nonnegative-variable form, a basic feasible solution is a feasible point uniquely determined by a linearly independent selection of tight constraint equations.
To see that an optimal basic feasible solution exists here, write the standard form as A z = b , z ≥ 0 Az=b,z\ge0 A z = b , z ≥ 0 , with objective c ⋅ z c\cdot z c ⋅ z . An optimum exists by Theorem 2.3. Choose an optimum z z z with the smallest number of positive coordinates, let S S S be the set of their indices, and write A S A_S A S for the corresponding column submatrix of A A A . If the columns A S A_S A S were dependent, a nonzero vector d d d supported on S S S would satisfy A d = 0 Ad=0 A d = 0 . Both z + t d z+td z + t d and z − t d z-td z − t d are feasible for all sufficiently small t > 0 t>0 t > 0 , so optimality forces c ⋅ d = 0 c\cdot d=0 c ⋅ d = 0 . Reverse d d d if necessary so it has a positive coordinate, and take t = min i : d i > 0 z i / d i t=\min_{i:d_i>0} z_i/d_i t = min i : d i > 0 z i / d i . Then z − t d z-td z − t d is another optimum, is nonnegative, and has strictly fewer positive coordinates, a contradiction. Thus the columns A S A_S A S are independent.
If S S S is empty, the optimum is the zero vector. Otherwise select ∣ S ∣ |S| ∣ S ∣ independent rows of A S A_S A S ; the resulting square system, together with the zero coordinates outside S S S , uniquely determines the optimum. These are independent tight equations, so the point is a basic feasible solution.
Cramer’s rule expresses each nonzero coordinate as a ratio of determinants of this system. Clear denominators in each row, including its right side. If its integer entries have absolute value at most 2 B b i t 2^{B_{\mathrm{bit}}} 2 B bit and its square size is r r r , the determinant expansion has r ! r! r ! terms of absolute value at most 2 r B b i t 2^{rB_{\mathrm{bit}}} 2 r B bit . Every numerator and nonzero denominator therefore has binary length at most r B b i t + ⌈ log 2 ( r ! ) ⌉ + 2 rB_{\mathrm{bit}}+\lceil\log_2(r!)\rceil+2 r B bit + ⌈ log 2 ( r !)⌉ + 2 , a polynomial bound.
The optimal value of s s s is positive when the strict system is feasible, so the optimal point satisfies the strict signed gap. Discarding s s s and the standard-form slack variables, and replacing each split pair of nonnegative variables by its difference, yields a rational point of the original strict system, the required certificate. By Cramer’s rule all nonzero coordinates of the basic feasible solution share the denominator given by the determinant of the square system, so each such difference has that denominator and a numerator of binary length at most one more than the bound above. The guessed states give actual ReLU values, including either state at a zero preactivation.
Checking the patterns, node values, and signed gap takes polynomial time. Non-equivalence is therefore in NP, proving the verification claim. Guessing a polynomial-size permitted network and applying this coNP test gives synthesis in Σ 2 P \Sigma^{\mathrm{P}}_2 Σ 2 P .
For hardness start with CNF unsatisfiability. Its coNP-completeness follows from circuit satisfiability by the following translation. Introduce one Boolean variable per gate, exclude every invalid assignment to a gate’s input and output variables by a clause, and require the output variable to be true. Each gate has a constant-size truth table, so the resulting CNF has polynomial size and is satisfiable precisely when the circuit is. Empty clauses or a formula with no clauses can be recognized directly and replaced by fixed unsatisfiable or satisfiable formulas with nonempty clauses. Apply Lemma 5.1 and use target F Φ F_\Phi F Φ , inventory W = ( 0 ) W=(0) W = ( 0 ) , and budget one. Every permitted network computes zero, and an output with a zero bias is legal and uses the occurrence exactly once. The lemma shows that synthesis succeeds precisely when Φ \Phi Φ is unsatisfiable. This proves the lower bound for both inventory variants. □
At Most Two Computational Vertices
A small computational budget permits reconstruction from finitely many target values even though equality must hold on all real inputs. The output may have direct input edges, and these edges are included in the reconstruction.
Theorem 5.3 (Two-vertex exact synthesis; restatement of Theorem 1.2). Full-space exact synthesis of a supplied rational ReLU target, with at-most-once inventory, an affine output, and at most two computational vertices, is coNP-complete under deterministic polynomial-time many-one reductions. The input dimension is unrestricted.
Proof. Write T T T for the supplied target throughout this proof. Write m = ∣ W ∣ m=|W| m = ∣ W ∣ for the number of inventory occurrences, and choose an integer R ≥ 2 R\ge2 R ≥ 2 bounding the absolute numerators and positive denominators of the reduced inventory values. The binary length of R R R is polynomial in the input length. Put B = 4 R 2 + 4 B=4R^2+4 B = 4 R 2 + 4 , and let v ∈ Q d v\in\mathbb{Q}^d v ∈ Q d have coordinates v i = B i v_i=B^i v i = B i , for 1 ≤ i ≤ d 1\le i\le d 1 ≤ i ≤ d . These coordinates also have polynomial binary length. We construct a polynomial list of legal candidates containing an exact witness whenever one exists.
A nonaffine network with at most two computational vertices has exactly one hidden vertex and computes
p ⋅ x + q + c ρ ( a ⋅ x + b ) , a ≠ 0 , c ≠ 0. p\cdot x+q+c\rho(a\cdot x+b), \qquad a\ne0,\quad c\ne0. p ⋅ x + q + c ρ ( a ⋅ x + b ) , a = 0 , c = 0.
Here p p p and q q q are the direct output coefficients and the output bias, a a a and b b b are the input weights and bias of the hidden vertex, and c c c is its output gain. Both a ≠ 0 a\ne0 a = 0 and c ≠ 0 c\ne0 c = 0 hold, because a = 0 a=0 a = 0 makes the hidden value the constant ρ ( b ) \rho(b) ρ ( b ) and c = 0 c=0 c = 0 removes the hidden term, and either would make the computed function affine. Every nonzero displayed coefficient is one inventory occurrence; zero positions may be omitted. If k k k is the largest index with a k ≠ 0 a_k \ne0 a k = 0 , then ∣ a k ∣ ≥ 1 / R \lvert a_k\rvert\ge1/R ∣ a k ∣ ≥ 1/ R and
∣ a ⋅ v ∣ ≥ B k R − R ∑ i < k B i > 3 B k 4 R > 2 R . \lvert a \cdot v\rvert\ge\frac{B^k}{R}-R\sum_{i<k}B^i>\frac{3B^k}{4R}>2R. ∣ a ⋅ v ∣ ≥ R B k − R i < k ∑ B i > 4 R 3 B k > 2 R .
Indeed the geometric sum is less than B k / ( B − 1 ) B^k/(B-1) B k / ( B − 1 ) , and R / ( B − 1 ) < 1 / ( 4 R ) R/(B-1)<1/(4R) R / ( B − 1 ) < 1/ ( 4 R ) . Since ∣ b ∣ , ∣ a i ∣ ≤ R \lvert b\rvert,\lvert a_i\rvert\le R ∣ b ∣ , ∣ a i ∣ ≤ R , the hidden preactivation has opposite signs at v v v and − v -v − v , and adding any coordinate unit vector e i e_i e i preserves each sign. Thus the target values at these points recover both affine pieces of every true nonaffine witness:
p s , i = T ( s v + e i ) − T ( s v ) , q s = T ( s v ) − p s ⋅ ( s v ) , s ∈ { + 1 , − 1 } . p_{s,i}=T(sv+e_i)-T(sv),\qquad q_s=T(sv)-p_s\cdot(sv),\qquad s\in\{+1,-1\}. p s , i = T ( s v + e i ) − T ( s v ) , q s = T ( s v ) − p s ⋅ ( s v ) , s ∈ { + 1 , − 1 } .
For each choice of the sign s s s at which the hidden vertex is inactive and of a nonzero output-gain occurrence c ∈ W c\in W c ∈ W , form
a i = ( p − s , i − p s , i ) / c , b = ( q − s − q s ) / c . a_i=(p_{-s,i}-p_{s,i})/c,\qquad b=(q_{-s}-q_s)/c. a i = ( p − s , i − p s , i ) / c , b = ( q − s − q s ) / c .
Here p s p_s p s is the vector with coordinates p s , i p_{s,i} p s , i . The inactive sign is not known in advance, so both values of s s s are tried; only at the inactive sign are p s p_s p s and q s q_s q s the direct output coefficients p p p and the output bias q q q . The candidate graph for the sign s s s uses p s p_s p s as its direct output coefficients and q s q_s q s as its output bias. Omit zero positions, require a ≠ 0 a\ne0 a = 0 , and retain this graph only if all nonzero positions can be assigned distinct occurrences of their numerical values. This is a polynomial-time multiset check. Every retained graph is legal: the hidden vertex has a nonzero input edge and reaches the output through the nonzero gain c c c . There are at most 2 m 2m 2 m such candidates.
It remains to include every possible affine witness. Evaluate the target at zero and at every coordinate unit vector. Let L 0 ( x ) = α ⋅ x + β L_0(x)=\alpha\cdot x+\beta L 0 ( x ) = α ⋅ x + β be their affine interpolant, where β = T ( 0 ) \beta=T(0) β = T ( 0 ) and α i = T ( e i ) − T ( 0 ) \alpha_i=T(e_i)-T(0) α i = T ( e i ) − T ( 0 ) . Its realizability with at most two vertices is decidable in polynomial time. First test a graph with no hidden vertex by matching its nonzero coefficients to the inventory. If all coefficients vanish, a present zero occurrence is necessary and sufficient, since by Definition 2.1 the output must have an incoming edge or a present bias to be legal. Next consider a constant hidden vertex, meaning a hidden vertex whose incoming edges, if any, all carry zero, so that its value is ρ ( b ) \rho(b) ρ ( b ) for its bias b b b . When that bias is present, deleting those zero edges changes neither the function nor legality, so we may assume the vertex has no incoming edge. Enumerate distinct nonzero occurrences b , c b,c b , c for its bias and output gain, and an optional nonzero output-bias occurrence q q q , allowing the bias to be absent with value zero. Test
β = q + c ρ ( b ) \beta=q+c\rho(b) β = q + c ρ ( b )
and simultaneously match the nonzero direct slopes α i \alpha_i α i , with all numerical multiplicities respected. These tests cover every witness whose constant hidden vertex has a nonzero bias and a nonzero output gain; the remaining constant cases reduce to the no-hidden test, as shown below.
A hidden vertex with a nonconstant preactivation and a nonzero output gain cannot produce an affine function, even after the affine skip terms are added. Indeed, write that function as p ⋅ x + q + c ρ ( a ⋅ x + b ) p\cdot x+q+c\rho(a\cdot x+b) p ⋅ x + q + c ρ ( a ⋅ x + b ) , with a ≠ 0 , c ≠ 0 a\ne0,c\ne0 a = 0 , c = 0 . On a ⋅ x + b < 0 a\cdot x+b<0 a ⋅ x + b < 0 its coefficient vector is p p p , and on a ⋅ x + b > 0 a\cdot x+b>0 a ⋅ x + b > 0 it is p + c a p+ca p + c a . Both open halfspaces are nonempty: the points x = ( − b − 1 ) a / ( a ⋅ a ) x=(-b-1)a/(a\cdot a) x = ( − b − 1 ) a / ( a ⋅ a ) and x = ( − b + 1 ) a / ( a ⋅ a ) x=(-b+1)a/(a\cdot a) x = ( − b + 1 ) a / ( a ⋅ a ) lie in them, respectively. Each point admits sufficiently small perturbations in every coordinate while staying in its halfspace, so near each point these perturbations read off that halfspace’s coefficient vector. A globally affine function has a single coefficient vector everywhere, which would then equal both p p p and p + c a p+ca p + c a . Their difference is c a ≠ 0 ca\ne0 c a = 0 , a contradiction.
A zero output gain contributes nothing, so its hidden vertex can be pruned. If the output is then left with no incoming edge and no present bias, the zero occurrence of that gain can serve as a present zero bias, which makes the output legal. A hidden vertex whose incoming edge weights and bias are all zero likewise computes zero and can be pruned, leaving a no-hidden witness that may use one of the vertex’s zero occurrences as its output bias. A constant hidden vertex with zero bias likewise computes zero and reduces to the no-hidden test in the same way. These cases exhaust all affine witnesses. If any test succeeds, append one legal realization of L 0 L_0 L 0 .
The resulting list has at most 2 m + 1 2m+1 2 m + 1 graphs. If the supplied cap is one, use only the no-hidden affine feasibility test and discard every reconstructed graph with two vertices.
All target evaluations and candidate coefficients have polynomial bit length. The bounds in the proof of Theorem 7.1 are stated for Boolean inputs, so we repeat the argument for the integer probe points used here. Let L ≥ 2 L \ge2 L ≥ 2 be the input length, let D D D be the product of the positive denominators of all parameter positions of T T T , and let x x x be one of the probe points 0 0 0 , e i e_i e i , s v sv s v or s v + e i sv+e_i s v + e i . Every coordinate of x x x is an integer of absolute value at most X = B d + 1 X = B^d+1 X = B d + 1 . Freezing the activation pattern of T T T at x x x expresses T ( x ) T(x) T ( x ) as a sum of bias-to-output path products and of input-to-output path products, each of the latter multiplied by one coordinate of x x x . A path uses each parameter position at most once and the coordinates of x x x are integers, so D D D times each such term is an integer, and D T ( x ) DT(x) D T ( x ) is an integer.
Every parameter of T T T has absolute value at most 2 L 2^L 2 L , every preactivation has at most L L L terms, T T T has at most L L L computational vertices, and D < 2 L D < 2^L D < 2 L . Every input value is at most X X X in absolute value, and every bias is at most 2 L ≤ 2 L X 2^L \le2^L X 2 L ≤ 2 L X . Induction in topological order therefore bounds the absolute value of the j j j th computational value by ( L 2 L ) j X (L2^L)^jX ( L 2 L ) j X , since a ReLU does not increase absolute value. Hence ∣ D T ( x ) ∣ ≤ 2 L ( L 2 L ) L X |DT(x)| \le2^L(L2^L)^L X ∣ D T ( x ) ∣ ≤ 2 L ( L 2 L ) L X , whose binary length is at most L + L log 2 L + L 2 + log 2 X + 1 L+L\log_2 L+L^2+\log_2 X+1 L + L log 2 L + L 2 + log 2 X + 1 . This is polynomial in the input length, because log 2 X ≤ 1 + d log 2 B \log_2 X \le1+d\log_2 B log 2 X ≤ 1 + d log 2 B and the coordinates of v v v have polynomial binary length. The quantities p s , i p_{s,i} p s , i , q s q_s q s , a i a_i a i , b b b , α i \alpha_i α i and β \beta β arise from polynomially many such values by polynomially many subtractions, multiplications by the integers ± B i \pm B^i ± B i , sums of d d d terms and divisions by one inventory value, so they also have polynomial bit length.
If a true witness is nonaffine, its two pieces and gain put it on the list. If it is affine, it equals L 0 L_0 L 0 , whose feasibility test succeeds. Conversely each retained graph is a permitted candidate, so synthesis succeeds precisely when one listed graph equals T T T globally. By Theorem 5.2, each equality test is in coNP. A polynomial disjunction of these tests remains in coNP: a certificate for failure gives one polynomial-length inequivalence certificate for every listed graph. An empty list is rejected deterministically.
For hardness, use the W = ( 0 ) W=(0) W = ( 0 ) construction of Theorem 5.2 with cap two. The single occurrence still permits only one computational vertex, so the only candidate function is zero. The same reduction from CNF unsatisfiability applies. □
The reconstruction relies on having only one hidden vertex. For example, with cap three the legal network
h 1 = ρ ( x − 1 2 ) , h 2 = ρ ( x − 1 4 − 2 h 1 ) , N = h 2 h_1=\rho\left(x-\frac{1}{2}\right),\qquad h_2=\rho\left(x-\frac{1}{4}-2h_1\right),\qquad N=h_2 h 1 = ρ ( x − 2 1 ) , h 2 = ρ ( x − 4 1 − 2 h 1 ) , N = h 2
uses W = ( 1 , 1 , 1 , − 2 , − 1 2 , − 1 4 ) W=(1,1,1,-2,-\frac{1}{2},-\frac{1}{4}) W = ( 1 , 1 , 1 , − 2 , − 2 1 , − 4 1 ) , with the third occurrence of 1 1 1 as the unit output edge from h 2 h_2 h 2 , and equals ρ ( 1 4 − ∣ x − 1 2 ∣ ) \rho\left(\frac{1}{4}-\left|x-\frac{1}{2}\right|\right) ρ ( 4 1 − x − 2 1 ) . Here R = 4 R=4 R = 4 and v = 68 v=68 v = 68 , and the function vanishes at all six probe points 0 , 1 , 68 , 69 , − 68 , − 67 0,1,68,69,-68,-67 0 , 1 , 68 , 69 , − 68 , − 67 . Thus those probes alone do not reconstruct a general cap-three witness. This example limits the stated reconstruction procedure and does not classify larger budgets.
A Fixed Computational-Node Cap
A fixed number of computational vertices still permits arbitrarily many input coordinates and inventory occurrences. Nevertheless, a polynomially constructible set of inputs distinguishes all different functions realizable with that cap. The set depends on the inventory and the cap, rather than on the target. Recall that DP consists of languages A ∩ C ‾ A\cap\overline{C} A ∩ C with A , C ∈ N P A,C\in\mathrm{NP} A , C ∈ NP .
Theorem 5.4 (Fixed-cap exact synthesis; restatement of Theorem 1.2). For each fixed integer H ≥ 1 H \ge1 H ≥ 1 , full-space exact synthesis of a supplied rational ReLU target with computational-node cap H H H belongs to DP, under either inventory rule. The dimension, inventory size and target size are unrestricted.
We first locate disagreements in low-dimensional coordinate subspaces. Recall that the support of a vector is the set of coordinates where it is nonzero.
Lemma 5.5 (Sparse disagreements). If two ReLU networks with affine outputs and respectively h 1 , h 2 h_1,h_2 h 1 , h 2 hidden vertices compute different functions on R d \mathbb{R}^d R d , they disagree at a vector whose support has size at most h 1 + h 2 + 1 h_1+h_2+1 h 1 + h 2 + 1 .
Proof. Choose an input x x x where the outputs differ. For each hidden vertex v v v in either network, let a v ∈ R d a_v\in\mathbb{R}^d a v ∈ R d contain the coefficients of its direct input edges, with zero for an absent edge. Its preactivation is therefore a v ⋅ x + b v a_v\cdot x+b_v a v ⋅ x + b v plus a linear combination of its hidden predecessors’ values. For network i ∈ { 1 , 2 } i\in\{1,2\} i ∈ { 1 , 2 } , let p i ∈ R d p_i\in\mathbb{R}^d p i ∈ R d be its direct input-to-output coefficient vector. Form a matrix A A A with one row a v a_v a v for every hidden vertex in the two networks and one final row p 1 − p 2 p_1-p_2 p 1 − p 2 . It has h 1 + h 2 + 1 h_1+h_2+1 h 1 + h 2 + 1 rows.
Choose a set J J J of column indices whose columns form a basis of the column space of A A A , meaning the linear span of its columns, and write A J A_J A J for their submatrix. Because A x Ax A x is in that column space, there is a vector u u u satisfying A J u = A x A_Ju = Ax A J u = A x . Set x J ′ = u x'_J = u x J ′ = u and every coordinate outside J J J to zero. Then A x ′ = A x Ax' = Ax A x ′ = A x , and x ′ x' x ′ has support of size at most
∣ J ∣ = rank A ≤ h 1 + h 2 + 1. |J| = \operatorname{rank} A \le h_1 + h_2 + 1. ∣ J ∣ = rank A ≤ h 1 + h 2 + 1.
Every hidden vertex has the same value at x ′ x' x ′ and x x x . Indeed, process each network in topological order. At a hidden vertex v v v , the direct input contribution is unchanged because a v ⋅ x ′ = a v ⋅ x a_v \cdot x' = a_v \cdot x a v ⋅ x ′ = a v ⋅ x . Its bias is unchanged, and the induction hypothesis preserves the values of all hidden predecessors. Thus its entire preactivation, and hence its ReLU value, is unchanged. The two output biases and all hidden-to-output contributions are consequently unchanged as well. Their direct input difference is preserved by the last row of A A A : ( p 1 − p 2 ) ⋅ x ′ = ( p 1 − p 2 ) ⋅ x (p_1-p_2)\cdot x' = (p_1-p_2)\cdot x ( p 1 − p 2 ) ⋅ x ′ = ( p 1 − p 2 ) ⋅ x . The output difference is therefore the same nonzero number at both inputs. If A A A has rank zero, take J = ∅ J = \varnothing J = ∅ and x ′ = 0 x' = 0 x ′ = 0 ; the same argument applies. □
The sparse disagreement bound permits a finite set of tests for the whole candidate class.
Lemma 5.6 (A separating set for a fixed cap). For fixed H H H , one can construct in deterministic polynomial time a polynomial-size set S ⊆ Q d S \subseteq\mathbb{Q}^d S ⊆ Q d of polynomial-bit vectors such that any two legal cap-H H H candidates agreeing on S S S agree on all of R d \mathbb{R}^d R d .
Proof. Put r = 2 H − 1 r = 2H - 1 r = 2 H − 1 . A cap-H H H candidate has at most H − 1 H - 1 H − 1 hidden vertices, since the cap counts the affine output, so the support bound h 1 + h 2 + 1 h_1+h_2+1 h 1 + h 2 + 1 of Lemma 5.5 is at most r r r for two candidates. Enumerate the coordinate subsets J ⊆ [ d ] J \subseteq[d] J ⊆ [ d ] of size at most r r r , including the empty subset. There are at most ∑ j = 0 r d j \sum_{j=0}^{r}d^j ∑ j = 0 r d j of them. For each J J J , enumerate topologically labeled local graphs with input vertices for the coordinates in J J J and at most H H H computational vertices, the last of which is the affine output. Assign distinct occurrences of W W W to present edges and biases, allowing unused occurrences. Only for this auxiliary enumeration, allow parameterless vertices and vertices without a path to the output. A parameterless vertex has value zero. These relaxed graphs are test functions, not enlarged legal synthesis witnesses.
There are at most C = H ( r + H + 1 ) C = H(r+H+1) C = H ( r + H + 1 ) edge and bias positions in a local graph. Assigning either absence or one of the m m m occurrences to each position, then rejecting repeated occurrence IDs, gives at most a constant multiple of ( m + 1 ) C (m+1)^C ( m + 1 ) C descriptions. This is polynomial because H H H is fixed. A legal global candidate restricted to the coordinate subspace supported on J J J occurs in this enumeration: delete its edges from coordinates outside J J J , retaining every other position, vertex and occurrence ID. On that subspace the deleted edges contribute zero. A deletion may leave a vertex with no parameters, and such a vertex is one of the relaxed parameterless vertices, whose value is zero, so the restriction is still described locally.
For each pair of local descriptions, enumerate their activation masks and the two signs of an output disagreement. There are constantly many masks. Their masked computations and preactivation inequalities are affine; adding a strict signed output gap gives a rational linear feasibility problem. A weak choice at a zero preactivation gives the actual ReLU value, so these systems cover all degeneracies. Rational linear programming decides feasibility in polynomial time. For a strict gap introduce 0 ≤ δ ≤ 1 0 \le\delta\le1 0 ≤ δ ≤ 1 , require the gap to be at least δ \delta δ , and maximize δ \delta δ . A strict gap exists exactly when the maximum is positive. The polynomial-bit witness bound is the same rational determinant argument used in Theorem 5.2. The local affine coefficients have polynomial bit length by path expansion. Thus a feasible pair supplies a polynomial-bit rational distinguishing input.
Call two local descriptions inequivalent when they compute different functions of the coordinates in J J J . For every inequivalent pair, select one distinguishing input from a feasible system, and embed it in R d \mathbb{R}^d R d by putting zero outside J J J . Let S S S be the set of these selected distinguishing points. Its construction and size are polynomial. If legal global candidates differ, Lemma 5.5 gives a coordinate set of size at most r r r on which their restrictions differ. The corresponding pair of local descriptions has a selected distinguishing point in S S S . Consequently agreement on S S S implies global agreement. This reasoning also covers an empty separating set. □
The separating set removes ambiguity among consistent candidates, while the supplied target may still require a universal equality test.
Proof of Theorem 5.4. Construct S S S and evaluate the supplied target F F F at each of its points. All these evaluations have polynomial bit complexity by the rational path bound. Let A A A be the language asserting that some legal candidate agrees with F F F on S S S . Let C C C assert that some such consistent legal candidate disagrees with F F F at some real input. Both are in NP: guess the graph and occurrence assignment, check legality and the sample equalities, and for C C C additionally use the polynomial-bit inequivalence certificate of Theorem 5.2. For exactly-once use, the legality check additionally requires every occurrence to be assigned; the same separating set is valid for this subclass.
If an exact witness exists, every consistent candidate computes its function by Lemma 5.6, so A A A holds and C C C fails. Conversely, if A A A holds and C C C fails, any consistent candidate is an exact witness. The synthesis language is therefore A ∩ C ‾ A \cap\overline{C} A ∩ C , proving the claimed upper bound. □
Theorem 5.4 is a bound for each fixed cap; it gives no DP upper bound for arbitrary supplied caps. The probe enumeration grows as ( d + m ) O ( H 2 ) (d+m)^{O(H^2)} ( d + m ) O ( H 2 ) . For cap two, Theorem 5.3 gives the stronger at-most-once classification. The coNP lower bound with W = ( 0 ) W=(0) W = ( 0 ) remains valid for every fixed cap.
Three Computational Vertices
For cap three the preceding DP upper bound is attained: Theorem 5.9 proves matching DP-hardness. The numerical part uses an explicit affine target; a separate target component enforces unsatisfiability. We use the distinct-number form of Numerical Matching with Target Sums (NMTS). Strong NP-completeness means that NP-completeness persists when the numerical integers are encoded in unary, with encoding length proportional to their values.
Theorem 5.7 (Numerical matching [22 Appendix, Corollary 8] ). Given three lists of positive integers ( a j ) j = 1 d (a_j)_{j=1}^{d} ( a j ) j = 1 d , ( b j ) j = 1 d (b_j)_{j=1}^{d} ( b j ) j = 1 d , ( t i ) i = 1 d (t_i)_{i=1}^{d} ( t i ) i = 1 d , deciding whether there are permutations π , σ \pi,\sigma π , σ of { 1 , … , d } \{1,\ldots,d\} { 1 , … , d } such that a π ( i ) + b σ ( i ) = t i a_{\pi(i)}+b_{\sigma(i)}=t_i a π ( i ) + b σ ( i ) = t i for every i i i is strongly NP-complete, even when all 3 d 3d 3 d integers are pairwise distinct and ∑ j a j + ∑ j b j = ∑ i t i \sum_j a_j+\sum_j b_j=\sum_i t_i ∑ j a j + ∑ j b j = ∑ i t i .
These promises will ensure distinct inventory values without changing the matching question. An affine target is homogeneous here when its constant term is zero.
Theorem 5.8 (Affine synthesis with cap three). Full-space synthesis of an explicitly encoded rational affine target with computational-node cap three is NP-complete under either inventory rule. Hardness holds for homogeneous targets with distinct positive integer slopes and distinct nonzero signed integer inventory values.
Proof. Membership follows by guessing the legal graph and occurrence assignment. At most two hidden vertices give four activation masks. For each mask, Theorem 2.3 tests either strict sign of a disagreement with the affine target by maximizing a slack in [ 0 , 1 ] [0,1] [ 0 , 1 ] , as in Theorem 5.2. This is polynomial time for the guessed candidate.
For hardness, use Numerical Matching with Target Sums on positive integers ( a j ) j = 1 d , ( b j ) j = 1 d , ( t i ) i = 1 d (a_j)_{j=1}^{d},(b_j)_{j=1}^{d},(t_i)_{i=1}^{d} ( a j ) j = 1 d , ( b j ) j = 1 d , ( t i ) i = 1 d , all 3 d 3d 3 d numbers distinct. It asks for permutations π , σ \pi,\sigma π , σ with a π ( i ) + b σ ( i ) = t i a_{\pi(i)}+b_{\sigma(i)}=t_i a π ( i ) + b σ ( i ) = t i . Theorem 5.7 supplies strong NP-hardness with these promises and matching total sums. We may assume d ≥ 3 d \ge3 d ≥ 3 : the smaller cases have constantly many matchings to test and can be replaced by fixed distinct-number YES or NO instances. For example, a = ( 1 , 2 , 3 ) a=(1,2,3) a = ( 1 , 2 , 3 ) , b = ( 10 , 20 , 30 ) b=(10,20,30) b = ( 10 , 20 , 30 ) and t = ( 11 , 22 , 33 ) t=(11,22,33) t = ( 11 , 22 , 33 ) form a YES instance, while t = ( 11 , 21 , 34 ) t=(11,21,34) t = ( 11 , 21 , 34 ) gives a NO instance; both have matching total sums. Put
R = max { a j , b j , t i } , M = 10 R + 10 , B = M + R , R=\max\{a_j,b_j,t_i\},\qquad M=10R+10,\qquad B=M+R, R = max { a j , b j , t i } , M = 10 R + 10 , B = M + R ,
and supply
W = ( M + a j ) j ⊎ ( M + b j ) j ⊎ ( − M − b j ) j ⊎ ( 1 , − 1 ) , T ( x ) = ∑ i = 1 d ( 2 M + t i ) x i . W=(M+a_j)_j \mathbin{\uplus} (M+b_j)_j \mathbin{\uplus} (-M-b_j)_j \mathbin{\uplus} (1,-1),\qquad T(x)=\sum_{i=1}^{d}(2M+t_i)x_i. W = ( M + a j ) j ⊎ ( M + b j ) j ⊎ ( − M − b j ) j ⊎ ( 1 , − 1 ) , T ( x ) = i = 1 ∑ d ( 2 M + t i ) x i .
The unit occurrences are 1 , − 1 1,-1 1 , − 1 ; call all other occurrences nonunit. Here m = 3 d + 2 m=3d+2 m = 3 d + 2 , all values are distinct nonzero integers, every nonunit magnitude lies in [ M + 1 , B ] [M+1,B] [ M + 1 , B ] , and
2 M + t i > B , ( M + 1 ) 2 > 3 M + 2 R . 2M+t_i>B,\qquad(M+1)^2>3M+2R. 2 M + t i > B , ( M + 1 ) 2 > 3 M + 2 R .
The transformation has polynomial binary length.
A source matching gives
L ( x ) = ∑ i ( M + b σ ( i ) ) x i , h + = ρ ( L ) , h − = ρ ( − L ) , N = ∑ i ( M + a π ( i ) ) x i + h + − h − . L(x)=\sum_i(M+b_{\sigma(i)})x_i,\qquad h_+=\rho(L),\qquad h_-=\rho(-L),\qquad N=\sum_i(M+a_{\pi(i)})x_i+h_+-h_-. L ( x ) = i ∑ ( M + b σ ( i ) ) x i , h + = ρ ( L ) , h − = ρ ( − L ) , N = i ∑ ( M + a π ( i ) ) x i + h + − h − .
The identity ρ ( L ) − ρ ( − L ) = L \rho(L)-\rho(-L)=L ρ ( L ) − ρ ( − L ) = L proves N = T N=T N = T . The three input-edge lists and the two unit output gains use every occurrence exactly once. There are exactly three computational vertices, no biases or parallel edges, and each hidden vertex reaches the output.
Conversely take any at-most-once cap-three witness for T T T , including possible biases and nested vertices. Let N ~ \widetilde{N} N be the computation obtained by deleting its biases, before pruning zero vertices. For all real s , t s,t s , t , ∣ ρ ( s ) − ρ ( t ) ∣ ≤ ∣ s − t ∣ |\rho(s)-\rho(t)|\leq|s-t| ∣ ρ ( s ) − ρ ( t ) ∣ ≤ ∣ s − t ∣ : the same-sign cases follow directly from the definition, and in the opposite-sign case the nonzero output difference is at most the input difference. Also ρ ( λ t ) = λ ρ ( t ) \rho(\lambda t)=\lambda\rho(t) ρ ( λ t ) = λ ρ ( t ) for λ > 0 \lambda>0 λ > 0 .
For each input vertex v v v set C v = 0 C_v=0 C v = 0 , and for each computational vertex v v v , in topological order, put
C v = ∣ b v ∣ + ∑ u : ( u , v ) ∈ E ∣ a u v ∣ C u . C_v=|b_v|+\sum_{u:(u,v)\in E}|a_{uv}|C_u. C v = ∣ b v ∣ + u : ( u , v ) ∈ E ∑ ∣ a uv ∣ C u .
Write z v z_v z v and z ~ v \widetilde{z}_v z v for the original and bias-free vertex values. The preceding ReLU inequality and the triangle inequality prove, by induction in the same topological order,
∣ z v ( λ x ) λ − z ~ v ( x ) ∣ ≤ C v λ . \left|\frac{z_v(\lambda x)}{\lambda}-\widetilde{z}_v(x)\right|\leq\frac{C_v}{\lambda}. λ z v ( λ x ) − z v ( x ) ≤ λ C v .
At a hidden vertex this uses the already bounded predecessor differences and the divided bias b v / λ b_v/\lambda b v / λ ; the affine output satisfies the same estimate without applying ReLU. The target is positively homogeneous, T ( λ x ) = λ T ( x ) T(\lambda x)=\lambda T(x) T ( λ x ) = λ T ( x ) , and N = T N=T N = T . Thus the output estimate gives ∣ N ~ ( x ) − T ( x ) ∣ ≤ C o u t / λ |\widetilde{N}(x)-T(x)|\leq C_{\mathrm{out}}/\lambda ∣ N ( x ) − T ( x ) ∣ ≤ C out / λ for every λ > 0 \lambda>0 λ > 0 . A positive difference would violate this bound for sufficiently large λ \lambda λ , so N ~ = T \widetilde{N}=T N = T .
Repeatedly delete newly parameterless hidden zero vertices together with their outgoing edges, until none remains, then prune vertices not reaching the output. Because T ≠ 0 T\neq0 T = 0 , the output remains legal. We obtain a bias-free at-most-once witness with no larger cap and no new occurrence.
Let p i p_i p i be its direct input-output coefficient, or zero if absent, so ∣ p i ∣ ≤ B |p_i|\leq B ∣ p i ∣ ≤ B . The hidden computation must supply the linear residual r ⋅ x r\cdot x r ⋅ x , where
r i = 2 M + t i − p i , 1 < M + 1 − R ≤ r i ≤ 3 M + 2 R . r_i=2M+t_i-p_i,\qquad1<M+1-R\leq r_i\leq3M+2R. r i = 2 M + t i − p i , 1 < M + 1 − R ≤ r i ≤ 3 M + 2 R .
No-hidden synthesis would require the unavailable slope 2 M + t i 2M+t_i 2 M + t i . With a single hidden vertex, its contribution has a fixed sign on all inputs and cannot equal this nonzero linear residual. Thus there are two hidden vertices. Order them and write
h 1 = ρ ( a ⋅ x ) , h 2 = ρ ( c ⋅ x + u h 1 ) , N = p ⋅ x + v h 1 + w h 2 . h_1=\rho(a\cdot x),\qquad h_2=\rho(c\cdot x+uh_1),\qquad N=p\cdot x+vh_1+wh_2. h 1 = ρ ( a ⋅ x ) , h 2 = ρ ( c ⋅ x + u h 1 ) , N = p ⋅ x + v h 1 + w h 2 .
An absent internal edge is denoted by u = 0 u=0 u = 0 . Reachability gives w ≠ 0 w\neq0 w = 0 , and a ≠ 0 a\neq0 a = 0 because otherwise there is only one effective hidden vertex. Also v ≠ 0 v\neq0 v = 0 , since w h 2 wh_2 w h 2 alone has a fixed sign.
On a ⋅ x < 0 a\cdot x<0 a ⋅ x < 0 we have r ⋅ x = w ρ ( c ⋅ x ) r\cdot x=w\rho(c\cdot x) r ⋅ x = w ρ ( c ⋅ x ) . If c c c is not proportional to a a a , set
z = c − c ⋅ a a ⋅ a a . z=c-\frac{c\cdot a}{a\cdot a}a. z = c − a ⋅ a c ⋅ a a .
The denominator is positive because a ≠ 0 a\neq0 a = 0 . Then a ⋅ z = 0 a\cdot z=0 a ⋅ z = 0 , while c ⋅ z = z ⋅ z > 0 c\cdot z=z\cdot z>0 c ⋅ z = z ⋅ z > 0 , since nonproportionality makes z ≠ 0 z\neq0 z = 0 . The line x 0 + s z x_0+sz x 0 + sz , with a ⋅ x 0 < 0 a\cdot x_0<0 a ⋅ x 0 < 0 , stays in that halfspace and makes the right side a nonaffine univariate hinge, contradicting the linear left side. Therefore c = t a c=ta c = t a . For t ≥ 0 t\geq0 t ≥ 0 , the residual vanishes on this open halfspace, contrary to its nonzero coefficients. Thus t < 0 t<0 t < 0 , and equality there gives r = w c r=wc r = w c globally.
If w w w is nonunit, every nonzero c i c_i c i must be a unit occurrence: otherwise ∣ w c i ∣ ≥ ( M + 1 ) 2 > 3 M + 2 R |wc_i|\geq(M+1)^2>3M+2R ∣ w c i ∣ ≥ ( M + 1 ) 2 > 3 M + 2 R . But all d ≥ 3 d\geq3 d ≥ 3 coefficients c i c_i c i are nonzero, because r = w c r=wc r = w c and every r i ≥ M + 1 − R > 1 r_i\geq M+1-R>1 r i ≥ M + 1 − R > 1 . Distinct nonzero c i c_i c i sit on distinct edges and therefore use distinct occurrences, while only two unit occurrences exist. Consequently w = 1 w=1 w = 1 or − 1 -1 − 1 . Every skip p i p_i p i is now nonzero, since p i = 0 p_i=0 p i = 0 would imply ∣ c i ∣ = 2 M + t i > B |c_i|=2M+t_i>B ∣ c i ∣ = 2 M + t i > B . Moreover every a i ≠ 0 a_i\neq0 a i = 0 , since c = t a c=ta c = t a and every c i ≠ 0 c_i\neq0 c i = 0 . The witness therefore has d d d input edges into each hidden vertex, d d d skips, and the two nonzero gains v , w v,w v , w . These 3 d + 2 = m 3d+2=m 3 d + 2 = m distinct positions consume the entire inventory. No occurrence is left for the internal edge, so u = 0 u=0 u = 0 .
On a ⋅ x > 0 a\cdot x>0 a ⋅ x > 0 , we now have h 2 = 0 h_2=0 h 2 = 0 , hence r = v a r=va r = v a . The same magnitude and unit-scarcity argument forces v v v to be a unit. Since v = w t v=wt v = w t and t < 0 t<0 t < 0 , the gains are opposite units and t = − 1 t=-1 t = − 1 , giving c = − a c=-a c = − a . Both unit occurrences are used as gains. The two hidden input edges of each coordinate must therefore use a positive and a negative occurrence of equal nonunit magnitude. The only available pairs are M + b j , − M − b j M+b_j,-M-b_j M + b j , − M − b j ; source distinctness excludes every M + a j M+a_j M + a j value from those pairs. All pairs are used bijectively, leaving exactly the M + a j M+a_j M + a j values for skips. As every residual slope is positive,
2 M + t i = ( M + a π ( i ) ) + ( M + b σ ( i ) ) . 2M+t_i=(M+a_{\pi(i)})+(M+b_{\sigma(i)}). 2 M + t i = ( M + a π ( i ) ) + ( M + b σ ( i ) ) .
which extracts an NMTS matching. The converse covered the larger at-most-once class, and the forward witness used every occurrence, so hardness holds under both inventory rules. □
The explicit affine classification combines with a universal target condition.
Theorem 5.9 (ReLU-target synthesis with cap three; restatement of Theorem 1.2 ). Full-space exact synthesis of a supplied rational ReLU target with computational-node cap three is DP-complete under deterministic polynomial-time many-one reductions, under either inventory rule.
Proof. Membership is Theorem 5.4 . For hardness combine the NMTS instance above with a CNF formula Φ \Phi Φ . Let F Φ ( z ) F_\Phi(z) F Φ ( z ) be the nonnegative network of Lemma 5.1 , whose input z z z has one coordinate for each variable of Φ \Phi Φ . With three fresh coordinates y y y , supply
T ^ ( x , z , y ) = T ( x ) + min { F Φ ( z ) , 1 , ρ ( y 1 ) , ρ ( y 2 ) , ρ ( y 3 ) } . \widehat{T}(x,z,y)=T(x)+\min\{F_\Phi(z),1,\rho(y_1),\rho(y_2),\rho(y_3)\}. T ( x , z , y ) = T ( x ) + min { F Φ ( z ) , 1 , ρ ( y 1 ) , ρ ( y 2 ) , ρ ( y 3 )} .
keeping the inventory and cap three unchanged. Binary minimum and the signed-pair construction in Lemma 5.1 compile this target in polynomial size. If Φ \Phi Φ is unsatisfiable, the added term is zero. A matching then gives the preceding witness using all occurrences. Conversely, consider a witness on the inputs ( x , z , y ) (x,z,y) ( x , z , y ) that computes T ( x ) T(x) T ( x ) , which is the target T ^ \widehat{T} T when Φ \Phi Φ is unsatisfiable. Restrict it by setting the added inputs z , y z,y z , y to zero and deleting their edges. Repeatedly pruning parameterless zero vertices and vertices not reaching the output gives a legal at-most-once witness for the affine target T T T ; the nonzero T T T keeps the output legal. Thus a NO matching instance cannot become YES. This holds even when the expanded witness uses every occurrence exactly once, because its restriction is still an at-most-once witness, which the converse in the proof of Theorem 5.8 excludes.
If Φ \Phi Φ is satisfiable, fix a Boolean z ∗ z^* z ∗ with F Φ ( z ∗ ) = 1 F_\Phi(z^*)=1 F Φ ( z ∗ ) = 1 and fix x = 0 x=0 x = 0 . The target slice is
Q ( y ) = min { 1 , ρ ( y 1 ) , ρ ( y 2 ) , ρ ( y 3 ) } . Q(y)=\min\{1,\rho(y_1),\rho(y_2),\rho(y_3)\}. Q ( y ) = min { 1 , ρ ( y 1 ) , ρ ( y 2 ) , ρ ( y 3 )} .
It has five distinct affine formulas on nonempty open sets: zero, one, and y 1 , y 2 , y 3 y_1,y_2,y_3 y 1 , y 2 , y 3 . The zero formula occurs when y 1 < 0 y_1<0 y 1 < 0 , one when all coordinates exceed one, and y i y_i y i when it is positive, below one, and smaller than both others. Any cap-three candidate, even after arbitrary inputs are fixed, has at most two hidden vertices and four masks, hence at most four affine output formulas.
On an open set where the target equals an affine form L L L , some masked formula must equal L L L identically. Otherwise each difference between a masked formula and L L L is an affine function that is not identically zero, and the zero sets of these finitely many differences, called proper affine zero sets, would cover that open set. Each proper affine zero set is a hyperplane or empty. A hyperplane cannot contain an open ball: moving a point slightly in the direction of its nonzero normal changes the defining affine value. Inside any open ball one can therefore choose a point off the hyperplane and then a smaller open ball around that point disjoint from it, since its affine value stays nonzero under sufficiently small perturbations. Repeating this choice for finitely many hyperplanes leaves a nonempty open ball outside their union. Hence finitely many proper affine zero sets cannot cover the open set in question. Five distinct formulas therefore cannot be represented by four masks.
The constructed instance is YES precisely when NMTS is YES and Φ \Phi Φ is unsatisfiable. For every language A ∩ C ‾ A\cap\overline{C} A ∩ C with A , C ∈ N P A,C\in\mathrm{NP} A , C ∈ NP , independently reduce A A A to NMTS and C C C to CNF satisfiability, then apply this construction. It is a polynomial-time many-one reduction from that language, proving DP-hardness directly from the class definition. Both inventory rules follow from the same forward and converse arguments. □
These classifications distinguish cap two from cap three while leaving unrestricted-budget synthesis open. They do not automatically prove hardness with any larger fixed cap: extra hidden vertices can invalidate the numerical converse.
Uniform Approximation on a Compact Box
By Theorem 5.4 , the full-space exact problem has a DP upper bound at every fixed cap. We now change the requirement to uniform approximation on a supplied compact box. The target is a supplied rational ReLU network F F F , and the input also lists rational endpoints a j < b j a_j < b_j a j < b j and a rational tolerance ε ≥ 0 \varepsilon\ge0 ε ≥ 0 . With B = ∏ j = 1 d [ a j , b j ] \mathcal{B} = \prod_{j=1}^{d}[a_j,b_j] B = ∏ j = 1 d [ a j , b j ] , the requirement is
∣ N ( x ) − F ( x ) ∣ ≤ ε for every x ∈ B . |N(x)-F(x)| \le\varepsilon\qquad\text{for every } x \in\mathcal{B}. ∣ N ( x ) − F ( x ) ∣ ≤ ε for every x ∈ B .
A violation for a fixed candidate is certified by activation masks, the box constraints, and one strict signed error inequality. The rational linear feasibility argument in Theorem 5.2 therefore puts candidate verification in coNP and synthesis in Σ 2 P \Sigma^P_2 Σ 2 P . This Σ 2 P \Sigma^P_2 Σ 2 P upper bound for uniform approximation is tight even with no hidden candidate vertex and an inventory of identical positive occurrences, by Theorem 6.2 .
A Covering Source
The target supplies the complexity of the uniform error test. We first state the succinct covering problem used in the reduction. For a Boolean circuit D D D on s s s input bits, its one-set is { y ∈ { 0 , 1 } s : D ( y ) = 1 } \{y \in\{0,1\}^{s}: D(y)=1\} { y ∈ { 0 , 1 } s : D ( y ) = 1 } . A family of sets covers the Boolean cube when every cube point belongs to at least one set. Such covering formulations are standard sources of second-level hardness [Uma99]; the circuit version below has a direct proof.
Lemma 6.1 (Circuit-specified set cover). Given Boolean circuits D 1 , … , D n D_1,\ldots,D_n D 1 , … , D n on the same input bits and an integer 1 ≤ k < n 1 \le k < n 1 ≤ k < n , deciding whether some k k k or fewer of their one-sets cover the Boolean cube is Σ 2 P \Sigma^P_2 Σ 2 P -complete. Hardness holds when every one-set is nonempty and the full family covers the cube.
Proof. Membership follows by guessing the chosen indices and universally checking the input bits. For hardness let p ≥ 1 , s ≥ 0 p \ge1,s \ge0 p ≥ 1 , s ≥ 0 be integers and let C C C be a Boolean circuit on p + s p+s p + s input bits. Start with the quantified source sentence
∃ u ∈ { 0 , 1 } p ∀ y ∈ { 0 , 1 } s C ( u , y ) = 1. \exists u \in\{0,1\}^{p}\ \forall y \in\{0,1\}^{s}\ C(u,y)=1. ∃ u ∈ { 0 , 1 } p ∀ y ∈ { 0 , 1 } s C ( u , y ) = 1.
An unused existential bit can be added if necessary. These circuit sentences are complete for the defining existential-universal polynomial-time predicate class. Indeed, a polynomial-time deterministic computation has polynomially many configuration bits and steps. One transition is computed by a polynomial-size Boolean circuit, and unrolling the polynomially many transitions yields a polynomial-size circuit for the predicate. Thus any predicate in the definition of Σ 2 P \Sigma^P_2 Σ 2 P has the required circuit representation.
Use universe coordinates ( u ′ , y , w ) (u',y,w) ( u ′ , y , w ) , where u ′ u' u ′ has p p p bits and the r = ⌈ log 2 ( p + 2 ) ⌉ r=\lceil\log_2(p+2)\rceil r = ⌈ log 2 ( p + 2 )⌉ bits of w w w encode an integer. Each code c c c specifies the subcube of universe points with w = c w=c w = c : code zero specifies the main subcube, codes 1 , … , p 1,\ldots,p 1 , … , p specify pair-control subcubes, and code p + 1 p+1 p + 1 specifies one additional control subcube. Write U ( w ) U(w) U ( w ) for the predicate w > p + 1 w>p+1 w > p + 1 , covering any unused codes. For 1 ≤ i ≤ p 1 \le i \le p 1 ≤ i ≤ p and b ∈ { 0 , 1 } b \in\{0,1\} b ∈ { 0 , 1 } , define
D i , b = ( w = 0 ∧ u i ′ ≠ b ) ∨ ( w = i ) ∨ U ( w ) , D_{i,b}=(w=0 \land u'_i \ne b)\lor(w=i)\lor U(w), D i , b = ( w = 0 ∧ u i ′ = b ) ∨ ( w = i ) ∨ U ( w ) ,
and define
D ∗ = ( w = 0 ∧ C ( u ′ , y ) ) ∨ ( w = p + 1 ) ∨ U ( w ) . D_*=(w=0 \land C(u',y))\lor(w=p+1)\lor U(w). D ∗ = ( w = 0 ∧ C ( u ′ , y )) ∨ ( w = p + 1 ) ∨ U ( w ) .
These are 2 p + 1 2p+1 2 p + 1 circuits of polynomial size; choose budget p + 1 p+1 p + 1 . Every set is nonempty because it contains its designated control subcube. The full family covers every code: at code zero, either D i , 0 D_{i,0} D i , 0 or D i , 1 D_{i,1} D i , 1 holds for any fixed i i i , and the remaining codes have their control or unused-code coverage.
Only D ∗ D_* D ∗ covers code p + 1 p+1 p + 1 , and code i i i is covered only by D i , 0 D_{i,0} D i , 0 and by D i , 1 D_{i,1} D i , 1 , each of which covers it on its own. The budget therefore forces D ∗ D_* D ∗ and exactly one choice D i , b i D_{i,b_i} D i , b i for each i i i . At code zero, the chosen sets D 1 , b 1 , … , D p , b p D_{1,b_1},\ldots,D_{p,b_p} D 1 , b 1 , … , D p , b p cover every u ′ ≠ b u'\ne b u ′ = b , where b = ( b 1 , … , b p ) b=(b_1,\ldots,b_p) b = ( b 1 , … , b p ) . They cover no point with u ′ = b u'=b u ′ = b , so coverage of that remaining subcube is exactly C ( b , y ) = 1 C(b,y)=1 C ( b , y ) = 1 for every y y y . This proves both directions and polynomial-time many-one hardness. □ \square □
A polynomial is multilinear if each variable has exponent at most one in every monomial. Its extrema on a box occur at vertices: with the other coordinates fixed, it is affine in one coordinate, whose maximizing or minimizing endpoint can be chosen in turn. This fact will control fractional input and gate coordinates without interpreting them as Boolean values.
Theorem 6.2 (Compact approximation; restatement of Theorem 1.3 ). Uniform-approximation synthesis for a rational ReLU target on a rational box is Σ 2 P \Sigma_{2}^{P} Σ 2 P -complete under deterministic polynomial-time many-one reductions. Hardness holds on the unit box, at tolerance one, with candidate node budget one and an inventory consisting of identical positive rational occurrences. The same completeness holds for the restriction to explicitly listed multilinear polynomial targets of degree at most five with candidate node budget one. Both assertions hold for at-most-once and exactly-once use, with affine scalar outputs.
Proof. We first construct a polynomial target with a positive integer tolerance, replace it by a ReLU target preserving the uniform-error condition, and then rescale to tolerance one. Take circuits D 1 , … , D n D_{1},\ldots,D_{n} D 1 , … , D n and budget 1 ≤ k < n 1 \le k < n 1 ≤ k < n from Lemma 6.1. Introduce feature coordinates x 1 , … , x n x_{1},\ldots,x_{n} x 1 , … , x n , source input coordinates y y y , and coordinates z z z for all internal gate values of these circuits. The feature coordinate x i x_{i} x i is required to equal circuit D i D_{i} D i ’s output in a consistent Boolean assignment. Two further coordinates s , t s,t s , t will select calibration conditions. All coordinates range independently over [ 0 , 1 ] [0,1] [ 0 , 1 ] .
For every gate, form a multilinear polynomial that is zero at consistent Boolean gate assignments and one at inconsistent ones. For distinct gate coordinates a , b , v a,b,v a , b , v , use
q A N D ( a , b , v ) = v + a b − 2 a b v , q_{\mathrm{AND}}(a,b,v) = v+ab-2abv, q AND ( a , b , v ) = v + ab − 2 ab v ,
q O R ( a , b , v ) = ( 1 − v ) ( a + b − a b ) + v ( 1 − a ) ( 1 − b ) . q_{\mathrm{OR}}(a,b,v) = (1-v)(a+b-ab)+v(1-a)(1-b). q OR ( a , b , v ) = ( 1 − v ) ( a + b − ab ) + v ( 1 − a ) ( 1 − b ) .
For a NOT gate and an equality constraint, respectively, use
q N O T ( a , v ) = 1 − a − v + 2 a v , q E Q ( a , v ) = a + v − 2 a v . q_{\mathrm{NOT}}(a,v) = 1-a-v+2av,\qquad q_{\mathrm{EQ}}(a,v) = a+v-2av. q NOT ( a , v ) = 1 − a − v + 2 a v , q EQ ( a , v ) = a + v − 2 a v .
A fixed-zero or fixed-one wire uses v v v or 1 − v 1-v 1 − v . Repeated gate inputs can be simplified or replaced by fresh copy coordinates with equality constraints. Every local polynomial has degree at most three and, by its Boolean values and multilinearity, lies in [ 0 , 1 ] [0,1] [ 0 , 1 ] throughout its local cube. Let Q ( x , y , z ) Q(x,y,z) Q ( x , y , z ) be their sum, including the output equalities, and let g ≥ 1 g \ge1 g ≥ 1 be their number. Then 0 ≤ Q ≤ g 0 \le Q \le g 0 ≤ Q ≤ g everywhere. At Boolean vertices, Q = 0 Q=0 Q = 0 means complete gate and output consistency; every other vertex has Q ≥ 1 Q \ge1 Q ≥ 1 . Expanding these constant-size local formulas gives a polynomial-size explicit monomial list.
Put S = ∑ i x i S=\sum_{i}x_{i} S = ∑ i x i , choose the integer E = n + k + g + 1 E=n+k+g+1 E = n + k + g + 1 , and define
P = − E + S + s t ( 2 E + 1 − Q − S ) . P=-E+S+st(2E+1-Q-S). P = − E + S + s t ( 2 E + 1 − Q − S ) .
This polynomial is multilinear of degree at most five and has polynomially many explicitly listed coefficients of polynomial bit length. Initially use tolerance E E E , node budget one, and k k k occurrences of the value one. Every candidate is therefore
N = b + ∑ j ∈ J ξ j , N=b+\sum_{j\in J}\xi_{j}, N = b + j ∈ J ∑ ξ j ,
where ξ = ( x , y , z , s , t ) \xi=(x,y,z,s,t) ξ = ( x , y , z , s , t ) , the index set J J J lists the distinct coordinates that receive an input edge, and b b b is zero for an absent bias or one for a present bias. At most k k k positions are occupied, or exactly k k k under the exactly-once rule. The output must occupy at least one position.
Set s = t = 0 s=t=0 s = t = 0 and x = 0 x=0 x = 0 . The target is − E -E − E , independently of y , z y,z y , z , so the error bound forces the nonnegative candidate value to be zero. This excludes its bias and every edge from y , z y,z y , z : any such coordinate can be set to one at a calibration vertex. The calibration vertices ( s , t ) = ( 1 , 0 ) (s,t)=(1,0) ( s , t ) = ( 1 , 0 ) and ( 0 , 1 ) (0,1) ( 0 , 1 ) , again with x = y = z = 0 x=y=z=0 x = y = z = 0 , exclude the two selector edges. Every successful candidate must consequently be
N = ∑ i ∈ I x i , 1 ≤ ∣ I ∣ ≤ k , N=\sum_{i\in I}x_{i},\qquad1\le|I|\le k, N = i ∈ I ∑ x i , 1 ≤ ∣ I ∣ ≤ k ,
with equality in the cardinality bound for exactly-once use.
For any Boolean source input y y y , choose its unique consistent gate values and feature outputs, and set s = t = 1 s=t=1 s = t = 1 . Then Q = 0 Q=0 Q = 0 and P = E + 1 P=E+1 P = E + 1 . The inequality P − N ≤ E P-N\le E P − N ≤ E requires ∑ i ∈ I D i ( y ) ≥ 1 \sum_{i\in I}D_{i}(y)\ge1 ∑ i ∈ I D i ( y ) ≥ 1 . Thus a successful candidate extracts a cover.
Conversely, choose a cover I I I with ∣ I ∣ ≤ k |I|\le k ∣ I ∣ ≤ k . For exactly-once use, pad it to k k k distinct indices; this is possible because k < n k<n k < n , and adding sets preserves coverage. Use the one-node candidate N = ∑ i ∈ I x i N=\sum_{i\in I}x_i N = ∑ i ∈ I x i . At a Boolean vertex with s t = 0 st=0 s t = 0 , the two error inequalities are
S − 2 E ≤ N ≤ S , (1) S-2E\le N\le S, \tag*{(1)} S − 2 E ≤ N ≤ S , (1)
which hold because $0 \le N \le S \le n < 2E$. At a vertex with s = t = 1 s=t=1 s = t = 1 , they are
1 − Q ≤ N ≤ 2 E + 1 − Q . 1-Q \le N \le2E+1-Q. 1 − Q ≤ N ≤ 2 E + 1 − Q .
For the lower bound, a consistent vertex is covered and an inconsistent vertex has Q ≥ 1 Q \ge1 Q ≥ 1 . For the upper bound, use N ≤ k N \le k N ≤ k , Q ≤ g Q \le g Q ≤ g , and 2 E + 1 − g ≥ k 2E+1-g \ge k 2 E + 1 − g ≥ k . Both signed errors are multilinear, so their vertex bounds extend to the entire unit box. This proves the reduction for the polynomial target.
To produce a ReLU target, write the multilinear polynomial as P ( ξ ) = ∑ A c A ∏ j ∈ A ξ j P(\xi)=\sum_A c_A\prod_{j\in A}\xi_j P ( ξ ) = ∑ A c A ∏ j ∈ A ξ j , with at most five coordinates in each support A A A . Replace it by the standard Lovasz extension of its Boolean vertex values [19 Section 4] :
F ( ξ ) = ∑ A c A min j ∈ A ξ j , F(\xi)=\sum_A c_A\min_{j\in A}\xi_j, F ( ξ ) = A ∑ c A j ∈ A min ξ j ,
where the minimum for A = ∅ A=\varnothing A = ∅ is defined to be one. This is the established min-polynomial representation, not a new extension. Each nonempty minimum has an explicit constant-size rational ReLU implementation, valid on all of R D \mathbb{R}^D R D and in particular on the box. Remove the supports with c A = 0 c_A=0 c A = 0 . For a remaining nonempty support A = { j 1 , … , j h } A=\{j_1,\ldots,j_h\} A = { j 1 , … , j h } , put m A , 1 = ξ j 1 m_{A,1}=\xi_{j_1} m A , 1 = ξ j 1 , and for 2 ≤ t ≤ h 2\le t\le h 2 ≤ t ≤ h introduce a hidden vertex v A , t v_{A,t} v A , t with
v A , t = ρ ( m A , t − 1 − ξ j t ) , m A , t = m A , t − 1 − v A , t . v_{A,t}=\rho\left(m_{A,t-1}-\xi_{j_t}\right),\qquad m_{A,t}=m_{A,t-1}-v_{A,t}. v A , t = ρ ( m A , t − 1 − ξ j t ) , m A , t = m A , t − 1 − v A , t .
The identity min { a , b } = a − ρ ( a − b ) \min\{a,b\}=a-\rho(a-b) min { a , b } = a − ρ ( a − b ) , valid for all real a , b a,b a , b , and induction on t t t give m A , h = min j ∈ A ξ j m_{A,h}=\min_{j\in A}\xi_j m A , h = min j ∈ A ξ j . Since m A , t − 1 = ξ j 1 − ∑ t ′ = 2 t − 1 v A , t ′ m_{A,t-1}=\xi_{j_1}-\sum_{t'=2}^{t-1}v_{A,t'} m A , t − 1 = ξ j 1 − ∑ t ′ = 2 t − 1 v A , t ′ , the preactivation of v A , t v_{A,t} v A , t has an edge of weight one from ξ j 1 \xi_{j_1} ξ j 1 , an edge of weight minus one from ξ j t \xi_{j_t} ξ j t , edges of weight minus one from v A , 2 , … , v A , t − 1 v_{A,2},\ldots,v_{A,t-1} v A , 2 , … , v A , t − 1 , and no bias. The affine output is c ∅ + ∑ A ≠ ∅ c A ( ξ j 1 − ∑ t = 2 h v A , t ) c_{\varnothing}+\sum_{A\ne\varnothing}c_A(\xi_{j_1}-\sum_{t=2}^h v_{A,t}) c ∅ + ∑ A = ∅ c A ( ξ j 1 − ∑ t = 2 h v A , t ) , with j 1 j_1 j 1 and h h h those of A A A : its bias is the constant term c ∅ = − E ≠ 0 c_{\varnothing}=-E\ne0 c ∅ = − E = 0 , each v A , t v_{A,t} v A , t enters it with weight − c A ≠ 0 -c_A\ne0 − c A = 0 , and each coordinate enters it with its collected coefficient when that is nonzero, so negative coefficients are output weights and there are no parallel edges. Ordering the vertices support by support and then by t t t makes every edge point forward, every hidden vertex has two input edges and an edge to the output, and there are at most four hidden vertices per support. Thus F F F is an explicitly encoded rational ReLU target of polynomial size.
We verify that this replacement preserves every affine candidate’s error bound. Let D D D be the total number of coordinates in ξ \xi ξ . Sort their values as r 1 ≥ ⋯ ≥ r D r_1\ge\cdots\ge r_D r 1 ≥ ⋯ ≥ r D , and put r 0 = 1 , r D + 1 = 0 r_0=1,r_{D+1}=0 r 0 = 1 , r D + 1 = 0 . Let a j a^j a j be the Boolean vertex whose first j j j coordinates in that order are one, and write a i j a_i^j a i j for its coordinate i i i . The numbers λ j = r j − r j + 1 \lambda_j=r_j-r_{j+1} λ j = r j − r j + 1 , for 0 ≤ j ≤ D 0\le j\le D 0 ≤ j ≤ D , are nonnegative, sum to one, and satisfy ξ = ∑ j λ j a j \xi=\sum_j\lambda_j a^j ξ = ∑ j λ j a j . For every monomial support, including the empty support,
min i ∈ A ξ i = ∑ j = 0 D λ j ∏ i ∈ A a i j . \min_{i\in A}\xi_i=\sum_{j=0}^{D}\lambda_j\prod_{i\in A}a_i^j. i ∈ A min ξ i = j = 0 ∑ D λ j i ∈ A ∏ a i j .
Therefore F ( ξ ) = ∑ j λ j P ( a j ) F(\xi)=\sum_j\lambda_jP(a^j) F ( ξ ) = ∑ j λ j P ( a j ) , while an affine candidate satisfies N ( ξ ) = ∑ j λ j N ( a j ) N(\xi)=\sum_j\lambda_jN(a^j) N ( ξ ) = ∑ j λ j N ( a j ) . The signed difference is a convex combination of its Boolean vertex differences. Its absolute value is consequently at most E E E everywhere if it is at most E E E at every vertex. The reverse implication is immediate because the vertices belong to the box. This argument includes negative coefficients, constant terms, and ties in the coordinate ordering.
Finally divide the target and every inventory occurrence by E E E , and set tolerance one. For a one-node affine candidate this uniformly divides its output by E E E , so legal descriptions correspond exactly to those above. The inventory now consists of k k k occurrences of 1 / E 1/E 1/ E , and the cover witness has at least one input edge and no bias. The reduction is polynomial in the original circuit input length and respects both inventory rules.
For ReLU targets, membership is the activation-pattern argument given at the start of this section. For the polynomial restriction, the candidate is affine and the explicitly listed target is multilinear, so a strict error violation has a box vertex witness, specified by one endpoint-choice bit per coordinate. Checking that witness uses polynomial-time rational arithmetic. Guessing the permitted candidate and universally checking the vertices proves membership in Σ 2 P \Sigma^P_2 Σ 2 P . Both completeness assertions follow.
Exact Population Risk on the Boolean Cube
A target given explicitly also permits an exact population-risk question without a sampling oracle. A probability distribution specifies the probabilities of possible inputs. A density is a nonnegative function of integral one; probabilities of regions are its integrals over those regions. For a random input X X X with probability distribution ν \nu ν and a real function f f f , the expectation E X ∼ ν [ f ( X ) ] \mathbb{E}_{X\sim\nu}[f(X)] E X ∼ ν [ f ( X )] is the average of f f f under that probability distribution. On a finite input set it is a probability-weighted sum, and for the continuous input distributions used here it is an integral against the specified density.
All continuous integrals below use Lebesgue measure, the usual volume on Euclidean space. A measurable function is one whose superlevel sets { x : f ( x ) > t } \{x:f(x)>t\} { x : f ( x ) > t } are measurable for every real t t t . For an event A A A , its indicator 1 A \mathbf{1}_{A} 1 A is one on A A A and zero otherwise, so Pr [ A ] = E [ 1 A ] \Pr[A]=\mathbb{E}[\mathbf{1}_{A}] Pr [ A ] = E [ 1 A ] . Independent coordinates have the product of their separate probability distributions as their joint probability distribution. A fair bit takes each value in { 0 , 1 } \{0,1\} { 0 , 1 } with probability 1 / 2 1/2 1/2 .
A measurable real-valued function f f f is integrable under ν \nu ν when E ν [ ∣ f ∣ ] < ∞ \mathbb{E}_{\nu}[|f|]<\infty E ν [ ∣ f ∣ ] < ∞ . A measurable real-valued function f f f is square-integrable under a probability distribution ν \nu ν when E ν [ f 2 ] < ∞ \mathbb{E}_{\nu}[f^{2}]<\infty E ν [ f 2 ] < ∞ ; write
∥ f ∥ 2 , ν = ( E ν [ f 2 ] ) 1 / 2 . \|f\|_{2,\nu}=\left(\mathbb{E}_{\nu}[f^{2}]\right)^{1/2}. ∥ f ∥ 2 , ν = ( E ν [ f 2 ] ) 1/2 .
The pointwise inequality 2 ∣ f g ∣ ≤ f 2 + g 2 2|fg|\le f^{2}+g^{2} 2∣ f g ∣ ≤ f 2 + g 2 , obtained from ( ∣ f ∣ − ∣ g ∣ ) 2 ≥ 0 (|f|-|g|)^{2}\ge0 ( ∣ f ∣ − ∣ g ∣ ) 2 ≥ 0 , shows that f g fg f g is integrable whenever f , g f,g f , g are square-integrable.
For square-integrable f , g f,g f , g , Cauchy–Schwarz is the inequality
∣ E ν [ f g ] ∣ ≤ ∥ f ∥ 2 , ν ∥ g ∥ 2 , ν . |\mathbb{E}_{\nu}[fg]|\le\|f\|_{2,\nu}\|g\|_{2,\nu}. ∣ E ν [ f g ] ∣ ≤ ∥ f ∥ 2 , ν ∥ g ∥ 2 , ν .
To verify it, suppose first that E ν [ g 2 ] = 0 \mathbb{E}_{\nu}[g^{2}]=0 E ν [ g 2 ] = 0 . Then g = 0 g=0 g = 0 except on a set of probability zero, and both sides vanish. Otherwise expand the nonnegative average E ν [ ( f − t g ) 2 ] \mathbb{E}_{\nu}[(f-tg)^{2}] E ν [( f − t g ) 2 ] and take t = E ν [ f g ] / E ν [ g 2 ] t=\mathbb{E}_{\nu}[fg]/\mathbb{E}_{\nu}[g^{2}] t = E ν [ f g ] / E ν [ g 2 ] . This gives
( E ν [ f g ] ) 2 ≤ E ν [ f 2 ] E ν [ g 2 ] . \left(\mathbb{E}_{\nu}[fg]\right)^{2}\le\mathbb{E}_{\nu}[f^{2}]\mathbb{E}_{\nu}[g^{2}]. ( E ν [ f g ] ) 2 ≤ E ν [ f 2 ] E ν [ g 2 ] .
Expanding the square and applying Cauchy–Schwarz gives
E ν [ ( f + g ) 2 ] ≤ ∥ f ∥ 2 , ν 2 + 2 ∥ f ∥ 2 , ν ∥ g ∥ 2 , ν + ∥ g ∥ 2 , ν 2 = ( ∥ f ∥ 2 , ν + ∥ g ∥ 2 , ν ) 2 . \mathbb{E}_{\nu}[(f+g)^{2}]\le\|f\|_{2,\nu}^{2}+2\|f\|_{2,\nu}\|g\|_{2,\nu}+\|g\|_{2,\nu}^{2}=\left(\|f\|_{2,\nu}+\|g\|_{2,\nu}\right)^{2}. E ν [( f + g ) 2 ] ≤ ∥ f ∥ 2 , ν 2 + 2∥ f ∥ 2 , ν ∥ g ∥ 2 , ν + ∥ g ∥ 2 , ν 2 = ( ∥ f ∥ 2 , ν + ∥ g ∥ 2 , ν ) 2 .
Taking nonnegative square roots proves the triangle inequality ∥ f + g ∥ 2 , ν ≤ ∥ f ∥ 2 , ν + ∥ g ∥ 2 , ν \|f+g\|_{2,\nu}\le\|f\|_{2,\nu}+\|g\|_{2,\nu} ∥ f + g ∥ 2 , ν ≤ ∥ f ∥ 2 , ν + ∥ g ∥ 2 , ν .
We also use convex averaging, or Jensen’s inequality: if X X X and f ( X ) f(X) f ( X ) are integrable and f : R → R f:\mathbb{R}\to\mathbb{R} f : R → R is convex, then f ( E [ X ] ) ≤ E [ f ( X ) ] f(\mathbb{E}[X])\le\mathbb{E}[f(X)] f ( E [ X ]) ≤ E [ f ( X )] . For completeness we prove it; in this argument only, m = E [ X ] m=\mathbb{E}[X] m = E [ X ] denotes the mean, not the inventory size ∣ W ∣ |W| ∣ W ∣ . Convexity makes every secant slope ( f ( m ) − f ( u ) ) / ( m − u ) (f(m)-f(u))/(m-u) ( f ( m ) − f ( u )) / ( m − u ) , with u < m u<m u < m , at most every secant slope ( f ( v ) − f ( m ) ) / ( v − m ) (f(v)-f(m))/(v-m) ( f ( v ) − f ( m )) / ( v − m ) , with v > m v>m v > m . The supremum of the former and infimum of the latter are finite: use u = m − 1 u=m-1 u = m − 1 and v = m + 1 v=m+1 v = m + 1 to bound the two collections on the needed sides. Choose a real c c c between them. Then f ( t ) ≥ f ( m ) + c ( t − m ) f(t)\ge f(m)+c(t-m) f ( t ) ≥ f ( m ) + c ( t − m ) for every real t t t , on either side of m m m . Averaging at t = X t=X t = X proves the inequality.
Jensen’s inequality also applies after fixing some independent input coordinates, called observed coordinates below, by using the product probability distribution of the remaining coordinates. Conditioning on specified observed coordinates means fixing those coordinates and averaging with their conditional probability distribution; the other coordinates keep their original probability distributions when all input coordinates are independent.
Let T : R d → R T:\mathbb{R}^{d}\to\mathbb{R} T : R d → R be a supplied rational ReLU network with an affine output. Write U d U_d U d for the uniform distribution on { 0 , 1 } d \{0,1\}^{d} { 0 , 1 } d , whose coordinates are independent fair bits. For a permitted network N N N , define
R T ( N ) = E X ∼ U d [ ( N ( X ) − T ( X ) ) 2 ] = 2 − d ∑ x ∈ { 0 , 1 } d ( N ( x ) − T ( x ) ) 2 . R_T(N)=\mathbb{E}_{X\sim U_d}[(N(X)-T(X))^{2}]=2^{-d}\sum_{x\in\{0,1\}^{d}}(N(x)-T(x))^{2}. R T ( N ) = E X ∼ U d [( N ( X ) − T ( X ) ) 2 ] = 2 − d x ∈ { 0 , 1 } d ∑ ( N ( x ) − T ( x ) ) 2 .
The decision input consists of T , W , n max T,W,n_{\max} T , W , n m a x and a rational threshold τ \tau τ . The question is whether some permitted network satisfies R T ( N ) ≤ τ R_T(N)\le\tau R T ( N ) ≤ τ . Candidate networks have affine outputs. Unless exactly-once use is specified, occurrences are used at most once. All input coordinates and the target graph are listed explicitly.
The relevant counting class is PP. A language belongs to this class if a polynomial-time machine making independent fair binary choices accepts an input precisely when strictly more than half its computation paths accept. The class N P P P \mathrm{NP}^{\mathrm{PP}} NP PP allows a nondeterministic polynomial-time machine to query a language in PP. The input encodings and running times here are ordinary binary encodings and bit running times.
Theorem 7.1 (Exact population risk; restatement of Theorem 1.4). Under either inventory rule, deciding whether R T ( N ) ≤ τ R_T(N) \le\tau R T ( N ) ≤ τ for some permitted network is N P P P \mathrm{NP}^{\mathrm{PP}} NP PP -complete under deterministic polynomial-time many-one reductions. Hardness holds for integer inventory, with computational-node cap two under at-most-once use and cap m = ∣ W ∣ m = |W| m = ∣ W ∣ under exactly-once use. Both caps include the affine output. The hard targets take only the values − 1 , + 1 -1,+1 − 1 , + 1 on the Boolean cube.
For the lower bound we use an existential choice followed by a strict majority test. The following is the source problem, including the threshold convention needed below.
Theorem 7.2 (Littman, Goldsmith, and Mundhenk [28 Theorem 15] ). Given a Boolean formula φ ( x , z ) \varphi(x,z) φ ( x , z ) , deciding whether there exists an assignment to x x x such that strictly more than half the assignments to z z z satisfy φ ( x , z ) \varphi(x,z) φ ( x , z ) is N P P P \mathrm{NP}^{\mathrm{PP}} NP PP -complete.
The parity of a list of bits is their sum modulo two. We use a parity construction from probabilistic explanation complexity. It prevents a candidate with limited coordinate access from observing the random bits used by the source formula.
Lemma 7.3 (Parity replacement; Koriche, Lagniez, and Tran [25 Definition 4 and Lemma 2] ). Replace each of several independent fair bits by the parity of its own block of k + 1 k + 1 k + 1 fresh independent fair bits. Conditioning on the values of at most k k k fresh coordinates in total leaves these block parities independent and uniform. They also remain independent of all other independent coordinates outside the replacement blocks.
The cited construction was used for conditional relevance of sparse explanations. Here the additional ingredient is an integer-output inequality for unconditional squared loss. The source problem, parity replacement, and counting framework are established ingredients; the occurrence-constrained synthesis reduction is proved below.
Proof of Theorem 7.1. Membership. Let L ≥ 2 L \ge2 L ≥ 2 be the total binary input length and m = ∣ W ∣ m = |W| m = ∣ W ∣ . A guessed legal network has at most m m m computational vertices, because every vertex requires a distinct incoming-edge or bias occurrence. Its graph and occurrence assignment therefore have polynomial length. All graph conditions in Definition 2.1, including the cap, can be checked in polynomial time.
Let D D D be the product of the positive denominators of the occurrences in W W W and of all parameter positions in the supplied target. Repeated target values contribute separate denominator factors for their separate positions. The binary length of D D D is at most L L L . Fix a Boolean input x x x . Replacing each active hidden ReLU by the identity and each inactive hidden ReLU by zero gives an acyclic affine computation with the same value at x x x . At a zero preactivation either choice gives that value. Its expansion is a sum of input-to-output and bias-to-output path products. Every path uses a parameter position at most once. For the candidate, distinct positions also use distinct inventory occurrences. Every path denominator consequently divides D D D , so both D N ( x ) DN(x) D N ( x ) and D T ( x ) DT(x) D T ( x ) are integers.
Exact evaluation has polynomial bit complexity. For an explicit bound, every parameter has absolute value at most 2 L 2^L 2 L , every affine preactivation has at most L L L terms, and each network has at most L L L computational vertices. Induction through its vertices gives absolute value at most ( L 2 L ) L (L2^L)^L ( L 2 L ) L on Boolean inputs. A ReLU does not increase this bound. Together with the common denominator, this bounds every intermediate numerator by a polynomial number of bits. In particular,
A ( x ) = ( D N ( x ) − D T ( x ) ) 2 A(x) = (DN(x) - DT(x))^2 A ( x ) = ( D N ( x ) − D T ( x ) ) 2
is a nonnegative integer with A ( x ) < 2 K A(x) < 2^K A ( x ) < 2 K , where the uniform choice K = 8 ( L + 1 ) 2 K = 8(L+1)^2 K = 8 ( L + 1 ) 2 suffices. Indeed log 2 max { 1 , A ( x ) } \log_2 \max\{1,A(x)\} log 2 max { 1 , A ( x )} is at most 2 L + 2 + 2 L log 2 L + 2 L 2 < K 2L + 2 + 2L\log_2 L + 2L^2 < K 2 L + 2 + 2 L log 2 L + 2 L 2 < K .
A polynomial-time counting machine guesses d d d input bits and K K K bits specifying an integer t ∈ { 0 , … , 2 K − 1 } t \in\{0,\ldots,2^K - 1\} t ∈ { 0 , … , 2 K − 1 } . It accepts exactly when t < A ( x ) t < A(x) t < A ( x ) . Its number of accepting paths is
S = ∑ x ∈ { 0 , 1 } d A ( x ) S = \sum_{x \in\{0,1\}^d} A(x) S = x ∈ { 0 , 1 } d ∑ A ( x )
and its total number of paths is Q = 2 d + K Q = 2^{d+K} Q = 2 d + K . The desired risk condition is
S ≤ J , J = ⌊ τ D 2 2 d ⌋ . S \le J,\qquad J = \left\lfloor\tau D^2 2^d \right\rfloor. S ≤ J , J = ⌊ τ D 2 2 d ⌋ .
Compute J J J by exact rational arithmetic. If J < 0 J < 0 J < 0 , reject; if J ≥ Q J \ge Q J ≥ Q , accept. For 0 ≤ J < Q 0 \le J < Q 0 ≤ J < Q , use a machine with one additional fair selector bit. On the first selector branch, simulate the counting machine and accept exactly when it rejects, giving Q − S Q-S Q − S accepting paths. On the second, accept exactly when the remaining d + K d+K d + K bits specify an integer less than J + 1 J+1 J + 1 , giving J + 1 J+1 J + 1 accepting paths. Of its 2 Q 2Q 2 Q paths, exactly Q − S + J + 1 Q-S+J+1 Q − S + J + 1 accept. This is strictly greater than Q Q Q precisely when S ≤ J S \le J S ≤ J . The fixed candidate can thus be checked with a PP query. Guessing the candidate proves membership in N P P P \mathrm{NP}^{\mathrm{PP}} NP PP .
Inventory and candidate witnesses for hardness. Take an instance φ ( x , z ) \varphi(x,z) φ ( x , z ) of Theorem 7.2 , with e e e existential bits and n n n random bits. Adding ignored variables preserves the answer, so assume e , n ≥ 1 e,n \ge1 e , n ≥ 1 . For a ∈ { 0 , 1 } e a \in\{0,1\}^e a ∈ { 0 , 1 } e , write
p ( a ) = 2 − n # { z ∈ { 0 , 1 } n : φ ( a , z ) = 1 } . p(a) = 2^{-n}\#\{z \in\{0,1\}^n:\varphi(a,z)=1\}. p ( a ) = 2 − n # { z ∈ { 0 , 1 } n : φ ( a , z ) = 1 } .
Supply e + 2 e+2 e + 2 occurrences of + 1 +1 + 1 , e e e occurrences of − 1 -1 − 1 , and one occurrence of each value
1 , 0 , − 1 , … , 1 − e . 1,0,-1,\ldots,1-e. 1 , 0 , − 1 , … , 1 − e .
The last group has e + 1 e+1 e + 1 occurrences; thus m = 3 e + 3 m=3e+3 m = 3 e + 3 . The three groups have separate occurrence identities even when their numerical values agree. Set the computational cap to two.
In the hardness construction, x ∈ { 0 , 1 } e x \in\{0,1\}^e x ∈ { 0 , 1 } e denotes the existential coordinates of an input; the share coordinates introduced with the target below make up the rest. For every a a a , a permitted hidden neuron can compute
h a ( x ) = ρ ( ∑ i : a i = 1 x i − ∑ i : a i = 0 x i + 1 − ∣ a ∣ ) , h_a(x)=\rho\left(\sum_{i:a_i=1}x_i-\sum_{i:a_i=0}x_i+1-|a|\right), h a ( x ) = ρ ( i : a i = 1 ∑ x i − i : a i = 0 ∑ x i + 1 − ∣ a ∣ ) ,
where ∣ a ∣ |a| ∣ a ∣ is the number of its one coordinates. Use an affine output with the single edge + 1 +1 + 1 from this neuron. The positive input edges and output edge consume at most e + 1 e+1 e + 1 occurrences from the first group. The negative input edges consume at most e e e from the second. The bias is assigned its separate occurrence in the last group; a zero bias may instead be omitted. There are no parallel edges, both computational vertices are legal, and the hidden vertex reaches the output. On Boolean inputs the preactivation is one minus the number of coordinates where x x x differs from a a a . Consequently h a ( x ) = 1 h_a(x)=1 h a ( x ) = 1 when x = a x=a x = a , and h a ( x ) = 0 h_a(x)=0 h a ( x ) = 0 otherwise.
The target. Replace every random bit z j z_j z j , and one additional fair bit u u u , by the parity of a disjoint block of m + 1 m+1 m + 1 fresh coordinates, which we call share coordinates. There are now
d = e + ( n + 1 ) ( m + 1 ) d=e+(n+1)(m+1) d = e + ( n + 1 ) ( m + 1 )
Boolean input coordinates: the e e e existential coordinates x x x and the share coordinates. In the definition of F F F and in the conditioning argument below, u u u and every z j z_j z j denote the parities of their blocks. Define the Boolean function F = u ∨ φ ( x , z ) F=u\lor\varphi(x,z) F = u ∨ φ ( x , z ) with these parities, and let the target be T = 2 F − 1 T=2^F-1 T = 2 F − 1 on the cube. The XOR of two bits is one precisely when the bits differ. Each parity has a linear-size XOR circuit, and XOR has a constant-size expression using AND and NOT. The circuit for F F F therefore has polynomial size.
This circuit compiles into a polynomial-size rational ReLU target. For Boolean gate values a , b a,b a , b , NOT is ρ ( 1 − a ) \rho(1-a) ρ ( 1 − a ) and AND is ρ ( a + b − 1 ) \rho(a+b-1) ρ ( a + b − 1 ) . Replace OR and XOR by their fixed AND/NOT expressions first, and create the corresponding hidden ReLU vertices in circuit order. If an AND gate has the same predecessor twice, use one edge of weight two instead of parallel edges. Constants are bias-only vertices when needed, and unused vertices can be removed. All resulting gate values are Boolean on the cube. The final affine output 2 F − 1 2F-1 2 F − 1 gives the prescribed signed target. Target parameters are independently supplied as part of its explicit graph and need not belong to W W W . This construction imposes no requirement on target values outside the cube.
For a fixed x x x , the parities are independent fair bits. Therefore
Pr [ F = 1 ∣ x ] = 1 + p ( x ) 2 , E [ T ∣ x ] = p ( x ) , T 2 = 1. \Pr[F=1\mid x]=\frac{1+p(x)}{2},\qquad\mathbb{E}[T\mid x]=p(x),\qquad T^2=1. Pr [ F = 1 ∣ x ] = 2 1 + p ( x ) , E [ T ∣ x ] = p ( x ) , T 2 = 1.
The last equality holds at every Boolean input.
All competing graphs in a no instance. Fix any permitted candidate N N N . Let S N S_N S N be the set of fresh share coordinates that occur on an input edge of its graph. Each distinct such coordinate requires a parameter occurrence, so ∣ S N ∣ ≤ m |S_N| \le m ∣ S N ∣ ≤ m . Including coordinates on zero-weight edges in S N S_N S N preserves this bound. Condition on all e e e original x x x -coordinates and on the share coordinates in S N S_N S N . The candidate value is determined by this conditioning. Every parity block retains at least one unconditioned coordinate. By Lemma 7.3 with k = m k=m k = m , the block parities remain independent fair bits, independently of x x x . The conditional target mean is consequently p ( x ) p(x) p ( x ) .
Every candidate value is an integer on the cube. This follows by induction because the supplied parameters and Boolean inputs are integers, integer affine operations preserve integrality, and ReLU preserves integers. In a no instance, 0 ≤ p ( x ) ≤ 1 / 2 0 \le p(x) \le1/2 0 ≤ p ( x ) ≤ 1/2 for every x x x . For every integer b b b ,
b 2 − 2 b p ( x ) ≥ 0. b^2-2bp(x) \ge0. b 2 − 2 b p ( x ) ≥ 0.
If b ≤ 0 b \le0 b ≤ 0 , both summands b 2 b^2 b 2 and − 2 b p ( x ) -2bp(x) − 2 b p ( x ) are nonnegative. If b ≥ 1 b \ge1 b ≥ 1 , the expression is at least b 2 − b = b ( b − 1 ) ≥ 0 b^2-b=b(b-1) \ge0 b 2 − b = b ( b − 1 ) ≥ 0 . Thus, on every event specified by the conditioning,
E [ ( N − T ) 2 ∣ x , ( X j ) j ∈ S N ] = 1 + N 2 − 2 N p ( x ) ≥ 1. \mathbb{E}[(N-T)^2\mid x,(X_j)_{j\in S_N}]=1+N^2-2Np(x)\ge1. E [( N − T ) 2 ∣ x , ( X j ) j ∈ S N ] = 1 + N 2 − 2 N p ( x ) ≥ 1.
Here X j X_j X j denotes coordinate j j j of the random input X ∼ U d X \sim U_d X ∼ U d . Averaging shows R T ( N ) ≥ 1 R_T(N) \ge1 R T ( N ) ≥ 1 for every permitted graph, including arbitrary sharing, skips, biases, and negative affine outputs.
A yes instance and the threshold. If p ( a ) > 1 / 2 p(a)>1/2 p ( a ) > 1/2 , use the permitted indicator network h a h_a h a . The event x = a x=a x = a has probability 2 − e 2^{-e} 2 − e , and h a 2 = h a h_a^2=h_a h a 2 = h a on the cube. It follows that
R T ( h a ) = 1 + 2 − e ( 1 − 2 p ( a ) ) . R_T(h_a)=1+2^{-e}(1-2p(a)). R T ( h a ) = 1 + 2 − e ( 1 − 2 p ( a )) .
Since p ( a ) p(a) p ( a ) is an integer multiple of 2 − n 2^{-n} 2 − n , strict majority implies p ( a ) − 1 / 2 ≥ 2 − n p(a)-1/2 \ge2^{-n} p ( a ) − 1/2 ≥ 2 − n . Hence
R T ( h a ) ≤ 1 − 2 1 − e − n < 1 − 2 − e − n < 1. R_T(h_a)\le1-2^{1-e-n}<1-2^{-e-n}<1. R T ( h a ) ≤ 1 − 2 1 − e − n < 1 − 2 − e − n < 1.
Choose τ = 1 − 2 − e − n \tau=1-2^{-e-n} τ = 1 − 2 − e − n . This rational has polynomial binary length. A source yes instance has feasible risk below τ \tau τ , while a source no instance has no feasible risk at most τ \tau τ . The dimension, target graph, inventory and threshold are all constructed in polynomial time. This proves the at-most-once hardness.
Exactly-once use. Membership uses the same graph certificate and counting test, with the additional polynomial-time check that every occurrence is assigned. For hardness retain the inventory, target, and threshold above, but set the cap to m m m . The no-instance argument covered every graph using the inventory and did not require cap two, so it remains valid.
For a yes witness a a a , omit the indicator neuron’s bias when ∣ a ∣ = 1 |a|=1 ∣ a ∣ = 1 . Let δ \delta δ be one if this bias is nonzero and zero otherwise. The two-vertex indicator then uses e + 1 + δ e+1+\delta e + 1 + δ occurrences, leaving
ℓ = m − e − 1 − δ = 2 e + 2 − δ ≥ 3 \ell=m-e-1-\delta=2e+2-\delta\ge3 ℓ = m − e − 1 − δ = 2 e + 2 − δ ≥ 3
unused occurrences. The sole zero occurrence is unused, because all indicator input and output weights are units and its present bias is nonzero.
List the unused occurrences as q 0 , q 1 , … , q ℓ − 1 q_0,q_1,\ldots,q_{\ell-1} q 0 , q 1 , … , q ℓ − 1 , with q 0 = 0 q_0=0 q 0 = 0 . Add hidden vertices
z 0 = ρ ( q 0 ) , z j = ρ ( q j z j − 1 ) ( 1 ≤ j ≤ ℓ − 2 ) , z_0=\rho(q_0),\qquad z_j=\rho(q_jz_{j-1})\quad(1\le j\le\ell-2), z 0 = ρ ( q 0 ) , z j = ρ ( q j z j − 1 ) ( 1 ≤ j ≤ ℓ − 2 ) ,
and add the term q ℓ − 1 z ℓ − 2 q_{\ell-1}z_{\ell-2} q ℓ − 1 z ℓ − 2 to the existing affine output. The first vertex has a present zero bias and every later vertex has an incoming edge. Each value is zero, by induction, regardless of the signs of the chain weights. The output and its risk are unchanged.
The chain consumes all ℓ \ell ℓ unused occurrences: one bias, ℓ − 2 \ell-2 ℓ − 2 internal edges, and one output edge. Its ℓ − 1 \ell-1 ℓ − 1 new hidden vertices give ℓ + 1 = m − e − δ ≤ m \ell+1=m-e-\delta\le m ℓ + 1 = m − e − δ ≤ m computational vertices in total. All new vertices have structural paths to the output. The added output predecessor differs from the indicator neuron, so no parallel edge is introduced. This is a legal exactly-once witness attaining the same risk below τ \tau τ . The reduction is polynomial, completing the exactly-once hardness and the theorem. □ \square □
The loss threshold in this reduction separates risks by an exponentially small amount. The theorem is an exact decision classification and gives no constant-gap approximation hardness. Sampling bounds at a fixed accuracy concern a different requirement.
The signed-label reduction used integer output values. Allowing a single half occurrence gives a finer output grid and permits ordinary binary labels.
Corollary 7.4 (Binary targets). Under either inventory rule, population squared-risk synthesis is N P P P \mathrm{NP}^{\mathrm{PP}} NP PP -complete even when the target takes values only in { 0 , 1 } \{0,1\} { 0 , 1 } on the Boolean cube and the inventory consists of integers and exactly one occurrence of 1 / 2 1/2 1/2 . Hardness holds with cap three under at-most-once use and cap m = ∣ W ∣ m=|W| m = ∣ W ∣ under exactly-once use, counting the affine output.
Proof. The upper bound follows from Theorem 7.1 . For hardness take the same existential-majority source φ ( x , z ) \varphi(x,z) φ ( x , z ) , with e , n ≥ 1 e,n\ge1 e , n ≥ 1 , and retain p ( a ) = Pr z [ φ ( a , z ) = 1 ] p(a)=\Pr_z[\varphi(a,z)=1] p ( a ) = Pr z [ φ ( a , z ) = 1 ] . Supply e + 2 e+2 e + 2 positive units, e e e negative units, the separate bias occurrences 1 , 0 , − 1 , … , 1 − e 1,0,-1,\ldots,1-e 1 , 0 , − 1 , … , 1 − e , and one occurrence of 1 / 2 1/2 1/2 . There are m = 3 e + 4 m=3e+4 m = 3 e + 4 occurrences in total. Replace every random source bit and the additional bit u u u by parity blocks of size m + 1 m+1 m + 1 , as before. Use the binary target
Y = F = u ∨ φ ( x , z ) , q ( x ) = E [ Y ∣ x ] = 1 + p ( x ) 2 . Y=F=u\vee\varphi(x,z),\qquad q(x)=\mathbb{E}[Y\mid x]=\frac{1+p(x)}{2}. Y = F = u ∨ φ ( x , z ) , q ( x ) = E [ Y ∣ x ] = 2 1 + p ( x ) . The preceding Boolean-gate compilation supplies this target as a polynomial-size rational ReLU network with an affine output, without the final signed transformation.
Every candidate value on the cube belongs to 1 2 Z \frac{1}{2}\mathbb{Z} 2 1 Z , the set of integer multiples of one half, whose elements we call half-integers. Indeed, freeze its actual activation pattern and use the path expansion from the preceding proof. Every path product contains only integers and at most one factor 1 / 2 1/2 1/2 , since there is just one such occurrence and a path cannot reuse it. Every product is a half-integer, and so is their sum. This argument permits sharing, reconverging paths, and any placement or nonuse of the half occurrence.
Fix any candidate and condition on all existential coordinates and its observed fresh coordinates. At most m m m fresh coordinates are observed, so Lemma 7.3 leaves the conditional target mean equal to q ( x ) q(x) q ( x ) . The candidate value is fixed by the conditioning and is a half-integer. In a source no instance, 1 / 2 ≤ q ( x ) ≤ 3 / 4 1/2\le q(x)\le3/4 1/2 ≤ q ( x ) ≤ 3/4 . For every half-integer b b b , the conditional squared loss minus 1 / 4 1/4 1/4 is
b 2 + ( 1 − 2 b ) q ( x ) − 1 4 . b^2+(1-2b)q(x)-\frac{1}{4}. b 2 + ( 1 − 2 b ) q ( x ) − 4 1 . There is no half-integer strictly between 1 / 2 1/2 1/2 and one. If b ≤ 1 / 2 b\le1/2 b ≤ 1/2 , the coefficient 1 − 2 b 1-2b 1 − 2 b is nonnegative, and the expression is at least
b 2 + 1 − 2 b 2 − 1 4 = ( b − 1 / 2 ) 2 ≥ 0. b^2+\frac{1-2b}{2}-\frac{1}{4}=(b-1/2)^2\ge0. b 2 + 2 1 − 2 b − 4 1 = ( b − 1/2 ) 2 ≥ 0. If b ≥ 1 b\ge1 b ≥ 1 , that coefficient is nonpositive, and the expression is at least
b 2 + 3 4 ( 1 − 2 b ) − 1 4 = ( b − 1 ) ( b − 1 / 2 ) ≥ 0. b^2+\frac{3}{4}(1-2b)-\frac{1}{4}=(b-1)(b-1/2)\ge0. b 2 + 4 3 ( 1 − 2 b ) − 4 1 = ( b − 1 ) ( b − 1/2 ) ≥ 0. Averaging proves risk at least 1 / 4 1/4 1/4 for every legal candidate, independently of its cap.
For a source yes witness a a a , use the earlier indicator g = h a g=h_a g = h a , then a second hidden vertex h = ρ ( 1 + g ) h=\rho(1+g) h = ρ ( 1 + g ) , and affine output N = h / 2 N=h/2 N = h /2 . This has three computational vertices and equals ( 1 + h a ) / 2 (1+h_a)/2 ( 1 + h a ) /2 , since h a ≥ 0 h_a\ge0 h a ≥ 0 . The positive input edges, the edge g → h g\to h g → h , and the bias of h h h use ∣ a ∣ + 2 ≤ e + 2 |a|+2\le e+2 ∣ a ∣ + 2 ≤ e + 2 positive unit occurrences. The negative input edges use at most e e e negative units; the bias of g g g uses its separate 1 − ∣ a ∣ 1-|a| 1 − ∣ a ∣ occurrence and is omitted if zero. The half occurrence is the output edge. Every vertex is legal and reaches the output, and there are no parallel edges.
Outside x = a x=a x = a , prediction 1 / 2 1/2 1/2 has squared error exactly 1 / 4 1/4 1/4 for either binary label. At x = a x=a x = a , prediction one has conditional mean squared error 1 − q ( a ) 1-q(a) 1 − q ( a ) . Changing the prediction there therefore changes its conditional risk by
1 − q ( a ) − 1 4 = 3 4 − q ( a ) = 1 − 2 p ( a ) 4 . 1-q(a)-\frac{1}{4}=\frac{3}{4}-q(a)=\frac{1-2p(a)}{4}. 1 − q ( a ) − 4 1 = 4 3 − q ( a ) = 4 1 − 2 p ( a ) . Consequently
R Y ( N ) = 1 4 + 2 − e 1 − 2 p ( a ) 4 ≤ 1 4 − 2 − e − n − 1 . R_Y(N)=\frac{1}{4}+2^{-e}\frac{1-2p(a)}{4}\le\frac{1}{4}-2^{-e-n-1}. R Y ( N ) = 4 1 + 2 − e 4 1 − 2 p ( a ) ≤ 4 1 − 2 − e − n − 1 . The inequality uses p ( a ) − 1 / 2 ≥ 2 − n p(a)-1/2\ge2^{-n} p ( a ) − 1/2 ≥ 2 − n , exactly as before. Choose the polynomial-length rational threshold τ = 1 / 4 − 2 − e − n − 2 \tau=1/4-2^{-e-n-2} τ = 1/4 − 2 − e − n − 2 . The yes risk is strictly below it, and every no risk is at least 1 / 4 1/4 1/4 . This proves at-most-once hardness at cap three.
For exactly-once use, set the cap to m m m . Let δ \delta δ indicate whether the nonzero bias of g g g is present. The displayed witness consumes e + 3 + δ e+3+\delta e + 3 + δ occurrences, leaving
ℓ = m − e − 3 − δ = 2 e + 1 − δ ≥ 2 \ell= m-e-3-\delta= 2e+1-\delta\ge2 ℓ = m − e − 3 − δ = 2 e + 1 − δ ≥ 2
unused, including the sole zero. The half occurrence is already assigned to the output edge. Use the zero-chain construction in the proof of Theorem 7.1 on these ℓ \ell ℓ leftovers. It consumes them all and adds ℓ − 1 \ell-1 ℓ − 1 hidden vertices without changing the output. When ℓ = 2 \ell=2 ℓ = 2 , it consists of a zero-bias hidden vertex and its edge to the output; no internal chain edge is needed. The resulting computational count is 3 + ℓ − 1 = ℓ + 2 = m − e − 1 − δ ≤ m 3+\ell-1=\ell+2=m-e-1-\delta\le m 3 + ℓ − 1 = ℓ + 2 = m − e − 1 − δ ≤ m . The chain is legal and has a distinct final output predecessor, as in that proof. The cap-independent no-instance inequality still excludes every exactly-once candidate. This proves the second hardness assertion. □
The corollary does not assert binary-target hardness for an entirely integer inventory, or cap three under exactly-once use. It retains an exponentially small decision gap and does not establish fixed-gap approximation hardness.
Exact Population Risk under Continuous Uniform Inputs
The distribution changes the arithmetic of exact risk. Let T , N : R d → R T,N:\mathbb{R}^{d}\to\mathbb{R} T , N : R d → R be rational ReLU networks with affine outputs, and define
R T c o n t ( N ) = ∫ [ 0 , 1 ] d ( N ( x ) − T ( x ) ) 2 d x . R^{\mathrm{cont}}_{T}(N)=\int_{[0,1]^{d}}(N(x)-T(x))^{2}\,dx. R T cont ( N ) = ∫ [ 0 , 1 ] d ( N ( x ) − T ( x ) ) 2 d x .
The cube has volume one, so this is squared population risk for continuous uniform inputs. For analysis, the same notation applies to measurable real-valued functions whose squared difference has finite integral. In the synthesis problems, the supplied target and candidate descriptions remain networks. In fixed-network comparison both networks are supplied. In synthesis only T T T is supplied, and N N N must obey the inventory and the computational cap. The threshold τ \tau τ is a binary rational in both problems. Fixed-network comparison asks whether the risk of the supplied N N N is at most τ \tau τ , and synthesis asks whether some legal candidate has risk at most τ \tau τ .
The appropriate comparison class permits different denominators on different counting paths. Following de Campos, Stamoulis, and Weyland [10 Definition 9] , write P P Q ( p t ) \mathrm{PP}^{(pt)}_{\mathbb{Q}} PP Q ( pt ) for the languages expressible as
F ( a ) = ∑ w ∈ { 0 , 1 } p ( ∣ a ∣ ) u ( a , w ) v ( a , w ) ≥ t ( a ) . F(a)=\sum_{w\in\{0,1\}^{p(|a|)}}\frac{u(a,w)}{v(a,w)}\ge t(a). F ( a ) = w ∈ { 0 , 1 } p ( ∣ a ∣ ) ∑ v ( a , w ) u ( a , w ) ≥ t ( a ) .
Here a a a is the ordinary binary input, p p p is a polynomial, u u u is a signed integer, v v v is a positive integer, and the weights and rational threshold t t t are computed exactly in polynomial time. Their individual binary lengths are polynomially bounded. The definition requires the weights to be computed exactly, not merely approximated. In particular, it does not include the undecidable comparisons possible in the broader approximation-defined model of that source. Taking integer counting weights shows that the class contains PP.
We use the following recent arithmetic result with its version stated explicitly. The notation BPP means polynomial-time computation with independent fair random bits and error at most 1 / 3 1/3 1/3 on every input. Oracle superscripts have the same meaning as for N P P P \mathrm{NP}^{\mathrm{PP}} NP PP . A dyadic rational is an integer divided by a power of two.
Theorem 8.1 (Meiburg [29 Theorem 12] ).
For exactly polynomial-time computable rational weights,
P P Q ( p t ) ⊆ B P P P P ∩ N P P P . \mathrm{PP}^{(pt)}_{\mathbb{Q}}\subseteq\mathrm{BPP}^{\mathrm{PP}}\cap\mathrm{NP}^{\mathrm{PP}}. PP Q ( pt ) ⊆ BPP PP ∩ NP PP .
This is an import from the cited preprint, rather than a new general arithmetic bound. Our contribution here is the exact ReLU integration characterization, including a one-dimensional realization of arbitrary rational weighted sums. Previous work already studies ReLU integration hardness, including fixed dimension [14 Proposition 2.7 and Theorem 3.2] , and probabilistic neural verification through weighted model integration [30 ] .
Theorem 8.2 (Continuous exact risk; restatement of Theorem 1.4). Comparing the continuous squared risk of two supplied networks with a rational threshold is P P Q ( p t ) \mathrm{PP}^{(\mathrm{pt})}_{\mathbb{Q}} PP Q ( pt ) -complete under deterministic polynomial-time many-one reductions. Hardness holds with d = 1 d=1 d = 1 , N = 0 N=0 N = 0 , dyadic target parameters, and 0 ≤ T ≤ 1 0 \le T \le1 0 ≤ T ≤ 1 . Under either inventory rule, continuous-risk synthesis is P P Q ( p t ) \mathrm{PP}^{(\mathrm{pt})}_{\mathbb{Q}} PP Q ( pt ) -hard even with W = ( 0 ) W=(0) W = ( 0 ) and cap one, and belongs to
N P P P Q ( p t ) ∩ B P P N P P P ⊆ P S P A C E . \mathrm{NP}^{\mathrm{PP}^{(\mathrm{pt})}_{\mathbb{Q}}} \cap\mathrm{BPP}^{\mathrm{NP}^{\mathrm{PP}}} \subseteq\mathrm{PSPACE}. NP PP Q ( pt ) ∩ BPP NP PP ⊆ PSPACE . The randomized synthesis upper bound uses Theorem 8.1.
We first establish the geometric encoding needed for membership. Recall the affine hyperplanes, halfspaces and open balls defined in § 2 . A set is closed when the points outside it form an open set. A set is convex when it contains every segment between two of its points: λ x + ( 1 − λ ) y \lambda x+(1-\lambda)y λ x + ( 1 − λ ) y belongs to the set whenever x , y x,y x , y do and 0 ≤ λ ≤ 1 0 \le\lambda\le1 0 ≤ λ ≤ 1 . Affine halfspaces are convex by linearity, and intersections preserve this property. A polytope is a bounded intersection of finitely many closed affine halfspaces.
A hyperplane supports a polytope when it meets the polytope and the polytope lies wholly in one of its two closed halfspaces. A face is the intersection of the polytope with a supporting hyperplane, and the whole polytope is also a face. The polytope and its faces are therefore convex.
An inequality is tight at a point when equality holds. The affine hull of a nonempty set consists of all finite affine combinations of its points. The dimension of the affine hull is the dimension of the linear span of the differences of the set’s points. A set’s boundary consists of the points whose every open ball contains both a point in the set and a point outside it. The relative boundary is defined in the same way, with balls taken within the set’s affine hull and outside points taken from that hull. The relative interior consists of the points of the set that have a ball within the affine hull contained in the set. A polytope is full-dimensional in R d \mathbb{R}^{d} R d when it contains an open ball in R d \mathbb{R}^{d} R d . A vertex is a zero-dimensional face, identified with its unique point. A facet of a face is a subface of dimension one smaller than that face. A face flag is a strictly nested sequence of faces, complete when it contains one face of every dimension.
Points v 0 , … , v k v_0,\ldots,v_k v 0 , … , v k are affinely independent when v 1 − v 0 , … , v k − v 0 v_1-v_0,\ldots,v_k-v_0 v 1 − v 0 , … , v k − v 0 are linearly independent. Their simplex is the set of convex combinations ∑ i = 0 k λ i v i \sum_{i=0}^{k}\lambda_i v_i ∑ i = 0 k λ i v i , where λ i ≥ 0 \lambda_i \ge0 λ i ≥ 0 and ∑ i λ i = 1 \sum_i \lambda_i=1 ∑ i λ i = 1 ; these weights are its barycentric coordinates. A triangulation decomposes a polytope into simplices such that any two distinct simplices intersect, if at all, in a common face. Lexicographic tie breaking compares the first coordinate at which two vectors differ.
A bounded set has measure zero, or volume zero, if for every ϵ > 0 \epsilon>0 ϵ > 0 it can be covered by boxes whose total volume is less than ϵ \epsilon ϵ . A finite union of such sets has the same property, by dividing ϵ \epsilon ϵ among their covers.
Each bounded part of a proper affine hyperplane has volume zero. To see this in a bounded cube, choose one nonzero normal coefficient and solve for that coordinate as an affine function of the others. Partition the other coordinates into boxes of side 1 / n 1/n 1/ n . There are O ( n d − 1 ) O(n^{d-1}) O ( n d − 1 ) such boxes, and the affine graph above each has height variation at most C / n C/n C / n , where C C C is the sum of the absolute remaining coefficient ratios. Cover that piece of the graph by a box with that base and height at most ( C + 2 ) / n (C+2)/n ( C + 2 ) / n . The sum of these box volumes is O ( 1 / n ) O(1/n) O ( 1/ n ) , and hence can be made smaller than any positive ϵ \epsilon ϵ . This also covers d = 1 d=1 d = 1 , with an empty base and a shrinking interval.
A bounded lower-dimensional polytope lies in a proper affine hyperplane and consequently has volume zero. For a full-dimensional polytope, a boundary point must make some nonconstant defining inequality tight, since finitely many strictly satisfied inequalities admit a small ball. Its boundary is therefore contained in finitely many proper hyperplanes and has volume zero. Changing a bounded integrand on these sets does not affect its integral: the absolute change is at most its bound times the volume of a covering set. For polynomials restricted to bounded polytopes, integration by successive one-dimensional slices agrees with volume integration. Their boundaries have volume zero, and polynomial oscillations on fine boxes tend uniformly to zero, so upper and lower box sums give the same limit. These are the integration conventions used below.
Lemma 8.3 (Polynomial-length rational integration leaves). From two supplied rational ReLU networks one can construct a polynomial-time algorithm whose input is a binary word of one fixed length, polynomial in the size of the two networks, and whose output is a nonnegative rational of polynomial binary
length, such that the sum of these outputs over all words of that length equals the continuous squared risk of the two networks.
Proof. Let L ≥ 2 L \ge2 L ≥ 2 bound the full input encoding length, and let p p p be the total number of hidden vertices of both networks. Freeze one of the 2 p 2^p 2 p activation patterns: replace each active ReLU by the identity and each inactive ReLU by zero. Every computed quantity becomes a rational affine form. Recursively substituting these affine computations expresses each coefficient as a sum of input-to-vertex or bias-to-vertex path products, as in the earlier exact-evaluation argument of Theorem 7.1 .
Let D D D be the product of all positive position denominators in both supplied networks. Each directed path uses a position at most once, so multiplication by D D D turns its term into an integer product of used numerators and unused denominators. The full encoding length L L L bounds the number of parameter positions and the sum of their numerical bit lengths; in particular each such scaled product has absolute value at most 2 2 L 2^{2L} 2 2 L .
An acyclic path is determined by its set of positions, so the two networks together supply at most 2 L + 1 2^{L+1} 2 L + 1 terms for any coefficient of their difference. Thus each scaled numerator has at most 3 L + 3 3L+3 3 L + 3 bits, including its sign, and the common positive denominator has at most L + 1 L+1 L + 1 bits. These bounds apply to frozen preactivations as well as the output difference, and are smaller than b 0 = 4 ( L + 1 ) 3 b_0 = 4(L+1)^3 b 0 = 4 ( L + 1 ) 3 for L ≥ 2 L \ge2 L ≥ 2 .
Intersect the frozen preactivation sign inequalities with the cube. Write the resulting polytope as P = { x : a i ⋅ x ≤ b i , 1 ≤ i ≤ M } P = \{x : a_i \cdot x \le b_i,\ 1 \le i \le M\} P = { x : a i ⋅ x ≤ b i , 1 ≤ i ≤ M } , with a i ∈ Q d a_i \in\mathbb{Q}^d a i ∈ Q d , b i ∈ Q b_i \in\mathbb{Q} b i ∈ Q , and M M M polynomial. Discard patterns for which P P P is not full-dimensional. This test is exact rational linear programming: reject any false constant row 0 ≤ b i 0 \le b_i 0 ≤ b i with b i < 0 b_i < 0 b i < 0 , ignore true constant rows in this test, and maximize a common slack δ ∈ [ 0 , 1 ] \delta\in[0,1] δ ∈ [ 0 , 1 ] in all remaining inequalities. The maximum is positive precisely when an open ball lies in P P P . All coefficients have polynomial binary length and δ ≤ 1 \delta\le1 δ ≤ 1 bounds the objective. Apply Theorem 2.3 first to reject infeasibility and otherwise to compute the maximum exactly, including when the feasible region is lower-dimensional. Also discard any pattern declaring active a gate whose frozen preactivation is the zero affine form.
The retained polytopes cover the cube up to a set of measure zero and have disjoint interiors. To see coverage, exclude the cube boundary and all zero hyperplanes of nonconstant affine forms arising in any frozen pattern. There are finitely many such hyperplanes. At any remaining point, choose the active state for each gate whose actual preactivation is positive and the inactive state otherwise. Nonzero frozen preactivations are then strictly signed in a neighborhood, while identically zero ones are inactive, so the resulting polytope is full-dimensional and retained. For disjointness, suppose two retained patterns overlap on an open set. At their first differing gate the earlier computations agree, so its preactivation is zero on that open set. The frozen affine form is therefore identically zero, contradicting retention of the pattern that declares it active.
On every activation polytope, including those discarded as lower-dimensional, the frozen computation gives the actual network values in topological order. Indeed, each labeled ReLU substitution agrees with max { 0 , t } \max\{0,t\} max { 0 , t } under its sign constraint, and either substitution gives zero when t = 0 t = 0 t = 0 . Each frozen affine output difference has d + 1 d+1 d + 1 coefficients of absolute value at most 2 b 0 2^{b_0} 2 b 0 . Thus ∣ N ( x ) − T ( x ) ∣ ≤ ( d + 1 ) 2 b 0 |N(x) - T(x)| \le(d+1)2^{b_0} ∣ N ( x ) − T ( x ) ∣ ≤ ( d + 1 ) 2 b 0 throughout the cube, and its square is bounded.
This squared difference is also measurable. For t ≥ 0 t \ge0 t ≥ 0 , its superlevel set is a finite union of activation regions intersected with either a ( x ) > t a(x) > \sqrt{t} a ( x ) > t or a ( x ) < − t a(x) < -\sqrt{t} a ( x ) < − t , where a a a is the corresponding frozen affine output difference. For t < 0 t < 0 t < 0 , the superlevel set is the whole cube. These sets have boundaries in finitely many proper affine hyperplanes, so the box covers above make the upper and lower sums of their indicators agree; in particular, they are measurable for volume. The boundedness and measurability just established show that lower-dimensional pieces and boundaries do not affect integration.
On a retained P P P , let g g g be the frozen affine output difference. We enumerate a rational triangulation without storing it. Retain every original inequality row, including constant rows, for this enumeration. A nonempty face is uniquely described by its full tight set J J J : the rows that are equalities throughout the face. We justify this description and the facet coverage used below.
Every nonempty convex subset of R d \mathbb{R}^d R d has a relative-interior point. Choose one point v 0 v_0 v 0 in the set, then successively add a point outside the current affine span while possible. Each addition increases the rank of the differences from v 0 v_0 v 0 , so at most d d d points are added. For the resulting family v 0 , … , v k v_0,\ldots,v_k v 0 , … , v k , these difference columns are independent and the affine span is the set’s affine hull. The simplex of these points lies in the set by convexity.
The equal-weight mean ( v 0 + ⋯ + v k ) / ( k + 1 ) (v_0+\cdots+v_k)/(k+1) ( v 0 + ⋯ + v k ) / ( k + 1 ) has all barycentric coordinates equal to 1 / ( k + 1 ) 1/(k+1) 1/ ( k + 1 ) . For k ≥ 1 k \ge1 k ≥ 1 , select k k k coordinate rows making the k k k difference columns a nonsingular square matrix, as proved in Section 2. Solving this system expresses λ 1 , … , λ k \lambda_1,\ldots,\lambda_k λ 1 , … , λ k as affine functions of the point’s coordinates within this affine hull; λ 0 = 1 − ∑ i = 1 k λ i \lambda_0=1-\sum_{i=1}^{k}\lambda_i λ 0 = 1 − ∑ i = 1 k λ i is affine too. Each coordinate changes by at most a fixed sum of absolute coefficients times the maximum coordinate change of the point. Thus all these finitely many positive coordinates remain positive on a sufficiently small relative ball about the mean, placing that ball in the simplex and in the set. For k = 0 k=0 k = 0 , the set is the single point v 0 v_0 v 0 , which is its own relative interior.
Choose such a point z z z for a nonempty face F F F , and let I I I be the original rows tight at z z z . If a row tight at z z z were slack at some y ∈ F y \in F y ∈ F , the point z + ε ( z − y ) z+\varepsilon(z-y) z + ε ( z − y ) , for sufficiently small ε > 0 \varepsilon>0 ε > 0 , would still lie in F F F , because z z z is a relative-interior point, but would violate that row. Hence every row in I I I is tight throughout F F F .
Conversely, let x ∈ P x \in P x ∈ P satisfy equality in all rows of I I I . For sufficiently small ε > 0 \varepsilon>0 ε > 0 , the point y = z + ε ( z − x ) y=z+\varepsilon(z-x) y = z + ε ( z − x ) also lies in P P P : rows in I I I stay equalities, and the other finitely many rows have positive slack at z z z . Write F = { u ∈ P : c ⋅ u = α } F=\{u \in P:c\cdot u=\alpha\} F = { u ∈ P : c ⋅ u = α } , with c ⋅ u ≤ α c\cdot u\le\alpha c ⋅ u ≤ α on P P P ; for F = P F=P F = P , take c = 0 c=0 c = 0 , α = 0 \alpha=0 α = 0 . Since z z z is a convex combination of x x x and y y y with both coefficients positive, equality c ⋅ z = α c\cdot z=\alpha c ⋅ z = α forces c ⋅ x = c ⋅ y = α c\cdot x=c\cdot y=\alpha c ⋅ x = c ⋅ y = α . Thus x ∈ F x \in F x ∈ F , proving that the rows in I I I describe exactly F F F and are its full tight set. In the other direction, any nonempty intersection obtained by making selected original rows tight is a face: equality in the sum of those valid inequalities forces equality in each selected row. The empty selection gives P P P .
We also need that each relative boundary point of a positive-dimensional face lies in a facet. Work within that face’s affine hull, and remove redundant inequalities, meaning rows whose removal leaves the set unchanged. Every remaining row is nonconstant in this affine hull. For one such row, nonredundancy supplies a point satisfying all the other rows but violating that row. Join it to a relative-interior point of the face, where all remaining rows are strict. Their segment meets the chosen row’s equality hyperplane before its violating endpoint, with every other row still strict. A small neighborhood of this meeting point within that hyperplane still satisfies the other rows. The row therefore defines a face of dimension one smaller, a facet.
Any point at which every remaining row is strict has a relative open ball inside the face. Thus every relative boundary point is tight in a remaining row and belongs to its facet. These facets cut faces of P P P by the preceding tight-row argument, so they occur among the enumerated faces. This geometric justification does not change the enumeration’s original row set.
For a proposed subset J ⊆ { 1 , … , M } J\subseteq\{1,\ldots,M\} J ⊆ { 1 , … , M } , solve the following linear program in the variables ( x , δ ) (x,\delta) ( x , δ ) , which we call the center program of J J J :
maximize δ , 0 ≤ δ ≤ 1 , x ∈ P , a j ⋅ x = b j ( j ∈ J ) , a i ⋅ x ≤ b i − δ ( i ∉ J ) . \begin{aligned}
\text{maximize}\quad& \delta,\qquad0\le\delta\le1,\qquad x\in P,\\
& a_j\cdot x=b_j \qquad(j\in J),\\
& a_i\cdot x\le b_i-\delta\qquad(i\notin J).
\end{aligned} maximize δ , 0 ≤ δ ≤ 1 , x ∈ P , a j ⋅ x = b j ( j ∈ J ) , a i ⋅ x ≤ b i − δ ( i ∈ / J ) .
A positive optimum occurs exactly when J J J is the full tight set of a nonempty face. Indeed, such a point is in that face and strictly satisfies all other rows; conversely a relative-interior point has a positive minimum of those finitely many slacks. If there are no other rows, the independent bound δ ≤ 1 \delta\le1 δ ≤ 1 gives the same test.
For each J J J with positive optimum, choose one optimal vertex of this bounded program, with lexicographic tie breaking, and denote its x x x -coordinate by z J z_J z J . This choice can be made with polynomially many exact rational linear programs. First fix δ \delta δ to its computed maximum, then minimize x 1 x_1 x 1 , fix that value, and continue through x d x_d x d . All these objectives are bounded because x ∈ [ 0 , 1 ] d x\in[0,1]^d x ∈ [ 0 , 1 ] d and 0 ≤ δ ≤ 1 0\le\delta\le1 0 ≤ δ ≤ 1 .
The final point is unique. If it lay strictly between two different feasible points of the original program, optimality would force both endpoints to have its δ \delta δ -value, then its x 1 x_1 x 1 -value, and successively every coordinate, a contradiction. Consider the constraints of the center program of J J J , without the coordinate-fixing constraints, that are tight at this point. They contain a full independent square subsystem in the variables ( x , δ ) (x,\delta) ( x , δ ) . Otherwise the coefficient vectors of these tight constraints span a proper subspace of R d + 1 \mathbb{R}^{d+1} R d + 1 , so some nonzero direction y ∈ R d + 1 y\in\mathbb{R}^{d+1} y ∈ R d + 1 is orthogonal to all of them. Moving from the final point by ε y \varepsilon y ε y or − ε y -\varepsilon y − ε y keeps every tight constraint tight, and for sufficiently small ε > 0 \varepsilon> 0 ε > 0 it keeps every other constraint of the center program satisfied, since those finitely many constraints have positive slack at the final point. The final point would then be the midpoint of two different feasible points of the center program, which the preceding paragraph excludes. Those independent tight equations expose a single point, so the final point is a vertex, in the sense defined earlier in this section, of the feasible set of the center program in the variables ( x , δ ) (x,\delta) ( x , δ ) .
Cramer’s rule using these rows of the center program gives a polynomial bit bound for every coordinate. All values fixed in the successive programs are coordinates of this final point, so their encodings have the same uniform polynomial bound. There are only d + 1 d+1 d + 1 optimization calls. The point z J z_J z J is a relative-interior point of the face and has polynomial-bit coordinates by Cramer’s rule. The face dimension is d − rank { a j : j ∈ J } d-\operatorname{rank}\{a_j:j\in J\} d − rank { a j : j ∈ J } , since all other inequalities are strict there. Thus all tests and this choice of point are polynomial-time exact computations.
For a nonempty face F F F , let J ( F ) J(F) J ( F ) be its full tight set of original inequality rows and write z F = z J ( F ) z_F=z_{J(F)} z F = z J ( F ) . Enumerate complete face flags F 0 ⊊ ⋯ ⊊ F d = P F_0\subsetneq\cdots\subsetneq F_d=P F 0 ⊊ ⋯ ⊊ F d = P , where dim F i = i \dim F_i=i dim F i = i , by their full tight sets. The full tight-row description gives F i ⊆ F i + 1 F_i\subseteq F_{i+1} F i ⊆ F i + 1 exactly when J ( F i + 1 ) ⊆ J ( F i ) J(F_{i+1})\subseteq J(F_i) J ( F i + 1 ) ⊆ J ( F i ) : inclusion of faces forces every row tight on the larger face to be tight on the smaller, and the reverse row inclusion forces every point of the smaller face into the larger. Test these row-set inclusions and the prescribed dimensions in polynomial time; the distinct dimensions ensure strict nesting. A face of dimension d d d is P P P , because every proper face lies in a proper supporting hyperplane.
For each flag form the simplex with vertices z F 0 , … , z F d z_{F_0},\ldots,z_{F_d} z F 0 , … , z F d . These points are affinely independent. For each i ≥ 1 i\geq1 i ≥ 1 , the faces F i − 1 ⊊ F i F_{i-1}\subsetneq F_i F i − 1 ⊊ F i are distinct, so their full tight sets are distinct, and the row-set inclusion criterion above gives J ( F i ) ⊊ J ( F i − 1 ) J(F_i)\subsetneq J(F_{i-1}) J ( F i ) ⊊ J ( F i − 1 ) . Choose a row j ∈ J ( F i − 1 ) ∖ J ( F i ) j\in J(F_{i-1})\setminus J(F_i) j ∈ J ( F i − 1 ) ∖ J ( F i ) . That row is an equality throughout F i − 1 F_{i-1} F i − 1 , hence at every affine combination of its points, so the affine hull of F i − 1 F_{i-1} F i − 1 lies in { x : a j ⋅ x = b j } \{x:a_j\cdot x=b_j\} { x : a j ⋅ x = b j } . The point z F i z_{F_i} z F i is feasible for the center program of J ( F i ) J(F_i) J ( F i ) with a positive value of δ \delta δ , and j ∉ J ( F i ) j\notin J(F_i) j ∈ / J ( F i ) , so a j ⋅ z F i ≤ b j − δ < b j a_j\cdot z_{F_i}\leq b_j-\delta<b_j a j ⋅ z F i ≤ b j − δ < b j . Thus z F i z_{F_i} z F i lies outside the affine hull of F i − 1 F_{i-1} F i − 1 . That hull contains the preceding points z F 0 , … , z F i − 1 z_{F_0},\ldots,z_{F_{i-1}} z F 0 , … , z F i − 1 , because each z F l z_{F_l} z F l with l < i l<i l < i lies in F l ⊆ F i − 1 F_l\subseteq F_{i-1} F l ⊆ F i − 1 . Suppose z F 0 , … , z F i − 1 z_{F_0},\ldots,z_{F_{i-1}} z F 0 , … , z F i − 1 are affinely independent, and take a vanishing linear combination of the differences z F l − z F 0 z_{F_l}-z_{F_0} z F l − z F 0 , 1 ≤ l ≤ i 1\leq l\leq i 1 ≤ l ≤ i . A nonzero coefficient on z F i − z F 0 z_{F_i}-z_{F_0} z F i − z F 0 would place z F i z_{F_i} z F i in the affine hull of the preceding points, so that coefficient is zero, and the remaining coefficients vanish by the assumed independence. Induction on i i i , starting from the single point z F 0 z_{F_0} z F 0 , proves the claim.
These simplices triangulate P P P up to their boundaries: they cover P P P and have pairwise disjoint interiors. We prove this by induction on the dimension k k k of a nonempty face F F F of P P P , with the following assertion. The simplices with vertices z F 0 , … , z F k z_{F_0},\ldots,z_{F_k} z F 0 , … , z F k , over all complete face flags F 0 ⊂ ⋯ ⊂ F k = F F_0\subset\cdots\subset F_k=F F 0 ⊂ ⋯ ⊂ F k = F with dim F i = i \dim F_i=i dim F i = i , cover F F F and have pairwise disjoint relative interiors in the affine hull of F F F . The case F = P F=P F = P is the claim. For the induction step, k ≥ 1 k\geq1 k ≥ 1 , the induction hypothesis triangulates each facet of F F F in this way, using the same selected points on shared subfaces. Let c = z F c=z_F c = z F be the selected relative-interior point of F F F . For a facet simplex S S S , its convex hull with c c c is { ( 1 − t ) y + t c : y ∈ S , 0 ≤ t ≤ 1 } \{(1-t)y+tc:y\in S,0\leq t\leq1\} {( 1 − t ) y + t c : y ∈ S , 0 ≤ t ≤ 1 } . Coning S S S to c c c means taking this set. For a positive-dimensional face, a ray from c c c in its affine span is a set { c + t v : t ≥ 0 } \{c+tv:t\geq0\} { c + t v : t ≥ 0 } , where v ≠ 0 v\neq0 v = 0 is a direction parallel to that span.
The face is closed, bounded and convex, and c c c is in its relative interior, so its intersection with this ray is a closed segment from c c c to a unique boundary endpoint. The segment has positive finite length, and an interior endpoint would permit extension. Every point strictly between c c c and the endpoint is in the relative interior: a small ball about c c c inside the face scales to a small ball about that point under convex combination with the endpoint. Thus only the endpoint is on the boundary within the face’s affine span.
Each point of the face other than c c c lies on one such segment. By the facet coverage just proved, the facet simplices cover these boundary endpoints and have disjoint relative interiors, consistently on shared subfaces. Their convex hulls with c c c therefore cover the face and have disjoint relative interiors in the face’s affine span; two simplex relative interiors could overlap only if the corresponding ray endpoint lay in both facet-simplex relative interiors. This proves the assertion in every dimension, starting with a single point.
We use affine changes of variables only on bounded polytopes with polynomial integrands. A coordinate swap preserves the integral after permuting the variables. Adding a multiple of one coordinate to another translates the corresponding one-dimensional slices and also preserves it. Scaling one coordinate by a nonzero scalar multiplies slice integrals by the absolute value of that scalar. Translation has factor one. Every invertible linear matrix reduces to the identity by these elementary row operations. The determinant row laws proved in Section 2 show that their accumulated volume factor is the absolute determinant of that matrix.
For an invertible d × d d \times d d × d matrix A A A , a vector b b b , a bounded polytope Q Q Q and a polynomial h h h , define the affine image A Q + b = { A y + b : y ∈ Q } AQ+b = \{Ay+b : y \in Q\} A Q + b = { A y + b : y ∈ Q } . The preceding coordinate substitutions give
∫ A Q + b h ( x ) d x = ∣ det A ∣ ∫ Q h ( A y + b ) d y . \int_{AQ+b} h(x)\,\mathrm{d}x = \lvert\det A\rvert\int_Q h(Ay+b)\,\mathrm{d}y. ∫ A Q + b h ( x ) d x = ∣ det A ∣ ∫ Q h ( A y + b ) d y .
The standard simplex is Δ d = { ( λ 1 , … , λ d ) : λ i ≥ 0 , ∑ i = 1 d λ i ≤ 1 } \Delta_d = \{(\lambda_1,\ldots,\lambda_d) : \lambda_i \ge0,\sum_{i=1}^d \lambda_i \le1\} Δ d = {( λ 1 , … , λ d ) : λ i ≥ 0 , ∑ i = 1 d λ i ≤ 1 } , with λ 0 = 1 − ∑ i = 1 d λ i \lambda_0 = 1-\sum_{i=1}^d \lambda_i λ 0 = 1 − ∑ i = 1 d λ i . The slice and box-sum conventions above justify composing these substitutions. For a simplex, use A = ( v 1 − v 0 , … , v d − v 0 ) A=(v_1-v_0,\ldots,v_d-v_0) A = ( v 1 − v 0 , … , v d − v 0 ) , b = v 0 b=v_0 b = v 0 , and the standard simplex as Q Q Q . Its columns are independent by the simplex hypothesis. For d = 0 d=0 d = 0 , the empty determinant and the point-volume convention are both one.
For simplex vertices v 0 , … , v d v_0,\ldots,v_d v 0 , … , v d , put g i = g ( v i ) g_i=g(v_i) g i = g ( v i ) , where g g g is the frozen affine output difference on the retained polytope. The integral of g 2 g^2 g 2 over the simplex is
∣ det ( v 1 − v 0 , … , v d − v 0 ) ∣ d ! ( ∑ i = 0 d g i ) 2 + ∑ i = 0 d g i 2 ( d + 1 ) ( d + 2 ) . \frac{\lvert\det(v_1-v_0,\ldots,v_d-v_0)\rvert}{d!}\frac{\left(\sum_{i=0}^d g_i\right)^2+\sum_{i=0}^d g_i^2}{(d+1)(d+2)}. d ! ∣ det ( v 1 − v 0 , … , v d − v 0 )∣ ( d + 1 ) ( d + 2 ) ( ∑ i = 0 d g i ) 2 + ∑ i = 0 d g i 2 .
This formula follows from the moments of the barycentric coordinates under the uniform distribution on the simplex, which we prove below: E [ λ i 2 ] = 2 / ( ( d + 1 ) ( d + 2 ) ) \mathbb{E}[\lambda_i^2]=2/((d+1)(d+2)) E [ λ i 2 ] = 2/ (( d + 1 ) ( d + 2 )) and E [ λ i λ j ] = 1 / ( ( d + 1 ) ( d + 2 ) ) \mathbb{E}[\lambda_i\lambda_j]=1/((d+1)(d+2)) E [ λ i λ j ] = 1/ (( d + 1 ) ( d + 2 )) for i ≠ j i\ne j i = j .
For integers a , b ≥ 0 a,b\ge0 a , b ≥ 0 , integration by parts gives
∫ 0 1 t a ( 1 − t ) b d t = a ! b ! ( a + b + 1 ) ! . \int_0^1 t^a(1-t)^b\,\mathrm{d}t=\frac{a!b!}{(a+b+1)!}. ∫ 0 1 t a ( 1 − t ) b d t = ( a + b + 1 )! a ! b ! .
Indeed, for a = 0 a=0 a = 0 the integral is 1 / ( b + 1 ) 1/(b+1) 1/ ( b + 1 ) ; for a ≥ 1 a\ge1 a ≥ 1 , the boundary term vanishes and integration by parts multiplies the integral with exponents ( a − 1 , b + 1 ) (a-1,b+1) ( a − 1 , b + 1 ) by a / ( b + 1 ) a/(b+1) a / ( b + 1 ) . Repeating this a a a times gives the displayed factorial ratio.
For nonnegative integers k 0 , … , k d k_0,\ldots,k_d k 0 , … , k d , we now prove
∫ Δ d ∏ i = 0 d λ i k i d λ = ∏ i = 0 d k i ! ( d + ∑ i = 0 d k i ) ! . \int_{\Delta_d}\prod_{i=0}^d\lambda_i^{k_i}\,\mathrm{d}\lambda=\frac{\prod_{i=0}^d k_i!}{\left(d+\sum_{i=0}^d k_i\right)!}. ∫ Δ d i = 0 ∏ d λ i k i d λ = ( d + ∑ i = 0 d k i ) ! ∏ i = 0 d k i ! .
For d = 0 d=0 d = 0 , the single point has zero-dimensional volume one and both sides are one. For d ≥ 1 d\ge1 d ≥ 1 , put λ d = t \lambda_d=t λ d = t and λ i = ( 1 − t ) μ i \lambda_i=(1-t)\mu_i λ i = ( 1 − t ) μ i for 0 ≤ i < d 0\le i<d 0 ≤ i < d , where ( μ 1 , … , μ d − 1 ) ∈ Δ d − 1 (\mu_1,\ldots,\mu_{d-1})\in\Delta_{d-1} ( μ 1 , … , μ d − 1 ) ∈ Δ d − 1 and μ 0 = 1 − ∑ i = 1 d − 1 μ i \mu_0=1-\sum_{i=1}^{d-1}\mu_i μ 0 = 1 − ∑ i = 1 d − 1 μ i , so that ∑ i = 0 d − 1 μ i = 1 \sum_{i=0}^{d-1}\mu_i=1 ∑ i = 0 d − 1 μ i = 1 . The substitution of the d − 1 d-1 d − 1 free coordinates μ 1 , … , μ d − 1 \mu_1,\ldots,\mu_{d-1} μ 1 , … , μ d − 1 scales their volume by ( 1 − t ) d − 1 (1-t)^{d-1} ( 1 − t ) d − 1 . The induction hypothesis gives the factor ∏ i < d k i ! / ( d − 1 + ∑ i < d k i ) ! \prod_{i<d}k_i!/(d-1+\sum_{i<d}k_i)! ∏ i < d k i ! / ( d − 1 + ∑ i < d k i )! . The remaining integral is
∫ 0 1 t k d ( 1 − t ) d − 1 + ∑ i < d k i d t = k d ! ( d − 1 + ∑ i < d k i ) ! ( d + ∑ i = 0 d k i ) ! , \int_0^1 t^{k_d}(1-t)^{d-1+\sum_{i<d}k_i}\,\mathrm{d}t=\frac{k_d!(d-1+\sum_{i<d}k_i)!}{(d+\sum_{i=0}^d k_i)!}, ∫ 0 1 t k d ( 1 − t ) d − 1 + ∑ i < d k i d t = ( d + ∑ i = 0 d k i )! k d ! ( d − 1 + ∑ i < d k i )! ,
proving the formula.
At zero exponents the volume is 1 / d ! 1/d! 1/ d ! , so multiplying by d ! d! d ! converts these integrals to uniform-simplex expectations and gives the two stated moments. Expanding ( ∑ i λ i g i ) 2 (\sum_i\lambda_i g_i)^2 ( ∑ i λ i g i ) 2 gives the squared-affine integral formula above. Determinants, factorials, and affine evaluations here use polynomially many polynomial-bit rationals, so each contribution has polynomial bit length.
The following chain of explicit bounds makes this precise: it ends with one polynomial that bounds the numerator and denominator lengths of every contribution, so each leaf output is a rational of polynomial bit length. Let b 0 = 4 ( L + 1 ) 3 b_0=4(L+1)^3 b 0 = 4 ( L + 1 ) 3 bound frozen numerator and denominator lengths and set
b 1 = ⌈ log 2 ( ( d + 1 ) ! ) ⌉ + ( d + 1 ) ( d + 4 ) b 0 + 2 , b_1=\lceil\log_2((d+1)!)\rceil+(d+1)(d+4)b_0+2, b 1 = ⌈ log 2 (( d + 1 )!)⌉ + ( d + 1 ) ( d + 4 ) b 0 + 2 ,
b 2 = 4 ( d + 2 ) ( b 0 + b 1 + 1 ) , b_2=4(d+2)(b_0+b_1+1), b 2 = 4 ( d + 2 ) ( b 0 + b 1 + 1 ) ,
b 3 = 10 ( d + 2 ) 2 ( b 2 + 1 ) , b v o l = 10 ( d + 2 ) 3 ( b 1 + 1 ) . (2) b_3=10(d+2)^2(b_2+1),\qquad b_{\mathrm{vol}}=10(d+2)^3(b_1+1). \tag*{(2)} b 3 = 10 ( d + 2 ) 2 ( b 2 + 1 ) , b vol = 10 ( d + 2 ) 3 ( b 1 + 1 ) . (2)
Write n = d + 1 n=d+1 n = d + 1 . A row of the center program of a subset J J J , displayed earlier in this proof, has d + 2 d+2 d + 2 rational entries, including its right-hand side. Clearing their denominators bounds each integer entry in absolute value by 2 ( d + 2 ) b 0 2^{(d+2)b_0} 2 ( d + 2 ) b 0 . The determinant expansion therefore bounds each Cramer numerator and denominator by n ! 2 n ( d + 2 ) b 0 n!2^{n(d+2)b_0} n ! 2 n ( d + 2 ) b 0 , whose binary length is smaller than b 1 b_1 b 1 .
An affine evaluation g ( z J ) g(z_J) g ( z J ) is a sum of n n n terms: the constant coefficient of g g g , and each other coefficient of g g g times the corresponding coordinate of z J z_J z J . Each term has numerator and positive denominator length at most b 0 + b 1 b_0+b_1 b 0 + b 1 . Putting the n n n terms of an affine evaluation over the product denominator gives numerator length at most
n ( b 0 + b 1 ) + ⌈ log 2 n ⌉ + 2 n(b_0+b_1)+\lceil\log_2 n\rceil+2 n ( b 0 + b 1 ) + ⌈ log 2 n ⌉ + 2
and denominator length at most n ( b 0 + b 1 ) + 1 n(b_0+b_1)+1 n ( b 0 + b 1 ) + 1 . Both bounds are smaller than b 2 = 4 ( n + 1 ) ( b 0 + b 1 + 1 ) b_2=4(n+1)(b_0+b_1+1) b 2 = 4 ( n + 1 ) ( b 0 + b 1 + 1 ) .
Next put the n n n values g i g_i g i over their product denominator. The squared-value bracket ( ∑ i g i ) 2 + ∑ i g i 2 (\sum_i g_i)^2+\sum_i g_i^2 ( ∑ i g i ) 2 + ∑ i g i 2 then has numerator length at most 2 n b 2 + ⌈ log 2 ( n 2 + n ) ⌉ + 2 2nb_2+\lceil\log_2(n^2+n)\rceil+2 2 n b 2 + ⌈ log 2 ( n 2 + n )⌉ + 2 and denominator length at most 2 n b 2 + 1 2nb_2+1 2 n b 2 + 1 . Both bounds are smaller than b 3 = 10 ( n + 1 ) 2 ( b 2 + 1 ) b_3=10(n+1)^2(b_2+1) b 3 = 10 ( n + 1 ) 2 ( b 2 + 1 ) .
For d ≥ 1 d\ge1 d ≥ 1 , each coordinate difference has numerator magnitude below 2 2 b 1 + 1 2^{2b_1+1} 2 2 b 1 + 1 and positive denominator below 2 2 b 1 2^{2b_1} 2 2 b 1 . Clear all d 2 d^2 d 2 such denominators in the volume determinant and divide by d ! d! d ! . The determinant expansion gives respective numerator and denominator length bounds
2 d 3 b 1 + d + ⌈ log 2 ( d ! ) ⌉ + 2 , 2 d 3 b 1 + ⌈ log 2 ( d ! ) ⌉ + 1. 2d^3b_1+d+\lceil\log_2(d!)\rceil+2,\qquad2d^3b_1+\lceil\log_2(d!)\rceil+1. 2 d 3 b 1 + d + ⌈ log 2 ( d !)⌉ + 2 , 2 d 3 b 1 + ⌈ log 2 ( d !)⌉ + 1.
They are smaller than b v o l b_{\mathrm{vol}} b vol . For d = 0 d=0 d = 0 , the volume is one and obeys the same bound.
Multiplying the volume by the squared-value bracket and dividing by n ( n + 1 ) n(n+1) n ( n + 1 ) therefore shows that
b v o l + b 3 + ⌈ log 2 ( ( d + 1 ) ( d + 2 ) ) ⌉ + 4 b_{\mathrm{vol}+b_3+\lceil\log_2((d+1)(d+2))\rceil+4} b vol + b 3 + ⌈ l o g 2 (( d + 1 ) ( d + 2 ))⌉ + 4
bounds each contribution’s numerator and positive denominator length.
An activation pattern together with the d + 1 d+1 d + 1 tight sets of a face flag is a binary word of length p + ( d + 1 ) M p+(d+1)M p + ( d + 1 ) M . Use a fixed polynomial bound on this length and accept only zero padding after these fields, so each flag is counted once. Every invalid word contributes 0 / 1 0/1 0/1 . The valid contributions sum to the risk by the preceding partition and triangulation arguments. □
The hardness direction uses narrow triangular bumps whose widths encode reciprocals. The folding itself is classical [33 Lemmas 3.10–3.11] ; we give the exact weighted construction and its integral.
Proof of Theorem 8.2. Membership of fixed comparison follows from Lemma 8.3: negate each risk leaf and compare their sum with − τ -\tau − τ in the definition of P P Q ( p t ) \mathrm{PP}^{(\mathrm{pt})}_{\mathbb{Q}} PP Q ( pt ) . A guessed legal synthesis graph has polynomial description length, so one such oracle query gives the synthesis upper bound N P P P Q ( p t ) \mathrm{NP}^{\mathrm{PP}^{(\mathrm{pt})}_{\mathbb{Q}}} NP PP Q ( pt ) .
For hardness, fix a language represented by F ( a ) ≥ t ( a ) F(a)\ge t(a) F ( a ) ≥ t ( a ) as above and one input a a a . Choose a polynomial integer q ≥ 1 q\ge1 q ≥ 1 such that ∣ u ( a , w ) ∣ ≤ 2 q |u(a,w)|\le2^q ∣ u ( a , w ) ∣ ≤ 2 q and 1 ≤ v ( a , w ) ≤ 2 q 1\le v(a,w)\le2^q 1 ≤ v ( a , w ) ≤ 2 q . Put K = 2 q K=2^q K = 2 q , b = 2 q + 2 b=2q+2 b = 2 q + 2 , and
U ( w ) = K v ( a , w ) − u ( a , w ) , 0 ≤ U ( w ) ≤ 2 2 q + 1 < 2 b . U(w)=Kv(a,w)-u(a,w),\qquad0\le U(w)\le2^{2q+1}<2^b. U ( w ) = K v ( a , w ) − u ( a , w ) , 0 ≤ U ( w ) ≤ 2 2 q + 1 < 2 b .
With a a a fixed, write v ( w ) = v ( a , w ) v(w)=v(a,w) v ( w ) = v ( a , w ) . Boolean circuits of polynomial size compute the digits of v ( w ) − 1 v(w)-1 v ( w ) − 1 and the predicate j < U ( w ) j<U(w) j < U ( w ) , where j j j is a b b b -bit unsigned integer. Such circuits are obtained by unrolling the fixed polynomial-time weight evaluator and polynomial-bit integer arithmetic. Let n = p ( ∣ a ∣ ) + b ≥ 1 n=p(|a|)+b\ge1 n = p ( ∣ a ∣ ) + b ≥ 1 , so that an n n n -bit word encodes the pair ( w , j ) (w,j) ( w , j ) . No padding of the source counting word is needed.
Write
θ ( t ) = ρ ( 2 t ) − 2 ρ ( 2 t − 1 ) + ρ ( 2 t − 2 ) . \theta(t)=\rho(2t)-2\rho(2t-1)+\rho(2t-2). θ ( t ) = ρ ( 2 t ) − 2 ρ ( 2 t − 1 ) + ρ ( 2 t − 2 ) .
The power θ i \theta^i θ i denotes iteration: θ 0 ( t ) = t \theta^0(t)=t θ 0 ( t ) = t and θ i + 1 = θ ∘ θ i \theta^{i+1}=\theta\circ\theta^i θ i + 1 = θ ∘ θ i . Let z i = θ i ( x ) z_i=\theta^i(x) z i = θ i ( x ) , put η = 2 − n \eta=2^{-n} η = 2 − n , and define h = η θ n + 1 ( x ) h=\eta\theta^{n+1}(x) h = η θ n + 1 ( x ) . For 0 ≤ i < n 0\le i<n 0 ≤ i < n , set s i = 2 z i − 1 s_i=2z_i-1 s i = 2 z i − 1 . The i i i th itinerary bit is 1 { s i > 0 } \mathbf{1}_{\{s_i>0\}} 1 { s i > 0 } , so a zero value of s i s_i s i gives bit zero.
For a real x x x and a nonempty set A ⊆ R A\subseteq\mathbb{R} A ⊆ R , write dist ( x , A ) = inf a ∈ A ∣ x − a ∣ \operatorname{dist}(x,A)=\inf_{a\in A}|x-a| dist ( x , A ) = inf a ∈ A ∣ x − a ∣ . The map θ \theta θ is zero outside [ 0 , 1 ] [0,1] [ 0 , 1 ] and has the two branches 2 t , 2 − 2 t 2t,2-2t 2 t , 2 − 2 t inside. The itinerary bit is one exactly when z i ≥ 1 / 2 z_i\ge1/2 z i ≥ 1/2 , so for z i ∈ [ 0 , 1 ] z_i\in[0,1] z i ∈ [ 0 , 1 ] it records that the next iterate z i + 1 = θ ( z i ) z_{i+1}=\theta(z_i) z i + 1 = θ ( z i ) is computed on the branch 2 − 2 t 2-2t 2 − 2 t . Composing its inverse branches t / 2 , 1 − t / 2 t/2,1-t/2 t /2 , 1 − t /2 shows that the 2 n 2^n 2 n open dyadic cells of width η \eta η realize all strict sign words of ( s 0 , … , s n − 1 ) (s_0,\ldots,s_{n-1}) ( s 0 , … , s n − 1 ) exactly once. On each cell h h h has slopes 2 , − 2 2,-2 2 , − 2 and is twice the distance to its endpoints. The zero set of s i s_i s i consists of the odd multiples of 2 − ( i + 1 ) 2^{-(i+1)} 2 − ( i + 1 ) , all of which are fine-cell endpoints. On the unit interval,
∣ s i ( x ) ∣ = 2 i + 1 dist ( x , { s i = 0 } ) ≥ h ( x ) . |s_i(x)| = 2^{i+1}\operatorname{dist}(x,\{s_i=0\}) \ge h(x). ∣ s i ( x ) ∣ = 2 i + 1 dist ( x , { s i = 0 }) ≥ h ( x ) .
Set r = 1 − θ n + 1 ( x ) r=1-\theta^{n+1}(x) r = 1 − θ n + 1 ( x ) , and let ε = η / 4 \varepsilon=\eta/4 ε = η /4 . For each itinerary coordinate use the clipped ramp
β i ( x ) = ρ ( ( s i + ε ) / ( 2 ε ) ) − ρ ( ( s i − ε ) / ( 2 ε ) ) . \beta_i(x)=\rho\left((s_i+\varepsilon)/(2\varepsilon)\right)-\rho\left((s_i-\varepsilon)/(2\varepsilon)\right). β i ( x ) = ρ ( ( s i + ε ) / ( 2 ε ) ) − ρ ( ( s i − ε ) / ( 2 ε ) ) .
It lies in [ 0 , 1 ] [0,1] [ 0 , 1 ] everywhere. Where r ≤ 1 / 2 r\le1/2 r ≤ 1/2 , which is the central half of each fine cell, we have h ≥ η / 2 = 2 ε h\ge\eta/2=2\varepsilon h ≥ η /2 = 2 ε , so each β i \beta_i β i is exactly its Boolean itinerary bit.
Compile the source circuit with AND ( u , v ) = ρ ( u + v − 1 ) \operatorname{AND}(u,v)=\rho(u+v-1) AND ( u , v ) = ρ ( u + v − 1 ) and NOT ( u ) = ρ ( 1 − u ) \operatorname{NOT}(u)=\rho(1-u) NOT ( u ) = ρ ( 1 − u ) . Every gate preserves [ 0 , 1 ] [0,1] [ 0 , 1 ] on fractional inputs, while on the central half of a fine cell, where r ≤ 1 / 2 r\le1/2 r ≤ 1/2 , all gates compute the correct Boolean values. For 0 ≤ k < q 0\le k<q 0 ≤ k < q , let c k ∈ [ 0 , 1 ] c_k\in[0,1] c k ∈ [ 0 , 1 ] be the compiled circuit’s outputs for the binary digits of v ( w ) − 1 v(w)-1 v ( w ) − 1 , and let A ∈ [ 0 , 1 ] A\in[0,1] A ∈ [ 0 , 1 ] output the predicate j < U ( w ) j<U(w) j < U ( w ) . On Boolean circuit inputs the convention is v ( w ) − 1 = ∑ k = 0 q − 1 2 k c k v(w)-1=\sum_{k=0}^{q-1}2^k c_k v ( w ) − 1 = ∑ k = 0 q − 1 2 k c k , with c k ∈ { 0 , 1 } c_k\in\{0,1\} c k ∈ { 0 , 1 } .
Define
P = r + ∑ k 2 k min { r , c k } , B = ρ ( 1 − 2 P ) , T = min { B , A } . P=r+\sum_k 2^k\min\{r,c_k\},\qquad B=\rho(1-2P),\qquad T=\min\{B,A\}. P = r + k ∑ 2 k min { r , c k } , B = ρ ( 1 − 2 P ) , T = min { B , A } .
Since P ≥ r P\ge r P ≥ r , the function B B B is zero wherever r ≥ 1 / 2 r\ge1/2 r ≥ 1/2 . Within the central half, c k c_k c k is Boolean, so min { r , c k } = r c k \min\{r,c_k\}=rc_k min { r , c k } = r c k and P = v ( w ) r P=v(w)r P = v ( w ) r . There A A A is either zero or one. It follows on the entire cell, including its boundary, that
T ( x ) = { ρ ( 1 − 2 v ( w ) r ( x ) ) , j < U ( w ) , 0 , j ≥ U ( w ) . T(x)=
\begin{cases}
\rho(1-2v(w)r(x)), & j<U(w),\\
0, & j\ge U(w).
\end{cases} T ( x ) = { ρ ( 1 − 2 v ( w ) r ( x )) , 0 , j < U ( w ) , j ≥ U ( w ) .
Both sides vanish outside the central half. Outside [ 0 , 1 ] [0,1] [ 0 , 1 ] , r = 1 r=1 r = 1 and the same vanishing holds. Thus no fractional decoding region contributes to the integral.
A selected cell has a triangular bump of height one and slope magnitude 4 v ( w ) / η 4v(w)/\eta 4 v ( w ) / η on each side. Each side’s squared integral is its height cubed divided by three times that magnitude. The two sides therefore contribute η / ( 6 v ( w ) ) \eta/(6v(w)) η / ( 6 v ( w )) . There are exactly U ( w ) U(w) U ( w ) selected integers j j j for each w w w , giving
∫ 0 1 T ( x ) 2 d x = 2 − n 6 ∑ w U ( w ) v ( w ) = 2 − n 6 ( K 2 p ( ∣ a ∣ ) − F ( a ) ) . \int_0^1 T(x)^2\,dx=\frac{2^{-n}}{6}\sum_w\frac{U(w)}{v(w)}=\frac{2^{-n}}{6}\left(K2^{p(|a|)}-F(a)\right). ∫ 0 1 T ( x ) 2 d x = 6 2 − n w ∑ v ( w ) U ( w ) = 6 2 − n ( K 2 p ( ∣ a ∣ ) − F ( a ) ) .
Comparison with τ = 2 − n ( K 2 p ( ∣ a ∣ ) − t ( a ) ) / 6 \tau=2^{-n}(K2^{p(|a|)}-t(a))/6 τ = 2 − n ( K 2 p ( ∣ a ∣ ) − t ( a )) /6 is consequently equivalent to F ( a ) ≥ t ( a ) F(a)\ge t(a) F ( a ) ≥ t ( a ) , including equality. All threshold data have polynomial bit length. If only nonnegative risk thresholds are allowed and this threshold is negative, F ≤ K 2 p < t F\le K2^p<t F ≤ K 2 p < t , so send the instance to the fixed NO instance T = 1 T=1 T = 1 , τ = 0 \tau=0 τ = 0 .
All displayed functions have polynomial-size ordinary ReLU realizations with dyadic parameters. For each fold, create its three hinge gates and a ReLU of their nonnegative affine sum. The nonnegative quantity r r r can similarly be a ReLU of 1 − z n + 1 1-z_{n+1} 1 − z n + 1 . Use min ( a , b ) = a − ρ ( a − b ) \min(a,b)=a-\rho(a-b) min ( a , b ) = a − ρ ( a − b ) , storing positive and negative parts of signed intermediate quantities when needed. All circuit outputs and final quantities declared nonnegative may be passed through a ReLU unchanged.
Powers of two used in the slopes and clipped ramps have polynomial-bit exponents. Repeated predecessors in an affine expression have their coefficients combined, so no parallel edges are required. Remove unused target vertices; a constant zero vertex, if needed, has a present zero bias.
The resulting target satisfies the conditions of Definition 2.1 other than the computational-vertex bound and the inventory, lies in [ 0 , 1 ] [0,1] [ 0 , 1 ] , and is supplied independently of the candidate inventory. Taking W = ( 0 ) W=(0) W = ( 0 ) and cap one forces the candidate to be zero under either inventory rule, and a zero output bias supplies a legal witness. This proves both comparison completeness and synthesis hardness.
Finally, apply Theorem 8.1 to a fixed-candidate comparison. Let s s s bound a padded candidate certificate’s length. Amplify its B P P P P \mathrm{BPP}^{\mathrm{PP}} BPP PP algorithm to error at most 2 − s − 3 2^{-s-3} 2 − s − 3 using polynomially many independent repetitions. A union bound over all 2 s 2^s 2 s certificates makes the same random tape correct for every legal candidate with probability at least 7 / 8 7/8 7/8 . For a fixed tape, asking whether some legal certificate accepts is an N P P P \mathrm{NP}^{\mathrm{PP}} NP PP query. This gives the stated randomized synthesis upper bound. The containment in PSPACE also follows directly. Polynomial-time nondeterministic or randomized paths can be enumerated with polynomial-size counters, and a PP query can be evaluated by enumerating its polynomial-length random paths and counting acceptances in polynomial space. Reusing this space through the nested oracle simulations preserves a polynomial space bound. □
This result distinguishes fixed-network comparison from existential synthesis. It does not show that continuous-risk synthesis is complete for its upper-bound class. It also does not reduce exact comparison to an additive approximation with only polynomially many precision bits.
The Length of the Exact Rational Value
A threshold comparison need not print the value it compares. Lawrence [26 ] already exhibited rational polytopes with compact descriptions and volumes whose reduced denominators have no polynomial bit bound. The following construction gives an explicit one-dimensional squared-risk counterpart with dyadic network parameters.
Theorem 8.4 (Long risk denominators). For every sufficiently large integer n n n , an explicit dyadic ReLU network T n : R → [ 0 , 1 ] T_n:\mathbb{R}\to[0,1] T n : R → [ 0 , 1 ] has O ( n log n ) O(n\log n) O ( n log n ) binary encoding length, while the reduced denominator of
∫ 0 1 T n ( x ) 2 d x \int_0^1 T_n(x)^2\,\mathrm{d}x ∫ 0 1 T n ( x ) 2 d x has Ω ( 2 n ) \Omega(2^n) Ω ( 2 n ) bits. Thus no polynomial in network encoding length bounds all such exact risk denominators, even with W = ( 0 ) W=(0) W = ( 0 ) and cap one.
The number-theoretic input is the following established asymptotic consequence of explicit prime estimates. Write π ( t ) \pi(t) π ( t ) for the number of primes at most t t t , and log \log log for the natural logarithm in this estimate.
Theorem 8.5 (Dusart Dusart [11 Theorem 6.9, inequality (6.7)] ). As t → ∞ t\to\infty t → ∞ ,
π ( t ) = t log t + t log 2 t + O ( t log 3 t ) . \pi(t)=\frac{t}{\log t}+\frac{t}{\log^2 t}+O\left(\frac{t}{\log^3 t}\right). π ( t ) = log t t + log 2 t t + O ( log 3 t t ) .
The cited bounds are unconditional. We use only their asymptotic form and require no explicit finite starting value of n n n .
Proof of Theorem 8.4. Use the tent map θ \theta θ from the proof of Theorem 8.2 . Put
η = 2 − n , B n ( x ) = ∑ i = 1 n θ i ( x ) , a = 4 ⋅ 2 ⌈ log 2 ( n + 1 ) ⌉ , c = n + 1 , C = 2 a , \eta=2^{-n},\qquad B_n(x)=\sum_{i=1}^{n}\theta^i(x),\qquad a=4\cdot2^{\lceil\log_2(n+1)\rceil},\qquad c=n+1,\qquad C=2a, η = 2 − n , B n ( x ) = i = 1 ∑ n θ i ( x ) , a = 4 ⋅ 2 ⌈ l o g 2 ( n + 1 )⌉ , c = n + 1 , C = 2 a ,
and define
T n ( x ) = C − 1 ρ ( a θ n + 1 ( x ) + B n ( x ) − c ) . T_n(x)=C^{-1}\rho\left(a\theta^{n+1}(x)+B_n(x)-c\right). T n ( x ) = C − 1 ρ ( a θ n + 1 ( x ) + B n ( x ) − c ) .
Both a , C a,C a , C are powers of two, with 4 ( n + 1 ) ≤ a < 8 ( n + 1 ) 4(n+1)\leq a<8(n+1) 4 ( n + 1 ) ≤ a < 8 ( n + 1 ) . On the unit interval, 0 ≤ B n ≤ n 0\leq B_n\leq n 0 ≤ B n ≤ n and 0 ≤ θ n + 1 ≤ 1 0\leq\theta^{n+1}\leq1 0 ≤ θ n + 1 ≤ 1 , so 0 ≤ T n ≤ ( a − 1 ) / C < 1 0\leq T_n\leq(a-1)/C<1 0 ≤ T n ≤ ( a − 1 ) / C < 1 . Outside the interval all the positive iterates vanish, hence T n = 0 T_n=0 T n = 0 .
Each fold uses four ReLUs as in the preceding proof. Its output is shared by the next fold and the final sum, with all parameter positions explicitly listed. The final hidden preactivation uses the n n n coefficients one, the coefficient a a a , and bias − c -c − c ; its affine output uses C − 1 C^{-1} C − 1 . There are 4 ( n + 1 ) + 2 4(n+1)+2 4 ( n + 1 ) + 2 computational vertices and O ( n ) O(n) O ( n ) positions, no parallel edges, and every vertex reaches the output. The parameter lengths and vertex-index lengths are O ( log n ) O(\log n) O ( log n ) , giving the asserted encoding bound.
For each integer 0 ≤ j < 2 n 0\leq j<2^n 0 ≤ j < 2 n , let I j = ( j η , ( j + 1 ) η ) I_j=(j\eta,(j+1)\eta) I j = ( j η , ( j + 1 ) η ) . On this open cell the function B n B_n B n is affine with slope
s j = ∑ i = 1 n ε i , j 2 i , ε i , j ∈ { − 1 , 1 } . s_j=\sum_{i=1}^{n}\varepsilon_{i,j}2^i,\qquad\varepsilon_{i,j}\in\{-1,1\}. s j = i = 1 ∑ n ε i , j 2 i , ε i , j ∈ { − 1 , 1 } .
Every sign vector occurs once. Indeed, if the tent branch signs are β 1 , … , β n \beta_1,\ldots,\beta_n β 1 , … , β n , then ε i = β 1 ⋯ β i \varepsilon_i=\beta_1\cdots\beta_i ε i = β 1 ⋯ β i ; conversely β 1 = ε 1 \beta_1=\varepsilon_1 β 1 = ε 1 and β i = ε i ε i − 1 \beta_i=\varepsilon_i\varepsilon_{i-1} β i = ε i ε i − 1 . The inverse-branch cell correspondence is bijective, so the numbers s j / 2 s_j/2 s j /2 are precisely all odd integers from − ( 2 n − 1 ) -(2^n-1) − ( 2 n − 1 ) to 2 n − 1 2^n-1 2 n − 1 . The identity θ i ( 1 − x ) = θ i ( x ) \theta^i(1-x)=\theta^i(x) θ i ( 1 − x ) = θ i ( x ) for i ≥ 1 i\geq1 i ≥ 1 also shows that reflected cells have opposite slopes and equal midpoint values.
At a fine-cell endpoint the last preactivation is B n − c ≤ − 1 B_n-c\leq-1 B n − c ≤ − 1 . At its midpoint it is at least a − c > 0 a-c>0 a − c > 0 . Write
u j = c − B n ( j η ) , D = a 2 n + 1 , H j = a − u j + s j η / 2 , p j = ( D + s j ) / 2. (3) u_j=c-B_n(j\eta),\qquad D=a2^{n+1},\qquad H_j=a-u_j+s_j\eta/2,\qquad p_j=(D+s_j)/2. \tag*{(3)} u j = c − B n ( j η ) , D = a 2 n + 1 , H j = a − u j + s j η /2 , p j = ( D + s j ) /2. (3)
Here 1 ≤ u j ≤ n + 1 1 \le u_j \le n+1 1 ≤ u j ≤ n + 1 , and H j H_j H j is the positive midpoint height before multiplication by C − 1 C^{-1} C − 1 . The two half-cell slopes are D + s j D+s_j D + s j and − ( D − s j ) -(D-s_j) − ( D − s j ) , both with positive magnitudes. There is therefore exactly one zero in each half, and the positive portions are two linear triangles of height H j H_j H j . Their squared integrals give
I n = ∫ 0 1 T n 2 = 1 3 C 2 ∑ j H j 3 ( 1 D + s j + 1 D − s j ) = 1 3 C 2 ∑ j H j 3 p j . I_n=\int_0^1 T_n^2=\frac{1}{3C^2}\sum_j H_j^3\left(\frac{1}{D+s_j}+\frac{1}{D-s_j}\right)=\frac{1}{3C^2}\sum_j\frac{H_j^3}{p_j}. I n = ∫ 0 1 T n 2 = 3 C 2 1 j ∑ H j 3 ( D + s j 1 + D − s j 1 ) = 3 C 2 1 j ∑ p j H j 3 .
For the last equality, reindex the terms H j 3 / ( D − s j ) H_j^3/(D-s_j) H j 3 / ( D − s j ) by the reflection x ↦ 1 − x x\mapsto1-x x ↦ 1 − x , which permutes the cells. Reflected cells have opposite slopes and equal midpoint values, hence equal heights H j H_j H j , so these terms sum to ∑ j H j 3 / ( D + s j ) \sum_j H_j^3/(D+s_j) ∑ j H j 3 / ( D + s j ) . The sum over j j j is therefore 2 ∑ j H j 3 / ( D + s j ) 2\sum_j H_j^3/(D+s_j) 2 ∑ j H j 3 / ( D + s j ) , which equals ∑ j H j 3 / p j \sum_j H_j^3/p_j ∑ j H j 3 / p j because D + s j = 2 p j D+s_j=2p_j D + s j = 2 p j .
Set U j = 2 n + 1 u j U_j=2^{n+1}u_j U j = 2 n + 1 u j and E j = 2 n + 1 H j E_j=2^{n+1}H_j E j = 2 n + 1 H j . Iterate values at dyadic endpoints have denominators dividing 2 n 2^n 2 n , so these are integers, and
E j = 2 p j − U j , I n = 1 3 C 2 ⋅ 2 3 ( n + 1 ) ∑ j E j 3 p j . E_j=2p_j-U_j,\qquad I_n=\frac{1}{3C^2\cdot2^{3(n+1)}}\sum_j\frac{E_j^3}{p_j}. E j = 2 p j − U j , I n = 3 C 2 ⋅ 2 3 ( n + 1 ) 1 j ∑ p j E j 3 .
Moreover,
0 < U j ≤ 2 n + 1 ( n + 1 ) < ( a − 1 ) 2 n < p j . 0<U_j\le2^{n+1}(n+1)<(a-1)2^n<p_j. 0 < U j ≤ 2 n + 1 ( n + 1 ) < ( a − 1 ) 2 n < p j .
The p j p_j p j run once through all odd integers in ( ( a − 1 ) 2 n , ( a + 1 ) 2 n ) ((a-1)2^n,(a+1)2^n) (( a − 1 ) 2 n , ( a + 1 ) 2 n ) .
Let p > 3 p>3 p > 3 be a prime in that interval. Exactly one denominator p j p_j p j equals p p p . Every other denominator is less than 2 p 2p 2 p and differs from p p p , because the ratio of the interval endpoints is ( a + 1 ) / ( a − 1 ) < 2 (a+1)/(a-1)<2 ( a + 1 ) / ( a − 1 ) < 2 . None is divisible by p p p . The corresponding numerator obeys E j ≡ − U j ≢ 0 ( m o d p ) E_j\equiv-U_j\not\equiv0\pmod p E j ≡ − U j ≡ 0 ( mod p ) , and since p p p is prime, p p p does not divide E j 3 E_j^3 E j 3 . Combining the other terms over a denominator coprime to p p p therefore cannot cancel this term’s factor p p p in the denominator. The common prefactor has denominator divisible only by two and three, so it cannot cancel it either. Thus every prime in the interval divides the reduced denominator of I n I_n I n .
To count them, put X = a 2 n X=a2^n X = a 2 n , Y = 2 n Y=2^n Y = 2 n , and F ( t ) = t / log t + t / log 2 t F(t)=t/\log t+t/\log^2t F ( t ) = t / log t + t / log 2 t . In this proof, f = Θ ( g ) f=\Theta(g) f = Θ ( g ) means both f = O ( g ) f=O(g) f = O ( g ) and f = Ω ( g ) f=\Omega(g) f = Ω ( g ) , and o ( 1 ) o(1) o ( 1 ) denotes a quantity tending to zero as n → ∞ n\to\infty n → ∞ . Then log X = Θ ( n ) \log X=\Theta(n) log X = Θ ( n ) , a = Θ ( n ) a=\Theta(n) a = Θ ( n ) , and
F ′ ( t ) = 1 log t − 2 log 3 t . F'(t)=\frac{1}{\log t}-\frac{2}{\log^3t}. F ′ ( t ) = log t 1 − log 3 t 2 .
Uniformly for X − Y ≤ t ≤ X + Y X-Y\le t\le X+Y X − Y ≤ t ≤ X + Y , the derivative is ( 1 + o ( 1 ) ) / log X (1+o(1))/\log X ( 1 + o ( 1 )) / log X , since Y / X = 1 / a → 0 Y/X=1/a\to0 Y / X = 1/ a → 0 . By Theorem 8.5 ,
π ( X + Y ) − π ( X − Y ) = ( 1 + o ( 1 ) ) 2 Y log X + O ( X log 3 X ) = Ω ( 2 n / n ) . \pi(X+Y)-\pi(X-Y)=(1+o(1))\frac{2Y}{\log X}+O\left(\frac{X}{\log^3 X}\right)=\Omega(2^n/n). π ( X + Y ) − π ( X − Y ) = ( 1 + o ( 1 )) log X 2 Y + O ( log 3 X X ) = Ω ( 2 n / n ) .
The error relative to 2 Y / log X 2Y/\log X 2 Y / log X is O ( a / log 2 X ) = O ( 1 / n ) O(a/\log^2X)=O(1/n) O ( a / log 2 X ) = O ( 1/ n ) , so it does not remove the positive main term.
Each counted prime has Θ ( n ) \Theta(n) Θ ( n ) binary logarithm. Their product, which divides the reduced denominator, consequently has Ω ( 2 n ) \Omega(2^n) Ω ( 2 n ) bits. The interval endpoints are even, so endpoint conventions do not change the count for large n n n . This proves the claimed output-size bound. □
The exponential bound is in the construction parameter n n n ; the ordinary indexed encoding has length O ( n log n ) O(n\log n) O ( n log n ) . The conclusion is a failure of every polynomial output-length bound, not a proof that threshold comparison lies outside PP or another decision class.
Synthesis with a Binary Accuracy Gap
By Proposition 9.3 , synthesis with a binary accuracy gap has a separator in N P P P \mathrm{NPPP} NPPP , and the matching hardness holds already for continuous uniform inputs on the cube. The reduction below supplies an integer inventory for which every legal candidate under cap two can be analyzed, whatever role each occurrence plays in it. Balanced corners, with equally many zeros and ones, give all intended corner functions the same bias, so the inventory needs only one bias occurrence. This leaves no second bias occurrence for a candidate to reassign to a steeper input edge.
For a prescribed input distribution, a binary accuracy gap supplies a rational threshold τ \tau τ and a positive rational gap δ \delta δ , both in binary. The synthesis promise is YES if some legal candidate has squared risk at most τ \tau τ , and NO if every legal candidate has risk at least τ + δ \tau+\delta τ + δ . These are disjoint promised input sets; inputs in neither set have no required answer. No unary encoding of 1 / δ 1/\delta 1/ δ and no bounded-loss assumption are imposed. A separator is an ordinary language containing every promised YES instance and no promised NO instance. A polynomial-time many-one promise reduction maps promised YES inputs to YES inputs and promised NO inputs to NO inputs.
Theorem 8.6. (Continuous uniform synthesis with a binary gap; restatement of Theorem 1.5). For continuous uniform inputs on [ 0 , 1 ] d [0,1]^d [ 0 , 1 ] d , the squared-risk synthesis promise problem of Proposition 9.3 is N P P P \mathrm{NP}^{\mathrm{PP}} NP PP -complete under deterministic polynomial-time many-one promise reductions, under either inventory rule. Hardness holds with integer inventory and computational-node cap two, including the affine output. The target is an independently supplied rational ReLU network and the positive gap is encoded in ordinary binary.
Membership follows from Proposition 9.3 ; we prove hardness. The dimension in the reduction is explicit, and may include auxiliary coordinates that every competing graph is permitted to use. We write ∥ f ∥ 2 2 = ∫ f 2 \lVert f\rVert_2^2=\int f^2 ∥ f ∥ 2 2 = ∫ f 2 for uniform-cube integrals in the dimensions in question.
Lemma 8.7. (Balanced corner inventory). Let e ≥ 2 e\ge2 e ≥ 2 be an integer, and set D = 2 e D=2e D = 2 e , K = D + 2 K=D+2 K = D + 2 , M = 2 10 K 2 M=2^{10K^2} M = 2 10 K 2 , and b = 1 − e M b=1-eM b = 1 − e M . Supply e e e occurrences of M M M , e e e occurrences of − M -M − M , one occurrence of b b b , and one occurrence of 1 1 1 , with cap two. On [ 0 , 1 ] d [0,1]^d [ 0 , 1 ] d define
U ( x ) = ρ ( 1 − M ∑ i = 1 D min { x i , 1 − x i } ) . U(x)=\rho\left(1-M\sum_{i=1}^{D}\min\{x_i,1-x_i\}\right). U ( x ) = ρ ( 1 − M i = 1 ∑ D min { x i , 1 − x i } ) . For every corner a ∈ { 0 , 1 } D a\in\{0,1\}^{D} a ∈ { 0 , 1 } D , define
h a ( x ) = ρ ( 1 − M ∑ i = 1 D ∣ x i − a i ∣ ) , h_a(x)=\rho\left(1-M\sum_{i=1}^{D}|x_i-a_i|\right), h a ( x ) = ρ ( 1 − M i = 1 ∑ D ∣ x i − a i ∣ ) , and write ∣ a ∣ = ∑ i = 1 D a i |a|=\sum_{i=1}^{D}a_i ∣ a ∣ = ∑ i = 1 D a i for its Hamming weight. Then the following two statements hold. First, for any integer d a u x ≥ 0 d_{\mathrm{aux}}\ge0 d aux ≥ 0 , additional independent uniform coordinates z ∈ [ 0 , 1 ] d a u x z\in[0,1]^{d_{\mathrm{aux}}} z ∈ [ 0 , 1 ] d aux , legal at-most-once candidate N ( x , z ) N(x,z) N ( x , z ) , and measurable μ ( x ) \mu(x) μ ( x ) with 0 ≤ μ ≤ U / 2 0\le\mu\le U/2 0 ≤ μ ≤ U /2 ,
∫ [ 0 , 1 ] D + d a u x ( N 2 − 2 N μ ) d x d z ≥ 0. \int_{[0,1]^{D+d_{\mathrm{aux}}}}\left(N^2-2N\mu\right)\,dx\,dz\ge0. ∫ [ 0 , 1 ] D + d aux ( N 2 − 2 N μ ) d x d z ≥ 0. Second, if ∣ a ∣ = e |a|=e ∣ a ∣ = e , then h a h_a h a agrees on [ 0 , 1 ] D [0,1]^D [ 0 , 1 ] D with a legal candidate consuming every supplied occurrence.
Proof. Here the positive support of a nonnegative function h h h means the set { x : h ( x ) > 0 } \{x:h(x)>0\} { x : h ( x ) > 0 } . For M ≥ 4 M\ge4 M ≥ 4 , the positive supports of the h a h_a h a , over all corners, are disjoint and U = ∑ a h a U=\sum_a h_a U = ∑ a h a , with 0 ≤ U ≤ 1 0\le U\le1 0 ≤ U ≤ 1 . The elementary simplex integral is
∫ y i ≥ 0 ∑ i = 1 j y i ≤ 1 ( 1 − ∑ i y i ) q d y = q ! ( j + q ) ! ( j , q ∈ Z ≥ 0 ) . \int_{\substack{y_i\ge0\\ \sum_{i=1}^{j}y_i\le1}}\left(1-\sum_i y_i\right)^q\,dy=\frac{q!}{(j+q)!}\qquad(j,q\in\mathbb{Z}_{\ge0}). ∫ y i ≥ 0 ∑ i = 1 j y i ≤ 1 ( 1 − i ∑ y i ) q d y = ( j + q )! q ! ( j , q ∈ Z ≥ 0 ) .
For j = 0 j=0 j = 0 both sides equal one. For larger j j j , integrating one coordinate and rescaling the remaining simplex proves the formula inductively, using ∫ 0 1 ( 1 − t ) j + q − 1 d t = 1 / ( j + q ) \int_0^1(1-t)^{j+q-1}\,dt=1/(j+q) ∫ 0 1 ( 1 − t ) j + q − 1 d t = 1/ ( j + q ) . Consequently, with
I j = ( j + 2 ) ! M j , E 0 = 2 D I D , I_j=\frac{(j+2)!}{M^j},\qquad E_0=2^D I_D, I j = M j ( j + 2 )! , E 0 = 2 D I D ,
we have ∥ h a ∥ 2 2 = I D \lVert h_a\rVert_2^2=I_D ∥ h a ∥ 2 2 = I D and ∥ U ∥ 2 2 = E 0 < 1 / 256 \lVert U\rVert_2^2=E_0<1/256 ∥ U ∥ 2 2 = E 0 < 1/256 .
On the cube, ∣ x i − a i ∣ |x_i-a_i| ∣ x i − a i ∣ equals 1 − x i 1-x_i 1 − x i if a i = 1 a_i=1 a i = 1 and x i x_i x i if a i = 0 a_i=0 a i = 0 . Hence, if ∣ a ∣ = e |a|=e ∣ a ∣ = e , then 1 − M ∣ a ∣ = 1 − e M = b 1-M|a|=1-eM=b 1 − M ∣ a ∣ = 1 − e M = b and
h a ( x ) = ρ ( b + M ∑ i : a i = 1 x i − M ∑ i : a i = 0 x i ) . h_a(x)=\rho\left(b+M\sum_{i:a_i=1}x_i-M\sum_{i:a_i=0}x_i\right). h a ( x ) = ρ ( b + M i : a i = 1 ∑ x i − M i : a i = 0 ∑ x i ) .
This preactivation has e e e positive M M M -edges, e e e negative M M M -edges and bias b b b ; the output edge uses the unit occurrence. It has two legal computational vertices and uses all m = D + 2 m=D+2 m = D + 2 occurrences.
We now examine every competing graph. A candidate with no hidden vertex is integer-affine. Otherwise it has exactly one hidden vertex and can be written
N = A + g ρ ( ℓ ) , N=A+g\rho(\ell), N = A + g ρ ( ℓ ) ,
where A A A is the direct affine output part, ℓ \ell ℓ is the hidden integer-affine preactivation, and g g g is its nonzero output-edge occurrence. Both formal affine branches A A A and A + g ℓ A+g\ell A + g ℓ have integer coefficients. Suppose neither branch is identically zero. The candidate equals A A A on the halfspace { ℓ ≤ 0 } \{\ell\le0\} { ℓ ≤ 0 } and A + g ℓ A+g\ell A + g ℓ on { ℓ > 0 } \{\ell> 0\} { ℓ > 0 } , and one of these two halfspaces has cube measure at least 1 / 2 1/2 1/2 . On it, N N N equals a nonzero integer-affine form Q = c + ∑ i = 1 D + d a u x α i ξ i Q=c+\sum_{i=1}^{D+d_{\mathrm{aux}}}\alpha_i\xi_i Q = c + ∑ i = 1 D + d aux α i ξ i , where ξ = ( x , z ) \xi=(x,z) ξ = ( x , z ) and c , α i c,\alpha_i c , α i are integers. If Q Q Q is constant, its magnitude is at least one. Otherwise choose j j j with α j ≠ 0 \alpha_j\ne0 α j = 0 ; then ∣ α j ∣ ≥ 1 |\alpha_j|\ge1 ∣ α j ∣ ≥ 1 , and conditioning on the other coordinates gives
Pr [ ∣ Q ∣ < 1 / 8 ] ≤ 1 / 4. \Pr[|Q|<1/8]\le1/4. Pr [ ∣ Q ∣ < 1/8 ] ≤ 1/4.
Indeed the admissible interval in that coordinate has length at most 1 / ( 4 ∣ α j ∣ ) 1/(4|\alpha_j|) 1/ ( 4∣ α j ∣ ) . Thus the integral of Q 2 Q^2 Q 2 on any set of measure at least 1 / 2 1/2 1/2 is at least ( 1 / 2 − 1 / 4 ) ( 1 / 8 ) 2 = 1 / 256 (1/2-1/4)(1/8)^2=1/256 ( 1/2 − 1/4 ) ( 1/8 ) 2 = 1/256 . This proves ∥ N ∥ 2 ≥ 1 / 16 \|N\|_2\ge1/16 ∥ N ∥ 2 ≥ 1/16 , also for any nonzero integer-affine candidate with no hidden vertex.
Cauchy–Schwarz then gives
∫ ( N 2 − 2 N μ ) ≥ ∥ N ∥ 2 2 − ∫ ∣ N ∣ U ≥ ∥ N ∥ 2 ( ∥ N ∥ 2 − ∥ U ∥ 2 ) ≥ 0. \int(N^2-2N\mu)\ge\|N\|_2^2-\int|N|U\ge\|N\|_2(\|N\|_2-\|U\|_2)\ge0. ∫ ( N 2 − 2 N μ ) ≥ ∥ N ∥ 2 2 − ∫ ∣ N ∣ U ≥ ∥ N ∥ 2 ( ∥ N ∥ 2 − ∥ U ∥ 2 ) ≥ 0.
A zero candidate is immediate. This argument includes halfspaces of measure zero or one.
If a formal branch is identically zero, then N = g ρ ( q ) N=g\rho(q) N = g ρ ( q ) , where q = ℓ q=\ell q = ℓ if A = 0 A=0 A = 0 , and q = − ℓ q=-\ell q = − ℓ if A + g ℓ = 0 A+g\ell=0 A + g ℓ = 0 . The latter identity follows from ρ ( ℓ ) − ℓ = ρ ( − ℓ ) \rho(\ell)-\ell=\rho(-\ell) ρ ( ℓ ) − ℓ = ρ ( − ℓ ) ; the sign of g g g is unchanged. For g < 0 g<0 g < 0 , the candidate is nonpositive and soundness holds pointwise. The only positive occurrences in the inventory are M M M and 1 1 1 , since − M -M − M and b = 1 − e M b=1-eM b = 1 − e M are negative, so only g = 1 g=1 g = 1 or g = M g=M g = M remains. Consider first the case q = − ℓ q=-\ell q = − ℓ , in which the active branch A + g ℓ A+g\ell A + g ℓ , the formula for N N N where ℓ > 0 \ell>0 ℓ > 0 , is identically zero. Then A = − g ℓ A=-g\ell A = − g ℓ , so each nonzero hidden coefficient a a a , including a bias, requires the output coefficient − g a -ga − g a on the same input or as the output bias. That output coefficient is a single inventory occurrence, because Definition 2.1 allows at most one edge for each ordered pair of vertices and each vertex has at most one bias. For g = M g=M g = M , a hidden coefficient of magnitude at least M M M would require an output coefficient of magnitude at least M 2 > e M M^2>eM M 2 > e M , which is unavailable in the inventory. Only the sole unit occurrence remains; then ℓ \ell ℓ is either an input coordinate or the constant one and ρ ( − ℓ ) = 0 \rho(-\ell)=0 ρ ( − ℓ ) = 0 on the cube. For g = 1 g=1 g = 1 , the hidden value b b b would require the output coefficient − b = e M − 1 -b=eM-1 − b = e M − 1 , which is absent from the inventory. All hidden coefficients are therefore multiples of M M M , and a positive vertex maximum of q = − ℓ q=-\ell q = − ℓ is at least M M M .
Consider next A = 0 A=0 A = 0 , and write B = max ℓ B=\max\ell B = max ℓ , attained at a cube vertex. For g = 1 g=1 g = 1 , the unit occurrence is already the gain. Without bias b b b , the b b b -occurrence is either unused or a negative input coefficient, contributing zero to the vertex maximum; all other contributions to the maximum are multiples of M M M . Thus B > 0 B>0 B > 0 implies B ≥ M B\ge M B ≥ M . With bias b b b , let p ≤ e p\le e p ≤ e be the number of positive input edges. Then B = 1 − M ( e − p ) B=1-M(e-p) B = 1 − M ( e − p ) , positive only when p = e p=e p = e . In that case B = 1 B=1 B = 1 . Write J J J for the set of coordinates selected by the hidden vertex and k = ∣ J ∣ k=|J| k = ∣ J ∣ , so e ≤ k ≤ D e\le k\le D e ≤ k ≤ D . For j ∈ J j\in J j ∈ J , the distance dist j \operatorname{dist}_j dist j is the selected coordinate or one minus that coordinate, and exactly e e e distances are of the latter type. The candidate is then
N = ρ ( 1 − M ∑ j ∈ J dist j ) . N=\rho\left(1-M\sum_{j\in J}\operatorname{dist}_j\right). N = ρ 1 − M j ∈ J ∑ dist j .
Selected coordinates may be original or auxiliary.
For g = M g=M g = M , one positive M M M occurrence is consumed. With bias b b b , the vertex maximum B B B is b b b plus the sum of the positive input coefficients. Those are at most the e − 1 e-1 e − 1 remaining occurrences of M M M and possibly the unit, so
B ≤ ( 1 − e M ) + ( e − 1 ) M = 1 − M , B\le(1-eM)+(e-1)M=1-M, B ≤ ( 1 − e M ) + ( e − 1 ) M = 1 − M ,
or B ≤ 2 − M < 0 B\le2-M<0 B ≤ 2 − M < 0 if the unit is also an input coefficient. Hence a nonzero candidate has no such bias. Its maximum is a multiple of M M M or a multiple plus one. If 0 < B < M 0<B<M 0 < B < M , then B = 1 B=1 B = 1 . The unit is either the bias, with no positive M M M -input, or an input coefficient. In the latter case there are no positive M M M -inputs if the bias is absent, or exactly one if the bias is − M -M − M . Counting the remaining negative M M M occurrences and the optional negative b b b -input gives at most e + 2 ≤ D e+2\le D e + 2 ≤ D nonzero input coordinates in every such case. In the bias-− M -M − M case one negative occurrence has already been consumed by that bias.
For the following norm estimates, write B = max q B=\max q B = max q , with the maximum taken over the full input cube. We first show that every case with B ≥ M B\ge M B ≥ M has a norm large enough for the Cauchy–Schwarz estimate. above; the cases with B = 1 B = 1 B = 1 follow. Every hidden input coefficient has magnitude at most e M eM e M , with at most K K K nonzero input coordinates. If B ≥ M B \ge M B ≥ M , a one-sided cube of side r = 1 / ( 2 K ( e + 1 ) ) r = 1/(2K(e+1)) r = 1/ ( 2 K ( e + 1 )) at a maximizing vertex has q ≥ M / 2 q \ge M/2 q ≥ M /2 . It follows that
∥ g ρ ( q ) ∥ 2 2 ≥ M 2 r K / 4 ≥ 1 / 256. \lVert g_\rho(q)\rVert_2^2 \ge M^2r^K/4 \ge1/256. ∥ g ρ ( q ) ∥ 2 2 ≥ M 2 r K /4 ≥ 1/256.
The final inequality follows from log 2 ( ( 2 K ( e + 1 ) ) K ) ≤ 3 K 2 \log_2((2K(e+1))^K) \le3K^2 log 2 (( 2 K ( e + 1 ) ) K ) ≤ 3 K 2 and M 2 = 2 20 K 2 M^2 = 2^{20K^2} M 2 = 2 20 K 2 .
For g = M , B = 1 g = M, B = 1 g = M , B = 1 , there are at most D D D input coordinates. The same argument with side 1 / ( 2 D ( e + 1 ) M ) 1/(2D(e+1)M) 1/ ( 2 D ( e + 1 ) M ) gives
∥ N ∥ 2 2 ≥ M 2 − D 4 ( 2 D ( e + 1 ) ) D ≥ E 0 . \lVert N\rVert_2^2 \ge\frac{M^{2-D}}{4(2D(e+1))^D} \ge E_0. ∥ N ∥ 2 2 ≥ 4 ( 2 D ( e + 1 ) ) D M 2 − D ≥ E 0 .
The last inequality follows from M 2 ≥ 2 D + 3 ( 2 D ( e + 1 ) ) D M^2 \ge2^{D+3}(2D(e+1))^D M 2 ≥ 2 D + 3 ( 2 D ( e + 1 ) ) D , whose right-hand side has binary logarithm at most 4 K 2 4K^2 4 K 2 . Fewer or no input coordinates only improve these weaker bounds. Cauchy–Schwarz proves soundness in both cases.
Only the case g = 1 , B = 1 g = 1, B = 1 g = 1 , B = 1 remains, in which the candidate is N = ρ ( 1 − M ∑ j ∈ J dist j ) N = \rho(1 - M\sum_{j\in J}\operatorname{dist}_j) N = ρ ( 1 − M ∑ j ∈ J dist j ) , as displayed above. Let s s s selected coordinates be original and t t t be auxiliary, so s + t = k ≤ D s+t = k \le D s + t = k ≤ D , and put l = D − s l = D-s l = D − s . The squared norm of N N N is I k I_k I k . For each selected original coordinate i i i , the distance dist i \operatorname{dist}_i dist i equals ∣ x i − a i ∣ \lvert x_i-a_i\rvert ∣ x i − a i ∣ for one value a i ∈ { 0 , 1 } a_i \in\{0,1\} a i ∈ { 0 , 1 } , its corner sign. Since U = ∑ a h a U = \sum_a h_a U = ∑ a h a , the product N U NU N U can be nonzero only on the positive support of a summand h a h_a h a whose corner a a a has these corner signs on the selected original coordinates. Writing V = ρ ( 1 − M ∑ i = 1 s dist i ) V = \rho(1-M\sum_{i=1}^s\operatorname{dist}_i) V = ρ ( 1 − M ∑ i = 1 s dist i ) , integration of N N N over the selected auxiliary coordinates gives V t + 1 / ( ( t + 1 ) ! M t ) V^{t+1}/((t+1)!M^t) V t + 1 / (( t + 1 )! M t ) . Call the l l l original coordinates that N N N does not select missing. Integration of U U U over the missing coordinates gives 2 l V l + 1 / ( ( l + 1 ) ! M l ) 2^lV^{l+1}/((l+1)!M^l) 2 l V l + 1 / (( l + 1 )! M l ) , one contribution for each choice of the missing corner signs. Since N N N does not depend on the missing coordinates and U U U does not depend on the auxiliary ones, ∫ N U \int NU ∫ N U is the integral, over the s s s selected original coordinates, of the product of these two partial integrals. Applying the simplex integral to this product, 2 l V t + l + 2 / ( ( t + 1 ) ! ( l + 1 ) ! M t + l ) 2^lV^{t+l+2}/((t+1)!(l+1)!M^{t+l}) 2 l V t + l + 2 / (( t + 1 )! ( l + 1 )! M t + l ) , gives
C : = ∫ N U = 2 D − s ( t + D − s + 2 ) ! ( t + 1 ) ! ( D − s + 1 ) ! ( t + D + 2 ) ! M t + D . C := \int NU = \frac{2^{D-s}(t+D-s+2)!}{(t+1)!(D-s+1)!(t+D+2)!M^{t+D}}. C := ∫ N U = ( t + 1 )! ( D − s + 1 )! ( t + D + 2 )! M t + D 2 D − s ( t + D − s + 2 )! .
If s = D , t = 0 s = D,t = 0 s = D , t = 0 , then C = I D = ∥ N ∥ 2 2 C = I_D = \lVert N\rVert_2^2 C = I D = ∥ N ∥ 2 2 . Otherwise l ≥ 1 l \ge1 l ≥ 1 , and
C I k = 2 l − 1 ( t + l + 2 ) ! ( s + t + 2 ) ! ( t + 1 ) ! ( l + 1 ) ! ( D + t + 2 ) ! M − l ≤ 2 3 D + 1 M ≤ 1 2 . \frac{C}{I_k} = \frac{2^{l-1}(t+l+2)!(s+t+2)!}{(t+1)!(l+1)!(D+t+2)!}M^{-l} \le\frac{2^{3D+1}}{M} \le\frac{1}{2}. I k C = ( t + 1 )! ( l + 1 )! ( D + t + 2 )! 2 l − 1 ( t + l + 2 )! ( s + t + 2 )! M − l ≤ M 2 3 D + 1 ≤ 2 1 .
Here ( s + t + 2 ) ! ≤ ( D + t + 2 ) ! (s+t+2)! \le(D+t+2)! ( s + t + 2 )! ≤ ( D + t + 2 )! . The remaining factorial ratio is the binomial coefficient ( t + l + 2 t + 1 ) = ( t + l + 2 ) ! / ( ( t + 1 ) ! ( l + 1 ) ! ) \binom{t+l+2}{t+1} = (t+l+2)!/((t+1)!(l+1)!) ( t + 1 t + l + 2 ) = ( t + l + 2 )! / (( t + 1 )! ( l + 1 )!) , which counts subsets of size t + 1 t+1 t + 1 in a set of size t + l + 2 t+l+2 t + l + 2 . It is at most 2 t + l + 2 2^{t+l+2} 2 t + l + 2 , the number of all subsets of that set. Since N ≥ 0 N \ge0 N ≥ 0 and 2 μ ≤ U 2\mu\le U 2 μ ≤ U , we have 2 N μ ≤ N U 2N\mu\le NU 2 N μ ≤ N U pointwise, so
∫ ( N 2 − 2 N μ ) ≥ ∥ N ∥ 2 2 − ∫ N U = I k − C . \int(N^2-2N\mu) \ge\lVert N\rVert_2^2-\int NU = I_k-C. ∫ ( N 2 − 2 N μ ) ≥ ∥ N ∥ 2 2 − ∫ N U = I k − C .
In the case s = D s = D s = D , t = 0 t = 0 t = 0 , the right side is exactly zero, which the lemma permits, since it asserts only a nonnegative integral. Otherwise C ≤ I k / 2 C \le I_k/2 C ≤ I k /2 , and the right side is at least I k / 2 > 0 I_k/2 > 0 I k /2 > 0 . This proves the required nonnegative integral. All possible affine branches, gains and occurrence roles have been covered. □ \square □
We next compile the probabilistic source into an actual continuous target. The discontinuous rounded variables used in the analysis are not supplied as network operations.
Proof of Theorem 8.6. Take the strict-majority source of Theorem 7.2 , with existential bits w ∈ { 0 , 1 } e w \in\{0,1\}^e w ∈ { 0 , 1 } e and random bits z ∈ { 0 , 1 } n z \in\{0,1\}^n z ∈ { 0 , 1 } n . Adding ignored variables, we may assume e ≥ 2 e \ge2 e ≥ 2 and n ≥ 1 n \ge1 n ≥ 1 . Replace its existential encoding by a = ( w , 1 − w ) ∈ { 0 , 1 } D a = (w,1-w) \in\{0,1\}^D a = ( w , 1 − w ) ∈ { 0 , 1 } D , where D = 2 e D = 2e D = 2 e . Let ψ ( a , z ) \psi(a,z) ψ ( a , z ) assert the original formula on the first e e e bits and all complementary-pair tests. Put p ( a ) = Pr z [ ψ ( a , z ) = 1 ] p(a) = \Pr_z[\psi(a,z)=1] p ( a ) = Pr z [ ψ ( a , z ) = 1 ] . Every assignment with positive acceptance probability is balanced. A YES source has some p ( a ) > 1 / 2 p(a) > 1/2 p ( a ) > 1/2 , whereas a NO source has 0 ≤ p ( a ) ≤ 1 / 2 0 \le p(a) \le1/2 0 ≤ p ( a ) ≤ 1/2 for every a a a . Use precisely the inventory and cap of Lemma 8.7 .
Represent each source random bit and one further bit u u u by the parity of its own block of m + 1 m+1 m + 1 fresh uniform continuous coordinates rounded at 1 / 2 1/2 1/2 , where m = D + 2 m = D+2 m = D + 2 is the number of inventory occurrences. There are S = ( n + 1 ) ( m + 1 ) S = (n+1)(m+1) S = ( n + 1 ) ( m + 1 ) share coordinates and total dimension d = D + S d = D+S d = D + S . For analysis let a ( x ) a(x) a ( x ) round the original coordinates, let F ∗ = u ∨ ψ ( a ( x ) , z ) F^* = u \vee\psi(a(x),z) F ∗ = u ∨ ψ ( a ( x ) , z ) , and set T ∗ = U ( x ) ( 2 F ∗ − 1 ) T^* = U(x)(2F^* - 1) T ∗ = U ( x ) ( 2 F ∗ − 1 ) . A candidate reads at most m m m distinct share coordinates because each input edge consumes an occurrence. Condition on the original x x x and on those read continuous values. Each parity block retains an independent unobserved uniform coordinate, whose rounded bit is fair. Its parity is therefore fair independently of all other blocks and of the conditioning. This is the independent-block argument of Lemma 7.3 , applied directly to the rounded unseen coordinates.
Consequently
E [ T ∗ ∣ x , read share values ] = U ( x ) p ( a ( x ) ) , ( T ∗ ) 2 = U ( x ) 2 . \mathbb{E}[T^* \mid x,\text{ read share values}] = U(x)p(a(x)), \qquad(T^*)^2 = U(x)^2. E [ T ∗ ∣ x , read share values ] = U ( x ) p ( a ( x )) , ( T ∗ ) 2 = U ( x ) 2 .
In a NO instance, Lemma 8.7 gives R T ∗ c o n t ( N ) ≥ E 0 R^{\mathrm{cont}}_{T^*}(N) \ge E_0 R T ∗ cont ( N ) ≥ E 0 for every legal candidate. In a YES instance use the balanced witness h a h_a h a . Its support lies at the selected corner and has squared norm I D I_D I D , so
R T ∗ c o n t ( h a ) = E 0 + ( 1 − 2 p ( a ) ) I D ≤ E 0 − 2 η , η = I D 2 − n , R^{\mathrm{cont}}_{T^*}(h_a) = E_0 + (1-2p(a))I_D \le E_0 - 2\eta, \qquad\eta= I_D2^{-n}, R T ∗ cont ( h a ) = E 0 + ( 1 − 2 p ( a )) I D ≤ E 0 − 2 η , η = I D 2 − n ,
since a strict majority exceeds one half by at least 2 − n 2^{-n} 2 − n .
Let 0 < ε ≤ 1 0 < \varepsilon\le1 0 < ε ≤ 1 be a rational transition width, to be chosen below. To obtain the actual target, decode each original coordinate by clip ( 2 x i − 1 / 2 ) \operatorname{clip}(2x_i - 1/2) clip ( 2 x i − 1/2 ) , where clip ( t ) = min { 1 , max { 0 , t } } \operatorname{clip}(t) = \min\{1,\max\{0,t\}\} clip ( t ) = min { 1 , max { 0 , t }} . This equals the corner bit wherever U > 0 U > 0 U > 0 , since those coordinates are within 1 / M ≤ 1 / 4 1/M \le1/4 1/ M ≤ 1/4 of a cube corner. Decode a share coordinate y y y by
clip ( ( y − 1 / 2 ) / ε + 1 / 2 ) . \operatorname{clip}\left((y-1/2)/\varepsilon+ 1/2\right). clip ( ( y − 1/2 ) / ε + 1/2 ) .
It equals the rounded bit outside a transition interval of length ε \varepsilon ε . Compile the parities and F = u ∨ ψ F = u \lor\psi F = u ∨ ψ by the AND/NOT ReLU identities in Theorem 7.1 , decomposing OR and XOR first. Every intermediate value remains in [ 0 , 1 ] [0,1] [ 0 , 1 ] , and agrees with its Boolean value outside the share transitions wherever U > 0 U > 0 U > 0 .
Supply the target
T = 2 ρ ( U + F − 1 ) − U . T = 2\rho(U+F-1)-U. T = 2 ρ ( U + F − 1 ) − U .
This has a polynomial-size rational ReLU graph. Indeed min { x i , 1 − x i } = x i − ρ ( 2 x i − 1 ) \min\{x_i,1-x_i\} = x_i-\rho(2x_i-1) min { x i , 1 − x i } = x i − ρ ( 2 x i − 1 ) , so U U U itself has such a graph, and clipping and the final displayed gate use affine operations and ReLUs. Signed intermediate values can be passed by the signed-pair construction of Lemma 5.1 . No inventory constraint is imposed on target parameters.
We have ∣ T ∣ ≤ U |T| \le U ∣ T ∣ ≤ U , and T = T ∗ T = T^* T = T ∗ except on the union of the share transitions, whose probability is at most S ε S\varepsilon S ε . Where U = 0 U = 0 U = 0 , both targets are zero regardless of the original decoders.
A uniform bound for all candidates on the cube is
∣ N ∣ ≤ H : = ( ( m + 1 ) ( e M + 1 ) ) m + 1 . |N| \le H := ((m+1)(eM+1))^{m+1}. ∣ N ∣ ≤ H := (( m + 1 ) ( e M + 1 ) ) m + 1 .
Topological induction gives this bound, since each affine preactivation has at most m m m terms, each parameter has magnitude at most e M eM e M , and there are at most m m m computational vertices. Thus this weaker bound suffices and has polynomial binary length. Since ∣ T ∣ , ∣ T ∗ ∣ ≤ 1 |T|,|T^*| \le1 ∣ T ∣ , ∣ T ∗ ∣ ≤ 1 , their squared losses differ by at most 4 ( H + 1 ) 4(H+1) 4 ( H + 1 ) on any input.
Choose
ε = η 16 S ( H + 1 ) . \varepsilon= \frac{\eta}{16S(H+1)}. ε = 16 S ( H + 1 ) η .
The two risks then differ by at most η / 4 \eta/4 η /4 , simultaneously for every candidate. Hence actual YES risk is at most E 0 − 7 η / 4 E_0-7\eta/4 E 0 − 7 η /4 , whereas every actual NO risk is at least E 0 − η / 4 E_0-\eta/4 E 0 − η /4 .
Set
τ = E 0 − 3 η / 2 , δ = η . \tau= E_0 - 3\eta/2, \qquad\delta= \eta. τ = E 0 − 3 η /2 , δ = η .
These are positive rational inputs with polynomial binary encodings; factorials and M D M^D M D have polynomial bit length. The YES risk is below τ \tau τ , and every NO risk is above τ + δ = E 0 − η / 2 \tau+\delta= E_0-\eta/2 τ + δ = E 0 − η /2 . All graphs, dimensions, occurrences and decoder coefficients are constructed in polynomial time. The YES witness consumes every occurrence and the NO argument covered at-most-once use, proving hardness for both inventory rules with cap two. Combining this reduction with Proposition 9.3 proves completeness. □ \square □
Discarding the gap field also gives N P P P \mathrm{NP}^{\mathrm{PP}} NP PP -hardness of exact continuous-risk synthesis. The binary gap above may be exponentially small. The result does not classify the inverse-gap clipped-loss problem, and it does not identify the exact-threshold upper bound of Theorem 8.2 with N P P P \mathrm{NP}^{\mathrm{PP}} NP PP .
Exact Continuous-Risk Synthesis
Exact synthesis under continuous uniform inputs is classified by the exact comparison class of Section 8 preceded by one discrete existential choice. For a class C \mathcal{C} C , write ∃ ⋅ C \exists\cdot\mathcal{C} ∃ ⋅ C for the languages L L L for which there are a polynomial s s s and a language B ∈ C B \in\mathcal{C} B ∈ C such that
a ∈ L ⟺ ∃ y ∈ { 0 , 1 } s ( ∣ a ∣ ) ( a , y ) ∈ B . a \in L \quad\Longleftrightarrow\quad\exists y \in\{0,1\}^{s(\lvert a\rvert)}\ (a,y) \in B. a ∈ L ⟺ ∃ y ∈ { 0 , 1 } s (∣ a ∣) ( a , y ) ∈ B .
This is one existential string followed by one test in C \mathcal{C} C . The definition does not identify this class with an adaptive oracle class. We use C = P P Q ( p t ) \mathcal{C} = \mathrm{PP}^{(\mathrm{pt})}_{\mathbb{Q}} C = PP Q ( pt ) , with the exact rational-weight convention specified in Section 8.
Theorem 8.8 (Exact continuous-risk synthesis; restatement of Theorem 1.4). Under continuous uniform inputs on [ 0 , 1 ] d [0,1]^d [ 0 , 1 ] d , squared-risk ReLU-network synthesis with a rational threshold is complete for ∃ ⋅ P P Q ( p t ) \exists\cdot\mathrm{PP}^{(\mathrm{pt})}_{\mathbb{Q}} ∃ ⋅ PP Q ( pt ) under deterministic polynomial-time many-one reductions, under either inventory rule. The target is an independently supplied rational ReLU network and the dimension is listed explicitly. Hardness holds with integer inventory and computational-node cap two, including the affine output. The hard targets have dyadic parameters and satisfy 0 ≤ T ≤ 1 0 \le T \le1 0 ≤ T ≤ 1 on the integration cube.
The proof needs a stronger version of the separation estimate in Lemma 8.7 . It allows a target mean μ \mu μ up to U U U instead of U / 2 U/2 U /2 , and its only exceptions are the balanced corner functions.
Lemma 8.9 (Separation from the balanced corner functions). Use the inventory, e , D , K , M , b , U e,D,K,M,b,U e , D , K , M , b , U , and corner functions of Lemma 8.7 . For any number of additional independent uniform coordinates z z z , any legal at-most-once candidate N ( x , z ) N(x,z) N ( x , z ) , and any measurable 0 ≤ μ ( x ) ≤ U ( x ) 0 \le\mu(x) \le U(x) 0 ≤ μ ( x ) ≤ U ( x ) ,
∫ ( N 2 − 2 N μ ) d x d z ≥ 0 , \int(N^2 - 2N\mu)\,dx\,dz \ge0, ∫ ( N 2 − 2 N μ ) d x d z ≥ 0 , unless the function computed by N N N is h a ( x ) h_a(x) h a ( x ) for a corner a ∈ { 0 , 1 } D a \in\{0,1\}^D a ∈ { 0 , 1 } D of Hamming weight e e e .
Proof. We retain the complete graph and occurrence-role classification in the proof of Lemma 8.7 . Write I j = 2 / ( ( j + 2 ) ! M j ) I_j = 2/((j+2)!M^j) I j = 2/ (( j + 2 )! M j ) and E 0 = ∥ U ∥ 2 2 = 2 D I D E_0 = \lVert U\rVert_2^2 = 2^D I_D E 0 = ∥ U ∥ 2 2 = 2 D I D . For the larger allowed mean, the Cauchy–Schwarz estimate of that proof becomes
∫ ( N 2 − 2 N μ ) ≥ ∥ N ∥ 2 ( ∥ N ∥ 2 − 2 ∥ U ∥ 2 ) . \int(N^2 - 2N\mu) \ge\lVert N\rVert_2(\lVert N\rVert_2 - 2\lVert U\rVert_2). ∫ ( N 2 − 2 N μ ) ≥ ∥ N ∥ 2 (∥ N ∥ 2 − 2 ∥ U ∥ 2 ) .
A nonzero integer-affine candidate, a candidate with neither formal affine branch zero, and each of the classified large-maximum cases have ∥ N ∥ 2 ≥ 1 / 16 \lVert N\rVert_2 \ge1/16 ∥ N ∥ 2 ≥ 1/16 . Here E 0 ≤ 2 D + 1 / M D < 1 / 1024 E_0 \le2^{D+1}/M^D < 1/1024 E 0 ≤ 2 D + 1 / M D < 1/1024 , so that estimate remains nonnegative. Zero functions and negative-gain functions satisfy the assertion pointwise. The active-branch-zero cases either vanish or have a maximum at least M M M , as established there.
For the remaining gain-M M M , maximum-one case, that proof gives
∥ N ∥ 2 2 ≥ M 2 − D 4 ( 2 D ( e + 1 ) ) D . \lVert N\rVert_2^2 \ge\frac{M^{2-D}}{4(2D(e+1))^D}. ∥ N ∥ 2 2 ≥ 4 ( 2 D ( e + 1 ) ) D M 2 − D .
This bound is at least 4 E 0 4E_0 4 E 0 whenever
M 2 ≥ 2 D + 5 ( 2 D ( e + 1 ) ) D . M^2 \ge2^{D+5}(2D(e+1))^D. M 2 ≥ 2 D + 5 ( 2 D ( e + 1 ) ) D .
The binary logarithm of the right side is at most 4 K 2 4K^2 4 K 2 , using D ≥ 4 D \ge4 D ≥ 4 , K = D + 2 K = D+2 K = D + 2 , and 2 D ( e + 1 ) = D ( D + 2 ) ≤ K 2 2D(e+1) = D(D+2) \le K^2 2 D ( e + 1 ) = D ( D + 2 ) ≤ K 2 ; the left side has logarithm 20 K 2 20K^2 20 K 2 . Thus Cauchy–Schwarz applies in this case as well.
Only the unit-gain candidates with maximum one remain; we call them templates . In that proof each template has the form ρ ( 1 − M ∑ j ∈ J dist j ) \rho(1-M\sum_{j\in J}\operatorname{dist}_j) ρ ( 1 − M ∑ j ∈ J dist j ) for its set J J J of selected coordinates. If a template selects s s s original and t t t auxiliary coordinates, put k = s + t ≤ D k=s+t\le D k = s + t ≤ D and l = D − s l=D-s l = D − s . Its squared norm is I k I_k I k . The exact mixed-coordinate calculation in that proof gives, unless s = D s=D s = D and t = 0 t=0 t = 0 ,
∫ N U I k = 2 l − 1 ( t + l + 2 ) ! ( s + t + 2 ) ! ( t + 1 ) ! ( l + 1 ) ! ( D + t + 2 ) ! M − l ≤ 2 3 D + 1 M < 1 2 . \frac{\int NU}{I_k}
=
\frac{2^{l-1}(t+l+2)!(s+t+2)!}{(t+1)!(l+1)!(D+t+2)!}M^{-l}
\le\frac{2^{3D+1}}{M} < \frac{1}{2}. I k ∫ N U = ( t + 1 )! ( l + 1 )! ( D + t + 2 )! 2 l − 1 ( t + l + 2 )! ( s + t + 2 )! M − l ≤ M 2 3 D + 1 < 2 1 .
Since these templates are nonnegative and μ ≤ U \mu\le U μ ≤ U , their integral ∫ ( N 2 − 2 N μ ) \int(N^2-2N\mu) ∫ ( N 2 − 2 N μ ) is at least I k − 2 ∫ N U ≥ 0 I_k-2\int NU \ge0 I k − 2 ∫ N U ≥ 0 . The exceptional templates use all D D D original coordinates and no auxiliary coordinate. In the occurrence classification of that proof, such a template is ρ ( 1 − M ∑ i = 1 D dist i ) \rho(1-M\sum_{i=1}^{D}\operatorname{dist}_{i}) ρ ( 1 − M ∑ i = 1 D dist i ) , where exactly e e e of the distances have the form 1 − x i 1-x_i 1 − x i and the others have the form x i x_i x i . Setting a i = 1 a_i=1 a i = 1 for the former coordinates and a i = 0 a_i=0 a i = 0 for the others gives dist i = ∣ x i − a i ∣ \operatorname{dist}_i=|x_i-a_i| dist i = ∣ x i − a i ∣ on the cube, so the template computes h a h_a h a with ∣ a ∣ = e |a|=e ∣ a ∣ = e . This exhausts every case of that classification, including direct skips, unused occurrences, biases, and arbitrary auxiliary-coordinate access. □
We next implement a uniform variable that stays independent of the raw coordinates a candidate can read. The underlying modular-sharing mechanism is classical [6 Section 4] ; its independent-share formulation is stated explicitly in [DC14, Scheme 3.1]. We give a continuous triangular realization and prove the conditional identity needed here.
Lemma 8.10. (A continuous uniform variable hidden from a bounded number of coordinates.) For an integer r ≥ 1 r \ge1 r ≥ 1 , define
Θ r ( t ) = 2 ρ ( t ) + 4 ∑ j = 1 2 r − 1 ( − 1 ) j ρ ( t − j / 2 ) + 2 ρ ( t − r ) . \Theta_r(t)=2\rho(t)+4\sum_{j=1}^{2r-1}(-1)^j\rho(t-j/2)+2\rho(t-r). Θ r ( t ) = 2 ρ ( t ) + 4 j = 1 ∑ 2 r − 1 ( − 1 ) j ρ ( t − j /2 ) + 2 ρ ( t − r ) . If Y 1 , … , Y r Y_1,\ldots,Y_r Y 1 , … , Y r are independent uniform variables on [ 0 , 1 ] [0,1] [ 0 , 1 ] , then Θ r ( ∑ j Y j ) \Theta_r(\sum_jY_j) Θ r ( ∑ j Y j ) is uniform on [ 0 , 1 ] [0,1] [ 0 , 1 ] and independent of any selected collection of at most r − 1 r-1 r − 1 of them. The function Θ r \Theta_r Θ r has a rational ReLU realization of size O ( r ) O(r) O ( r ) .
Proof. For each integer k ∈ { 0 , … , r − 1 } k\in\{0,\ldots,r-1\} k ∈ { 0 , … , r − 1 } , the displayed expression on [ k , k + 1 ] [k,k+1] [ k , k + 1 ] is the triangle with values zero, one, and zero at k , k + 1 / 2 , k + 1 k,k+1/2,k+1 k , k + 1/2 , k + 1 . It vanishes outside [ 0 , r ] [0,r] [ 0 , r ] . The slopes follow by adding the changes at successive half-integers; their magnitudes inside are two. Condition on every variable except one unobserved Y j Y_j Y j . The sum of the conditioned variables is a fixed c ∈ [ 0 , r − 1 ] c\in[0,r-1] c ∈ [ 0 , r − 1 ] . Because Θ r \Theta_r Θ r has period one on [ 0 , r ] [0,r] [ 0 , r ] and each triangle consists of two branches, every bounded measurable f f f satisfies
∫ 0 1 f ( Θ r ( c + t ) ) d t = ∫ 0 1 f ( Θ 1 ( t ) ) d t = ∫ 0 1 f ( u ) d u . \int_0^1 f(\Theta_r(c+t))\,dt=\int_0^1 f(\Theta_1(t))\,dt=\int_0^1 f(u)\,du. ∫ 0 1 f ( Θ r ( c + t )) d t = ∫ 0 1 f ( Θ 1 ( t )) d t = ∫ 0 1 f ( u ) d u .
Averaging over the other unobserved variables shows that, given the observed variables, Θ r ( ∑ j Y j ) \Theta_r(\sum_jY_j) Θ r ( ∑ j Y j ) is conditionally uniform on [ 0 , 1 ] [0,1] [ 0 , 1 ] , as asserted, and independent of them. The defining sum of 2 r + 1 2r+1 2 r + 1 ReLU terms gives the stated realization. □
Proof of Theorem 8.8. For membership, existentially specify a padded candidate graph and its occurrence assignment. Its length is polynomial, and legality is polynomial-time decidable under either rule. For a legal graph, Lemma 8.3 provides short rational leaves r w r_w r w summing to its risk. In a bitstring equality, 0 denotes the all-zero word of the required length, including the empty word when that length is zero. The single weighted comparison
∑ w ( − r w + τ 1 { w = 0 } ) ≥ 0 \sum_w(-r_w+\tau\mathbf{1}_{\{w=0\}})\ge0 w ∑ ( − r w + τ 1 { w = 0 } ) ≥ 0
holds exactly when the candidate’s risk is at most the threshold τ \tau τ . For an illegal certificate use constant negative weights. This proves membership directly in the stated existential closure.
For hardness, fix an input to any language in that closure. Its condition is an existential string y y y followed by a comparison of an exactly polynomial-time computable rational weighted sum with a rational threshold t = t 1 / t 2 t=t_1/t_2 t = t 1 / t 2 , where t 2 > 0 t_2>0 t 2 > 0 . If the counting word has length p p p , replace each weight u / v u/v u / v by
2 p t 2 u − t 1 v 2 p t 2 v . \frac{2^pt_2u-t_1v}{2^pt_2v}. 2 p t 2 v 2 p t 2 u − t 1 v .
Summing the new weights over all 2 p 2^p 2 p counting words subtracts t t t from the sum, and every new numerator and denominator still has polynomial binary length. Pad ignored existential coordinates so their number e e e is at least two. An ignored counting bit, with each weight divided by two, ensures p ≥ 1 p\ge1 p ≥ 1 without changing the sum. Choose a polynomially bounded integer q ≥ 1 q\ge1 q ≥ 1 such that the normalized evaluators satisfy
∣ u ( y , w ) ∣ ≤ Q : = 2 q , 1 ≤ v ( y , w ) ≤ Q . |u(y,w)|\le Q:=2^q,\qquad1\le v(y,w)\le Q. ∣ u ( y , w ) ∣ ≤ Q := 2 q , 1 ≤ v ( y , w ) ≤ Q .
We must therefore encode
∃ y ∈ { 0 , 1 } e F y ≥ 0 , F y = ∑ w ∈ { 0 , 1 } p u ( y , w ) v ( y , w ) . \exists y \in\{0,1\}^{e}\ F_y \ge0,\qquad F_y=\sum_{w\in\{0,1\}^{p}}\frac{u(y,w)}{v(y,w)}. ∃ y ∈ { 0 , 1 } e F y ≥ 0 , F y = w ∈ { 0 , 1 } p ∑ v ( y , w ) u ( y , w ) .
No positive lower bound on a nonzero ∣ F y ∣ |F_y| ∣ F y ∣ is assumed.
Use the balanced inventory of Lemma 8.7 , with D = 2 e D=2e D = 2 e and m = D + 2 m=D+2 m = D + 2 occurrences. For a ∈ { 0 , 1 } D a\in\{0,1\}^{D} a ∈ { 0 , 1 } D , use the source evaluators if a = ( y , 1 − y ) a=(y,1-y) a = ( y , 1 − y ) ; otherwise set u = − 1 , v = 1 u=-1,v=1 u = − 1 , v = 1 . Write the resulting sum as F a F_a F a , and put A = Q 2 p A=Q2^p A = Q 2 p . Every invalid code has F a = − 2 p < 0 F_a=-2^p<0 F a = − 2 p < 0 , and ∣ F a ∣ ≤ A |F_a|\le A ∣ F a ∣ ≤ A for every code. The source is YES precisely when some balanced code has F a ≥ 0 F_a\ge0 F a ≥ 0 .
Write clip ( t ) = ρ ( t ) − ρ ( t − 1 ) \operatorname{clip}(t)=\rho(t)-\rho(t-1) clip ( t ) = ρ ( t ) − ρ ( t − 1 ) . On the positive support of U U U , the functions
d i ( x ) = clip ( 2 x i − 1 / 2 ) d_i(x)=\operatorname{clip}(2x_i-1/2) d i ( x ) = clip ( 2 x i − 1/2 )
are exactly the bits of the unique corner supporting U U U . Indeed every coordinate is within 1 / M ≤ 1 / 4 1/M\le1/4 1/ M ≤ 1/4 of that corner. In this proof, I 2 I_2 I 2 and I 3 I_3 I 3 denote the second and third moments of a corner function h a h_a h a , displayed next; they are not the quantities I j = 2 / ( ( j + 2 ) ! M j ) I_j=2/((j+2)!M^j) I j = 2/ (( j + 2 )! M j ) of Lemma 8.9 at j = 2 , 3 j=2,3 j = 2 , 3 , whose subscript counts selected coordinates. The simplex formula used in Lemma 8.7 gives
I 2 = ∫ h a 2 = 2 ( D + 2 ) ! M D , I 3 = ∫ h a 3 = 6 ( D + 3 ) ! M D , I 3 I 2 = 3 D + 3 ≤ 3 7 . I_2=\int h_a^2=\frac{2}{(D+2)!M^D},\qquad I_3=\int h_a^3=\frac{6}{(D+3)!M^D},\qquad\frac{I_3}{I_2}=\frac{3}{D+3}\le\frac{3}{7}. I 2 = ∫ h a 2 = ( D + 2 )! M D 2 , I 3 = ∫ h a 3 = ( D + 3 )! M D 6 , I 2 I 3 = D + 3 3 ≤ 7 3 .
For j = 2 , 3 j=2,3 j = 2 , 3 , disjoint corner supports give R j U = 2 D I j R^U_j=2^D I_j R j U = 2 D I j .
We first construct a payload using a local uniform coordinate t ∈ [ 0 , 1 ] t\in[0,1] t ∈ [ 0 , 1 ] . Set b 1 = 2 q + 2 b_1=2q+2 b 1 = 2 q + 2 , n = p + b 1 n=p+b_1 n = p + b 1 , and η = 2 − n \eta=2^{-n} η = 2 − n . Use the tent decoder in the proof of Theorem 8.2 : the n n n itinerary bits, recalled below, encode ( w , j ) (w,j) ( w , j ) , where j j j is a b 1 b_1 b 1 -bit unsigned integer. For completeness, recall the tent map θ ( t ) = ρ ( 2 t ) − 2 ρ ( 2 t − 1 ) + ρ ( 2 t − 2 ) \theta(t)=\rho(2t)-2\rho(2t-1)+\rho(2t-2) θ ( t ) = ρ ( 2 t ) − 2 ρ ( 2 t − 1 ) + ρ ( 2 t − 2 ) of that proof, whose power θ i \theta^i θ i denotes iteration. Let z 0 = t z_0=t z 0 = t , z i = θ i ( t ) z_i=\theta^i(t) z i = θ i ( t ) , and s i = 2 z i − 1 s_i=2z_i-1 s i = 2 z i − 1 for 0 ≤ i < n 0\le i<n 0 ≤ i < n , and put r t = 1 − θ n + 1 ( t ) r_t=1-\theta^{n+1}(t) r t = 1 − θ n + 1 ( t ) . As in that proof, the i i i th itinerary bit is 1 { s i > 0 } \mathbf{1}_{\{s_i>0\}} 1 { s i > 0 } . With ϵ = η / 4 \epsilon=\eta/4 ϵ = η /4 , the ramps
β i = ρ ( s i + ϵ 2 ϵ ) − ρ ( s i − ϵ 2 ϵ ) \beta_i=\rho\left(\frac{s_i+\epsilon}{2\epsilon}\right)-\rho\left(\frac{s_i-\epsilon}{2\epsilon}\right) β i = ρ ( 2 ϵ s i + ϵ ) − ρ ( 2 ϵ s i − ϵ )
lie in [ 0 , 1 ] [0,1] [ 0 , 1 ] and equal the itinerary bits wherever r t ≤ 1 / 2 r_t\le1/2 r t ≤ 1/2 . The fine cells are the 2 n 2^n 2 n open dyadic cells of width η \eta η from that proof, which realize all strict sign words of ( s 0 , … , s n − 1 ) (s_0,\ldots,s_{n-1}) ( s 0 , … , s n − 1 ) exactly once. Each fine cell thus realizes one word and has two affine branches on which r t r_t r t traverses [ 0 , 1 ] [0,1] [ 0 , 1 ] . Thus integration on that cell is η \eta η times integration against a uniform r t r_t r t .
Compile the source evaluator, its validity test, the digits c k c_k c k of v − 1 v-1 v − 1 for 0 ≤ k < q 0\le k<q 0 ≤ k < q , and the selection predicate
J : j < Q v + u J:\quad j<Qv+u J : j < Q v + u
by Boolean AND and NOT gates, realized as in Theorem 8.2 . All gate values stay in [ 0 , 1 ] [0,1] [ 0 , 1 ] , even outside exact decoding regions. Since 0 ≤ Q v + u ≤ 2 Q 2 < 2 b 1 0\le Qv+u\le2Q^2<2^{b_1} 0 ≤ Q v + u ≤ 2 Q 2 < 2 b 1 , this predicate selects exactly Q v + u Qv+u Q v + u integers for each w w w . Define
P = r t + ∑ k = 0 q − 1 2 k min { r t , c k } , B U ( x , t ) = min { ρ ( U ( x ) − 4 P ) , J } . P=r_t+\sum_{k=0}^{q-1}2^k\min\{r_t,c_k\},\qquad B_U(x,t)=\min\{\rho(U(x)-4P),J\}. P = r t + k = 0 ∑ q − 1 2 k min { r t , c k } , B U ( x , t ) = min { ρ ( U ( x ) − 4 P ) , J } .
No product of two variable signals occurs in these formulas. If ρ ( U ( x ) − 4 P ) > 0 \rho(U(x)-4P)>0 ρ ( U ( x ) − 4 P ) > 0 , then r t ≤ P < U / 4 ≤ 1 / 4 r_t\le P<U/4\le1/4 r t ≤ P < U /4 ≤ 1/4 . Both the itinerary and original-coordinate decoders are then Boolean, so P = v r t P=vr_t P = v r t and J J J is the exact selection bit. Consequently the payload on a fine cell is ρ ( U − 4 v r t ) \rho(U-4vr_t) ρ ( U − 4 v r t ) if selected, and zero otherwise. Where ρ ( U ( x ) − 4 P ) = 0 \rho(U(x)-4P)=0 ρ ( U ( x ) − 4 P ) = 0 the payload vanishes; if U = 0 U=0 U = 0 it vanishes independently of every decoded value. It is continuous and zero at t = 0 , 1 t=0,1 t = 0 , 1 .
A selected cell contributes
∫ c e l l B U d t = η U 2 8 v , ∫ c e l l B U 2 d t = η U 3 12 v . \int_{\mathrm{cell}} B_U\,dt=\frac{\eta U^2}{8v},\qquad\int_{\mathrm{cell}} B_U^2\,dt=\frac{\eta U^3}{12v}. ∫ cell B U d t = 8 v η U 2 , ∫ cell B U 2 d t = 12 v η U 3 .
These follow by integrating U − 4 v r U-4vr U − 4 v r and its square over 0 ≤ r ≤ U / ( 4 v ) 0\le r\le U/(4v) 0 ≤ r ≤ U / ( 4 v ) . On the support of the corner a a a , summing selected words therefore gives
∫ 0 1 B U d t = η U 2 8 ( A + F a ) , ∫ 0 1 B U 2 d t = η U 3 12 ( A + F a ) . \int_0^1 B_U\,dt=\frac{\eta U^2}{8}(A+F_a),\qquad\int_0^1 B_U^2\,dt=\frac{\eta U^3}{12}(A+F_a). ∫ 0 1 B U d t = 8 η U 2 ( A + F a ) , ∫ 0 1 B U 2 d t = 12 η U 3 ( A + F a ) .
In particular 3 ∫ B U 2 = 2 U ∫ B U 3\int B_U^2=2U\int B_U 3 ∫ B U 2 = 2 U ∫ B U , exactly. For x ∈ [ 0 , 1 ] D x\in[0,1]^D x ∈ [ 0 , 1 ] D and s ∈ [ 0 , 1 ] s\in[0,1] s ∈ [ 0 , 1 ] , fix λ = 1 / 8 \lambda=1/8 λ = 1/8 and introduce the background
E U ( x , s ) = ρ ( ( s − 2 λ ) / λ ) − ρ ( ( s − 2 λ ) / λ − U ( x ) ) . \mathcal{E}_U(x,s)=\rho\left((s-2\lambda)/\lambda\right)-\rho\left((s-2\lambda)/\lambda-U(x)\right). E U ( x , s ) = ρ ( ( s − 2 λ ) / λ ) − ρ ( ( s − 2 λ ) / λ − U ( x ) ) .
As a function of s s s , it is zero on [ 0 , 2 λ ] [0,2\lambda] [ 0 , 2 λ ] , rises linearly to U U U , and equals U U U on [ 3 λ , 1 ] [3\lambda,1] [ 3 λ , 1 ] . Place three copies of the payload at
B 1 ( x , s ) = B U ( x , clip ( s / λ ) ) , B_1(x,s)=B_U\left(x,\operatorname{clip}(s/\lambda)\right), B 1 ( x , s ) = B U ( x , clip ( s / λ ) ) ,
B 2 ( x , s ) = B U ( x , clip ( ( s − λ ) / λ ) ) , B_2(x,s)=B_U\left(x,\operatorname{clip}((s-\lambda)/\lambda)\right), B 2 ( x , s ) = B U ( x , clip (( s − λ ) / λ ) ) ,
B 3 ( x , s ) = B U ( x , clip ( ( s − 3 λ ) / λ ) ) . B_3(x,s)=B_U\left(x,\operatorname{clip}((s-3\lambda)/\lambda)\right). B 3 ( x , s ) = B U ( x , clip (( s − 3 λ ) / λ ) ) .
and put T 0 = E U + B 1 + B 2 − B 3 T_0=\mathcal{E}_U+B_1+B_2-B_3 T 0 = E U + B 1 + B 2 − B 3 . Since the payload vanishes at t = 0 t=0 t = 0 and t = 1 t=1 t = 1 , these copies are supported in [ 0 , λ ] [0,\lambda] [ 0 , λ ] , [ λ , 2 λ ] [\lambda,2\lambda] [ λ , 2 λ ] , and [ 3 λ , 4 λ ] [3\lambda,4\lambda] [ 3 λ , 4 λ ] , respectively. The background is zero on the first two intervals and equals U U U on the third. Thus 0 ≤ T 0 ≤ U 0\le T_0\le U 0 ≤ T 0 ≤ U . Relative to the background alone, the copies change the energy ∫ 0 1 T 0 2 d s \int_0^1 T_0^2\,ds ∫ 0 1 T 0 2 d s by
λ ( 3 ∫ B U 2 − 2 U ∫ B U ) = 0. \lambda\left(3\int B_U^2-2U\int B_U\right)=0. λ ( 3 ∫ B U 2 − 2 U ∫ B U ) = 0.
Direct integration of the capped ramp gives
∫ 0 1 T 0 2 d s = ( 1 − 2 λ ) U 2 − 2 λ 3 U 3 , \int_0^1 T_0^2\,ds=(1-2\lambda)U^2-\frac{2\lambda}{3}U^3, ∫ 0 1 T 0 2 d s = ( 1 − 2 λ ) U 2 − 3 2 λ U 3 ,
μ ( x ) : = ∫ 0 1 T 0 d s = ( 1 − 2 λ ) U + c U 2 + λ η A 8 F a U 2 , \mu(x):=\int_0^1 T_0\,ds=(1-2\lambda)U+cU^2+\frac{\lambda\eta_A}{8}F_aU^2, μ ( x ) := ∫ 0 1 T 0 d s = ( 1 − 2 λ ) U + c U 2 + 8 λ η A F a U 2 ,
c = − λ 2 + λ η A 8 . c=-\frac{\lambda}{2}+\frac{\lambda\eta_A}{8}. c = − 2 λ + 8 λ η A .
The signed multiplicity of the payload copies in the first moment is 2 − 1 = 1 2-1=1 2 − 1 = 1 . The formula for μ \mu μ applies on each corner support; it is zero elsewhere, and 0 ≤ μ ≤ U 0\le\mu\le U 0 ≤ μ ≤ U .
Supply m + 1 m+1 m + 1 fresh raw uniform coordinates Y Y Y , and define the actual target
T ( x , Y ) = T 0 ( x , Θ m + 1 ( ∑ j = 1 m + 1 Y j ) ) . T(x,Y)=T_0\left(x,\Theta_{m+1}\left(\sum_{j=1}^{m+1}Y_j\right)\right). T ( x , Y ) = T 0 ( x , Θ m + 1 ( j = 1 ∑ m + 1 Y j ) ) .
Every candidate reads at most m m m distinct raw share coordinates, since each read requires an input-edge occurrence. By Lemma 8.10 , conditioning on all original coordinates and on every raw share value read by a candidate leaves the virtual coordinate uniform. Both displayed moments of T 0 T_0 T 0 therefore remain exact for T T T under this conditioning. The target’s energy is the known rational
R 0 = ∫ T 2 = 2 D ( 3 4 I 2 − 1 12 I 3 ) , \mathcal{R}_0=\int T^2=2^D\left(\frac{3}{4}I_2-\frac{1}{12}I_3\right), R 0 = ∫ T 2 = 2 D ( 4 3 I 2 − 12 1 I 3 ) ,
independent of all source sums F a F_a F a .
Choose the rational threshold
τ = R 0 + I 2 − 2 ( ( 1 − 2 λ ) I 2 + c I 3 ) = R 0 − 1 2 I 2 + λ ( 1 − η A 4 ) I 3 . \begin{aligned}
\tau=\mathcal{R}_0+I_2-2\left((1-2\lambda)I_2+cI_3\right) \\
&=\mathcal{R}_0-\frac{1}{2}I_2+\lambda\left(1-\frac{\eta_A}{4}\right)I_3.
\end{aligned} τ = R 0 + I 2 − 2 ( ( 1 − 2 λ ) I 2 + c I 3 ) = R 0 − 2 1 I 2 + λ ( 1 − 4 η A ) I 3 .
Here η A = 1 / ( 4 Q ) ≤ 1 / 8 \eta_A=1/(4Q)\le1/8 η A = 1/ ( 4 Q ) ≤ 1/8 , so the last correction is positive. Moreover,
R 0 − τ ≥ 25 56 I 2 > 0 , R 0 ≥ 2 D 5 7 I 2 ≥ 80 7 I 2 . \mathcal{R}_0-\tau\ge\frac{25}{56}I_2>0,\qquad\mathcal{R}_0\ge2^D\frac{5}{7}I_2\ge\frac{80}{7}I_2. R 0 − τ ≥ 56 25 I 2 > 0 , R 0 ≥ 2 D 7 5 I 2 ≥ 7 80 I 2 .
In particular 0 < τ < R 0 0<\tau<\mathcal{R}_0 0 < τ < R 0 .
For every candidate, the exact conditional first moment gives
R T c o n t ( N ) = R 0 + ∫ ( N 2 − 2 N μ ) . R_T^{\mathrm{cont}}(N)=\mathcal{R}_0+\int\left(N^2-2N\mu\right). R T cont ( N ) = R 0 + ∫ ( N 2 − 2 N μ ) .
Unless its function is a balanced original-coordinate h a h_a h a , Lemma 8.9 makes this risk at least R 0 > τ \mathcal{R}_0 > \tau R 0 > τ . For such a corner, U = h a U=h_a U = h a on its support, and the first-moment formula instead gives
R T c o n t ( h a ) = τ − λ η 4 I 3 F a . R_T^{\mathrm{cont}}(h_a)=\tau-\frac{\lambda\eta}{4}I_3F_a. R T cont ( h a ) = τ − 4 λ η I 3 F a .
A YES source therefore has a corner witness of risk at most τ \tau τ ; if its F a = 0 F_a=0 F a = 0 , the corner risk is exactly τ \tau τ , preserving the non-strict convention without a margin assumption. For a NO source, every valid and invalid code has F a < 0 F_a<0 F a < 0 , so each corner has risk strictly above τ \tau τ ; every other candidate was already excluded. The YES witness consumes every occurrence, whereas the NO argument covered the at-most-once superset. This proves hardness under both rules.
Finally, the source evaluators unroll into polynomial-size Boolean circuits, and their ReLU compilation uses only polynomially many positions. The fold depth n n n , decoder slopes, Q , M Q,M Q , M , factorials in the moments, and the threshold all have polynomial binary encoding length. The dimension is D + m + 1 D+m+1 D + m + 1 , listed explicitly. The three payload copies and continuous sharing use polynomially many rational target parameters. Minimum, clipping, and signed intermediate expressions have the ordinary ReLU implementations described in Theorem 8.2 ; repeated predecessors are combined, so no parallel edge is needed. The target inventory is independent of the supplied candidate inventory. Thus this is a polynomial-time many-one reduction to the exact synthesis problem. □
Synthesis under Gaussian Inputs
Changing the input probability distribution preserves the occurrence constraint but changes the arithmetic of exact loss. Let γ d \gamma_d γ d be the standard Gaussian distribution on R d \mathbb{R}^d R d , with density
ψ d ( x ) = ( 2 π ) − d / 2 exp ( − ∥ x ∥ 2 2 / 2 ) . \psi_d(x)=(2\pi)^{-d/2}\exp(-\lVert x\rVert_2^2/2). ψ d ( x ) = ( 2 π ) − d /2 exp ( − ∥ x ∥ 2 2 /2 ) .
We use the vector norms defined in § 2 , so ∥ x ∥ 2 2 = ∑ i x i 2 \lVert x\rVert_2^2=\sum_i x_i^2 ∥ x ∥ 2 2 = ∑ i x i 2 . The one-dimensional density and its upper tail are
ϕ ( t ) = ψ 1 ( t ) = ( 2 π ) − 1 / 2 e − t 2 / 2 , Φ ‾ ( t ) = ∫ t ∞ ϕ ( u ) d u . \phi(t)=\psi_1(t)=(2\pi)^{-1/2}e^{-t^2/2},\qquad\overline{\Phi}(t)=\int_t^\infty\phi(u)\,du. ϕ ( t ) = ψ 1 ( t ) = ( 2 π ) − 1/2 e − t 2 /2 , Φ ( t ) = ∫ t ∞ ϕ ( u ) d u .
For a real random variable V V V with E [ V 2 ] < ∞ \mathbb{E}[V^2]<\infty E [ V 2 ] < ∞ , its mean is E [ V ] \mathbb{E}[V] E [ V ] and its variance is E [ ( V − E [ V ] ) 2 ] \mathbb{E}[(V-\mathbb{E}[V])^2] E [( V − E [ V ] ) 2 ] . It is centered when its mean is zero. For v > 0 v>0 v > 0 , a centered normal of variance v v v has density
f v ( t ) = ( 2 π v ) − 1 / 2 exp ( − t 2 / ( 2 v ) ) . f_v(t)=(2\pi v)^{-1/2}\exp(-t^2/(2v)). f v ( t ) = ( 2 π v ) − 1/2 exp ( − t 2 / ( 2 v )) .
This is the density of v Z \sqrt{v}Z v Z for a standard normal Z Z Z ; the moment calculation below gives its stated mean and variance.
Independent coordinates have product density. If independent centered normals have positive variances s , t s,t s , t , the density of their sum at r r r is ∫ R f s ( x ) f t ( r − x ) d x \int_{\mathbb{R}}f_s(x)f_t(r-x)\,dx ∫ R f s ( x ) f t ( r − x ) d x , by substituting r = x + y r=x+y r = x + y in that product density. The identity
x 2 s + ( r − x ) 2 t = s + t s t ( x − s r s + t ) 2 + r 2 s + t \frac{x^2}{s}+\frac{(r-x)^2}{t}=\frac{s+t}{st}\left(x-\frac{sr}{s+t}\right)^2+\frac{r^2}{s+t} s x 2 + t ( r − x ) 2 = s t s + t ( x − s + t sr ) 2 + s + t r 2
shows that this integral is f s + t ( r ) f_{s+t}(r) f s + t ( r ) : the remaining Gaussian integral is 2 π s t / ( s + t ) \sqrt{2\pi st/(s+t)} 2 π s t / ( s + t ) . Scaling a standard normal by a nonzero real a a a gives density f a 2 f_{a^2} f a 2 , also for negative a a a because ϕ \phi ϕ is even. For independent standard normal X i X_i X i , repeatedly adding these independent summands proves that ∑ i a i X i \sum_i a_iX_i ∑ i a i X i is a centered normal of variance ∑ i a i 2 \sum_i a_i^2 ∑ i a i 2 whenever the coefficient vector is nonzero. If every coefficient is zero, the sum is identically zero. Functions of disjoint sets of independent coordinates remain independent, because their joint probabilities factor under the product probability distribution.
Evenness of ϕ \phi ϕ makes every integrable odd moment of Z Z Z zero. Since ϕ ′ ( z ) = − z ϕ ( z ) \phi'(z)=-z\phi(z) ϕ ′ ( z ) = − z ϕ ( z ) , integration by parts gives
∫ R z m ϕ ( z ) d z = ( m − 1 ) ∫ R z m − 2 ϕ ( z ) d z ( m = 2 , 4 ) . \int_{\mathbb{R}}z^m\phi(z)\,dz=(m-1)\int_{\mathbb{R}}z^{m-2}\phi(z)\,dz\qquad(m=2,4). ∫ R z m ϕ ( z ) d z = ( m − 1 ) ∫ R z m − 2 ϕ ( z ) d z ( m = 2 , 4 ) .
The boundary term z m − 1 ϕ ( z ) z^{m-1}\phi(z) z m − 1 ϕ ( z ) vanishes at both infinities, since Gaussian exponential decay dominates each fixed power. Starting from ∫ ϕ = 1 \int\phi=1 ∫ ϕ = 1 , this yields E [ Z 2 ] = 1 \mathbb{E}[Z^2]=1 E [ Z 2 ] = 1 and E [ Z 4 ] = 3 \mathbb{E}[Z^4]=3 E [ Z 4 ] = 3 , and hence the claimed variances after scaling.
For square-integrable functions, meaning E X ∼ γ d [ f ( X ) 2 ] < ∞ \mathbb{E}_{X\sim\gamma_d}[f(X)^2]<\infty E X ∼ γ d [ f ( X ) 2 ] < ∞ , write
∥ f ∥ 2 , γ d 2 = E X ∼ γ d [ f ( X ) 2 ] , ⟨ f , g ⟩ γ d = E X ∼ γ d [ f ( X ) g ( X ) ] . \|f\|_{2,\gamma_d}^2=\mathbb{E}_{X\sim\gamma_d}[f(X)^2],\qquad\langle f,g\rangle_{\gamma_d}=\mathbb{E}_{X\sim\gamma_d}[f(X)g(X)]. ∥ f ∥ 2 , γ d 2 = E X ∼ γ d [ f ( X ) 2 ] , ⟨ f , g ⟩ γ d = E X ∼ γ d [ f ( X ) g ( X )] .
We abbreviate the function norm to ∥ f ∥ 2 \|f\|_2 ∥ f ∥ 2 within the Gaussian arguments; vector norms retain their coordinate definitions above. Functions are orthogonal when this inner product is zero.
For a supplied rational ReLU target T T T and a permitted candidate N N N , define
R T G ( N ) = E X ∼ γ d [ ( N ( X ) − T ( X ) ) 2 ] . R_T^{\mathrm{G}}(N)=\mathbb{E}_{X\sim\gamma_d}[(N(X)-T(X))^2]. R T G ( N ) = E X ∼ γ d [( N ( X ) − T ( X ) ) 2 ] .
Gaussian-risk synthesis asks whether some legal occurrence assignment has risk at most a supplied rational threshold. All statements below concern this synthesis question; calculations for a fixed candidate are supporting algorithms. Gaussian agnostic learning of a freely chosen ReLU has different input and output requirements [16 ] . Weighted model integration also distinguishes its exact integration procedures from Gaussian Monte Carlo approximation [30 Section IV-B] .
Exact Comparison and Effective Approximation
Finite candidate enumeration does not by itself decide a non-strict inequality between computable real risks. A real number is computable here if an algorithm produces a rational interval of arbitrarily prescribed width containing it. A language is recursively enumerable if an algorithm halts and accepts exactly its members, with no termination requirement on other inputs. A language whose complement is recursively enumerable is called co-recursively enumerable. We use these notions only for the following exact synthesis bound, without claiming a running-time bound.
A function f f f is K K K -Lipschitz for a vector norm ∥ ⋅ ∥ \|\cdot\| ∥ ⋅ ∥ if ∣ f ( x ) − f ( y ) ∣ ≤ K ∥ x − y ∥ |f(x)-f(y)|\le K\|x-y\| ∣ f ( x ) − f ( y ) ∣ ≤ K ∥ x − y ∥ for every pair of inputs. The following growth estimate is used both for the semidecision bounds below and for the accuracy-gap result of Proposition 9.3.
Lemma 9.1 (Uniform growth and Lipschitz bounds). From a synthesis input and its rational ReLU target, one can compute a positive integer H H H of polynomial binary length such that every legal candidate and the target satisfy
∣ N ( x ) ∣ , ∣ T ( x ) ∣ ≤ H ( 1 + ∥ x ∥ ∞ ) , |N(x)|,|T(x)|\le H(1+\|x\|_\infty), ∣ N ( x ) ∣ , ∣ T ( x ) ∣ ≤ H ( 1 + ∥ x ∥ ∞ ) , and both functions have Lipschitz constant at most H H H for the infinity norm. Thus F = N − T F=N-T F = N − T has Lipschitz constant at most 2 H 2H 2 H and ∣ F ( x ) ∣ ≤ 2 H ( 1 + ∥ x ∥ ∞ ) |F(x)|\le2H(1+\|x\|_\infty) ∣ F ( x ) ∣ ≤ 2 H ( 1 + ∥ x ∥ ∞ ) .
Proof. Let L ≥ 2 L\ge2 L ≥ 2 bound the complete input length, all vertex and parameter-position counts, and the explicit dimension. Candidate counts are bounded by the inventory length, even if the supplied cap is larger. Every target and inventory parameter has absolute value at most 2 L 2^L 2 L . Put A = 1 + ( L + 1 ) 2 L A=1+(L+1)2^L A = 1 + ( L + 1 ) 2 L and H = A L + 1 H=A^{L+1} H = A L + 1 . In topological order, if previous growth and Lipschitz bounds are at most A j A^j A j , each next affine preactivation has at most L L L incoming terms and one bias, and has bounds at most A j + 1 A^{j+1} A j + 1 . The ReLU is one-Lipschitz and maps zero to zero, so it preserves these bounds. The affine output obeys the same estimate. There are at most L L L computational vertices in each graph; inputs start with bound one. The chosen H H H therefore suffices uniformly, and log 2 H = O ( L 2 + L log ( L + 1 ) ) \log_2 H=O(L^2+L\log(L+1)) log 2 H = O ( L 2 + L log ( L + 1 )) . The difference bounds follow by the triangle inequality.
The growth bound makes Gaussian tail error effectively controllable. This yields terminating certificates for strict inequalities on the appropriate side.
Proposition 9.2 (Semidecision bounds for Gaussian synthesis). Under either inventory rule, Gaussian-risk synthesis with a non-strict rational threshold is co-recursively enumerable; the version with a strict threshold is recursively enumerable. The zero-threshold non-strict problem is exactly full-space exact synthesis.
Proof. There are effectively finitely many legal graphs and occurrence assignments: each computational vertex consumes a distinct occurrence through an incoming position or bias, so a legal graph has at most min ( m , n max ) \min(m,n_{\max}) min ( m , n m a x ) computational vertices, where n max n_{\max} n m a x is the cap. They can all be enumerated and their legality tested. We show that each risk is uniformly computable. Put A 0 = 2 H A_0 = 2H A 0 = 2 H from Lemma 9.1. The squared difference is bounded by 2 A 0 2 ( 1 + ∥ x ∥ 2 2 ) 2A_0^2(1+\lVert x\rVert_2^2) 2 A 0 2 ( 1 + ∥ x ∥ 2 2 ) . For an integer B ≥ 1 B \ge1 B ≥ 1 , integration by parts gives
Pr [ ∣ Z ∣ > B ] ≤ 2 ϕ ( B ) B , E [ Z 2 1 { ∣ Z ∣ > B } ] ≤ 2 ϕ ( B ) ( B + B − 1 ) \Pr[|Z| > B] \le\frac{2\phi(B)}{B}, \qquad\mathbb{E}[Z^2\mathbf{1}_{\{|Z|>B\}}] \le2\phi(B)(B+B^{-1}) Pr [ ∣ Z ∣ > B ] ≤ B 2 ϕ ( B ) , E [ Z 2 1 { ∣ Z ∣ > B } ] ≤ 2 ϕ ( B ) ( B + B − 1 )
for a standard normal Z Z Z . A union bound over a coordinate with ∣ x i ∣ > B |x_i| > B ∣ x i ∣ > B , using independence for the squares of the other coordinates, bounds the contribution to the risk from outside [ − B , B ] d [-B,B]^d [ − B , B ] d by
4 d A 0 2 ϕ ( B ) ( B + d + 1 B ) . 4dA_0^2\phi(B)\left(B+\frac{d+1}{B}\right). 4 d A 0 2 ϕ ( B ) ( B + B d + 1 ) .
This bound tends to zero as B B B grows, at a computable rate. On the compact box, the squared difference and Gaussian density have computable Lipschitz constants, so rational grid sums with certified density approximations provide arbitrarily small error. The explicit implementation in Proposition 9.3 below supplies such approximations without any exact transcendental comparison. Together these give rational upper and lower bounds of arbitrary precision for every risk.
For non-strict synthesis, run these approximations on the finite list until every lower bound exceeds the threshold. This accepts exactly the complement; an empty candidate list is accepted immediately. For strict synthesis, accept when some upper bound falls below the threshold. These procedures terminate on precisely the claimed sides, including negative thresholds. At threshold zero, continuity and the strictly positive Gaussian density imply that zero risk is equivalent to equality everywhere: a nonzero difference at one point has nonzero magnitude on a neighborhood of positive Gaussian measure. □
A Binary-Encoded Accuracy Gap
We use the binary-gap synthesis promise defined in Section 8.2, for either the continuous uniform probability distribution or the standard Gaussian probability distribution. Let τ \tau τ and δ > 0 \delta> 0 δ > 0 be its binary rational threshold and gap. Exact threshold equality is avoided by the separation between its promised YES and NO risks.
Proposition 9.3 (Synthesis with a binary gap). Under either inventory rule, the stated risk promise problem has a separator in N P P P \mathrm{NP}^{\mathrm{PP}} NP PP , for continuous uniform inputs on [ 0 , 1 ] d [0,1]^d [ 0 , 1 ] d and for standard Gaussian inputs on R d \mathbb{R}^d R d .
Proof. Use the uniform bound H H H of Lemma 9.1, which is computable before a candidate is guessed. In this proof, R ( N ) R(N) R ( N ) denotes R T c o n t ( N ) R_T^{\mathrm{cont}}(N) R T cont ( N ) or R T G ( N ) R_T^{\mathrm{G}}(N) R T G ( N ) , according to the specified input probability distribution. For a fixed legal candidate N N N , set F = N − T F=N-T F = N − T and f = F 2 f=F^2 f = F 2 . We construct a polynomial-time fair-bit experiment with an acceptance probability p N p_N p N and a known positive rational multiplier D D D such that
∣ D p N − R ( N ) ∣ ≤ δ / 4. |Dp_N-R(N)| \le\delta/4. ∣ D p N − R ( N ) ∣ ≤ δ /4.
Both D D D and its inverse have polynomial binary length. The test p N ≤ ( τ + δ / 2 ) / D p_N \le(\tau+\delta/2)/D p N ≤ ( τ + δ /2 ) / D is a total PP predicate by the exact counting-threshold transformation in Theorem 7.1; thresholds outside [ 0 , 1 ] [0,1] [ 0 , 1 ] are decided directly. A YES witness makes this predicate accept, and every candidate on a NO instance makes it reject. Guessing the legal graph and using that total predicate proves the separator claim, without making a promise about individual oracle queries.
For uniform inputs, f ≤ M = 16 H 2 f \le M = 16H^2 f ≤ M = 16 H 2 on the cube and its infinity-norm Lipschitz constant is at most 16 H 2 16H^2 16 H 2 . Couple a point X X X to the grid point with coordinates Y i = 2 − b ⌊ 2 b X i ⌋ Y_i=2^{-b}\lfloor2^bX_i\rfloor Y i = 2 − b ⌊ 2 b X i ⌋ . Then ∣ E [ f ( X ) ] − E [ f ( Y ) ] ∣ ≤ 16 H 2 2 − b \lvert\mathbb{E}[f(X)]-\mathbb{E}[f(Y)]\rvert\le16H^2 2^{-b} ∣ E [ f ( X )] − E [ f ( Y )]∣ ≤ 16 H 2 2 − b . Choose b b b to make this at most δ / 8 \delta/8 δ /8 . The grid point is sampled with fair bits, and f ( Y ) f(Y) f ( Y ) is exactly computable in polynomial time. With an independent uniform s s s -bit integer V V V , accept when
V < ⌊ 2 s f ( Y ) M ⌋ . V < \left\lfloor\frac{2^s f(Y)}{M} \right\rfloor. V < ⌊ M 2 s f ( Y ) ⌋ .
Taking M 2 − s ≤ δ / 8 M2^{-s} \le\delta/8 M 2 − s ≤ δ /8 gives the required approximation with D = M D=M D = M . The chosen b , s b,s b , s are polynomial in the input length because H H H has polynomial binary length and log 2 ( 1 / δ ) \log_2(1/\delta) log 2 ( 1/ δ ) is at most polynomial in the binary length of the positive rational δ \delta δ .
For Gaussian inputs put A 0 = 2 H A_0=2H A 0 = 2 H , choose k k k with A 0 ≤ 2 k A_0 \le2^k A 0 ≤ 2 k , and let b δ ≥ 1 b_\delta\ge1 b δ ≥ 1 be a bit-length bound giving δ ≥ 2 − b δ \delta\ge2^{-b_\delta} δ ≥ 2 − b δ . Choose
B = 8 ( k + b δ + d + 2 ) . (4) B = 8(k+b_\delta+d+2). \tag*{(4)} B = 8 ( k + b δ + d + 2 ) . (4)
The tail estimate in Proposition 9.2 is at most δ / 8 \delta/8 δ /8 . Indeed ϕ ( B ) ≤ 2 − B 2 / 2 \phi(B) \le2^{-B^2/2} ϕ ( B ) ≤ 2 − B 2 /2 , and the prefactor 4 d A 0 2 ( B + ( d + 1 ) / B ) 4dA_0^2(B+(d+1)/B) 4 d A 0 2 ( B + ( d + 1 ) / B ) has base-two logarithm at most 2 k + 2 d + B + 3 2k+2d+B+3 2 k + 2 d + B + 3 . The chosen B B B makes this logarithm minus B 2 / 2 B^2/2 B 2 /2 at most − b δ − 3 -b_\delta-3 − b δ − 3 .
On [ − B , B ] d [-B,B]^d [ − B , B ] d , put U = 2 H ( B + 1 ) U=2H(B+1) U = 2 H ( B + 1 ) and V B = ( 2 B ) d V_B=(2B)^d V B = ( 2 B ) d . The function g = f ψ d g=f\psi_d g = f ψ d is bounded by U 2 U^2 U 2 and has infinity-norm Lipschitz constant at most
K G = 4 H U + U 2 d B . K_G=4HU+U^2dB. K G = 4 H U + U 2 d B .
Here f f f has Lipschitz constant at most 4 H U 4HU 4 H U and ψ d ≤ 1 \psi_d\le1 ψ d ≤ 1 . The gradient ∇ ψ d = ( ∂ ψ d / ∂ x i ) i = 1 d \nabla\psi_d=(\partial\psi_d/\partial x_i)_{i=1}^d ∇ ψ d = ( ∂ ψ d / ∂ x i ) i = 1 d of the density satisfies ∥ ∇ ψ d ∥ 1 ≤ d B \|\nabla\psi_d\|_1\le dB ∥∇ ψ d ∥ 1 ≤ d B on the box, where ∥ v ∥ 1 = ∑ i ∣ v i ∣ \|v\|_1=\sum_i|v_i| ∥ v ∥ 1 = ∑ i ∣ v i ∣ for a vector v v v . A uniform dyadic grid has mesh 2 B 2 − b 2B2^{-b} 2 B 2 − b ; choose b b b so that V B K G 2 B 2 − b ≤ δ / 32 V_BK_G2B2^{-b}\le\delta/32 V B K G 2 B 2 − b ≤ δ /32 . At every sampled point, approximate g g g by a rational g ~ ∈ [ 0 , U 2 ] \widetilde g\in[0,U^2] g ∈ [ 0 , U 2 ] to error at most δ / ( 32 V B ) \delta/(32V_B) δ / ( 32 V B ) . It suffices to approximate the density to that error divided by U 2 U^2 U 2 , then multiply by the exactly computed f f f ; clipping to the stated interval does not increase the error. Finally, as in the uniform case, draw the grid point Y Y Y with fair bits and an independent uniform s s s -bit integer V V V , and accept when
V < ⌊ 2 s g ~ ( Y ) M ⌋ , M = U 2 + 1 , V<\left\lfloor\frac{2^s\widetilde g(Y)}{M}\right\rfloor,\qquad M=U^2+1, V < ⌊ M 2 s g ( Y ) ⌋ , M = U 2 + 1 ,
with V B M 2 − s ≤ δ / 32 V_BM2^{-s}\le\delta/32 V B M 2 − s ≤ δ /32 . Since 0 ≤ g ~ < M 0\le\widetilde g<M 0 ≤ g < M , the acceptance probability p N p_N p N differs from E [ g ~ ( Y ) ] / M \mathbb{E}[\widetilde g(Y)]/M E [ g ( Y )] / M by at most 2 − s 2^{-s} 2 − s . The multiplier is now D = V B M D=V_BM D = V B M . The tail, grid, density, and vertical errors total at most δ / 8 + 3 δ / 32 = 7 δ / 32 < δ / 4 \delta/8+3\delta/32=7\delta/32<\delta/4 δ /8 + 3 δ /32 = 7 δ /32 < δ /4 . All bounds and precision choices have polynomial binary length.
For clarity, the density approximations themselves have polynomial bit running time. At a grid point the rational q = ∥ Y ∥ 2 2 / 2 q=\|Y\|_2^2/2 q = ∥ Y ∥ 2 2 /2 has polynomial numerical magnitude, at most d B 2 / 2 dB^2/2 d B 2 /2 . For x ≥ 0 x\ge0 x ≥ 0 and an integer n ≥ 0 n\ge0 n ≥ 0 , let S n ( x ) = ∑ i = 0 n ( − x ) i / i ! S_n(x)=\sum_{i=0}^n(-x)^i/i! S n ( x ) = ∑ i = 0 n ( − x ) i / i ! . Repeated integration by parts gives the exact remainder
exp ( − x ) − S n ( x ) = ( − 1 ) n + 1 n ! ∫ 0 x exp ( − t ) ( x − t ) n d t . \exp(-x)-S_n(x)=\frac{(-1)^{n+1}}{n!}\int_0^x\exp(-t)(x-t)^n\,dt. exp ( − x ) − S n ( x ) = n ! ( − 1 ) n + 1 ∫ 0 x exp ( − t ) ( x − t ) n d t .
Its absolute value is at most x n + 1 / ( n + 1 ) ! x^{n+1}/(n+1)! x n + 1 / ( n + 1 )! . The sign of the remainder shows that S 2 k + 1 ( x ) ≤ exp ( − x ) ≤ S 2 k ( x ) S_{2k+1}(x)\le\exp(-x)\le S_{2k}(x) S 2 k + 1 ( x ) ≤ exp ( − x ) ≤ S 2 k ( x ) for integers k ≥ 0 k\ge0 k ≥ 0 , and the two bounds differ by x 2 k + 1 / ( 2 k + 1 ) ! x^{2k+1}/(2k+1)! x 2 k + 1 / ( 2 k + 1 )! . When 0 ≤ x ≤ 1 / 2 0\le x\le1/2 0 ≤ x ≤ 1/2 , this difference is at most 2 − 2 k − 1 2^{-2k-1} 2 − 2 k − 1 . Choose the least integer j ≥ 0 j\ge0 j ≥ 0 with q / 2 j ≤ 1 / 2 q/2^j\le1/2 q / 2 j ≤ 1/2 , apply these bounds there, and clip their endpoints to [ 0 , 1 ] [0,1] [ 0 , 1 ] . Squaring the endpoints j j j times encloses exp ( − q ) \exp(-q) exp ( − q ) . Each squaring increases their distance by at most a factor of two, so the total amplification is at most 2 j ≤ max { 1 , 4 q } 2^j\le\max\{1,4q\} 2 j ≤ max { 1 , 4 q } . Polynomially many terms and polynomial extra precision therefore suffice; exact rational numerator and denominator lengths stay polynomial, including after the j j j squarings.
For completeness, n ! ≥ ( n / 3 ) n n!\ge(n/3)^n n ! ≥ ( n /3 ) n for integers n ≥ 1 n\ge1 n ≥ 1 . The binomial expansion gives
( 1 + 1 / n ) n = ∑ i = 0 n n ( n − 1 ) ⋯ ( n − i + 1 ) i ! n i ≤ ∑ i = 0 n 1 i ! < 3 , (1+1/n)^n=\sum_{i=0}^n\frac{n(n-1)\cdots(n-i+1)}{i!n^i}\le\sum_{i=0}^n\frac1{i!}<3, ( 1 + 1/ n ) n = i = 0 ∑ n i ! n i n ( n − 1 ) ⋯ ( n − i + 1 ) ≤ i = 0 ∑ n i ! 1 < 3 ,
where i ! ≥ 2 i − 1 i!\ge2^{i-1} i ! ≥ 2 i − 1 for i ≥ 2 i\ge2 i ≥ 2 bounds the tail by a geometric sum. The factorial estimate now follows by induction from 1 ! ≥ 1 / 3 1!\ge1/3 1 ! ≥ 1/3 : multiplication by n + 1 n+1 n + 1 preserves it because ( n + 1 ) ( n / 3 ) n ≥ ( ( n + 1 ) / 3 ) n + 1 (n+1)(n/3)^n\ge((n+1)/3)^{n+1} ( n + 1 ) ( n /3 ) n ≥ (( n + 1 ) /3 ) n + 1 . This supplies the factorial estimate used in the later Gaussian calibration.
The density constant ( 2 π ) − d / 2 (2\pi)^{-d/2} ( 2 π ) − d /2 also requires rational enclosures of π \pi π . Use the integral definition
arctan x = ∫ 0 x ( 1 + t 2 ) − 1 d t \arctan x=\int_0^x(1+t^2)^{-1}\,dt arctan x = ∫ 0 x ( 1 + t 2 ) − 1 d t
and the usual analytic normalization π = 4 arctan 1 \pi=4\arctan1 π = 4 arctan 1 . For 0 ≤ x < 1 0\le x<1 0 ≤ x < 1 , integrating the finite geometric identity gives, for integers m ≥ 1 m\ge1 m ≥ 1 ,
arctan x = ∑ i = 0 m − 1 ( − 1 ) i x 2 i + 1 2 i + 1 + ( − 1 ) m ∫ 0 x t 2 m 1 + t 2 d t . \arctan x=\sum_{i=0}^{m-1}\frac{(-1)^ix^{2i+1}}{2i+1}+(-1)^m\int_0^x\frac{t^{2m}}{1+t^2}\,dt. arctan x = i = 0 ∑ m − 1 2 i + 1 ( − 1 ) i x 2 i + 1 + ( − 1 ) m ∫ 0 x 1 + t 2 t 2 m d t .
The last term has absolute value at most x 2 m + 1 / ( 2 m + 1 ) x^{2m+1}/(2m+1) x 2 m + 1 / ( 2 m + 1 ) . Machin’s identity states that
π = 16 arctan ( 1 / 5 ) − 4 arctan ( 1 / 239 ) . \pi=16\arctan(1/5)-4\arctan(1/239). π = 16 arctan ( 1/5 ) − 4 arctan ( 1/239 ) .
To verify it, write A ( x ) = arctan x A(x)=\arctan x A ( x ) = arctan x . For nonnegative x , y x,y x , y with x y < 1 xy<1 x y < 1 , A ( x ) + A ( y ) = A ( x + y 1 − x y ) A(x)+A(y)=A\left(\frac{x+y}{1-xy}\right) A ( x ) + A ( y ) = A ( 1 − x y x + y ) : differentiating both sides in y y y gives 1 1 + y 2 \frac{1}{1+y^2} 1 + y 2 1 , and the sides agree at y = 0 y=0 y = 0 . Consequently,
2 A ( 1 / 5 ) = A ( 5 / 12 ) , 4 A ( 1 / 5 ) = A ( 120 / 119 ) , A ( 1 ) + A ( 1 / 239 ) = A ( 120 / 119 ) . 2A(1/5)=A(5/12),\qquad4A(1/5)=A(120/119),\qquad A(1)+A(1/239)=A(120/119). 2 A ( 1/5 ) = A ( 5/12 ) , 4 A ( 1/5 ) = A ( 120/119 ) , A ( 1 ) + A ( 1/239 ) = A ( 120/119 ) .
Thus π = 16 A ( 1 / 5 ) − 4 A ( 1 / 239 ) \pi=16A(1/5)-4A(1/239) π = 16 A ( 1/5 ) − 4 A ( 1/239 ) , and the two series tails decrease geometrically. They give rational enclosures of π \pi π to any prescribed binary precision in polynomial time.
Since 1 / 2 ≤ ( 1 + t 2 ) − 1 ≤ 1 1/2\le(1+t^2)^{-1}\le1 1/2 ≤ ( 1 + t 2 ) − 1 ≤ 1 on [ 0 , 1 ] [0,1] [ 0 , 1 ] , the normalization gives 2 ≤ π ≤ 4 2\le\pi\le4 2 ≤ π ≤ 4 . Rational bisection encloses the square root of a positive rational argument by comparing squared rational midpoints with that argument. Apply this operation to both rational endpoints of each enclosing interval. Starting with a rational interval containing the root, each comparison halves its width, so polynomially many steps give the required binary precision. Integer powering and reciprocal operations then give ( 2 π ) − d / 2 (2\pi)^{-d/2} ( 2 π ) − d /2 with polynomially many extra precision bits: on the interval 2 ≤ t ≤ 4 2\le t\le4 2 ≤ t ≤ 4 , the absolute value of the derivative of t ↦ ( 2 t ) − d / 2 t\mapsto(2t)^{-d/2} t ↦ ( 2 t ) − d /2 is at most d d d . All interval endpoints have polynomial bit length, and exact rational arithmetic preserves that bound. Thus the whole acceptance predicate is deterministic polynomial time in its fair bits and the guessed graph. No exact Gaussian sampling or transcendental equality test is used. □
The proposition does not classify exact synthesis at an arbitrary threshold. It also differs from the model of Open Problem 2 in the Introduction, which uses uniform Boolean inputs, clips the squared loss at one, and supplies 1 / δ 1/\delta 1/ δ in unary, so that its gap is at least inverse-polynomial. In the proposition, the loss may be unbounded and the binary gap may be exponentially small.
Counting Hardness with a Forced Zero Candidate
Standard Gaussian inputs also support a counting lower bound. The following restriction separates the difficulty of risk comparison from the choice of an occurrence assignment: its prescribed multiset forces every candidate to compute zero. Boolean-to-Gaussian lifting is an established technique in learning theory [12 , 5 ] . Here an independent Gaussian coordinate supplies a homogeneous amplitude with second moment one; this permits an explicit rational threshold for a majority-counting reduction.
Theorem 9.4 (Forced-zero Gaussian synthesis). Under either inventory rule, standard Gaussian squared-risk synthesis with positive binary rational threshold τ \tau τ and positive binary rational gap δ \delta δ is PP-complete in the separator sense when restricted to W = ( 0 ) W=(0) W = ( 0 ) and cap one. Hardness holds for bias-free rational ReLU targets, and also when candidate graphs are required to be bias-free. For the same restrictions, the exact non-strict positive-threshold problem is PP-hard. These hardness statements use deterministic polynomial-time many-one reductions, with promise reductions for the gap problem.
Proof. We reduce from strict-majority circuit acceptance. Given a Boolean circuit C C C on n ≥ 1 n\ge1 n ≥ 1 inputs, the source question is whether Pr [ C ( B ) = 1 ] > 1 / 2 \Pr[C(B)=1]>1/2 Pr [ C ( B ) = 1 ] > 1/2 for a uniform Boolean vector B B B . This problem is PP-complete: the acceptance predicate of a polynomial-time machine using polynomially many fair bits has a polynomial-size Boolean circuit, and unused bits pad its random tape to a fixed length. An unused bit ensures n ≥ 1 n\ge1 n ≥ 1 . Set
ε = 2 − n − 5 n , τ = 1 2 − 2 − n − 1 , δ = 2 − n − 2 . \varepsilon=\frac{2^{-n-5}}{n},\qquad\tau=\frac{1}{2}-2^{-n-1},\qquad\delta=2^{-n-2}. ε = n 2 − n − 5 , τ = 2 1 − 2 − n − 1 , δ = 2 − n − 2 . These are positive rationals of polynomial binary length.
Use n + 1 n+1 n + 1 independent standard normal coordinates ( Z , X 1 , … , X n ) (Z,X_1,\ldots,X_n) ( Z , X 1 , … , X n ) . The target first computes
r = ∣ Z ∣ = ρ ( Z ) + ρ ( − Z ) , a i = min { r , ε − 1 ρ ( X i ) } . r=|Z|=\rho(Z)+\rho(-Z),\qquad a_i=\min\{r,\varepsilon^{-1}\rho(X_i)\}. r = ∣ Z ∣ = ρ ( Z ) + ρ ( − Z ) , a i = min { r , ε − 1 ρ ( X i )} . All these quantities have bias-free ReLU implementations. For nonnegative stored signals u , v u,v u , v , the formulas
min ( u , v ) = ρ ( u − ρ ( u − v ) ) , max ( u , v ) = ρ ( v + ρ ( u − v ) ) \min(u,v)=\rho(u-\rho(u-v)),\qquad\max(u,v)=\rho(v+\rho(u-v)) min ( u , v ) = ρ ( u − ρ ( u − v )) , max ( u , v ) = ρ ( v + ρ ( u − v )) use two ReLU vertices each. A nonnegative sum, including r r r , can be stored using one additional ReLU.
Simulate C C C on the signals a i a_i a i with AND implemented by minimum, OR by maximum, and NOT by u ↦ ρ ( r − u ) u\mapsto\rho(r-u) u ↦ ρ ( r − u ) . The constants FALSE and TRUE are represented by zero and r r r . Let v C v_C v C be the final signal and give the target the affine output
Induction over the circuit shows that every simulated circuit signal and T T T lies in [ 0 , r ] [0,r] [ 0 , r ] . Each gate needs only constantly many vertices and edges. The nonunit scaling is ϵ − 1 = n 2 n + 5 \epsilon^{-1}=n2^{n+5} ϵ − 1 = n 2 n + 5 , of O ( n + log n ) O(n+\log n) O ( n + log n ) bits, so the target has polynomial binary encoding. Repeated gate arguments may be simplified, or coincident affine terms combined, so no parallel edges are necessary. All target biases are absent. Prune unused computations; a zero target can instead be represented by one zero-weight input-output edge. Thus the target satisfies the graph conventions and is positively homogeneous, meaning T ( t x ) = t T ( x ) T(tx)=tT(x) T ( t x ) = tT ( x ) for every t ≥ 0 t\ge0 t ≥ 0 .
For analysis only, put
B i = 1 { X i > 0 } , J = r ( 1 − C ( B ) ) , E = ⋃ i = 1 n { ∣ X i ∣ < ϵ r } . B_i=\mathbf{1}_{\{X_i>0\}},\qquad J=r(1-C(B)),\qquad E=\bigcup_{i=1}^{n}\{|X_i|<\epsilon r\}. B i = 1 { X i > 0 } , J = r ( 1 − C ( B )) , E = i = 1 ⋃ n { ∣ X i ∣ < ϵr } .
Outside E E E , a i = r B i a_i=rB_i a i = r B i , and gate induction gives T = J T=J T = J . Everywhere T , J ∈ [ 0 , r ] T,J\in[0,r] T , J ∈ [ 0 , r ] , whence
∣ T 2 − J 2 ∣ ≤ r 2 1 E . |T^2-J^2|\le r^2\mathbf{1}_E. ∣ T 2 − J 2 ∣ ≤ r 2 1 E .
Conditionally on r r r , the bound 1 / 2 π 1/\sqrt{2\pi} 1/ 2 π on the standard Gaussian density gives
Pr [ ∣ X i ∣ < ϵ r ∣ r ] ≤ 2 ϵ r 2 π . \Pr[|X_i|<\epsilon r\mid r]\le\frac{2\epsilon r}{\sqrt{2\pi}}. Pr [ ∣ X i ∣ < ϵr ∣ r ] ≤ 2 π 2 ϵr .
The union bound therefore yields
∣ E [ T 2 ] − E [ J 2 ] ∣ ≤ 2 n ϵ 2 π E [ ∣ Z ∣ 3 ] = 4 n ϵ π ≤ 4 n ϵ = 2 − n − 3 . |\mathbb{E}[T^2]-\mathbb{E}[J^2]|\le\frac{2n\epsilon}{\sqrt{2\pi}}\mathbb{E}[|Z|^3]=\frac{4n\epsilon}{\pi}\le4n\epsilon=2^{-n-3}. ∣ E [ T 2 ] − E [ J 2 ] ∣ ≤ 2 π 2 n ϵ E [ ∣ Z ∣ 3 ] = π 4 n ϵ ≤ 4 n ϵ = 2 − n − 3 .
Indeed, substituting u = z 2 / 2 u=z^2/2 u = z 2 /2 in the Gaussian integral gives E [ ∣ Z ∣ 3 ] = 2 2 / π \mathbb{E}[|Z|^3]=2\sqrt{2/\pi} E [ ∣ Z ∣ 3 ] = 2 2/ π . The signs B i B_i B i are independent fair bits and are independent of r r r , while E [ r 2 ] = 1 \mathbb{E}[r^2]=1 E [ r 2 ] = 1 . Writing q = Pr [ C ( B ) = 0 ] q=\Pr[C(B)=0] q = Pr [ C ( B ) = 0 ] , we obtain
∣ E [ T 2 ] − q ∣ ≤ ϵ , ϵ = 2 − n − 3 . |\mathbb{E}[T^2]-q|\le\epsilon,\qquad\epsilon=2^{-n-3}. ∣ E [ T 2 ] − q ∣ ≤ ϵ , ϵ = 2 − n − 3 .
This bound includes the unbounded Gaussian amplitude; no separate truncation error is omitted.
Supply W = ( 0 ) W=(0) W = ( 0 ) and cap one, including the output. Every legal candidate computes zero, and a single zero-weight input-output edge is a legal candidate under either inventory rule, also with biases prohibited. Thus every legal candidate has risk R = E [ T 2 ] R=\mathbb{E}[T^2] R = E [ T 2 ] . On a source YES instance, q ≤ 1 / 2 − 2 − n q\le1/2-2^{-n} q ≤ 1/2 − 2 − n , so
R ≤ 1 2 − 2 − n + ϵ < τ . R\le\frac{1}{2}-2^{-n}+\epsilon<\tau. R ≤ 2 1 − 2 − n + ϵ < τ .
For a source NO instance, q ≥ 1 / 2 q\ge1/2 q ≥ 1/2 , so
R ≥ 1 2 − ϵ > 1 2 − 2 − n − 2 = τ + δ . R\ge\frac{1}{2}-\epsilon>\frac{1}{2}-2^{-n-2}=\tau+\delta. R ≥ 2 1 − ϵ > 2 1 − 2 − n − 2 = τ + δ .
Every source circuit maps into the promise, proving PP-hardness for both rules. Dropping δ \delta δ from the output of this same reduction proves exact positive-threshold PP-hardness.
For gap membership, use the fixed-candidate construction in Proposition 9.3 with the legal zero network. It produces a polynomial-time fair-bit experiment and a known positive polynomial-bit rational D D D . Here b b b denotes the total number of fair bits of that experiment, its grid and vertical bits, not the grid precision b b b of Proposition 9.3 . The experiment accepts on A A A of the 2 b 2^b 2 b outcomes, so its acceptance probability is p = A / 2 b p=A/2^b p = A / 2 b , satisfying ∣ D p − R ∣ ≤ δ / 4 |Dp-R|\le\delta/4 ∣ D p − R ∣ ≤ δ /4 . There is no graph guess because every candidate has the same function. Accept when p ≤ ( τ + δ / 2 ) / D p\le(\tau+\delta/2)/D p ≤ ( τ + δ /2 ) / D . The approximation margins give a separator for every promised instance. To see directly that this is a total PP predicate, let K = ⌊ 2 b ( τ + δ / 2 ) / D ⌋ K=\lfloor2^b(\tau+\delta/2)/D\rfloor K = ⌊ 2 b ( τ + δ /2 ) / D ⌋ . If K < 0 K<0 K < 0 reject, and if K ≥ 2 b K\ge2^b K ≥ 2 b accept. Otherwise a machine with b + 1 b+1 b + 1 fair bits has 2 b − A 2^b-A 2 b − A accepting paths in one half, by complementing the original experiment, and K + 1 K+1 K + 1 accepting paths in the other half. Its acceptance count exceeds 2 b 2^b 2 b exactly when A ≤ K A\le K A ≤ K . This proves the claimed PP separator and completes the classification.
The theorem gives no exact PP upper bound or exact equality procedure. Its PP lower bound also applies to the unrestricted synthesis problem, and Theorem 9.5 attains the N P P P \mathrm{NP}^{\mathrm{PP}} NP PP upper bound under either rule when the candidate function can vary. The construction uses an exponentially small binary gap; it does not settle the clipped inverse-polynomial-gap question in Open Problem 2.
A Matching Gaussian Classification with Two Vertices
Allowing a nonzero candidate adds an existential choice to Gaussian counting. We obtain a matching classification by representing each witness bit with opposite unit weights on a pair of Gaussian coordinates. Each witness then defines a rare hinge: the ReLU of the resulting signed sum of coordinates minus a large bias, which is positive only far in the Gaussian tail. Every such intended hinge uses the complete supplied multiset. Different signed coordinate choices have much smaller cross moments than their individual squared norms. A second Gaussian moment calculation excludes every competing cap-two graph outside this family.
Theorem 9.5 (Gaussian binary-gap synthesis; restatement of Theorem 1.5). Under either inventory rule, standard-Gaussian squared-risk ReLU synthesis with a positive binary rational threshold and a positive binary rational gap is N P P P \mathrm{NP}^{\mathrm{PP}} NP PP -complete in the separator sense, under deterministic polynomial-time many-one promise reductions. Hardness holds with integer inventory and cap two, including the affine output. With the same restrictions, the exact non-strict positive-threshold problem is N P P P \mathrm{NP}^{\mathrm{PP}} NP PP -hard under deterministic polynomial-time many-one reductions.
Proof. Membership for the gap problem is Proposition 9.3. For hardness use E-MAJSAT: given a Boolean circuit φ \varphi φ with e e e existential inputs and n n n random inputs, put p ( a ) = Pr z ∈ { 0 , 1 } n [ φ ( a , z ) = 1 ] p(a)=\Pr_{z\in\{0,1\}^{n}}[\varphi(a,z)=1] p ( a ) = Pr z ∈ { 0 , 1 } n [ φ ( a , z ) = 1 ] for each a ∈ { 0 , 1 } e a\in\{0,1\}^{e} a ∈ { 0 , 1 } e , and ask whether
∃ a ∈ { 0 , 1 } e p ( a ) > 1 2 . \exists a\in\{0,1\}^{e}\quad p(a)>\frac{1}{2}. ∃ a ∈ { 0 , 1 } e p ( a ) > 2 1 .
This is N P P P \mathrm{NP}^{\mathrm{PP}} NP PP -complete with exactly this fixed-witness, strict-majority convention [28 Appendix A, Theorem 15] . Ignored variables ensure e , n ≥ 1 e,n\ge1 e , n ≥ 1 . Put q = 2 − n q=2^{-n} q = 2 − n . A YES instance has some p ( a ) ≥ 1 / 2 + q p(a)\ge1/2+q p ( a ) ≥ 1/2 + q ; a NO instance has every p ( a ) ≤ 1 / 2 p(a)\le1/2 p ( a ) ≤ 1/2 .
Set
k = 2 e , E = k + n + 10 , B = 16 E 2 , d = k + n + 1 , η = 2 − 10 E . k=2e,\qquad E=k+n+10,\qquad B=16E^2,\qquad d=k+n+1,\qquad\eta=2^{-10E}. k = 2 e , E = k + n + 10 , B = 16 E 2 , d = k + n + 1 , η = 2 − 10 E .
The independent standard normal coordinates are ( Y 1 , Y 1 ′ , … , Y e , Y e ′ , R 1 , … , R n , U ) (Y_1,Y'_1,\ldots,Y_e,Y'_e,R_1,\ldots,R_n,U) ( Y 1 , Y 1 ′ , … , Y e , Y e ′ , R 1 , … , R n , U ) . Recall from § 9 that a centered random variable has mean zero. Write X X X for this raw input vector and put D i = Y i − Y i ′ D_i=Y_i-Y'_i D i = Y i − Y i ′ ; these differences are independent centered normals of variance two, independent of the verifier coordinates R 1 , … , R n , U R_1,\ldots,R_n,U R 1 , … , R n , U . Supply cap two and the multiset
W = ( + 1 ) e + 1 ⊎ ( − 1 ) e ⊎ { − B } . W=(+1)^{e+1}\uplus(-1)^e\uplus\{-B\}. W = ( + 1 ) e + 1 ⊎ ( − 1 ) e ⊎ { − B } .
For a signed witness a ∈ { − 1 , + 1 } e a\in\{-1,+1\}^{e} a ∈ { − 1 , + 1 } e , write a ˉ = ( a + 1 ) / 2 ∈ { 0 , 1 } e \bar a=(a+1)/2\in\{0,1\}^{e} a ˉ = ( a + 1 ) /2 ∈ { 0 , 1 } e for its Boolean encoding and define
h a = ρ ( ∑ i = 1 e a i D i − B ) , H = ρ ( ∑ i = 1 e ∣ D i ∣ − B ) = max a h a . h_a=\rho\left(\sum_{i=1}^{e}a_iD_i-B\right),\qquad H=\rho\left(\sum_{i=1}^{e}|D_i|-B\right)=\max_a h_a. h a = ρ ( i = 1 ∑ e a i D i − B ) , H = ρ ( i = 1 ∑ e ∣ D i ∣ − B ) = a max h a .
The equality H = max a h a H=\max_a h_a H = max a h a holds because ∑ i a i D i ≤ ∑ i ∣ D i ∣ \sum_i a_iD_i\le\sum_i|D_i| ∑ i a i D i ≤ ∑ i ∣ D i ∣ , with equality when each a i a_i a i is a sign of D i D_i D i , and ρ \rho ρ is nondecreasing. The envelope H H H has an integer-weight ReLU implementation with 2 e + 1 2e+1 2 e + 1 hidden vertices. For each i i i , two hidden vertices compute ρ ( Y i − Y i ′ ) = ρ ( D i ) \rho(Y_i-Y'_i)=\rho(D_i) ρ ( Y i − Y i ′ ) = ρ ( D i ) and ρ ( Y i ′ − Y i ) = ρ ( − D i ) \rho(Y'_i-Y_i)=\rho(-D_i) ρ ( Y i ′ − Y i ) = ρ ( − D i ) , whose sum is ∣ D i ∣ |D_i| ∣ D i ∣ . One further hidden vertex receives these 2 e 2e 2 e values with weight one and has bias − B -B − B , so it computes ρ ( ∑ i ( ρ ( D i ) + ρ ( − D i ) ) − B ) = H \rho(\sum_i(\rho(D_i)+\rho(-D_i))-B)=H ρ ( ∑ i ( ρ ( D i ) + ρ ( − D i )) − B ) = H .
Every intended h a h_a h a uses all occurrences exactly once: the two input weights ( a i , − a i ) (a_i,-a_i) ( a i , − a i ) consume one positive and one negative unit per pair, the hidden bias consumes − B -B − B , and the output edge consumes the remaining positive unit. Thus it is a legal cap-two candidate under either rule. Its raw coefficient vector has k = 2 e k=2e k = 2 e nonzero unit entries, so its hidden preactivation has variance k k k .
We first provide quantitative bounds for these functions. For integers 1 ≤ j ≤ 2 k 1\le j\le2k 1 ≤ j ≤ 2 k , let
s j = E [ ρ ( j Z − B ) 2 ] , s = s k , s_j=\mathbb{E}\left[\rho\left(\sqrt{j}Z-B\right)^2\right],\qquad s=s_k, s j = E [ ρ ( j Z − B ) 2 ] , s = s k ,
where Z Z Z is standard normal. Using the Gaussian density ϕ \phi ϕ and its upper tail Φ ‾ \overline{\Phi} Φ , we show
exp ( − B 2 / ( 2 j ) ) 200 B 3 ≤ s j ≤ 2 j ( B + 1 ) exp ( − B 2 / ( 2 j ) ) . (5) \frac{\exp(-B^2/(2j))}{200B^3}\le s_j\le2j(B+1)\exp(-B^2/(2j)). \tag*{(5)} 200 B 3 exp ( − B 2 / ( 2 j )) ≤ s j ≤ 2 j ( B + 1 ) exp ( − B 2 / ( 2 j )) . (5)
For the lower bound put t = B / j ≥ 1 t=B/\sqrt{j}\ge1 t = B / j ≥ 1 and integrate over [ t + t − 1 , t + 2 t − 1 ] [t+t^{-1},t+2t^{-1}] [ t + t − 1 , t + 2 t − 1 ] . The squared excess is at least j / B 2 j/B^2 j / B 2 , the interval has length j / B \sqrt{j}/B j / B , and the density of Z Z Z on it is at least
exp ( − 4 ) exp ( − t 2 / 2 ) / 2 π > exp ( − t 2 / 2 ) / 200. \exp(-4)\exp(-t^2/2)/\sqrt{2\pi}>\exp(-t^2/2)/200. exp ( − 4 ) exp ( − t 2 /2 ) / 2 π > exp ( − t 2 /2 ) /200.
Their product is at least the stated lower bound because j ≥ 1 j \ge1 j ≥ 1 .
For the upper bound, integration by parts and Φ ‾ ( t ) ≤ ϕ ( t ) / t \overline{\Phi}(t) \le\phi(t)/t Φ ( t ) ≤ ϕ ( t ) / t yield
s j ≤ j E [ Z 2 1 { Z > t } ] = j ( t ϕ ( t ) + Φ ‾ ( t ) ) ≤ 2 j ( B + 1 ) exp ( − t 2 / 2 ) . s_j \le j\mathbb{E}[Z^2\mathbf{1}_{\{Z>t\}}] = j(t\phi(t)+\overline{\Phi}(t)) \le2j(B+1)\exp(-t^2/2). s j ≤ j E [ Z 2 1 { Z > t } ] = j ( tϕ ( t ) + Φ ( t )) ≤ 2 j ( B + 1 ) exp ( − t 2 /2 ) .
For completeness, the same upper bound holds for any positive real variance v v v with t = B / v ≥ 1 t=B/\sqrt{v} \ge1 t = B / v ≥ 1 . Shifting the integral gives
E [ ρ ( v Z − B ) 2 ] = v ϕ ( t ) ∫ 0 ∞ u 2 exp ( − t u − u 2 / 2 ) d u ≤ v 2 exp ( − t 2 / 2 ) , \mathbb{E}[\rho(\sqrt{v}Z-B)^2] = v\phi(t)\int_0^\infty u^2\exp(-tu-u^2/2)\,du \le\frac{v}{2}\exp(-t^2/2), E [ ρ ( v Z − B ) 2 ] = v ϕ ( t ) ∫ 0 ∞ u 2 exp ( − t u − u 2 /2 ) d u ≤ 2 v exp ( − t 2 /2 ) ,
since ∫ 0 ∞ u 2 exp ( − u 2 / 2 ) d u = π / 2 \int_0^\infty u^2\exp(-u^2/2)\,du=\sqrt{\pi/2} ∫ 0 ∞ u 2 exp ( − u 2 /2 ) d u = π /2 . This is smaller than the stated upper estimate.
The quantities s j s_j s j increase with j j j : add an independent centered Gaussian increment and apply Jensen’s inequality to the convex function x ↦ ρ ( x − B ) 2 x\mapsto\rho(x-B)^2 x ↦ ρ ( x − B ) 2 .
Identify an intended witness with its raw coefficient vector a ~ = ( a 1 , − a 1 , … , a e , − a e , 0 , … , 0 ) ∈ R d \widetilde{a}=(a_1,-a_1,\ldots,a_e,-a_e,0,\ldots,0)\in\mathbb{R}^d a = ( a 1 , − a 1 , … , a e , − a e , 0 , … , 0 ) ∈ R d . For any signed coordinate vector w ∈ { − 1 , 0 , 1 } d w\in\{-1,0,1\}^d w ∈ { − 1 , 0 , 1 } d with exactly k k k nonzero entries, distinct from an intended vector a ~ \widetilde{a} a , we have ∥ w + a ~ ∥ 2 2 ≤ 4 k − 2 \lVert w+\widetilde{a}\rVert_2^2\le4k-2 ∥ w + a ∥ 2 2 ≤ 4 k − 2 . Writing h w = ρ ( w ⋅ X − B ) h_w=\rho(w\cdot X-B) h w = ρ ( w ⋅ X − B ) , the arithmetic-geometric mean inequality gives pointwise
h w h a ≤ 1 4 ρ ( ( w + a ~ ) ⋅ X − 2 B ) 2 . h_wh_a\le\frac{1}{4}\rho((w+\widetilde{a})\cdot X-2B)^2. h w h a ≤ 4 1 ρ (( w + a ) ⋅ X − 2 B ) 2 .
If one factor on the left is zero this is immediate; otherwise both preactivations are positive and their sum is the argument on the right. Applying the bound above for a general variance v v v , or using zero if w + a ~ = 0 w+\widetilde{a}=0 w + a = 0 , shows
E [ h w h a ] ≤ c : = 12 k B exp ( − B 2 / ( 2 k − 1 ) ) . \mathbb{E}[h_wh_a]\le c:=12kB\exp(-B^2/(2k-1)). E [ h w h a ] ≤ c := 12 k B exp ( − B 2 / ( 2 k − 1 )) .
This also covers vectors w w w with nonzero entries on the verifier coordinates.
Put
κ = 2400 k B 4 exp ( − B 2 / ( 4 k 2 ) ) . \kappa=2400kB^4\exp(-B^2/(4k^2)). κ = 2400 k B 4 exp ( − B 2 / ( 4 k 2 )) .
Taking ratios in Equation (5) gives
s k − 1 / s ≤ κ ( k ≥ 2 ) , c / s ≤ κ , s / s k + 1 ≤ κ . s_{k-1}/s\le\kappa\quad(k\ge2),\qquad c/s\le\kappa,\qquad s/s_{k+1}\le\kappa. s k − 1 / s ≤ κ ( k ≥ 2 ) , c / s ≤ κ , s / s k + 1 ≤ κ .
Each ratio divides the upper bound in Equation (5) , or the definition of c c c , by the lower bound there. For the first ratio,
s k − 1 s ≤ 400 ( k − 1 ) ( B + 1 ) B 3 exp ( − B 2 2 ( k − 1 ) + B 2 2 k ) = 400 ( k − 1 ) ( B + 1 ) B 3 exp ( − B 2 2 k ( k − 1 ) ) . \frac{s_{k-1}}{s}\le400(k-1)(B+1)B^3\exp\left(-\frac{B^2}{2(k-1)}+\frac{B^2}{2k}\right)=400(k-1)(B+1)B^3\exp\left(-\frac{B^2}{2k(k-1)}\right). s s k − 1 ≤ 400 ( k − 1 ) ( B + 1 ) B 3 exp ( − 2 ( k − 1 ) B 2 + 2 k B 2 ) = 400 ( k − 1 ) ( B + 1 ) B 3 exp ( − 2 k ( k − 1 ) B 2 ) .
For the second,
c s ≤ 2400 k B 4 exp ( − B 2 2 k − 1 + B 2 2 k ) = 2400 k B 4 exp ( − B 2 2 k ( 2 k − 1 ) ) . \frac{c}{s}\le2400kB^4\exp\left(-\frac{B^2}{2k-1}+\frac{B^2}{2k}\right)=2400kB^4\exp\left(-\frac{B^2}{2k(2k-1)}\right). s c ≤ 2400 k B 4 exp ( − 2 k − 1 B 2 + 2 k B 2 ) = 2400 k B 4 exp ( − 2 k ( 2 k − 1 ) B 2 ) .
For the third, where k + 1 ≤ 2 k k+1\le2k k + 1 ≤ 2 k keeps s k + 1 s_{k+1} s k + 1 within the range of Equation (5) ,
s s k + 1 ≤ 400 k ( B + 1 ) B 3 exp ( − B 2 2 k + B 2 2 ( k + 1 ) ) = 400 k ( B + 1 ) B 3 exp ( − B 2 2 k ( k + 1 ) ) . \frac{s}{s_{k+1}}\le400k(B+1)B^3\exp\left(-\frac{B^2}{2k}+\frac{B^2}{2(k+1)}\right)=400k(B+1)B^3\exp\left(-\frac{B^2}{2k(k+1)}\right). s k + 1 s ≤ 400 k ( B + 1 ) B 3 exp ( − 2 k B 2 + 2 ( k + 1 ) B 2 ) = 400 k ( B + 1 ) B 3 exp ( − 2 k ( k + 1 ) B 2 ) .
The denominators 2 k ( k − 1 ) 2k(k-1) 2 k ( k − 1 ) , 2 k ( 2 k − 1 ) 2k(2k-1) 2 k ( 2 k − 1 ) and 2 k ( k + 1 ) 2k(k+1) 2 k ( k + 1 ) are at most 4 k 2 4k^2 4 k 2 because k ≥ 1 k\ge1 k ≥ 1 , so each exponential is at most exp ( − B 2 / ( 4 k 2 ) ) \exp(-B^2/(4k^2)) exp ( − B 2 / ( 4 k 2 )) . Since B + 1 ≤ 2 B B+1\le2B B + 1 ≤ 2 B , each prefactor is at most 2400 k B 4 2400kB^4 2400 k B 4 .
Our deliberately large choice of B B B ensures
κ ≤ η , s 2 k ≤ 2 − 20 E , 2 k s ≤ η . (6) \kappa\le\eta,\qquad s_{2k}\le2^{-20E},\qquad2^k s\le\eta. \tag*{(6)} κ ≤ η , s 2 k ≤ 2 − 20 E , 2 k s ≤ η . (6)
Here log 2 ( 2400 k B 4 ) ≤ 12 E \log_2(2400kB^4)\le12E log 2 ( 2400 k B 4 ) ≤ 12 E for E ≥ 12 E\ge12 E ≥ 12 , whereas B 2 / ( 4 k 2 ) ≥ 64 E 2 B^2/(4k^2)\ge64E^2 B 2 / ( 4 k 2 ) ≥ 64 E 2 and exp ( − x ) ≤ 2 − x \exp(-x)\le2^{-x} exp ( − x ) ≤ 2 − x . For the other two bounds, B + 1 ≤ 2 E B+1\le2^E B + 1 ≤ 2 E , 4 k ≤ 2 E 4k\le2^E 4 k ≤ 2 E , and the exponents B 2 / ( 4 k ) ≥ 64 E 3 B^2/(4k)\ge64E^3 B 2 / ( 4 k ) ≥ 64 E 3 , B 2 / ( 2 k ) ≥ 128 E 3 B^2/(2k)\ge128E^3 B 2 / ( 2 k ) ≥ 128 E 3 dominate the respective prefactors.
A convenient explicit lower bound from Equation (5) is
0 < L : = 2 − B 2 − 4 B − 10 ≤ s . 0<L:=2^{-B^2-4B-10}\le s. 0 < L := 2 − B 2 − 4 B − 10 ≤ s .
Indeed exp ( − B 2 / ( 2 k ) ) ≥ 2 − B 2 \exp(-B^2/(2k))\ge2^{-B^2} exp ( − B 2 / ( 2 k )) ≥ 2 − B 2 , using exp ( 1 ) < 4 \exp(1)<4 exp ( 1 ) < 4 , and 200 B 3 ≤ 2 4 B + 10 200B^3\le2^{4B+10} 200 B 3 ≤ 2 4 B + 10 .
We next construct the independently supplied target. Set
ε = η L 256 d e 3 . \varepsilon= \frac{\eta L}{256de^{3}}. ε = 256 d e 3 η L .
Here e e e is the number of existential inputs of φ \varphi φ , not Euler’s number. The target reads a bit from each coordinate t t t among the differences D i D_i D i and the verifier coordinates R j , U R_j,U R j , U , using the signal
min { H , ε − 1 ρ ( t ) } . \min\{H,\varepsilon^{-1}\rho(t)\}. min { H , ε − 1 ρ ( t )} .
Simulate the Boolean circuit F ( a , z , u ) = u ∨ φ ( a , z ) F(a,z,u)=u\vee\varphi(a,z) F ( a , z , u ) = u ∨ φ ( a , z ) , whose arguments a a a , z z z and u u u are the bits read from the D i D_i D i , the R j R_j R j and U U U , respectively, with amplitude H H H : AND and OR are minimum and maximum, NOT is v ↦ ρ ( H − v ) v\mapsto\rho(H-v) v ↦ ρ ( H − v ) , and constants are zero and H H H . Use the two-ReLU minimum and maximum implementations from the proof of Theorem 9.4. If v F v_F v F is the final signal, take the affine target output T = 2 v F − H T=2v_F-H T = 2 v F − H . Every simulated circuit signal is in [ 0 , H ] [0,H] [ 0 , H ] , and ∣ T ∣ ≤ H |T|\le H ∣ T ∣ ≤ H .
Absolute values of the rational differences, the sum defining H H H , and the gate simulation all have polynomial size. The integers B B B and ε − 1 \varepsilon^{-1} ε − 1 have polynomial binary length, as do all target coefficients. Coincident terms may be combined to avoid parallel edges, and unused vertices may be pruned. The target has biases; no homogeneity restriction is claimed in this theorem.
For analysis put bit ( t ) = 1 { t > 0 } \operatorname{bit}(t)=\mathbf{1}_{\{t>0\}} bit ( t ) = 1 { t > 0 } , applied coordinatewise to a vector, and define
J = H ( 2 F ( bit D , bit R , bit U ) − 1 ) . J=H\bigl(2F(\operatorname{bit}D,\operatorname{bit}R,\operatorname{bit}U)-1\bigr). J = H ( 2 F ( bit D , bit R , bit U ) − 1 ) .
In particular, bit ( 0 ) = 0 \operatorname{bit}(0)=0 bit ( 0 ) = 0 . For fixed witness signs a a a , independence and the extra fair bit u u u give
E R , U [ 2 F ( a ˉ , bit R , bit U ) − 1 ] = p ( a ˉ ) . \mathbb{E}_{R,U}\bigl[2F(\bar a,\operatorname{bit}R,\operatorname{bit}U)-1\bigr]=p(\bar a). E R , U [ 2 F ( a ˉ , bit R , bit U ) − 1 ] = p ( a ˉ ) .
Let E \mathcal{E} E be the union of the events ∣ D i ∣ < ε H |D_i|<\varepsilon H ∣ D i ∣ < ε H , ∣ R j ∣ < ε H |R_j|<\varepsilon H ∣ R j ∣ < ε H , and ∣ U ∣ < ε H |U|<\varepsilon H ∣ U ∣ < ε H over the decoded coordinates. Outside this event the simulation equals J J J , and everywhere ∣ T − J ∣ ≤ 2 H 1 E |T-J|\le2H\mathbf{1}_{\mathcal{E}} ∣ T − J ∣ ≤ 2 H 1 E . Put S = E [ H 2 1 E ] S=\mathbb{E}[H^2\mathbf{1}_{\mathcal{E}}] S = E [ H 2 1 E ] . We claim
S ≤ 64 d e ε 3 ≤ η s / 4. S\le64de\varepsilon^3\le\eta s/4. S ≤ 64 d e ε 3 ≤ η s /4.
Indeed H ≤ Q = ∑ i ∣ D i ∣ H\le Q=\sum_i|D_i| H ≤ Q = ∑ i ∣ D i ∣ , and convexity gives E [ Q 3 ] ≤ e 2 ∑ i E [ ∣ D i ∣ 3 ] ≤ 6 e 3 \mathbb{E}[Q^3]\le e^2\sum_i\mathbb{E}[|D_i|^3]\le6e^3 E [ Q 3 ] ≤ e 2 ∑ i E [ ∣ D i ∣ 3 ] ≤ 6 e 3 , since E [ ∣ D i ∣ 3 ] = 8 / π < 6 \mathbb{E}[|D_i|^3]=8/\sqrt{\pi}<6 E [ ∣ D i ∣ 3 ] = 8/ π < 6 . Every decoded coordinate has density at most 1 / 2 1/2 1/2 . For an independent verifier coordinate, conditioning on H H H bounds its contribution by ε E [ H 3 ] ≤ 6 ε e 3 \varepsilon\mathbb{E}[H^3]\le6\varepsilon e^3 ε E [ H 3 ] ≤ 6 ε e 3 .
For D i D_i D i , put Q − i = ∑ j ≠ i ∣ D j ∣ Q_{-i}=\sum_{j\ne i}|D_j| Q − i = ∑ j = i ∣ D j ∣ . On ∣ D i ∣ < ε H |D_i|<\varepsilon H ∣ D i ∣ < ε H , since ε ≤ 1 / 2 \varepsilon\le1/2 ε ≤ 1/2 , we have ∣ D i ∣ < 2 ε Q − i |D_i|<2\varepsilon Q_{-i} ∣ D i ∣ < 2 ε Q − i and H 2 ≤ 4 Q − i 2 H^2\le4Q_{-i}^2 H 2 ≤ 4 Q − i 2 . Conditioning on the other differences bounds the contribution by 8 ε E [ Q − i 3 ] ≤ 48 ε e 3 8\varepsilon\mathbb{E}[Q_{-i}^3]\le48\varepsilon e^3 8 ε E [ Q − i 3 ] ≤ 48 ε e 3 . This includes e = 1 e=1 e = 1 , when the dependent event has zero contribution. There are e + n + 1 ≤ d e+n+1\le d e + n + 1 ≤ d events; the union bound and the chosen ε \varepsilon ε prove the claim.
Put C = E [ T 2 ] C=\mathbb{E}[T^2] C = E [ T 2 ] and C 0 = 2 e s C_0=2^e s C 0 = 2 e s . Since H = max a h a H=\max_a h_a H = max a h a , choosing a maximizing hinge at each point bounds the sum of the remaining squared hinges by their products with that maximizer. Summing the cross-moment bounds over ordered pairs gives
0 ≤ C 0 − E [ H 2 ] ≤ 2 2 e c . 0\le C_0-\mathbb{E}[H^2]\le2^{2e}c. 0 ≤ C 0 − E [ H 2 ] ≤ 2 2 e c .
Next, ∣ T ∣ ≤ H |T|\le H ∣ T ∣ ≤ H everywhere, and outside E \mathcal{E} E we have T = J T=J T = J , so T 2 = J 2 = H 2 T^2=J^2=H^2 T 2 = J 2 = H 2 because 2 F − 1 ∈ { − 1 , 1 } 2F-1\in\{-1,1\} 2 F − 1 ∈ { − 1 , 1 } . Hence
0 ≤ E [ H 2 ] − C ≤ S . 0\le\mathbb{E}[H^2]-C\le S. 0 ≤ E [ H 2 ] − C ≤ S .
Adding these two displays gives
0 ≤ C 0 − C ≤ Δ : = 2 2 e c + S . 0\le C_0-C\le\Delta:=2^{2e}c+S. 0 ≤ C 0 − C ≤ Δ := 2 2 e c + S .
For a signed witness a a a , its difference-sign orthant is { X : a i D i ≥ 0 for every i } \{X:a_iD_i\ge0\text{ for every }i\} { X : a i D i ≥ 0 for every i } . On this orthant the intended hinge satisfies H = h a H=h_a H = h a . Define the random signed label ℓ a = 2 F ( a ˉ , bit R , bit U ) − 1 \ell_a=2F(\bar a,\operatorname{bit}R,\operatorname{bit}U)-1 ℓ a = 2 F ( a ˉ , bit R , bit U ) − 1 , whose mean over R , U R,U R , U is p ( a ˉ ) p(\bar a) p ( a ˉ ) . Each D i D_i D i has a continuous normal density, so the union of the events D i = 0 D_i=0 D i = 0 has probability zero and may be omitted in these expectations. Off these events, bit D = a ˉ \operatorname{bit}D=\bar a bit D = a ˉ on the orthant, so there J = h a ℓ a J=h_a\ell_a J = h a ℓ a and h a 2 = h a 2 ℓ a h_a^2=h_a^2\ell_a h a 2 = h a 2 ℓ a . Outside the difference-sign orthant, the pointwise difference between h a J h_aJ h a J and h a 2 ℓ a h_a^2\ell_a h a 2 ℓ a is at most 2 h a H 2h_aH 2 h a H . There H = h a ′ H=h_{a'} H = h a ′ , where the signed witness a ′ ≠ a a'\ne a a ′ = a records the signs of the D i D_i D i , so the difference is at most 2 h a h a ′ 2h_ah_{a'} 2 h a h a ′ . Since h a h_a h a depends only on the differences and ℓ a \ell_a ℓ a only on R , U R,U R , U , averaging the labels gives E [ h a 2 ℓ a ] = p ( a ˉ ) E [ h a 2 ] = p ( a ˉ ) s \mathbb{E}[h_a^2\ell_a]=p(\bar a)\mathbb{E}[h_a^2]=p(\bar a)s E [ h a 2 ℓ a ] = p ( a ˉ ) E [ h a 2 ] = p ( a ˉ ) s . Summing the bounds E [ h a h a ′ ] ≤ c \mathbb{E}[h_ah_{a'}]\le c E [ h a h a ′ ] ≤ c over the 2 e − 1 2^e-1 2 e − 1 witnesses a ′ ≠ a a'\ne a a ′ = a therefore yields
∣ E [ h a J ] − p ( a ˉ ) s ∣ ≤ 2 e + 1 c . \left|\mathbb{E}[h_aJ]-p(\bar a)s\right|\le2^{e+1}c. ∣ E [ h a J ] − p ( a ˉ ) s ∣ ≤ 2 e + 1 c .
Moreover h a ≤ H h_a \le H h a ≤ H , so ∣ E [ h a ( T − J ) ] ∣ ≤ 2 S |\mathbb{E}[h_a(T-J)]| \le2S ∣ E [ h a ( T − J )] ∣ ≤ 2 S . Thus
∣ E [ h a T ] − p ( a ˉ ) s ∣ ≤ A : = 2 e + 1 c + 2 S . |\mathbb{E}[h_aT]-p(\bar{a})s| \le A := 2^{e+1}c+2S. ∣ E [ h a T ] − p ( a ˉ ) s ∣ ≤ A := 2 e + 1 c + 2 S .
The preceding bounds imply
Δ ≤ q s / 128 , A ≤ q s / 128 , 2 2 e κ s ≤ q s / 128. (7) \Delta\le qs/128,\qquad A \le qs/128,\qquad2\sqrt{2^e}\,\kappa s \le qs/128. \tag*{(7)} Δ ≤ q s /128 , A ≤ q s /128 , 2 2 e κ s ≤ q s /128. (7)
For explicit comparison, Δ / s ≤ ( 2 2 e + 1 / 4 ) 2 − 10 E \Delta/s \le(2^{2e}+1/4)2^{-10E} Δ/ s ≤ ( 2 2 e + 1/4 ) 2 − 10 E , A / s ≤ ( 2 e + 1 + 1 / 2 ) 2 − 10 E A/s \le(2^{e+1}+1/2)2^{-10E} A / s ≤ ( 2 e + 1 + 1/2 ) 2 − 10 E , and the last left side divided by s s s is at most 2 1 + e / 2 − 5 E 2^{1+e/2-5E} 2 1 + e /2 − 5 E . Each is at most 2 − n − 7 2^{-n-7} 2 − n − 7 because E = k + n + 10 E=k+n+10 E = k + n + 10 and e ≤ k e \le k e ≤ k .
It remains to exclude every other legal graph. With cap two a candidate is affine, or has one hidden vertex and the form
N ( X ) = g ρ ( w ⋅ X + b ) + A 0 ( X ) , N(X)=g\rho(w\cdot X+b)+A_0(X), N ( X ) = g ρ ( w ⋅ X + b ) + A 0 ( X ) ,
where A 0 A_0 A 0 is affine and the displayed present coefficients use distinct occurrences of W W W . The hidden-to-output edge is compulsory by reachability, and all coefficients are integers. If the hidden input vector is zero, or if there is no hidden vertex, the output is an integer affine function. It is either zero or has squared Gaussian norm at least one. In the latter case the triangle inequality and C 0 ≤ η C_0\le\eta C 0 ≤ η make its risk greater than C 0 C_0 C 0 .
Suppose w ≠ 0 w\ne0 w = 0 , and − B -B − B is not the hidden bias. Then b ∈ { − 1 , 0 , 1 } b\in\{-1,0,1\} b ∈ { − 1 , 0 , 1 } , σ = ∥ w ∥ 2 ≥ 1 \sigma=\|w\|_2\ge1 σ = ∥ w ∥ 2 ≥ 1 , and ∣ g ∣ ≥ 1 |g|\ge1 ∣ g ∣ ≥ 1 . The linear-combination rule established at the start of Section 9 makes Z = w ⋅ X / σ Z=w\cdot X/\sigma Z = w ⋅ X / σ standard normal. Integration by parts twice on ( − b / σ , ∞ ) (-b/\sigma,\infty) ( − b / σ , ∞ ) gives
E [ ρ ( σ Z + b ) ( Z 2 − 1 ) ] = σ ϕ ( b / σ ) . \mathbb{E}[\rho(\sigma Z+b)(Z^2-1)]=\sigma\phi(b/\sigma). E [ ρ ( σ Z + b ) ( Z 2 − 1 )] = σ ϕ ( b / σ ) .
Indeed ϕ ′ ′ = ( z 2 − 1 ) ϕ \phi''=(z^2-1)\phi ϕ ′′ = ( z 2 − 1 ) ϕ ; the first boundary term vanishes because the hinge is zero at its breakpoint, and the remaining integral is − σ ∫ − b / σ ∞ ϕ ′ ( z ) d z -\sigma\int_{-b/\sigma}^{\infty}\phi'(z)\,dz − σ ∫ − b / σ ∞ ϕ ′ ( z ) d z .
The polynomial Z 2 − 1 Z^2-1 Z 2 − 1 is the second Hermite polynomial in the normalization used here. The preceding moment calculation gives E [ Z 2 ] = 1 \mathbb{E}[Z^2]=1 E [ Z 2 ] = 1 and E [ Z 4 ] = 3 \mathbb{E}[Z^4]=3 E [ Z 4 ] = 3 . Thus constants are orthogonal to Z 2 − 1 Z^2-1 Z 2 − 1 . The reflection X ↦ − X X\mapsto-X X ↦ − X preserves the standard Gaussian density and Z 2 − 1 Z^2-1 Z 2 − 1 , while changing the sign of every linear function of X X X ; the inner product of every linear function of X X X with Z 2 − 1 Z^2-1 Z 2 − 1 is therefore zero. Every affine function of X X X is consequently orthogonal to Z 2 − 1 Z^2-1 Z 2 − 1 , whose squared norm is 3 − 2 + 1 = 2 3-2+1=2 3 − 2 + 1 = 2 . Cauchy–Schwarz therefore gives
∥ N ∥ 2 2 ≥ g 2 σ 2 ϕ ( b / σ ) 2 2 ≥ ϕ ( 1 ) 2 2 > 1 100 . \|N\|_2^2\ge\frac{g^2\sigma^2\phi(b/\sigma)^2}{2}\ge\frac{\phi(1)^2}{2}>\frac{1}{100}. ∥ N ∥ 2 2 ≥ 2 g 2 σ 2 ϕ ( b / σ ) 2 ≥ 2 ϕ ( 1 ) 2 > 100 1 .
Since ∥ T ∥ 2 2 ≤ C 0 ≤ η \|T\|_2^2\le C_0\le\eta ∥ T ∥ 2 2 ≤ C 0 ≤ η , every such candidate has risk greater than C 0 C_0 C 0 , by the triangle inequality. This covers − B -B − B as an input weight, output gain, output bias or unused occurrence, and permits arbitrary affine skips and cancellations.
If − B -B − B is the hidden bias, all other coefficients are units and g = ± 1 g=\pm1 g = ± 1 . After the output edge is assigned, at most 2 e = k 2e=k 2 e = k units remain for hidden input edges. The hidden term is then ± h w \pm h_w ± h w , and its squared norm E [ h w 2 ] \mathbb{E}[h_w^2] E [ h w 2 ] is at most s k ≤ s 2 k s_k\le s2^k s k ≤ s 2 k . If A 0 ≠ 0 A_0\ne0 A 0 = 0 , it has norm at least one, so ∥ N ∥ 2 2 ≥ 1 − s 2 k \|N\|_2^2\ge1-\sqrt{s2^k} ∥ N ∥ 2 2 ≥ 1 − s 2 k ; its risk again exceeds C 0 C_0 C 0 . Otherwise A 0 = 0 A_0=0 A 0 = 0 and N = ± h w N=\pm h_w N = ± h w , with w w w having j ≤ k j\le k j ≤ k nonzero unit entries. Under exactly-once use j = k j=k j = k is forced: every remaining unit must occupy a distinct hidden input edge. When A 0 = 0 A_0=0 A 0 = 0 , no direct input-to-output edge and no output bias can be present. Such an edge or bias would contribute a nonzero coefficient to A 0 A_0 A 0 , and cancelling it would require a parallel edge or a second output bias. This accounts for every exactly-once occurrence without padding or increasing the cap.
Under at-most-once use, for 1 ≤ j < k 1\le j<k 1 ≤ j < k , monotonicity and Cauchy–Schwarz give
R T G ( N ) ≥ C − 2 C s j ≥ C − 2 2 e κ s ≥ C − q s / 128. R_T^G(N)\ge C-2\sqrt{Cs_j}\ge C-2\sqrt{2^e}\,\kappa s\ge C-qs/128. R T G ( N ) ≥ C − 2 C s j ≥ C − 2 2 e κ s ≥ C − q s /128.
For j = 0 j=0 j = 0 the candidate is the zero function. For j = k j=k j = k , if w w w is outside the intended paired family, the cross-moment bound applies to every intended a a a , so
∣ E [ h w T ] ∣ ≤ E [ h w H ] ≤ ∑ a E [ h w h a ] ≤ 2 e c . |\mathbb{E}[h_wT]|\le\mathbb{E}[h_wH]\le\sum_a\mathbb{E}[h_wh_a]\le2^e c. ∣ E [ h w T ] ∣ ≤ E [ h w H ] ≤ a ∑ E [ h w h a ] ≤ 2 e c .
The risk is then at least C − 2 e + 1 c ≥ C − 2 A C-2^{e+1}c\ge C-2A C − 2 e + 1 c ≥ C − 2 A , for either output sign. For an intended vector, on a source NO instance p ( a ˉ ) ≤ 1 / 2 p(\bar{a})\le1/2 p ( a ˉ ) ≤ 1/2 gives R T G ( h a ) ≥ C − 2 A R_T^G(h_a)\ge C-2A R T G ( h a ) ≥ C − 2 A . Also p ( a ˉ ) ≥ 0 p(\bar{a})\ge0 p ( a ˉ ) ≥ 0 , so $R_T^G(-h_a)\ge C+s-2A\ge C-2A, covering even any sign placements excluded by inventory balance. Every zero function has risk C C C . Combining all cases proves, under either inventory rule,
R T G ( N ) ≥ C 0 − 3 q s / 128 for every legal candidate on a source NO instance. R_T^G(N) \ge C_0 - 3qs/128 \quad\text{for every legal candidate on a source NO instance.} R T G ( N ) ≥ C 0 − 3 q s /128 for every legal candidate on a source NO instance.
On a source YES instance choose p ( a ˉ ) ≥ 1 / 2 + q p(\bar a) \ge1/2+q p ( a ˉ ) ≥ 1/2 + q . Its exactly-once legal hinge satisfies
R T G ( h a ) = C + s − 2 E [ h a T ] ≤ C 0 − 2 q s + 2 A ≤ C 0 − ( 2 − 1 / 64 ) q s . R_T^G(h_a) = C+s-2\mathbb{E}[h_aT] \le C_0-2qs+2A \le C_0-(2-1/64)qs. R T G ( h a ) = C + s − 2 E [ h a T ] ≤ C 0 − 2 q s + 2 A ≤ C 0 − ( 2 − 1/64 ) q s .
Finally we supply rational thresholds, including a finite-bit method of obtaining their calibration. One can compute a positive dyadic s ^ \hat{s} s ^ with
∣ s ^ − s ∣ ≤ θ : = 2 − k − n − 12 L |\hat{s}-s| \le\theta:= 2^{-k-n-12}L ∣ s ^ − s ∣ ≤ θ := 2 − k − n − 12 L
in polynomial time. Here is an explicit implementation, so no exact Gaussian comparison is assumed. Write
s = ( 2 π k ) − 1 / 2 ∫ B ∞ ( y − B ) 2 exp ( − y 2 / ( 2 k ) ) d y . s=(2\pi k)^{-1/2}\int_B^\infty(y-B)^2\exp(-y^2/(2k))\,\mathrm{d}y. s = ( 2 π k ) − 1/2 ∫ B ∞ ( y − B ) 2 exp ( − y 2 / ( 2 k )) d y .
Choose an integer b ≥ 1 b \ge1 b ≥ 1 with 2 − b ≤ θ 2^{-b} \le\theta 2 − b ≤ θ , and put M = 8 ( k + b + B + 10 ) M=8(k+b+B+10) M = 8 ( k + b + B + 10 ) . In this calibration, b b b is a precision, M M M a truncation point and N N N below a polynomial degree; they are unrelated to the hidden bias b b b and the candidate N N N above. The omitted tail is at most 2 k ( M + 1 ) exp ( − M 2 / ( 2 k ) ) < 2 − b − 4 2k(M+1)\exp(-M^2/(2k))<2^{-b-4} 2 k ( M + 1 ) exp ( − M 2 / ( 2 k )) < 2 − b − 4 : use the Gaussian second-moment tail, M ≥ 8 k M\ge8k M ≥ 8 k , and log 2 ( 2 k ( M + 1 ) ) ≤ k + M + 2 \log_2(2k(M+1))\le k+M+2 log 2 ( 2 k ( M + 1 )) ≤ k + M + 2 . On [ B , M ] [B,M] [ B , M ] , approximate the exponential uniformly to ξ = 2 − b − 6 / M 3 \xi=2^{-b-6}/M^3 ξ = 2 − b − 6 / M 3 by its Taylor polynomial in y 2 / ( 2 k ) y^2/(2k) y 2 / ( 2 k ) . A degree N ≥ 8 M 2 + 4 ( b + ⌈ log 2 ( M 3 ) ⌉ + 10 ) N\ge8M^2+4(b+\lceil\log_2(M^3)\rceil+10) N ≥ 8 M 2 + 4 ( b + ⌈ log 2 ( M 3 )⌉ + 10 ) suffices: the integral remainder and factorial estimate in Proposition 9.3 give an error at most ( 3 M 2 / ( 2 k ( N + 1 ) ) ) N + 1 ≤ ( 3 / 16 ) N + 1 ≤ ξ (3M^2/(2k(N+1)))^{N+1}\le(3/16)^{N+1}\le\xi ( 3 M 2 / ( 2 k ( N + 1 )) ) N + 1 ≤ ( 3/16 ) N + 1 ≤ ξ . All degrees and numerical bounds are polynomial in k , B , b k,B,b k , B , b .
Approximate ( 2 π k ) − 1 / 2 (2\pi k)^{-1/2} ( 2 π k ) − 1/2 to ξ \xi ξ , using the π \pi π series and square-root bisection already described in Proposition 9.3. Exact integration of the rational polynomial times ( y − B ) 2 (y-B)^2 ( y − B ) 2 , followed by multiplication by that approximation, has error at most 3 ξ M 3 3\xi M^3 3 ξ M 3 . Rounding the result dyadically to error at most 2 − b − 4 2^{-b-4} 2 − b − 4 , and adding the tail bound, gives total error below 2 − b ≤ θ 2^{-b}\le\theta 2 − b ≤ θ . Polynomially many rational coefficients have polynomial bit length, their common denominator has polynomial bit length, and exact integration and polynomial multiplication preserve polynomial bit complexity. Since θ < s / 2 \theta<s/2 θ < s /2 , the resulting s ^ \hat{s} s ^ is positive. The requested b b b is polynomial in e + n e+n e + n , by the explicit formula for L L L .
Take
τ = ( 2 e − q ) s ^ , δ = ( q / 4 ) s ^ . \tau=(2^e-q)\hat{s},\qquad\delta=(q/4)\hat{s}. τ = ( 2 e − q ) s ^ , δ = ( q /4 ) s ^ .
Both rationals are positive and have polynomial binary encoding. The calibration errors for τ \tau τ and τ + δ \tau+\delta τ + δ , compared respectively with C 0 − q s C_0-qs C 0 − q s and C 0 − 3 q s / 4 C_0-3qs/4 C 0 − 3 q s /4 , are at most 2 e θ ≤ q s / 4096 2^e\theta\le qs/4096 2 e θ ≤ q s /4096 . The displayed YES and NO risk bounds therefore imply respectively R T G ( h a ) < τ R_T^G(h_a)<\tau R T G ( h a ) < τ and R T G ( N ) > τ + δ R_T^G(N)>\tau+\delta R T G ( N ) > τ + δ for every legal NO candidate. This is the required promise reduction. Every zero candidate has risk C ≥ C 0 − q s / 128 > τ C\ge C_0-qs/128>\tau C ≥ C 0 − q s /128 > τ . Thus the reduction requires a nonzero occurrence choice. Dropping δ \delta δ proves the exact positive-threshold hardness claim.
The exponentially small binary gap and unbounded squared loss also leave Open Problem 2 open. For the exact Gaussian problem, Theorem 9.5 strengthens the lower bound but supplies neither an equality algorithm nor an exact N P P P \mathrm{NP}^{\mathrm{PP}} NP PP upper bound.
A Fixed Absolute Gap with Unbounded Loss
The exponentially small gap in the preceding reduction can be enlarged while retaining the node cap and both inventory rules. This uses large parameter values and a large threshold; the loss remains unbounded. The distinction matters for comparison with the clipped-loss model of Open Problem 2.
Corollary 9.6 (Fixed absolute Gaussian gap). With the absolute gap fixed to 1 / 4 1/4 1/4 , standard-Gaussian unbounded squared-risk synthesis is N P P P \mathrm{NP}^{\mathrm{PP}} NP PP -complete in the separator sense under either inventory rule. Hardness holds with integer inventory and cap two. The forced-zero restriction W = ( 0 ) W=(0) W = ( 0 ) , cap one, is PP-complete for this fixed gap under either rule, even for bias-free rational targets. The positive rational threshold is part of the input and is not bounded by a numerical constant. Both hardness statements use deterministic polynomial-time many-one promise reductions.
Proof. We first strengthen Theorem 9.5 by rescaling the reduction in its proof, which we call the base reduction. Use its parameters e , n , k , E , B , q , s , s ^ , η , L , τ , δ e,n,k,E,B,q,s,\hat{s},\eta,L,\tau,\delta e , n , k , E , B , q , s , s ^ , η , L , τ , δ , target T T T , and reference energy C 0 = 2 e s C_0=2^e s C 0 = 2 e s . In particular, η = 2 − 10 E \eta=2^{-10E} η = 2 − 10 E is the error scale of the base reduction and L = 2 − B 2 − 4 B − 10 L=2^{-B^2-4B-10} L = 2 − B 2 − 4 B − 10 is its positive lower bound on s s s . Choose K K K as the least nonnegative integer power of two satisfying
K 4 q s ^ ≥ 1. K^4q\hat{s}\ge1. K 4 q s ^ ≥ 1.
The base estimates ensure q s ^ < 1 q\hat{s}<1 q s ^ < 1 , so minimality gives
1 ≤ K 4 q s ^ < 16. 1\le K^4q\hat{s}<16. 1 ≤ K 4 q s ^ < 16.
Supply target T ∗ = K 2 T T_\ast=K^2T T ∗ = K 2 T , cap two, inventory
W ∗ = ( + K ) e + 1 ⊎ ( − K ) e ⊎ { − K B } , W_\ast=(+K)^{e+1}\uplus(-K)^e\uplus\{-KB\}, W ∗ = ( + K ) e + 1 ⊎ ( − K ) e ⊎ { − K B } ,
and threshold τ ∗ = K 4 ( 2 e − q ) s ^ \tau_\ast=K^4(2^e-q)\hat{s} τ ∗ = K 4 ( 2 e − q ) s ^ . Each intended paired hinge becomes K 2 h a K^2h_a K 2 h a : scaling its hidden parameters multiplies its hidden value by K K K , and scaling its output gain contributes another K K K . It uses every occurrence exactly once.
We must control all other graphs because output skips and biases use a single scaled occurrence, so their coefficients lie in K Z K\mathbb{Z} K Z rather than K 2 Z K^2\mathbb{Z} K 2 Z . The base precision gives s / 2 ≤ s ^ ≤ 3 s / 2 s/2\le\hat{s}\le3s/2 s /2 ≤ s ^ ≤ 3 s /2 , and hence
K 4 < 32 q s , K 2 C 0 ≤ 6 ⋅ 2 e s / q ≤ 6 ⋅ 2 − 5 E + n / 2 < 1 100 . K^4<\frac{32}{qs},\qquad K^2C_0\le6\cdot2^e\sqrt{s/q}\le6\cdot2^{-5E+n/2}<\frac{1}{100}. K 4 < q s 32 , K 2 C 0 ≤ 6 ⋅ 2 e s / q ≤ 6 ⋅ 2 − 5 E + n /2 < 100 1 .
Here k = 2 e k=2e k = 2 e , 2 k s ≤ 2 − 10 E 2^ks\le2^{-10E} 2 k s ≤ 2 − 10 E , and E = k + n + 10 E=k+n+10 E = k + n + 10 . Thus K C 0 < 1 / 10 K\sqrt{C_0}<1/10 K C 0 < 1/10 . Every candidate with no input-dependent hidden vertex is affine with coefficients in K Z K\mathbb{Z} K Z . Such a candidate is zero or has Gaussian norm at least K K K . In the latter case its distance from T ∗ T_\ast T ∗ is at least K − K 2 C 0 > 9 K / 10 K-K^2\sqrt{C_0}>9K/10 K − K 2 C 0 > 9 K /10 , so its risk exceeds K 4 C 0 K^4C_0 K 4 C 0 . This class includes a candidate with a constant hidden gate, since the contribution of such a gate has coefficients in K 2 Z ⊆ K Z K^2\mathbb{Z}\subseteq K\mathbb{Z} K 2 Z ⊆ K Z .
For an input-dependent hidden vertex, if − K B -KB − K B is not its bias, the hidden bias is absent or ± K \pm K ± K , its input-vector norm is at least K K K , and its output gain has magnitude at least K K K . The calculation using the second Hermite polynomial Z 2 − 1 Z^2-1 Z 2 − 1 in the proof of Theorem 9.5 therefore gives
∥ N ∥ 2 2 ≥ K 4 ϕ ( 1 ) 2 / 2 > K 4 / 100. \lVert N\rVert_2^2\ge K^4\phi(1)^2/2>K^4/100. ∥ N ∥ 2 2 ≥ K 4 ϕ ( 1 ) 2 /2 > K 4 /100.
This bound remains valid with every affine skip and bias, including all other placements of − K B -KB − K B . Since ∥ T ∗ ∥ 2 2 ≤ K 4 C 0 \lVert T_\ast\rVert_2^2\le K^4C_0 ∥ T ∗ ∥ 2 2 ≤ K 4 C 0 and C 0 ≤ η C_0\le\eta C 0 ≤ η , its risk is greater than K 4 C 0 K^4C_0 K 4 C 0 .
If − K B -KB − K B is the hidden bias, write
N = ± K 2 h w + K A , N=\pm K^2h_w+KA, N = ± K 2 h w + K A ,
where A A A is integer affine and w w w has j ≤ k j\le k j ≤ k nonzero unit entries. If A ≠ 0 A\ne0 A = 0 , its norm is at least one; monotonicity gives ∥ h w ∥ 2 ≤ s \lVert h_w\rVert_2\le\sqrt{s} ∥ h w ∥ 2 ≤ s (or zero for j = 0 j=0 j = 0 ). Consequently
∥ N − T ∗ ∥ 2 ≥ K ( 1 − K ( s + C 0 ) ) > 4 K / 5 , \lVert N-T_\ast\rVert_2\ge K\left(1-K(\sqrt{s}+\sqrt{C_0})\right)>4K/5, ∥ N − T ∗ ∥ 2 ≥ K ( 1 − K ( s + C 0 ) ) > 4 K /5 ,
using s ≤ C 0 s\le C_0 s ≤ C 0 . Its risk again exceeds K 4 C 0 K^4C_0 K 4 C 0 . With A = 0 A=0 A = 0 , the candidate ± K 2 h w \pm K^2h_w ± K 2 h w is K 2 K^2 K 2 times the base candidate ± h w = ± ρ ( w ⋅ X − B ) \pm h_w=\pm\rho(w\cdot X-B) ± h w = ± ρ ( w ⋅ X − B ) , whose hidden bias is − B -B − B , or zero when j = 0 j=0 j = 0 . Under exactly-once use, the same occurrence count forces j = k j=k j = k ; under at-most-once use, the proof of Theorem 9.5 already handles every j < k j<k j < k . All corresponding risk bounds therefore multiply exactly by K 4 K^4 K 4 . The base YES witness has risk below τ ∗ \tau_\ast τ ∗ , and every base NO candidate has risk above
τ ∗ + K 4 ( q / 4 ) s ^ ≥ τ ∗ + 1 / 4. \tau_\ast+K^4(q/4)\hat{s}\ge\tau_\ast+1/4. τ ∗ + K 4 ( q /4 ) s ^ ≥ τ ∗ + 1/4.
The discarded large-risk cases also exceed this threshold, because C 0 > τ + δ C_0>\tau+\delta C 0 > τ + δ for the unscaled base reduction. Indeed τ + δ = ( 2 e − 3 q / 4 ) s ^ \tau+\delta=(2^e-3q/4)\hat{s} τ + δ = ( 2 e − 3 q /4 ) s ^ , and the base calibration ∣ s ^ − s ∣ ≤ θ |\hat{s}-s|\le\theta ∣ s ^ − s ∣ ≤ θ , with 2 e θ ≤ q s / 4096 2^e\theta\le qs/4096 2 e θ ≤ q s /4096 , gives
τ + δ ≤ ( 2 e − 3 q 4 ) s + 2 e θ ≤ C 0 − ( 3 4 − 1 4096 ) q s < C 0 , \tau+\delta\le\left(2^e-\frac{3q}{4}\right)s+2^e\theta\le C_0-\left(\frac{3}{4}-\frac{1}{4096}\right)qs<C_0, τ + δ ≤ ( 2 e − 4 3 q ) s + 2 e θ ≤ C 0 − ( 4 3 − 4096 1 ) q s < C 0 ,
since q , s > 0 q,s>0 q , s > 0 . Each discarded risk therefore exceeds K 4 C 0 > K 4 ( τ + δ ) = τ ∗ + K 4 ( q / 4 ) s ^ K^4C_0>K^4(\tau+\delta)=\tau_\ast+K^4(q/4)\hat{s} K 4 C 0 > K 4 ( τ + δ ) = τ ∗ + K 4 ( q /4 ) s ^ . Thus the reduction has the required fixed gap under both rules.
All parameters have polynomial binary length. Indeed s ^ ≥ L / 2 = 2 − B 2 − 4 B − 11 \hat{s} \ge L/2 = 2^{-B^2-4B-11} s ^ ≥ L /2 = 2 − B 2 − 4 B − 11 , so
log 2 K ≤ ⌊ B 2 + 4 B + n + 11 4 ⌋ . \log_2 K \le\left\lfloor\frac{B^2+4B+n+11}{4} \right\rfloor. log 2 K ≤ ⌊ 4 B 2 + 4 B + n + 11 ⌋ .
Rational comparisons find K K K in polynomial time; K B , K 2 KB,K^2 K B , K 2 , the scaled target, and τ ∗ \tau_* τ ∗ then have polynomial encodings. The separator in Proposition 9.3 applies with the fixed gap 1 / 4 1/4 1/4 , completing this classification. For the forced-zero restriction, start with the PP reduction in Theorem 9.4, whose gap is δ = 2 − n − 2 \delta= 2^{-n-2} δ = 2 − n − 2 . Multiply its target by λ = 2 ⌈ ( n + 2 ) / 2 ⌉ \lambda= 2^{\lceil(n+2)/2\rceil} λ = 2 ⌈( n + 2 ) /2 ⌉ and its positive threshold by λ 2 \lambda^2 λ 2 . The sole candidate remains zero, all risks multiply by λ 2 \lambda^2 λ 2 , and the resulting NO margin is at least λ 2 δ ≥ 1 > 1 / 4 \lambda^2\delta\ge1 > 1/4 λ 2 δ ≥ 1 > 1/4 . Target size and coefficient bit lengths remain polynomial, and bias-freeness is preserved. The fixed-candidate PP separator still applies, proving the second classification. □
This is an absolute-gap result with no bound on the numerical threshold. In the cap-two reduction, writing t = K 4 q ^ s ^ ∈ [ 1 , 16 ) t = K^4 \hat{q}\hat{s} \in[1,16) t = K 4 q ^ s ^ ∈ [ 1 , 16 ) gives
τ ∗ = t ( 2 e + n − 1 ) . \tau_* = t(2^{e+n}-1). τ ∗ = t ( 2 e + n − 1 ) .
Thus the threshold can be exponentially large even though its encoding is polynomial. Clipping the loss destroys the scaling identity, so the corollary does not resolve Open Problem 2 or give a bounded-loss constant-gap classification.
Parameter Placements in the Boolean Construction
This appendix proves the placement statement used in the Boolean hardness reduction. The case analysis has four structural stages. First, coefficient and mixed-difference obstructions restrict gain positions; Lemmas A.10 to A.14 exclude the unintended external and internal cases. Second, Lemmas A.15 and A.16 place the two finite negative parameters at hidden biases and the gains at those biased vertices’ output edges. Third, Lemma A.17 excludes actual coupling edges, using coefficient identities on the clean query cube. Finally, Lemmas A.18 and A.19 use local coefficient comparisons and the full target to identify one shared affine score. The final subsection assembles these statements into Lemma 4.9 ; no later global identification is used to justify an earlier edge exclusion.
All parameters and inputs are those of the calibrated construction. For the candidate at hand, the clean block is a query block supplied by Lemma 4.7 , with every coordinate outside it fixed to zero. Its restricted cube is the clean query cube, also called the clean cube or the query cube, and the restriction of f f f to it is the clean query target, or the query target. A clean query input is a query coordinate of the clean block. Unless a face or the full target is named, a winner, clean winner or clean query winner is a winning vertex of the clean query cube; in Lemma A.19 , which uses the full target, a winner is a winning vertex of f f f . At a tested input, a coordinate is selected when its value there is one, and an input edge, direct query term or input primitive is selected when its coordinate is selected. The selected query sum, or selected power-of-two sum, of a hidden vertex is the sum of the integers 2 i 2^i 2 i over its selected direct query inputs, where F 2 i + 2 w F2^i + 2w F 2 i + 2 w is the actual value on that input edge. A coordinate is a supported coordinate of a hidden vertex, and of its direct query sum or direct affine form, when its direct input edge into that vertex is present, that is, when it lies in the support of that vertex’s direct input coefficient vector.
We retain the actual activation patterns and the formal identities justified by Lemma 4.6 . A missing edge or bias has coefficient zero and consumes no occurrence. Hidden vertices are numbered in topological order, with at most three of them. For the fixed tested input, η v ∈ { 0 , 1 } \eta_v \in\{0,1\} η v ∈ { 0 , 1 } is the actual activation indicator of hidden vertex v v v , as defined in § 2 . The same convention gives η h , η g \eta_h,\eta_g η h , η g when the vertices are named h , g h,g h , g ; the input is suppressed from the notation. The tuple of these indicators is the actual activation pattern at the input, also called its activation mask, or simply its mask; the mask of a single hidden vertex is its indicator. Freezing the pattern replaces each hidden ReLU by its affine preactivation or by zero according to its indicator, as in Lemma 4.6 .
A frozen transfer from a vertex u u u to a later vertex v v v is the sum over directed paths from u u u to v v v of the edge-weight product times the activation indicators of the hidden vertices after u u u on that path. It is zero if there is no contributing path. This is the coefficient transmitting the value at u u u through the frozen computation, by expansion in topological order. The transfer is short when each of its paths passes through at most one hidden vertex strictly between u u u and v v v . A frozen expression is a vertex value, preactivation, transfer or output computed with the pattern frozen in this way and treated as a Laurent polynomial in the formal variables w , Γ w,\Gamma w , Γ recalled below. The frozen pattern, or frozen mask, is the actual pattern used for it.
The source vertex of a gain edge is its gain source, and the gain is said to originate at that vertex. The prefix ending at a hidden vertex consists of that vertex and the earlier hidden vertices from which it can be reached, with the edges and biases entering them. Its value is the value of that vertex, and we also speak of the prefix ending at h h h when that value is named h h h . The direct affine form of a hidden vertex is the sum of its direct query terms and its bias. A first affine query prefix is a prefix ending at a hidden vertex that, under the frozen pattern, receives no contribution from any earlier hidden vertex, for instance because every hidden predecessor is inactive. Its preactivation is then its direct affine form, an affine function of the clean query inputs called its first affine form; its value is a first affine value, and its final vertex is a first affine source when a gain originates there.
A gain relay is a hidden-to-hidden gain edge; it carries the value of its source vertex, multiplied by the gain, into its target vertex. The ordinary output coefficient of a hidden vertex is the coefficient of its hidden-to-output edge when that edge does not carry a gain, and zero when that edge is absent. A hinge, also called a gate, is a hidden ReLU vertex; the first, middle and last gates are the hidden vertices in topological order. On a set of inputs, a hinge is effective, or genuine, unless it equals its preactivation throughout that set or vanishes throughout it.
Recall that w = M − 1 w=M^{-1} w = M − 1 and Γ = M 9 \Gamma=M^9 Γ = M 9 are the two formal variables of the construction. Put D = w − 1 D=w^{-1} D = w − 1 , so that the two gain occurrences + M , − M +M,-M + M , − M are + D , − D +D,-D + D , − D . When + D +D + D is a hidden-to-output edge, its gain source is the positive-output source ; when − D -D − D is one, its gain source is the negative-output source . A path, transfer or frozen expression is gain-free when no gain occurrence enters it, that is, when no edge or bias contributing to it carries + D +D + D or − D -D − D ; a vertex is gain-free when its value is. When both gains are hidden-to-output edges, the signed gain contribution at an input is D D D times the value of the source of + D +D + D minus D D D times the value of the source of − D -D − D , and its value there is the signed gain value.
By Lemma 4.6 , the frozen output at a tested input is identically its value zero or one, so its coefficient of every nonzero power of w w w vanishes. The coefficient of w − 1 w^{-1} w − 1 in the frozen output, or in a term, path or transfer contributing to it, is called its inverse coefficient. The inverse equation, or inverse-w w w equation, is the requirement that the inverse coefficient of the frozen output vanish; inverse-square terms and equations refer in the same way to w − 2 w^{-2} w − 2 . For a gain-free expression h h h under a frozen pattern, write h 0 h^0 h 0 for its coefficient of w 0 w^0 w 0 , still a Laurent polynomial in Γ \Gamma Γ . This notation does not assert that h 0 h^0 h 0 numerically dominates every positive power of w w w , since Γ = M 9 \Gamma=M^9 Γ = M 9 . Signs are determined after the prescribed specialization. For a gain-free frozen expression h h h , write h ( 1 ) h^{(1)} h ( 1 ) for its derivative : its formal derivative at w = 0 w=0 w = 0 , holding Γ \Gamma Γ fixed, or equivalently its coefficient of w 1 w^1 w 1 . This does not differentiate the activation pattern or the numerical specialization. The leading value of a gain-free expression h h h is h 0 h^0 h 0 .
For a nongain primitive c c c , its leading value c 0 c^0 c 0 is also called a nongain leading primitive, and c − c 0 c-c^0 c − c 0 is its correction. More generally, the correction of a gain-free expression h h h is h − h 0 h-h^0 h − h 0 ; when it is a scalar multiple of w w w , that scalar is also called its correction. Every nongain leading primitive is a signed finite value F n Fn F n , a value Γ \Gamma Γ , or a value s 0 = 1 / ( q Γ ) s_0 = 1/(q\Gamma) s 0 = 1/ ( q Γ ) . For a finite primitive, the integer n n n satisfies 1 ≤ ∣ n ∣ ≤ Q 1 \le|n| \le Q 1 ≤ ∣ n ∣ ≤ Q ; this generic n n n is not the number of source numbers. The values Γ \Gamma Γ and s 0 s_0 s 0 are the nonfinite primitives. When an ordinary scalar coefficient c c c is finite or absent, write c 0 = F n c c^0 = F n_c c 0 = F n c , where n c ∈ Z n_c \in\mathbb{Z} n c ∈ Z and n c = 0 n_c = 0 n c = 0 for an absent coefficient. The same indexed convention applies to other scalar coefficient names, such as n t , n a , n b n_t,n_a,n_b n t , n a , n b . Dividing such a finite leading value by F F F normalizes it, so the normalized value of F n c F n_c F n c is n c n_c n c . For a direct affine form U U U , write L U L_U L U for the normalized finite part of U 0 U^0 U 0 , so that U 0 U^0 U 0 is F L U F L_U F L U plus its Γ \Gamma Γ and s 0 s_0 s 0 terms; for a first affine form it is the first affine leading sum. The finite negative primitives both have leading value − P -P − P . A positive finite primitive has correction zero, 2 w 2w 2 w , − w -w − w , or − 3 w / 2 -3w/2 − 3 w /2 ; a finite negative has correction zero or w w w .
The Γ \Gamma Γ -degree and w w w -degree of a monomial c Γ j w k c\Gamma^j w^k c Γ j w k , with c c c a nonzero scalar, are j j j and k k k , so a factor s 0 = 1 / ( q Γ ) s_0 = 1/(q\Gamma) s 0 = 1/ ( q Γ ) has Γ \Gamma Γ -degree − 1 -1 − 1 . An expression has Γ \Gamma Γ -degree j j j , or w w w -degree k k k , when all its monomials do, and it has nonnegative, nonzero, positive or negative such degree when all its monomials do; its constant Γ \Gamma Γ -degree part is the sum of its monomials of Γ \Gamma Γ -degree zero. Since Γ s 0 = 1 / q \Gamma s_0 = 1/q Γ s 0 = 1/ q , one Γ \Gamma Γ factor and one s 0 s_0 s 0 factor in a product multiply to the constant 1 / q 1/q 1/ q , and such a pair of factors is called a balanced pair, or a balanced Γ , s 0 \Gamma,s_0 Γ , s 0 pair. A product or path contribution is balanced when it contains at least one Γ \Gamma Γ or s 0 s_0 s 0 factor and its Γ \Gamma Γ and s 0 s_0 s 0 factors are equal in number, so that they form balanced pairs and its Γ \Gamma Γ -degree is zero. It is pure balanced when it has no other factor, so that its value is 1 / q 1/q 1/ q or 1 / q 2 1/q^2 1/ q 2 .
The winning stencil of the clean query cube, also called its ordinary winning stencil, consists of its all-ones vertex and its single-coordinate deletions. These vertices win by Lemma 4.7 and affinely span the query cube. We use the corresponding stencil, the winning stencil of the face or face stencil, on faces obtained by fixing at most two coordinates to zero or one coordinate to one. Those vertices also win: even deleting three coordinates leaves a weighted sum at least 3 Q − 3 ( Q + 1 ) / 2 > Q 3Q - 3(Q + 1)/2 > Q 3 Q − 3 ( Q + 1 ) /2 > Q . Winning span, or winning affine span, is the consequence of this spanning used throughout: an affine function of the free query coordinates, whose coefficients may be Laurent polynomials in Γ \Gamma Γ , that takes one value at every vertex of the winning stencil of the cube or face in question equals that constant, so its query coefficients vanish; in particular, one that vanishes at every winner vanishes identically. When two-coordinate deletions are used in addition, they are specified explicitly.
The robust face statement of Lemma 4.7 is its last assertion, which covers exactly these faces. The robust variation argument is the paragraph of the proof of Lemma A.14 headed “A first finite-negative-biased hinge has a robust leading variation”; it shows that a first hidden vertex with a finite negative bias takes three distinct leading values at winners of the clean cube. For a hinge with a finite negative bias and direct query sum U U U , a leading-zero boundary is a tested input at which the normalized leading value of U U U equals Q Q Q , so that the leading preactivation vanishes and the hinge value is its correction. The score of a hinge is its preactivation without its negative bias. When the score is affine, the effective coefficient of a coordinate x j x_j x j is its total coefficient there: the direct input coefficient plus the sharing weight times the wire’s coefficient of x j x_j x j , with the sharing weight and wire as in Lemma 4.9.
Coefficient and Transfer Comparisons
Several comparisons recur in the placement cases. We prove them once with the full numerical ranges, rather than infer distinctness solely from genericity. In particular, equality of leading finite values does not identify the two negative primitives.
Lemma A.1 (Positive primitives and short products). Two positive nongain primitives with the same zeroth w w w -coefficient have the same full value. A nonzero product of two nongain leading primitives cannot equal a sum or difference of at most two nongain leading primitives. The latter assertion permits signed finite factors and signed finite summands.
Proof. Among positive finite leading integers, the source values β j \beta_j β j are below T T T , the query values are powers of two at most ( Q + 1 ) / 2 (Q+1)/2 ( Q + 1 ) /2 , and the anchor value Q − T Q-T Q − T is larger than both ranges. The control value is Q Q Q . The source values are not powers of two. Thus equality can occur only within a single numerical family, whose full correction is fixed. Equal Γ \Gamma Γ or s 0 s_0 s 0 values are already identical.
For products, the six unordered type pairs have leading forms
F 2 n 1 n 2 , F n Γ , F n s 0 , Γ 2 , 1 / q , s 0 2 . F^2 n_1 n_2,\quad Fn\Gamma,\quad Fn s_0,\quad\Gamma^2,\quad1/q,\quad s_0^2. F 2 n 1 n 2 , F n Γ , F n s 0 , Γ 2 , 1/ q , s 0 2 .
A sum of two finite primitives has absolute value at most 2 P < F 2 2P < F^2 2 P < F 2 , so it cannot equal the first form. At exponent Γ \Gamma Γ , or at exponent Γ − 1 \Gamma^{-1} Γ − 1 after scaling by q q q , a sum of two primitives has coefficient of absolute
өзиниң value at most two. The corresponding finite-times-nonfinite product has coefficient of absolute value at least F > 2 F>2 F > 2 . A square Γ 2 \Gamma^2 Γ 2 or s 0 2 s_0^2 s 0 2 has an exponent unavailable to a primitive sum. Finally, a sum with no nonzero Γ \Gamma Γ -exponent has coefficient an integer multiple of F F F , and hence cannot equal 1 / q 1/q 1/ q . These cases also cover absent summands and every permitted sign. □
The next obstruction applies at one winning point, before any global graph normalization. It also applies after gain terms cancel exactly in a frozen expression. It does not require deleting their edges from the network.
Lemma A.2 (The gain-free coefficient obstruction). In an actual frozen output expression, the paths avoiding both gain occurrences have Γ 0 w 0 \Gamma^0w^0 Γ 0 w 0 -coefficient
F Z q 2 + k 1 q + k 2 q 2 , Z ∈ Z , 0 ≤ k 1 , k 2 ≤ p . \frac{FZ}{q^2}+\frac{k_1}{q}+\frac{k_2}{q^2},\qquad Z\in\mathbb{Z},\quad0\leq k_1,k_2\leq p. q 2 F Z + q k 1 + q 2 k 2 , Z ∈ Z , 0 ≤ k 1 , k 2 ≤ p . This coefficient cannot equal one. Consequently, at an output equal to one, the sum of all gain-containing path contributions cannot be identically zero. If all gain-containing contributions vanish or cancel exactly, output one is impossible.
Proof. A gain-free path has nonnegative w w w -degree. For its zeroth coefficient, every finite factor therefore contributes its F n F^n F n term, giving a factor F F F . A balanced product contains at most two Γ , s 0 \Gamma,s_0 Γ , s 0 pairs, so its denominator divides q 2 q^2 q 2 . This accounts for the term F Z / q 2 FZ/q^2 F Z / q 2 , with arbitrary signs coming only from finite negative factors.
The length of a path is its number of parameter factors, counting the input edge or bias at which it starts. That input edge or bias is the origin of the path, and origin counts count such starting parameters. Elsewhere, as in “at the origin” or “the origin test”, the origin is the all-zero input. The origin test is the requirement that the candidate output the target value zero at the origin, and a unit test is the same requirement at a unit vector, where the target is also zero by Lemma 4.8 . Without a finite factor, a constant Γ \Gamma Γ -degree requires either one balanced pair on a length-two path or two pairs on a length-four path. Their values are 1 / q 1/q 1/ q and 1 / q 2 1/q^2 1/ q 2 , respectively, and their signs are positive. For a length-two path, a selector used as an input edge or bias enters a hidden vertex, and the Γ \Gamma Γ factor must then be that vertex’s output edge. Such a selector therefore has only one possible final Γ \Gamma Γ edge. A selector used as the output edge can instead meet two Γ \Gamma Γ origins, but this consumes both available Γ \Gamma Γ occurrences and supplies at most two paths in total. Otherwise, when no selector output edge meets two Γ \Gamma Γ origins, each pair can be charged to a distinct selector occurrence. Thus the number k 1 k_1 k 1 of such paths is at most p p p , using p ≥ 4 p\geq4 p ≥ 4 .
A length-four path must use all three hidden vertices in their topological order. Its last three edges are then fixed. If the initial occurrence is a selector, at most p p p such initial occurrences are available. If it is a Γ \Gamma Γ occurrence, the last three edges contain one Γ \Gamma Γ and two selectors, leaving at most one possible initial Γ \Gamma Γ occurrence. These alternatives cannot occur together for the same fixed last three edges. This proves k 2 ≤ p k_2\leq p k 2 ≤ p .
Equality to one would give
F Z + q k 1 + k 2 = q 2 . FZ+qk_1+k_2=q^2. F Z + q k 1 + k 2 = q 2 . However, 0 ≤ q k 1 + k 2 ≤ q 2 − 1 < F 0\leq qk_1+k_2\leq q^2-1<F 0 ≤ q k 1 + k 2 ≤ q 2 − 1 < F and q 2 < F q^2<F q 2 < F , which contradicts this equation modulo F F F . The cancellations in the statement concern formal contributions of gain-containing paths, so they leave the above calculation of the gain-free Γ 0 w 0 \Gamma^0w^0 Γ 0 w 0 -coefficient valid under the original pattern. □
The query target also requires negative parameters at two distinct hidden preactivations. This elementary allocation statement restricts signed output terms in the subsequent cases.
Lemma A.3 (Negative allocation). Let a permitted candidate represent the clean query target of Lemma 4.7 on its restricted cube. Then at least two distinct hidden vertices must receive negative preactivation contributions. At most one of the three supplied negative occurrences can therefore be an output parameter. If all output parameters are positive, the output bias is absent, every direct input-to-output term vanishes on the restriction, and every hidden value with a direct edge to the output is zero at the origin and at the unit vectors of the restricted cube. If the candidate represents the full target f f f and all its present output parameters are positive, all direct input-to-output edges are absent globally.
Proof. A hidden vertex with only nonnegative incoming coefficients, a nonnegative bias and nonnegative predecessor values applies ρ \rho ρ to a nonnegative expression. Its ReLU is the identity on the restricted cube. If negative contributions enter at most one hidden vertex, substitute these positive identity gates in topological order. The result is an affine function plus a scalar multiple of the sole remaining ReLU of an affine function. This contradicts Lemma 4.7. A negative input edge fixed to zero contributes nothing and cannot create an additional effective nonlinearity. Only three negative occurrences are supplied, so at most one can be assigned to the output.
With positive output parameters every output summand is nonnegative. A positive output bias would contradict the target value zero at the origin. A present positive edge from a query input directly to the output would contradict its value zero at the corresponding restricted unit vector. Other external inputs are fixed to zero and their direct output terms vanish on the restriction. For the full target, every unit vector also loses. For a source coordinate, β j ≤ Q − 1 \beta_j \le Q-1 β j ≤ Q − 1 gives F β j ≤ P − F < P − w F\beta_j \le P-F < P-w F β j ≤ P − F < P − w . An anchor baseline is F ( Q − T ) − w < P − w F(Q-T)-w < P-w F ( Q − T ) − w < P − w , since T > 0 T>0 T > 0 . A query baseline satisfies ( P − w ) − ( F 2 i + 2 w ) = F ( Q − 2 i ) − 3 w > 0 (P-w)-(F2^i+2w)=F(Q-2^i)-3w>0 ( P − w ) − ( F 2 i + 2 w ) = F ( Q − 2 i ) − 3 w > 0 , since Q − 2 i ≥ 1 Q-2^i\ge1 Q − 2 i ≥ 1 and F > 3 w F>3w F > 3 w . The control baseline is P − 3 w / 2 < P − w P-3w/2<P-w P − 3 w /2 < P − w . Thus f ( e t ) = 0 f(e_t)=0 f ( e t ) = 0 for every coordinate t t t , and the same positive-summand argument excludes every direct input-to-output edge globally. The remaining nonnegative hidden summands must individually vanish at all those tests. □
Helmbold and Long proved a sign restriction for networks with nonnegative edge weights [21 Lemma 12] . For a Boolean function, call it supermodular when every pairwise mixed difference, with all other coordinates fixed, is nonnegative. We record the Boolean-cube form of their restriction and its consequence for the query target.
Lemma A.4 (Nonnegative-edge restriction). A ReLU network with nonnegative edge weights, arbitrary hidden biases and an affine output with nonnegative edge weights is monotone and supermodular on the Boolean cube. An arbitrary output bias does not affect the supermodular conclusion. Such a network cannot represent the query target.
Proof. Monotonicity follows by induction through the nonnegative edges. Positive sums preserve supermodularity, and affine input terms have zero mixed difference. It remains to check that taking positive parts preserves the mixed-difference inequality. On one two-coordinate face, let A A A and E E E be the preactivation values at the corners where both free coordinates are zero and where both are one, respectively. Let B B B and C C C be the values at the two mixed corners, with the first free coordinate equal to one at B B B . Monotonicity gives A ≤ B , C ≤ E A\le B,C\le E A ≤ B , C ≤ E , and supermodularity gives A + E ≥ B + C A+E\ge B+C A + E ≥ B + C . If A ≥ 0 A\ge0 A ≥ 0 , taking positive parts changes no term. If E ≤ 0 E\le0 E ≤ 0 , all four positive parts are zero. Otherwise A < 0 ≤ E A<0\le E A < 0 ≤ E . When both B , C ≤ 0 B,C\le0 B , C ≤ 0 , the required inequality is E ≥ 0 E\ge0 E ≥ 0 . When exactly one is positive, it is bounded above by E E E . When both are positive, E ≥ B + C − A ≥ B + C E\ge B+C-A\ge B+C E ≥ B + C − A ≥ B + C . Thus positive part preserves the mixed-difference inequality in every case. Induction proves the assertion for hidden nodes and their nonnegative output sum. The query target has an OR face, a two-coordinate face on which it is the OR of the free coordinates, given by the weight-( Q − 1 ) (Q-1) ( Q − 1 ) background in the proof of Lemma 4.7; its mixed difference is negative, so the target cannot belong to this class. □
The remaining short-transfer equalities require slightly more than a primitive-versus-product comparison. They occur when two gain paths could meet at a later vertex. All missing coefficients below are interpreted as zero.
Lemma A.5 (Two short-transfer equations). Let b b b be a positive nongain leading primitive and c 3 c_3 c 3 a nonzero nongain leading primitive. Let c 2 , e c_2,e c 2 , e be nongain leading primitives or zero, with signed finite values permitted. Then
c 2 = c 3 ( b − e ) c_2=c_3(b-e) c 2 = c 3 ( b − e ) forces c 2 = 0 c_2=0 c 2 = 0 and b = e b=e b = e . Also
b ( c 2 + c 3 e ) = c 3 b(c_2+c_3e)=c_3 b ( c 2 + c 3 e ) = c 3 is impossible. In the first conclusion, b = e b=e b = e identifies their full positive primitive values.
Proof. Suppose first that c 2 ≠ 0 c_2\ne0 c 2 = 0 , and divide the first equality by c 3 c_3 c 3 . If both c 2 , c 3 c_2,c_3 c 2 , c 3 are finite, their ratio is a nonzero rational of absolute value at most Q Q Q . A nonzero finite difference b − e b-e b − e is an integer multiple of F > Q F > Q F > Q ; a difference involving nonfinite primitives has a nonzero Γ \Gamma Γ -exponent unless the identical terms cancel to zero. Neither possibility gives that ratio.
If exactly one of c 2 , c 3 c_2,c_3 c 2 , c 3 is finite, the ratio is a single Γ \Gamma Γ or s 0 s_0 s 0 monomial whose coefficient, relative to that primitive, is either q F n qF n q F n or 1 / ( F n ) 1/(F n) 1/ ( F n ) . It cannot match the unit coefficient of one nonfinite primitive in b − e b-e b − e ; an additional finite term or a second exponent cannot help. If both are nonfinite, their ratio is one or has exponent two or minus two in Γ \Gamma Γ . A difference of primitives cannot give these values: its exponents are only minus one, zero and one, and its nonzero constant coefficient is divisible by F F F .
Thus c 2 ≠ 0 c_2 \ne0 c 2 = 0 is impossible. With c 2 = 0 c_2=0 c 2 = 0 , the equation and c 3 ≠ 0 c_3 \ne0 c 3 = 0 give b = e b=e b = e . The equal value is positive, so Lemma A.1 identifies the full primitives.
For the second equation, first take c 3 c_3 c 3 nonfinite and compare the coefficient at its own Γ \Gamma Γ -exponent, dividing by its coefficient. The term b c 2 bc_2 b c 2 can contribute only an integer multiple of F F F at this exponent. The term b c 3 e bc_3e b c 3 e contributes either an integer multiple of F F F , or the single positive balanced value 1 / q 1/q 1/ q , or zero. The resulting equation is 1 = F Z + I / q 1=FZ+I/q 1 = F Z + I / q with integer Z Z Z and I ∈ { 0 , 1 } I \in\{0,1\} I ∈ { 0 , 1 } . Clearing q q q contradicts reduction modulo F F F .
It remains that c 3 = F n 3 c_3=Fn_3 c 3 = F n 3 is finite, with n 3 ≠ 0 n_3 \ne0 n 3 = 0 . If b b b is finite, every constant-degree contribution to b c 2 + b c 3 e bc_2+bc_3e b c 2 + b c 3 e is divisible by F 2 F^2 F 2 . Equality to F n 3 Fn_3 F n 3 would make n 3 n_3 n 3 divisible by F F F , although 0 < ∣ n 3 ∣ ≤ Q < F 0<|n_3|\le Q<F 0 < ∣ n 3 ∣ ≤ Q < F .
If b b b is nonfinite, the only constant-degree contributions to b c 2 bc_2 b c 2 and b e be b e are balanced products. Write their indicators as I 1 , I 2 ∈ { 0 , 1 } I_1,I_2 \in\{0,1\} I 1 , I 2 ∈ { 0 , 1 } . The constant-degree equation becomes
I 1 + F n 3 I 2 = q F n 3 . I_1+Fn_3I_2=qFn_3. I 1 + F n 3 I 2 = q F n 3 .
Its right side after rearrangement has absolute value ∣ F n 3 ( q − I 2 ) ∣ ≥ F ( q − 1 ) > 1 |Fn_3(q-I_2)| \ge F(q-1)>1 ∣ F n 3 ( q − I 2 ) ∣ ≥ F ( q − 1 ) > 1 , whereas I 1 ≤ 1 I_1\le1 I 1 ≤ 1 . This contradiction exhausts the types and signs. □
Neither gain may be placed at the affine output bias. Unlike the later graph cases, its proof uses only the origin.
Lemma A.6 (Gain output biases). A candidate for f f f cannot assign either + D +D + D or − D -D − D to its output bias.
Proof. Suppose first that the output bias is − D -D − D . Since the target is zero at the origin, the other output paths must contribute coefficient one at w − 1 Γ 0 w^{-1}\Gamma^0 w − 1 Γ 0 there. Only paths containing + D +D + D can supply this coefficient. Finite factors contribute an integer multiple of F / q F/q F / q . A path without a finite factor must contain one balanced Γ , s 0 \Gamma,s_0 Γ , s 0 pair in addition to + D +D + D .
At the all-zero input there are at most two such paths in a graph with at most three hidden vertices. Indeed, suppose first that + D +D + D is either the bias of the first hidden vertex or an output edge, in the latter case the final edge of its path. Then the other two factors must form the balanced pair, and the path passes through two hidden vertices, one of which carries + D +D + D as its bias or output edge. The other hidden vertex has at most two choices, so there are at most two such paths. If + D +D + D is an internal edge, the only possible origin of the path is the bias of the source vertex of + D +D + D . The path then ends at the output edge of the target vertex of + D +D + D , so there is at most one such path. If + D +D + D is the bias of the middle hidden vertex, at most one path has its other two factors forming a balanced pair. If + D +D + D is the bias of the last hidden vertex, there is no such path. An input gain contributes zero at the origin. Two balanced pairs and a gain need five factors and do not occur.
Thus the coefficient equation has the form
F Z + k = q , Z ∈ Z , 0 ≤ k ≤ 2 , FZ+k=q,\qquad Z\in\mathbb{Z},\qquad0\le k\le2, F Z + k = q , Z ∈ Z , 0 ≤ k ≤ 2 , after clearing q q q . It is impossible modulo F F F , since 2 < q < F 2<q<F 2 < q < F .
For output bias + D +D + D , the negative-gain paths must instead supply coefficient one. Finite-factor paths give an F / q 2 F/q^2 F / q 2 multiple. Every gain path with no finite factor again has only one balanced pair, with nonnegative count at most p p p . The coefficient equation after clearing q 2 q^2 q 2 is
F Z + q k = q 2 , 0 ≤ k ≤ p . FZ+qk=q^2,\qquad0\le k\le p. F Z + q k = q 2 , 0 ≤ k ≤ p . Since q k ≤ q ( q − 1 ) < q 2 < F qk\le q(q-1)<q^2<F q k ≤ q ( q − 1 ) < q 2 < F , this too is impossible modulo F F F . If the other gain is unused, there is no inverse coefficient to cancel the bias at all.
A Single Effective Gain
We first exclude candidates in which one gain is unavailable on the query restriction. The two signs require different statements. The negative-gain statement is pointwise; the positive internal-gain statement uses equality on the whole restricted cube. This distinction is needed when a gain is assigned to an input coordinate fixed to zero.
Lemma A.7 (The pointwise negative-gain obstruction). At a clean query winner, output one is impossible if every individual active path contribution containing + D +D + D is zero. The occurrence − D -D − D may occupy any permitted position. In particular this applies when + D +D + D is unused, lies on a fixed-zero input edge, or is an output edge whose source is inactive.
Proof. Fix the actual pattern at the tested point. The + D +D + D contributions vanish by hypothesis. The paths avoiding both gains have constant coefficient F Z / q 2 + r 0 FZ/q^2+r_0 F Z / q 2 + r 0 , where
r 0 = k 1 q + k 2 q 2 , 0 ≤ r 0 < 1 , r_0 = \frac{k_1}{q}+\frac{k_2}{q^2}, \qquad0 \le r_0 < 1, r 0 = q k 1 + q 2 k 2 , 0 ≤ r 0 < 1 ,
by Lemma A.2. There must therefore be a nonzero − D -D − D contribution. All selected external-input primitives are actual query values, even though their levels need not match their coordinate labels. Every nonzero path under discussion also avoids + D +D + D , since individual path contributions containing that occurrence vanish by hypothesis. Terms with that occurrence are discarded only as zero contributions in the actual pattern.
An output edge.
Here − D -D − D is the hidden-to-output edge from a hidden vertex h h h , and h h h also denotes that vertex’s frozen value. Write the frozen output as A − D h A-Dh A − D h , where A A A contains all other output terms, including any signed finite ones. The inverse-w w w equation forces h 0 = 0 h^0=0 h 0 = 0 as a Laurent polynomial in Γ \Gamma Γ . The next display groups the terms of a constant coefficient by their number of finite factors, and its symbols are explained after it in the order of its terms. For a prefix ending at h h h with at most three hidden vertices, the constant Γ \Gamma Γ -coefficient of h 0 h^0 h 0 has the form
F 3 A 3 + F 2 A 2 + F L + F N 0 q + k q = 0 , A 2 , A 3 , L , N 0 ∈ Z , 0 ≤ k ≤ p . F^3A_3+F^2A_2+FL+\frac{FN_0}{q}+\frac{k}{q}=0,\qquad A_2,A_3,L,N_0\in\mathbb{Z},\qquad0\le k\le p. F 3 A 3 + F 2 A 2 + F L + q F N 0 + q k = 0 , A 2 , A 3 , L , N 0 ∈ Z , 0 ≤ k ≤ p .
A term with three finite factors belongs to F 3 A 3 F^3A_3 F 3 A 3 , and one with two finite factors belongs to F 2 A 2 F^2A_2 F 2 A 2 . The integer L L L is the normalized sum of direct query leading values at the final prefix vertex and its finite bias.
Only one three-factor path shape passes through all three hidden vertices: an origin at the first hidden vertex, followed by the edge from the first to the second hidden vertex and the edge from the second to the third. We call the paths of this shape the three-factor chain. If a path of that chain has a balanced Γ , s 0 \Gamma,s_0 Γ , s 0 pair and one finite factor, the contribution of such paths to the display is F N 0 / q FN_0/q F N 0 / q , which defines the integer N 0 N_0 N 0 ; otherwise N 0 = 0 N_0=0 N 0 = 0 . The finite factor of such a path lies in one of two places. Either it is the origin, and these origins together give the finite-first sum, the finite direct query inputs and bias of the first hidden vertex, times the two nonfinite edges. Or it is one of the two internal edges, and that finite edge is multiplied by a nonnegative origin count: the number of origins of the chain that carry the Γ \Gamma Γ or s 0 s_0 s 0 value completing a nonbalanced pair. Input-origin nonfinite terms are absent on the clean block: every input edge from a coordinate of the block carries a query value, and every other input is fixed to zero. The only possible nonfinite origin of the chain is therefore the bias of the first hidden vertex, so this count is zero or one, and in particular at most p p p . All remaining balanced two-factor paths give k / q k/q k / q .
The direct and finite-first sums have absolute value at most ( 3 r + 1 ) Q (3r+1)Q ( 3 r + 1 ) Q , and an internal normalized product has absolute value at most p Q pQ pQ . Since q < r < Q q<r<Q q < r < Q and the calibrated construction takes F > 8 Q 4 F>8Q^4 F > 8 Q 4 , it follows that ∣ q L + N 0 ∣ < F |qL+N_0|<F ∣ q L + N 0 ∣ < F . Clearing a q-reduced modulo F F F first gives k = 0 k=0 k = 0 . Dividing by F F F and reducing modulo F F F then gives q L + N 0 = 0 qL+N_0=0 q L + N 0 = 0 ; the higher powers do not affect this reduction. Thus N 0 ≡ 0 ( m o d q ) N_0\equiv0\pmod q N 0 ≡ 0 ( mod q ) .
The Γ 0 w 1 \Gamma^0w^1 Γ 0 w 1 -coefficient of h h h has the following form for an integer Z 1 Z_1 Z 1 :
F Z 1 2 q + δ + E q . \frac{FZ_1}{2q}+\delta+\frac{E}{q}. 2 q F Z 1 + δ + q E .
The term δ = 2 a + δ b i a s \delta=2a+\delta_{\mathrm{bias}} δ = 2 a + δ bias is the correction of its direct selected-query sum and finite bias; here a ∈ { 0 , … , 3 r } a\in\{0,\ldots,3r\} a ∈ { 0 , … , 3 r } counts the direct query inputs equal to one at the tested point. The half-integer E E E is q q q times the correction of the same possible balanced three-factor chain. Every other differentiated term retains a finite factor and belongs to F Z 1 / ( 2 q ) F\mathbb{Z}_1/(2q) F Z 1 / ( 2 q ) . If E < 0 E<0 E < 0 , that chain’s finite factor must be the anchor baseline B = F ( Q − T ) − w B=F(Q-T)-w B = F ( Q − T ) − w or the control baseline C = P − 3 2 w C=P-\frac{3}{2}w C = P − 2 3 w of the calibrated construction, the only primitives with a negative correction. For a finite-first sum, one selected query already makes its correction positive, contrary to E < 0 E<0 E < 0 . With none selected, its normalized leading value is Q − T Q-T Q − T or Q Q Q , which is nonzero modulo q q q and contradicts N 0 ≡ 0 ( m o d q ) N_0\equiv0\pmod q N 0 ≡ 0 ( mod q ) . For a finite internal factor, its leading value is nonzero modulo q q q and is multiplied by a positive count less than q q q , which again contradicts N 0 ≡ 0 ( m o d q ) N_0\equiv0\pmod q N 0 ≡ 0 ( mod q ) . Hence E ≥ 0 E\geq0 E ≥ 0 .
The output-one equation consequently gives, for an integer Z 2 Z_2 Z 2 ,
F Z 2 2 q 2 + r 0 − δ − E q = 1. \frac{FZ_2}{2q^2}+r_0-\delta-\frac{E}{q}=1. 2 q 2 F Z 2 + r 0 − δ − q E = 1.
After clearing 2 q 2 2q^2 2 q 2 , the part not visibly divisible by F F F is
2 q k 1 + 2 k 2 − 2 q 2 δ − 2 q E − 2 q 2 , 2qk_1+2k_2-2q^2\delta-2qE-2q^2, 2 q k 1 + 2 k 2 − 2 q 2 δ − 2 q E − 2 q 2 ,
an integer, since δ \delta δ and E E E are half-integers. Here ∣ δ ∣ ≤ 6 r + 2 |\delta|\leq6r+2 ∣ δ ∣ ≤ 6 r + 2 , because a ≤ 3 r a\leq3r a ≤ 3 r and a finite bias has correction of absolute value at most two. Also ∣ E ∣ ≤ 6 r + 2 |E|\leq6r+2 ∣ E ∣ ≤ 6 r + 2 . When the finite factor of the chain is its origin, E E E is the correction of the finite-first sum, which is twice its number of selected query inputs, at most 3 r 3r 3 r , plus the correction of the first finite bias, of absolute value at most two. When the finite factor is an internal edge, E E E is the correction of that edge, of absolute value at most two, times the origin count, which is at most one. These two shapes cannot occur together, since in the first both internal edges of the chain are nonfinite and in the second exactly one is. With 0 ≤ k 1 , k 2 ≤ p = q − 1 0\leq k_1,k_2\leq p=q-1 0 ≤ k 1 , k 2 ≤ p = q − 1 , so that
2 q k 1 + 2 k 2 ≤ 2 q 2 − 2 , 2qk_1+2k_2\leq2q^2-2, 2 q k 1 + 2 k 2 ≤ 2 q 2 − 2 ,
and with q , r < Q q,r<Q q , r < Q , the displayed integer has absolute value at most
2 q 2 ( 6 r + 4 ) + 2 q ( 6 r + 2 ) < 12 Q 3 + 20 Q 2 + 4 Q < 50 Q 3 < 8 Q 4 < F , 2q^2(6r+4)+2q(6r+2)<12Q^3+20Q^2+4Q<50Q^3<8Q^4<F, 2 q 2 ( 6 r + 4 ) + 2 q ( 6 r + 2 ) < 12 Q 3 + 20 Q 2 + 4 Q < 50 Q 3 < 8 Q 4 < F ,
where 50 Q 3 < 8 Q 4 50Q^3<8Q^4 50 Q 3 < 8 Q 4 holds because Q = 2 r − 1 ≥ 127 Q=2^r-1\geq127 Q = 2 r − 1 ≥ 127 . Reduction modulo F F F therefore gives literal equality of this small part.
We name these quantities for later use. The period is F / ( 2 q 2 ) F/(2q^2) F / ( 2 q 2 ) : before 2 q 2 2q^2 2 q 2 is cleared, every term containing a factor F F F is an integer multiple of it. The small residue of a constant coefficient is the sum of its terms without a factor F F F . Here the small residue is r 0 − δ − E / q r_0-\delta-E/q r 0 − δ − E / q , and it is smaller in absolute value than the period. The gain residue and the gain-free residue are the parts of the small residue contributed by gain paths and by gain-free paths, respectively. Here they are − δ − E / q -\delta-E/q − δ − E / q and r 0 r_0 r 0 . Since r 0 < 1 r_0<1 r 0 < 1 and E ≥ 0 E\geq0 E ≥ 0 , the literal equality r 0 − δ − E / q = 1 r_0-\delta-E/q=1 r 0 − δ − E / q = 1 requires δ < 0 \delta<0 δ < 0 . Only a positive B B B or C C C final bias has a negative correction, so L ≥ Q − T L\geq Q-T L ≥ Q − T . Also N 0 ≥ − p Q N_0\geq-pQ N 0 ≥ − pQ , including every signed internal finite choice. It follows that
0 = q L + N 0 ≥ q ( Q − T ) − p Q = Q − q T > 0. 0=qL+N_0\geq q(Q-T)-pQ=Q-qT>0. 0 = q L + N 0 ≥ q ( Q − T ) − pQ = Q − q T > 0.
The final strict inequality follows from Q > q 2 Q>q^2 Q > q 2 and Q > 8 T 2 Q>8T^2 Q > 8 T 2 . This is a contradiction.
An edge entering the last hidden vertex.
Let z z z be that last vertex’s nonzero output coefficient and u u u the source of the gain edge. The inverse coefficient forces u 0 = 0 u^0=0 u 0 = 0 . If z z z is finite, the constant gain coefficient is an F / ( 2 q ) F/(2q) F / ( 2 q ) multiple, which changes only the divisible part of the preceding residue calculation. Indeed a two-hidden nongain prefix has half-integer Γ w 1 0 \Gamma^0_{w_1} Γ w 1 0 -coefficient; a balanced pair without a finite factor has no w w w -correction. The correction of z z z multiplies u 0 = 0 u^0=0 u 0 = 0 . Thus the small residue remains r 0 < 1 r_0<1 r 0 < 1 , contradicting output one.
For a nonfinite z z z , write the two-hidden prefix as v = ρ ( V + t h 1 ) v=\rho(V+th_1) v = ρ ( V + t h 1 ) , allowing a missing first vertex or edge. Here h 1 h_1 h 1 is the value of the first hidden vertex, t t t is the coefficient of its edge into u u u , and V V V collects the direct inputs and bias of u u u . The constant gain coefficient then needs the w w w -coefficient of u u u at the Γ \Gamma Γ -degree opposite to that of z z z , and this coefficient can arise in only two ways: from the correction of a finite t t t , or from the derivative of h 1 h_1 h 1 carried by a matching nonfinite t t t . If t t t is finite, its correction multiplies the count of opposite-degree origins entering h 1 h_1 h 1 . The corresponding coefficient of u 0 u^0 u 0 has equation Γ n t k + k ′ = 0 \Gamma n_t k+k'=0 Γ n t k + k ′ = 0 , where k k k and k ′ k' k ′ are the numbers of opposite-degree origins entering h 1 h_1 h 1 and V V V , respectively, k ′ ≤ p k'\leq p k ′ ≤ p , and n t ≠ 0 n_t\ne0 n t = 0 . As F > p F>p F > p , this forces both counts to be zero, and the derivative vanishes.
In the second way, t t t is the matching Γ \Gamma Γ or s 0 s_0 s 0 primitive, and the coefficient of t t t in u 0 u^0 u 0 satisfies F L 1 + k ′ = 0 FL_1+k'=0 F L 1 + k ′ = 0 , where F L 1 FL_1 F L 1 is the Γ 0 \Gamma^0 Γ 0 -coefficient of h 1 0 h_1^0 h 1 0 . Again L 1 = k ′ = 0 L_1=k'=0 L 1 = k ′ = 0 .
In this second way, the affine preactivation of h 1 h_1 h 1 is a sum of positive query inputs and a bias with zero finite leading value. Its finite bias is therefore absent or negative, since a positive finite bias could not cancel those inputs. Its derivative d = 2 a 1 + δ b i a s d = 2a_1 + \delta_{\mathrm{bias}} d = 2 a 1 + δ bias is nonnegative, where a 1 a_1 a 1 counts its query inputs equal to one and δ b i a s \delta_{\mathrm{bias}} δ bias is the correction of its finite bias. The gain then contributes a small constant residue − d / q -d/q − d / q , or zero if the first vertex is inactive. These are all possibilities, because a two-factor path has no other opposite-degree source. The full small residue is therefore at most r 0 < 1 r_0 < 1 r 0 < 1 , which again cannot be one after clearing 2 q 2 2q^2 2 q 2 and reducing modulo F F F .
The first-to-middle edge.
All gain paths have combined form − D h 1 Θ -Dh_1\Theta − D h 1 Θ , with Θ = b + η c s \Theta= b + \eta cs Θ = b + η cs , where η ∈ { 0 , 1 } \eta\in\{0,1\} η ∈ { 0 , 1 } is the actual last-vertex indicator. Here b b b and c c c are the output coefficients of the middle and last hidden vertices, s s s is the coefficient of the edge from the middle to the last hidden vertex, and absent edges are zero. If a gain path is present, Θ 0 ≠ 0 \Theta^0 \ne0 Θ 0 = 0 by Lemma A.1. The inverse coefficient thus forces h 1 0 = 0 h_1^0 = 0 h 1 0 = 0 . The derivative d d d of its affine preactivation is nonnegative by the preceding clean-input argument. The constant Γ \Gamma Γ -degree of Θ 0 \Theta^0 Θ 0 is an F F F -multiple plus either zero or 1 / q 1/q 1/ q ; only a positive balanced pair supplies the latter. Hence the small gain residue is zero or − d / q -d/q − d / q . It cannot supplement r 0 < 1 r_0 < 1 r 0 < 1 to give one.
Biases and external-input edges.
A negative gain on an external-input edge is fixed to zero on the clean block and leaves no gain contribution. A middle hidden bias has inverse coefficient − Θ 0 -\Theta^0 − Θ 0 , with nonzero short transfer whenever a gain path is present. For a bias at the last hidden vertex, the transfer from that vertex to the output is its nonzero output-edge coefficient. A first hidden bias − D -D − D makes that gate inactive throughout the clean cube: its finite query inputs sum to at most 3 r P + 6 r w < M 3r_P+6r_w < M 3 r P + 6 r w < M , and no nonfinite input edge is active there. An output bias − D -D − D has inverse coefficient minus one with no remaining gain path to cancel it. These cases exhaust every possible position. □
The next obstruction has the opposite sign but applies to the whole query function. Positive output coefficients permit the origin and unit tests that its proof needs.
Lemma A.8 (A sole positive internal gain). Suppose − D -D − D is unused in a legal candidate representing the entire clean query target. Then + D +D + D cannot be a hidden-to-hidden edge or a hidden bias.
Proof. By Lemma A.3 , both finite negatives enter distinct hidden preactivations. Thus all output parameters are positive. The output bias and skips are absent, and directly output-connected hidden values vanish at the origin and unit vectors. At every winner the positive gain must contribute to the constant coefficient, since otherwise Lemma A.2 applies.
An edge into the last hidden vertex.
Let z > 0 z > 0 z > 0 be the output coefficient of the last hidden vertex, and u = ρ ( V + t ρ ( U ) ) u = \rho(V+t\rho(U)) u = ρ ( V + tρ ( U )) the at-most-two-hidden prefix, with u u u the source of the gain edge and V , t V,t V , t as in the case “An edge entering the last hidden vertex” of the proof of Lemma A.7 , and U U U the affine preactivation of the first hidden vertex, so that ρ ( U ) \rho(U) ρ ( U ) is the value h 1 h_1 h 1 there. At every winner this last vertex is active and u 0 = 0 u^0 = 0 u 0 = 0 . If z z z is finite, the Γ 0 w 1 − 1 \Gamma^0w_1^{-1} Γ 0 w 1 − 1 -coefficient of u u u is a half-integer, so the gain contribution is an F / 2 F/2 F /2 multiple. It cannot change the small residue in Lemma A.2 to one.
Let z = Γ z = \Gamma z = Γ . A nonzero constant gain coefficient requires a Γ − 1 w 1 \Gamma^{-1}w^1 Γ − 1 w 1 term in u u u , so the first vertex is active at every winner; otherwise the remaining direct affine terms supply no such derivative. For finite t t t , the Γ − 1 w 0 \Gamma^{-1}w^0 Γ − 1 w 0 equation has the form F n t k + k ′ = 0 F n_t k + k' = 0 F n t k + k ′ = 0 , with origin counts bounded by p < F p < F p < F . Both counts vanish and the required derivative is zero.
For t = s 0 t = s_0 t = s 0 , the equation is F L U + k ′ = 0 F L_U + k' = 0 F L U + k ′ = 0 , where L U L_U L U is the finite first affine leading sum. It forces L U = 0 L_U = 0 L U = 0 at every winner and k ′ = 0 k' = 0 k ′ = 0 . The all-ones query vertex and its single deletions win and affinely span the query cube. Since the first vertex is active there, its affine form L U L_U L U vanishes identically. All its finite input coefficients and finite bias are absent, and so is their w w w -correction.
For t = Γ t = \Gamma t = Γ or a missing t t t , no opposite-degree derivative exists. Thus no type gives the required constant coefficient. For z = s 0 z = s_0 z = s 0 , interchange Γ \Gamma Γ and s 0 s_0 s 0 in this argument; the origin count is then at most two, still below F F F .
The first-to-middle edge and later biases.
For the first-to-middle edge, the gain contribution is D p ( U ) ( b + η c s ) Dp(U)(b+\eta cs) D p ( U ) ( b + η cs ) , with positive output coefficients b , c b,c b , c when present and an arbitrary nongain internal s s s . The transfer’s zeroth coefficient is nonzero
whenever any gain path is present, by Lemma A.1. The first source is active at every winner and its affine U 0 U^0 U 0 vanishes there by the inverse equation. Winning affine span makes U 0 U^0 U 0 identically zero, removing every query input and its bias, hence the entire source. This contradicts its required activity. A middle gain bias has that same nonzero short transfer and an uncancelled inverse coefficient. A last gain bias has its nonzero output coefficient and is excluded in the same way.
A first hidden bias.
The first gate is h 1 = D + I 1 h_1=D+I_1 h 1 = D + I 1 , with I 1 I_1 I 1 a positive query sum. It is active throughout the clean cube and has no direct output edge, because h 1 ( 0 ) > 0 h_1(0)>0 h 1 ( 0 ) > 0 . The two remaining vertices have the form
h 2 = ρ ( V + t h 1 ) , h 3 = ρ ( W + u h 1 + s h 2 ) , N = b h 2 + c h 3 . h_2=\rho(V+th_1), \qquad h_3=\rho(W+uh_1+sh_2), \qquad N=bh_2+ch_3. h 2 = ρ ( V + t h 1 ) , h 3 = ρ ( W + u h 1 + s h 2 ) , N = b h 2 + c h 3 .
All output coefficients are positive when present. Its gain transfer under a pattern is
Z = η 2 b t + η 3 c u + η 2 η 3 c s t . Z=\eta_2bt+\eta_3cu+\eta_2\eta_3cst. Z = η 2 b t + η 3 c u + η 2 η 3 cs t .
Since + D +D + D enters only as the bias of h 1 h_1 h 1 and − D -D − D is unused, the inverse coefficient is Z 0 Z^0 Z 0 . Suppose that only one output branch is present, or that only one of h 2 , h 3 h_2,h_3 h 2 , h 3 is active. Then Z 0 Z^0 Z 0 is one of b 0 t 0 b^0t^0 b 0 t 0 , c 0 u 0 c^0u^0 c 0 u 0 and c 0 s 0 t 0 c^0s^0t^0 c 0 s 0 t 0 , except when only the c c c branch is present and both h 2 , h 3 h_2,h_3 h 2 , h 3 are active, where Z 0 = c 0 ( u 0 + s 0 t 0 ) Z^0=c^0(u^0+s^0t^0) Z 0 = c 0 ( u 0 + s 0 t 0 ) . A single present term is a product of nonzero leading primitives, each a nonzero scalar times a power of Γ \Gamma Γ , and is therefore nonzero. In the exceptional case, with u u u and s t st s t both present, u 0 + s 0 t 0 = 0 u^0+s^0t^0=0 u 0 + s 0 t 0 = 0 would make the nonzero product s 0 t 0 s^0t^0 s 0 t 0 equal to the negative of one primitive, contrary to Lemma A.1; with only one of them present, a single nonzero term remains. If all gain paths vanish, the coefficient obstruction applies. Thus b , c b,c b , c are present and both h 2 , h 3 h_2,h_3 h 2 , h 3 are active at every winner, with
b 0 t 0 + c 0 u 0 + c 0 s 0 t 0 = 0. b^0t^0+c^0u^0+c^0s^0t^0=0. b 0 t 0 + c 0 u 0 + c 0 s 0 t 0 = 0.
A negative finite t t t makes h 2 h_2 h 2 inactive unless V V V has a Γ \Gamma Γ bias; that bias instead makes it positive at the origin, contradicting its positive output edge. There are no active Γ \Gamma Γ input edges on the clean block. A positive finite or Γ \Gamma Γ value of t t t also makes h 2 ( 0 ) > 0 h_2(0)>0 h 2 ( 0 ) > 0 , even against a finite negative bias. A missing t t t leaves a single nonzero inverse transfer, or no gain path. Therefore t = s 0 t=s_0 t = s 0 . Since s 0 D > 0 s_0D>0 s 0 D > 0 , the origin test forces one finite negative to be the bias of h 2 h_2 h 2 . The other must be an edge u u u or s s s ; otherwise every term of the displayed transfer has positive sign.
If s = − P a s=-P_a s = − P a , where P a ∈ { P − w , P } P_a\in\{P-w,P\} P a ∈ { P − w , P } , then u u u and the bias of W W W are nonnegative. The origin test forces both absent. The transfer would require b 0 = P c 0 b^0=P c^0 b 0 = P c 0 , contrary to Lemma A.1.
If u = − P a u=-P_a u = − P a , divide the leading transfer by c 0 s 0 c^0s_0 c 0 s 0 to get
b 0 c 0 + s 0 = P q Γ . \frac{b^0}{c^0}+s^0=Pq\Gamma. c 0 b 0 + s 0 = P q Γ.
Every term on the left is positive. The ratio can have the required Γ \Gamma Γ -exponent only as q F n Γ qFn\Gamma q F n Γ , from finite b b b and selector c c c , or Γ / ( F n ) \Gamma/(Fn) Γ/ ( F n ) , from b = Γ b=\Gamma b = Γ and finite c c c . The optional s s s must be absent or Γ \Gamma Γ . In the first ratio case, a present s s s would give q F n + 1 = P q qFn+1=Pq q F n + 1 = P q , impossible modulo q q q . With s s s absent, b b b must be the control primitive C C C and c = s 0 c=s_0 c = s 0 . The full gain transfer is then s 0 ( C − P a ) s_0(C-P_a) s 0 ( C − P a ) , whose contribution after multiplication by D D D has no constant Γ \Gamma Γ -degree. The gain-free coefficient obstruction still excludes output one. In the second ratio case, either 1 / ( F n ) 1/(Fn) 1/ ( F n ) or 1 + 1 / ( F n ) 1+1/(Fn) 1 + 1/ ( F n ) would have to equal P q Pq P q , which is impossible in magnitude. This exhausts the first-bias case and all internal gain positions. □
The internal-gain analysis leaves the case of a positive gain on an output edge. Its exclusion will force the negative gain to contribute at every clean winner.
Lemma A.9 (A positive output gain without a negative-gain path). At a clean query winner, a candidate cannot have + D +D + D on a hidden-to-output edge if every individual active path contribution containing − D -D − D is zero. Other output coefficients may be signed finite primitives.
Proof. Let h h h be the source of the positive output gain. Its inverse-w w w coefficient forces h 0 = 0 h^0=0 h 0 = 0 as an entire Laurent polynomial in Γ \Gamma Γ . The other nonzero paths avoid both gains, so their constant coefficient has the form in Lemma A.2. For a prefix containing three hidden vertices, expand the constant-degree prefix equation as
F 3 A 3 + F 2 A 2 + F L + F N 0 q + k q = 0. (8) F^3A_3+F^2A_2+FL+\frac{FN_0}{q}+\frac{k}{q}=0. \tag*{(8)} F 3 A 3 + F 2 A 2 + F L + q F N 0 + q k = 0. (8)
The definitions and bounds of L , N 0 , k L,N_0,k L , N 0 , k are those in the output-edge case of Lemma A.7 . They follow from the unique possible three-factor chain and the direct input/bias terms, irrespective of the signs of the other output coefficients. The same successive reductions give k = 0 k=0 k = 0 , q L + N 0 = 0 qL+N_0=0 q L + N 0 = 0 , and N 0 ≡ 0 ( m o d q ) N_0\equiv0\pmod{q} N 0 ≡ 0 ( mod q ) . The Γ 0 w 1 \Gamma^{0}w^{1} Γ 0 w 1 -coefficient has form F Z 2 q 2 + δ + E q \frac{FZ}{2q^{2}}+\delta+\frac{E}{q} 2 q 2 F Z + δ + q E . After combining it with all other output paths, clearing 2 q 2 2q^{2} 2 q 2 and reducing modulo F F F , the small-residue equation is
δ + E q + r 0 = 1 , 0 ≤ r 0 < 1. \delta+\frac{E}{q}+r_0=1,\qquad0\le r_0<1. δ + q E + r 0 = 1 , 0 ≤ r 0 < 1.
Thus 0 < δ + E / q ≤ 1 0<\delta+E/q\le1 0 < δ + E / q ≤ 1 . This argument uses no positivity assumption on the other output terms.
If the balanced three-factor chain has an internal finite factor, N 0 N_0 N 0 is a nonzero-modulo-q q q normalized finite value times a positive origin count less than q q q . It cannot be zero modulo q q q , so such a chain is absent. The remaining possible chain has its finite factor first. Its first vertex has only query inputs and one bias. When the chain is active, the normalized finite leading sum of its first vertex is the integer N 0 N_0 N 0 of the display; otherwise N 0 = 0 N_0=0 N 0 = 0 . If that bias is finite and the chain is active, this sum is nonnegative, since an active vertex has nonnegative leading preactivation. If its bias is nonfinite, that finite sum consists only of positive query terms and is also nonnegative. As in Lemma A.7 , a negative derivative E E E would require a B B B or C C C bias with no selected queries, contradicting N 0 ≡ 0 ( m o d q ) N_0\equiv0\pmod{q} N 0 ≡ 0 ( mod q ) . Therefore N 0 ≥ 0 N_0\ge0 N 0 ≥ 0 , E ≥ 0 E\ge0 E ≥ 0 , and L = − N 0 / q ≤ 0 L=-N_0/q\le0 L = − N 0 / q ≤ 0 .
A positive finite final bias would make L > 0 L>0 L > 0 , impossible. With a negative finite final bias and any selected direct query, δ ≥ 2 \delta\ge2 δ ≥ 2 , contrary to the small-residue interval. With that bias and no direct query, L = − Q L=-Q L = − Q , hence N 0 = q Q N_0=qQ N 0 = q Q . The normalized finite first bias is at most Q Q Q , including zero when absent or nonfinite. Its selected query sum is therefore q Q qQ q Q minus that bias, and is at least ( q − 1 ) Q (q-1)Q ( q − 1 ) Q . Every query weight is at most ( Q + 1 ) / 2 (Q+1)/2 ( Q + 1 ) /2 , requiring at least 2 q − 2 2q-2 2 q − 2 selected inputs. Its derivative is at least 2 ( 2 q − 2 ) − 3 / 2 > q 2(2q-2)-3/2>q 2 ( 2 q − 2 ) − 3/2 > q , which again makes δ + E / q > 1 \delta+E/q>1 δ + E / q > 1 .
If the final bias is absent or nonfinite, L ≥ 0 L\ge0 L ≥ 0 ; thus L = N 0 = 0 L=N_0=0 L = N 0 = 0 and δ = 0 \delta=0 δ = 0 . A nonzero finite-first derivative then requires a negative finite first bias and a selected power-of-two sum equal to Q Q Q . The integer Q = 2 r − 1 Q=2^r-1 Q = 2 r − 1 has r r r binary ones, so such a sum uses at least r r r terms. Its derivative satisfies E ≥ 2 r > q E\ge2r>q E ≥ 2 r > q . Otherwise E = 0 E=0 E = 0 . Both possibilities contradict 0 < δ + E / q ≤ 1 0<\delta+E/q\le1 0 < δ + E / q ≤ 1 .
If the prefix ending at h h h contains at most two hidden vertices, there is no finite-balanced three-factor chain, so N 0 = E = 0 N_0=E=0 N 0 = E = 0 . The same reduction gives L = 0 L=0 L = 0 and 0 < δ ≤ 1 0<\delta\le1 0 < δ ≤ 1 . An absent or nonfinite final bias gives δ = 0 \delta=0 δ = 0 , and a positive finite one gives L > 0 L>0 L > 0 . A negative finite bias requires selected query sum Q Q Q , hence δ ≥ 2 r \delta\ge2r δ ≥ 2 r . This excludes the shorter prefixes too.
The pointwise lemmas also show why input-gain positions can be eliminated without assuming that deleting a path preserves a ReLU’s activation pattern. Only one case needs an actual graph pruning operation.
Lemma A.10 (Input gains and legal pruning). A candidate representing f f f cannot put either gain on an external-input edge, and cannot leave one gain unused.
Proof. Choose the block in Lemma 4.7 . Every gain input edge is fixed to zero there. If + D +D + D is unused or an input edge, Lemma A.7 excludes a winning output; when both gains are absent the coefficient obstruction also applies directly. If − D -D − D is unused or an input edge and + D +D + D is an output edge, Lemma A.9 applies. Output gain biases are excluded by Lemma A.6 .
It remains that + D +D + D is internal and − D -D − D is unavailable on the restriction. Delete all external-input edges fixed to zero. Repeatedly remove a hidden vertex with no incoming edge or present bias, together with its outgoing edges: it is identically zero, so each such deletion preserves the restricted function. Remove vertices without any path to the output. All these operations discard occurrences, create no parallel edges and increase no node count. The final output cannot be parameterless, because the query target is not identically zero. The remaining network is legal, represents the whole query target and has − D -D − D unused. If + D +D + D remains, its internal role is unchanged and Lemma A.8 applies. If it has been discarded too, Lemma A.2 applies. This pruning is justified by identically zero contributions, rather than by preserving an activation pattern after an arbitrary deletion.
:::
Both Gains on Internal Edges
After the preceding exclusions, every used gain has a hidden source or is a hidden bias. For two hidden-to-hidden gain edges, the three topological pairs below exhaust the possible shapes. Ordinary output coefficients may still be signed finite values.
Lemma A.11 (Two internal gain edges). No candidate for the clean query target places both gains on hidden-to-hidden edges.
Proof. Number the hidden vertices 1 , 2 , 3 1,2,3 1 , 2 , 3 , allowing missing vertices and edges. Here i j ij ij denotes the edge from hidden vertex i i i to hidden vertex j j j . The gain-edge pairs are serial ( 12 , 23 ) (12,23) ( 12 , 23 ) , fanout ( 12 , 13 ) (12,13) ( 12 , 13 ) , or fanin ( 13 , 23 ) (13,23) ( 13 , 23 ) . In the case names below, in headings and in the text, bias one, two or three (or the last bias) is a gain bias at that hidden vertex, output one, two or three is the hidden-to-output edge from that vertex carrying a gain, a sign word such as positive gives that gain’s sign, and a slash, a comma or the word and joins the positions of the two gains. For instance, edge 12/bias one places one gain on the edge from vertex one to vertex two and the other at the bias of vertex one. Either sign order is permitted. We apply the coefficient obstruction only to the actual frozen expression, or after full gain cancellation.
Serial edges.
If vertex one is inactive at a winner, only the last gain relay remains, with a first affine query source. Its inverse equation forces that source to be w K wK w K for an integer K K K . The constant gain contribution is an F / 2 F/2 F /2 multiple for a finite output coefficient, or has nonzero Γ \Gamma Γ -degree for a nonfinite one. The gain-free small residue cannot then become one. The same argument applies if vertex three is inactive and only the first gain relay remains. If vertex two is inactive, both gain paths disappear. Thus all three are active at every winner.
The unique inverse-square path forces the first affine source’s zeroth coefficient to vanish. Its active value is therefore w K wK w K , where K K K is a positive integer at most 6 r + 1 < F 6r+1 < F 6 r + 1 < F . A nonfinite bias could not occur in a zero zeroth coefficient, and a positive finite bias could not cancel the selected query inputs. The derivative consequently comes only from query corrections and a negative finite bias, giving the stated integer bound. At vertex two the gain contributes + K +K + K or − K -K − K to its zeroth coefficient. All its other scalar terms are multiples of F F F , and any nonfinite bias has a distinct Γ \Gamma Γ -degree. Its zeroth coefficient is therefore nonzero. The last gain edge creates an unmatched inverse-w w w term at vertex three and its nonzero output edge. No earlier output can cancel it, since the first gain has already been multiplied by w K wK w K . This excludes both serial sign orders.
Fanout edges.
Write c 2 , c 3 c_2,c_3 c 2 , c 3 for the output coefficients and e e e for the possible edge 2 → 3 2 \to3 2 → 3 . Up to overall sign, the gain contribution is
D h 1 ( η 2 c 2 − η 3 c 3 + η 2 η 3 c 3 e ) . D h_1\left(\eta_2 c_2-\eta_3 c_3+\eta_2\eta_3 c_3 e\right). D h 1 ( η 2 c 2 − η 3 c 3 + η 2 η 3 c 3 e ) .
Each winner requires h 1 > 0 h_1>0 h 1 > 0 and a nonzero gain contribution. If e e e is present and both branches active, the leading bracket cannot vanish by Lemma A.1. If a branch is missing or inactive, the same comparison excludes cancellation between its one primitive and one product; a single surviving term is nonzero. With e e e absent, nontrivial leading cancellation requires both branches active and c 2 0 = c 3 0 c_2^0=c_3^0 c 2 0 = c 3 0 . Two negative output occurrences are forbidden by Lemma A.3. The equal output primitives are therefore positive and identical in full. The entire gain contribution then cancels, and Lemma A.2 excludes one.
In every remaining winning pattern, the inverse equation forces h 1 0 = 0 h_1^0=0 h 1 0 = 0 . The first vertex is active at all those winners, so this is an affine identity on the all-ones vertex and its single deletions. They affinely span the query cube. Every query coefficient and its bias leading term must vanish, removing all corresponding full primitives and their corrections. The source is identically zero, contrary to its required activity.
Fanin edges.
The last vertex must be active at every winner. A finite last output coefficient again makes the constant gain contribution an F / ( 2 q ) F/(2q) F / ( 2 q ) multiple, and the small gain-free residue cannot give one. Let its output coefficient be Γ \Gamma Γ or s 0 s_0 s 0 . If one gain prefix is inactive, the other is a first affine query prefix with zero zeroth coefficient. Its w w w -derivative has no opposite Γ \Gamma Γ -degree, so the constant gain coefficient is zero. The coefficient obstruction excludes that pattern. Both prefixes are therefore active at every winner.
Write their values as h 1 = ρ ( U ) h_1=\rho(U) h 1 = ρ ( U ) , h 2 = ρ ( V + t h 1 ) h_2=\rho(V+t h_1) h 2 = ρ ( V + t h 1 ) . The inverse equation gives
V 0 = ( 1 − t 0 ) U 0 (9) V^0=(1-t^0)U^0 \tag*{(9)} V 0 = ( 1 − t 0 ) U 0 (9)
on the winning stencil. Since the points of the winning stencil affinely span the query cube, this is an affine identity on the entire query cube. If U U U has no query coefficient, this identity removes every query coefficient of V V V too. Both prefix vertices are then constant functions, leaving only the last effective hinge, which contradicts Lemma 4.7. Otherwise compare a positive query coefficient of U 0 U^0 U 0 with the corresponding nonnegative coefficient of V 0 V^0 V 0 . A positive finite t 0 > 1 t^0>1 t 0 > 1 gives a negative right side. A Γ \Gamma Γ or s 0 s_0 s 0 value gives an unmatched negative nonzero Γ \Gamma Γ -degree. A negative finite value − P -P − P gives a multiplier 1 + P 1+P 1 + P , larger than every available ratio of direct query coefficients. Thus t t t is absent.
The direct leading query coefficients then agree in full. The prefix biases also have equal leading values; their only possible unequal full values are the two finite negatives, differing by w w w . Consequently h 1 − h 2 h_1-h_2 h 1 − h 2 on the winning patterns is zero or ± w \pm w ± w , with no Γ \Gamma Γ -dependence. Multiplying by the nonfinite output coefficient gives no constant Γ \Gamma Γ -degree gain contribution. The gain-free coefficient obstruction supplies the final contradiction. □
Internal Gain Biases
The remaining all-internal placements contain at least one gain bias. Two elementary consequences of the preceding coefficient comparisons will be used in their case analysis.
A first affine query prefix cannot have zeroth coefficient one. An at-most-two-hidden nongain prefix cannot have it either: its constant-degree coefficient is F Z + I / q FZ+I/q F Z + I / q , with I ∈ { 0 , 1 } I\in\{0,1\} I ∈ { 0 , 1 } , because only its single internal edge can complete a balanced nonfinite pair from a bias origin. The equation F Z + I / q = 1 FZ+I/q=1 F Z + I / q = 1 is impossible modulo F F F after clearing q q q .
We will also use the following single-relay consequence at a winner. If the only surviving gain is an edge from a first affine query prefix to a final active hidden vertex, its inverse equation forces that source to be w K wK w K , with integer K K K . A finite output coefficient makes its constant contribution an F / 2 F/2 F /2 multiple; a nonfinite output coefficient gives nonzero Γ \Gamma Γ -degree. Other paths have the residue of Lemma A.2, so they cannot complete the output to one. A relay whose source or target vertex is inactive has no gain contribution and is excluded directly. These observations concern actual patterns, not a replacement by a numerically similar graph.
Lemma A.12 (Gain biases and mixed internal placements). No candidate for the clean query target places both gains internally with at least one as a hidden bias.
Proof. Let c i c_i c i be the ordinary output coefficient from hidden vertex i i i , and e i j e_{ij} e ij an ordinary internal edge when it is not itself a gain. Absent coefficients are zero. The finite signs of these coefficients remain unrestricted except that at most one output occurrence is negative, by Lemma A.3. The short-product and short-transfer lemmas apply with these signs.
Biases at vertices one and two.
If the first bias is − D -D − D , its gate is identically zero on the clean cube. The second gate with bias + D +D + D is then affine and active throughout that cube, leaving only the third effective hinge. If the first bias is + D +D + D , write h 1 = D + I 1 h_1=D+I_1 h 1 = D + I 1 . The second preactivation has the form
( e 12 − 1 ) D + e 12 I 1 + I 2 . (e_{12}-1)D+e_{12}I_1+I_2. ( e 12 − 1 ) D + e 12 I 1 + I 2 .
For a positive finite or Γ \Gamma Γ value of e 12 e_{12} e 12 , it is positive throughout the cube. For a negative finite value, a selector, or a missing edge, it is negative throughout the cube. Here finite query sums are bounded by 3 r P + 6 r w 3rP+6rw 3 r P + 6 r w , Γ D \Gamma D Γ D is larger than that scale, and s 0 < 1 s_0<1 s 0 < 1 . Thus the first two vertices are affine or zero, and only the third can remain nonlinear. Both sign orders contradict Lemma 4.7.
Biases at vertices two and three.
Up to sign, the inverse gain transfer is
η 2 c 2 + η 3 c 3 ( η 2 e 23 − 1 ) . \eta_2c_2+\eta_3c_3(\eta_2e_{23}-1). η 2 c 2 + η 3 c 3 ( η 2 e 23 − 1 ) .
A single active term cannot cancel. When both are active and e 23 e_{23} e 23 is present, cancellation would make one product equal a sum or difference of two primitives, contrary to Lemma A.1. When e 23 e_{23} e 23 is absent, it requires c 2 0 = c 3 0 c_2^0=c_3^0 c 2 0 = c 3 0 . They cannot both be negative output occurrences, and positive equality identifies their full values. The gain-bias terms then cancel exactly. The coefficient obstruction excludes output one in this last possibility as well.
Biases at vertices one and three.
A first bias − D -D − D is inactive, so the only possible last gain bias has an uncancelled output transfer, or no gain path survives. Take a first bias + D +D + D , giving h 1 = D + I 1 h_1 = D+I_1 h 1 = D + I 1 . The middle gate is affine or zero except when its incoming edge from h 1 h_1 h 1 is absent or a selector and its own bias is finite negative. We verify this classification. A positive finite or Γ \Gamma Γ incoming edge makes the middle gate active everywhere, even against a finite negative bias. A negative finite incoming edge makes it inactive everywhere unless its bias is Γ \Gamma Γ ; that bias makes it active everywhere. An absent or selector edge leaves only a finite negative bias capable of producing a genuine hinge; every other bias gives an everywhere nonnegative preactivation or a constant zero. If the middle gate is affine or zero, only the last gate remains nonlinear, contradicting the query target.
In the exceptional case the middle gate is zero at the origin. If e 13 e_{13} e 13 is positive finite or Γ \Gamma Γ , the last gate, whose bias is − D -D − D , is active at the origin. The inverse equation is
c 1 0 + c 3 0 ( e 13 0 − 1 ) = 0 , c_1^0+c_3^0(e_{13}^0-1)=0, c 1 0 + c 3 0 ( e 13 0 − 1 ) = 0 ,
impossible by Lemma A.1.
For negative finite, selector or absent e 13 e_{13} e 13 , the last gate is zero at the origin, forcing c 1 c_1 c 1 absent. At each winner both middle and last gates must be active; otherwise there is one nonzero transfer, or no gain contribution. A missing e 12 e_{12} e 12 cannot cancel the last bias transfer, so e 12 = s 0 e_{12}=s_0 e 12 = s 0 . The inverse equation becomes
s 0 c 2 0 + c 3 0 ( e 13 0 − 1 + s 0 e 23 0 ) = 0. s_0c_2^0+c_3^0(e_{13}^0-1+s_0e_{23}^0)=0. s 0 c 2 0 + c 3 0 ( e 13 0 − 1 + s 0 e 23 0 ) = 0.
Put H = 1 + P H=1+P H = 1 + P when e 13 e_{13} e 13 is finite negative, and H = 1 H=1 H = 1 otherwise. Any selector e 13 e_{13} e 13 has different degree from the coefficient compared below. If c 3 0 = F n c_3^0=Fn c 3 0 = F n is finite, its constant-degree equation is
F n ( H − I / q ) = J / q , I , J ∈ { 0 , 1 } . Fn(H-I/q)=J/q,\qquad I,J\in\{0,1\}. F n ( H − I / q ) = J / q , I , J ∈ { 0 , 1 } .
The absolute value of the left side is at least F ( 1 − 1 / q ) > 1 F(1-1/q)>1 F ( 1 − 1/ q ) > 1 , while the right side is at most 1 / q 1/q 1/ q . If c 3 = Γ c_3=\Gamma c 3 = Γ , comparison at degree one gives H = I / q H=I/q H = I / q , again impossible. If c 3 = s 0 c_3=s_0 c 3 = s 0 , comparison at degree minus one gives H = F Z + I / q H=FZ+I/q H = F Z + I / q . Clearing q q q and reducing modulo F F F gives q = I q=I q = I , since P P P is divisible by F F F . This too is impossible. These comparisons exhaust the output types, including signed finite output coefficients.
A gain edge whose source vertex has the other gain as its bias.
Consider edge 12 and bias one. A negative first gain bias is inactive and removes every gain path. A positive first bias is active; an active gain-edge path then has a nonzero inverse-square transfer proportional to c 2 + η 3 c 3 e 23 c_2+\eta_3c_3e_{23} c 2 + η 3 c 3 e 23 . This transfer cannot vanish by Lemma A.1. If those paths are absent, the remaining first-bias transfer is c 1 + η 3 c 3 e 13 c_1+\eta_3c_3e_{13} c 1 + η 3 c 3 e 13 , likewise nonzero unless all gain paths disappear. Thus edge 12/bias one is excluded.
For edge 13/bias one, a positive first bias and active last vertex give a unique nonzero inverse-square path; with the last vertex inactive, the remaining transfer is c 1 + η 2 c 2 e 12 c_1+\eta_2c_2e_{12} c 1 + η 2 c 2 e 12 . A negative first bias is again inactive.
For edge 23/bias two, an inactive middle vertex removes both gains, while an active middle and last vertex give a unique inverse-square bias-to-edge path. With only the middle active, its gain-bias output coefficient is uncancelled or every gain path disappears. This covers the three source-bias placements, with either gain sign on the edge.
A gain edge whose target vertex has the other gain as its bias.
These are edge 12/bias two, edge 13/bias three, and edge 23/bias three. The gain-dependent part of the target preactivation is σ D ( h − 1 ) \sigma D(h-1) σ D ( h − 1 ) , with σ ∈ { − 1 , 1 } \sigma\in\{-1,1\} σ ∈ { − 1 , 1 } and a nongain source prefix h h h . An active gain contribution has a nonzero short transfer to the output. The inverse coefficient therefore forces h 0 = 1 h^0=1 h 0 = 1 . The first- and two-hidden-prefix obstruction proved before the lemma excludes this. Inactive contributions leave the gain-free coefficient obstruction.
Edge 12 and bias three.
If the last vertex is inactive, the single-relay argument applies. If the middle is inactive and the last active, its gain-bias output transfer is uncancelled. Thus both are active at every winner and
h 1 0 ( c 2 0 + c 3 0 e 23 0 ) = c 3 0 h_1^0(c_2^0+c_3^0e_{23}^0)=c_3^0 h 1 0 ( c 2 0 + c 3 0 e 23 0 ) = c 3 0
The transfer is a nonzero constant Laurent polynomial. Winning affine span removes every query coefficient of the first source. It is a positive primitive bias b b b , and the equation becomes
b 0 ( c 2 0 + c 3 0 e 23 0 ) = c 3 0 . b^{0}(c_{2}^{0}+c_{3}^{0}e_{23}^{0})=c_{3}^{0}. b 0 ( c 2 0 + c 3 0 e 23 0 ) = c 3 0 .
contrary to Lemma A.5 .
Edge 13 and bias two.
An inactive middle vertex gives the single relay; an inactive last vertex leaves an uncancelled bias transfer or no gain contribution. Both are active at every winner, with
c 3 0 h 1 0 = c 2 0 + c 3 0 e 23 0 . c_{3}^{0}h_{1}^{0}=c_{2}^{0}+c_{3}^{0}e_{23}^{0}. c 3 0 h 1 0 = c 2 0 + c 3 0 e 23 0 .
The right side is nonzero: otherwise Lemma A.1 forces c 2 = e 23 = 0 c_{2}=e_{23}=0 c 2 = e 23 = 0 , leaving the middle vertex without a path to output. The first source is active, and winning affine span makes it a positive primitive bias b b b . The first short-transfer equation forces c 2 = 0 c_{2}=0 c 2 = 0 and b = e 23 b=e_{23} b = e 23 in full. The two gain paths then cancel exactly in the winning pattern, so the coefficient obstruction applies.
Edge 23 and bias one.
A negative first bias leaves the single relay. A positive first bias is active throughout the cube. If the last vertex is inactive, its remaining transfer is c 1 + η 2 c 2 e 12 c_{1}+\eta_{2}c_{2}e_{12} c 1 + η 2 c 2 e 12 , which cannot cancel unless all paths disappear. If the middle is inactive, the transfer c 1 + c 3 e 13 c_{1}+c_{3}e_{13} c 1 + c 3 e 13 has the same property. Both middle and last are therefore active at every winner. Their unique inverse-square path forces e 12 e_{12} e 12 absent. The middle source is now query-affine and satisfies
c 3 0 h 2 0 = c 1 0 + c 3 0 e 13 0 . c_{3}^{0}h_{2}^{0}=c_{1}^{0}+c_{3}^{0}e_{13}^{0}. c 3 0 h 2 0 = c 1 0 + c 3 0 e 13 0 .
If the right side were zero, its two terms would both be absent, leaving the first vertex with no output path after e 12 = 0 e_{12}=0 e 12 = 0 . Thus winning affine span makes h 2 h_{2} h 2 a positive primitive bias b b b . The first short-transfer equation gives c 1 = 0 c_{1}=0 c 1 = 0 and b = e 13 b=e_{13} b = e 13 in full. Again the gain terms cancel exactly, contradicting the coefficient obstruction.
The three bias pairs, the three source-bias placements, the three target-bias placements and the three remaining mixed placements exhaust all cases. □
A Positive Output Gain and an Internal Negative Gain
We next place + D +D + D on an output edge and − D -D − D internally. At every clean winner the positive-output source is active by Lemma A.7 . A negative-gain path is also active there by Lemma A.9. These are pointwise conclusions and allow signed finite coefficients on other output edges. Write z i z_i z i for the ordinary output coefficient of hidden vertex i i i , that is, the coefficient of its hidden-to-output edge when that edge does not carry a gain. Likewise, e i j e_{ij} e ij denotes the edge from hidden vertex i i i to hidden vertex j j j when it does not carry a gain; absent coefficients are zero.
Lemma A.13 (The positive-output mixed orientation). No candidate for the clean query target places + D +D + D on a hidden-to-output edge and − D -D − D on a hidden edge or hidden bias.
Proof. Case names follow the key in the proof of Lemma A.11 . All coefficient equations below use winning patterns. Whenever a first affine source is active at all winners and its zeroth coefficient vanishes there, winning affine span makes that entire affine expression zero. Its query coefficients and bias are then absent in full, so the source cannot be active. We use this implication repeatedly.
We also record the two-hidden-prefix comparison needed in several cases. For h 1 = ρ ( U ) h_{1}=\rho(U) h 1 = ρ ( U ) , h 2 = ρ ( V + t h 1 ) h_{2}=\rho(V+th_{1}) h 2 = ρ ( V + t h 1 ) , suppose both are active and V 0 + t 0 U 0 = 0 V^{0}+t^{0}U^{0}=0 V 0 + t 0 U 0 = 0 . Write
U 0 = F L U + g U Γ + s U s 0 , V 0 = F L V + g V Γ + s V s 0 . U^{0}=FL_{U}+g_{U}\Gamma+s_{U}s_{0},\qquad V^{0}=FL_{V}+g_{V}\Gamma+s_{V}s_{0}. U 0 = F L U + g U Γ + s U s 0 , V 0 = F L V + g V Γ + s V s 0 .
Here L U L_{U} L U and L V L_{V} L V are integers, the normalized finite parts. The nonfinite counts, g U , g V g_{U},g_{V} g U , g V of Γ \Gamma Γ and s U , s V s_{U},s_{V} s U , s V of s 0 s_{0} s 0 , are zero or one, since the active input edges are finite queries and each direct affine form has one bias. For finite t 0 = F n t t^{0}=F n_{t} t 0 = F n t , its nonfinite equations are F n t g U + g V = 0 F n_{t}g_{U}+g_{V}=0 F n t g U + g V = 0 and F n t s U + s V = 0 F n_{t}s_{U}+s_{V}=0 F n t s U + s V = 0 , forcing all counts zero because F > p F>p F > p . The scalar equation then forces L U = L V = 0 L_{U}=L_{V}=0 L U = L V = 0 , since the direct normalized sums are smaller than F F F . For t = Γ t=\Gamma t = Γ , the degree-two equation gives g U = 0 g_{U}=0 g U = 0 , the degree-minus-one equation gives s V = 0 s_{V}=0 s V = 0 , the degree-one equation F L U + g V = 0 FL_{U}+g_{V}=0 F L U + g V = 0 gives both terms zero, and the scalar equation
F L V + s U / q = 0 FL_V+s_U/q=0 F L V + s U / q = 0 gives both zero. For t = s 0 t=s_0 t = s 0 , interchange the two nonfinite types and obtain the same conclusion. With t t t absent, only V 0 = 0 V^0=0 V 0 = 0 is required.
A − D -D − D edge on a path to the positive-output source.
For edge 12/output two or edge 13/output three, the source first gate and the positive-output gate are active at every winner. The unique inverse-square coefficient forces the first affine source’s zeroth coefficient to vanish. Winning span gives the contradiction described above.
For edge 23/output three, write h 2 = ρ ( V + t h 1 ) h_2=\rho(V+th_1) h 2 = ρ ( V + t h 1 ) , and let X X X be the direct affine form of hidden vertex three, so that its preactivation is X + e 13 h 1 − D h 2 X+e_{13}h_1-Dh_2 X + e 13 h 1 − D h 2 . At every winner vertex three is active by Lemma A.7, and h 2 h_2 h 2 is active by Lemma A.9, since every path containing − D -D − D passes through h 2 h_2 h 2 . Every path containing a gain ends with the output edge + D +D + D of vertex three, so the gain contribution is
D ( X + e 13 h 1 ) − D 2 h 2 , D(X+e_{13}h_1)-D^2h_2, D ( X + e 13 h 1 ) − D 2 h 2 ,
with h 1 , h 2 h_1,h_2 h 1 , h 2 gain-free. Its w − 2 w^{-2} w − 2 coefficient is − h 2 0 -h_2^0 − h 2 0 , so the inverse-square equation gives h 2 0 = 0 h_2^0=0 h 2 0 = 0 . Its w − 1 w^{-1} w − 1 coefficient is X 0 + e 1 0 h 1 0 − h 2 ( 1 ) X^0+e_{1}^{0}h_1^0-h_2^{(1)} X 0 + e 1 0 h 1 0 − h 2 ( 1 ) , which the inverse-w w w equation requires to vanish.
If h 1 h_1 h 1 is inactive at a winner, then h 2 = V h_2=V h 2 = V with V 0 = 0 V^0=0 V 0 = 0 , so h 2 = w δ h_2=w\delta h 2 = w δ , where δ = V ( 1 ) \delta=V^{(1)} δ = V ( 1 ) denotes the w w w -coefficient of V V V at that winner. A nonfinite bias would leave a term of nonzero Γ \Gamma Γ -degree in V 0 V^0 V 0 , so the bias of V V V is finite or absent. If no query input of V V V were selected, V V V would be its bias alone, whose leading value is nonzero when present, and V 0 = 0 V^0=0 V 0 = 0 would give V = 0 V=0 V = 0 , contrary to the activity of h 2 h_2 h 2 . The selected query inputs have positive leading values, so the bias of V V V is a finite negative, of normalized leading value − Q -Q − Q , and the selected query sum is Q Q Q . A sum of k k k powers of two has at most k k k binary ones, and Q = 2 r − 1 Q=2^r-1 Q = 2 r − 1 has r r r of them, so at least r r r and at most 3 r 3r 3 r query inputs are selected. The correction of a finite negative bias is zero or one, so 2 r ≤ δ ≤ 6 r + 1 < F 2r\leq\delta\leq6r+1<F 2 r ≤ δ ≤ 6 r + 1 < F . Since h 1 = 0 h_1=0 h 1 = 0 and h 2 ( 1 ) = δ h_2^{(1)}=\delta h 2 ( 1 ) = δ , the inverse-w w w equation now reads X 0 = δ X^0=\delta X 0 = δ . The Γ 0 \Gamma^0 Γ 0 -coefficient of X 0 X^0 X 0 is F F F times the normalized finite leading sum of X X X , because a nonfinite bias contributes only at Γ \Gamma Γ -degree one or minus one. It cannot equal the integer δ \delta δ , which lies strictly between zero and F F F . Thus h 1 h_1 h 1 is active at every winner.
The two-hidden-prefix comparison removes both leading affine forms when t t t is present, or V 0 V^0 V 0 alone when it is absent. Winning span makes the requisite active source identically zero.
For edge 12/output three with edge 23 present, the same unique inverse-square path forces the first source zero. If edge 23 is absent, vertices two and three can be interchanged; this is the edge 13/output-two case below. These cases cover every placement of − D -D − D on an edge that lies on a path to the vertex whose output edge carries + D +D + D .
Edge 13, positive output one.
The inverse equation is
U 0 ( 1 − z 3 0 ) = 0. U^0(1-z_3^0)=0. U 0 ( 1 − z 3 0 ) = 0.
No primitive has leading value one, so U 0 U^0 U 0 vanishes at every winner and winning span excludes the active first source.
Edge 12, positive output one.
The inverse equation has factor
U 0 ( 1 − z 2 0 − η 3 z 3 0 e 23 0 ) = 0. U^0(1-z_2^0-\eta_3z_3^0e_{23}^0)=0. U 0 ( 1 − z 2 0 − η 3 z 3 0 e 23 0 ) = 0.
The second factor cannot be zero. Its constant Γ \Gamma Γ -coefficient is 1 − F Z − I / q 1-FZ-I/q 1 − F Z − I / q , with Z ∈ Z Z\in\mathbb{Z} Z ∈ Z and I ∈ { 0 , 1 } I\in\{0,1\} I ∈ { 0 , 1 } , because the only product without a finite factor is a positive balanced pair. Clearing q q q and reducing modulo F F F excludes equality to zero. Again the first source vanishes on the winning span, a contradiction.
Edge 23, positive output one.
All three gates are active at every winner, and the inverse identity is
U 0 = z 3 0 ( V 0 + t 0 U 0 ) . U^0=z_3^0(V^0+t^0U^0). U 0 = z 3 0 ( V 0 + t 0 U 0 ) .
It is affine on the winning stencil.
If a query coefficient U j = F u j > 0 U_j=Fu_j>0 U j = F u j > 0 is present and z 3 z_3 z 3 is finite, a finite or absent t t t makes the right coefficient divisible by F 2 F^2 F 2 . After division by F F F , it would make u j u_j u j a multiple of F F F , although u j ≤ Q < F u_j\leq Q<F u j ≤ Q < F . A nonfinite t t t instead introduces an unmatched Γ \Gamma Γ -degree. For nonfinite z 3 z_3 z 3 , a finite t t t leaves the right side at the degree of z 3 z_3 z 3 , so it cannot equal the nonzero scalar left side. Equal nonfinite types give a degree-two term. Opposite types give the balanced product 1 / q 1/q 1/ q , but then the direct V j V_j V j term must be zero and the remaining equality is U j = U j / q U_j = U_j/q U j = U j / q , impossible. For nonfinite z 3 z_3 z 3 and missing t t t , the right side z 3 0 V j z_3^0V_j z 3 0 V j is zero or has the nonzero Γ \Gamma Γ -degree of z 3 z_3 z 3 . Thus U U U has no query coefficient. The affine identity then removes every query coefficient of V V V too, leaving only the last effective hinge and contradicting the query target.
Edge 13, positive output two.
All three gates are active at every winner and
V 0 = ( z 3 0 − t 0 ) U 0 . V^0 = (z_3^0-t^0)U^0. V 0 = ( z 3 0 − t 0 ) U 0 .
If U U U has no query coefficient, neither has V V V , and again only one effective hinge remains. Otherwise compare a positive direct query coefficient of U U U . A nonzero finite difference z 3 0 − t 0 z_3^0-t^0 z 3 0 − t 0 has an extra factor F F F , larger than any available normalized direct coefficient. Different nonfinite types or one finite and one nonfinite type leave unmatched degrees. Thus z 3 0 = t 0 z_3^0=t^0 z 3 0 = t 0 and every query coefficient of V V V is zero; the full affine identity also removes its bias. If t t t is negative, h 2 = ρ ( t h 1 ) = 0 h_2=\rho(th_1)=0 h 2 = ρ ( t h 1 ) = 0 , contrary to positive-output activity. If t t t is positive, equality of positive leading primitives is full equality z 3 = t z_3=t z 3 = t . The gain contribution then cancels exactly in the winning pattern, which the coefficient obstruction excludes.
Edge 23, positive output two.
The inverse equation ( 1 − z 3 0 ) h 2 0 = 0 (1-z_3^0)h_2^0=0 ( 1 − z 3 0 ) h 2 0 = 0 forces h 2 0 = 0 h_2^0=0 h 2 0 = 0 . If h 1 h_1 h 1 is inactive at a winner, then h 2 = w δ h_2=w\delta h 2 = w δ , where, as in the edge 23/output-three case, δ ≥ 2 r \delta\ge2r δ ≥ 2 r is the w w w -coefficient of h 2 h_2 h 2 . The gain contribution has small constant residue δ \delta δ : the z 3 z_3 z 3 term is an F F F -multiple when finite and has nonzero Γ \Gamma Γ -degree otherwise. The equation δ + r 0 = 1 \delta+r_0=1 δ + r 0 = 1 , where r 0 r_0 r 0 is the gain-free residue, with 0 ≤ r 0 < 1 0\le r_0<1 0 ≤ r 0 < 1 as in the proof of Lemma A.7, is impossible. Therefore h 1 h_1 h 1 is active at every winner. When t t t is present, the two-hidden-prefix comparison forces both leading affine forms zero, and winning span excludes the active source. When t = 0 t=0 t = 0 , only V 0 = 0 V^0=0 V 0 = 0 is needed; winning span makes V V V identically zero, contradicting the required activity of h 2 = V h_2=V h 2 = V . This finishes all nine negative-edge/positive-output placements.
A negative bias at the first vertex or at the output source.
A first hidden bias − D -D − D is inactive on the clean cube, contrary to the necessary negative-gain path. If the negative bias is at the positive-output source itself, its direct inverse-square coefficient is nonzero. If a middle negative bias feeds a last positive-output source through a present edge, the unique inverse-square transfer is that nonzero edge coefficient. If the edge is absent, interchange the two incomparable vertices and use the last-bias/output-two case below.
Bias three, positive output one.
The inverse equation gives U 0 = z 3 0 U^0=z_3^0 U 0 = z 3 0 on every winner. Winning span removes all query coefficients of the first source and matches its bias leading value with z 3 0 z_3^0 z 3 0 . A negative finite match would make that first gate inactive. A positive match is full equality by Lemma A.1, including the nonfinite types. Thus the two gain terms cancel exactly, contradicting the coefficient obstruction.
Bias three, positive output two.
At every winner h 2 0 = z 3 0 h_2^0=z_3^0 h 2 0 = z 3 0 , with h 2 = ρ ( V + t ρ ( U ) ) h_2=\rho(V+t\rho(U)) h 2 = ρ ( V + tρ ( U )) . If t , z 3 t,z_3 t , z 3 are finite, with z 3 0 = F n z z_3^0=Fn_z z 3 0 = F n z and L U , L V L_U,L_V L U , L V as in the two-hidden-prefix comparison, the constant-degree equation is
F L V + η 1 F 2 n t L U = F n z . FL_V+\eta_1F^2n_tL_U=Fn_z. F L V + η 1 F 2 n t L U = F n z .
The same equation has right side zero for nonfinite z 3 z_3 z 3 . For nonfinite t t t , the only additional constant-degree term is an optional positive balanced bias pair 1 / q 1/q 1/ q . Reduction modulo F F F after clearing q q q eliminates that pair. In every case L V = n z L_V=n_z L V = n z , or zero for nonfinite z 3 z_3 z 3 , at all winners. Winning span removes every direct query input of V V V and identifies its finite bias leading value when present.
If z 3 z_3 z 3 is finite negative, V V V uses the other finite negative bias. Both finite negatives are then consumed, and the first gate is positive-only. If t t t is absent, h 2 = V h_2=V h 2 = V is a negative constant and is inactive. If t t t is present, the identity h 2 0 = z 3 0 h_2^0=z_3^0 h 2 0 = z 3 0 , compared in every Γ \Gamma Γ -degree and not only in the constant degree, forces any active first gate’s leading value to be zero, whether t t t is finite or nonfinite. A nonzero positive-only first affine gate cannot have that property. An inactive first gate leaves negative V V V and makes h 2 h_2 h 2 inactive too. Hence a finite z 3 z_3 z 3 must be positive, and its matching finite bias is equal in full. Their w w w -corrections agree. If z 3 z_3 z 3 is nonfinite, both finite corrections are zero.
For finite t t t , the remaining Γ 0 w 1 \Gamma^0w^1 Γ 0 w 1 contribution to h 2 h_2 h 2 through the first gate is an F / 2 F/2 F /2 multiple. For nonfinite t t t , it has nonzero Γ \Gamma Γ -degree, since the derivative U ( 1 ) U^{(1)} U ( 1 ) of the first gate’s affine form U U U is a scalar. Thus the constant gain coefficient is the difference of the matching final corrections, namely zero, plus an F / 2 F/2 F /2 multiple, so its gain residue is zero. It cannot complete the gain-free residue below one to the required output.
Bias two, positive output one. Here the first gate is hidden vertex one, which carries the positive output gain, and U U U is its affine preactivation. The inverse equation makes U 0 U^0 U 0 equal to one of at most two fixed Laurent values,
U 0 = z 2 0 + η 3 z 3 0 e 23 0 . U^0 = z_2^0 + \eta_3 z_3^0 e_{23}^0. U 0 = z 2 0 + η 3 z 3 0 e 23 0 .
If it had two positive query coefficients, the all-ones vertex, one deletion and two deletions would give three distinct values, all at winners. Thus it has at most one clean query input. Fix that input to zero if it exists, so that the first gate is constant on the resulting face. If that constant is not positive, the first gate is inactive, so the source of + D +D + D is inactive and Lemma A.7 excludes the winners of this face. Otherwise let that constant be the positive primitive b b b . The middle preactivation is
− D + t b + I 2 , -D+tb+I_2, − D + t b + I 2 ,
where I 2 I_2 I 2 is a direct query sum.
Under Γ = M 9 \Gamma=M^9 Γ = M 9 , the leading term of a nongain primitive is a multiple of M − 9 M^{-9} M − 9 , M 0 M^0 M 0 or M 9 M^9 M 9 , according as it is a selector, a finite primitive or Γ \Gamma Γ . A product of two nongain primitives therefore has leading specialized M M M -exponent in { − 18 , − 9 , 0 , 9 , 18 } \{-18,-9,0,9,18\} { − 18 , − 9 , 0 , 9 , 18 } ; in particular this exponent is never one, the M M M -exponent of D = M D=M D = M . At exponent zero its coefficient is bounded in terms of P 2 P^2 P 2 ; every query sum is at most 3 r P + 6 r w 3rP+6rw 3 r P + 6 r w . The chosen M M M therefore makes the middle gate’s sign constant on the entire face: a positive exponent-nine or exponent-eighteen product dominates, a negative such product keeps it off, and every smaller product leaves − D -D − D dominant. The first gate is constant and the middle gate affine or zero, leaving only the third effective hinge. This contradicts the robust face statement of Lemma 4.7 . The stated bias cases exhaust all positions and complete the proof. □
A Negative Output Gain and an Internal Positive Gain
The opposite mixed orientation has positive ordinary output branches, but those branches need not vanish at the origin and the unit vectors of the restricted cube before this is proved. We use the pointwise negative-gain obstruction to ensure that the positive gain contributes on every query winner.
Lemma A.14 (The negative-output mixed orientation). No candidate for the clean query target places − D -D − D on a hidden-to-output edge and + D +D + D on a hidden edge or hidden bias.
Proof. Let P ∗ P_\ast P ∗ denote either positive magnitude P − w P-w P − w or P P P of a finite negative occurrence. Both have w 0 w^0 w 0 value P = F Q P=FQ P = F Q . As at the start of this appendix, h 0 h^0 h 0 denotes the w 0 w^0 w 0 -coefficient of a gain-free frozen expression h h h , retaining its Laurent dependence on Γ \Gamma Γ . Every positive finite primitive has w 0 = F k w^0=Fk w 0 = F k , with integer 1 ≤ k ≤ Q 1\leq k\leq Q 1 ≤ k ≤ Q . A direct query coefficient is at most F Q FQ F Q . The total leading query sum at a vertex is at most 3 r F Q < F 2 3rFQ<F^2 3 r F Q < F 2 .
We will also need bounds on actual values, after numerical specialization. Every actual query primitive F 2 i + 2 w F2^i+2w F 2 i + 2 w is at most P = F Q P=FQ P = F Q , since Q − 2 i ≥ 1 Q-2^i\geq1 Q − 2 i ≥ 1 and F > 2 w F>2w F > 2 w . Thus a direct query sum is at most A = 3 r P A=3rP A = 3 r P . Put U 0 = 5 r P U_0=5rP U 0 = 5 r P and U 1 = 8 r P 2 U_1=8rP^2 U 1 = 8 r P 2 . A first gain-free vertex without a Γ \Gamma Γ bias has value at most A + P ≤ U 0 A+P\leq U_0 A + P ≤ U 0 : all finite positive primitives are at most P P P , a selector is below one, and a negative bias only lowers its ReLU value. Here P ≥ 6 P\geq6 P ≥ 6 , r < Q < P r<Q<P r < Q < P , and F > 8 Q 4 > 10 r F>8Q^4>10r F > 8 Q 4 > 10 r ; the choice of M M M gives D = M > 2 H 0 D=M>2H_0 D = M > 2 H 0 , and the definition of H 0 H_0 H 0 gives 2 H 0 > 2 ⋅ 16 ⋅ 64 P 4 = 2048 P 4 2H_0>2\cdot16\cdot64P^4=2048P^4 2 H 0 > 2 ⋅ 16 ⋅ 64 P 4 = 2048 P 4 . Consequently
P U 1 + U 0 = 8 r P 3 + 5 r P < 13 P 4 < D / 64. PU_1+U_0=8rP^3+5rP<13P^4<D/64. P U 1 + U 0 = 8 r P 3 + 5 r P < 13 P 4 < D /64.
Also P < D P<D P < D and D ≥ 2 D\geq2 D ≥ 2 , so
2 P D + D + P U 1 + U 0 < 3 D 2 < D 9 = Γ , P − w > P / 2. 2PD+D+PU_1+U_0<3D^2<D^9=\Gamma,\qquad P-w>P/2. 2 P D + D + P U 1 + U 0 < 3 D 2 < D 9 = Γ , P − w > P /2.
These explicit inequalities will justify each comparison with the positive-gain bias.
Both finite negatives enter distinct hidden vertices, by Lemma A.3 : at least two distinct hidden vertices must receive negative contributions on the clean cube, − D -D − D is an output edge, and the two finite negatives are the only other negative occurrences. Thus precisely two hidden vertices can supply effective ReLU nonlinearities; the remaining hidden vertex is a positive wire. Call a hidden vertex negative-receiving when a finite negative occurrence enters it. Making either negative-receiving vertex globally linear or inactive leaves at most one effective hinge, contradicting the target. The same argument applies after fixing at most two query coordinates to zero.
Elementary facts used.
Short internal transfers do not vanish at w 0 w^0 w 0 .
We call the comparisons of this paragraph the short-transfer comparison. Between two distinct hidden vertices there are at most a direct path and a two-edge path. If either exists in the frozen mask, their combined gain-free transfer has nonzero w 0 w^0 w 0 .
Cancellation would make the w 0 w^0 w 0 value of the direct edge the negative of that of the two-edge product. At equal Γ \Gamma Γ exponents, with signs absorbed into the integers, it would require one of the following, which compare a finite direct edge with two finite edges, a Γ \Gamma Γ or s 0 s_0 s 0 direct edge with the same primitive times a finite edge, and a finite direct edge with a balanced pair, respectively:
None is possible. The same reasoning applies to a transfer from one hidden vertex to positive output edges through at most one other hidden vertex.
Also, a transfer from a hidden vertex to positive outputs through at most one other vertex cannot have w 0 w^0 w 0 identically 1 1 1 . Its Γ \Gamma Γ coefficient is
F Z + n q , Z ∈ Z , n ∈ { 0 , 1 } . FZ + \frac{n}{q}, \qquad Z \in\mathbb{Z}, \quad n \in\{0,1\}. F Z + q n , Z ∈ Z , n ∈ { 0 , 1 } .
The term n / q n/q n / q can arise only from Γ s 0 = 1 / q \Gamma s_0 = 1/q Γ s 0 = 1/ q , the only two-edge Γ 0 \Gamma^0 Γ 0 path without a factor F F F . Equality with one would give q F Z = q − n qFZ = q-n q F Z = q − n , a multiple of F F F lying strictly between zero and F F F , which is impossible.
A first finite-negative-biased hinge has a robust leading variation.
We call the argument of this paragraph the robust variation argument. Such a first hidden vertex is
H ( x ) = ρ ( F ( L ( x ) − Q ) + w ( 2 k ( x ) + δ b ) ) , δ b ∈ { 0 , 1 } , H(x) = \rho\left(F(L(x)-Q) + w(2k(x) + \delta_b)\right), \qquad\delta_b \in\{0,1\}, H ( x ) = ρ ( F ( L ( x ) − Q ) + w ( 2 k ( x ) + δ b ) ) , δ b ∈ { 0 , 1 } ,
where L ( x ) L(x) L ( x ) is the sum of the supported direct query powers of two times their input bits, and k ( x ) k(x) k ( x ) counts those inputs equal to one. The bias is − P + w δ b -P + w\delta_b − P + w δ b , with δ b = 0 \delta_b = 0 δ b = 0 or 1 1 1 according to which finite negative occurrence is used.
If no query coordinate is supported, this hinge is identically zero and leaves at most one effective nonlinearity. Thus at least one is supported. Put E 0 = L ( 1 ) − Q E_0 = L(1)-Q E 0 = L ( 1 ) − Q . Then
E 0 > max { nonzero coefficients of L } . E_0 > \max\{\text{nonzero coefficients of }L\}. E 0 > max { nonzero coefficients of L } .
Otherwise, fixing a coordinate with the largest coefficient in L L L and one further supported coordinate to zero makes this hinge identically inactive. If it has only one supported coordinate, fixing that coordinate suffices. Either outcome leaves at most one effective hinge on a subcube whose target still requires two by Lemma 4.7.
There are at least two supported coordinates, since one alone cannot have excess larger than its own weight. Let i i i and j j j be two distinct supported coordinates of H H H , and let 2 i 0 , 2 j 0 2^{i_0},2^{j_0} 2 i 0 , 2 j 0 be their coefficients in L L L , so that i 0 , j 0 i_0,j_0 i 0 , j 0 are the levels of the occurrences on their input edges into H H H . Consequently H 0 H^0 H 0 takes three distinct values at suitable points
1 , 1 − e i , 1 − e i − e j , 1,\qquad1-e_i,\qquad1-e_i-e_j, 1 , 1 − e i , 1 − e i − e j ,
all of which win. The first two leading values are F E 0 FE_0 F E 0 and F ( E 0 − 2 i 0 ) > 0 F(E_0-2^{i_0})>0 F ( E 0 − 2 i 0 ) > 0 . The third is max { 0 , F ( E 0 − 2 i 0 − 2 j 0 ) } \max\{0,F(E_0-2^{i_0}-2^{j_0})\} max { 0 , F ( E 0 − 2 i 0 − 2 j 0 )} , strictly smaller than the second, whether it is zero or positive. Moreover, H 0 > 0 H^0>0 H 0 > 0 at 1 1 1 and at every one-coordinate deletion.
We will also use the resulting strict variation at a later active gate G G G whose only possible hidden predecessor is such a first hinge H H H ; we call the following calculation the strict-variation calculation. We call such a hinge independent, since it has no hidden predecessor of its own. Suppose G G G has a finite negative bias and is active on the winning stencil. There its leading value has the form
G 0 ( x ) = F Z ( x ) − P + c 0 H 0 ( x ) , G^0(x) = FZ(x) - P + c^0H^0(x), G 0 ( x ) = F Z ( x ) − P + c 0 H 0 ( x ) ,
where the edge c c c from H H H to G G G is a positive ordinary edge or zero, and Z Z Z is the direct query sum of this gate normalized by F F F .
If c ≠ 0 c \ne0 c = 0 , choose a supported coordinate j j j of H H H , of occurrence level i j i_j i j on its input edge into H H H . Both stencil values of H H H are active, so their leading difference is F 2 i j > 0 F2^{i_j} > 0 F 2 i j > 0 . Let d j ≥ 0 d_j \ge0 d j ≥ 0 be the direct normalized query coefficient of G G G . Deleting that coordinate changes G 0 G^0 G 0 by
F d j + c 0 F 2 i j ≠ 0. Fd_j+c^0F2^{i_j}\ne0. F d j + c 0 F 2 i j = 0.
Both terms have nonnegative Laurent coefficients and the second is nonzero. If c = 0 c=0 c = 0 , any present direct query instead gives a strictly positive difference by itself. With neither a predecessor edge nor a direct query, the negative bias makes G G G inactive, contradicting the hypothesis. Thus such a G G G cannot have constant leading value on the winning stencil. The same difference calculation, without the term − P -P − P , shows that a positive wire constant on this stencil has no direct query and no edge from a varying H H H . It is then its actual constant bias.
Positive primitives with the same w 0 w^0 w 0 value are identical.
The only possible concern is among finite primitives. The construction’s size separation makes Q − T Q-T Q − T larger than 2 r − 1 2^{r-1} 2 r − 1 and every β j \beta_j β j , while Q Q Q is distinct from all these values. The β j \beta_j β j are not powers of two. Thus equal leading values occur only between identical numerical primitive types, including duplicate occurrences.
The following constant-only exclusions follow directly by comparing Γ \Gamma Γ exponents and finite magnitudes. For positive primitive leading values b , t b,t b , t , a positive primitive or absent c c c , and a positive primitive b ′ b' b ′ , which occurs only in the last two equations, none of
( c − t ) b = P , ( c − 1 / t ) b = P , b ′ = ( P + t ) b , b ′ = ( P + 1 / t ) b (10) \begin{aligned}
(c-t)b&=P, & (c-1/t)b&=P,\\
b'&=(P+t)b, & b'&=(P+1/t)b \tag*{(10)}
\end{aligned} ( c − t ) b b ′ = P , = ( P + t ) b , ( c − 1/ t ) b b ′ = P , = ( P + 1/ t ) b (10)
holds.
For the first equality, finite b = F ℓ b=F\ell b = F ℓ would require c − t = Q / ℓ c-t=Q/\ell c − t = Q / ℓ , a positive scalar smaller than F F F . A nonzero scalar difference of primitives is an integer multiple of F F F , so this is impossible. For b = Γ b=\Gamma b = Γ , the required coefficient of Γ − 1 \Gamma^{-1} Γ − 1 is P P P , while a difference supplies only zero or ± 1 / q \pm1/q ± 1/ q . For b = s 0 b=s_0 b = s 0 , the required coefficient of Γ \Gamma Γ is P q Pq P q , while a difference supplies only zero or ± 1 \pm1 ± 1 .
For the reciprocal equality and finite b b b , a finite t = F n t=Fn t = F n makes c − 1 / t c-1/t c − 1/ t either negative, when c = 0 c=0 c = 0 , or at least F − 1 / F > Q F-1/F>Q F − 1/ F > Q , when c c c is finite positive. A nonfinite c c c leaves a nonzero degree. A nonfinite t t t cannot give a scalar: its reciprocal can only meet the opposite primitive with coefficient 1 / q − 1 1/q-1 1/ q − 1 or 1 − q 1-q 1 − q , neither zero. For nonfinite b b b , a finite t t t leaves a nonzero scalar that no finite c c c can cancel, again by F > 1 F>1 F > 1 . If t t t has the same nonfinite type as b b b , its reciprocal has the degree needed on the right, but its coefficient is negative and adding the only available opposite primitive leaves it negative. If t t t has the opposite type, cancellation of its unwanted reciprocal degree would require one of the same impossible equalities 1 / q = 1 1/q=1 1/ q = 1 or 1 = q 1=q 1 = q . These cases exhaust the second equality.
For each of the last two equalities, all contributions on the right are positive. A finite b b b makes its P b Pb P b contribution exceed the largest finite primitive P P P , and a nonfinite candidate on the left cannot contain that positive scalar term. A nonfinite b b b gives a coefficient of at least P > 1 P>1 P > 1 at its own degree, whereas the corresponding available primitive has coefficient one relative to that type. Additional positive terms cannot cancel it or reduce that coefficient.
The positive gain + D +D + D on an internal edge.
Let that edge be u → v u\to v u → v .
First, there is a winner with h u 0 ≠ 0 h_u^0\ne0 h u 0 = 0 . This is immediate if u u u is a first positive wire: since an active + D +D + D path is necessary, u u u is active at every winner, and an active first positive wire has a nonzero w 0 w^0 w 0 coefficient, because its value is a sum of positive primitives, whose positive leading values cannot cancel. If u u u is a first negative-biased hinge, it follows from the robust variation argument.
If u u u is second, let g g g be its sole possible hidden predecessor. Suppose h u 0 = 0 h_u^0=0 h u 0 = 0 at every winner. Since an active + D +D + D path is necessary, u u u is active at every winner.
If g g g is a first negative-biased hinge, its leading value changes by a positive multiple of F F F between 1 1 1 and a suitable one-coordinate deletion. A positive edge from g g g gives a nonzero change in h u 0 h_u^0 h u 0 . A negative finite edge gives a change of magnitude at least P F PF P F , which cannot be canceled by
the direct query change, at most F Q FQ F Q . A Γ \Gamma Γ or s 0 s_0 s 0 edge gives a nonzero change in its separate Γ \Gamma Γ exponent. If the edge is absent, vanishing on the whole stencil forces all direct queries and the bias of u u u absent, making u = 0 u=0 u = 0 .
If g g g is a positive wire, u u u must receive a finite negative. With a negative bias, all query coefficients are positive. Vanishing on the stencil removes every query, leaving an impossible primitive-product equality with P P P . With a negative edge from g g g , a coefficient involving a query of g g g has magnitude at least P F PF P F and cannot cancel a direct query coefficient. Again all queries disappear. Let B g , B u B_g,B_u B g , B u be the actual optional positive bias values at g , u g,u g , u , with zero for absence. The remaining equation is B u 0 = P B g 0 B_u^0=PB_g^0 B u 0 = P B g 0 . If B g ≠ 0 B_g\ne0 B g = 0 , Lemma A.1 excludes this equality, including an absent B u B_u B u . If B g = 0 B_g=0 B g = 0 , then B u 0 = 0 B_u^0=0 B u 0 = 0 , so B u = 0 B_u=0 B u = 0 , since every present positive primitive has a nonzero leading value. Both vertices are then exactly zero throughout the clean cube, contradicting activity of u u u at every winner.
This proves that the claimed winner exists.
Fix such a winner. Every path containing + D +D + D passes through the edge u → v u\to v u → v and then through v v v , so v v v is active there, since an active + D +D + D path is necessary; and u u u is active, since h u 0 ≠ 0 h_u^0\ne0 h u 0 = 0 . The value h u h_u h u is gain-free, because + D +D + D leaves u u u and − D -D − D is an output edge. If the − D -D − D source equals v v v , every path containing both gains consists of a path into u u u , the edge u → v u\to v u → v and the output edge − D -D − D , so the w − 2 w^{-2} w − 2 coefficient of the output is − h u 0 ≠ 0 -h_u^0\ne0 − h u 0 = 0 , which the output identity excludes. If the source equals u u u , let T v T_v T v be the frozen transfer from v v v to the output; it is gain-free, since no path from v v v uses the output edge of u u u . No path contains both gains, and the gain contribution is D h u ( T v − 1 ) Dh_u(T_v-1) D h u ( T v − 1 ) , whose inverse coefficient is h u 0 ( T v 0 − 1 ) h_u^0(T_v^0-1) h u 0 ( T v 0 − 1 ) . Since h u 0 ≠ 0 h_u^0\ne0 h u 0 = 0 and Laurent polynomials in Γ \Gamma Γ have no zero divisors, the w − 1 w^{-1} w − 1 equation forces T v 0 = 1 T_v^0=1 T v 0 = 1 . The transfer T v T_v T v passes through at most the remaining hidden vertex and ends at positive output edges, so this contradicts the short-transfer comparison.
The negative-output source is therefore the third vertex a a a , distinct from u , v u,v u , v . At such a winner it is active. Otherwise every − D -D − D contribution vanishes, and the inverse coefficient is h u 0 T v 0 h_u^0T_v^0 h u 0 T v 0 , where T v T_v T v is now the frozen transfer from v v v to the positive output edges. This transfer contains the active + D +D + D path, so T v 0 ≠ 0 T_v^0\ne0 T v 0 = 0 by the short-transfer comparison, and the w − 1 w^{-1} w − 1 term is nonzero. Nor can an active path run from v v v to a a a . Since u u u precedes v v v , such a path is the edge v → a v\to a v → a , and with v v v and a a a active it gives the w − 2 w^{-2} w − 2 coefficient − h u 0 -h_u^0 − h u 0 times the nonzero leading value of that edge, which is nonzero. Hence, if a a a is after v v v , the edge v → a v\to a v → a must be absent.
Let t > 0 t>0 t > 0 be the output edge from v v v . It must exist. On every winning mask the w − 1 w^{-1} w − 1 equation is
h a 0 = t 0 h u 0 . (11) h_a^0=t^0h_u^0. \tag*{(11)} h a 0 = t 0 h u 0 . (11)
In the following case equations, F Z FZ F Z denotes the corresponding direct query sum, b b b an optional positive primitive bias, and c c c a positive ordinary edge or zero. Their values may differ between cases. There are two structural possibilities.
First possibility: u u u is first and a a a is later.
First exclude a positive wire u u u with no bias and exactly one clean query input x j x_j x j . Fix x j = 1 x_j=1 x j = 1 . On this face u u u is the positive constant equal to its actual input primitive. At all-ones and every single deletion of a free coordinate, the face target wins by Lemma 4.7 . Equation (11) gives the same nonzero leading value t 0 u 0 t^0u^0 t 0 u 0 for a a a at all those points, so a a a is active there. The edge v → a v\to a v → a has already been excluded, and the only other possible hidden predecessor is the now constant u u u . Thus the leading value of the preactivation of a a a is affine in the free query coordinates. The face stencil affinely spans their coordinate space, so every free query coefficient in that preactivation is zero. Every present such direct edge has a positive leading query primitive F 2 i F2^i F 2 i ; hence all those edges are absent. Consequently a a a , like u u u , is an actual constant throughout the face.
Only v v v remains an effective ReLU, with affine preactivation, and the output is affine plus a scalar multiple of that one ReLU. This contradicts the assertion of Lemma 4.7 for a face with one coordinate fixed to one.
A positive wire with neither a query nor a bias is identically zero on the clean cube, contradicting the previously established winner with h u 0 ≠ 0 h_u^0\ne0 h u 0 = 0 . Every remaining first positive wire has either a positive bias or at least two query inputs, so u 0 > 0 u^0>0 u 0 > 0 at all-ones and every one-coordinate deletion. The same positivity for a first negatively biased hinge was proved above. Equation (11) now makes a a a active on this entire stencil. Its frozen leading affine formula can therefore be compared coefficient by coefficient there. In this case c c c is the edge from u u u to a a a .
If u u u is a negative-biased hinge, write
u 0 = F ( L − Q ) . u^{0}=F(L-Q). u 0 = F ( L − Q ) .
If a a a has a negative bias, then
F Z − P = ( t 0 − c 0 ) F ( L − Q ) . FZ-P=(t^{0}-c^{0})F(L-Q). F Z − P = ( t 0 − c 0 ) F ( L − Q ) . Comparing constants gives t 0 − c 0 = 1 t^{0}-c^{0}=1 t 0 − c 0 = 1 , impossible for positive primitives or an absent c c c .
If a a a has a negative edge from u u u , then
F Z + b 0 = ( t 0 + P ) F ( L − Q ) . FZ+b^{0}=(t^{0}+P)F(L-Q). F Z + b 0 = ( t 0 + P ) F ( L − Q ) . Its constant equation equates a nonnegative bias with − ( t 0 + P ) P < 0 -(t^{0}+P)P<0 − ( t 0 + P ) P < 0 .
If a a a is the positive wire, then
F Z + b 0 = ( t 0 − c 0 ) F ( L − Q ) . FZ+b^{0}=(t^{0}-c^{0})F(L-Q). F Z + b 0 = ( t 0 − c 0 ) F ( L − Q ) . Query coefficients and the constant force t 0 = c 0 t^{0}=c^{0} t 0 = c 0 , with no direct queries or bias at a a a . Lemma A.1 gives t = c t=c t = c exactly and hence h a = c h u h_a=ch_u h a = c h u exactly. At a winner, v v v is active, so the two gain-containing contributions cancel exactly, not merely at leading order. Lemma A.2 rules out output 1.
If u u u is a positive wire, a a a must receive a finite negative.
A negative bias gives
F Z − P = ( t 0 − c 0 ) u 0 . FZ-P=(t^{0}-c^{0})u^{0}. F Z − P = ( t 0 − c 0 ) u 0 . If u u u has any query, the query equations make the proportionality factor nonnegative, contradicting the negative constant. If u u u is constant, this is ( c 0 − t 0 ) b = P (c^{0}-t^{0})b=P ( c 0 − t 0 ) b = P , excluded by Equation (10) .
This covers both orders u , a , v u,a,v u , a , v and u , v , a u,v,a u , v , a .
Second possibility: a a a is first, followed by u , v u,v u , v .
If a a a is a first negative-biased hinge, write a 0 = F n a^{0}=Fn a 0 = F n , where n n n is an integer and at all-ones 0 < n < F 0<n<F 0 < n < F . In this subcase allow b b b to be an optional positive primitive bias or the remaining finite negative bias, and c c c to be a positive ordinary edge, zero, or the remaining finite negative edge from a a a to u u u . At most one of b , c b,c b , c can be negative, since the other finite negative has already been used at a a a . These are the exceptions to the default positive b , c b,c b , c convention above.
Since u u u is active,
F n = t 0 ( F Z + b 0 + c 0 F n ) . Fn=t^{0}(FZ+b^{0}+c^{0}Fn). F n = t 0 ( F Z + b 0 + c 0 F n ) .
If t 0 = F n t t^{0}=Fn_t t 0 = F n t is finite, only finite or absent b 0 = F n b , c 0 = F n c b^{0}=Fn_b,c^{0}=Fn_c b 0 = F n b , c 0 = F n c , with signed integers n b , n c n_b,n_c n b , n c , contribute to the F 0 F^{0} F 0 coefficient of the right side. That contribution is F 2 n t ( Z + n b + F n c n ) F^{2}n_t(Z+n_b+Fn_c n) F 2 n t ( Z + n b + F n c n ) , divisible by F 2 F^{2} F 2 , whereas F n Fn F n is not because 0 < n < F 0<n<F 0 < n < F . If t 0 = Γ t^{0}=\Gamma t 0 = Γ or s 0 s_0 s 0 , only an opposite positive nonfinite primitive in b b b or c c c contributes to that scalar coefficient. It is therefore
η + θ F n q , η , θ ∈ { 0 , 1 } . \frac{\eta+\theta Fn}{q},\qquad\eta,\theta\in\{0,1\}. q η + θ F n , η , θ ∈ { 0 , 1 } .
Equality with F n Fn F n would require ( q − θ ) F n = η (q-\theta)Fn=\eta ( q − θ ) F n = η , impossible since q ≥ 5 q\ge5 q ≥ 5 , F ( q − 1 ) > 1 F(q-1)>1 F ( q − 1 ) > 1 , and n ≥ 1 n\ge1 n ≥ 1 .
If a a a is a positive wire, u u u receives a finite negative. Comparing affine coefficients in Equation (11) :
F Z + b 0 = ( P + 1 / t 0 ) a 0 . FZ+b^{0}=(P+1/t^{0})a^{0}. F Z + b 0 = ( P + 1/ t 0 ) a 0 .
Queries violate P F > P F Q PF>P FQ P F > P F Q ; the constant-only case is excluded by (10) . Thus every internal-edge placement is excluded.
The positive gain + D +D + D as a hidden bias.
Let v v v be its vertex and a a a the negative-output source. If v = a v=a v = a , its bias and output edge give a unique nonzero inverse-square term. With at most two hidden vertices, a first positive-gain bias leaves only one negative-receiving vertex, contrary to Lemma A.3. Indeed, the preactivation of that first vertex v v v is its bias + D +D + D plus input terms, and on the clean cube every input edge either carries a positive query value or is fixed to zero, so v v v receives no negative contribution there; the only other hidden vertex is a a a . If the positive-gain bias is second, negative allocation requires a finite-negative-biased first hinge and a negative finite edge into the second vertex. That first hinge is bounded by U 0 U_0 U 0 , so the second preactivation is at least D − P U 0 > 0 D-PU_0>0 D − P U 0 > 0 throughout the cube. Only one effective nonlinearity remains, which is impossible. The bias cases below therefore have three hidden vertices.
At every winner, v v v has an active path to output. If a a a were reachable from v v v in that mask, the nonzero short transfer of the short-transfer comparison would give a nonzero w − 2 w^{-2} w − 2 coefficient. Thus it is not reachable.
If a a a were inactive, the remaining + D +D + D -to-positive-output transfer would have nonzero w 0 w^{0} w 0 , giving a nonzero w − 1 w^{-1} w − 1 term. Therefore the source a a a is active at every winner, and no active path goes from v v v to a a a .
The w − 1 w^{-1} w − 1 equation gives
h a 0 = T v 0 , h_a^{0}=T_v^{0}, h a 0 = T v 0 ,
where T v T_v T v is the gain-free transfer from the bias injection to positive output edges. Excluding a a a , there is at most one intervening hidden vertex. Consequently h a 0 h_a^{0} h a 0 assumes at most two fixed Laurent-polynomial values over all winners. If v v v is last, or the other vertex cannot lie on that transfer, it assumes one fixed value.
Consider the position of v v v .
The vertex v v v is first. It is a positive wire. The second vertex must be a genuine negative-receiving hinge. Here D ≤ v ≤ D + A < 2 D D\le v\le D+A<2D D ≤ v ≤ D + A < 2 D . With a negative edge from v v v and no Γ \Gamma Γ bias, the second preactivation is at most U 0 − ( P / 2 ) D < 0 U_0-(P/2)D<0 U 0 − ( P /2 ) D < 0 . With a Γ \Gamma Γ bias it is at least Γ − P ( D + A ) > 0 \Gamma-P(D+A)>0 Γ − P ( D + A ) > 0 . With a negative bias, a positive finite edge from v v v gives preactivation at least F D − P > 0 FD-P>0 F D − P > 0 , and a Γ \Gamma Γ edge gives at least Γ D − P > 0 \Gamma D-P>0 Γ D − P > 0 . Only an s 0 s_0 s 0 edge can retain a genuine hinge; its contribution is at most 2 D / ( q Γ ) < 1 2D/(q\Gamma)<1 2 D / ( q Γ ) < 1 .
If a a a is this second hinge, the absence of an active path from v v v to a a a removes the possible s 0 s_0 s 0 edge. It is then a first finite-negative-biased hinge, whose three distinct leading values from the robust variation argument contradict the at-most-two-values conclusion.
Otherwise a a a is last. The direct v → a v\to a v → a edge is absent by the active-path restriction. If the second hinge feeds a a a , it is active at all-ones. Indeed, its finite negative bias is its only negative parameter on the clean cube and v v v is a positive wire, so its preactivation is nondecreasing in every clean query coordinate; inactivity at all-ones would make it inactive throughout the clean cube, and it would not be a genuine hinge. Since v ≥ D > 0 v\ge D>0 v ≥ D > 0 everywhere and a a a is active at the winner all-ones, the absence of an active path from v v v to a a a removes the possible incoming s 0 s_0 s 0 edge of the second hinge from v v v . Thus both hinges are independent of v v v , and h a 0 h_a^{0} h a 0 must be constant over winners. A negative-biased a a a cannot be constant on the winning stencil, by the strict-variation calculation for a later active gate G G G in the elementary facts above. A negative edge from the finite first hinge makes a a a inactive at all-ones unless its bias is Γ \Gamma Γ : the hinge has value at least F F F there, giving preactivation at most U 0 − ( P / 2 ) F < 0 U_0-(P/2)F<0 U 0 − ( P /2 ) F < 0 . A Γ \Gamma Γ bias instead gives preactivation at least Γ − P U 0 > 0 \Gamma-PU_0>0 Γ − P U 0 > 0 throughout the cube. If the second hinge does not feed a a a , a a a itself is an independent first finite-negative-biased hinge, again contradicting the robust variation argument.
The vertex v v v is second. It cannot receive a finite negative. Its only possible negative is an edge from the first vertex. Without a Γ \Gamma Γ bias that first vertex is at most U 0 U_0 U 0 , so v v v 's preactivation is at least D − P U 0 > 0 D-PU_0>0 D − P U 0 > 0 . With a Γ \Gamma Γ bias the first vertex is at least Γ \Gamma Γ , and the preactivation of v v v is at most D + A − ( P / 2 ) Γ < 0 D+A-(P/2)\Gamma<0 D + A − ( P /2 ) Γ < 0 . Either way v v v cannot supply the required second nonlinearity.
Hence v v v is the positive wire; the first and last vertices receive the finite negatives. If a a a is first, the robust variation argument contradicts the at-most-two-values conclusion. If a a a is last, the absence of an active path from v v v to a a a removes v → a v \to a v → a , and h a 0 h_a^0 h a 0 equals the fixed output primitive of v v v . A negative-biased a a a cannot have this constant value on the stencil, by the same strict-variation calculation. A negative edge from the first finite hinge gives preactivation at most U 0 − ( P / 2 ) F < 0 U_0-(P/2)F<0 U 0 − ( P /2 ) F < 0 at all-ones unless the bias is Γ \Gamma Γ ; that bias gives the global lower bound Γ − P U 0 > 0 \Gamma-PU_0>0 Γ − P U 0 > 0 .
The vertex v v v is last. Here h a 0 h_a^0 h a 0 equals its fixed positive output primitive t 0 t^0 t 0 .
If both earlier vertices receive finite negatives, the first is a finite-negative-biased hinge. If a a a is first, the robust variation argument gives a contradiction. If a a a is second, the strict-variation calculation excludes a negative bias, and the preceding negative-edge argument excludes that alternative.
Otherwise one earlier vertex is the positive wire W W W , the other the sole earlier negative hinge H H H .
If H H H is first and a = H a=H a = H , the robust variation argument excludes it.
If H H H is first and a = W a=W a = W , the positive-wire version of the strict-variation calculation forces it to have no queries and no edge from H H H . It is a constant primitive bias, with W 0 = t 0 W^0=t^0 W 0 = t 0 . If the negative edge entering v v v comes from the first finite hinge H H H , its preactivation is at least D − P U 0 > 0 D-PU_0>0 D − P U 0 > 0 . If it instead comes from the constant W W W , a finite or s 0 s_0 s 0 bias there has value at most P P P , giving the lower bound D − P 2 > 0 D-P^2>0 D − P 2 > 0 . The remaining case is W = Γ W=\Gamma W = Γ . Its bias and the output primitive t = Γ t=\Gamma t = Γ consume both Γ \Gamma Γ occurrences, so no positive Γ \Gamma Γ edge from H H H remains to overcome − P ∗ Γ -P_* \Gamma − P ∗ Γ . The preactivation of v v v is then at most D + A + P U 0 − ( P − w ) Γ < 0 D+A+PU_0-(P-w)\Gamma<0 D + A + P U 0 − ( P − w ) Γ < 0 .
If W W W is first and a = H a=H a = H , the active hinge H H H has constant leading value on the whole winning stencil. With a negative bias, its nonnegative query coefficients must all vanish. With a negative edge from W W W , cancellation of a query coefficient would require a direct coefficient at least P F > F Q PF>FQ P F > F Q . Thus again every query vanishes. In either case H H H is constant on the entire cube and cannot supply a nonlinearity.
If W W W is first and a = W a=W a = W , constancy forces W W W to be a constant primitive bias equal to t t t .
In the case where W W W is first and a = W a=W a = W , suppose first that the constant W W W is finite or s 0 s_0 s 0 , so 0 < W ≤ P 0<W\le P 0 < W ≤ P . A Γ \Gamma Γ bias at H H H would give preactivation at least Γ − P 2 > 0 \Gamma-P^2>0 Γ − P 2 > 0 . A positive Γ \Gamma Γ edge from a constant finite W ≥ 1 W\ge1 W ≥ 1 also gives positive preactivation, at least Γ − P \Gamma-P Γ − P ; from W = s 0 W=s_0 W = s 0 it contributes only 1 / q 1/q 1/ q . In every remaining genuine-hinge case the direct query sum is at most A A A , an ordinary finite or selector bias is at most P P P , and a positive finite or selector product with W W W is at most P 2 P^2 P 2 . Including the balanced contribution gives
0 ≤ H ≤ A + P + P 2 + 1 ≤ U 1 . 0 \le H \le A+P+P^2+1 \le U_1. 0 ≤ H ≤ A + P + P 2 + 1 ≤ U 1 .
A negative edge to v v v from either W W W or H H H consequently leaves preactivation at least D − P U 1 > 0 D-PU_1>0 D − P U 1 > 0 .
Still in the case where W W W is first and a = W a=W a = W , suppose W = Γ W=\Gamma W = Γ . A finite or Γ \Gamma Γ positive edge to H H H gives preactivation at least Γ − P > 0 \Gamma-P>0 Γ − P > 0 . A negative edge gives preactivation at most A + Γ − ( P − w ) Γ < 0 A+\Gamma-(P-w)\Gamma<0 A + Γ − ( P − w ) Γ < 0 , even allowing a Γ \Gamma Γ bias. Thus a genuine nonlinear H H H can retain only an s 0 s_0 s 0 edge or no edge from W W W , giving H ≤ A + 1 ≤ U 1 H\le A+1\le U_1 H ≤ A + 1 ≤ U 1 . If the negative edge entering v v v comes from H H H , the lower bound is again D − P U 1 > 0 D-PU_1>0 D − P U 1 > 0 . If it comes from W = Γ W=\Gamma W = Γ , the bias at W W W and the output primitive t = Γ t=\Gamma t = Γ consume both Γ \Gamma Γ occurrences. The positive edge from H H H , if present, is therefore at most P P P , and v v v ’s preactivation is at most
D + A + P U 1 − ( P − w ) Γ < 0. D+A+PU_1-(P-w)\Gamma<0. D + A + P U 1 − ( P − w ) Γ < 0.
Every case contradicts either the formal output identity or the need for two effective nonlinearities. Absent edges throughout have coefficient zero. The coefficient obstruction in the cancellation case uses equality of the full gain contributions, not merely their leading terms.
Finite Negative Edges with Both Gains at Output
The previous lemmas leave both gains on distinct hidden-to-output edges. Negative allocation puts the two finite negatives into distinct hidden preactivations. The optional third hidden vertex is a positive wire. We first prove that neither finite negative can be an internal edge. The argument allows either gain to originate at the positive wire, because the negatively biased vertices have not yet been identified as the gain sources.
For this analysis put P a = P − δ a w P_a=P-\delta_a w P a = P − δ a w , P b = P − δ b w P_b=P-\delta_b w P b = P − δ b w , with { δ a , δ b } = { 0 , 1 } \{\delta_a,\delta_b\}=\{0,1\} { δ a , δ b } = { 0 , 1 } . The notation U , V , W U,V,W U , V , W denotes direct query sums, with a positive optional bias only when explicitly allowed. Every ordinary internal or output coefficient is positive or absent, because all negative occurrences have been assigned. The inverse output equation equates the zeroth w w w -coefficients of the two gain sources at every Boolean point, irrespective of its target label. At every clean winner both gain sources are active: an inactive positive source contradicts Lemma A.7 , and an inactive negative source contradicts Lemma A.9.
We will use a minimal leading-positive query coalition. For h = ρ ( U − P a ) h=\rho(U-P_a) h = ρ ( U − P a ) , a leading-positive query coalition , or simply a leading-positive coalition, is a selected set whose normalized query sum exceeds Q Q Q . Choose one with no proper subset still having this property. Such a set is called inclusion-minimal, and exists whenever any selected set exceeds Q Q Q , by repeatedly deleting coordinates while the property persists. Let 2 ℓ 2^\ell 2 ℓ be its smallest selected power of two and e e e its excess above Q Q Q . Deleting the coordinate of weight 2 ℓ 2^\ell 2 ℓ leaves sum at most Q Q Q , so e ≤ 2 ℓ e\le2^\ell e ≤ 2 ℓ ; positive integer excess gives 1 ≤ e 1\le e 1 ≤ e . The selected sum is divisible by 2 ℓ 2^\ell 2 ℓ , while Q = 2 r − 1 ≡ − 1 ( m o d 2 ℓ ) Q=2^r-1\equiv-1\pmod{2^\ell} Q = 2 r − 1 ≡ − 1 ( mod 2 ℓ ) , so e ≡ 1 ( m o d 2 ℓ ) e\equiv1\pmod{2^\ell} e ≡ 1 ( mod 2 ℓ ) . Therefore e = 1 e=1 e = 1 , and
h = F + w K 1 , K 1 = 2 k + δ a ≥ 4 , h=F+wK_1,\qquad K_1=2k+\delta_a\ge4, h = F + w K 1 , K 1 = 2 k + δ a ≥ 4 ,
where k k k is the number of selected query inputs and at least two are selected.
If no leading-positive coalition exists, h h h is zero or w K 1 wK_1 w K 1 , with every positive K 1 K_1 K 1 between two and 6 r + 1 6r+1 6 r + 1 . A direct query sum equal to Q Q Q uses at least r r r terms, so at that boundary the stronger lower bound K 1 ≥ 2 r K_1\ge2r K 1 ≥ 2 r holds. The gate h h h is exactly zero at the origin and unit vectors.
All residue comparisons below are literal after clearing 2 q 2 2q^2 2 q 2 and reducing modulo F F F . Every undisplayed constant-degree term contains a factor F F F , and the displayed residues and the remainder formed by the pure balanced contributions are smaller than the resulting period F / ( 2 q 2 ) F/(2q^2) F / ( 2 q 2 ) . This follows from F > 8 Q 4 F>8Q^4 F > 8 Q 4 , q , r < Q q,r<Q q , r < Q , and the bound 2 k + δ ≤ 6 r + 1 2k+\delta\le6r+1 2 k + δ ≤ 6 r + 1 for a direct correction with 0 ≤ k ≤ 3 r 0\le k\le3r 0 ≤ k ≤ 3 r selected query inputs and finite-bias correction δ ∈ { 0 , 1 } \delta\in\{0,1\} δ ∈ { 0 , 1 } . In particular a residue of magnitude at least two, or zero, cannot be turned into output one by a gain-free remainder in [ 0 , 1 ) [0,1) [ 0 , 1 ) . We continue to retain every actual activation pattern.
Lemma A.15 (The finite negative parameters are biases). If both gains are hidden-to-output edges in a candidate for the clean query target, both finite negative occurrences are hidden biases at distinct vertices.
Proof. At every clean query winner both gain sources are active, by Lemmas A.7 and A.9. Negative allocation puts the two finite negatives at distinct hidden receivers. If a receiver vanished identically on a face obtained by fixing at most two query coordinates to zero, all other positive gates could be substituted by their affine preactivations, leaving at most one effective hinge on that face. Lemma 4.7 excludes this possibility. We assume for a contradiction that at least one finite negative is an internal edge. The remaining hidden vertex, when present, is a positive wire. We first find one frozen expression valid on an affinely spanning winning stencil, then compare its affine constant term.
Gains at the first two vertices. Suppose the gains originate at the first two hidden vertices. At every winner their active values have the form
h = U + a , g = V + b + t h , h=U+a,\qquad g=V+b+th, h = U + a , g = V + b + t h ,
where U , V U,V U , V are direct query sums, a , b a,b a , b are biases or zero, and t t t is their internal edge or zero. Write μ j , ν j \mu_j,\nu_j μ j , ν j for the respective direct query coefficients of U , V U,V U , V . The inverse-w w w equation gives h 0 = g 0 h^0=g^0 h 0 = g 0 on the winning stencil. Affine spanning therefore gives, for every query coordinate j j j ,
ν j 0 = ( 1 − t 0 ) μ j 0 . \nu_j^0=(1-t^0)\mu_j^0. ν j 0 = ( 1 − t 0 ) μ j 0 .
If t ≠ 0 t\ne0 t = 0 and u j ≠ 0 u_j\ne0 u j = 0 , this equates the product t 0 u j 0 t^0u_j^0 t 0 u j 0 to the difference u j 0 − v j 0 u_j^0-v_j^0 u j 0 − v j 0 , contrary to Lemma A.1. Thus a present t t t forces all coefficients of U , V U,V U , V to vanish. Both gain sources are then constant, and the possible third vertex supplies only one effective hinge, again impossible.
Consequently t = 0 t=0 t = 0 , and the affine identities give U 0 = V 0 U^0=V^0 U 0 = V 0 and a 0 = b 0 a^0=b^0 a 0 = b 0 . Equal positive leading primitives have equal full values, and absent or nonfinite equal biases also agree in full. The only unequal biases with equal leading value are the two finite negatives. If they occur at h , g h,g h , g , the lemma already holds. Otherwise h = g h=g h = g on the whole query cube, so the gains cancel exactly; Lemma A.2 excludes a winner. This treats fewer than three hidden vertices as well.
One mask on a winning stencil. We may now assume that there are three hidden vertices, one gain originates at the last, and the other originates earlier. Only the remaining hidden vertex, which is not a gain source, can have a varying mask at winners. A positive wire can be replaced by its affine preactivation on the whole query cube. At a point where its value is zero, its nonnegative summands are zero; their frozen expressions are zero there too by the specialization argument in Lemma 4.6 . Thus this replacement preserves the formal output identities even at such a point. It suffices to make the negative-receiving vertices’ masks constant on a winning stencil, possibly after fixing at most two coordinates to zero.
We use the following consequences of the chosen scales:
F > 4 ( d + 2 ) , P − w > P / 2 , Γ > 2 ( d + 2 ) P 2 , ( P − w ) F / 2 > ( d + 2 ) P . F > 4(d+2), \qquad P-w > P/2, \qquad\Gamma> 2(d+2)P^2, \qquad(P-w)F/2 > (d+2)P. F > 4 ( d + 2 ) , P − w > P /2 , Γ > 2 ( d + 2 ) P 2 , ( P − w ) F /2 > ( d + 2 ) P .
The first two follow from the displayed bounds defining F , M F,M F , M . For the third, d < m d < m d < m , P ≥ 6 P \ge6 P ≥ 6 , and M > 2 H 0 > 2048 ( m + 1 ) F 4 M > 2H_0 > 2048(m+1)F^4 M > 2 H 0 > 2048 ( m + 1 ) F 4 , whereas Γ = M 9 > M \Gamma= M^9 > M Γ = M 9 > M . The fourth follows from the first two by multiplying ( P − w ) > P / 2 (P-w) > P/2 ( P − w ) > P /2 by F / 2 F/2 F /2 . Every direct query sum is at most d P dP d P , and an additional finite positive bias is at most P P P .
If the first vertex receives a finite negative, it is
h = ρ ( U − P a ) h = \rho(U-P_a) h = ρ ( U − P a )
with U U U a direct query sum. It is active at the all-ones vertex and at every vertex obtained from it by deleting up to two coordinates. For a gain source this follows from winner activity. Otherwise inactivity there, together with monotonicity, would make it vanish on the entire corresponding zero-fixed face, which was excluded above. Its leading preactivation is nonnegative at all those points: a negative integer multiple of F F F cannot be overcome by the correction, bounded by ( 6 r + 1 ) w < 1 (6r+1)w < 1 ( 6 r + 1 ) w < 1 . The direct query coefficient vector of U U U has support on at least two coordinates, since deleting a smaller support would leave the negative bias alone. At a single deletion choose a different coordinate in that support and delete it too. Restoring that coordinate adds at least F F F to the leading preactivation. Thus h 0 ≥ F h^0 \ge F h 0 ≥ F on the ordinary winning stencil, and h ≥ F h \ge F h ≥ F there because its correction is nonnegative.
If the second vertex is positive, its ReLU is already redundant. Otherwise the internal-negative assumption makes it ρ ( V + b − P b h ) \rho(V+b-P_bh) ρ ( V + b − P b h ) , with b b b positive or absent. For b ≠ Γ b \ne\Gamma b = Γ , its preactivation is at most ( d + 2 ) P − ( P − w ) F < 0 (d+2)P-(P-w)F < 0 ( d + 2 ) P − ( P − w ) F < 0 . For b = Γ b=\Gamma b = Γ , it is at least Γ − ( d + 2 ) P 2 > 0 \Gamma-(d+2)P^2 > 0 Γ − ( d + 2 ) P 2 > 0 , since h ≤ ( d + 2 ) P h \le(d+2)P h ≤ ( d + 2 ) P . Its mask is therefore constant on the stencil in either case.
Suppose instead that the first vertex is the positive wire A = U + a A=U+a A = U + a . If the second vertex has a negative bias, its form is
ρ ( V − P a + t A ) , t ≥ 0. \rho(V-P_a+tA), \qquad t \ge0. ρ ( V − P a + t A ) , t ≥ 0.
It is monotone. If it is a gain source, it is active at all winners. If it is not, inactivity at a winning deletion would make it vanish on that zero-fixed face, leaving only one effective negative receiver. Thus it is uniformly active on the winning stencil.
The remaining second-vertex form is
ρ ( V + b − P a A ) , A = U + a . \rho(V+b-P_aA), \qquad A=U+a. ρ ( V + b − P a A ) , A = U + a .
If this vertex is a gain source, its mask is already active on the stencil. Otherwise the wire itself is a gain source and is active at all winners. For a = Γ a=\Gamma a = Γ , the second preactivation has the following upper bound on the whole cube:
Γ + ( d + 2 ) P − ( P − w ) Γ < 0. \Gamma+(d+2)P-(P-w)\Gamma< 0. Γ + ( d + 2 ) P − ( P − w ) Γ < 0.
For a positive finite a a a , or when U U U has at least two supported coordinates, we have F / 2 ≤ A ≤ ( d + 2 ) P F/2 \le A \le(d+2)P F /2 ≤ A ≤ ( d + 2 ) P on the ordinary winning stencil. The preceding scale inequalities then make the second gate active when b = Γ b=\Gamma b = Γ , and inactive otherwise. If a = 0 a=0 a = 0 and U U U has at most one supported coordinate, a winning deletion makes the gain-bearing wire zero, impossible.
The only remaining case is a = s 0 a=s_0 a = s 0 with at most one supported query coordinate in U U U . Fix that coordinate to zero if it exists; then A = s 0 A=s_0 A = s 0 . For a positive finite b b b or b = Γ b=\Gamma b = Γ , the second gate is active everywhere on this face. For b = 0 b=0 b = 0 or b = s 0 b=s_0 b = s 0 , it is active precisely when V > 0 V>0 V > 0 , because
0 < P a s 0 < 1 < F . 0<P_as_0<1<F. 0 < P a s 0 < 1 < F .
If V V V has at least two free supported coordinates, it is active on the face’s winning stencil. If it has one, fix that coordinate to zero too; if it has none, no further fixing is needed. It is then uniformly inactive. At most two zero fixings have been used. These cases exhaust the orderings: the first vertex is either a negative-biased hinge or the positive wire; a negative edge into the second can only come from the first; and every last gain source is already active at winners.
We have obtained one frozen affine output expression valid at all points of a winning stencil on such a face. Those points affinely span its free query coordinates. Specialization makes the expression formally one at each stencil point, so each Laurent coefficient is the corresponding constant affine function on the whole face. In particular, the affine constant term is identically one. Zero fixings introduce no new constant terms.
The inverse coefficient’s affine constant term. Every finite leading primitive is F n Fn F n with
1 ≤ ∣ n ∣ ≤ Q , q ∤ n . 1 \leq|n| \leq Q,\qquad q \nmid n. 1 ≤ ∣ n ∣ ≤ Q , q ∤ n .
For sources this uses β j ≡ 3 ( m o d q ) \beta_j \equiv3 \pmod{q} β j ≡ 3 ( mod q ) ; for queries it uses that q q q is odd; for anchors, controls and the two negatives it uses q ∤ ( Q − T ) q \nmid(Q-T) q ∤ ( Q − T ) and q ∤ Q q \nmid Q q ∤ Q . The two nonfinite leading types are Γ , s 0 \Gamma,s_0 Γ , s 0 , with Γ s 0 = 1 / q \Gamma s_0 = 1/q Γ s 0 = 1/ q .
Let H + , H − H_+,H_- H + , H − be the frozen values of the positive- and negative-gain sources. The inverse coefficient says H + 0 − H − 0 = 0 H_+^0-H_-^0=0 H + 0 − H − 0 = 0 . Take its affine constant term and then its Γ 0 \Gamma^0 Γ 0 coefficient. A constant path starts with a bias and uses at most two internal edges. In the difference of the two gain sources there are at most two direct-bias terms, at most three two-factor terms, and just one possible three-factor term, the chain through all three hidden vertices. Indeed, number the hidden vertices 1 , 2 , 3 1,2,3 1 , 2 , 3 in topological order, so that the last gain source is vertex three and the other is vertex one or two. The bias-origin paths to vertex three are its own bias, the bias of vertex one times edge 13, the bias of vertex two times edge 23, and the bias of vertex one times edges 12 and 23. Those to vertex two are its own bias and the bias of vertex one times edge 12, and the only one to vertex one is its own bias. The earlier source therefore adds one direct-bias term and at most one two-factor term, and inactive vertices only remove terms.
In the difference H + 0 − H − 0 H_+^0-H_-^0 H + 0 − H − 0 , write F N 1 FN_1 F N 1 for the signed difference of the direct finite-bias terms. The quantities F 2 N 2 F^2N_2 F 2 N 2 and F 3 N 3 F^3N_3 F 3 N 3 collect the signed all-finite path contributions with two and three factors, respectively. Let k k k be the signed count of balanced two-factor paths, each contributing 1 / q 1/q 1/ q . Let N q N_q N q be zero if the unique three-factor term with one finite factor and one of each nonfinite type is absent. Otherwise let N q N_q N q be the signed integer n n n in its finite leading factor F n Fn F n , so that this term contributes F N q / q FN_q/q F N q / q . Hence
0 = F N 1 + F 2 N 2 + F 3 N 3 + F N q q + k q . 0 = FN_1 + F^2N_2 + F^3N_3 + \frac{FN_q}{q} + \frac{k}{q}. 0 = F N 1 + F 2 N 2 + F 3 N 3 + q F N q + q k .
and these integers satisfy
∣ N 1 ∣ ≤ 2 Q , ∣ N 2 ∣ ≤ 3 Q 2 , ∣ N 3 ∣ ≤ Q 3 , ∣ k ∣ ≤ 3. |N_1| \leq2Q,\qquad|N_2| \leq3Q^2,\qquad|N_3| \leq Q^3,\qquad|k| \leq3. ∣ N 1 ∣ ≤ 2 Q , ∣ N 2 ∣ ≤ 3 Q 2 , ∣ N 3 ∣ ≤ Q 3 , ∣ k ∣ ≤ 3.
If the mixed three-factor term is present, N q N_q N q is nonzero modulo q q q .
Multiply by q q q and reduce modulo F F F . Since ∣ k ∣ < F |k| < F ∣ k ∣ < F , this gives k = 0 k=0 k = 0 . After division by F F F , reduction modulo q q q gives N q ≡ 0 ( m o d q ) N_q \equiv0 \pmod{q} N q ≡ 0 ( mod q ) , so that three-factor term is absent and N q = 0 N_q=0 N q = 0 . The remaining equation is N 1 + F N 2 + F 2 N 3 = 0 N_1+FN_2+F^2N_3=0 N 1 + F N 2 + F 2 N 3 = 0 . Because ∣ N 1 ∣ < F |N_1|<F ∣ N 1 ∣ < F and ∣ N 2 ∣ < F |N_2|<F ∣ N 2 ∣ < F , successive reduction and division by F F F give N 1 = N 2 = N 3 = 0 N_1=N_2=N_3=0 N 1 = N 2 = N 3 = 0 . In particular the direct finite biases of the two gain sources have equal leading value. Under the internal-negative assumption they cannot be the two finite negatives. They are therefore both absent or nonfinite, or are equal positive finite primitives. Their full w w w -derivatives agree in each case.
The constant output coefficient. The constant w 0 Γ 0 w^0\Gamma^0 w 0 Γ 0 contribution of the gains is the difference of the w w w -derivatives of these two source constant expressions. The direct-bias derivatives cancel. A product with at least two finite factors has derivative divisible by F / 2 F/2 F /2 , since every finite correction is a half-integer multiple of w w w and one finite factor remains after differentiation. A three-factor product with one finite factor and the balanced nonfinite pair was just excluded. All-nonfinite products have derivative zero. Thus the gain-derived constant contribution belongs to ( F / 2 ) Z (F/2)\mathbb{Z} ( F /2 ) Z .
For gain-free constant paths, every finite factor contributes F F F at degree w 0 w^0 w 0 , and any denominator divides q 2 q^2 q 2 . The pure balanced paths contribute 1 / q 1/q 1/ q or 1 / q 2 1/q^2 1/ q 2 , positively. The path counts in Lemma A.2 also bound this bias-origin subset: taking the affine constant term only discards input-origin paths, and every pure balanced contribution is positive. The constant output identity consequently has form
1 = F Z 2 q 2 + k 1 q + k 2 q 2 , Z ∈ Z , 0 ≤ k 1 , k 2 ≤ p − q − 1. 1 = \frac{FZ}{2q^2} + \frac{k_1}{q} + \frac{k_2}{q^2},\qquad Z \in\mathbb{Z},\qquad0 \leq k_1,k_2 \leq p-q-1. 1 = 2 q 2 F Z + q k 1 + q 2 k 2 , Z ∈ Z , 0 ≤ k 1 , k 2 ≤ p − q − 1.
It implies
F Z = 2 ( q 2 − q k 1 − k 2 ) , 2 ≤ 2 ( q 2 − q k 1 − k 2 ) ≤ 2 q 2 < F , FZ = 2(q^2-qk_1-k_2), \qquad2 \le2(q^2-qk_1-k_2) \le2q^2 < F, F Z = 2 ( q 2 − q k 1 − k 2 ) , 2 ≤ 2 ( q 2 − q k 1 − k 2 ) ≤ 2 q 2 < F ,
which is impossible for an integer multiple of F F F . No internal finite-negative edge is therefore possible. Both finite negatives are biases at their distinct hidden receivers, as claimed. □
Output Gains Cannot Originate at a Positive Wire
By Lemma A.15 , the finite negatives are now biases at two distinct hidden vertices. The third vertex, if present, is positive and hence a wire. We still allow one gain to originate at that wire. In a topological order of the hidden vertices, the wire is upstream when it precedes both biased vertices, downstream when it follows both, and between them, a between wire, when it follows one biased vertex and precedes the other. The following proof excludes all three positions of the wire.
Lemma A.16 (The gain sources are the biased vertices). Let a permitted candidate represent the clean query target of Lemma 4.7 on its restricted cube. If both finite negatives are hidden biases and both gains are output edges, neither gain can originate at the remaining positive wire.
Proof. Let P 1 , P 2 P_1,P_2 P 1 , P 2 be the positive magnitudes of the first and second negatively biased vertices in topological order, so { P 1 , P 2 } = { P − w , P } \{P_1,P_2\}=\{P-w,P\} { P 1 , P 2 } = { P − w , P } . For h = ρ ( U − P 1 ) h=\rho(U-P_1) h = ρ ( U − P 1 ) , with direct query sum U U U , set K 1 = 0 K_1=0 K 1 = 0 at a tested input where h h h is inactive. When it is active, write h = F κ + w K 1 h=F\kappa+wK_1 h = F κ + w K 1 , where κ ∈ Z ≥ 0 \kappa\in\mathbb{Z}_{\geq0} κ ∈ Z ≥ 0 and 0 < K 1 ≤ 6 r + 1 0<K_1\leq6r+1 0 < K 1 ≤ 6 r + 1 . Thus K 1 K_1 K 1 is the frozen derivative contribution of the first hinge, including its activation indicator. At a leading-zero boundary the normalized sum of U U U is Q Q Q , so K 1 ≥ 2 r K_1\geq2r K 1 ≥ 2 r . A minimal leading-positive coalition gives κ = 1 \kappa=1 κ = 1 , as shown before Lemma A.15 . For the second hinge, K 2 K_2 K 2 denotes only the correction of its direct query sum and finite bias, excluding coupling terms. Its full derivative may be K 2 + t 0 K 1 K_2+t^0K_1 K 2 + t 0 K 1 or the coupled expression displayed below; only the direct quantity is bounded by 6 r + 1 6r+1 6 r + 1 . All output-gain sources are active at clean winners by Lemmas A.7 and A.9. The gain-free residues and the period comparison are those stated before Lemma A.15 .
An upstream wire.
A positive bias in the wire makes its zeroth coefficient nonzero everywhere. A positive query input does so on that input’s fixed-one face. Inverse equality activates the other gain source on that whole cube or face. Both gain sources are then active, hence equal to their affine preactivations, throughout it, so substitution leaves at most one effective hinge, contradicting the robust face statement of Lemma 4.7 . A wire with neither bias nor clean query input is zero on the restriction and cannot carry a gain at a clean winner. Thus an upstream gain-bearing wire is excluded.
A downstream wire.
Write
h = ρ ( U − P 1 ) , g = ρ ( V + t h − P 2 ) , u = W + a h + b g , h=\rho(U-P_1), \qquad g=\rho(V+th-P_2), \qquad u=W+ah+bg, h = ρ ( U − P 1 ) , g = ρ ( V + t h − P 2 ) , u = W + ah + b g ,
with direct query sums U , V U,V U , V and positive affine W W W . Both hinges are exactly zero at origin and units: there U U U and V V V are zero or a single query value, at most F ( Q + 1 ) / 2 + 2 w < P − w F(Q+1)/2+2w<P-w F ( Q + 1 ) /2 + 2 w < P − w , and h = 0 h=0 h = 0 removes the term t h th t h . Inverse equality for either pair involving forces W 0 = 0 W^0=0 W 0 = 0 at those tests, hence W = 0 W=0 W = 0 identically, since every present positive primitive has a nonzero leading value. All three hidden values are therefore zero at origin and units, and the output there is its bias plus at most one skip term. These ordinary output parameters are positive or absent and the target is zero at these tests, so the output bias and every skip edge from a clean query coordinate are absent; the other skip edges are fixed to zero on the restriction. Thus the ordinary output contribution, here the output bias and the skip edges, vanishes on the clean restriction.
First put the gains on h , u h,u h , u . Their inverse equation is
( 1 − a 0 ) h 0 = b 0 g 0 . (1-a^0)h^0=b^0g^0. ( 1 − a 0 ) h 0 = b 0 g 0 .
Suppose h 0 = F κ > 0 h^0=F\kappa>0 h 0 = F κ > 0 at a point. An active g g g then has g 0 ≥ 0 g^0\geq0 g 0 ≥ 0 at the prescribed specialization Γ = M g \Gamma=M^g Γ = M g . A finite t t t makes its positive product dominate the finite negative bias. A Γ \Gamma Γ edge gives a positive highest degree. A selector or absent edge can only activate with nonnegative scalar part. This sign assertion is used only at points where h 0 > 0 h^0>0 h 0 > 0 . A finite or Γ \Gamma Γ value of a 0 > 1 a^0>1 a 0 > 1 therefore has the wrong sign.
For absent or selector a a a , examine the remaining b , t b,t b , t types. A finite b b b and finite or absent t t t would make κ \kappa κ a multiple of F F F , although 0 < κ < F 0<\kappa<F 0 < κ < F . A nonfinite t t t instead gives a positive nonfinite coefficient, which cannot match the absent or negative selector coefficient on the left. For nonfinite b b b , the only scalar product available on the right is a balanced b t = 1 / q bt=1/q b t = 1/ q , giving scalar F κ / q F^{\kappa}/q F κ / q instead of F κ F^{\kappa} F κ . All other products have nonzero degree, and b = 0 b=0 b = 0 would require a 0 = 1 a^{0}=1 a 0 = 1 . Thus h 0 > 0 h^{0}>0 h 0 > 0 is impossible and h 0 = 0 h^{0}=0 h 0 = 0 everywhere.
At the all-ones winner h = w K 1 h=wK_{1} h = w K 1 , with K 1 ≥ 2 K_{1}\geq2 K 1 ≥ 2 , and u 0 = 0 u^{0}=0 u 0 = 0 . If b b b is present and g g g is active, inverse equality gives g 0 = 0 g^{0}=0 g 0 = 0 ; its direct query sum is then Q Q Q , and
g ( 1 ) = K 2 + t 0 K 1 , K 2 ≥ 2 r . g^{(1)}=K_{2}+t^{0}K_{1},\qquad K_{2}\geq2r. g ( 1 ) = K 2 + t 0 K 1 , K 2 ≥ 2 r .
The constant-degree residue of ( h − u ) / w (h-u)/w ( h − u ) / w is
K 1 − I K 1 / q , I ∈ { 0 , 1 } , K_{1}-IK_{1}/q,\qquad I\in\{0,1\}, K 1 − I K 1 / q , I ∈ { 0 , 1 } ,
where I = 1 I=1 I = 1 only for a balanced b t bt b t pair. It is at least 2 ( 1 − 1 / q ) > 1 2(1-1/q)>1 2 ( 1 − 1/ q ) > 1 . If b b b is absent or g g g inactive, that residue is K 1 K_{1} K 1 . Reversing gain signs negates it, and no gain-free remainder below one can repair either orientation.
Next put the gains on g , u g,u g , u . The inverse equation is
( 1 − b 0 ) g 0 = a 0 h 0 . (1-b^{0})g^{0}=a^{0}h^{0}. ( 1 − b 0 ) g 0 = a 0 h 0 .
If a = 0 a=0 a = 0 , then g 0 = 0 g^{0}=0 g 0 = 0 everywhere. At a winner g g g is active. Its direct sum has normalized value Q Q Q , and a present t t t forces h 0 = 0 h^{0}=0 h 0 = 0 by finite magnitude or nonfinite degree separation. Its derivative is K 2 + t 0 K 1 K_{2}+t^{0}K_{1} K 2 + t 0 K 1 , with K 2 ≥ 2 r K_{2}\geq2r K 2 ≥ 2 r and 0 ≤ K 1 ≤ 6 r + 1 0\leq K_{1}\leq6r+1 0 ≤ K 1 ≤ 6 r + 1 . The gain residue is
K 2 − I K 1 / q ≥ 2 r − ( 6 r + 1 ) / q > 1. K_{2}-IK_{1}/q\geq2r-(6r+1)/q>1. K 2 − I K 1 / q ≥ 2 r − ( 6 r + 1 ) / q > 1.
A missing t t t gives the same expression with I = 0 I=0 I = 0 . This excludes a = 0 a=0 a = 0 .
If a a a is present and h h h has a leading-positive coalition, choose a minimal one, so h 0 = F h^{0}=F h 0 = F . The inverse equation makes g g g active and gives
( 1 − b 0 ) ( v − Q + t 0 ) = a 0 , (1-b^{0})(v-Q+t^{0})=a^{0}, ( 1 − b 0 ) ( v − Q + t 0 ) = a 0 ,
where v v v is the direct normalized sum in V V V . Finite or Γ \Gamma Γ values b 0 > 1 b^{0}>1 b 0 > 1 have the wrong sign by the positive-h 0 h^{0} h 0 argument above. For b = s 0 b=s_{0} b = s 0 , a finite or absent t t t makes X = v − Q + t 0 X=v-Q+t^{0} X = v − Q + t 0 a scalar, so the left side X − X s 0 X-Xs_{0} X − X s 0 has scalar coefficient X X X and s 0 s_{0} s 0 -coefficient − X -X − X ; it is zero or has two monomials, and in neither case equals the present positive primitive a 0 a^{0} a 0 . For t = Γ t=\Gamma t = Γ , the scalar coefficient contains the nonintegral term ( v − Q ) − 1 / q (v-Q)-1/q ( v − Q ) − 1/ q , while the degree-one coefficient forces a = Γ a=\Gamma a = Γ . For t = s 0 t=s_{0} t = s 0 , a negative s 0 2 s_{0}^{2} s 0 2 coefficient is unmatched. Thus b = 0 b=0 b = 0 . Then v − Q = a 0 − t 0 v-Q=a^{0}-t^{0} v − Q = a 0 − t 0 ; since ∣ v − Q ∣ < F |v-Q|<F ∣ v − Q ∣ < F , finite differences force zero, and nonfinite differences must cancel identically. Hence v = Q v=Q v = Q and a = t a=t a = t in full. At this coalition g − u = V − P 2 = w K 2 g-u=V-P_{2}=wK_{2} g − u = V − P 2 = w K 2 , K 2 ≥ 2 r K_{2}\geq2r K 2 ≥ 2 r , whose signed gain value cannot be either Boolean label. Other ordinary terms have only an F F F -multiple scalar coefficient here.
If h h h has no leading-positive coalition, both h 0 , g 0 h^{0},g^{0} h 0 , g 0 vanish everywhere. At the all-ones winner the preceding derivative calculation again gives K 2 − I K 1 / q > 1 K_{2}-IK_{1}/q>1 K 2 − I K 1 / q > 1 . Thus both downstream gain pairs are excluded.
A between wire.
Write
h = ρ ( U − P 1 ) , u = V + a h , g = ρ ( W + b u + c h − P 2 ) , h=\rho(U-P_{1}),\qquad u=V+ah,\qquad g=\rho(W+bu+ch-P_{2}), h = ρ ( U − P 1 ) , u = V + ah , g = ρ ( W + b u + c h − P 2 ) ,
with direct query sums U , W U,W U , W and positive affine V V V . For gain pair h , u h,u h , u , the origin and the unit vectors of the restricted cube give V = 0 V=0 V = 0 . The inverse equation ( 1 − a 0 ) h 0 = 0 (1-a^{0})h^{0}=0 ( 1 − a 0 ) h 0 = 0 makes h 0 = 0 h^{0}=0 h 0 = 0 everywhere, and its all-ones gain residue is ± K 1 \pm K_{1} ± K 1 , K 1 ≥ 2 K_{1}\geq2 K 1 ≥ 2 . Both biased hinges and the wire are zero at origin and units, so the output there is its bias plus at most one skip term. These ordinary output parameters are positive or absent and the target is zero at these tests, so the output bias and every skip edge from a clean query coordinate are absent; the other skip edges are fixed to zero on the restriction. Thus the ordinary output contribution, here the output bias and the skip edges, vanishes on the clean restriction. The remaining gain-free terms cannot repair the residue.
For gain pair u , g u,g u , g , first remove V V V . A positive bias of V V V would give, at origin,
( b 0 − 1 ) V 0 = P . (b^{0}-1)V^{0}=P. ( b 0 − 1 ) V 0 = P .
A nonzero b b b would make its product with one primitive equal a sum of two primitives, contrary to Lemma A.1; absent b b b has the wrong sign. Hence V V V has no bias. At a unit vector with a present query coefficient of V V V , inverse equality requires
W j 0 + ( b 0 − 1 ) V j 0 = P . W_j^0+(b^0-1)V_j^0=P. W j 0 + ( b 0 − 1 ) V j 0 = P .
A finite b b b has an oversized F 2 F^2 F 2 contribution, a nonfinite b b b has unmatched degree, and absent b b b gives W j 0 − V j 0 < P W_j^0-V_j^0<P W j 0 − V j 0 < P , since W j 0 ≤ F ( Q + 1 ) / 2 W_j^0\le F(Q+1)/2 W j 0 ≤ F ( Q + 1 ) /2 . Thus V = 0 V=0 V = 0 identically and u = a h u=ah u = ah . If a = 0 a=0 a = 0 , the wire gain source is zero at a winner, already impossible.
For present a a a , a minimal leading-positive h h h coalition gives h 0 = F h^0=F h 0 = F , g 0 = a 0 F g^0=a^0F g 0 = a 0 F , and
v W − Q + ( b 0 − 1 ) a 0 + c 0 = 0. v_W-Q+(b^0-1)a^0+c^0=0. v W − Q + ( b 0 − 1 ) a 0 + c 0 = 0.
Here v W v_W v W is the direct normalized sum in W W W , with ∣ v W − Q ∣ < F \lvert v_W-Q\rvert<F ∣ v W − Q ∣ < F . Let a 0 = F ν a^0=F\nu a 0 = F ν be finite and suppose b 0 = F μ b^0=F\mu b 0 = F μ is finite, where ν , μ ≥ 1 \nu,\mu\ge1 ν , μ ≥ 1 because positive finite primitives have positive normalized leading values. A nonfinite c c c would be the only term at its F F F -degree, so c c c is finite or absent, with c 0 = F n c c^0=Fn_c c 0 = F n c and n c ≥ 0 n_c\ge0 n c ≥ 0 . The equation becomes v W − Q + F ( F μ ν − ν + n c ) = 0 v_W-Q+F(F\mu\nu-\nu+n_c)=0 v W − Q + F ( F μν − ν + n c ) = 0 , and reduction modulo F F F gives v W = Q v_W=Q v W = Q . Then F μ ν − ν + n c = 0 F\mu\nu-\nu+n_c=0 F μν − ν + n c = 0 , although F μ ν − ν + n c ≥ ( F − 1 ) ν > 0 F\mu\nu-\nu+n_c\ge(F-1)\nu>0 F μν − ν + n c ≥ ( F − 1 ) ν > 0 , which is impossible. A nonfinite b b b has an unmatched positive degree. For nonfinite a a a , finite b b b gives positive coefficient ( F μ − 1 ) a 0 (F\mu-1)a^0 ( F μ − 1 ) a 0 , and a same-type b b b gives a square degree, neither cancellable by positive c c c . An opposite-type b b b leaves v W − Q + 1 / q = 0 v_W-Q+1/q=0 v W − Q + 1/ q = 0 after the nonfinite terms cancel, impossible for integer v W − Q v_W-Q v W − Q . The only remaining choice is b = 0 b=0 b = 0 , c = a c=a c = a in full and v W = Q v_W=Q v W = Q . Then g − u = W − P 2 = w K 2 g-u=W-P_2=wK_2 g − u = W − P 2 = w K 2 , K 2 ≥ 2 r K_2\ge2r K 2 ≥ 2 r , again incompatible with either Boolean label.
Finally, if h h h has no leading-positive coalition, h 0 = u 0 = g 0 = 0 h^0=u^0=g^0=0 h 0 = u 0 = g 0 = 0 everywhere. At all-ones both gain sources are active, so h = w K 1 h=wK_1 h = w K 1 , the direct W W W -sum is Q Q Q , and
g ( 1 ) = K 2 + ( b 0 0 a 0 0 + c 0 0 ) K 1 , K 2 ≥ 2 r . g^{(1)}=K_2+(b_0^0a_0^0+c_0^0)K_1,\qquad K_2\ge2r. g ( 1 ) = K 2 + ( b 0 0 a 0 0 + c 0 0 ) K 1 , K 2 ≥ 2 r .
The constant residue of ( g − u ) / w (g-u)/w ( g − u ) / w is K 2 + I K 1 / q > 1 K_2+IK_1/q>1 K 2 + I K 1 / q > 1 , with I ∈ { 0 , 1 } I\in\{0,1\} I ∈ { 0 , 1 } . Reversing gain signs again negates it. This excludes both between-wire pairs and completes all wire positions. □
Removing Coupling Between the Biased Hinges
The output gains now originate at the two negatively biased vertices themselves. Both are active at every clean winner by Lemmas A.7 and A.9. This lets us compare their affine preactivations on a winning stencil, without first assuming a shared score. The resulting restrictions concern actual edges and will therefore hold on the full input cube.
Lemma A.17 (Parallelization). Consider any permitted candidate representing f f f in which both gains are hidden-to-output edges, both finite negatives are biases at two distinct hidden vertices, and the two gains originate at these two biased vertices. Suppose that its remaining hidden vertex, if present, is a positive wire. Write the two biased vertices as h , g h,g h , g , in their relative topological order, and the optional positive wire as u u u . Each possible position of u u u gives one of the forms below, and each form satisfies the stated conclusion. In the following forms, A , B A,B A , B are direct input sums, V V V is positive affine, { P 1 , P 2 } = { P − w , P } \{P_1,P_2\}=\{P-w,P\} { P 1 , P 2 } = { P − w , P } , and all lettered coefficients are positive nongain primitives or zero. If the wire is upstream, so that
u = V , h = ρ ( A + a u − P 1 ) , g = ρ ( B + b u + t h − P 2 ) , u=V,\qquad h=\rho(A+au-P_1),\qquad g=\rho(B+bu+th-P_2), u = V , h = ρ ( A + a u − P 1 ) , g = ρ ( B + b u + t h − P 2 ) , then the edge t t t is absent. If the wire is downstream, so that
h = ρ ( A − P 1 ) , g = ρ ( B + t h − P 2 ) , u = V + a h + b g , h=\rho(A-P_1),\qquad g=\rho(B+th-P_2),\qquad u=V+ah+bg, h = ρ ( A − P 1 ) , g = ρ ( B + t h − P 2 ) , u = V + ah + b g , then the edge t t t is absent. If the wire lies between h h h and g g g , so that
h = ρ ( A − P 1 ) , u = V + a h , g = ρ ( B + b u + c h − P 2 ) , h=\rho(A-P_1),\qquad u=V+ah,\qquad g=\rho(B+bu+ch-P_2), h = ρ ( A − P 1 ) , u = V + ah , g = ρ ( B + b u + c h − P 2 ) , then the edge c c c is absent and at least one of a , b a,b a , b is absent. The conclusions hold for actual edges globally, not just their contributions on the clean block.
Proof. Restrict to the clean query block. The two biased vertices h , g h,g h , g , from which the output gains originate, are active at the all-ones query vertex and every single deletion. Their inverse output equation equates their zeroth coefficients there. These points affinely span the query cube, so their affine query coefficients agree as Laurent polynomials. For each query coordinate j j j , write A j , B j , V j A_j,B_j,V_j A j , B j , V j for the corresponding zeroth direct coefficients. Each is zero or F F F times a power of two with exponent in { 0 , … , r − 1 } \{0,\ldots,r-1\} { 0 , … , r − 1 } .
For the upstream form the equality is
( 1 − t 0 ) ( A j + a 0 V j ) = B j + b 0 V j . (1-t^{0})(A_j+a^{0}V_j)=B_j+b^{0}V_j. ( 1 − t 0 ) ( A j + a 0 V j ) = B j + b 0 V j .
Some A j + a 0 V j A_j+a^{0}V_j A j + a 0 V j is nonzero; otherwise h h h is constant on the clean cube and only g g g remains an effective hinge. For that coordinate, finite or Γ \Gamma Γ values t 0 > 1 t^{0}>1 t 0 > 1 make the left side negative and the right side nonnegative at the prescribed specialization. If t = s 0 t=s_0 t = s 0 and a a a is finite or absent, its left side has a negative selector coefficient, whereas the right has only nonnegative coefficients. If a = Γ a=\Gamma a = Γ , the negative selector term forces A j = 0 A_j=0 A j = 0 . Then V j > 0 V_j>0 V j > 0 ; the degree-one term forces b = Γ b=\Gamma b = Γ , while the remaining scalar equation requires B j = − V j / q < 0 B_j=-V_j/q<0 B j = − V j / q < 0 . If a = s 0 a=s_0 a = s 0 , the negative square-selector term first forces V j = 0 V_j=0 V j = 0 , after which the negative selector multiple of A j A_j A j is unmatched. All positive types of t t t are excluded, so its edge is absent.
For the downstream form the query equation is
( 1 − t 0 ) A j = B j . (1-t^{0})A_j=B_j. ( 1 − t 0 ) A j = B j .
Some A j > 0 A_j>0 A j > 0 , since otherwise the first biased gate is identically zero and cannot be a gain source at a winner. Finite or Γ \Gamma Γ values of t t t give a negative left side. A selector gives an unmatched negative selector coefficient. Thus this t t t is absent too.
For the between form,
( 1 − b 0 a 0 − c 0 ) A j = B j + b 0 V j . (1-b^{0}a^{0}-c^{0})A_j=B_j+b^{0}V_j. ( 1 − b 0 a 0 − c 0 ) A j = B j + b 0 V j .
Choose A j > 0 A_j>0 A j > 0 , put R j = B j / A j R_j=B_j/A_j R j = B j / A j , T j = V j / A j T_j=V_j/A_j T j = V j / A j , and rearrange as
1 = c 0 + b 0 a 0 + R j + b 0 T j . 1=c^{0}+b^{0}a^{0}+R_j+b^{0}T_j. 1 = c 0 + b 0 a 0 + R j + b 0 T j .
Each nonzero ratio is a power of two at least 2 − ( r − 1 ) 2^{-(r-1)} 2 − ( r − 1 ) . Every term on the right is a nonnegative Laurent monomial. A nonfinite c c c has an exponent which cannot cancel, and a finite c c c is at least F > 1 F>1 F > 1 . Hence c c c is absent. If b 0 T j ≠ 0 b^{0}T_j\ne0 b 0 T j = 0 , a finite b b b gives a value at least F / 2 r − 1 > 1 F/2^{r-1}>1 F / 2 r − 1 > 1 , and a nonfinite b b b gives an uncancellable exponent. Thus b 0 T j = 0 b^{0}T_j=0 b 0 T j = 0 ; this does not assert T j = 0 T_j=0 T j = 0 when b b b is absent.
The product b 0 a 0 b^{0}a^{0} b 0 a 0 is either zero, larger than one if both factors are finite, of nonzero degree, or the balanced constant 1 / q 1/q 1/ q . The last possibility would require R j = 1 − 1 / q R_j=1-1/q R j = 1 − 1/ q . This positive number is neither zero nor a power of two: in lowest terms its denominator is the odd prime q q q , whereas a rational power of two has a power-of-two denominator. Therefore b 0 a 0 = 0 b^{0}a^{0}=0 b 0 a 0 = 0 , so at least one of the actual positive edges a , b a,b a , b is absent. Every excluded primitive has a nonzero zeroth coefficient, making these genuine edge absences.
When a = 0 a=0 a = 0 , the between wire is independent of h h h and can be reordered upstream. When b = 0 b=0 b = 0 , it can be reordered downstream. The graphs and parameter occurrences are unchanged by this topological reordering. Thus the full network has two parallel biased hinges with a positive wire upstream or downstream, or with that wire absent.
The Full-Cube Common Ramp
The graph restrictions have so far used one query block. Their actual edge absences hold globally, but equality of coefficient vectors on that block alone is insufficient for witness extraction. We now use every origin/unit test and the affinely spanning nonminimal winners of the full target. All positive source, anchor, query, control, Γ \Gamma Γ , and selector primitives are allowed in arbitrary input positions in this part of the argument.
Lemma A.18 (A local sharing comparison). Let U 0 ≠ 0 U_0\ne0 U 0 = 0 be a sum of at most two positive nongain leading primitives. Let a 0 , b 0 , d 0 , e 0 a_0,b_0,d_0,e_0 a 0 , b 0 , d 0 , e 0 be positive nongain leading primitives or zero. Then
( a 0 − b 0 ) U 0 = e 0 − d 0 . (a_0-b_0)U_0=e_0-d_0. ( a 0 − b 0 ) U 0 = e 0 − d 0 . forces a 0 = b 0 a_0=b_0 a 0 = b 0 and d 0 = e 0 d_0=e_0 d 0 = e 0 .
Proof. Write U 0 = F v + γ Γ + σ s 0 U_0=Fv+\gamma\Gamma+\sigma s_0 U 0 = F v + γ Γ + σ s 0 , with v ≥ 0 v\ge0 v ≥ 0 , γ , σ ∈ { 0 , 1 , 2 } \gamma,\sigma\in\{0,1,2\} γ , σ ∈ { 0 , 1 , 2 } , and at most two summands in all. A finite coefficient on the right is an F F F -multiple of magnitude at most P P P , while its coefficient relative to Γ \Gamma Γ or s 0 s_0 s 0 has absolute value at most one. If a 0 , b 0 a_0,b_0 a 0 , b 0 are finite or zero and unequal, their difference is a nonzero F F F -multiple. A finite summand in U 0 U_0 U 0 gives magnitude at least F 2 > P F^2>P F 2 > P , and a nonfinite summand gives coefficient magnitude at least F > 1 F>1 F > 1 . Thus this case requires equality.
For a 0 = Γ , b 0 = s 0 a_0=\Gamma,b_0=s_0 a 0 = Γ , b 0 = s 0 , the degree-two and degree-minus-two equations give γ = σ = 0 \gamma=\sigma=0 γ = σ = 0 . The remaining finite summand would have coefficient at least F F F at both nonfinite degrees, impossible on the right. For a 0 = Γ a_0=\Gamma a 0 = Γ and finite or zero b 0 b_0 b 0 , the degree-two equation first gives γ = 0 \gamma=0 γ = 0 . The degree-one equation then gives ν = 0 \nu=0 ν = 0 , since its coefficient F ν F\nu F ν is either zero or at least F F F . The remaining constant coefficient σ / q \sigma/q σ / q , with 1 ≤ σ ≤ 2 < q 1\leq\sigma\leq2<q 1 ≤ σ ≤ 2 < q , cannot be an F F F -multiple. For a 0 = s 0 a_0=s_0 a 0 = s 0 and finite or zero b 0 b_0 b 0 , use degree minus two, then degree minus one, then the same constant comparison γ / q \gamma/q γ / q . Reversing a 0 , b 0 a_0,b_0 a 0 , b 0 changes only the sign of both sides. Equal nonfinite types are already identical. These alternatives exhaust the types and prove a 0 = b 0 a_0=b_0 a 0 = b 0 ; the displayed equation then gives d 0 = e 0 d_0=e_0 d 0 = e 0 . □
This comparison is applied at an origin or unit point, where a positive affine wire has at most its bias and one input contribution. It also applies to a single wire input when global coefficient equality is established.
Lemma A.19 (Global common-ramp form). Suppose the finite negatives are the biases of two parallel hidden hinges, and their output edges are + D , − D +D,-D + D , − D . Suppose the optional third vertex is a positive wire upstream or downstream, with no path coupling the hinges. If the network equals f f f on the full Boolean cube, then it computes the function
D ( ρ ( S ( x ) − ( P − w ) ) − ρ ( S ( x ) − P ) ) , S ( x ) = a u ( x ) + ∑ j = 1 d b j x j . D\left(\rho\left(S(x)-(P-w)\right)-\rho\left(S(x)-P\right)\right), \qquad S(x)=au(x)+\sum_{j=1}^{d}b_jx_j. D ( ρ ( S ( x ) − ( P − w ) ) − ρ ( S ( x ) − P ) ) , S ( x ) = a u ( x ) + j = 1 ∑ d b j x j . Here u u u is an optional positive affine wire with one optional actual primitive per input and one optional actual primitive bias. The coefficient a a a is the weight of the sharing edge from u u u to each hinge, and b j b_j b j is the weight of the direct input edge from x j x_j x j to each hinge, as in Lemma 4.9. A nonzero a a a uses two identical-value actual occurrences, one for each sharing edge, and each nonzero b j b_j b j likewise uses two identical-value occurrences. All other output contributions vanish. If the wire was downstream, it can be omitted and a = 0 a=0 a = 0 .
Proof. Write h A h_A h A for the positive-gain hinge, h B h_B h B for the negative-gain hinge, and { P A , P B } = { P − w , P } \{P_A,P_B\}=\{P-w,P\} { P A , P B } = { P − w , P } for their bias magnitudes. The full target has all the properties of Lemma 4.8. We treat the two wire positions separately.
An upstream wire.
Let d A , d B ∈ R ≥ 0 d d_A,d_B\in\mathbb{R}_{\geq0}^{d} d A , d B ∈ R ≥ 0 d be the vectors of direct input coefficients, with absent edges represented by zeros. Write
u = v b + ∑ j = 1 d v j x j , h A = ρ ( a u + d A ⋅ x − P A ) , h B = ρ ( a ′ u + d B ⋅ x − P B ) , u=v_b+\sum_{j=1}^{d}v_jx_j,\qquad h_A=\rho(au+d_A\cdot x-P_A),\qquad h_B=\rho(a'u+d_B\cdot x-P_B), u = v b + j = 1 ∑ d v j x j , h A = ρ ( a u + d A ⋅ x − P A ) , h B = ρ ( a ′ u + d B ⋅ x − P B ) ,
where a a a and a ′ a' a ′ are the sharing weights into h A h_A h A and h B h_B h B , and N = D ( h A − h B ) + E ( x ) N=D(h_A-h_B)+E(x) N = D ( h A − h B ) + E ( x ) , where E E E is the nonnegative affine sum of ordinary output bias, skips and a possible wire output.
At an origin or unit point, the positive score of a hinge, its preactivation without its negative bias, is a direct primitive plus its sharing primitive times a sum of at most two primitives. We show that a hinge whose zeroth score, the w 0 w^0 w 0 -coefficient of this positive score, equals P P P at such a point is inactive. In that case every finite-times-finite product is too large, every unbalanced nonfinite degree cannot cancel, and any balanced contributions total k / q k/q k / q with 0 ≤ k ≤ 2 < q 0\leq k\leq2<q 0 ≤ k ≤ 2 < q . The latter cannot complete an integer F F F -multiple to P P P . The shared contribution must therefore be zero in full, and the only possible direct primitive with leading value P P P is the control primitive C C C . Its actual value P − 3 w / 2 P-3w/2 P − 3 w /2 is below both thresholds. Such a hinge is inactive.
At each of these target-zero points, a positive-only active hinge would make N > 0 N>0 N > 0 . A negative-only active hinge would require zeroth preactivation zero by the inverse equation, which the preceding comparison has just excluded. If both are active, inverse equality and Lemma A.18 identify their full scores at that point; if the wire value is zero, direct primitive equality suffices. Their full preactivation difference is then w w w or − w -w − w , so N = 1 + E N=1+E N = 1 + E or N = − 1 + E N=-1+E N = − 1 + E . The first cannot be zero. The second requires E = 1 E=1 E = 1 . At an origin or unit point every term of E E E is a positive primitive or a product of two positive primitives. Nonzero nonfinite degrees cannot cancel. A finite scalar term is at least F F F , while balanced products total at most 2 / q < 1 2/q<1 2/ q < 1 . Thus E E E cannot be one. Both hinges are consequently off at the origin and at every unit vector. The nonnegative affine function E E E vanishes there, so it is identically zero. Every coordinate has a strictly positive effective coefficient in the positive-gain score. Otherwise N = D ( h A − h B ) N=D(h_A-h_B) N = D ( h A − h B ) is nonincreasing in that coordinate, and monotonicity of f f f would make it inessential, contrary to Lemma 4.8. Let y = x + e j y=x+e_j y = x + e j be a nonminimal winner with x x x also winning. The positive hinge is active at x x x and its score strictly increases at y y y . If the negative hinge were off at y y y , it would also be off at x x x , making N ( y ) > N ( x ) = 1 N(y)>N(x)=1 N ( y ) > N ( x ) = 1 . Therefore both are active at every nonminimal winner.
Let A , B A,B A , B denote their affine preactivations, including biases. At those winners the formal output identity gives A − B = w A-B=w A − B = w . For each Laurent monomial the difference has an affine coefficient in x x x . The nonminimal winners affinely span the full input space, so A − B = w A-B=w A − B = w identically as a formal affine expression. If the wire bias is present, its leading constant identity forces a 0 = ( a ′ ) 0 a^0=(a')^0 a 0 = ( a ′ ) 0 , hence a = a ′ a=a' a = a ′ in full. If the wire bias is absent but some wire input v j v_j v j is present, use the leading coefficient identity
( a 0 − ( a ′ ) 0 ) v j 0 = ( d B ) j 0 − ( d A ) j 0 (a^0-(a')^0)v_j^0=(d_B)_j^0-(d_A)_j^0 ( a 0 − ( a ′ ) 0 ) v j 0 = ( d B ) j 0 − ( d A ) j 0
and Lemma A.18 to obtain the same full sharing equality. Once sharing agrees, every leading direct pair agrees, and Lemma A.1 identifies its full value. If the wire is absent or zero, direct equality alone suffices and it is omitted rather than retained as an illegal empty vertex.
The bias orientation is correct when the positive-gain hinge h A h_A h A carries the bias magnitude P − w P-w P − w , so that P A = P − w P_A=P-w P A = P − w and P B = P P_B=P P B = P . It is reversed when P A = P P_A=P P A = P and P B = P − w P_B=P-w P B = P − w . Reversed bias orientation would require ( a − a ′ ) v b = 2 w (a-a')v_b=2w ( a − a ′ ) v b = 2 w . For a present bias the sharing values are equal, and for an absent bias its product is zero. Either way this is impossible. Thus P A = P − w P_A=P-w P A = P − w , P B = P P_B=P P B = P , with a common full score S S S . The two distinct physical edges in every nonzero sharing or direct pair use distinct inventory occurrences, as claimed.
A downstream wire.
Write
h A = ρ ( d A ⋅ x − P A ) , h B = ρ ( d B ⋅ x − P B ) , u = V + θ A h A + θ B h B . h_A=\rho(d_A\cdot x-P_A),\qquad h_B=\rho(d_B\cdot x-P_B),\qquad u=V+\theta_Ah_A+\theta_Bh_B. h A = ρ ( d A ⋅ x − P A ) , h B = ρ ( d B ⋅ x − P B ) , u = V + θ A h A + θ B h B .
Let c u ≥ 0 c_u\ge0 c u ≥ 0 be the wire’s ordinary output coefficient, and put α = c u θ A ≥ 0 \alpha=c_u\theta_A\ge0 α = c u θ A ≥ 0 , β = c u θ B ≥ 0 \beta=c_u\theta_B\ge0 β = c u θ B ≥ 0 . Let c x ∈ R ≥ 0 d c_x\in\mathbb{R}_{\ge0}^d c x ∈ R ≥ 0 d list the remaining direct output coefficients, and let c b ≥ 0 c_b\ge0 c b ≥ 0 be the output bias. The remaining affine output is E 0 = c u V + c x ⋅ x + c b ≥ 0 E_0=c_uV+c_x\cdot x+c_b\ge0 E 0 = c u V + c x ⋅ x + c b ≥ 0 . Then
N = ( D + α ) h A − ( D − β ) h B + E 0 . N=(D+\alpha)h_A-(D-\beta)h_B+E_0. N = ( D + α ) h A − ( D − β ) h B + E 0 .
If D − β ≤ 0 D-\beta\le0 D − β ≤ 0 , this is a nonnegative combination of positive-input hinges and an affine function, hence supermodular by Lemma A.4. The target’s OR face excludes that case. Thus D − β > 0 D-\beta>0 D − β > 0 .
At origin both hinges are off, so E 0 ( 0 ) = 0 E_0(0)=0 E 0 ( 0 ) = 0 . At a unit vector the only primitive large enough to activate either direct hinge is Γ \Gamma Γ : every finite primitive and s 0 s_0 s 0 lies below both thresholds. If one is active, the inverse equation forces both active and their direct leading primitives equal. Both are Γ \Gamma Γ , consuming the two available occurrences. Their gain difference is 1 1 1 or − 1 -1 − 1 . The positive orientation cannot give zero; the negative orientation would require the total ordinary output at that unit to equal one.
No Γ \Gamma Γ remains for c u , θ A , θ B , V , c x c_u,\theta_A,\theta_B,V,c_x c u , θ A , θ B , V , c x . The ordinary output there is
c u V j + ( c x ) j + c u θ A ( Γ − P A ) + c u θ B ( Γ − P B ) . c_uV_j+(c_x)_j+c_u\theta_A(\Gamma-P_A)+c_u\theta_B(\Gamma-P_B). c u V j + ( c x ) j + c u θ A ( Γ − P A ) + c u θ B ( Γ − P B ) .
Its Γ 0 w 0 \Gamma^0w^0 Γ 0 w 0 -coefficient lies in ( F / q ) Z (F/q)\mathbb{Z} ( F / q ) Z . Direct finite values and finite products have factor F F F . A finite-selector product multiplied by the displayed Γ \Gamma Γ contributes F / q F/q F / q . Two selectors have nonzero degree after that multiplication, and the negative threshold parts still have factor F F F whenever their degree is zero. The origin test has removed all bias contributions. Since F > q F>q F > q , this coefficient cannot be one. Therefore both hinges are off at all units, and E 0 E_0 E 0 , a nonnegative affine function vanishing at origin and all units, is identically zero.
Every coefficient of d A d_A d A is strictly positive by essentiality: with it zero, the positive hinge is independent of that input while the negative hinge is nondecreasing, making N N N nonincreasing. At any winner h A > 0 h_A>0 h A > 0 because D − β > 0 D-\beta>0 D − β > 0 . For adjacent winning vertices y = x + e j y=x+e_j y = x + e j , an inactive h B ( y ) h_B(y) h B ( y ) would also make h B ( x ) = 0 h_B(x)=0 h B ( x ) = 0 and give
N ( y ) − N ( x ) = ( D + α ) ( d A ) j > 0 , N(y)-N(x)=(D+\alpha)(d_A)_j>0, N ( y ) − N ( x ) = ( D + α ) ( d A ) j > 0 ,
contrary to the equal labels one. Thus both hinges are active on the spanning nonminimal winners. Their inverse coefficients give ( d A 0 − d B 0 ) ⋅ y = 0 (d_A^0-d_B^0)\cdot y=0 ( d A 0 − d B 0 ) ⋅ y = 0 there. Affine spanning and full positive primitive identification yield d A = d B d_A=d_B d A = d B . Let S = d A ⋅ x S=d_A\cdot x S = d A ⋅ x , with every coefficient positive. If the bias orientation is reversed, so that P A = P P_A = P P A = P and P B = P − w P_B = P-w P B = P − w , the signed gain contribution on these winners is minus one. The ordinary term must equal two there, so the affine function
( α + β ) S − α P A − β P B (\alpha+\beta)S-\alpha P_A-\beta P_B ( α + β ) S − α P A − β P B
equals two on an affinely spanning set. It is therefore the constant two everywhere. Its slope coefficients force α + β = 0 \alpha+\beta=0 α + β = 0 , since every coefficient of S S S is positive. Nonnegativity gives α = β = 0 \alpha=\beta=0 α = β = 0 , making that function zero, a contradiction. Hence the bias orientation is correct.
At any nonminimal winner the signed gain contribution is exactly one, and both hinges are positive. The ordinary term α h A + β h B \alpha h_A+\beta h_B α h A + β h B must be zero, forcing α = β = 0 \alpha=\beta=0 α = β = 0 . Together with E 0 = 0 E_0=0 E 0 = 0 , this removes every ordinary contribution and gives the claimed common ramp with no sharing wire. A present downstream wire would have a positive output edge by reachability; all its incoming contributions then vanish, making it parameterless. It is consequently omitted in the legal common-ramp representation.
Completion of the Placement Argument
We can now combine the local restrictions with the full-cube rigidity statement. The logical order matters: coefficient comparisons on one query block force actual internal edges absent, and only then are full-coordinate scores identified.
Restatement of Lemma 4.9. Let W , f W,f W , f be the calibrated inventory and target above, with computational-node cap four and affine output. If any permitted network represents f f f on the Boolean cube, then some permitted network using a submultiset of the same occurrences has the form
D ( ρ ( S ( x ) − ( P − w ) ) − ρ ( S ( x ) − P ) ) , S ( x ) = a u ( x ) + ∑ j = 1 d b j x j . D\left(\rho\left(S(x)-(P-w)\right)-\rho\left(S(x)-P\right)\right), \qquad S(x)=au(x)+\sum_{j=1}^{d}b_jx_j. D ( ρ ( S ( x ) − ( P − w ) ) − ρ ( S ( x ) − P ) ) , S ( x ) = a u ( x ) + j = 1 ∑ d b j x j . The coefficients a , b 1 , … , b d , v b , v 1 , … , v d a,b_1,\ldots,b_d,v_b,v_1,\ldots,v_d a , b 1 , … , b d , v b , v 1 , … , v d are nonnegative reals; a zero denotes an absent parameter, and each nonzero coefficient is a single positive actual inventory primitive. The optional wire is u ( x ) = v b + ∑ j = 1 d v j x j u(x)=v_b+\sum_{j=1}^{d}v_jx_j u ( x ) = v b + ∑ j = 1 d v j x j , also called the sharing wire. It enters each hinge through one sharing edge of weight a a a , and each input x j x_j x j enters each hinge through one direct input edge of weight b j b_j b j . Every nonzero a a a uses two identical-value occurrences for its two sharing edges, and every nonzero b j b_j b j uses two identical-value occurrences for its two direct input edges. The two displayed biases and gains use their specified supplied occurrences. The hinges carry no other bias, so the score bias, the constant term of S S S , is a v b av_b a v b . The output has no ordinary output contribution, meaning no output bias, no skip edge and no edge from the wire to the output. A wire without incoming parameters, or with no sharing edge, is omitted.
Proof of Lemma 4.9. A permitted candidate has at most three hidden vertices. Choose its clean block from Lemma 4.7 and keep the actual activation patterns. The formal identities follow from Lemma 4.6 . If neither gain is used, Lemma A.2 excludes a winner. Lemma A.6 excludes both output-bias positions. Lemma A.10 excludes gain input edges and an unused gain, using legal pruning only in the case where + D +D + D is internal and − D -D − D is unavailable on the clean cube, which it reduces to Lemma A.8, or when pruning discards + D +D + D , to Lemma A.2 . Thus both gains have hidden sources or are hidden biases.
If both are internal edges, Lemma A.11 excludes them. If both are internal and one or both are biases, Lemma A.12 excludes them. If exactly one is an output edge, Lemmas A.13 and A.14 exclude its two sign orientations with every internal position of the other gain. The two gains must therefore be distinct hidden-to-output edges.
Lemma A.15 puts both finite negatives at distinct hidden biases. Both gains and both finite negatives are now placed, so the remaining hidden vertex, if present, receives only positive parameters and is a positive wire. Lemma A.16 puts the gains at those same biased vertices, rather than at the optional positive wire. The positive wire can be ordered upstream, downstream, or between the two biased vertices; incomparable and omitted cases are included by missing edges. Lemma A.17 removes every direct or wire-mediated coupling between the hinges. These are actual absent edges, so the conclusion holds on all coordinates, not merely on the chosen block. A between wire can be reordered upstream or downstream without changing an edge or replacing a numerical value.
The full-target properties of Lemma 4.8 now apply to Lemma A.19 . They force the correct bias/gain orientation, identical full sharing and direct coefficients, and zero ordinary output contribution. This is precisely the stated common-ramp form with actual paired occurrences. Any omitted zero contribution
only discards parameters and cannot increase the node count. The wire is retained only when it has an incoming parameter and a sharing path to output, so the resulting representation is legal. No step replaces a primitive by an effective numerical coefficient that would require a new inventory occurrence. □