Introduction
A stream changes, and so do the questions asked of it; often without foreknowledge. An inference is often the beginning of an analysis. A partition of the data, for example one obtained by normalized cuts, prompts questions about which directions support a group, where it is weakly connected to its surroundings, and how a new observation relates to it. An outlier score prompts related questions: which coordinates contributed, what values would be expected in their place, and what would change if a value were required to be larger than the current one? The answer to one question generates a need for further information and verification.
These questions differ in intent. Reading a prediction can hold the observations and reference values fixed. Testing a prescribed answer allows the other inferred values to respond. Scoring a possible insertion asks how the query would participate in the model. The distinctions are expressed through conditioning and normalization: what is held fixed, what may change, and against which population an answer is compared. Out-of-sample extensions of spectral methods make a related distinction between extending an answer and refitting its evidence [9 , 1 , 21 ] . Chaining inference routines can obscure these distinctions when each routine supplies its own neighborhood, scale, or reference population.
Decision forests have long served regression and classification [14 , 12 , 29 , 30 , 13 ] . Streaming tree learners also maintain fitted predictors as observations arrive [25 , 32 ] . These constructions reflect choices about which distinctions are relevant to a particular prediction task. However a directional explanation or a counterfactual likely asks for different distinctions which are discarded by a predictor. Geurts et al. [31 ] analyze how random splits on irrelevant attributes reduce the sample available for prediction, introducing bias that is not removed by ensemble averaging. Tree-specific explanation methods take a different approach, Lundberg et al. [50 ] develop efficient Shapley attributions. Using those attributions for another decision requires specifying the reference distribution and the intended use. Bhattacharjee et al. [10 ] , for example, show that small aggregate SHAP values over the data support need not justify discarding a feature, and derive guarantees using an extended distribution. These results reinforce the need to connect an explanation, or a question that is not known ex-ante, to original inference without constructing families of small estimators that fragment the sample or repeat work.
We believe that algorithms should explain their own work, without the need for depending on other algorithms for explanations. The observations, local geometry, and conditions used to obtain an answer should remain available to the questions that follow it. Relevance depends on the question: a distinction that contributes little to a prediction may still matter to a directional explanation or a counterfactual; and follow-up questions often arise on instances where prediction is ambiguous. Retaining that evidence also has an efficiency benefit. When an estimate and its directional contributions are accumulated in the same traversal, separate calls for the answer and its decomposition become one joint query and can be efficiently pipelined in a modern streaming system. For that pair, two tree traversals become one; richer interventions can reuse the geometry while specifying the additional computation they require.
Energy-based learning provides a uniform formulation of related inference tasks: observed variables are fixed and the remaining ones are inferred by minimizing a common energy [43 ] . For explanations, its terms must also be connected to the observations and directions retained by the algorithm, and the boundary conditions must specify what a query changes. Here the energy is derived from the local geometry represented by the trees. Its terms remain connected to the boxes, gaps, masses, and paths from which they were computed. Changing the question then changes the boundary conditions or the measurement of this same geometry. This gives prediction, explanation, and intervention a common calculation, with explicit evidence for each reading. We call this scalar energy the Hamiltonian; its noisy relaxation is described by Langevin dynamics with random thermal forces (i.e., Brownian motion) [41 , 44 ] (Pavliotis 2014). For the quadratic fields used here, conditioning the associated Gaussian law gives Kriging predictions and uncertainty [66 , 64 ] .
A random cut forest (RCF) provides one way to maintain this evidence as observations change over a dynamic data stream [36 ] . Its trees recursively cut data boxes, and insertions and deletions preserve their distribution; the estimators are kept apart from the maintained tree structures [33 ] . Scores, attributions, imputed values, and forecasts are already read from the same trees in OpenSearch (OpenSearch Project, 2026a,b; Random Cut Forest contributors, 2026). RCF also provides anomaly detection in Amazon Managed Service for Prometheus [4 ] .
This paper proposes the development of uniform, energy based analysis techniques that can function on dynamic data streams. It uses RCF both as an example that separates stream maintenance, as well as a class of trees that provides a wealth of properties that make the energy analysis over streams feasible. In particular this paper shows that many of the known uses of RCF can be reinterpreted as hitting functionals of random walks on the trees. We can then derive their energy, and use that energy to treat predictions, partitions, and coalitions together. The geometric identities and energy calculations are conditional on the chosen observations and reference sets; they require no particular sampling scheme. Maintaining the trees as their observations change is distinct from jointly adapting the reference and query sets. Axis-parallel cuts can adapt to affine structure: for width-proportional cuts we prove localization and stretch guarantees controlled by affine dimension, without estimating or rotating the support. This gives a coordinate-cut counterpart, under an affine-support assumption, to the intrinsic-dimension questions studied for random projection and other partition trees [20 , 71 ] . Theorem 9 transfers this geometric control to the averaged Hamiltonian.
Main Results We state the results informally, in the order of the sections that prove them.
Local cuts and affine dimension (Section 2). We prove localization and stretch guarantees controlled by affine dimension for axis-parallel RCF cuts, without identifying the support’s orientation or rotating the observations (Theorem 3). The guarantees persist above a bounded-noise scale (Corollary 1). Exact insertion/deletion consistency keeps this ℓ 1 \ell_1 ℓ 1 geometry maintainable as the sample changes (Proposition 1).
Hitting times and scores (§ 3 ). We give inverse-depth, range, and normalized displacement a common escape interpretation, identifying what each measurement reads from the same tree geometry (Theorem 5); collusive displacement averages the maximum mass-escape odds along a path (Corollary 2). Requiring a declared reference mass gives near-duplicate resilience without coalition search (Theorem 8). The same calculation retains 2 d 2d 2 d signed contributions that sum to the measurement (5) .
Sparse energy and diffusion (§ 4 ). We realize a dense pairwise agreement energy exactly by a sparse network on each tree (Theorem 12 ). Affine geometric control passes to the averaged Hamiltonian (Theorem 9) and bounds the statistical distinguishability of Gaussian mean shifts (39) . The free observation field’s merge contrasts relax at rates read directly from the hierarchy (55) ; grounded Langevin relaxation has stationary covariance equal to temperature times the Green’s function (45) .
Landmarks and Normalized Cuts (§ 5 ). We align different tree geometries on common references and compute forest affinity products and normalized-cut objectives by linear tree passes, without constructing a dense affinity matrix (Proposition 13 ). The pooled Kriging variance separates missing support, weak connection after first contact, and the cost of requiring common prediction weights across trees (Theorem 14 ); range escape is the initial rate of its support term.
Coalition games and strategic responses (§ 6 ). We construct a quadratic Hamiltonian and an exact potential game for arbitrary worths assigned to descendant coalitions. Its Green’s function is independent of the worths, and two tree passes find a pure equilibrium under participation clamps (Theorem 16 ). On the mass-resistance path, collusive displacement is the square of the largest relative change of the minimizing field under a one-clamp release, measured in the energy norm (79) .
Collective cut improvement and attribution (§ 6 ). We prove that a partition can resist every smaller move while a descendant-block move improves its normalized cut by an arbitrarily large factor (Theorem 19 ). The collusive cut proposes blocks by tree passes and tests a common move against the pooled objective (Theorem 20 ). Boundary prices attribute the fixed cut cost exactly to tree merges, to oriented faces together with unresolved leaf contributions, and to members (Theorem 17 ; (84) and (83) ).
Directional audits and perturbations (§ 7 ). We compute how directional geometry changes the minimized energy (Proposition 21 ) and sharply bound changes in harmonic predictions under conductance perturbations, with edge support and boundary values fixed (Theorem 22 ). Audit stiffness quantifies the energy cost of requiring a different answer at the probe ((92) ).
Measurements and an illustration (§ 8 ). Directional ratios ((101) ) compare a query’s expansion with a neighborhood of declared mass. A rotating star with noise coordinates compares these readings, the cuts, and their Nyström extension.
Local and Global Geometry
Approximate Nesting and Organization Each tree supplies an exact laminar hierarchy of descendant sets, with boxes nested along every ancestral chain. Inspecting both children or comparing siblings does not change this hierarchy. Query intent can nevertheless require a branching family of neighborhoods: masking coordinates or augmenting boxes by the query can produce overlapping boxes from different branches (§ 2.4 and § 2.5 ). Across trees, the resulting neighborhoods need not belong to one common nested hierarchy. We use approximately nested geometry for these combined readings; the per-tree escape identities and energy realizations remain exact. Common references align the trees for pooled predictions and cuts (§ 5 ).
In closing, the sections take multiple steps toward one goal. Section 2 discusses the affine geometry. § 3 explains what the forest already computes, as hitting times on its trees. § 4 and § 5 hold the boundary fixed and read the field it induces: predictions, their uncertainty, and the energy of a partition on common references. § 6 moves the boundary and lets the players respond. § 7 and § 8 keep the directional evidence, bound its response to perturbation, and compare the measurements on a rotating star. Diffusion ties these together: the multiscale structure of the trees becomes the multiscale structure of a dynamics, whose rates are read from the boxes and change only on an insertion or a deletion that affects the approximately nested geometry. Affine structure controls tree geometry; tree geometry controls energy; energy controls distinguishability of field responses. Hitting times, Laplacians, and Nyström extensions thus formalize the notion of diffusion dynamics in streaming data; estimates and objects of interest being formed and dissociated by clamping the energy of the operators, aided by the information geometry ([3 ] ) of approximately nested anisotropic local neighborhoods.
Local and Global Geometry
This section identifies the evidence a tree retains and the computations a query can reuse. We begin with the two sides of each coordinate, the boxes around the observations, and the observations visited by a query. We then define the cut family and explain why length allocation permits insertions and deletions while preserving the tree distribution. The same probabilities combine an answer and its directional contributions on the way back up the tree.
The global question is whether these local cuts preserve useful distances. We recall the original RCF stretch guarantee, then show how affine dimension controls localization and stretch without rotating the coordinates, including above a bounded-noise scale. The final two examples use these ingredients to explain a score, propose a replacement, and complete a time-series history. § 3 then reads the query probabilities as escape functionals.
Local Geometry and Query Traversals
Let S S S be a finite empirical sample in R d \mathbb{R}^d R d , allowing multiplicities or nonnegative weights. A rooted binary tree assigns each nonempty node v v v a sample S v S_v S v , mass m v m_v m v , and data box B v = bbox ( S v ) = ∏ j [ ℓ v j , u v j ] B_v = \operatorname{bbox}(S_v) = \prod_j[\ell_{vj},u_{vj}] B v = bbox ( S v ) = ∏ j [ ℓ v j , u v j ] . Write ψ = m r o o t \psi= m_{\mathrm{root}} ψ = m root for the total sample mass. These boxes are nested along ancestry. The routing cell, defined by the cuts, can contain a query outside its data box.
Definition 1 (Query-relative half dimensions). Fix a complete query x ∈ R d x \in\mathbb{R}^d x ∈ R d and write t + = max { t , 0 } t_+ = \max\{t,0\} t + = max { t , 0 } . The two half dimensions of coordinate j j j retain the nonnegative reaches
z x , j − ( y ) = ( x j − y j ) + , z x , j + ( y ) = ( y j − x j ) + . z^-_{x,j}(y) = (x_j-y_j)_+, \qquad z^+_{x,j}(y) = (y_j-x_j)_+. z x , j − ( y ) = ( x j − y j ) + , z x , j + ( y ) = ( y j − x j ) + . Their extents on S v S_v S v are the reaches from x x x to the faces of M v ( x ) = bbox ( B v ∪ { x } ) M_v(x) = \operatorname{bbox}(B_v \cup\{x\}) M v ( x ) = bbox ( B v ∪ { x }) :
a v j − = max y ∈ S v z x , j − ( y ) = ( x j − ℓ v j ) + , a v j + = max y ∈ S v z x , j + ( y ) = ( u v j − x j ) + . a^-_{vj} = \max_{y\in S_v} z^-_{x,j}(y) = (x_j-\ell_{vj})_+, \qquad a^+_{vj} = \max_{y\in S_v} z^+_{x,j}(y) = (u_{vj}-x_j)_+. a v j − = y ∈ S v max z x , j − ( y ) = ( x j − ℓ v j ) + , a v j + = y ∈ S v max z x , j + ( y ) = ( u v j − x j ) + . The 2 d 2d 2 d reaches retain both sides of the original d d d coordinates. For fixed x x x , the augmented boxes remain nested along ancestry.
Fix coordinate units. At a box B = ∏ j [ ℓ j , u j ] B = \prod_j[\ell_j,u_j] B = ∏ j [ ℓ j , u j ] , put r j = u j − ℓ j r_j = u_j-\ell_j r j = u j − ℓ j , R ( B ) = ∑ j r j R(B) = \sum_j r_j R ( B ) = ∑ j r j , and define the exterior face gaps
g j − = ( ℓ j − x j ) + , g j + = ( x j − u j ) + , G = ∑ j ( g j − + g j + ) . g^-_j = (\ell_j-x_j)_+, \qquad g^+_j = (x_j-u_j)_+, \qquad G = \sum_j(g^-_j+g^+_j). g j − = ( ℓ j − x j ) + , g j + = ( x j − u j ) + , G = j ∑ ( g j − + g j + ) .
Definition 2 (Query transcript). A transcript records the visited nodes, their ancestry, masses and boxes, including the origin of query-adjusted and sibling boxes. At a split on an observed coordinate, the query follows its child. A missing split coordinate admits both children; their projected boxes can overlap.
On the way down, the traversal identifies the relevant observations and geometry. On the way back up, it combines the child answers under the query’s declared branch probabilities. We call this upward combination a fold . It can carry a scalar estimate and its directional contributions together. The next subsection specifies the cut law and the probabilities used in this computation.
Cut Laws and Dynamic Maintenance
A cut law specifies which coordinate is cut and where its threshold falls. Repeating the law on the two children defines the distribution of a tree. We first give the family of cut laws, then ask which one can be maintained by inserting and deleting observations.
Definition 3 (The p p p -cut law and its tree distance). At a box with coordinate widths w j w_j w j , choose a positive-width axis with probability
π p , j ( w ) = w j p ∑ ℓ : w ℓ > 0 w ℓ p , w j > 0 , p ≥ 0 , (1) \pi_{p,j}(w) = \frac{w_j^p}{\sum_{\ell:w_\ell>0}w_\ell^p}, \qquad w_j>0,\quad p\ge0, \tag*{(1)} π p , j ( w ) = ∑ ℓ : w ℓ > 0 w ℓ p w j p , w j > 0 , p ≥ 0 , (1) then choose a uniform threshold on that axis and recurse. The exponent p = 0 p=0 p = 0 gives uniform coordinate allocation, as in isolation forests ([48 ] ); p = 1 p=1 p = 1 gives the length allocation of RCF; p = 2 p=2 p = 2 gives squared-width allocation. For p > 0 p>0 p > 0 put R p ( B ) = ( ∑ j w j p ) 1 / p R_p(B) = (\sum_j w_j^p)^{1/p} R p ( B ) = ( ∑ j w j p ) 1/ p , with R 1 ( B ) = R ( B ) R_1(B)=R(B) R 1 ( B ) = R ( B ) . For p ≥ 1 p\ge1 p ≥ 1 this is the ℓ p \ell_p ℓ p diameter of the box. For stored points and p > 0 p>0 p > 0 define d T , p ( a , b ) = R p ( B l c a ( a , b ) ) d_{T,p}(a,b)=R_p(B_{\mathrm{lca}(a,b)}) d T , p ( a , b ) = R p ( B lca ( a , b ) ) ; for p ≥ 1 p\ge1 p ≥ 1 write Δ p ( A ) = diam p ( A ) \Delta_p(A)=\operatorname{diam}_p(A) Δ p ( A ) = diam p ( A ) for their data diameter. Uniform allocation is an exponent-zero cut law, with no ℓ 0 \ell_0 ℓ 0 ruler intended.
Insertion and restriction. For p = 1 p=1 p = 1 , an auxiliary cut of M = bbox ( B ∪ { x } ) M=\operatorname{bbox}(B\cup\{x\}) M = bbox ( B ∪ { x }) separates the query on the exterior face intervals with probabilities
p j , ± ( x , B ) = g j ± R ( B ) + G , q ( x , B ) = G R ( B ) + G . (2) p_{j,\pm}(x,B)=\frac{g_j^{\pm}}{R(B)+G},\qquad q(x,B)=\frac{G}{R(B)+G}. \tag*{(2)} p j , ± ( x , B ) = R ( B ) + G g j ± , q ( x , B ) = R ( B ) + G G . (2)
The denominator is R ( M ) R(M) R ( M ) , since a j − + a j + = r j + g j − + g j + a_j^-+a_j^+=r_j+g_j^-+g_j^+ a j − + a j + = r j + g j − + g j + . At zero expanded range set these probabilities to zero. At a node v v v write G v = R ( M v ( x ) ) − R ( B v ) G_v=R(M_v(x))-R(B_v) G v = R ( M v ( x )) − R ( B v ) for its total exterior gap. If the cut does not isolate the query, its restriction to the old box has the original cut law. An existing cut of B B B therefore supplies the correct continuation of an insertion.
The restriction criterion also explains deletion. For any width-based coordinate probabilities π j ( w ) \pi_j(w) π j ( w ) and uniform thresholds, let a j ( B ) = π j ( w ( B ) ) / w j ( B ) a_j(B)=\pi_j(w(B))/w_j(B) a j ( B ) = π j ( w ( B )) / w j ( B ) be the cut-position density on a positive-width axis. For B ⊆ M B\subseteq M B ⊆ M ,
W j ( B , M ) = a j ( M ) a j ( B ) , E B W = ∑ j w j ( B ) a j ( M ) . (3) W_j(B,M)=\frac{a_j(M)}{a_j(B)},\qquad\mathbb{E}_B W=\sum_j w_j(B)a_j(M). \tag*{(3)} W j ( B , M ) = a j ( B ) a j ( M ) , E B W = j ∑ w j ( B ) a j ( M ) . (3)
The expectation is the probability that a cut of M M M lies in the old cut intervals. Conditional on those intervals, the old cut law is recovered exactly when W j W_j W j is constant there. Length allocation gives W j = R ( B ) / R ( M ) = 1 − q ( x , B ) W_j=R(B)/R(M)=1-q(x,B) W j = R ( B ) / R ( M ) = 1 − q ( x , B ) .
Proposition 1 (Length allocation and deletion). Suppose a recursive tree law chooses a positive-width coordinate j j j with probability π j ( w ) > 0 \pi_j(w)>0 π j ( w ) > 0 as a function of the widths w w w , then a uniform threshold on that coordinate. Deleting any stored location and contracting its parent preserves the construction law on the remaining sample if and only if
π j ( w ) = c j w j ∑ l c l w l ( c j > 0 ) \pi_j(w)=\frac{c_jw_j}{\sum_l c_lw_l}\qquad(c_j>0) π j ( w ) = ∑ l c l w l c j w j ( c j > 0 ) on the positive-width coordinates. The probability of separating two points of a box B B B is then their weighted ℓ 1 \ell_1 ℓ 1 distance divided by ∑ j c j R j ( B ) \sum_j c_jR_j(B) ∑ j c j R j ( B ) .
Proof. For these probabilities, rescaling the coordinates by c j c_j c j gives the ordinary random-cut construction, which is preserved by deletion [36 ] . For the converse, enlarge a box of widths w ′ w' w ′ by adding one point beyond just one face, producing widths w w w . A root cut outside the old box isolates the added point, leaving a fresh subtree of the old sample after contraction. A root cut inside the old box remains the root cut after deletion. The contracted root law is the mixture of these two laws. Whenever the old box admits a cut, the second has positive weight and must equal the original root law. By (3), the densities π j ( w ) / w j \pi_j(w)/w_j π j ( w ) / w j and π j ( w ′ ) / w j ′ \pi_j(w')/w'_j π j ( w ′ ) / w j ′ are proportional on the positive coordinates of w ′ w' w ′ . By enlarging one coordinate at a time to the coordinatewise maximum of any two width vectors, this fixes every density ratio on their shared positive coordinates. Comparing to an all-positive box gives the constants c j c_j c j , and normalization gives the stated probabilities. Integrating the cut density over the intervals between the two points gives the separation probability. □ \square □
Combining query responses. On a path v 0 , … , v h v_0,\ldots,v_h v 0 , … , v h from the root, put p i , j , ± = p j , ± ( x , B v i ) p_{i,j,\pm}=p_{j,\pm}(x,B_{v_i}) p i , j , ± = p j , ± ( x , B v i ) , q i = q ( x , B v i ) q_i=q(x,B_{v_i}) q i = q ( x , B v i ) , s 0 = 1 s_0=1 s 0 = 1 , and s i + 1 = s i ( 1 − q i ) s_{i+1}=s_i(1-q_i) s i + 1 = s i ( 1 − q i ) . The separation law, including a terminal state ∂ \partial ∂ , is
μ T , x { ( i , j , ± ) } = s i p j , ± ( x , B v i ) , μ T , x { ∂ } = s h + 1 . (4) \mu_{T,x}\{(i,j,\pm)\}=s_ip_{j,\pm}(x,B_{v_i}),\qquad\mu_{T,x}\{\partial\}=s_{h+1}. \tag*{(4)} μ T , x {( i , j , ± )} = s i p j , ± ( x , B v i ) , μ T , x { ∂ } = s h + 1 . (4)
Here s i s_i s i is survival to level i i i , whereas q i q_i q i is the local separation probability. For an unseen query taken to a singleton leaf the last hazard is one; a matching query or a truncated path retains a terminal atom with a declared payoff.
Proposition 2 (Transcript expectations). For a finite-valued scalar or vector payoff g g g , the separation law has total mass one and its expectation is computed on the way back up the path
H i = ∑ j , ± p i , j , ± g ( i , j , ± ) + ( 1 − q i ) H i + 1 . (5) H_i=\sum_{j,\pm}p_{i,j,\pm}g(i,j,\pm)+(1-q_i)H_{i+1}. \tag*{(5)} H i = j , ± ∑ p i , j , ± g ( i , j , ± ) + ( 1 − q i ) H i + 1 . (5) with H h + 1 = g ( ∂ ) H_{h+1}=g(\partial) H h + 1 = g ( ∂ ) and H 0 = ∫ g d μ T , x H_0=\int g\,\mathrm{d}\mu_{T,x} H 0 = ∫ g d μ T , x .
Proof. The level mass is s i q i = s i − s i + 1 s_iq_i=s_i-s_{i+1} s i q i = s i − s i + 1 . Summation telescopes, including the terminal atom. Expanding the recurrence gives the stated integral.
Reading another cut law. The maintained tree uses length allocation. A query can nevertheless estimate a reading under another cut law by likelihood weighting [67 ] . For a target coordinate law π j ( M ) \pi_j(M) π j ( M ) on the augmented box, its separating-face probabilities are
p j , ± t a r g e t ( x , B ) = π j ( M ) g j ± w j ( M ) , q t a r g e t = ∑ j , ± p j , ± t a r g e t . (6) p^{\mathrm{target}}_{j,\pm}(x,B)=\pi_j(M)\frac{g_j^\pm}{w_j(M)},\qquad q_{\mathrm{target}}=\sum_{j,\pm}p^{\mathrm{target}}_{j,\pm}. \tag*{(6)} p j , ± target ( x , B ) = π j ( M ) w j ( M ) g j ± , q target = j , ± ∑ p j , ± target . (6)
Flat coordinates contribute zero to (6) . For a nonseparating cut on axis j j j , reweight continuation by
W j ( B , M ) = w j ( B ) w j ( M ) π j ( M ) π 1 , j ( w ( B ) ) . (7) W_j(B,M)=\frac{w_j(B)}{w_j(M)}\frac{\pi_j(M)}{\pi_{1,j}(w(B))}. \tag*{(7)} W j ( B , M ) = w j ( M ) w j ( B ) π 1 , j ( w ( B )) π j ( M ) . (7)
This is the density ratio (3) , taking its numerator from the target law on M M M and its denominator from the maintained law on B B B . It requires positive base density where the target has nonseparating cuts. For a stored pair, let D T ( i , j ) D_T(i,j) D T ( i , j ) be its box distance at separation under a declared ruler, and P i j P_{ij} P ij its common path, including the separating cut. The corresponding change of law is
E p e − D T ( i , j ) / h = E 1 [ e − D T ( i , j ) / h ∏ v ∈ P i j π p , j ( v ) ( w v ) π 1 , j ( v ) ( w v ) ] . (8) \mathbb{E}_{p}e^{-D_T(i,j)/h}=\mathbb{E}_{1}\left[e^{-D_T(i,j)/h}\prod_{v\in P_{ij}}\frac{\pi_{p,j}(v)(w_v)}{\pi_{1,j}(v)(w_v)}\right]. \tag*{(8)} E p e − D T ( i , j ) / h = E 1 e − D T ( i , j ) / h v ∈ P ij ∏ π 1 , j ( v ) ( w v ) π p , j ( v ) ( w v ) . (8)
Uniform threshold densities cancel from the likelihood ratio. For a pair in the same leaf, stop the product at that leaf. Likelihood weighting estimates an average under another cut law; changing the ruler on fixed cuts (§ 4.2 ) needs no weights. Equation (8) is an expectation identity: finite weighted pair estimates need not form a positive semidefinite matrix. § 8.2 uses the query weights to compare cut laws with the same inverse-depth payoff.
Affine Dimension and Global Stretch
Having fixed the cut law, we ask which distances its boxes retain and how many levels are needed to localize observations. For coordinate vectors we write ℓ p \ell_p ℓ p ; L p ( μ ) L^p(\mu) L p ( μ ) denotes a function space with a declared measure. We begin with the original RCF stretch argument, then show how affine dimension gives control independently of the number of coordinates.
The original stretch identity. At a common cell of a stored pair, put h j = ∣ a j − b j ∣ h_j=|a_j-b_j| h j = ∣ a j − b j ∣ and δ = ∥ a − b ∥ p \delta=\lVert a-b\rVert_p δ = ∥ a − b ∥ p . For p ≥ 1 p\ge1 p ≥ 1 , its separation probability q p q_p q p satisfies
R p ( B ) q p = ∑ j : w j > 0 w j p − 1 h j R p ( B ) p − 1 ≤ δ . (9) R_p(B)q_p=\frac{\sum_{j:w_j>0}w_j^{p-1}h_j}{R_p(B)^{p-1}}\le\delta. \tag*{(9)} R p ( B ) q p = R p ( B ) p − 1 ∑ j : w j > 0 w j p − 1 h j ≤ δ . (9)
At p = 1 p=1 p = 1 this is equality. Consequently, with τ a b \tau_{ab} τ ab counting the common cuts through separation, the original RCF identity [36 Supplement, Lemma 7] is
E 1 d T , 1 ( a , b ) ∥ a − b ∥ 1 = E 1 τ a b . (10) \frac{\mathbb{E}_1 d_{T,1}(a,b)}{\lVert a-b\rVert_1}=\mathbb{E}_1\tau_{ab}. \tag*{(10)} ∥ a − b ∥ 1 E 1 d T , 1 ( a , b ) = E 1 τ ab . (10)
Writing R 0 = R ( B r o o t ) R_0=R(B_{\mathrm{root}}) R 0 = R ( B root ) and δ = ∥ a − b ∥ 1 \delta=\lVert a-b\rVert_1 δ = ∥ a − b ∥ 1 , the ambient bound from the same argument is
E 1 d T , 1 ( a , b ) δ = O ( d [ 1 + log ( R 0 / δ ) ] ) . (11) \frac{\mathbb{E}_1 d_{T,1}(a,b)}{\delta}=O\bigl(d[1+\log(R_0/\delta)]\bigr). \tag*{(11)} δ E 1 d T , 1 ( a , b ) = O ( d [ 1 + log ( R 0 / δ )] ) . (11)
For example, following the child containing a a a , a uniform cut removes at least one quarter of the selected width in expectation. Thus the expected range decreases by at least ∑ j w j 2 / ( 4 R ) ≥ R / ( 4 d ) \sum_j w_j^2/(4R)\ge R/(4d) ∑ j w j 2 / ( 4 R ) ≥ R / ( 4 d ) at each step. Once its range is less than δ \delta δ the pair has separated, giving the bound.
Control by affine dimension. The following range lemma replaces ambient coordinate counting with a property of the support. The localization theorem combines it with the net and cluster-pair argument of Dasgupta and Freund [20 ] , using finite-dimensional L p L^p L p geometry [45 , 18 ] .
Lemma 1 (Range of an affine set). Let S ⊂ R d S\subset\mathbb{R}^d S ⊂ R d lie in an affine subspace of dimension r ≥ 1 r\ge1 r ≥ 1 , fix 1 ≤ p < ∞ 1\le p<\infty 1 ≤ p < ∞ , and put Λ r , p = r max { 1 , p / 2 } \Lambda_{r,p}=r^{\max\{1,p/2\}} Λ r , p = r m a x { 1 , p /2 } . Every nonempty A ⊆ S A\subseteq S A ⊆ S satisfies
R p ( bbox ( A ) ) p ≤ Λ r , p Δ p ( A ) p . (12) R_p(\operatorname{bbox}(A))^p\le\Lambda_{r,p}\Delta_p(A)^p. \tag*{(12)} R p ( bbox ( A ) ) p ≤ Λ r , p Δ p ( A ) p . (12) In particular Λ r , p = r \Lambda_{r,p}=r Λ r , p = r for 1 ≤ p ≤ 2 1\le p\le2 1 ≤ p ≤ 2 . At p = 1 p=1 p = 1 the summed widths of a bounding box are at most r r r times its ℓ 1 \ell_1 ℓ 1 diameter, with equality for the cross-polytope A = { ± e 1 , … , ± e r } A=\{\pm e_1,\ldots,\pm e_r\} A = { ± e 1 , … , ± e r } .
Proof. Lewis’s normalization of the r r r -dimensional direction space gives a parametrization z j = λ j 1 / p ⟨ u j , t ⟩ z_j=\lambda_j^{1/p}\langle u_j,t\rangle z j = λ j 1/ p ⟨ u j , t ⟩ , where ∥ u j ∥ 2 = 1 \lVert u_j\rVert_2=1 ∥ u j ∥ 2 = 1 and ∑ j λ j u j u j T = I r \sum_j\lambda_j u_j u_j^\mathsf{T}=I_r ∑ j λ j u j u j T = I r ; coordinates that vanish on the space can be omitted [45 , 18 ] . Taking traces gives ∑ j λ j = r \sum_j\lambda_j=r ∑ j λ j = r . On ∥ t ∥ 2 = 1 \lVert t\rVert_2=1 ∥ t ∥ 2 = 1 , for p ≤ 2 p\le2 p ≤ 2 the sum ∑ j λ j ∣ ⟨ u j , t ⟩ ∣ p \sum_j\lambda_j|\langle u_j,t\rangle|^p ∑ j λ j ∣ ⟨ u j , t ⟩ ∣ p is at least one; for p ≥ 2 p\ge2 p ≥ 2 , the power-mean inequality bounds it below by r 1 − p / 2 r^{1-p/2} r 1 − p /2 . Thus ∥ t ∥ 2 ≤ r ( 1 / 2 − 1 / p ) + ∥ z ∥ p \lVert t\rVert_2\le r^{(1/2-1/p)_+}\lVert z\rVert_p ∥ t ∥ 2 ≤ r ( 1/2 − 1/ p ) + ∥ z ∥ p . Each coordinate width of A A A is at most λ j 1 / p r ( 1 / 2 − 1 / p ) + Δ p ( A ) \lambda_j^{1/p}r^{(1/2-1/p)_+}\Delta_p(A) λ j 1/ p r ( 1/2 − 1/ p ) + Δ p ( A ) . Summing their p p p th powers proves (12). For the equality, the cross-polytope has every width equal to 2 and ℓ 1 \ell_1 ℓ 1 diameter 2. At p = 2 p=2 p = 2 , the λ j \lambda_j λ j are the coordinate leverage scores of the subspace and sum to r r r .
Theorem 3 (Affine dimension, localization, and stretch). Fix 1 ≤ p < ∞ 1\le p<\infty 1 ≤ p < ∞ . Let S ⊂ R d S\subset\mathbb{R}^d S ⊂ R d have at least two distinct locations and lie in an affine subspace of dimension r ≥ 1 r\ge1 r ≥ 1 . Put Δ = Δ p ( S ) \Delta=\Delta_p(S) Δ = Δ p ( S ) and, with Λ r , p \Lambda_{r,p} Λ r , p of Lemma 1,
κ r , p = 4 p Λ r , p , K r , p ( η ) = ⌈ κ r , p [ 2 r log ( 1 + 16 κ r , p ) + log ( 1 / η ) ] ⌉ . \kappa_{r,p}=4^p\Lambda_{r,p},\qquad K_{r,p}(\eta)=\left\lceil\kappa_{r,p}\left[2r\log(1+16\kappa_{r,p})+\log(1/\eta)\right]\right\rceil. κ r , p = 4 p Λ r , p , K r , p ( η ) = ⌈ κ r , p [ 2 r log ( 1 + 16 κ r , p ) + log ( 1/ η ) ] ⌉ . In the random tree of the p p p -cut law, with probability at least 1 − η 1-\eta 1 − η , every depth-K r , p ( η ) K_{r,p}(\eta) K r , p ( η ) descendant has data diameter at most Δ / 2 \Delta/2 Δ/2 , for 0 < η < 1 0<\eta<1 0 < η < 1 . Terminal leaves have diameter zero. Moreover, for a , b ∈ S a,b\in S a , b ∈ S with δ = ∥ a − b ∥ p > 0 \delta=\lVert a-b\rVert_p>0 δ = ∥ a − b ∥ p > 0 ,
d T , p ( a , b ) ≥ δ , E d T , p ( a , b ) δ ≤ 2 K r , p ( 1 / 2 ) ( 1 + ⌈ log 2 ( Δ / δ ) ⌉ ) . (13) d_{T,p}(a,b)\ge\delta,\qquad\frac{\mathbb{E}d_{T,p}(a,b)}{\delta}\le2K_{r,p}(1/2)\left(1+\left\lceil\log_2(\Delta/\delta)\right\rceil\right). \tag*{(13)} d T , p ( a , b ) ≥ δ , δ E d T , p ( a , b ) ≤ 2 K r , p ( 1/2 ) ( 1 + ⌈ log 2 ( Δ/ δ ) ⌉ ) . (13) For fixed p p p the constant-probability depth is O p ( r 1 + max { 1 , p / 2 } log ( 1 + r ) ) O_p(r^{1+\max\{1,p/2\}}\log(1+r)) O p ( r 1 + m a x { 1 , p /2 } log ( 1 + r )) , independently of ambient dimension. At p = 1 p=1 p = 1 this reduces to K r , 1 ( η ) = ⌈ 4 r [ 2 r log ( 1 + 64 r ) + log ( 1 / η ) ] ⌉ K_{r,1}(\eta)=\lceil4r[2r\log(1+64r)+\log(1/\eta)]\rceil K r , 1 ( η ) = ⌈ 4 r [ 2 r log ( 1 + 64 r ) + log ( 1/ η )]⌉ . The construction need not identify or rotate the support.
Proof. Write Λ = Λ r , p \Lambda= \Lambda_{r,p} Λ = Λ r , p and κ = κ r , p \kappa= \kappa_{r,p} κ = κ r , p ; the geometric input is (12) . Take a maximal a 0 a_0 a 0 -separated net in S S S , with a 0 = Δ / ( 8 κ ) a_0 = \Delta/(8\kappa) a 0 = Δ/ ( 8 κ ) , and assign each point to a covering ball. Volume packing in the r r r -dimensional normed support gives at most M = ( 1 + 16 κ ) r M = (1 + 16\kappa)^r M = ( 1 + 16 κ ) r clusters C i C_i C i , each containing its center c i c_i c i , with diameter at most 2 a 0 2a_0 2 a 0 . If points of C i , C j C_i,C_j C i , C j can be more than Δ / 2 \Delta/2 Δ/2 apart, their centers are more than Δ / 2 − 2 a 0 \Delta/2 - 2a_0 Δ/2 − 2 a 0 apart. Let g j g_j g j and o j o_j o j be the coordinate gap and overlap lengths of their two boxes. Equation (12) gives
∥ g ∥ p ≥ Δ / 2 − ( 2 + 4 Λ 1 / p ) a 0 ≥ 5 Δ / 16 , ∥ o ∥ p ≤ 2 Λ 1 / p a 0 . \lVert g\rVert_p \ge\Delta/2 - (2 + 4\Lambda^{1/p})a_0 \ge5\Delta/16,\qquad\lVert o\rVert_p \le2\Lambda^{1/p}a_0. ∥ g ∥ p ≥ Δ/2 − ( 2 + 4 Λ 1/ p ) a 0 ≥ 5Δ/16 , ∥ o ∥ p ≤ 2 Λ 1/ p a 0 .
Call a cell mixed when it contains points from both clusters. At a mixed cell with widths w w w , a cut in a global gap interval leaves no mixed child. Two mixed children require a cut in an overlap interval. The corresponding cut densities are w j p − 1 / R p ( B ) p w_j^{p-1}/R_p(B)^p w j p − 1 / R p ( B ) p . Since w j ≥ g j w_j \ge g_j w j ≥ g j and R p ( B ) ≤ Λ 1 / p Δ R_p(B) \le\Lambda^{1/p}\Delta R p ( B ) ≤ Λ 1/ p Δ , Hölder’s inequality gives
∑ j : w j > 0 w j p − 1 ( g j − o j ) ≥ [ ( 5 / 16 ) p − 4 − ( p + 1 ) ] Δ p ≥ 4 − p Δ p . \sum_{j:w_j>0} w_j^{p-1}(g_j-o_j) \ge\left[(5/16)^p - 4^{-(p+1)}\right]\Delta^p \ge4^{-p}\Delta^p. j : w j > 0 ∑ w j p − 1 ( g j − o j ) ≥ [ ( 5/16 ) p − 4 − ( p + 1 ) ] Δ p ≥ 4 − p Δ p .
The expected number of mixed children is therefore at most 1 − 1 / κ 1 - 1/\kappa 1 − 1/ κ . The probability this pair has a mixed cell at depth k k k is at most e − k / κ e^{-k/\kappa} e − k / κ . A union bound over the M 2 M^2 M 2 cluster pairs proves localization.
Conditional on previous cuts, the same result applies to the pair’s current common cell. In a block of K r , p ( 1 / 2 ) K_{r,p}(1/2) K r , p ( 1/2 ) levels, with conditional probability at least 1 / 2 1/2 1/2 , the pair separates or that cell’s data diameter halves. After 1 + ⌈ log 2 ( Δ / δ ) ⌉ 1 + \lceil\log_2(\Delta/\delta)\rceil 1 + ⌈ log 2 ( Δ/ δ )⌉ successful blocks it must separate, so the expected number of shared cuts is at most twice that number of blocks. If h j = ∣ a j − b j ∣ h_j = |a_j-b_j| h j = ∣ a j − b j ∣ , the separation probability q q q at a common cell satisfies
R p ( B ) q = ∑ j : w j > 0 w j p − 1 h j R p ( B ) p − 1 ≤ ∥ a − b ∥ p = δ . R_p(B)q = \frac{\sum_{j:w_j>0} w_j^{p-1}h_j}{R_p(B)^{p-1}} \le\lVert a-b\rVert_p = \delta. R p ( B ) q = R p ( B ) p − 1 ∑ j : w j > 0 w j p − 1 h j ≤ ∥ a − b ∥ p = δ .
Summing conditional separation payoffs proves the upper bound in (13); domination follows because the separating box contains both points. At p = 1 p=1 p = 1 , the last inequality is the exact separation identity of Guha et al. [36 ] .
For p = 1 p=1 p = 1 on affine data, both (11) and (13) apply, so one can use their minimum. The ambient estimate can be smaller when r r r is comparable to d d d ; the affine estimate removes ambient dimension when many coordinates describe a low-dimensional support.
The range of $p. The theorem needs no assumption beyond the affine support. Domination holds for every p p p because the separating box contains both points, and the upper bound uses only Hölder’s inequality and Lemma 1. For $1 \le p \le 2$ the lemma’s constant is r r r , so the depth is O ( 4 p r 2 log ( 1 + r ) ) O(4^p r^2\log(1+r)) O ( 4 p r 2 log ( 1 + r )) throughout that range; for p > 2 p>2 p > 2 the lemma’s factor is r p / 2 r^{p/2} r p /2 , giving the depth stated in the theorem. What singles out p = 1 p=1 p = 1 is maintenance: by Proposition 1 it is the only exponent whose trees survive deletion, and the other laws are simulated from it by likelihood weights (§ 2.2 ).
What changes below one. Positive powers suppress vanishing coordinate widths, but the stretch estimate needs p ≥ 1 p \ge1 p ≥ 1 . For 0 < p < 1 0 < p < 1 0 < p < 1 , take S = { a = ( 0 , 0 ) , b = ( 0 , ε ) , c = ( 1 , 0 ) } S = \{a=(0,0), b=(0,\varepsilon), c=(1,0)\} S = { a = ( 0 , 0 ) , b = ( 0 , ε ) , c = ( 1 , 0 )} , with 0 < ε < 1 0 < \varepsilon< 1 0 < ε < 1 . The root separates a , b a,b a , b with probability ε p / ( 1 + ε p ) \varepsilon^p/(1+\varepsilon^p) ε p / ( 1 + ε p ) and then assigns the ruler R p ( B r o o t ) = ( 1 + ε p ) 1 / p ≥ 1 R_p(B_{\mathrm{root}})=(1+\varepsilon^p)^{1/p}\ge1 R p ( B root ) = ( 1 + ε p ) 1/ p ≥ 1 . Their own separation is ∥ a − b ∥ p = ε \lVert a-b\rVert_p=\varepsilon ∥ a − b ∥ p = ε , so
E p d T , p ( a , b ) ε ≥ ε p − 1 1 + ε p . (14) \frac{\mathbb{E}_p d_{\mathcal{T},p}(a,b)}{\varepsilon} \ge\frac{\varepsilon^{p-1}}{1+\varepsilon^p}. \tag*{(14)} ε E p d T , p ( a , b ) ≥ 1 + ε p ε p − 1 . (14)
This grows polynomially as ε ↓ 0 \varepsilon\downarrow0 ε ↓ 0 , already on three points in affine dimension two. Thus the logarithmic scale-ratio bound does not extend to these exponents. Here R p R_p R p is only a formal box ruler; ℓ p \ell_p ℓ p is not a norm below one.
Corollary 1 (Affine localization above a noise scale). Suppose each x ∈ S x \in S x ∈ S has a fixed representation x = s x + n x x=s_x+n_x x = s x + n x , where the s x s_x s x lie in one r r r -dimensional affine subspace, r ≥ 1 r \ge1 r ≥ 1 , and ∣ ( n x ) j ∣ ≤ σ j \lvert(n_x)_j\rvert\le\sigma_j ∣( n x ) j ∣ ≤ σ j . Put W = ∑ j σ j W=\sum_j \sigma_j W = ∑ j σ j . Every nonempty A ⊆ S A \subseteq S A ⊆ S satisfies
R ( bbox A ) ≤ r Δ 1 ( A ) + 2 ( r + 1 ) W . (15) R(\operatorname{bbox} A) \le r\Delta_1(A)+2(r+1)W. \tag*{(15)} R ( bbox A ) ≤ r Δ 1 ( A ) + 2 ( r + 1 ) W . (15) For ordinary width-proportional cuts, let Δ = Δ 1 ( S ) > 0 \Delta=\Delta_1(S)>0 Δ = Δ 1 ( S ) > 0 and suppose Δ ≥ 16 W \Delta\ge16W Δ ≥ 16 W . For 0 < η < 1 0<\eta<1 0 < η < 1 , with
K r n o i s e ( η ) = ⌈ 10 r [ 2 r log ( 1 + 72 r ) + log ( 1 / η ) ] ⌉ , K^{\mathrm{noise}}_r(\eta)=\left\lceil10r\left[2r\log(1+72r)+\log(1/\eta)\right]\right\rceil, K r noise ( η ) = ⌈ 10 r [ 2 r log ( 1 + 72 r ) + log ( 1/ η ) ] ⌉ , every depth-K r n o i s e ( η ) K^{\mathrm{noise}}_r(\eta) K r noise ( η ) descendant has data diameter at most Δ / 2 \Delta/2 Δ/2 with probability at least 1 − η 1-\eta 1 − η ; terminal leaves have diameter zero. Moreover, for a stored pair with δ = ∥ a − b ∥ 1 > 0 \delta=\lVert a-b\rVert_1>0 δ = ∥ a − b ∥ 1 > 0 and δ ≥ 16 W \delta\ge16W δ ≥ 16 W ,
d T , 1 ( a , b ) ≥ δ , E d T , 1 ( a , b ) δ ≤ 2 K r n o i s e ( 1 / 2 ) [ 1 + ⌈ log 2 ( Δ / δ ) ⌉ ] . (16) d_{T,1}(a,b)\ge\delta,\qquad\frac{\mathbb{E}d_{T,1}(a,b)}{\delta}\le2K^{\mathrm{noise}}_r(1/2)\left[1+\left\lceil\log_2(\Delta/\delta)\right\rceil\right]. \tag*{(16)} d T , 1 ( a , b ) ≥ δ , δ E d T , 1 ( a , b ) ≤ 2 K r noise ( 1/2 ) [ 1 + ⌈ log 2 ( Δ/ δ ) ⌉ ] . (16)
Proof. For A 0 = { s x : x ∈ A } A_0=\{s_x:x\in A\} A 0 = { s x : x ∈ A } , the coordinate noise adds at most 2 W 2W 2 W to the summed widths, and Δ 1 ( A 0 ) ≤ Δ 1 ( A ) + 2 W \Delta_1(A_0)\le\Delta_1(A)+2W Δ 1 ( A 0 ) ≤ Δ 1 ( A ) + 2 W . Lemma 1 proves (15). The noisy observations need not have a low-dimensional affine span, so cover their affine parts instead. These have diameter at most Δ + 2 W ≤ 9 Δ / 8 \Delta+2W\le9\Delta/8 Δ + 2 W ≤ 9Δ/8 . A maximal a 0 a_0 a 0 -separated net with a 0 = Δ / ( 32 r ) a_0=\Delta/(32r) a 0 = Δ/ ( 32 r ) has at most M = ( 1 + 72 r ) r M=(1+72r)^r M = ( 1 + 72 r ) r centers. Assign observations through their affine parts to covering centers. Each resulting cluster C i C_i C i satisfies
R i : = R ( bbox C i ) ≤ 2 r a 0 + 2 W ≤ 3 Δ / 16. R_i:=R(\operatorname{bbox} C_i)\le2ra_0+2W\le3\Delta/16. R i := R ( bbox C i ) ≤ 2 r a 0 + 2 W ≤ 3Δ/16.
For a pair of clusters containing observations more than Δ / 2 \Delta/2 Δ/2 apart, let g j , o j g_j,o_j g j , o j be the coordinate gap and overlap lengths of their boxes. The interval identity for hull width gives
∑ j ( g j − o j ) = R ( bbox ( C i ∪ C l ) ) − R i − R l > Δ / 8. \sum_j(g_j-o_j)=R(\operatorname{bbox}(C_i\cup C_l))-R_i-R_l>\Delta/8. j ∑ ( g j − o j ) = R ( bbox ( C i ∪ C l )) − R i − R l > Δ/8.
At a cell containing both cluster types, a cut in a global gap leaves no mixed child; two mixed children require a cut in a global overlap. Since its data box has R ( B ) ≤ r Δ + 2 ( r + 1 ) W ≤ 5 r Δ / 4 R(B)\le r\Delta+2(r+1)W\le5r\Delta/4 R ( B ) ≤ r Δ + 2 ( r + 1 ) W ≤ 5 r Δ/4 , the expected number of mixed children is at most 1 − 1 / ( 10 r ) 1-1/(10r) 1 − 1/ ( 10 r ) . The probability a fixed pair has any mixed cell at depth k k k is at most e − k / ( 10 r ) e^{-k/(10r)} e − k / ( 10 r ) . A union bound over M 2 M^2 M 2 pairs proves the diameter claim. A single cluster has diameter at most 3 Δ / 16 3\Delta/16 3Δ/16 .
Conditional on earlier cuts, the same argument applies to any current common cell while its diameter is at least δ ≥ 16 W \delta\ge16W δ ≥ 16 W . Each block of K r n o i s e ( 1 / 2 ) K^{\mathrm{noise}}_r(1/2) K r noise ( 1/2 ) levels has conditional probability at least 1 / 2 1/2 1/2 of separating the pair or halving that diameter. After 1 + ⌈ log 2 ( Δ / δ ) ⌉ 1+\lceil\log_2(\Delta/\delta)\rceil 1 + ⌈ log 2 ( Δ/ δ )⌉ successes the pair must separate. The exact separation probability at a common cell is δ / R ( B ) \delta/R(B) δ / R ( B ) , so the expected separating range divided by δ \delta δ equals the expected number of shared cuts, including the separating cut. The block bound proves (16); the separating box contains both points, proving domination. □ \square □
Theorem 9 transfers this affine control, including the noise floor, to the averaged Hamiltonian. Its Fisher-metric interpretation in § 4 bounds the statistical distinguishability of Gaussian mean shifts. Thus O ( r 2 log ( 1 + r ) ) O(r^2\log(1+r)) O ( r 2 log ( 1 + r )) levels give constant-probability contraction above a conservative noise scale, without identifying or rotating the support. Below that scale the noise may itself be high dimensional. § 8.2 uses a similar perturbation model when it pads the star with bounded noise coordinates.
Uniform coordinate allocation. Uniform coordinate allocation, the exponent-zero cut law of isolation forests, can require ambient-dimension depth even on affine dimension two. Fix 0 < ε < 1 0 < \varepsilon< 1 0 < ε < 1 and take
S m = { ( a , ε t / m , … , ε t / m ) : a ∈ { 0 , 1 } , t ∈ { 0 , 2 − m , … , 2 − 1 , 1 } } ⊂ R m + 1 . S_m=\{(a,\varepsilon t/m,\ldots,\varepsilon t/m):a\in\{0,1\},\ t\in\{0,2^{-m},\ldots,2^{-1},1\}\}\subset\mathbb{R}^{m+1}. S m = {( a , εt / m , … , εt / m ) : a ∈ { 0 , 1 } , t ∈ { 0 , 2 − m , … , 2 − 1 , 1 }} ⊂ R m + 1 .
Its ℓ 1 \ell_1 ℓ 1 diameter is 1 + ε 1+\varepsilon 1 + ε . The two points with t = 0 t=0 t = 0 and opposite a a a remain together until a cut on the first coordinate. Along their common branch, while t t t still varies, that coordinate is selected with probability 1 / ( m + 1 ) 1/(m+1) 1/ ( m + 1 ) . Every other cut replaces the maximum 2 − j 2^{-j} 2 − j by 2 − ( j + K ) 2^{-(j+K)} 2 − ( j + K ) , or zero if j + K > m j+K>m j + K > m , where P ( K = k ) = 2 − k \mathbb{P}(K=k)=2^{-k} P ( K = k ) = 2 − k and E K = 2 \mathbb{E}K=2 E K = 2 . After k k k cuts, a union bound and Markov’s inequality give probability at least 1 − k / ( m + 1 ) − 2 k / m 1-k/(m+1)-2k/m 1 − k / ( m + 1 ) − 2 k / m that neither a first-coordinate cut nor collapse of the t t t range has occurred. For k ≤ m / 12 k\le m/12 k ≤ m /12 this is at least 3 / 4 3/4 3/4 . The common cell then has diameter at least one, greater than half the original diameter. Thus no bound depending only on affine dimension can hold for this uniform cut law.
Affine dimension versus doubling dimension. Affine dimension controls the coordinate ranges in (12) . Bounded doubling dimension alone cannot replace it: even at p = 1 p=1 p = 1 a metric of doubling dimension at most two can have growing expected stretch under the coordinate cut law. For example, take the n = 2 m n=2^m n = 2 m leaves of a complete binary hierarchy and one coordinate for each nonroot vertex v v v . With ρ = 2 − 1 / 2 \rho=2^{-1/2} ρ = 2 − 1/2 , put x z ( v ) = ρ j x_z(v)=\rho^j x z ( v ) = ρ j when v v v is on leaf z z z ’s root path at depth j j j , and zero otherwise. Leaves whose common ancestor is at depth j j j have distance 2 ∑ ℓ = j + 1 m ρ ℓ 2\sum_{\ell=j+1}^{m}\rho^\ell 2 ∑ ℓ = j + 1 m ρ ℓ . This is an ultrametric of doubling dimension at most two: four grandchild clusters cover each cluster at half its diameter. Its diameter is bounded, but its range is R m = ∑ j = 1 m ( 2 ρ ) j = Θ ( n ) R_m=\sum_{j=1}^{m}(2\rho)^j=\Theta(\sqrt{n}) R m = ∑ j = 1 m ( 2 ρ ) j = Θ ( n ) .
Fix two leaves in opposite root branches. Until they separate, a nonseparating coordinate cut removes a descendant cluster. Write c = ρ / ( 1 − ρ ) c=\rho/(1-\rho) c = ρ / ( 1 − ρ ) and let P v = ∑ ℓ = j m 2 ℓ − j ρ ℓ P_v=\sum_{\ell=j}^{m}2^{\ell-j}\rho^\ell P v = ∑ ℓ = j m 2 ℓ − j ρ ℓ sum the coordinate weights indexed by a depth-j j j vertex and its descendants. Such a removal loses at most P v + c P_v+c P v + c range, including emptied ancestors, and ∑ v ρ depth ( v ) P v ≤ c R m \sum_v \rho^{\operatorname{depth}(v)}P_v\le cR_m ∑ v ρ depth ( v ) P v ≤ c R m by summing first over the descendant coordinates. While the remaining range is at least R m / 2 R_m/2 R m /2 , the expected nonseparating range loss per cut is thus at most 3 c 3c 3 c , and separation probability at most 4 c / R m 4c/R_m 4 c / R m . Stopping at separation or when range first falls below R m / 2 R_m/2 R m /2 , Markov’s inequality and a union bound give probability at least 1 − 10 c k / R m 1-10ck/R_m 1 − 10 c k / R m that neither has occurred after k k k cuts. Thus Ω ( n ) \Omega(\sqrt{n}) Ω ( n ) levels can pass with constant probability and no diameter decrease; the separation identity gives Ω ( n ) \Omega(\sqrt{n}) Ω ( n ) expected stretch as well.
Thus a bound on doubling dimension alone does not supply the localization or stretch guarantee of Theorem 3. The distinction concerns the coordinate representation of the metric: its doubling dimension does not control the summed widths used by the cuts.
Example I: From a Score to a Replacement
The upward recurrence (5) accumulates a score and its face contributions together: credit each separation payoff to the face on which it occurs. The contributions sum to the scalar score, and averaging them with the same weights across trees preserves this identity. § 3.1 defines the inverse-depth choice and its escape interpretation. Here we use the retained face and reference box to decide what to inspect: sum the two faces to rank coordinates, and keep their split to locate the direction contributing the score.
Let J J J contain the coordinates selected for investigation. A counterfactual replacement masks them and holds the remaining values fixed. At a cut on an observed coordinate the query follows its child; at a cut on J J J both children remain feasible. Leaves supply candidate values for J J J , and completed queries can be scored by the same recurrence. One selection rule compares branch candidates by score plus an independent uniform priority, favoring lower scores while retaining alternatives. Repeated selections are summarized by coordinatewise medians. The selection weights determine the distribution being summarized.
The replacement is in the observation’s units. Rescoring it checks whether the selected contributions fall, or the anomaly shifts to other directions. A reporting rule can also require a material change, such as a 20% deviation from the replacement, with an absolute-scale convention near zero. This chain of attribution, conditional replacement, and rescoring is used in OpenSearch [58 , 63 ] (Li and Kolchina 2025).
Imputation starts with a supplied mask and preserves the observed coordinates before detection. A prescribed fill rule is also possible: carrying a previous value forward bridges an expected reporting gap, while filling a missing count with zero can expose a cessation of activity (Li and Blomberg, 2025). The mask and fill choice remain part of the evidence for the resulting score.
Example II: Forecasting by Completion
A shingle of length k + 1 k+1 k + 1 pairs the history Z t = ( X t − k + 1 , … , X t ) Z_t=(X_{t-k+1},\ldots,X_t) Z t = ( X t − k + 1 , … , X t ) with the continuation Y t = X t + 1 Y_t=X_{t+1} Y t = X t + 1 . A forecast fixes the history at its observed value z t z_t z t and masks Y t Y_t Y t ; the same traversal supplies conditional continuations, summarized by their median [59 ] . Masking an earlier block instead gives a retrospective replacement, which can use readings already present later in the shingle. The mask specifies the question and the available evidence.
The cut law determines which distinctions this traversal can expose. Take the periodic signal 0 k 1 k 0^k1^k 0 k 1 k , k ≥ 2 k \ge2 k ≥ 2 , with independent continuous median-zero noise supported on [ − σ , σ ] [-\sigma,\sigma] [ − σ , σ ] , 0 < σ < 1 / 4 0 < \sigma< 1/4 0 < σ < 1/4 . Thresholding the last k k k readings at 1 / 2 1/2 1/2 identifies the next noiseless bit. At a node where s > 0 s > 0 s > 0 of the d = k + 1 d = k+1 d = k + 1 coordinates still distinguish phases, those widths are at least 1 − 2 σ 1-2\sigma 1 − 2 σ ; the other widths are at most 2 σ 2\sigma 2 σ . All widths are positive almost surely. Uniform coordinate allocation, as in isolation trees and fully randomized Extra-Trees, and the positive-power cut laws give [48 , 31 , 36 ]
p n o i s e ( 0 ) = d − s d , p n o i s e ( p ) ≤ ( d − s ) ( 2 σ ) p s ( 1 − 2 σ ) p + ( d − s ) ( 2 σ ) p , p > 0 (17) p_{\mathrm{noise}}^{(0)}=\frac{d-s}{d}, \qquad p_{\mathrm{noise}}^{(p)}\le\frac{(d-s)(2\sigma)^p}{s(1-2\sigma)^p+(d-s)(2\sigma)^p}, \qquad p>0 \tag*{(17)} p noise ( 0 ) = d d − s , p noise ( p ) ≤ s ( 1 − 2 σ ) p + ( d − s ) ( 2 σ ) p ( d − s ) ( 2 σ ) p , p > 0 (17)
for the share of cuts spent on noise-only coordinates. At p = 1 p=1 p = 1 this is length allocation. At fixed d d d , every p > 0 p>0 p > 0 makes this share vanish with σ \sigma σ ; uniform allocation does not. The bound follows by maximizing the noise widths and minimizing the phase-varying widths.
At a node restricted to the patterns ( 0 k , 1 ) (0^k,1) ( 0 k , 1 ) and ( 1 , 0 k − 1 , 0 ) (1,0^{k-1},0) ( 1 , 0 k − 1 , 0 ) , only the oldest lag and the continuation have unit-scale variation. As σ ↓ 0 \sigma\downarrow0 σ ↓ 0 , a clean oldest-lag cut has probability 1 / d 1/d 1/ d under uniform allocation and 1 / 2 1/2 1/2 under width-proportional allocation. That cut distinguishes the observed histories; a cut on the missing continuation requires branching. Thus vanishing noise can retain a fixed share of uniform cuts. Within a phase, common positive scaling of the noise leaves each normalized power cut law unchanged, so repeated observations still permit refinement of the phase cloud. The completion weights determine how that resolution enters the forecast. Proposition 4 states these facts.
Proposition 4 (Cut allocation on the block signal). At a node holding records from a set of phases, call a coordinate phase-varying when both binary values occur among its records, and let s s s be their number.
(i) At a node with s > 0 s>0 s > 0 , a cut falls on one of the other d − s d-s d − s coordinates with probability ( d − s ) / d (d-s)/d ( d − s ) / d under uniform allocation, and with probability at most the right side of (17) for every fixed p > 0 p>0 p > 0 .
(ii) In a box holding records of the two patterns a = ( 0 k , 1 ) a=(0^k,1) a = ( 0 k , 1 ) and b = ( 1 , 0 k − 1 , 0 ) b=(1,0^{k-1},0) b = ( 1 , 0 k − 1 , 0 ) only, a clean oldest-lag cut, which chooses coordinate one and a threshold in [ σ , 1 − σ ] [\sigma,1-\sigma] [ σ , 1 − σ ] , separates the two histories; its probabilities satisfy
p c l e a n ( 0 ) ≥ 1 − 2 σ d ( 1 + 2 σ ) , p c l e a n ( 1 ) ≥ 1 − 2 σ 2 + 2 d σ , (18) p_{\mathrm{clean}}^{(0)} \ge\frac{1-2\sigma}{d(1+2\sigma)}, \qquad p_{\mathrm{clean}}^{(1)} \ge\frac{1-2\sigma}{2+2d\sigma}, \tag*{(18)} p clean ( 0 ) ≥ d ( 1 + 2 σ ) 1 − 2 σ , p clean ( 1 ) ≥ 2 + 2 d σ 1 − 2 σ , (18)
which tend to 1 / d 1/d 1/ d and 1 / 2 1/2 1/2 as σ ↓ 0 \sigma\downarrow0 σ ↓ 0 . More generally, every fixed p > 0 p>0 p > 0 gives clean-cut probability tending to 1 / 2 1/2 1/2 at fixed d d d .
(iii) At a node whose records all come from one phase, the cut law for every fixed p ≥ 0 p\ge0 p ≥ 0 , with its threshold normalized to the box, does not depend on σ > 0 \sigma>0 σ > 0 .
Proof. (i) With bounded noise, each phase-varying width is at least 1 − 2 σ 1-2\sigma 1 − 2 σ and each other width is at most 2 σ 2\sigma 2 σ ; continuous noise makes every width positive almost surely at a node with two distinct records. Uniform allocation assigns ( d − s ) / d (d-s)/d ( d − s ) / d to the noise-only coordinates. Under exponent p > 0 p>0 p > 0 , their share is largest when their widths are 2 σ 2\sigma 2 σ and the others 1 − 2 σ 1-2\sigma 1 − 2 σ , which gives (17). (ii) In this box the oldest-lag and continuation widths lie in [ 1 − 2 σ , 1 + 2 σ ] [1-2\sigma,1+2\sigma] [ 1 − 2 σ , 1 + 2 σ ] and each of the other k − 1 k-1 k − 1 widths is at most 2 σ 2\sigma 2 σ . Under uniform allocation, coordinate one is chosen with probability 1 / d 1/d 1/ d and a clean threshold then with probability at least ( 1 − 2 σ ) / ( 1 + 2 σ ) (1-2\sigma)/(1+2\sigma) ( 1 − 2 σ ) / ( 1 + 2 σ ) . Under width-proportional allocation the coordinate weight cancels its width, leaving the length 1 − 2 σ 1-2\sigma 1 − 2 σ of the clean interval over the total extent, at most 2 + 2 d σ 2+2d\sigma 2 + 2 d σ . For every fixed p > 0 p>0 p > 0 , the two phase-varying widths tend to one, the other widths to zero, and a clean threshold occupies an asymptotically full share of the oldest-lag interval. This gives the same 1 / 2 1/2 1/2 limit. (iii) Records of one phase have the form b ϕ + σ u i b_\phi+\sigma u_i b ϕ + σ u i , so every coordinate width is σ \sigma σ times the corresponding width of the u i u_i u i ; all power-based coordinate probabilities and normalized thresholds are unchanged by a common positive factor, and the same holds at every node below.
Hitting Time Functionals
A forest is an ensemble of controlled random partitions of the space, and random walks and their hitting functionals are natural tools for analyzing such structures. The partitions are maintained and algorithmically designed, so they supply query-relative measurements of mass, extent, gaps, and scale through related stochastic experiments, and the relation among the experiments makes the measurements interpretable.
A surprising connection of studying hitting time is that some of the existing scores of RCF released in open source, Guha and Tokle [33 ] , can be reinterpreted as an escape score — indicating a curious view that anomaly attempts to escape being characterized by the remainder of data. § 3.1 reads inverse depth, displacement, and range escape as one escape measured in three lengths. § 3.2 reads escape at a declared mass, providing resilience to near duplicates without deleting records. § 3.3 then retains the separation locations as a hitting bundle: cut, passage, and calibration boxes with directional extents and variation across trees. The same additive path lengths will supply the pair interactions used by § 4 .
Sampling and conditioning. Let Q be a declared law on finite samples S S S . It may subsample or resample a fixed batch, or draw from a specified population; a point mass at the full batch gives the fixed-sample case. Conditional on S S S , draw a tree from the build law B ( ⋅ ∣ S ) \mathrm{B}(\cdot\mid S) B ( ⋅ ∣ S ) of § 2.2 . A query then supplies its separation law μ T , x \mu_{T,x} μ T , x . Unless an additional expectation is stated, a path identity holds conditional on the sampled records and the realized cuts. A theorem that averages cuts while fixing S S S keeps this conditioning explicit. A forest averages over the sampled records and the trees built on them. Different samples change the nearby observations, boxes, masses, and possible attachments. This sampling variation is present even in batch data, and is distinct from bounded coordinate perturbations in § 2.3 . We use T T T as shorthand for a tree realization together with its retained sample; its ensemble law is Q ( d S ) B ( d T ∣ S ) Q(\mathrm{d}S)B(\mathrm{d}T \mid S) Q ( d S ) B ( d T ∣ S ) . Independent draws give a Monte Carlo approximation to this declared average. The finite-tree identities and empirical averages themselves require no independence. These snapshot calculations do not specify how a stream maintains its samples.
Escape in Depth, Range, and Mass
For an unseen query followed to a singleton-location leaf, use the separation law (4) from § 2.1 . Its attachment probability s i q i s_iq_i s i q i places x x x beside v i v_i v i . In this virtual insertion a new parent u u u joins x x x and v i v_i v i , and the query becomes a leaf at depth i + 1 i+1 i + 1 . Write T i T_i T i for this augmented tree and r i r_i r i for its root. The new parent is also the root when v i v_i v i was the old root. For a seen query, consider removing the query and its prior copies and perform a shadow (pretend) operation of inserting all the copies simultaneously – but now the shadow insertion can be at any allowed level reproducing the distribution.
Definition 4 (Escape on the augmented tree). In an augmented tree with root r r r , give each edge a positive length and use its reciprocal as conductance, so a random walk chooses among neighboring edges in proportion to their conductances. Write L ( w ) L(w) L ( w ) for the total length from x x x to w w w along the root path, with L ( x ) = 0 L(x)=0 L ( x ) = 0 , and τ x + \tau_x^+ τ x + for the first positive return time to x x x . The walk started at x x x escapes if it reaches r r r before it returns to $x.
Theorem 5 (Scores as escape probabilities). Let u u u be the parent of x x x in the augmented tree. Then
P x ( τ r < τ x + ) = L ( u ) L ( r ) . (19) \mathbb{P}_{x}(\tau_r < \tau_x^+) = \frac{L(u)}{L(r)}. \tag*{(19)} P x ( τ r < τ x + ) = L ( r ) L ( u ) . (19)
Write C x = R ( M r o o t ( x ) ) C_x=R(M_{\mathrm{root}}(x)) C x = R ( M root ( x )) . When x x x attaches beside v i v_i v i , the three lengths of this section give
Table 1.
Averaged over the separation law, the first is the unseen-query inverse-depth score of the library, the second is the range escape (22) , and the third is normalized displacement (Corollary 2).
Proof. Hold x x x at voltage zero and r r r at one. Branches off their connecting path carry no current, and the voltage rises in proportion to cumulative resistance. After the obligatory first step to u u u , the chance of reaching r r r first is its voltage, L ( u ) / L ( r ) L(u)/L(r) L ( u ) / L ( r ) ([27 , 51 ] ). With unit lengths the first edge has length one and the path i + 1 i+1 i + 1 ; with range lengths they are R ( M v i ( x ) ) R(M_{v_i}(x)) R ( M v i ( x )) and C x C_x C x ; with mass lengths, excluding the query, they are m v i m_{v_i} m v i and ψ \psi ψ . □
Proposition 6 (Inverse-depth escape). Give each edge unit length. At an attachment beside v i v_i v i , the walk satisfies
P x T i ( τ r i < τ x + ) = 1 i + 1 . (20) \mathbb{P}_{x}^{T_i}(\tau_{r_i} < \tau_x^+) = \frac{1}{i+1}. \tag*{(20)} P x T i ( τ r i < τ x + ) = i + 1 1 . (20)
Averaging over the separation law gives
U ( T , x ) = ∑ i s i q i P x T i ( τ r i < τ x + ) = ∑ i s i q i i + 1 . (21) U(T,x)=\sum_i s_iq_i\mathbb{P}^{T_i}_x(\tau_{r_i}<\tau_x^+)=\sum_i\frac{s_iq_i}{i+1}. \tag*{(21)} U ( T , x ) = i ∑ s i q i P x T i ( τ r i < τ x + ) = i ∑ i + 1 s i q i . (21)
Thus U ( T , x ) U(T,x) U ( T , x ) is the escape probability when the attachment is drawn first and the walk is then run on that augmented tree.
Proof. The first edge has length one and the whole path has length i + 1 i+1 i + 1 . Apply (19) and average, using ∑ i s i q i = 1 \sum_i s_iq_i=1 ∑ i s i q i = 1 . □ \square □
A shallow attachment gives a larger escape probability. Ranking queries by this probability is the inverse-depth anomaly score. The forest averages U ( T , x ) U(T,x) U ( T , x ) over its trees. This is the unnormalized unseen-query calculation underlying the OpenSearch score [58 , 63 ] .
Proposition 7 (Range escape). Use range as the cumulative length on the augmented query path, with length zero at x x x ; contract any edge with zero range increment. The first edge has length R ( B u ) = R ( M v i ( x ) ) R(B_u)=R(M_{v_i}(x)) R ( B u ) = R ( M v i ( x )) , and the whole path has length C x C_x C x . With the gaps G v G_v G v and survivals s v s_v s v of § 2.1 ,
E [ P x ( τ r < τ x + ) ] = E R ( B u ) C x = ∑ v s v G v C x . (22) \mathbb{E}\left[\mathbb{P}_x(\tau_r<\tau_x^+)\right]=\frac{\mathbb{E}R(B_u)}{C_x}=\frac{\sum_v s_vG_v}{C_x}. \tag*{(22)} E [ P x ( τ r < τ x + ) ] = C x E R ( B u ) = C x ∑ v s v G v . (22) This is also the probability that an independent random cut of M r o o t ( x ) M_{\mathrm{root}}(x) M root ( x ) crosses the sampled separation box B u B_u B u .
Proof. Equation (19) gives R ( B u ) / C x R(B_u)/C_x R ( B u ) / C x at each attachment. Its expectation is ∑ v s v q v R ( M v ( x ) ) / C x = ∑ v s v G v / C x \sum_v s_vq_vR(M_v(x))/C_x=\sum_v s_vG_v/C_x ∑ v s v q v R ( M v ( x )) / C x = ∑ v s v G v / C x . An independent cut chooses a coordinate in proportion to its width and a uniform position on that coordinate. The fraction of those positions crossing B u B_u B u is R ( B u ) / C x R(B_u)/C_x R ( B u ) / C x .
Range escape measures the fraction of the tree’s spatial extent still in play at separation, averaged over the attachment. Replacing G v G_v G v in (22) by each of its face gaps gives contributions that sum to the same probability. The box ranges also give the tree distance used in the RCF analysis [36 ] . Put H v = R ( B v ) / 2 H_v=R(B_v)/2 H v = R ( B v ) /2 on a sample tree and give an edge from parent p p p to child v v v length H p − H v H_p-H_v H p − H v . Stored locations have height zero, so
d T ( a , b ) = 2 H lca ( a , b ) = R ( B lca ( a , b ) ) ≥ ∥ a − b ∥ 1 . (23) d_T(a,b)=2H_{\operatorname{lca}(a,b)}=R(B_{\operatorname{lca}(a,b)})\geq\lVert a-b\rVert_1. \tag*{(23)} d T ( a , b ) = 2 H lca ( a , b ) = R ( B lca ( a , b ) ) ≥ ∥ a − b ∥ 1 . (23)
For the escape, the cumulative range length is L = 2 H L=2H L = 2 H on the augmented query path. This common factor cancels from the ratio in (19) .
Corollary 2 (Displacement and CoDisp). Use original sample mass as the cumulative length on the query path, excluding the inserted query. When x x x attaches beside v i v_i v i , the first edge has length m v i m_{v_i} m v i and the whole path has length ψ \psi ψ . The escape probability is therefore m v i / ψ m_{v_i}/\psi m v i / ψ . Exactly the old records below v i v_i v i gain a branch decision on insertion, so normalized displacement is the expected mass escape [36 ] .
For CoDisp, consider a tree with root r r r containing x x x , and at each cut on its path take the whole child subtree containing x x x as the departing coalition. If its mass is a a a and its sibling’s mass is b b b , deleting that coalition saves one branch decision for each record in the sibling. The saving per departing record is b / a b/a b / a . CoDisp takes the largest such ratio over the path:
CoDisp ( T , x ) = max w ≠ r on the path of x m w ′ m w , \operatorname{CoDisp}(T,x)=\max_{w\ne r\text{ on the path of }x}\frac{m_{w'}}{m_w}, CoDisp ( T , x ) = w = r on the path of x max m w m w ′ , where m w m_w m w is the mass below w w w and w ′ w' w ′ is its sibling. The forest averages these per-tree maxima.
Proof. For displacement, the mass increments along the path are the masses of the siblings that join it. Apply (19) . For CoDisp, contraction of the departing subtree’s parent reduces the sibling’s depths by one and leaves all other surviving depths fixed, giving the saving b b b .
The ratio has a walk interpretation on the mass path as well. Place a ground below x x x , and give each path vertex cumulative resistance equal to its subtree mass, now including x x x . From w w w , the probability of reaching its parent before ground is p = a / ( a + b ) p = a/(a+b) p = a / ( a + b ) . The complementary odds are ( 1 − p ) / p = b / a (1-p)/p = b/a ( 1 − p ) / p = b / a .
Corollary 3 (Affine control of the three escapes). Fix the stored sample S S S and an unseen query x x x , and put δ = min y ∈ S ∥ x − y ∥ 1 > 0 \delta= \min_{y\in S}\lVert x-y\rVert_1 > 0 δ = min y ∈ S ∥ x − y ∥ 1 > 0 , Δ = diam 1 ( S ∪ { x } ) \Delta= \operatorname{diam}_1(S\cup\{x\}) Δ = diam 1 ( S ∪ { x }) , and C x = R ( bbox ( S ∪ { x } ) ) C_x = R(\operatorname{bbox}(S\cup\{x\})) C x = R ( bbox ( S ∪ { x })) . Suppose S ∪ { x } S\cup\{x\} S ∪ { x } has affine dimension r ≥ 1 r\ge1 r ≥ 1 , and set
a = 2 K r , 1 ( 1 / 2 ) [ 1 + ⌈ log 2 ( Δ / δ ) ⌉ ] . a = 2K_{r,1}(1/2)\left[1+\left\lceil\log_2(\Delta/\delta)\right\rceil\right]. a = 2 K r , 1 ( 1/2 ) [ 1 + ⌈ log 2 ( Δ/ δ ) ⌉ ] . Draw T T T on S S S and then the virtual attachment of x x x . Let D x D_x D x be its augmented depth, R ∗ R_* R ∗ the range of its first ancestor containing stored records, and m ∗ m_* m ∗ the stored mass at that ancestor. The expectations below include both cuts and attachment. The depth and range escapes satisfy
E D x ≤ a , E T U ( T , x ) = E 1 D x ≥ 1 a , \mathbb{E}D_x\le a,\qquad\mathbb{E}T_U(T,x)=\mathbb{E}\frac{1}{D_x}\ge\frac{1}{a}, E D x ≤ a , E T U ( T , x ) = E D x 1 ≥ a 1 , δ C x ≤ E R ∗ C x ≤ min { 1 , a δ C x } . (24) \frac{\delta}{C_x}\le\frac{\mathbb{E}R_*}{C_x}\le\min\left\{1,\frac{a\delta}{C_x}\right\}. \tag*{(24)} C x δ ≤ C x E R ∗ ≤ min { 1 , C x a δ } . (24) Let M x ( t ) M_x(t) M x ( t ) be the stored mass within ℓ 1 \ell_1 ℓ 1 distance t t t of x x x , and let ψ \psi ψ be the total stored mass. For every t > 0 t>0 t > 0 , the mass escape satisfies
E m ∗ ψ ≤ min { 1 , M x ( t ) ψ + a δ t } . (25) \frac{\mathbb{E}m_*}{\psi}\le\min\left\{1,\frac{M_x(t)}{\psi}+\frac{a\delta}{t}\right\}. \tag*{(25)} ψ E m ∗ ≤ min { 1 , ψ M x ( t ) + t a δ } . (25) If instead the augmented sample meets the noise model of Corollary 1 and δ ≥ 16 W \delta\ge16W δ ≥ 16 W , the same statements hold with K n o i s e ( 1 / 2 ) K_{\mathrm{noise}}(1/2) K noise ( 1/2 ) in place of K r , 1 ( 1 / 2 ) K_{r,1}(1/2) K r , 1 ( 1/2 ) , using the clean affine dimension r r r .
Proof. Insertion consistency makes the augmented tree a fresh random-cut tree on S ∪ { x } S\cup\{x\} S ∪ { x } . Apply the conditional contraction blocks of Theorem 3 along the cell containing x x x . After 1 + ⌈ log 2 ( Δ / δ ) ⌉ 1+\lceil\log_2(\Delta/\delta)\rceil 1 + ⌈ log 2 ( Δ/ δ )⌉ successful blocks its diameter is less than δ \delta δ , so it contains no stored record. The expected depth is at most a a a , and convexity of 1 / D x 1/D_x 1/ D x gives the depth escape bound. If y y y is a nearest stored location, then δ ≤ R ∗ ≤ d T ( x , y ) \delta\le R_*\le d_T(x,y) δ ≤ R ∗ ≤ d T ( x , y ) , so the stretch theorem and R ∗ ≤ C x R_*\le C_x R ∗ ≤ C x give the range bounds. On R ∗ ≤ t R_*\le t R ∗ ≤ t , every record at the first ancestor lies within distance t t t of x x x ; hence m ∗ ≤ M x ( t ) m_*\le M_x(t) m ∗ ≤ M x ( t ) . On the complementary event use m ∗ ≤ ψ m_*\le\psi m ∗ ≤ ψ and Markov’s inequality P ( R ∗ > t ) ≤ E R ∗ / t ≤ a δ / t \mathbb{P}(R_*>t)\le\mathbb{E}R_*/t\le a\delta/t P ( R ∗ > t ) ≤ E R ∗ / t ≤ a δ / t . The noisy contraction and stretch bounds apply by the same argument while the common cell has diameter at least δ ≥ 16 W \delta\ge16W δ ≥ 16 W .
Affine dimension thus controls how many decisions are needed to resolve the nearest gap and the range at which the query first meets the sample. The mass reading also depends on how much mass lies at that scale. Copies change this occupancy without changing the affine geometry. An arbitrary query increases the affine dimension of the stored sample by at most one. Below the noise scale, Theorem 9 supplies an additive range bound; the depth bound above requires resolution beyond that scale.
Nonparametric Escape and Near Duplicates
Exact duplicates add mass without increasing spatial extent. Near duplicates concentrate mass in a small range. A declared mass chooses how much support is required before a query reads its escape. These statements concern the records retained in each sampled tree; the sampling law determines how often a cloud supplies that support.
Definition 5 (Ancestor at a declared mass). On a complete query path v 0 , … , v h v_0,\ldots,v_h v 0 , … , v h of root mass ψ ≥ 1 \psi\ge1 ψ ≥ 1 , the mass selector c μ ( x ) c_\mu(x) c μ ( x ) is the deepest vertex containing at least mass μ \mu μ , in record units. For 0 < μ ≤ ψ 0 < \mu\le\psi 0 < μ ≤ ψ ,
i μ = max { i : m v i ≥ μ } , c μ ( x ) = v i μ . (26) i_\mu= \max\{i : m_{v_i} \ge\mu\}, \qquad c_\mu(x) = v_{i_\mu}. \tag*{(26)} i μ = max { i : m v i ≥ μ } , c μ ( x ) = v i μ . (26)
Escape at a declared mass. A query can separate beside a few near duplicates even when the cloud is far from the rest of the sample. Its separation box can then have only the range of the cloud, making ordinary range escape small. Choose a minimum mass μ \mu μ and read the escape at the first ancestor with that much support. This is an adaptive-neighbor construction in the tree’s range metric. It keeps the sample and the separation law fixed; it requires one query path and no coalition search. The calibration box of § 3.3 uses mass in the same way to set the scale of a comparison.
For an integer 1 ≤ μ ≤ ψ 1 \le\mu\le\psi 1 ≤ μ ≤ ψ , use the mass selector c μ ( x ) c_\mu(x) c μ ( x ) of Definition 5. Here μ \mu μ is a minimum number of stored records. Stop the separation experiment at c μ ( x ) c_\mu(x) c μ ( x ) : keep separations above it, and use its range for all outcomes that reach it. With v v v the separating vertex under (4) and v ∨ c v \vee c v ∨ c the higher of two vertices on the path, put
Esc μ ( T , x ) = E R ( B v ∨ c μ ( x ) ∪ { x } ) C , C = R ( B r o o t ∪ { x } ) . (27) \operatorname{Esc}_\mu(T,x) = \frac{\mathbb{E}\,R\left(B_{v\vee c_\mu(x)} \cup\{x\}\right)}{C}, \qquad C = R\left(B_{\mathrm{root}} \cup\{x\}\right). \tag*{(27)} Esc μ ( T , x ) = C E R ( B v ∨ c μ ( x ) ∪ { x } ) , C = R ( B root ∪ { x } ) . (27)
For a query that matches a stored location, the terminal outcome uses the range at c μ ( x ) c_\mu(x) c μ ( x ) . Set the measurement to zero when C = 0 C = 0 C = 0 .
In the tree augmented by virtual insertion, the numerator reads the range of the first ancestor of x x x containing at least μ \mu μ original records, counted with multiplicity. It is the distance to the μ \mu μ th stored neighbor in the augmented tree’s range metric (23) . Equivalently, put the query at voltage zero and the root at one, using cumulative range as resistance. The reading is the voltage at this ancestor, as in (19) ; § 4.1 develops the corresponding potential game.
Theorem 8 (Escape at a declared mass). Fix the stored sample and query x x x , and let D D D be a box containing x x x , with δ = R ( D ) \delta= R(D) δ = R ( D ) . Parts (i) and (ii) concern a fixed tree. Parts (iii)–(v) use the random-cut construction with coordinate choice proportional to width and a uniform threshold in the chosen coordinate.
(i) Esc 1 \operatorname{Esc}_1 Esc 1 is the range escape (22) , and Esc μ \operatorname{Esc}_\mu Esc μ is nondecreasing in μ \mu μ . Its numerator is computed by the fold (5) , stopped at c μ ( x ) c_\mu(x) c μ ( x ) .
(ii) Suppose C > 0 C > 0 C > 0 . If D D D contains fewer than μ \mu μ stored records and every stored point y y y outside it satisfies ∑ j dist ( y j , D j ) ≥ Γ \sum_j \operatorname{dist}(y_j,D_j) \ge\Gamma ∑ j dist ( y j , D j ) ≥ Γ , then Esc μ ( T , x ) ≥ Γ / C \operatorname{Esc}_\mu(T,x) \ge\Gamma/C Esc μ ( T , x ) ≥ Γ/ C .
(iii) Suppose there are stored records both inside and outside D D D , and the sum Γ ′ \Gamma' Γ ′ of the coordinate gaps between D D D and the bounding box of the outside records is positive. With probability at least Γ ′ / ( Γ ′ + δ ) \Gamma'/(\Gamma' + \delta) Γ ′ / ( Γ ′ + δ ) , the query’s branch reaches a node containing just the records in D D D before any cut on that branch crosses D D D .
(iv) Under the gap condition of (iii), if D D D contains at least μ \mu μ stored records, then
E T Esc μ ( T , x ) ≤ δ C + δ Γ ′ + δ . \mathbb{E}_{T}\operatorname{Esc}_{\mu}(T,x) \le\frac{\delta}{C}+\frac{\delta}{\Gamma' + \delta}. E T Esc μ ( T , x ) ≤ C δ + Γ ′ + δ δ .
(v) Let D D D be the bounding box of x x x and a group of at least μ \mu μ stored records. For each coordinate and side, let b j ± b_j^{\pm} b j ± be the distance from the face of D D D to the corresponding face of the root box with x x x included. If δ > 0 \delta> 0 δ > 0 , then
E T Esc μ ( T , x ) ≤ δ C ( 1 + ∑ j , ± log ( 1 + b j ± δ ) ) . \mathbb{E}_{T}\operatorname{Esc}_{\mu}(T,x) \le\frac{\delta}{C}\left(1+\sum_{j,\pm}\log\left(1+\frac{b_j^{\pm}}{\delta}\right)\right). E T Esc μ ( T , x ) ≤ C δ ( 1 + j , ± ∑ log ( 1 + δ b j ± ) ) .
If δ = 0 \delta= 0 δ = 0 , the measurement is zero. No gap between the bounding boxes is required.
Proof. (i) For μ = 1 \mu= 1 μ = 1 , c 1 ( x ) c_1(x) c 1 ( x ) is the routed leaf, so the stopping rule leaves the original range escape unchanged. Increasing μ \mu μ moves c μ ( x ) c_\mu(x) c μ ( x ) up the path, and the boxes are nested. To compute the numerator, initialize the fold at c μ ( x ) c_\mu(x) c μ ( x ) with R ( B c μ ( x ) ∪ { x } ) R(B_{c_\mu(x)} \cup\{x\}) R ( B c μ ( x ) ∪ { x }) and use the original separation payoffs above it. In the augmented tree, if v v v already contains μ \mu μ stored records, the new parent of x x x is the first ancestor meeting the mass requirement. Otherwise that ancestor is c μ ( x ) c_\mu(x) c μ ( x ) , whose box now includes x x x . At a match the new copy shares the original leaf. This gives the stated tree-neighbor interpretation in each case.
(ii) Every box used for a payoff contains at least μ \mu μ stored records, so it contains some y y y outside D D D . Its range is at least ∑ j ∣ x j − y j ∣ ≥ ∑ j dist ( y j , D j ) \sum_j |x_j-y_j| \ge\sum_j \operatorname{dist}(y_j,D_j) ∑ j ∣ x j − y j ∣ ≥ ∑ j dist ( y j , D j ) , which proves the floor.
(iii) At a node still containing the cloud and some outside records, the available separating gaps have total length at least Γ ′ \Gamma' Γ ′ , while thresholds through D D D have total length at most δ \delta δ . Cuts elsewhere remove only outside records, which cannot shrink the gaps. Conditional on the first cut in either a separating gap or D D D , it falls in a gap with probability at least Γ ′ / ( Γ ′ + δ ) \Gamma'/(\Gamma' + \delta) Γ ′ / ( Γ ′ + δ ) . Such a cut leaves the records of D D D together on the query’s branch.
(iv) The insertion law gives a random-cut tree on the augmented sample [36 ] ; adding a copy at a matching location changes masses without changing cuts. Apply (iii) to this augmented sample. On the event that the cloud is isolated intact, the first ancestor containing μ \mu μ original records lies within the cloud’s subtree and has range at most δ \delta δ . The normalized payoff is at most δ / C \delta/C δ / C on this event and at most one otherwise.
(v) Realize the cuts by an auxiliary Poisson process on position and time, with unit intensity over the augmented root box’s coordinate intervals laid end to end. On the query branch, accept the next mark in the current box’s intervals. Its position is uniform over their total length, giving width-proportional cuts. Disjoint intervals have independent marks; positions of total length r r r have no mark by time t t t with probability e − t r e^{-tr} e − t r [42 ] . The first mark through D D D occurs at time τ ∼ Exp ( δ ) \tau\sim\operatorname{Exp}(\delta) τ ∼ Exp ( δ ) . Until that mark, the cloud stays together, so the ancestor used by the measurement is no higher than the node where this cut occurs.
Let Z j ± Z_j^{\pm} Z j ± be that node’s extent beyond the corresponding face of D D D . For Z j ± Z_j^{\pm} Z j ± to exceed u u u , there can have been no mark before τ \tau τ in the adjoining interval of length u u u : such a mark would have cut off the records beyond it. These marks are independent of τ \tau τ , so
E Z j ± ≤ ∫ 0 b j ± E e − u τ d u = δ log ( 1 + b j ± δ ) . \mathbb{E}Z_j^{\pm} \le\int_0^{b_j^{\pm}} \mathbb{E}e^{-u\tau}\,du = \delta\log\left(1+\frac{b_j^{\pm}}{\delta}\right). E Z j ± ≤ ∫ 0 b j ± E e − uτ d u = δ log ( 1 + δ b j ± ) .
The node’s range is δ + ∑ j , ± Z j ± \delta+\sum_{j,\pm}Z_j^{\pm} δ + ∑ j , ± Z j ± . Taking expectations and dividing by C C C gives the bound. If δ = 0 \delta= 0 δ = 0 , at least μ \mu μ stored records coincide with x x x and the mass requirement is met at its leaf.
The two regimes give μ \mu μ its meaning. If a cloud contains fewer than μ \mu μ retained records and is separated from the remaining data, part (ii) gives a floor in every tree. If it contains at least μ \mu μ records, parts (iv)–(v) give vanishing mean escape as δ / C \delta/C δ / C tends to zero in fixed dimension. Thus near-duplicate resilience is a statement at a declared mass resolution. Once enough copies are present, they supply the requested support. The threshold applies to each tree’s retained sample, and increasing it can raise the reading for ordinary points as well as anomalies.
The random-tree bounds use width-proportional cuts. § 2.2 explains why uniform coordinate allocation can retain a nonvanishing share of cuts on shrinking nuisance coordinates.
Corollary 4 (Noisy copies at a declared mass). Suppose the query and k ≥ 1 k \ge1 k ≥ 1 stored records are perturbations of one clean location. The other records have a clean bounding box at summed coordinate gap Γ 0 > 2 W \Gamma_0 > 2W Γ 0 > 2 W from that location. Every record and the query are perturbed in coordinate j j j by at most σ j \sigma_j σ j , where W = ∑ j σ j W = \sum_j \sigma_j W = ∑ j σ j as in Corollary 1. Write C = R ( B r o o t ∪ { x } ) C = R(B_{\mathrm{root}} \cup\{x\}) C = R ( B root ∪ { x }) . For 1 ≤ μ ≤ ψ 1 \le\mu\le\psi 1 ≤ μ ≤ ψ ,
Esc μ ( T , x ) ≥ Γ 0 − 2 W C if k < μ , E T Esc μ ( T , x ) ≤ 2 W C + 2 W Γ 0 if k ≥ μ . \operatorname{Esc}_{\mu}(T,x) \ge\frac{\Gamma_0 - 2W}{C} \quad\text{if } k < \mu,\qquad\mathbb{E}_{T}\operatorname{Esc}_{\mu}(T,x) \le\frac{2W}{C} + \frac{2W}{\Gamma_0} \quad\text{if } k \ge\mu. Esc μ ( T , x ) ≥ C Γ 0 − 2 W if k < μ , E T Esc μ ( T , x ) ≤ C 2 W + Γ 0 2 W if k ≥ μ . The first inequality holds on every tree; the expectation in the second uses width-proportional cuts conditional on the noisy sample.
Proof. The box of the query and its k k k nearby records has range at most 2 W 2W 2 W . Its summed coordinate gap to the other records’ box is at least Γ 0 − 2 W \Gamma_0 - 2W Γ 0 − 2 W . Apply parts (ii) and (iv) of Theorem 8; the second bound uses δ / ( Γ ′ + δ ) ≤ 2 W / Γ 0 \delta/(\Gamma' + \delta) \le2W/\Gamma_0 δ / ( Γ ′ + δ ) ≤ 2 W / Γ 0 .
Thus insufficient nearby mass forces the reading across the gap, while sufficient nearby mass permits a reading on the noise-sized cloud. Sampling changes the retained count k k k and the gap; the corollary applies to each sample meeting its conditions.
Geometry and Regularization of the Hitting Bundle
The escape probabilities of Section 3.1 summarize how a query separates from one tree. The locations of those separations also select neighborhoods around it. Across a forest, the query paths, with their separation laws, masses, and boxes, form its hitting bundle . Figure 1 shows one such path; Figure 2 shows neighborhoods from several trees. The bundle retains both the geometry of each neighborhood and the variation between trees.
Figure 1. One query path through eight observations. (a) Nested data boxes and x = ( 4 , 2.5 ) x=(4,2.5) x = ( 4 , 2.5 ) ; the dashed box enlarges B v 4 B_{v_4} B v 4 by x x x , and its shaded extensions are empty. (b) The same ancestry, with attachment widths proportional to s i q i s_iq_i s i q i . (c) The separation law and the three escape readings of § 3.1 , with R 0 = 8.5 R_0=8.5 R 0 = 8.5 . Under Definition 6, the cut and passage boxes both equal B v 3 B_{v_3} B v 3 , and the calibration box is B v 0 B_{v_0} B v 0 . Colors identify ancestors.
Figure 2. One query and six trees from a 25-tree bundle, shown in coordinates 1 and 2; colors identify trees. Left: cut boxes (solid) and their enlargements by x x x (dashed); the shaded extensions are empty. Circles mark the gap-box centres on each tree’s largest-gap face. The arrow and ellipse give their mean displacement from x x x and one-standard-deviation spread over all 25 trees. Right: passage boxes of the same trees, at half the drawing scale. The black dashed box averages the query-relative half dimensions over all 25 trees.
Neighborhoods on one path. A query can traverse large boxes whose local separation probabilities are small. We start the neighborhood selection near the end of this part of the path. The cut box then uses a median attachment and the passage box a low attachment quantile. Both are restricted to lie at or below a common mass-selected ancestor; calibration selects the comparison neighborhood by mass.
On the complete path v 0 , … , v h v_0,\ldots,v_h v 0 , … , v h , choose ϵ ∈ ( 0 , 1 / 2 ) \epsilon\in(0,1/2) ϵ ∈ ( 0 , 1/2 ) and let k k k be the deepest index with q k ≤ ϵ q_k \le\epsilon q k ≤ ϵ , taking k = 0 k=0 k = 0 if there is none. Under the width-proportional query law, the probabilities q i q_i q i do not decrease down the nested boxes: exterior gaps grow as widths shrink. Conditional on reaching v k v_k v k , the cumulative attachment probability and its quantile levels, for 0 < α < 1 0 < \alpha< 1 0 < α < 1 , are
F k ( i ) = { 1 − ∏ ℓ = k i ( 1 − q ℓ ) , k ≤ i < h , 1 , i = h , Q α = min { i ∈ { k , … , h } : F k ( i ) > α } . F_k(i) =
\begin{cases}
1-\prod_{\ell=k}^{i}(1-q_\ell), & k \le i < h,\\
1, & i=h,
\end{cases}
\qquad
Q_\alpha= \min\{i \in\{k,\ldots,h\}: F_k(i) > \alpha\}. F k ( i ) = { 1 − ∏ ℓ = k i ( 1 − q ℓ ) , 1 , k ≤ i < h , i = h , Q α = min { i ∈ { k , … , h } : F k ( i ) > α } .
The terminal event is assigned to v h v_h v h , so Q α = h Q_\alpha=h Q α = h at an exact match. We use the mass selector c μ ( x ) c_\mu(x) c μ ( x ) of Definition 5 to restrict the selected ancestors.
Definition 6 (Cut, passage, and calibration boxes). Using the path and restarted law above:
The cut box is B c u t ( T , x ) B_{\mathrm{cut}}(T,x) B cut ( T , x ) , the data box at the deeper of the conditional-median vertex v Q 1 / 2 v_{Q_{1/2}} v Q 1/2 and c ψ ( x ) c_{\sqrt{\psi}}(x) c ψ ( x ) .
The passage box is B p a s s ( T , x ) B_{\mathrm{pass}}(T,x) B pass ( T , x ) , the data box at the deeper of v Q ε v_{Q_\varepsilon} v Q ε and c ψ ( x ) c_{\sqrt{\psi}}(x) c ψ ( x ) , using the low attachment quantile with the same cap.
For a declared μ c a l ∈ [ ψ , ψ ] \mu_{\mathrm{cal}}\in[\sqrt{\psi},\psi] μ cal ∈ [ ψ , ψ ] , the calibration box is B c a l ( T , x ; μ c a l ) = B c μ c a l ( x ) B_{\mathrm{cal}}(T,x;\mu_{\mathrm{cal}})=B_{c_{\mu_{\mathrm{cal}}}}(x) B cal ( T , x ; μ cal ) = B c μ cal ( x ) . The calibration mass can be selected by
μ ( θ ) = ψ ( 1 + θ ) / 2 , 0 ≤ θ ≤ 1. (28) \mu(\theta)=\psi^{(1+\theta)/2},\qquad0\le\theta\le1. \tag*{(28)} μ ( θ ) = ψ ( 1 + θ ) /2 , 0 ≤ θ ≤ 1. (28)
Each box retains its mass and its enlargement by the query, M = bbox ( B ∪ { x } ) M=\operatorname{bbox}(B\cup\{x\}) M = bbox ( B ∪ { x }) , together with the face gaps.
The examples use ε = 0.05 \varepsilon=0.05 ε = 0.05 and μ c a l = min { ψ , 2 ψ } \mu_{\mathrm{cal}}=\min\{\psi,2\sqrt{\psi}\} μ cal = min { ψ , 2 ψ } ; the latter corresponds to θ = 0.25 \theta=0.25 θ = 0.25 when ψ = 256 \psi=256 ψ = 256 . Restarting and the cap select the reported boxes; the root-based escapes of Section 3.1 retain their original law.
These are data boxes at three chosen vertices. Their query enlargements preserve the same nesting: B c u t ⊆ B p a s s ⊆ B c a l B_{\mathrm{cut}}\subseteq B_{\mathrm{pass}}\subseteq B_{\mathrm{cal}} B cut ⊆ B pass ⊆ B cal , with ties allowed. The low quantile selects the passage box; the conditional median can select a deeper cut box. Their shared cap allows the two boxes to coincide. Calibration selects mass independently of the query’s hazard.
Figure 1 illustrates the rules on one path. For ψ = 8 \psi=8 ψ = 8 , the cap c 8 ( x ) c_{\sqrt{8}}(x) c 8 ( x ) is v 3 v_3 v 3 . The uncapped passage candidate is v 1 v_1 v 1 and the median is v 3 v_3 v 3 , so both reported boxes are B v 3 B_{v_3} B v 3 . The default calibration mass exceeds the five records at v 1 v_1 v 1 , so its box is the root B v 0 B_{v_0} B v 0 . The table also shows the three escape readings developed in Section 3.1.
Geometry across trees. The trees can select different neighborhoods for the same query, both because they retain different records and because their cuts differ conditional on those records.
Definition 7 (Directional geometry of the hitting bundle). Fix coordinate units and a box-selection rule. For each realized tree T t T_t T t , let a t ( x ) ∈ R 2 d \mathbf{a}_t(x)\in\mathbb{R}^{2d} a t ( x ) ∈ R 2 d contain the query-relative half dimensions a t j − , a t j + a_{tj}^{-},a_{tj}^{+} a t j − , a t j + of its selected box, as in Definition 1. The vector retains elongation across coordinates and imbalance between opposite faces. Differences among the widths a t j − + a t j + a_{tj}^{-}+a_{tj}^{+} a t j − + a t j + record the selected box’s anisotropy in these units; differences between a t j − a_{tj}^{-} a t j − and a t j + a_{tj}^{+} a t j + record its asymmetry about the query. Its representative box has half dimensions
a ˉ j ± = 1 N ∑ t = 1 N a t j ± . \bar{a}_{j}^{\pm}=\frac{1}{N}\sum_{t=1}^{N}a_{tj}^{\pm}. a ˉ j ± = N 1 t = 1 ∑ N a t j ± .
Equal coordinate widths do not by themselves establish rotational isotropy of the underlying data.
Definition 8 (Mean gap displacement and its covariance: lead and swing). At each selected cut box choose a positive-gap face by a declared rule, and let c T ( x ) c_T(x) c T ( x ) be the centre of the gap box spanned by that face and x x x . Set c T ( x ) = x c_T(x)=x c T ( x ) = x if there is no positive gap. For Z T ( x ) = c T ( x ) − x Z_T(x)=c_T(x)-x Z T ( x ) = c T ( x ) − x , define the lead and swing by
ℓ g a p ( x ) = E T Z T ( x ) , Σ g a p ( x ) = E T [ ( Z T ( x ) − ℓ g a p ( x ) ) ( Z T ( x ) − ℓ g a p ( x ) ) T ] . \ell_{\mathrm{gap}}(x)=\mathbb{E}_T Z_T(x),\qquad
\Sigma_{\mathrm{gap}}(x)=\mathbb{E}_T\left[(Z_T(x)-\ell_{\mathrm{gap}}(x))(Z_T(x)-\ell_{\mathrm{gap}}(x))^\mathsf{T}\right]. ℓ gap ( x ) = E T Z T ( x ) , Σ gap ( x ) = E T [ ( Z T ( x ) − ℓ gap ( x )) ( Z T ( x ) − ℓ gap ( x ) ) T ] .
*when the second moment exists. In a finite forest, use its empirical averages. The figure chooses a largest-gap face.
Lead is mean gap displacement; swing is its covariance across trees. A long lead with small swing means the sampled trees locate similar empty margins. Both sampling and cut variation contribute to swing. For a displayed coordinate projection P P P , the mean arrow is P ℓ g a p P\ell_{\mathrm{gap}} P ℓ gap and the covariance ellipse uses P Σ g a p P T P\Sigma_{\mathrm{gap}}P^{\mathsf{T}} P Σ gap P T . These moments describe selected gap locations; the half dimensions retain the boxes’ extents as well.
Figure 2 shows a three-dimensional blade : coordinate 1 is uniform on [ − 5 , 5 ] [-5,5] [ − 5 , 5 ] and the transverse standard deviations are 0.5 and 0.12. The query x = ( 0 , 1.3 , 0 ) x=(0,1.3,0) x = ( 0 , 1.3 , 0 ) lies beside its narrow side; each of 25 trees holds ψ = 256 \psi=256 ψ = 256 observations. The mean passage box reaches 1.15 toward the data and only 0.01 away in coordinate 2. The swing is mainly along the blade, with standard deviation 0.55 against 0.10 across it: the trees agree on the separation side while their gap locations vary along the blade.
Directional readings. The same separating-face record also explains a scalar escape. For inverse depth, keeping that face gives its contributions:
A j , ± ( T , x ) = ∑ i s i p i , j , ± i + 1 , ∑ j , ± A j , ± ( T , x ) = U ( T , x ) . (29) A_{j,\pm}(T,x)=\sum_i\frac{s_i p_{i,j,\pm}}{i+1},\qquad\sum_{j,\pm}A_{j,\pm}(T,x)=U(T,x). \tag*{(29)} A j , ± ( T , x ) = i ∑ i + 1 s i p i , j , ± , j , ± ∑ A j , ± ( T , x ) = U ( T , x ) . (29)
The fold (5) computes the probability and these contributions together. The library’s attribution visitor carries out this calculation before normalization.
Anisotropic balloon estimates. Mass per unit extent gives a local density reading of the selected neighborhoods. Selecting a neighborhood at the query is the balloon view of density estimation
([69 ] ). The mass criterion plays the role of the neighbor count ([49 ] ); the tree supplies an anisotropic window. For a sample S S S of total mass n n n , a selected window A x A_x A x , and a reference measure λ \lambda λ with λ ( A x ) > 0 \lambda(A_x)>0 λ ( A x ) > 0 , its empirical occupancy per unit extent is
f ^ o c c ( x ) = m ( S ∩ A x ) / n λ ( A x ) . (30) \widehat{f}_{\mathrm{occ}}(x)=\frac{m(S\cap A_x)/n}{\lambda(A_x)}. \tag*{(30)} f occ ( x ) = λ ( A x ) m ( S ∩ A x ) / n . (30)
For example, λ \lambda λ may be ambient volume or a measure on an identified support. For a fixed-shape window of volume C h d Ch^d C h d , locally constant density gives k / n ≃ f ( x ) C h d k/n \simeq f(x)Ch^d k / n ≃ f ( x ) C h d : choosing mass chooses a bandwidth, while rectangular windows retain different lengths in different directions.
A scalar summary of the bundle also requires an aggregation rule. For the separation-law mass reading, fix a probe mass η ≥ 0 \eta\ge0 η ≥ 0 , put ξ v = m v + η \xi_v=m_v+\eta ξ v = m v + η , and define
a ( x ) = ∑ t = 1 N ∫ ξ v d μ T t , x ∑ t = 1 N n t , (31) a(x)=\frac{\sum_{t=1}^{N}\int\xi_v\,d\mu_{T_t,x}}{\sum_{t=1}^{N}n_t}, \tag*{(31)} a ( x ) = ∑ t = 1 N n t ∑ t = 1 N ∫ ξ v d μ T t , x , (31)
where n t n_t n t is tree t t t ’s sample mass and the terminal event receives its declared mass payoff. For a selected cut, passage, or calibration box, use its mass payoff in place of the integral. This specifies which mass reading accompanies the geometric summary.
Regularization of the bundle estimate. The representative geometry and the chosen mass reading specify the raw estimate. A regularization specifies how a vanishing extent is treated.
Definition 9 (Regularization of mass per unit extent). For the representative half dimensions of Definition 7, put
V b o x ( x ) = ∏ j = 1 d ( a ˉ j + a ˉ j + ) . V_{\mathrm{box}}(x)=\prod_{j=1}^{d}(\bar{a}_j+\bar{a}_j^{+}). V box ( x ) = j = 1 ∏ d ( a ˉ j + a ˉ j + ) . Given a ( x ) > 0 a(x)>0 a ( x ) > 0 , the additive regularization R γ \mathcal{R}_{\gamma} R γ adds γ ≥ 0 \gamma\geq0 γ ≥ 0 to volume per unit normalized mass:
f ^ R γ ( x ) = 1 V b o x ( x ) / a ( x ) + γ = a ( x ) V b o x ( x ) + γ a ( x ) , V b o x ( x ) + γ a ( x ) > 0. (32) \widehat{f}^{\mathcal{R}_{\gamma}}(x)=\frac{1}{V_{\mathrm{box}}(x)/a(x)+\gamma}=\frac{a(x)}{V_{\mathrm{box}}(x)+\gamma a(x)},\qquad V_{\mathrm{box}}(x)+\gamma a(x)>0. \tag*{(32)} f R γ ( x ) = V box ( x ) / a ( x ) + γ 1 = V box ( x ) + γ a ( x ) a ( x ) , V box ( x ) + γ a ( x ) > 0. (32)
Since a ( x ) a(x) a ( x ) is dimensionless, γ \gamma γ has volume units.
Thus γ \gamma γ sets a floor on volume per unit mass. Taking γ = 0 \gamma= 0 γ = 0 gives the raw ratio where the volume is positive; taking γ > 0 \gamma> 0 γ > 0 caps it at 1 / γ 1/\gamma 1/ γ , including when the observed volume vanishes. This operation regularizes a geometric reading; the measurement-error variance of Section 4.1 belongs to a different model. The order of aggregation is part of this estimate: average the half dimensions, form a volume, and divide the chosen mass reading by its regularized value. Averaging per-tree ratios is a different choice. To report empirical occupancy of the representative box itself, count its contents and use (30) .
On lower-dimensional data, a density on the support requires a measure on that support; an ambient-volume comparison can instead use the declared regularization above. Dropping zero-width coordinates amounts to choosing a coordinate support. These finite neighborhood readings retain their mass, extents, and gaps as geometric evidence. As with other balloon estimates, global normalization is a separate requirement [69 , 39 ] .
Directional mass-to-extent estimates. The balloon reading can also use each separating face and its associated length. Normalize the nonterminal directional law, retaining its terminal probability separately. For σ ∈ { − , + } \sigma\in\{-,+\} σ ∈ { − , + } , let w j σ w_{j\sigma} w j σ be the probability of separation on face ( j , σ ) (j,\sigma) ( j , σ ) and D j σ D_{j\sigma} D j σ the integral of the expanded width a ˉ j + a ˉ j + \bar{a}_j + \bar{a}_j^{+} a ˉ j + a ˉ j + on those events. Put L j σ = D j σ / w j σ L_{j\sigma} = D_{j\sigma}/w_{j\sigma} L j σ = D j σ / w j σ when w j σ > 0 w_{j\sigma} > 0 w j σ > 0 . The face identifies the side of separation; the width supplies its length scale. For a declared moment order p > 0 p > 0 p > 0 , independent of the cut and field-energy exponents, the same additive regularization gives
V d i r ( p ) ( x ) = ∑ j , σ : w j σ > 0 w j σ L j σ p , R ^ d i r γ , p ( x ) = a ( x ) V d i r ( p ) ( x ) + γ a ( x ) . (33) V_{\mathrm{dir}}^{(p)}(x)=\sum_{j,\sigma:w_{j\sigma}>0}w_{j\sigma}L_{j\sigma}^{p},\qquad\widehat{\mathcal{R}}_{\mathrm{dir}}^{\gamma,p}(x)=\frac{a(x)}{V_{\mathrm{dir}}^{(p)}(x)+\gamma a(x)}. \tag*{(33)} V dir ( p ) ( x ) = j , σ : w j σ > 0 ∑ w j σ L j σ p , R dir γ , p ( x ) = V dir ( p ) ( x ) + γ a ( x ) a ( x ) . (33)
Pooling opposite faces first gives w j = w j − + w j + w_j = w_{j-} + w_{j+} w j = w j − + w j + and D j = D j − + D j + D_j = D_{j-} + D_{j+} D j = D j − + D j + . Applying the power to L j = D j / w j L_j = D_j/w_j L j = D j / w j generally changes the estimate when p ≠ 1 p \ne1 p = 1 . Keeping the faces therefore matters even for a scalar summary. In common coordinate units, V d i r ( p ) V_{\mathrm{dir}}^{(p)} V dir ( p ) and its regularization scale γ \gamma γ have units of length to the power p p p . Taking p = d p = d p = d gives volume units to this mixture of directional lengths; its reference measure and exponent remain declared choices.
Laplacians and Hamiltonians on a Tree
Section 3 asks whether a walk escapes before it returns. To predict a value, supply values at reference vertices and let the walk return the first prescribed value it reaches. An escape probability is the case in which one boundary has value one and the other zero. This hitting functional therefore gives both a probability and a weighted prediction. The chosen energy below specifies the walk’s transition weights; the prescribed boundary values determine which question is being asked.
To couple values across the observations, charge disagreements according to how strongly the geometry connects them. The graph Laplacian records these couplings; its quadratic form is the Hamiltonian. Minimizing that energy over unknown values gives the prediction with the least disagreement allowed by the evidence. This passage from sampled geometry to diffusion energy has a long history. Laplacian eigenmaps and diffusion maps construct weighted interactions from observations and use their Laplacians to describe geometry and diffusion [7 , 19 ] . The sampling measure and the normalization affect which manifold operator a graph approximates [38 ] . Here the declared sample and the maintained tree determine the finite geometry; the affine bounds connect its interaction energy directly to distances between the observed locations.
Shared ancestry explains why a tree can also describe uncertainty. Imagine an uncertain value at the root, followed by an independent, mean-zero change on each edge. Two observations inherit the same changes until their paths part, then acquire separate changes. Their covariance is precisely the variance accumulated on that shared path. A measurement on one branch gives evidence about this shared part of a second branch’s value. Changes on an unobserved branch remain uncertain.
We first condition on the retained sample. Theorem 9 averages the cut law on this fixed sample; each network construction is then computed on one realized tree. Sampling different records changes the geometry as well as the answer, according to the law of § 3 . § 5 puts trees with different retained samples on common reference observations before combining their answers. The Gaussian uncertainty below is conditional uncertainty in values on the chosen geometry.
Additive geometry and interaction. An affinity is a weight coupling two observation values. The tree supplies their unique path and its additive length. To make the affinity independent of path subdivision, require survival through consecutive pieces to multiply. Its length-based factor then satisfies
S ( 0 ) = 1 , 0 < S ( ℓ ) ≤ 1 , S ( a + b ) = S ( a ) S ( b ) . S(0) = 1,\qquad0 < S(\ell) \le1,\qquad S(a+b) = S(a)S(b). S ( 0 ) = 1 , 0 < S ( ℓ ) ≤ 1 , S ( a + b ) = S ( a ) S ( b ) .
For a positive, continuous, nonconstant survival law, taking − log S -\log S − log S gives a continuous additive function. Hence, for some length scale h > 0 h > 0 h > 0 ,
S ( ℓ ) = e − ℓ / h . S(\ell) = e^{-\ell/h}. S ( ℓ ) = e − ℓ / h .
The requirement concerns composition on a fixed geometry. It specifies how the geometric contributions at different levels combine before boundary values or labels are supplied.
On the range tree of (23) , with edge lengths ℓ e \ell_e ℓ e , the pair affinity is
K i j = ∏ e on the path i ↔ j e − ℓ e / h = e − d T ( i , j ) / h . K_{ij} = \prod_{e\text{ on the path }i\leftrightarrow j} e^{-\ell_e/h} = e^{-d_T(i,j)/h}. K ij = e on the path i ↔ j ∏ e − ℓ e / h = e − d T ( i , j ) / h .
The scale h h h is fixed for one field: an affinity falls by a factor e e e over distance h h h . A declared calibration box (Definition 6) can supply that length; § 5.1 sets a common scale across the forest. Changing h h h changes the coupling, while rescaling both h h h and the geometric lengths only changes units.
Range escape and affinity. Sample U U U under the attachment law of § 3.1 , put C x = R ( M r o o t ( x ) ) > 0 C_x = R(M_{\mathrm{root}}(x)) > 0 C x = R ( M root ( x )) > 0 , and let Z = R ( B U ) / C x Z = R(B_U)/C_x Z = R ( B U ) / C x . The affinity at spatial scale h h h is e − λ Z e^{-\lambda Z} e − λ Z , where λ = C x / h \lambda= C_x/h λ = C x / h is the dimensionless inverse scale. Hence
F x ( λ ) = E [ 1 − e − λ Z ] = λ E Z + O ( λ 2 ) . (34) F_x(\lambda) = \mathbb{E}[1-e^{-\lambda Z}] = \lambda\mathbb{E}Z + O(\lambda^2). \tag*{(34)} F x ( λ ) = E [ 1 − e − λ Z ] = λ E Z + O ( λ 2 ) . (34)
The initial slope is range escape (22) ; the finite quantity records affinity lost at the chosen spatial scale. For a declared-mass first-passage trial, use the normalized range Z μ Z_\mu Z μ of its first augmented ancestor with at least μ \mu μ original records: F x , μ ′ ( 0 ) = E S μ ( T , x ) F'_{x,\mu}(0) = \mathbb{E}S_\mu(T,x) F x , μ ′ ( 0 ) = E S μ ( T , x ) . For an unseen query, Corollary 3 controls the expected initial slope through its nearest geometric gap and the affine dimension of the augmented sample.
§ 4.1 computes predictions and responses of a network energy; § 4.2 realizes the complete affinity energy on a sparse tree network.
Shared Ancestry, Prediction, and Equilibrium
A reference set P P P is the collection of observations against which an answer is read. For now these observations have vertices on one tree. Some may supply measured values; in a grouping problem their values may instead be the class indicators being tested. They are not automatically all fixed. A clamp prescribes a value, such as a measurement, a proposed class, or a zero prior mean. A free vertex is one whose value is still to be determined. It may be an unmeasured observation, a query, or an internal vertex carrying the shared part of its descendants’ values. These are values on the retained geometry; solving for them does not move the data or rebuild the cuts.
An edge’s conductance c u v c_{uv} c uv specifies how strongly its two values are coupled. Its reciprocal is resistance: a large resistance permits a larger difference at lower energy cost. Geometric lengths determine these couplings through the constructions below.
Definition 10 (Hamiltonian H \mathcal{H} H and prescribed values). For an undirected network with conductances c u v > 0 c_{uv}>0 c uv > 0 and Laplacian L L L , the Hamiltonian H \mathcal{H} H of the field f f f is its total weighted disagreement energy:
H ( f ) = 1 2 f T L f = 1 2 ∑ { u , v } c u v ( f u − f v ) 2 , (35) \mathcal{H}(f)=\frac{1}{2}f^{\mathsf{T}}Lf=\frac{1}{2}\sum_{\{u,v\}}c_{uv}(f_u-f_v)^2, \tag*{(35)} H ( f ) = 2 1 f T L f = 2 1 { u , v } ∑ c uv ( f u − f v ) 2 , (35) counting each edge once. Prescribe f A = y A f_A=y_A f A = y A at a set of vertices A A A , and write F F F for the remaining vertices. Each connected component must reach a prescribed value to determine its absolute level; a ground supplies a fixed zero value when a prior mean is required. The equilibrium minimizes H \mathcal{H} H over f F f_F f F . For pair affinities K K K on observations P P P , coupling every distinct pair directly with conductance K i j K_{ij} K ij gives the agreement energy
H a f f ( y ) = 1 2 y T L K y = 1 2 ∑ i < j K i j ( y i − y j ) 2 , L K = diag ( K 1 ) − K . \mathcal{H}_{\mathrm{aff}}(y)=\frac{1}{2}y^{\mathsf{T}}L_Ky=\frac{1}{2}\sum_{i<j}K_{ij}(y_i-y_j)^2,\qquad L_K=\operatorname{diag}(K\mathbf{1})-K. H aff ( y ) = 2 1 y T L K y = 2 1 i < j ∑ K ij ( y i − y j ) 2 , L K = diag ( K 1 ) − K . Self-affinities cancel from L K L_K L K .
Theorem 9 (Affine geometry controls the Hamiltonian). Fix a retained sample S S S with at least two distinct locations, write δ i j = ∥ x i − x j ∥ 1 \delta_{ij}=\lVert x_i-x_j\rVert_1 δ ij = ∥ x i − x j ∥ 1 , Δ = diam 1 ( S ) \Delta=\operatorname{diam}_1(S) Δ = diam 1 ( S ) , and let δ ∗ = min δ i j > 0 δ i j \delta_*=\min_{\delta_{ij}>0}\delta_{ij} δ ∗ = min δ ij > 0 δ ij . For s > 0 s>0 s > 0 define the direct distance energy
H S ( 1 ) ( f ) = 1 2 ∑ i < j e − δ i j / s ( f i − f j ) 2 \mathcal{H}^{(1)}_S(f)=\frac{1}{2}\sum_{i<j}e^{-\delta_{ij}/s}(f_i-f_j)^2 H S ( 1 ) ( f ) = 2 1 i < j ∑ e − δ ij / s ( f i − f j ) 2 and let H h T \mathcal{H}^{\mathsf{T}}_h H h T use the tree affinities e − d T ( x i , x j ) / h e^{-d_{\mathsf{T}}(x_i,x_j)/h} e − d T ( x i , x j ) / h at a fixed h > 0 h>0 h > 0 . If S S S lies in an r r r -dimensional affine subspace, put
a = 2 K r , 1 ( 1 / 2 ) [ 1 + ⌈ log 2 ( Δ / δ ∗ ) ⌉ ] . a=2K_{r,1}(1/2)\left[1+\left\lceil\log_2(\Delta/\delta_*)\right\rceil\right]. a = 2 K r , 1 ( 1/2 ) [ 1 + ⌈ log 2 ( Δ/ δ ∗ ) ⌉ ] . For every field f f f , averaging width-proportional trees conditional on S S S gives
H h / a ( 1 ) ( f ) ≤ E T H h T ( f ) ≤ H h ( 1 ) ( f ) . (36) \mathcal{H}^{(1)}_{h/a}(f)\leq\mathbb{E}_{\mathsf{T}}\mathcal{H}^{\mathsf{T}}_h(f)\leq\mathcal{H}^{(1)}_h(f). \tag*{(36)} H h / a ( 1 ) ( f ) ≤ E T H h T ( f ) ≤ H h ( 1 ) ( f ) . (36) Equivalently, the corresponding Laplacians are ordered as quadratic forms. This is a comparison of the averaged energy before minimization. In the noise model of Corollary 1, let r r r be the clean affine dimension and W > 0 W>0 W > 0 . Using the observed distances above, set
t W = max { δ ∗ , 16 W } , b W = 20 r W 1 { δ ∗ < 16 W } , t_W=\max\{\delta_*,16W\},\qquad b_W=20rW\,\mathbf{1}_{\{\delta_*<16W\}}, t W = max { δ ∗ , 16 W } , b W = 20 r W 1 { δ ∗ < 16 W } , a W = 2 K r n o i s e ( 1 / 2 ) [ 1 + ⌈ log 2 max { Δ , t W } t W ⌉ ] . a_W=2K^{\mathrm{noise}}_r(1/2)\left[1+\left\lceil\log_2\frac{\max\{\Delta,t_W\}}{t_W}\right\rceil\right]. a W = 2 K r noise ( 1/2 ) [ 1 + ⌈ log 2 t W max { Δ , t W } ⌉ ] .
Then
e − b W / h H h / a W ( 1 ) ( f ) ≤ E T H T h ( f ) ≤ H h ( 1 ) ( f ) . (37) e^{-b_W/h}\mathcal{H}_{h/a_W}^{(1)}(f) \le\mathbb{E}_T\mathcal{H}_T^h(f) \le\mathcal{H}_h^{(1)}(f). \tag*{(37)} e − b W / h H h / a W ( 1 ) ( f ) ≤ E T H T h ( f ) ≤ H h ( 1 ) ( f ) . (37)
In particular, when all distinct locations are separated by at least 16 W 16W 16 W , the additional attenuation is absent.
Proof. For δ i j > 0 \delta_{ij}>0 δ ij > 0 , Theorem 3 gives d T ( x i , x j ) ≥ δ i j d_T(x_i,x_j)\ge\delta_{ij} d T ( x i , x j ) ≥ δ ij and E T d T ( x i , x j ) ≤ a δ i j \mathbb{E}_T d_T(x_i,x_j)\le a\delta_{ij} E T d T ( x i , x j ) ≤ a δ ij . Since e − z / h e^{-z/h} e − z / h is convex,
e − a δ i j / h ≤ e − E T d T ( x i , x j ) / h ≤ E T e − d T ( x i , x j ) / h ≤ e − δ i j / h . e^{-a\delta_{ij}/h}\le e^{-\mathbb{E}_T d_T(x_i,x_j)/h}\le\mathbb{E}_T e^{-d_T(x_i,x_j)/h}\le e^{-\delta_{ij}/h}. e − a δ ij / h ≤ e − E T d T ( x i , x j ) / h ≤ E T e − d T ( x i , x j ) / h ≤ e − δ ij / h .
Coincident records have distance zero and affinity one in both constructions. Multiply by ( f i − f j ) 2 / 2 (f_i-f_j)^2/2 ( f i − f j ) 2 /2 and sum.
For the noisy case, follow a distinct pair’s common cell until it separates or the cell’s data diameter falls below t W t_W t W . Corollary 1, applied conditionally in blocks, bounds the expected number of these cuts by a W a_W a W . At each such cut the separation probability is δ i j / R ( B ) \delta_{ij}/R(B) δ ij / R ( B ) , so its expected separating-range payoff is δ i j \delta_{ij} δ ij . Summing the stopped payoffs bounds the contribution before this stopping point by a W δ i j a_W\delta_{ij} a W δ ij . If t W = δ ∗ ≥ 16 W t_W=\delta_* \ge16W t W = δ ∗ ≥ 16 W , no distinct pair remains together in a cell of diameter less than t W t_W t W . Otherwise t W = 16 W t_W=16W t W = 16 W . If A A A is the set of records in the stopped cell, every subsequent separating range is at most
R ( bbox A ) ≤ r diam 1 ( A ) + 2 ( r + 1 ) W ≤ ( 18 r + 2 ) W ≤ 20 r W , R(\operatorname{bbox} A)\le r\operatorname{diam}_1(A)+2(r+1)W\le(18r+2)W\le20rW, R ( bbox A ) ≤ r diam 1 ( A ) + 2 ( r + 1 ) W ≤ ( 18 r + 2 ) W ≤ 20 r W ,
by (15). Thus E T d T ( x i , x j ) ≤ a W δ i j + b W \mathbb{E}_T d_T(x_i,x_j)\le a_W\delta_{ij}+b_W E T d T ( x i , x j ) ≤ a W δ ij + b W . Jensen’s inequality and summation prove (37).
The affine theorem therefore bounds the strength of the averaged coupling by changing the spatial scale of a direct distance energy. Below the noise floor, unresolved geometry adds the attenuation e − b W / h e^{-b_W/h} e − b W / h . The comparison uses the same stored observations in every tree; routing a different reference set is the further construction of Section 5.
Information geometry of field responses. Fix the retained observations, bandwidth h h h , and a nonempty prescribed boundary A A A , with nonempty free set F F F . Pool the observation Laplacians as L ˉ h = E T L h T \bar{L}_h=\mathbb{E}_T L_h^T L ˉ h = E T L h T and put Q h = ( L ˉ h ) F F Q_h=(\bar{L}_h)_{FF} Q h = ( L ˉ h ) F F , which is positive definite. At temperature ϑ > 0 \vartheta>0 ϑ > 0 , consider the Gaussian mean family P m = N ( m , ϑ Q h − 1 ) P_m=\mathcal{N}(m,\vartheta Q_h^{-1}) P m = N ( m , ϑ Q h − 1 ) on the free values. For a mean shift ξ ∈ R F \xi\in\mathbb{R}^F ξ ∈ R F , extend ξ \xi ξ by zero on A A A to obtain ξ ~ \widetilde{\xi} ξ . The Fisher metric of this family is
g m ( ξ , ξ ) = 1 ϑ ξ T Q h ξ = 2 ϑ E T H h T ( ξ ~ ) , (38) g_m(\xi,\xi)=\frac{1}{\vartheta}\xi^{\mathsf{T}}Q_h\xi=\frac{2}{\vartheta}\mathbb{E}_T\mathcal{H}_h^T(\widetilde{\xi}), \tag*{(38)} g m ( ξ , ξ ) = ϑ 1 ξ T Q h ξ = ϑ 2 E T H h T ( ξ ) , (38)
and, since the covariance is fixed, KL ( P m ∥ P m + ξ ) = 1 2 g m ( ξ , ξ ) \operatorname{KL}(P_m\mathbin{\|}P_{m+\xi})=\frac{1}{2}g_m(\xi,\xi) KL ( P m ∥ P m + ξ ) = 2 1 g m ( ξ , ξ ) [3 , 2 ] . The field energy therefore measures the statistical distinguishability of mean shifts. Under the affine-support assumptions of Theorem 9,
2 ϑ H h / a ( 1 ) ( ξ ~ ) ≤ g m ( ξ , ξ ) ≤ 2 ϑ H h ( 1 ) ( ξ ~ ) . (39) \frac{2}{\vartheta}\mathcal{H}_{h/a}^{(1)}(\widetilde{\xi})\le g_m(\xi,\xi)\le\frac{2}{\vartheta}\mathcal{H}_h^{(1)}(\widetilde{\xi}). \tag*{(39)} ϑ 2 H h / a ( 1 ) ( ξ ) ≤ g m ( ξ , ξ ) ≤ ϑ 2 H h ( 1 ) ( ξ ) . (39)
The noisy version replaces a a a by a W a_W a W and multiplies the lower bound by e − b W / h e^{-b_W/h} e − b W / h , as in (37). This construction forms the Gaussian law after pooling energies. Affine structure controls tree geometry; tree geometry controls energy; energy controls distinguishability of field responses.
The equilibrium can be found by local agreement: each free value equals the conductance-weighted average of its neighbors. It is also a potential game. A player controls one free value and pays for its disagreement with its neighbors; a unilateral change has exactly the same cost in the player’s objective and in the total energy.
Proposition 10 (A potential game). Let each free vertex u u u be a player that chooses f u ∈ R f_u \in\mathbb{R} f u ∈ R at the cost J u ( f ) = 1 2 ∑ v c u v ( f u − f v ) 2 J_u(f)=\frac{1}{2}\sum_v c_{uv}(f_u-f_v)^2 J u ( f ) = 2 1 ∑ v c uv ( f u − f v ) 2 , with f A = y A f_A=y_A f A = y A .
(i) A change of f u f_u f u alone changes J u J_u J u and H \mathcal{H} H by the same amount, so the game is an exact potential game with potential H \mathcal{H} H [54 ] . Each best response is the conductance-weighted average of the neighbors, and the unique Nash equilibrium [56 ] is the harmonic extension f F ⋆ = − L F F − 1 L F A y A f_F^\star=-L_{FF}^{-1}L_{FA}y_A f F ⋆ = − L F F − 1 L F A y A , the minimizer of H \mathcal{H} H . Equivalently, a walk with transition probabilities c u v / ∑ w c u w c_{uv}/\sum_w c_{uw} c uv / ∑ w c u w returns the prescribed value at its first hit of A A A , and f u ⋆ = E u [ y X τ A ] f_u^\star=\mathbb{E}_u[y_{X_{\tau_A}}] f u ⋆ = E u [ y X τ A ] for u ∈ F u\in F u ∈ F .
(ii) On the tree of Proposition 7, with the query at zero and the root at one, the equilibrium value at the vertex above the query is the escape (19) , and at every vertex of the path it is the probability that a walk from there reaches the root before the query.
Proof. (i) J u J_u J u and H \mathcal{H} H share the terms on the edges at u u u , and no other term of either depends on f u f_u f u . Setting ∂ J u / ∂ f u = ∑ v c u v ( f u − f v ) \partial J_u/\partial f_u=\sum_v c_{uv}(f_u-f_v) ∂ J u / ∂ f u = ∑ v c uv ( f u − f v ) to zero gives the weighted average. Since every component of F F F reaches A A A , L F F L_{FF} L F F is positive definite and H \mathcal{H} H is strictly convex in f F f_F f F . A profile at which no player can lower its own cost has ∂ H / ∂ f u = ∂ J u / ∂ f u = 0 \partial\mathcal{H}/\partial f_u=\partial J_u/\partial f_u=0 ∂ H / ∂ f u = ∂ J u / ∂ f u = 0 for every u u u , so it is the unique minimizer, and the minimizer is such a profile. The first-step equation for the expected boundary value is the same weighted-average equation, with the same prescribed values on A A A , so uniqueness gives the hitting representation. (ii) Branches off the path end in free vertices and carry no current, so on the path the equilibrium is linear in the cumulative resistance L L L : it is L ( w ) / L ( r o o t ) L(w)/L(\mathrm{root}) L ( w ) / L ( root ) at w w w , the probability that the walk from w w w meets the root first, and at the vertex above the query it is (19) .
The matrix G = ( L F F ) − 1 G=(L_{FF})^{-1} G = ( L F F ) − 1 records how these predictions respond to additional evidence. Its column at q q q is the change caused by a unit current applied at q q q , with the prescribed values held fixed. This response matrix is the grounded Green’s function . It too is a hitting functional. Run a continuous-time walk that jumps from u u u to v v v at rate c u v c_{uv} c uv and stops on reaching A A A . Then
G ( u , v ) = E u [ ∫ 0 τ A 1 { X t = v } d t ] . (40) G(u,v)=\mathbb{E}_u\left[\int_0^{\tau_A}1\{X_t=v\}\,dt\right]. \tag*{(40)} G ( u , v ) = E u [ ∫ 0 τ A 1 { X t = v } d t ] . (40)
Its entries count time spent at a specified vertex before the fixed evidence is reached, rather than the whole duration of the walk. The stopped transition matrix is e − t L F F e^{-tL_{FF}} e − t L F F , whose time integral is L F F − 1 L_{FF}^{-1} L F F − 1 . The jump probabilities at departures are those of Proposition 10; the mean holding time at u u u is 1 / ∑ v c u v 1/\sum_v c_{uv} 1/ ∑ v c uv . In particular, the diagonal records time at the query, the same response that will give its model variance. The following construction identifies it directly from shared ancestry.
A grounded network for shared variation. Let T = ( W , E ) T=(W,\mathcal{E}) T = ( W , E ) have positive geometric edge lengths ℓ e \ell_e ℓ e , after zero-length edges are contracted. Put δ u = d T ( r o o t , u ) \delta_u=d_T(\mathrm{root},u) δ u = d T ( root , u ) and ψ ( u ) = e 2 δ u / h \psi(u)=e^{2\delta_u/h} ψ ( u ) = e 2 δ u / h for a declared spatial scale h > 0 h>0 h > 0 . Give the root an independent Gaussian value of variance one; along p → c p\to c p → c add an independent Gaussian increment of variance ψ ( c ) − ψ ( p ) \psi(c)-\psi(p) ψ ( c ) − ψ ( p ) . The corresponding grounded network N N N has root-to-ground resistance one and edge resistance ψ ( c ) − ψ ( p ) \psi(c)-\psi(p) ψ ( c ) − ψ ( p ) . Thus each resistance is the variance of the change carried by that edge. Define
R u v = e − d T ( u , v ) / h , ρ e = e − ℓ e / h , K i j = R i j ( i , j ∈ P ) , b u = ∑ j ∈ P R u j . (41) R_{uv}=e^{-d_T(u,v)/h},\qquad\rho_e=e^{-\ell_e/h},\qquad K_{ij}=R_{ij}\ (i,j\in P),\qquad b_u=\sum_{j\in P}R_{uj}. \tag*{(41)} R uv = e − d T ( u , v ) / h , ρ e = e − ℓ e / h , K ij = R ij ( i , j ∈ P ) , b u = j ∈ P ∑ R u j . (41)
Rescaling each Gaussian value to variance one will give R R R . On the RCF range geometry already defined in (23) , the same construction can use cumulative variance φ ( v ) = e − R ( B v ) / h \varphi(v)=e^{-R(B_v)/h} φ ( v ) = e − R ( B v ) / h . Nested ranges make its increments nonnegative, and each observation has variance one. The identity d T ( i , j ) = R ( B l c a ( i , j ) ) d_T(i,j)=R(B_{\mathrm{lca}(i,j)}) d T ( i , j ) = R ( B lca ( i , j ) ) then gives the observation covariance without a rescaling.
Proposition 11 (The grounded Green’s function). Let L N L_N L N be the Laplacian with the ground held at zero , and G p r i o r = L N − 1 G^{\mathrm{prior}} = L_N^{-1} G prior = L N − 1 . Then G p r i o r ( u , v ) = ψ ( lca ( u , v ) ) G^{\mathrm{prior}}(u,v) = \psi(\operatorname{lca}(u,v)) G prior ( u , v ) = ψ ( lca ( u , v )) and R = E G p r i o r E R = EG^{\mathrm{prior}}E R = E G prior E for E = diag ( e − δ u / h ) E = \operatorname{diag}(e^{-\delta_u/h}) E = diag ( e − δ u / h ) . On the range metric of (23) , use φ ( v ) = e − R ( B v ) / h \varphi(v) = e^{-R(B_v)/h} φ ( v ) = e − R ( B v ) / h in place of ψ \psi ψ , and root resistance φ ( root ) \varphi(\operatorname{root}) φ ( root ) . Then E = diag ( e H v / h ) E = \operatorname{diag}(e^{H_v/h}) E = diag ( e H v / h ) , and G p r i o r G^{\mathrm{prior}} G prior itself restricts to K K K on the observations:
K i j = φ ( lca ( i , j ) ) , φ ( v ) = e − R ( B v ) / h . (42) K_{ij} = \varphi(\operatorname{lca}(i,j)), \qquad\varphi(v) = e^{-R(B_v)/h}. \tag*{(42)} K ij = φ ( lca ( i , j )) , φ ( v ) = e − R ( B v ) / h . (42)
Proof. A unit current from u u u to the ground flows along its path to the root and through the ground edge. The potential at v v v is the resistance of the part of that path shared with the path from v v v to ground. That resistance telescopes to ψ ( lca ( u , v ) ) \psi(\operatorname{lca}(u,v)) ψ ( lca ( u , v )) . The identity for R R R follows from d T ( u , v ) = δ u + δ v − 2 δ lca ( u , v ) d_T(u,v) = \delta_u + \delta_v - 2\delta_{\operatorname{lca}(u,v)} d T ( u , v ) = δ u + δ v − 2 δ lca ( u , v ) . On the range tree the same sum telescopes to φ ( lca ( u , v ) ) \varphi(\operatorname{lca}(u,v)) φ ( lca ( u , v )) , and d T ( u , v ) = 2 H lca ( u , v ) − H u − H v d_T(u,v) = 2H_{\operatorname{lca}(u,v)} - H_u - H_v d T ( u , v ) = 2 H lca ( u , v ) − H u − H v gives the indicated rescaling. Since H i = 0 H_i = 0 H i = 0 for each observation, that rescaling is one on P P P .
In the Gaussian reading, the covariance is the sum of increment variances shared by the two paths. In the electrical reading, a unit current to ground follows the unique path, and its potential on another branch is the shared resistance. These are the same sum. Upward fusion of the evidence and downward prediction therefore compute conditioning on a tree in linear time ([17 ] ). For the range construction, h [ φ ( c ) − φ ( p ) ] → R ( B p ) − R ( B c ) h[\varphi(c) - \varphi(p)] \to R(B_p) - R(B_c) h [ φ ( c ) − φ ( p )] → R ( B p ) − R ( B c ) as h → ∞ h \to\infty h → ∞ . This recovers the resistance increments used for the escapes in § 3.1 ; prescribing the query and root values gives that earlier hitting experiment.
Prediction, uncertainty, and testing a value. Now suppose the references P P P carry measured values y P y_P y P . On the range tree, conditioning this Gaussian model gives simple Kriging ([66 , 64 ] ): the measurements are held fixed, the remaining values are predicted, and the prior ground stays at zero. Equivalently, a walk returns the value of the measurement or ground that it hits first. Independent measurement error of variance ϵ \epsilon ϵ , the Kriging nugget, is represented by a resistance ϵ \epsilon ϵ between each latent reference value and its prescribed measurement. At zero nugget they coincide. The prediction is G p r i o r ( q , P ) ( K + ϵ I ) − 1 y P G^{\mathrm{prior}}(q,P)(K+\epsilon I)^{-1}y_P G prior ( q , P ) ( K + ϵ I ) − 1 y P , and the conditioned Green’s function is
C P , ϵ p r i o r ( u , v ) = G p r i o r ( u , v ) − G p r i o r ( u , P ) ( K + ϵ I ) − 1 G p r i o r ( P , v ) . (43) C^{\mathrm{prior}}_{P,\epsilon}(u,v) = G^{\mathrm{prior}}(u,v) - G^{\mathrm{prior}}(u,P)(K+\epsilon I)^{-1}G^{\mathrm{prior}}(P,v). \tag*{(43)} C P , ϵ prior ( u , v ) = G prior ( u , v ) − G prior ( u , P ) ( K + ϵ I ) − 1 G prior ( P , v ) . (43)
At ϵ = 0 \epsilon= 0 ϵ = 0 one can use the pseudoinverse for redundant coincident measurements. The diagonal C P , ϵ p r i o r ( q , q ) C^{\mathrm{prior}}_{P,\epsilon}(q,q) C P , ϵ prior ( q , q ) is the remaining variance of the latent value at q q q under this model. Electrically it is the effective resistance to the prescribed measurements and ground: a weakly supported value can change substantially in response to a small applied current. If it is positive, clamping q q q a distance δ \delta δ from its equilibrium and letting the other free values respond gives
Δ f ( v ) = δ C P , ϵ p r i o r ( v , q ) C P , ϵ p r i o r ( q , q ) , Δ H min = δ 2 2 C P , ϵ p r i o r ( q , q ) . (44) \Delta f(v) = \delta\frac{C^{\mathrm{prior}}_{P,\epsilon}(v,q)}{C^{\mathrm{prior}}_{P,\epsilon}(q,q)}, \qquad\Delta H_{\min} = \frac{\delta^2}{2C^{\mathrm{prior}}_{P,\epsilon}(q,q)}. \tag*{(44)} Δ f ( v ) = δ C P , ϵ prior ( q , q ) C P , ϵ prior ( v , q ) , Δ H m i n = 2 C P , ϵ prior ( q , q ) δ 2 . (44)
Indeed a unit current at q q q gives the column C P , ϵ p r i o r ( ⋅ , q ) C^{\mathrm{prior}}_{P,\epsilon}(\mathord{\cdot},q) C P , ϵ prior ( ⋅ , q ) ; scaling that current imposes the desired change, and its energy is half the current times the change. Across an edge of conductance c c c , if the far branch has effective conductance a a a to its fixed evidence, the change is multiplied by c / ( c + a ) c/(c+a) c / ( c + a ) : the edge current must equal the current into the branch. A weak connection attenuates a change when the branch has its own evidence. A branch with no fixed evidence follows the change at equilibrium even across a weak edge. The response thus records both the geometry of the gap and the evidence beyond it.
Field fluctuations and Langevin dynamics. Allow the inferred values to fluctuate about the prediction while holding the sampled geometry and prescribed values fixed. In overdamped Langevin dynamics (Pavliotis, 2014, Chapters 3–4), the free values descend the energy while receiving independent noise:
d f F = − ( L F F f F + L F A y A ) d t + 2 ϑ d W t . (45) df_F = -\left(L_{FF}f_F + L_{FA}y_A\right)\,dt + \sqrt{2\vartheta}\,dW_t. \tag*{(45)} d f F = − ( L F F f F + L F A y A ) d t + 2 ϑ d W t . (45)
Here ϑ > 0 \vartheta> 0 ϑ > 0 sets the fluctuation level of the field values and W W W is standard Brownian motion on R F \mathbb{R}^{F} R F . Its stationary law is N ( f F ⋆ , ϑ L F F − 1 ) \mathcal{N}(f_F^\star,\vartheta L_{FF}^{-1}) N ( f F ⋆ , ϑ L F F − 1 ) : the hitting prediction is its mean, and the response matrix is its covariance at unit noise level. Fixing all other values gives conditional variance ϑ / d u \vartheta/d_u ϑ / d u at u u u , where d u = ∑ v c u v d_u=\sum_v c_{uv} d u = ∑ v c uv ; letting the other free values fluctuate gives marginal variance ϑ G ( u , u ) \vartheta G(u,u) ϑG ( u , u ) . The deterministic mean relaxes as e − t L F F e^{-tL_{FF}} e − t L F F . On the grounded Gaussian tree above, unit noise gives the independent increment model itself. Resampling one vertex at a time from its conditional Gaussian law also samples this joint law. The Gaussian reading, the hitting prediction, and the potential game thus describe one specified energy and boundary.
Realizing the Energy of Agreement
For proposed observation values y y y , Definition 10 gives the agreement cost H a f f ( y ) \mathcal{H}_{\mathrm{aff}}(y) H aff ( y ) ; twice this cost is the cut weight when y y y is a class indicator. The cost involves every pair, although the geometry is a tree. The next theorem realizes it by a sparse network.
A separate terminal carries each observation’s prescribed value. The other vertices transmit agreement and are free to settle. Eliminating them means minimizing over their values while retaining the exact energy and response at the observation terminals.
Theorem 12 (Tree realization of Laplace affinities). Give each edge { u , v } ∈ E \{u,v\}\in\mathcal{E} { u , v } ∈ E conductance
c u v = b u b v ρ u v 1 − ρ u v 2 , (46) c_{uv}=\frac{b_u b_v\rho_{uv}}{1-\rho_{uv}^{2}}, \tag*{(46)} c uv = 1 − ρ uv 2 b u b v ρ uv , (46) and attach a separate terminal j ′ j' j ′ to each observation j ∈ P j\in P j ∈ P by an edge of conductance b j b_j b j . Eliminating the free vertices W W W leaves L K L_K L K on the terminal set P ′ = { j ′ : j ∈ P } P'=\{j':j\in P\} P ′ = { j ′ : j ∈ P } . Equivalently, at terminal potentials y y y the minimum energy of the sparse network is
min f P ′ = y H ( f ) = 1 2 y T L K y . (47) \min_{f_{P'}=y}\mathcal{H}(f)=\frac{1}{2}y^{\mathsf{T}}L_Ky. \tag*{(47)} f P ′ = y min H ( f ) = 2 1 y T L K y . (47) Write L a f f L^{\mathrm{aff}} L aff for the Laplacian of the full sparse network. The reduction preserves terminal currents, harmonic equilibria, and effective capacities.
Proof. Let s = 1 P s=\mathbf{1}_P s = 1 P and b = R s b=Rs b = R s , and set g = E − 1 b = G p r i o r E s g=E^{-1}b=G^{\mathrm{prior}}Es g = E − 1 b = G prior E s . The rescaled matrix Q = diag ( g ) L N diag ( g ) Q=\operatorname{diag}(g)L_N\operatorname{diag}(g) Q = diag ( g ) L N diag ( g ) has off-diagonal entry − g c g p / ( ψ ( c ) − ψ ( p ) ) -g_cg_p/(\psi(c)-\psi(p)) − g c g p / ( ψ ( c ) − ψ ( p )) on an edge, the negative of (46). Its row sum at u u u is g u ( L N g ) u = b u s u g_u(L_Ng)_u=b_us_u g u ( L N g ) u = b u s u . The rescaling is a Doob transform: the row sums are the conductances to the new terminals, with no leakage elsewhere. Hence Q Q Q is the interior block of the proposed network. Write D = diag ( b ) D=\operatorname{diag}(b) D = diag ( b ) , so Q − 1 = D − 1 R D − 1 Q^{-1}=D^{-1}RD^{-1} Q − 1 = D − 1 R D − 1 . With H j , u = b j 1 { u = j } H_{j,u}=b_j\mathbf{1}\{u=j\} H j , u = b j 1 { u = j } the terminal-to-interior conductance matrix, the Schur complement is
diag ( b P ) − H Q − 1 H T = diag ( K 1 ) − R P P = L K . \operatorname{diag}(b_P)-HQ^{-1}H^{\mathsf{T}}=\operatorname{diag}(K\mathbf{1})-R_{PP}=L_K. diag ( b P ) − H Q − 1 H T = diag ( K 1 ) − R P P = L K .
Eliminating by minimizing the quadratic energy gives (47). ∎
The resulting terminal matrix is the Schur complement, also called Kron reduction [26 ] ; star–mesh transformations use the same elimination [73 ] . This is an exact representation of the chosen affinity energy, so the equilibrium and the compensating current of a clamp can be computed by tree elimination without forming the complete graph. § 6.1 works it on six observations.
The distinction from the preceding covariance calculation matters. Reducing the original grounded Gaussian tree gives K − 1 K^{-1} K − 1 when K K K is nonsingular. Reducing the transformed, ungrounded network above gives L K L_K L K . The former retains a prior mean and describes prediction error; the latter penalizes differences between observation values. Formula (44) applies to either calculation using its own grounded response matrix and prescribed values.
Total affinity in two passes. The total affinities b u b_u b u measure how much reference mass is coupled to each vertex; they need no pairwise matrix. Root the tree and write U v U_v U v for the affinity mass contributed by observations in the descendant subtree. An upward pass followed by a downward pass gives
U v = 1 { v ∈ P } + ∑ c : c c h i l d o f v ρ v c U c , b r o o t = U r o o t , b v = U v + ρ p v ( b p − ρ p v U v ) . (48) U_v = \mathbf{1}\{v \in P\} + \sum_{c:\,c\ \mathrm{child\ of}\ v} \rho_{vc} U_c,\qquad b_{\mathrm{root}} = U_{\mathrm{root}},\qquad b_v = U_v + \rho_{pv}(b_p - \rho_{pv}U_v). \tag*{(48)} U v = 1 { v ∈ P } + c : c child of v ∑ ρ v c U c , b root = U root , b v = U v + ρ p v ( b p − ρ p v U v ) . (48)
Hence (46) is available after linear work in ∣ W ∣ |W| ∣ W ∣ .
Changing the geometric ruler. The realization uses distances on the retained tree, so the same cuts can support another choice of geometric lengths. Let w v = ( R j ( B v ) ) j = 1 d w_v = (R_j(B_v))_{j=1}^{d} w v = ( R j ( B v ) ) j = 1 d be the coordinate widths of a stored box. Using heights H v = ∥ w v ∥ 2 / 2 H_v = \lVert w_v\rVert_2/2 H v = ∥ w v ∥ 2 /2 gives nonnegative edge lengths, since the boxes are nested. With singleton-location leaves, the tree distance of stored observations is then ∥ w lca ( i , j ) ∥ 2 \lVert w_{\operatorname{lca}(i,j)}\rVert_2 ∥ w lca ( i , j ) ∥ 2 , which dominates their Euclidean distance. The contribution of coordinate j j j to the edge from parent p p p to child v v v is
( w p j + w v j ) ( w p j − w v j ) 2 ( ∥ w p ∥ 2 + ∥ w v ∥ 2 ) . (49) \frac{(w_{pj}+w_{vj})(w_{pj}-w_{vj})}{2(\lVert w_p\rVert_2+\lVert w_v\rVert_2)}. \tag*{(49)} 2 (∥ w p ∥ 2 + ∥ w v ∥ 2 ) ( w p j + w v j ) ( w p j − w v j ) . (49)
The entries are nonnegative and sum to H p − H v H_p-H_v H p − H v . Splitting each width decrement into the two occupied half-width decrements of § 7.1 retains all 2 d 2d 2 d directions. Recomputing affinities from these lengths gives another field on the same maintained hierarchy; Theorem 12 still supplies its exact sparse network.
The quadratic power in the field energy acts on differences of values independently of this spatial ruler. § 6.4 uses it to read C o D i s p \sqrt{\mathrm{CoDisp}} CoDisp as a relative field-change magnitude.
Energy within Nested Neighborhoods
The tree realization explains why the dense energy can be solved sparsely. Its nested neighborhoods also explain what the energy charges for, and how a proposed grouping can be evaluated by counts. We use the RCF range metric of (23) throughout, with φ ( v ) = e − R ( B v ) / h \varphi(v)=e^{-R(B_v)/h} φ ( v ) = e − R ( B v ) / h ; two observations have affinity φ ( lca ( i , j ) ) \varphi(\operatorname{lca}(i,j)) φ ( lca ( i , j )) . Write P v P_v P v for the reference observations below v v v and n v n_v n v for their number. Here the tree retains the occupied descendant sets; § 5.1 constructs this representation when the references are routed through a separately sampled tree.
Put K i i = 1 K_{ii}=1 K ii = 1 , the affinity of an observation with itself at its leaf, and L K = diag ( K 1 ) − K L_K=\operatorname{diag}(K\mathbf{1})-K L K = diag ( K 1 ) − K , in which the diagonal cancels. For a vertex v v v , its degree is the total affinity of a new observation leaf joining at v v v . It couples to descendants at φ ( v ) \varphi(v) φ ( v ) and to the remaining references through the ancestors of v v v :
deg ( v ) = ∑ j ∈ P φ ( lca ( v , j ) ) = φ ( v ) n v + Out v , Out r o o t = 0 , Out c = Out v + φ ( v ) ( n v − n c ) , (50) \begin{aligned}
\operatorname{deg}(v) &= \sum_{j\in P} \varphi(\operatorname{lca}(v,j)) = \varphi(v)n_v+\operatorname{Out}_v,\\
\operatorname{Out}_{\mathrm{root}} &= 0,\qquad\operatorname{Out}_c = \operatorname{Out}_v+\varphi(v)(n_v-n_c), \tag*{(50)}
\end{aligned} deg ( v ) Out root = j ∈ P ∑ φ ( lca ( v , j )) = φ ( v ) n v + Out v , = 0 , Out c = Out v + φ ( v ) ( n v − n c ) , (50)
where c c c is a child of v v v , lca ( v , j ) \operatorname{lca}(v,j) lca ( v , j ) is the lowest common ancestor of v v v and the leaf of j j j , and φ = 1 \varphi=1 φ = 1 at a leaf. One upward pass for the counts and one downward pass for Out \operatorname{Out} Out give every degree; the degree of a leaf is the row sum of K K K for each reference at that leaf.
The same counts construct the network. Since d T ( v , j ) = 2 H lca ( v , j ) − H v d_T(v,j)=2H_{\operatorname{lca}(v,j)}-H_v d T ( v , j ) = 2 H lca ( v , j ) − H v , the total affinities and conductances of Theorem 12 are determined by the degrees (50):
b v = e H v / h deg ( v ) , c p c = deg ( p ) deg ( c ) φ ( c ) − φ ( p ) , (51) b_v=e^{H_v/h}\operatorname{deg}(v),\qquad c_{pc}=\frac{\operatorname{deg}(p)\operatorname{deg}(c)}{\varphi(c)-\varphi(p)}, \tag*{(51)} b v = e H v / h deg ( v ) , c p c = φ ( c ) − φ ( p ) deg ( p ) deg ( c ) , (51)
for every vertex v v v and every edge from a child c c c to its parent p p p . At the leaf of an observation j j j that is alone there, H = 0 H=0 H = 0 and b j = 1 + ∑ i ≠ j K i j b_j=1+\sum_{i\ne j}K_{ij} b j = 1 + ∑ i = j K ij . Observations that share a leaf u u u hang from it by separate pendant edges, each of conductance deg ( u ) \operatorname{deg}(u) deg ( u ) : the proof of Theorem 12 goes through with s s s counting the observations at each vertex. One array of degrees, computed by the two passes of (50), therefore gives the sparse network whose Schur complement is L K L_K L K , and each induced tree of § 5.1 has one.
Proposition 13 (Computing with nested neighborhoods). Use φ ( v ) = e − R ( B v ) / h \varphi(v)=e^{-R(B_v)/h} φ ( v ) = e − R ( B v ) / h and K i j = φ ( lca ( i , j ) ) K_{ij}=\varphi(\operatorname{lca}(i,j)) K ij = φ ( lca ( i , j )) on the range tree of (23), with φ = 1 \varphi=1 φ = 1 at observation leaves. Write par ( v ) \operatorname{par}(v) par ( v ) for the parent of v v v , with φ ( par ( r o o t ) ) = 0 \varphi(\operatorname{par}(\mathrm{root}))=0 φ ( par ( root )) = 0 . Then
K = ∑ v ( φ ( v ) − φ ( par ( v ) ) ) 1 P v 1 P v T , (52) K=\sum_v\bigl(\varphi(v)-\varphi(\operatorname{par}(v))\bigr)\mathbf{1}_{P_v}\mathbf{1}_{P_v}^{\mathsf{T}}, \tag*{(52)} K = v ∑ ( φ ( v ) − φ ( par ( v )) ) 1 P v 1 P v T , (52) the sum running over all vertices, leaves included. Consequently:
(i) K K K is positive semidefinite.
(ii) For x ∈ R P × r x\in\mathbb{R}^{P\times r} x ∈ R P × r , let X v X_v X v sum the rows of x x x over P v P_v P v , upward, and put O r o o t = 0 O_{\mathrm{root}}=0 O root = 0 and O c = O v + φ ( v ) X c ′ O_c=O_v+\varphi(v)X_{c'} O c = O v + φ ( v ) X c ′ for each child c c c of v v v with sibling c ′ c' c ′ , downward. Then ( K x ) i = X u + O u (Kx)_i=X_u+O_u ( K x ) i = X u + O u for i i i at leaf u u u . The work is O ( r ∣ T ∣ ) O(r|T|) O ( r ∣ T ∣ ) , and x = 1 x=\mathbf{1} x = 1 gives the degrees (50).
(iii) For a labeling of P P P into k k k classes, let n v ( c ) n_v(c) n v ( c ) count the observations of class c c c in P v P_v P v . Pair each internal vertex v v v with the ordered pairs ( a , b ) (a,b) ( a , b ) and ( b , a ) (b,a) ( b , a ) of its children, and each leaf u u u with ( u , u ) (u,u) ( u , u ) . The cut and volume of class c c c ,
cut ( c ) = ∑ v φ ( v ) ∑ ( a , b ) n a ( c ) ( n b − n b ( c ) ) , vol ( c ) = ∑ i : c ( i ) = c ( K 1 ) i , \operatorname{cut}(c)=\sum_v\varphi(v)\sum_{(a,b)}n_a(c)\bigl(n_b-n_b(c)\bigr),\qquad
\operatorname{vol}(c)=\sum_{i:c(i)=c}(K\mathbf{1})_i, cut ( c ) = v ∑ φ ( v ) ( a , b ) ∑ n a ( c ) ( n b − n b ( c ) ) , vol ( c ) = i : c ( i ) = c ∑ ( K 1 ) i , follow from one upward pass of the counts, in O ( k ∣ T ∣ ) O(k|T|) O ( k ∣ T ∣ ) .
Proof. For i ≠ j i\ne j i = j the vertices whose sets contain both are the ancestors of lca ( i , j ) \operatorname{lca}(i,j) lca ( i , j ) , itself included, and the coefficients in (52) telescope to φ ( lca ( i , j ) ) \varphi(\operatorname{lca}(i,j)) φ ( lca ( i , j )) ; for i = j i=j i = j they telescope to the value one at the leaf. Every coefficient is nonnegative because φ \varphi φ does not decrease downward, so K K K is a nonnegative combination of positive semidefinite matrices. For (ii), the observations j j j with lca ( i , j ) = v \operatorname{lca}(i,j)=v lca ( i , j ) = v , for v v v a proper ancestor of the leaf u u u of i i i , are those below the child of v v v off the path to u u u , and those at u u u itself have affinity one; summing φ ( v ) \varphi(v) φ ( v ) times their sums along the path is the downward recursion. For (iii), group the ordered pairs ( i , j ) (i,j) ( i , j ) with c ( i ) = c ≠ c ( j ) c(i)=c\ne c(j) c ( i ) = c = c ( j ) by lca ( i , j ) \operatorname{lca}(i,j) lca ( i , j ) .
The nonnegative block weights make K K K a valid covariance, and this property survives averaging. More directly, the two passes give the interactions needed for a proposed assignment without writing the pairwise matrix. The argument uses the retained descendant sets; § 5.1 constructs them for references routed through a tree that was built from another sample. For weighted observations, use weighted sums and the product of the two weights in each pair’s energy, including for time-decay weights. The same formulas hold for any declared node affinity whose increments are nonnegative, as they are automatically for nested box ranges.
Which neighborhoods charge for disagreement. The block decomposition (52) expresses the affinity energy as disagreement within the retained neighborhoods. Let f ˉ v \bar{f}_v f ˉ v be the mean potential on the n v n_v n v observations below v v v . Then
H T ( f ) = 1 2 ∑ v ( φ ( v ) − φ ( par ( v ) ) ) n v ∑ i ∈ P v ( f i − f ˉ v ) 2 . (53) \mathcal{H}_T(f)=\frac{1}{2}\sum_v\bigl(\varphi(v)-\varphi(\operatorname{par}(v))\bigr)n_v\sum_{i\in P_v}(f_i-\bar{f}_v)^2. \tag*{(53)} H T ( f ) = 2 1 v ∑ ( φ ( v ) − φ ( par ( v )) ) n v i ∈ P v ∑ ( f i − f ˉ v ) 2 . (53)
Indeed the Laplacian of the block 1 P v 1 P v T \mathbf{1}_{P_v}\mathbf{1}_{P_v}^{\mathsf{T}} 1 P v 1 P v T has quadratic form n v ∑ i ∈ P v ( f i − f ˉ v ) 2 n_v\sum_{i\in P_v}(f_i-\bar{f}_v)^2 n v ∑ i ∈ P v ( f i − f ˉ v ) 2 . For φ ( v ) = e − R ( B v ) / h \varphi(v)=e^{-R(B_v)/h} φ ( v ) = e − R ( B v ) / h , draw a resolution from Exp ( 1 / h ) \operatorname{Exp}(1/h) Exp ( 1/ h ) and take the maximal boxes whose ranges are at most that resolution. A box is selected with probability φ ( v ) − φ ( par ( v ) ) \varphi(v)-\varphi(\operatorname{par}(v)) φ ( v ) − φ ( par ( v )) , the chance that the resolution lies between its range and its parent’s. The energy is half the expected mass-weighted variation within these boxes. This is where the cut geometry enters a field calculation: it selects the neighborhoods within which disagreement costs energy.
What each merge contributes. Write c v = n v [ φ ( v ) − φ ( par ( v ) ) ] c_v=n_v[\varphi(v)-\varphi(\operatorname{par}(v))] c v = n v [ φ ( v ) − φ ( par ( v ))] . The affinity operator is a finite hierarchical Laplacian ([8 ] ): if each observation in P v P_v P v is refreshed at rate c v c_v c v by a draw uniform on P v P_v P v , then i i i jumps to j ≠ i j\ne i j = i at rate ∑ v : i , j ∈ P v c v / n v = φ ( lca ( i , j ) ) = K i j \sum_{v:i,j\in P_v}c_v/n_v=\varphi(\operatorname{lca}(i,j))=K_{ij} ∑ v : i , j ∈ P v c v / n v = φ ( lca ( i , j )) = K ij , so − L K -L_K − L K is the generator, and its Dirichlet energy is (53). Split each within-box variation at the merge of the two children a , b a,b a , b of v v v ,
∑ i ∈ P v ( f i − f ˉ v ) 2 = ∑ i ∈ P a ( f i − f ˉ a ) 2 + ∑ i ∈ P b ( f i − f ˉ b ) 2 + n a n b n v ( f ˉ a − f ˉ b ) 2 . \sum_{i\in P_v}(f_i-\bar{f}_v)^2=\sum_{i\in P_a}(f_i-\bar{f}_a)^2+\sum_{i\in P_b}(f_i-\bar{f}_b)^2+\frac{n_an_b}{n_v}(\bar{f}_a-\bar{f}_b)^2. i ∈ P v ∑ ( f i − f ˉ v ) 2 = i ∈ P a ∑ ( f i − f ˉ a ) 2 + i ∈ P b ∑ ( f i − f ˉ b ) 2 + n v n a n b ( f ˉ a − f ˉ b ) 2 .
and collect the coefficient of each merge, ∑ w ⪰ v c w = deg ( v ) \sum_{w\succeq v}c_w=\operatorname{deg}(v) ∑ w ⪰ v c w = deg ( v ) by (50), the sum over v v v and its ancestors. With one observation at each leaf,
2 H T ( f ) = ∑ v : a , b c h i l d r e n deg ( v ) n a n b n v ( f ˉ a − f ˉ b ) 2 ; (54) 2\mathcal{H}_T(f)=\sum_{v:a,b\ \mathrm{children}}\operatorname{deg}(v)\frac{n_an_b}{n_v}(\bar{f}_a-\bar{f}_b)^2; \tag*{(54)} 2 H T ( f ) = v : a , b children ∑ deg ( v ) n v n a n b ( f ˉ a − f ˉ b ) 2 ; (54)
a leaf u u u holding several adds deg ( u ) \operatorname{deg}(u) deg ( u ) times its own within-leaf variation. The factor after deg ( v ) \operatorname{deg}(v) deg ( v ) is the increase in the within-group sum of squares when the two children merge, the criterion of Ward [74 ] . For f = 1 A f=\mathbf{1}_A f = 1 A the child means are the shares of A A A in each child, and the left side is the numerator of the normalized cut (66) : each merge charges for how differently its two children support membership in A A A , weighted by the affinity of a virtual observation placed at the merge.
Relaxation across a merge. Consider the observation energy H T ( f ) = 1 2 f T L K f \mathcal{H}_T(f)=\frac{1}{2}f^{\mathsf{T}}L_Kf H T ( f ) = 2 1 f T L K f with one field coordinate per record. For the children a , b a,b a , b of v v v , the normalized difference of their means is
A v ( t ) = n a n b n v ( f ˉ a ( t ) − f ˉ b ( t ) ) . A_v(t)=\sqrt{\frac{n_an_b}{n_v}}\bigl(\bar{f}_a(t)-\bar{f}_b(t)\bigr). A v ( t ) = n v n a n b ( f ˉ a ( t ) − f ˉ b ( t ) ) .
Its contribution in (54) is 1 2 deg ( v ) A v 2 \frac{1}{2}\deg(v)A_v^2 2 1 deg ( v ) A v 2 . The neighborhood averages within either child leave this contrast unchanged, whereas averaging over v v v or an ancestor sets it to zero. Thus the heat flow generated by − L K -L_K − L K relaxes this contrast at the rate
E A v ( t ) = e − t deg ( v ) A v ( 0 ) , deg ( v ) = ∑ w ≻ v c w . (55) \mathbb{E}A_v(t)=e^{-t\deg(v)}A_v(0),\qquad\deg(v)=\sum_{w\succ v}c_w. \tag*{(55)} E A v ( t ) = e − t d e g ( v ) A v ( 0 ) , deg ( v ) = w ≻ v ∑ c w . (55)
For Langevin dynamics of these free observation values, projecting (45) onto the normalized contrast gives
d A v ( t ) = − deg ( v ) A v ( t ) d t + 2 ϑ d B v ( t ) , dA_v(t)=-\deg(v)A_v(t)\,dt+\sqrt{2\vartheta}\,dB_v(t), d A v ( t ) = − deg ( v ) A v ( t ) d t + 2 ϑ d B v ( t ) ,
where B v B_v B v is standard Brownian motion. This is an Ornstein–Uhlenbeck process: its relaxation time is 1 / deg ( v ) 1/\deg(v) 1/ deg ( v ) and its stationary variance is ϑ / deg ( v ) \vartheta/\deg(v) ϑ / deg ( v ) . The common offset requires anchoring. Fixing individual observation values changes the contrast dynamics; the conditioned response in § 4.1 then gives the uncertainty.
These relaxation rates belong to the observation Laplacian L K L_K L K . The sparse realization of Theorem 12 preserves its equilibrium responses; running dynamics on all auxiliary vertices need not preserve the same transient rates.
Updating the energy as the data change. An insertion or deletion changes the stored boxes and masses along its path. The new box ranges determine the affinities. Once the references are located, two passes for the updated counts and degrees give the sparse-network conductances and merge relaxation rates. One array therefore links the current geometry, its predictions, and its dynamics.
For a committed insertion, the effect on existing affinities is explicit. At fixed h h h , attach x x x above v v v and put G w = R ( B w ∪ { x } ) − R ( B w ) G_w=R(B_w\cup\{x\})-R(B_w) G w = R ( B w ∪ { x }) − R ( B w ) . An old stored pair keeps its lowest common ancestor w w w : its affinity is multiplied by e − G w / h e^{-G_w/h} e − G w / h when w w w is strictly above v v v , and is unchanged otherwise. Insertion above the old root therefore deepens the old leaves while preserving their pairwise affinities. With s w = P ( v strictly below w ) s_w=\mathbb{P}(v\text{ strictly below }w) s w = P ( v strictly below w ) under the separation law, the expected old-pair affinity at w w w is
e − R ( B w ) / h [ 1 − s w ( 1 − e − G w / h ) ] . e^{-R(B_w)/h}\left[1-s_w\left(1-e^{-G_w/h}\right)\right]. e − R ( B w ) / h [ 1 − s w ( 1 − e − G w / h ) ] .
It retains the nested form of Proposition 13. The expected loss summed over unordered pairs of stored copies is
∑ w on the query path m w ′ m w ′ ′ e − R ( B w ) / h s w ( 1 − e − G w / h ) , \sum_{w\text{ on the query path}}m_{w'}m_{w''}e^{-R(B_w)/h}s_w\left(1-e^{-G_w/h}\right), w on the query path ∑ m w ′ m w ′′ e − R ( B w ) / h s w ( 1 − e − G w / h ) ,
where w ′ w' w ′ , w ′ ′ w'' w ′′ are the children of w w w . Thus the changed stored boxes lie on the insertion path. External references can also be rerouted by the new cut. Insertion can change the network support, which the conductance comparisons of § 7.4 hold fixed.
Even a fixed mode with drift − λ ( A v − m ( t ) ) -\lambda(A_v-m(t)) − λ ( A v − m ( t )) toward a moving prescribed value m ( t ) m(t) m ( t ) can lag behind it. For λ = deg ( v ) > 0 \lambda=\deg(v)>0 λ = deg ( v ) > 0 , the mean error e ( t ) = E A v ( t ) − m ( t ) e(t)=\mathbb{E}A_v(t)-m(t) e ( t ) = E A v ( t ) − m ( t ) obeys e ′ = − λ e − m ′ e'=-\lambda e-m' e ′ = − λ e − m ′ . When ∣ m ′ ∣ ≤ v 0 |m'|\leq v_0 ∣ m ′ ∣ ≤ v 0 ,
∣ e ( t ) ∣ ≤ e − λ t ∣ e ( 0 ) ∣ + v 0 λ ( 1 − e − λ t ) . |e(t)|\leq e^{-\lambda t}|e(0)|+\frac{v_0}{\lambda}\left(1-e^{-\lambda t}\right). ∣ e ( t ) ∣ ≤ e − λ t ∣ e ( 0 ) ∣ + λ v 0 ( 1 − e − λ t ) .
Tracking changes of the hierarchy further requires a common representation of the values and control of the changing equilibrium. § 5 places the trees on common reference observations; § 7.4 bounds the response to finite conductance changes. § 5.1 gives a residual bound for reusing a field on the new references.
Landmarks, Routing, and Normalized Cuts
In this section we discuss streaming segmentation based on energy and Laplacians; in particular we focus on normalized cuts. The challenge is to create a common reference set P P P , often called landmarks, which supplies the observations being compared. When reference values are supplied, the covariance interpretation gives Kriging, or Gaussian process prediction [66 , 64 ] . For grouping, the affinities define a Laplacian on the same observations. Routing P P P gives nested reference sets and counts within each hierarchy. Inherited box ranges, measured at a common spatial scale, give the affinities and degrees; their average defines one energy on P P P . Its leading spectral modes are computed iteratively from tree products, and Nyström extends the representation to new observations [28 ] . Recursive landmarks and multi-centroid representations have been used in stream segmentation through weighted center summaries [34 , 35 ] . Here the RCF supplies the geometry used both to group the references and to read a new observation, providing an alternative based on the maintained energy.
Section 5.1 constructs that geometry. Section 5.2 reads separation, support, and pooled prediction uncertainty. Section 5.3 evaluates normalized cuts and extends a fixed reference partition. Algorithm 1 collects the construction and assignment interface. Section 5.4 then distinguishes what a query may change, and Section 6 attributes boundary costs and tests changes of membership. Every tree calculation remains exact within its own hierarchy; partition optimization uses the chosen normalized-cut procedure.
Putting the Trees on Common Observations
The reference set P P P contains the observations we want to compare, group, or use as prediction evidence. It can be chosen from a current sample or supplied separately. It need not be the sample that built any one tree, and its members need not have measured response values. Routing sends each reference through the existing split decisions to its reached leaf, as in ordinary tree prediction [15 , 13 ] . The cuts and stored boxes remain fixed. Repeating this in every tree places its affinities on the same observations. For distinct references i , j i,j i , j , let v t ( i , j ) v_t(i,j) v t ( i , j ) be their parting vertex in tree t t t and define
K t ( i , j ) = φ t ( v t ( i , j ) ) , φ t ( v ) = e − R ( B v t ) / h , (56) K_t(i,j)=\varphi_t(v_t(i,j)),\qquad\varphi_t(v)=e^{-R(B_v^t)/h}, \tag*{(56)} K t ( i , j ) = φ t ( v t ( i , j )) , φ t ( v ) = e − R ( B v t ) / h , (56)
Here B v t B_v^t B v t is the data box of v v v in tree t t t , computed from that tree’s own sample. Set K t ( i , j ) = 1 K_t(i,j)=1 K t ( i , j ) = 1 when i i i and j j j reach the same leaf, including i = j i=j i = j . The forest affinity on P P P is
K ˉ = 1 N ∑ t = 1 N K t , d ˉ = K ˉ 1 , L ˉ = diag ( d ˉ ) − K ˉ . \bar{K}=\frac{1}{N}\sum_{t=1}^{N}K_t,\qquad\bar{d}=\bar{K}\mathbf{1},\qquad\bar{L}=\operatorname{diag}(\bar{d})-\bar{K}. K ˉ = N 1 t = 1 ∑ N K t , d ˉ = K ˉ 1 , L ˉ = diag ( d ˉ ) − K ˉ .
The sets of references routed through descendants are exactly nested along each path. Their membership is determined by the partition cuts, so a routed point can lie outside the data box of a vertex it passes. For example, in one dimension two points arbitrarily far to the right of every stored sample reach the same extreme leaf in every tree, giving affinity one regardless of their separation. The randomized passive placement uses the query’s enlargement of the attachment box to measure support outside the sample.
Lemma 2 (Induced trees). Route P P P through tree t t t , and keep the vertices at which points of P P P part and the leaves they reach. This induced tree T t ( P ) T_t(P) T t ( P ) has at most $2|P|-1$ vertices, its leaves hold the points of P P P that reach a common leaf of t t t , and K t K_t K t has the form of Proposition 13 on it, with $\varphi_t
inherited from t t t and one at its leaves. Hence K t ⪰ 0 K_t \succeq0 K t ⪰ 0 , K ˉ ⪰ 0 \bar{K} \succeq0 K ˉ ⪰ 0 with unit diagonal, every point has positive affinity to every other, and a product K ˉ x \bar{K}x K ˉ x with r r r columns costs O ( N r ∣ P ∣ ) O(Nr|P|) O ( N r ∣ P ∣ ) once the induced trees are built. With the conductances (51) computed from the degrees of T t ( P ) T_t(P) T t ( P ) , Theorem 12 realizes each L t L_t L t exactly.
Proof. A vertex of T t ( P ) T_t(P) T t ( P ) is a vertex of t t t , and a descendant in T t ( P ) T_t(P) T t ( P ) is a descendant in t t t , whose data box is nested in its ancestor’s; so φ t \varphi_t φ t does not decrease down T t ( P ) T_t(P) T t ( P ) , and K t ( i , j ) = φ t ( lca T t ( P ) ( i , j ) ) K_t(i,j) = \varphi_t(\operatorname{lca}_{T_t(P)}(i,j)) K t ( i , j ) = φ t ( lca T t ( P ) ( i , j )) by the definition of v t v_t v t . Every internal vertex of T t ( P ) T_t(P) T t ( P ) splits its points into two nonempty sets, which bounds the size. Positivity of the affinities follows from φ t > 0 \varphi_t > 0 φ t > 0 . The rest is Proposition 13 and (51) applied to T t ( P ) T_t(P) T t ( P ) .
The induced tree uses the original cut and box at each retained vertex. Its reference counts n v = ∣ P v ∣ n_v = |P_v| n v = ∣ P v ∣ are computed after routing, while the retained boxes still describe the tree’s construction sample. The reference counts of § 4.3 and the stored sample masses m v m_v m v therefore have separate roles. An out-of-sample row uses the same paths. For a query q q q , each sibling that parts from its path supplies its references at the parting affinity, and references at its leaf have affinity one. This gives the row and its products with reference vectors in one traversal per tree, using subtree sums when only the products are needed. Routing can assign affinity one to two points outside the sample that reach the same leaf. The randomized placement of § 5.1 additionally measures enlargement of the attachment box.
A common spatial scale. The scale h h h is fixed for all pairs being compared. The calibration boxes of § 3.3 offer a choice tied to mass: take the median, over trees and reference observations, of the calibration range at mass μ ( θ ) \mu(\theta) μ ( θ ) in (28) . This uses the same declared neighborhood mass as the ratios of § 8.1 . A query-specific passage box supplies a local scale for that query; a common affinity field requires a common choice of h h h . The experiments use h h h equal to five percent of the median root range of the selected trees. The same positive spatial scale is used in the trees being compared. Compute it once for a reference model and share it across that model’s queries. Across updates, retaining h h h fixes the comparison scale; reselecting it changes that scale.
Moving references and reusing computations. The sampling target matters. For repeated data D s = g s ( D ) D_s = g_s(D) D s = g s ( D ) with known point identities, one can sample identities P ⊆ D P \subseteq D P ⊆ D once and move their coordinates as P s = g s ( P ) P_s = g_s(P) P s = g s ( P ) . A uniform identity sample remains uniform over the current indexed records. A reservoir over arriving records instead samples the observed history [72 ] ; these are different reference laws. § 3 allows either, once declared.
Cached routes remain valid while their split decisions remain valid. At an unchanged cut x j ≥ τ x_j \ge\tau x j ≥ τ , the sufficient condition ∣ Δ x j ∣ < ∣ x j − τ ∣ |\Delta x_j| < |x_j-\tau| ∣Δ x j ∣ < ∣ x j − τ ∣ certifies that a moving point stays on its old side. A changed cut or a structural update requires rechecking the affected route. The route and its affinity values are separate: box ranges, bandwidth, reference counts, and class or spectral sums can change even when the route does not. Rotation preserves Euclidean distances but generally changes axis-parallel box ranges.
Previous partitions, spectral vectors, or fields can initialize the next solve after identifying or evaluating them on the new references. For a current grounded field system L F F f F ∗ = b L_{FF}f_F^* = b L F F f F ∗ = b , let g g g be such an initial field and r = b − L F F g r = b-L_{FF}g r = b − L F F g its current residual. If L F F ⪰ λ I L_{FF} \succeq\lambda I L F F ⪰ λ I , λ > 0 \lambda> 0 λ > 0 , then
∥ g − f F ∗ ∥ 2 ≤ ∥ r ∥ 2 λ , ∥ E f ( t ) − f F ∗ ∥ 2 ≤ e − λ t ∥ r ∥ 2 λ when E f ( 0 ) = g . (57) \lVert g-f_F^*\rVert_2 \le\frac{\lVert r\rVert_2}{\lambda}, \qquad\lVert\mathbb{E}_f(t)-f_F^*\rVert_2 \le e^{-\lambda t}\frac{\lVert r\rVert_2}{\lambda} \quad\text{when } \mathbb{E}_f(0)=g. \tag*{(57)} ∥ g − f F ∗ ∥ 2 ≤ λ ∥ r ∥ 2 , ∥ E f ( t ) − f F ∗ ∥ 2 ≤ e − λ t λ ∥ r ∥ 2 when E f ( 0 ) = g . (57)
The first bound follows from f F ∗ − g = L F F − 1 F R f_F^* - g = L_{FF}^{-1}F_R f F ∗ − g = L F F − 1 F R ; the second applies the Langevin mean relaxation of (45) with the current operator held fixed between updates. Thus the residual measures what must be corrected, and the restoring rate determines how much relaxation is needed.
Randomized placement of a passive probe. The simplest passive placement puts q q q at the leaf reached by routing x x x . For a randomized placement, draw the separation vertex v v v as in Section 3.1 and put H = R ( B v ∪ { x } ) / 2 H = R(B_v \cup\{x\})/2 H = R ( B v ∪ { x }) /2 . Place an attachment vertex at height H H H on the ancestral path above v v v , extending it above the root if needed. Attach q q q there as a height-zero leaf through an edge of length H H H . If a reference j j j parts from this path at w j w_j w j , with w j = v w_j = v w j = v for descendants of v v v , its passive row entry is
K q j = exp ( − max { R ( B v ∪ { x } ) , R ( B w j ) } h ) . K_{qj} = \exp\left(-\frac{\max\{R(B_v \cup\{x\}), R(B_{w_j})\}}{h}\right). K q j = exp ( − h max { R ( B v ∪ { x }) , R ( B w j )} ) .
Subdividing the path preserves all old pairwise distances. The first contact scale D ∗ D_* D ∗ used in the next subsection is the minimum of these ranges over the reference set. With a reference below v v v it is R ( B v ∪ { x } ) R(B_v \cup\{x\}) R ( B v ∪ { x }) , the same augmented range used in range escape. A query equal to a stored location uses its leaf, contracting the zero-length attachment.
For a fixed query, adjoining x x x to every box on a chain preserves containment: the box enclosing B v ∪ { x } B_v \cup\{x\} B v ∪ { x } is contained in the one enclosing B p ∪ { x } B_p \cup\{x\} B p ∪ { x } whenever v v v is a child of p p p . Subdivision preserves the descendant hierarchy of the references. A clamp fixes values with the tree and gauge held fixed. As the query, its placement, or the tree changes, the neighborhoods being compared can belong to different chains. Exact identities are applied within a realization before the responses or affinities are pooled.
Separation, Support, and Pooling
The trees can agree about a prediction while disagreeing about its support. A query may join the references at a small box in one realization and only at a large box in another. We now keep the reference set fixed and average over both the tree and any randomized placement. A mean separation scale alone loses the distinction between these realizations. Let D D D be the range at which a pair, or a probe and a reference set, first share a box, a random scale over the maintained trees and, for a passive probe, its placements, with law P D P_D P D . Its pooled affinity defines a separation cost:
F h = − h log E e − D / h = inf Q ≪ P D { E Q D + h K L ( Q ∥ P D ) } . (58) F_h = -h\log\mathbb{E}e^{-D/h} = \inf_{Q\ll P_D}\left\{\mathbb{E}_Q D + h\,\mathrm{KL}(Q\mathbin\Vert P_D)\right\}. \tag*{(58)} F h = − h log E e − D / h = Q ≪ P D inf { E Q D + h KL ( Q ∥ P D ) } . (58)
The minimizing law Q h Q_h Q h tilts P D P_D P D by e − D / h / E e − D / h e^{-D/h}/\mathbb{E}e^{-D/h} e − D / h / E e − D / h . Subtracting the infimum from the expression in braces gives h K L ( Q ∥ Q h ) h\,\mathrm{KL}(Q\mathbin\Vert Q_h) h KL ( Q ∥ Q h ) . Thus short separation paths lower the cost, but favoring a rare path costs information. If a box of range d d d is found with probability p p p and the other scales are large against h h h , then F h ≈ d + h log ( 1 / p ) F_h \approx d + h\log(1/p) F h ≈ d + h log ( 1/ p ) : finding a close neighbor once is different from finding one reliably.
For a passive probe q q q , fix a common nonempty reference set P P P . Let ξ \xi ξ denote a draw of a tree and a realized height-zero placement, as specified in Section 5.1. In the resulting range tree, u ∗ ( ξ ) u_*(\xi) u ∗ ( ξ ) is the first ancestor of q q q with a reference descendant, including the leaf when it already contains a reference. Write D ∗ ( ξ ) D_*(\xi) D ∗ ( ξ ) for its range, twice its height. The reference affinities are fixed under placement in each tree. Let Q ξ = K ξ ( P , P ) + ϵ I Q_\xi= K_\xi(P,P) + \epsilon I Q ξ = K ξ ( P , P ) + ϵ I , k ξ = K ξ ( P , q ) k_\xi= K_\xi(P,q) k ξ = K ξ ( P , q ) and let w ξ w_\xi w ξ minimize the per-realization error
J ξ ( w ) = 1 − 2 k ξ T w + w T Q ξ w . J_\xi(w) = 1 - 2k_\xi^{\mathsf{T}}w + w^{\mathsf{T}}Q_\xi w. J ξ ( w ) = 1 − 2 k ξ T w + w T Q ξ w .
A weight w i w_i w i specifies the coefficient of reference i i i in a linear prediction. Choosing a different vector w ξ w_\xi w ξ in each realization conditions there before averaging. Pooling first instead asks for one common weight vector, w ⋆ w_{\star} w ⋆ , minimizing E ξ J ξ ( w ) \mathbb{E}_{\xi}J_{\xi}(w) E ξ J ξ ( w ) . A bar denotes this same average on K K K and k k k . The pooled variance is
V ϵ p o o l ( q ) = min w E ξ J ξ ( w ) = 1 − k ˉ T ( K ˉ + ϵ I ) + k ˉ . (59) V_{\epsilon}^{\mathrm{pool}}(q)=\min_{w}\mathbb{E}_{\xi}J_{\xi}(w)=1-\bar{k}^{\mathsf{T}}(\bar{K}+\epsilon I)^{+}\bar{k}. \tag*{(59)} V ϵ pool ( q ) = w min E ξ J ξ ( w ) = 1 − k ˉ T ( K ˉ + ϵ I ) + k ˉ . (59)
The inverse is ordinary for ϵ > 0 \epsilon>0 ϵ > 0 and may be a pseudoinverse at zero nugget.
The next decomposition answers three questions in order: how far the query is from any reference on its branch, how firmly the references constrain it after first contact, and how much is lost by requiring one predictor to work across the realizations.
Theorem 14 (Support and pooled Kriging variance). Use the grounded range circuit of Proposition 11 , with each reference measurement connected through resistance ϵ \epsilon ϵ and held at zero for the resistance calculation. Let R ξ , ϵ r e m R_{\xi,\epsilon}^{\mathrm{rem}} R ξ , ϵ rem be the effective resistance from u ⋆ ( ξ ) u_{\star}(\xi) u ⋆ ( ξ ) to those measurements and prior ground after removing the unsupported probe branch below u ⋆ ( ξ ) u_{\star}(\xi) u ⋆ ( ξ ) . Then
V ϵ p o o l ( q ) = E ξ [ 1 − e − D ⋆ ( ξ ) / h ] + E ξ R ξ , ϵ r e m V_{\epsilon}^{\mathrm{pool}}(q)=\mathbb{E}_{\xi}\left[1-e^{-D_{\star}(\xi)/h}\right]+\mathbb{E}_{\xi}R_{\xi,\epsilon}^{\mathrm{rem}} V ϵ pool ( q ) = E ξ [ 1 − e − D ⋆ ( ξ ) / h ] + E ξ R ξ , ϵ rem + E ξ [ ( w ⋆ − w ξ ) T Q ξ ( w ⋆ − w ξ ) ] . (60) \qquad+\mathbb{E}_{\xi}\left[(w_{\star}-w_{\xi})^{\mathsf{T}}Q_{\xi}(w_{\star}-w_{\xi})\right]. \tag*{(60)} + E ξ [ ( w ⋆ − w ξ ) T Q ξ ( w ⋆ − w ξ ) ] . (60) At positive nugget the last term vanishes exactly when the optimal weights are common to the realizations. The first term is the probability of no reference in the probe’s block at exponential resolution. In particular,
V ϵ p o o l ( q ) ≥ 1 − e − F h ( q ; P ) / h , F h ( q ; P ) = − h log E ξ e − D ⋆ ( ξ ) / h . (61) V_{\epsilon}^{\mathrm{pool}}(q)\ge1-e^{-\mathcal{F}_{h}(q;P)/h},\qquad\mathcal{F}_{h}(q;P)=-h\log\mathbb{E}_{\xi}e^{-D_{\star}(\xi)/h}. \tag*{(61)} V ϵ pool ( q ) ≥ 1 − e − F h ( q ; P ) / h , F h ( q ; P ) = − h log E ξ e − D ⋆ ( ξ ) / h . (61) At zero nugget, hold the finite ensemble, placement law and ranges fixed as h → ∞ h\to\infty h → ∞ . For an edge e = ( p , c ) e=(p,c) e = ( p , c ) of the realized tree let S e S_e S e be the probe and reference indices below c c c , and write R p , R c R_p,R_c R p , R c for the corresponding vertex ranges, including the attachment heights. Then
lim h → ∞ h V 0 p o o l ( q ) = min 1 T w = 1 E ξ ∑ e = ( p , c ) ( R p − R c ) ( 1 { q ∈ S e } − ∑ i ∈ P ∩ S e w i ) 2 . (62) \lim_{h\to\infty}hV_{0}^{\mathrm{pool}}(q)=\min_{\mathbf{1}^{\mathsf{T}}w=1}\mathbb{E}_{\xi}\sum_{e=(p,c)}(R_p-R_c)\left(\mathbf{1}\{q\in S_e\}-\sum_{i\in P\cap S_e}w_i\right)^2. \tag*{(62)} h → ∞ lim h V 0 pool ( q ) = 1 T w = 1 min E ξ e = ( p , c ) ∑ ( R p − R c ) ( 1 { q ∈ S e } − i ∈ P ∩ S e ∑ w i ) 2 . (62) This is the least mean dissipation of a unit probe current with common signed return currents at the references.
Proof. In the grounded circuit of a realization the diagonal of the conditioned Green’s function is the effective resistance from q q q to the fixed measurements and ground. Every edge from q q q to u ⋆ u_{\star} u ⋆ carries the full unit current, since no branch off that segment contains a fixed measurement. Its resistance telescopes to 1 − e − D ⋆ / h 1-e^{-D_{\star}/h} 1 − e − D ⋆ / h and the rest is R ξ , ϵ r e m R_{\xi,\epsilon}^{\mathrm{rem}} R ξ , ϵ rem . This gives J ξ ( w ξ ) J_{\xi}(w_{\xi}) J ξ ( w ξ ) . Expanding the quadratic around w ξ w_{\xi} w ξ gives J ξ ( w ) − J ξ ( w ξ ) = ( w − w ξ ) T Q ξ ( w − w ξ ) J_{\xi}(w)-J_{\xi}(w_{\xi})=(w-w_{\xi})^{\mathsf{T}}Q_{\xi}(w-w_{\xi}) J ξ ( w ) − J ξ ( w ξ ) = ( w − w ξ ) T Q ξ ( w − w ξ ) , also when Q ξ Q_{\xi} Q ξ is singular. Averaging at w ⋆ w_{\star} w ⋆ proves (60). A resolution S ∼ Exp ( 1 / h ) S\sim\operatorname{Exp}(1/h) S ∼ Exp ( 1/ h ) places a reference in the probe’s block exactly when S ≥ D ⋆ S\ge D_{\star} S ≥ D ⋆ , proving the probability claim.
For the limit, inject 1 1 1 at q q q and − w i -w_i − w i at each reference. The quadratic J ξ ( w ) J_{\xi}(w) J ξ ( w ) at zero nugget is the power in the grounded tree. The current on each edge is the sum of injections below it, and the current to ground is 1 − 1 T w 1-\mathbf{1}^{\mathsf{T}}w 1 − 1 T w . If R max R_{\max} R m a x bounds the ranges in the finite ensemble, the edge resistances multiplied by h h h lie between e − R max / h ( R p − R c ) e^{-R_{\max}/h}(R_p-R_c) e − R m a x / h ( R p − R c ) and R p − R c R_p-R_c R p − R c . The scaled ground resistance is between h e − R max / h he^{-R_{\max}/h} h e − R m a x / h and h h h . Thus the minimum is bounded above by the right side of (62), and bounded below by e − R max / h e^{-R_{\max}/h} e − R m a x / h times the minimum of that quadratic edge cost plus h ( 1 − 1 T w ) 2 h(1-\mathbf{1}^{\mathsf{T}}w)^2 h ( 1 − 1 T w ) 2 . These penalized minima tend to the equality-constrained minimum. Indeed, the possible vectors of edge currents and the current to ground form a finite-dimensional closed affine space, after weighting the edge currents by the square roots of their lengths and ensemble probabilities. A bounded objective keeps the weighted currents bounded and drives the ground current to zero. Any limit point is feasible for the equality constraint.
The decomposition separates unsupported distance, the strength of the reference connection after first contact, and incompatibility of the reconstruction weights across trees. The slope of the unsupported term in 1 / h 1/h 1/ h at zero is E ξ D ∗ \mathbb{E}_{\xi}D_* E ξ D ∗ . In a tree with its stored sample as reference and uncapped placement, D ∗ = R ( B v ∪ { x } ) D_* = R(B_v \cup\{x\}) D ∗ = R ( B v ∪ { x }) at the sampled separation vertex v v v . Dividing the slope by R ( B r o o t ∪ { x } ) R(B_{\mathrm{root}} \cup\{x\}) R ( B root ∪ { x }) gives the range escape (22) . With one reference at separation scale D D D and zero nugget, the unsupported term is 1 − e − D / h 1-e^{-D/h} 1 − e − D / h and the full variance is 1 − e − 2 D / h 1-e^{-2D/h} 1 − e − 2 D / h . The latter includes resistance beyond first contact. A placement cap or a different reference set changes the first contact scale used in all these formulas.
For this stored-reference, uncapped placement of an unseen query, Corollary 3 gives D ∗ ≥ δ D_* \ge\delta D ∗ ≥ δ and E ξ D ∗ ≤ a δ \mathbb{E}_{\xi}D_* \le a\delta E ξ D ∗ ≤ a δ , with the nearest distance δ \delta δ and affine factor a a a defined there. Concavity of 1 − e − z / h 1-e^{-z/h} 1 − e − z / h gives
1 − e − δ / h ≤ E ξ [ 1 − e − D ∗ / h ] ≤ 1 − e − a δ / h . (63) 1-e^{-\delta/h} \le\mathbb{E}_{\xi}\left[1-e^{-D_*/h}\right] \le1-e^{-a\delta/h}. \tag*{(63)} 1 − e − δ / h ≤ E ξ [ 1 − e − D ∗ / h ] ≤ 1 − e − a δ / h . (63)
The full pooled variance is at least the left side; the upper bound here concerns its unsupported summand. In the noisy case, the stopped-range estimate in the proof of Theorem 9, applied to the augmented sample, replaces a δ a\delta a δ by a W δ + b W a_W\delta+b_W a W δ + b W . Thus the affine geometry also controls the first contribution to uncertainty.
Nyström coordinates and support. Let D ‾ = diag ( d ‾ ) \overline{D}=\operatorname{diag}(\overline{d}) D = diag ( d ) and let z l z_l z l be an orthonormal eigenvector of D ‾ − 1 / 2 K ‾ D ‾ − 1 / 2 \overline{D}^{-1/2}\overline{K}\overline{D}^{-1/2} D − 1/2 K D − 1/2 with eigenvalue μ l > 0 \mu_l>0 μ l > 0 . For a probe row k q = K ‾ ( P , q ) k_q=\overline{K}(P,q) k q = K ( P , q ) and d q = 1 T k q d_q=\mathbf{1}^{\mathsf{T}}k_q d q = 1 T k q , its Nyström coordinate is
z l ( q ) = 1 μ l ∑ j ∈ P k q ( j ) z l ( j ) d q d ‾ j . (64) z_l(q)=\frac{1}{\mu_l}\sum_{j\in P}\frac{k_q(j)z_l(j)}{\sqrt{d_q\overline{d}_j}}. \tag*{(64)} z l ( q ) = μ l 1 j ∈ P ∑ d q d j k q ( j ) z l ( j ) . (64)
For a positive semidefinite augmented affinity with probe diagonal one, the variance not represented by retained modes I I I is
V I s p e c ( q ) = 1 − d q ∑ l ∈ I μ l z l ( q ) 2 . (65) V_I^{\mathrm{spec}}(q)=1-d_q\sum_{l\in I}\mu_l z_l(q)^2. \tag*{(65)} V I spec ( q ) = 1 − d q l ∈ I ∑ μ l z l ( q ) 2 . (65)
With all positive modes this is the zero-nugget Kriging variance (59) for the same row. This is a support diagnostic of the declared geometry. The rotating-star assignment uses routed rows; its two pre-frame support diagnostics share a randomized-placement row (§ 8.2 ).
Which quantities can be averaged directly. Fix a query and supplied reference values. Let m T ( x ) m_T(x) m T ( x ) and v T ( x ) v_T(x) v T ( x ) be the conditional prediction mean and variance on a sampled tree. Mixing these conditional prediction laws under the declared ensemble law gives mean E T m T ( x ) \mathbb{E}_T m_T(x) E T m T ( x ) and variance
E T v T ( x ) + Var T ( m T ( x ) ) . \mathbb{E}_T v_T(x)+\operatorname{Var}_T(m_T(x)). E T v T ( x ) + Var T ( m T ( x )) .
The first term is uncertainty with the sampled geometry fixed; the second includes changes in predictions caused by sampling and tree construction. Pooling before prediction defines other models.
The affinity energies average directly because L K ‾ = E L K t L_{\overline{K}}=\mathbb{E}L_{K_t} L K = E L K t . Their equilibria need not average to the equilibrium of that common energy. Gaussian prediction also involves an inverse: averaging the per-tree Gaussian energies gives precision E [ ( K t + ϵ I ) − 1 ] \mathbb{E}[(K_t+\epsilon I)^{-1}] E [( K t + ϵ I ) − 1 ] , whereas averaging the covariances gives precision ( E K t + ϵ I ) − 1 (\mathbb{E}K_t+\epsilon I)^{-1} ( E K t + ϵ I ) − 1 . For ϵ > 0 \epsilon>0 ϵ > 0 ,
E [ ( K t + ϵ I ) − 1 ] ⪰ ( E K t + ϵ I ) − 1 . \mathbb{E}\left[(K_t+\epsilon I)^{-1}\right]\succeq(\mathbb{E}K_t+\epsilon I)^{-1}. E [ ( K t + ϵ I ) − 1 ] ⪰ ( E K t + ϵ I ) − 1 .
Thus these two pooling choices define different prediction models. For the pooled covariance model just analyzed, the exact excess over the average per-realization conditioned variance is the nonnegative common-weight cost in (60). This is why the order of solving and pooling is part of the estimator, even though every one-tree calculation is exact.
Here the pooling rule specifies which prediction problem is solved. Section 6.4 considers a further choice after a per-tree response has been computed: averaging its squared relative magnitude, or averaging its relative magnitude and then squaring. For CoDisp the latter gives a smoothed ensemble reading while keeping each tree’s game and energy fixed. The Kriging calculation and this reporting choice illustrate different places where the order of operations matters.
Cut Energy
Section 5.1 fixes the reference observations and gives the tree products and query rows. We now use the affinity Hamiltonian of Section 4 to evaluate a partition on those references. Each tree contributes through its own exact hierarchy. The pooled affinity can combine incompatible hierarchies, and the optimization and spectral extension use the sum of the tree operators.
For a partition A 1 , … , A k A_1,\ldots,A_k A 1 , … , A k of P P P , the normalized cut [65 ] under K ˉ \bar{K} K ˉ is
Ncut ( A ) = ∑ c = 1 k cut ( A c ) vol ( A c ) , vol ( A ) = ∑ i ∈ A d ˉ i , cut ( A ) = vol ( A ) − ∑ i , j ∈ A K ˉ i j . (66) \operatorname{Ncut}(A)=\sum_{c=1}^{k}\frac{\operatorname{cut}(A_c)}{\operatorname{vol}(A_c)},\qquad\operatorname{vol}(A)=\sum_{i\in A}\bar{d}_i,\qquad\operatorname{cut}(A)=\operatorname{vol}(A)-\sum_{i,j\in A}\bar{K}_{ij}. \tag*{(66)} Ncut ( A ) = c = 1 ∑ k vol ( A c ) cut ( A c ) , vol ( A ) = i ∈ A ∑ d ˉ i , cut ( A ) = vol ( A ) − i , j ∈ A ∑ K ˉ ij . (66)
For the walk with transitions K ˉ i j / d ˉ i \bar{K}_{ij}/\bar{d}_i K ˉ ij / d ˉ i , the ratio cut ( A ) / vol ( A ) \operatorname{cut}(A)/\operatorname{vol}(A) cut ( A ) / vol ( A ) is the probability of leaving A A A in one step conditional on starting in A A A under the stationary law [53 ] .
For a public boundary on tree t t t , clamp each reference terminal to its membership in A A A . If H t a f f \mathcal{H}^{\mathrm{aff}}_t H t aff is the energy of the sparse affinity network for K t K_t K t , then
cut ( A ) = 1 A T L ˉ 1 A = 2 N ∑ t = 1 N min f P ′ = 1 A H t a f f ( f ) . (67) \operatorname{cut}(A)=\mathbf{1}_A^{\mathsf{T}}\bar{L}\mathbf{1}_A=\frac{2}{N}\sum_{t=1}^{N}\min_{f_{P'}=\mathbf{1}_A}\mathcal{H}^{\mathrm{aff}}_t(f). \tag*{(67)} cut ( A ) = 1 A T L ˉ 1 A = N 2 t = 1 ∑ N f P ′ = 1 A min H t aff ( f ) . (67)
This follows from (47). The free voltages respond to the whole coalition boundary. Each cut term compares the current at unit voltage across the boundary with the total degree of the members. Section 6.5 reads the same cut as the share of affinity that the members of each class give up, and Section 6.6 treats relocation as a potential game.
Which scales and coordinates carry a cut. Let 0 = λ 1 < λ 2 ≤ ⋯ 0=\lambda_1<\lambda_2\le\cdots 0 = λ 1 < λ 2 ≤ ⋯ be the generalized eigenvalues of ( L ˉ , D ˉ ) (\bar{L},\bar{D}) ( L ˉ , D ˉ ) and put μ l = 1 − λ l \mu_l=1-\lambda_l μ l = 1 − λ l . The random-walk interpretation is that of Meilă and Shi [53 ] ; the multiway spectral start and Lloyd refinement use established normalized-cut procedures [57 , 23 ] . The elbow uses ratios λ k + 1 / λ k \lambda_{k+1}/\lambda_k λ k + 1 / λ k with λ k > 0 \lambda_k>0 λ k > 0 . Testing counts up to k max k_{\max} k m a x needs the first k max + 1 k_{\max}+1 k m a x + 1 eigenvalues; the corresponding k k k slow modes give the row embedding of the spectral start.
To examine coordinate j j j , multiply its box width by θ j \theta_j θ j in the affinity, holding cuts and bandwidth fixed. Put V j = − ∂ θ j K ˉ V_j=-\partial_{\theta_j}\bar{K} V j = − ∂ θ j K ˉ at θ = 1 \theta=1 θ = 1 ; its ( i , p ) (i,p) ( i , p ) entry is the forest mean of R j ( B v ) / φ ( v ) / h R_j(B_v)/\varphi(v)/h R j ( B v ) / φ ( v ) / h at their parting boxes and also zero within a tree. For selected modes I I I separated from the remaining spectrum, take D ˉ \bar{D} D ˉ -orthonormal eigenvectors K ˉ u l = μ l D ˉ u l \bar{K}u_l=\mu_l\bar{D}u_l K ˉ u l = μ l D ˉ u l with u l T D ˉ u l = 1 u_l^{\mathsf{T}}\bar{D}u_l=1 u l T D ˉ u l = 1 . Differentiating this equation gives
I j : = ∂ θ j ∑ l ∈ I μ l = ∑ l ∈ I [ − u l T V j u l + μ l ∑ i u l ( i ) 2 ( V j 1 ) i ] . (68) I_j:=\partial_{\theta_j}\sum_{l\in I}\mu_l=\sum_{l\in I}\left[-u_l^{\mathsf{T}}V_ju_l+\mu_l\sum_i u_l(i)^2(V_j\mathbf{1})_i\right]. \tag*{(68)} I j := ∂ θ j l ∈ I ∑ μ l = l ∈ I ∑ [ − u l T V j u l + μ l i ∑ u l ( i ) 2 ( V j 1 ) i ] . (68)
The second term includes the changing degrees. A positive I j I_j I j reduces the sum of the selected relaxed cuts. Importance per share divides I j / ∑ a = 1 d ∣ A a ∣ I_j/\sum_{a=1}^{d}|A_a| I j / ∑ a = 1 d ∣ A a ∣ by the coordinate’s fraction of affinity-weighted range, 1 ⊤ V j 1 / ∑ a = 1 d 1 ⊤ V a 1 1^{\top}V_{j1}/\sum_{a=1}^{d}1^{\top}V_{a1} 1 ⊤ V j 1 / ∑ a = 1 d 1 ⊤ V a 1 . The selection ranks positive ratios with positive range share and keeps those above the largest consecutive ratio when it reaches the gap threshold used for relaxed cuts. If no ratio is positive, it keeps every coordinate. The experiment’s final ablation recomputes bandwidth after selection; the derivative itself has fixed bandwidth.
Extending a fixed reference partition. Having grouped P P P , assign a new observation using that frozen construction. The tree geometry, bandwidth, reference degrees, and fitted class statistics are held fixed. The query supplies only its routed row k q = K ˉ ( P , q ) k_q=\bar{K}(P,q) k q = K ˉ ( P , q ) and d q = 1 ⊤ k q d_q=1^{\top}k_q d q = 1 ⊤ k q .
The fixed-class mean rule compares the query with the existing class means in the normalized affinity geometry. For V A = vol ( A ) V_A=\operatorname{vol}(A) V A = vol ( A ) and B A = ∑ i , j ∈ A K ˉ i j B_A=\sum_{i,j\in A}\bar{K}_{ij} B A = ∑ i , j ∈ A K ˉ ij , it chooses the class minimizing
− 2 K ˉ ( q , A ) d q V A + B A V A 2 . -\frac{2\bar{K}(q,A)}{d_qV_A}+\frac{B_A}{V_A^2}. − d q V A 2 K ˉ ( q , A ) + V A 2 B A .
The Nyström readout instead extends the spectral start by (64) , normalizes the resulting row, and assigns it to an existing center of that spectral embedding [28 ] . The experiment uses the fixed-class mean rule after refinement and compares it with the Nyström readout of the spectral start. These are two specified readouts of the frozen construction. Joining the query to the graph and recomputing a partition would ask a different question, treated in § 5.4 .
Table 1 (PDF p. 40).
The passes give exact products and cut evaluations after routing; finding the partition takes the iterations of the chosen solver. The same fitted statistics serve repeated probes. Model updates can reuse routes and initial solutions under § 5.1 ’s conditions.
What the Query Changes
The construction above assigns an observation using fixed references. Other query intents change different parts of the calculation. Write x x x for the feature vector and q q q for its vertex in a field calculation.
A passive query reads a prediction, class, or support against fixed reference affinities and values. It may use routing, or the randomized attachment of § 5.1 when its distance outside the stored support matters. A self-referential query also prescribes the value or class to be tested. Its clamp lets other free values respond; joining a class changes the class and degree accounting. § 6.1 and § 6.6 price those changes.
RCF scoring considers participation : the separation law averages over where x x x would join the tree. Inverse depth reads the inserted point’s private escape, and displacement its effect on the old records. These virtual insertions need not commit an update. An actual insertion changes stored boxes and may reroute external references, as in § 4.3 .
Clamping, Games, and Cuts
Section 4 supplied a Hamiltonian, and Section 5 put the trees on common references. With the reference values fixed, minimizing the energy fills the unknown values with the least disagreement. An intervention asks: if we insist on a different value or class, how must the remaining predictions adjust, and what additional energy is required? A clamp imposes that requirement; releasing it measures the energy saved when the requirement is removed. Moving a group between classes also changes its boundary and the volumes used to normalize its cost.
The group-clamp identity prices the first change. Stopping and participation games then specify which members can act together. CoDisp reads one release on a mass-resistance path. Finally, the same descendant sets attribute the cost of a fixed partition and supply joint moves that improve a cut even when smaller moves fail.
Imposing and Releasing a Boundary
A private prediction or escape reads one coordinate of the harmonic field under fixed evidence. To test an intervention, impose a new boundary and let the other voltages adjust. On a fixed circuit let f ⋆ f^\star f ⋆ be the equilibrium and G = ( L F F ) − 1 G=(L_{FF})^{-1} G = ( L F F ) − 1 the Green’s function on its free vertices. Prescribing z z z on a set C ⊆ F C\subseteq F C ⊆ F increases the minimized energy by
H C ( z ) − H ( f ⋆ ) = 1 2 ( z − f C ⋆ ) T ( G C C ) − 1 ( z − f C ⋆ ) . (69) \mathcal{H}_{C}(z)-\mathcal{H}(f^\star)=\frac{1}{2}(z-f^\star_{C})^{\mathsf{T}}(G_{CC})^{-1}(z-f^\star_{C}). \tag*{(69)} H C ( z ) − H ( f ⋆ ) = 2 1 ( z − f C ⋆ ) T ( G C C ) − 1 ( z − f C ⋆ ) . (69)
The required current on C C C is ( G C C ) − 1 ( z − f C ⋆ ) (G_{CC})^{-1}(z-f^\star_C) ( G C C ) − 1 ( z − f C ⋆ ) . The old equilibrium cancels the linear term in the energy, leaving half the current paired with the voltage change. A singleton clamp gives the audit (92) . Releasing a clamp measures the same difference in the other direction.
For the range-affinity Hamiltonian on the common stored observations of Theorem 9, fix a nonempty clamp set A A A and values y A y_A y A . Write E s ( 1 ) ( y A ) = min f A = y A H s ( f ) \mathcal{E}_{s}^{(1)}(y_A)=\min_{f_A=y_A}\mathcal{H}^{s}(f) E s ( 1 ) ( y A ) = min f A = y A H s ( f ) . Minimizing the pointwise energy comparison over the same free values preserves its order:
E h / a ( 1 ) ( y A ) ≤ min f A = y A E T H h T ( f ) ≤ E h ( 1 ) ( y A ) . (70) \mathcal{E}_{h/a}^{(1)}(y_A)\leq\min_{f_A=y_A}\mathcal{E}_{T}\mathcal{H}_{h}^{T}(f)\leq\mathcal{E}_{h}^{(1)}(y_A). \tag*{(70)} E h / a ( 1 ) ( y A ) ≤ f A = y A min E T H h T ( f ) ≤ E h ( 1 ) ( y A ) . (70)
The noisy version uses a W a_W a W and multiplies the lower bound by e − b W / h e^{-b_W/h} e − b W / h . These are total boundary costs for the pooled range-affinity energy. The incremental price in (69) is read from its Green’s function; the mass path used for CoDisp below retains its own conductances.
Two clusters and a void. Figure 3 applies the realization of Theorem 12 to six observations. Use dimensionless leaf-edge lengths 0.35 at observations 1 and 6 and 0.30 at the other four, inner-edge lengths 0.25, and void length 4; each probe subdivides its indicated edge at the midpoint. The probe inside the left cluster reads 0.955; the probe in the void reads 0.501. Their reference supports are 1.96 and 0.50, and their audit stiffnesses 1.28 and 0.24. Thus the same deviation from the predicted value costs about a fifth as much energy in the void. The prediction is available in both places; the clamp calculation records how strongly the evidence holds it.
Figure 3. Six observations in two clusters, their complete affinity graph (a), and its exact sparse realization (b). Edge widths in (b) show conductance, squares are observation terminals, and diamonds are probes. With terminals 1 and 5 held at one and zero, (c) shows the energy needed to prescribe a value z z z at each probe. The probe in the void has a prediction near one half and a much flatter audit.
Passage Times and Stopping Coalitions
Changing an absorbing boundary changes both the first place reached and the cost collected before stopping. On the fixed finite network of § 4.1 , let A A A be absorbing and F F F its complement, with every component meeting A A A . At each visit to v v v , spend mean time η v > 0 \eta_v > 0 η v > 0 , then choose the next vertex with probability c v w / d v c_{vw}/d_v c v w / d v , where d v = ∑ w c v w d_v = \sum_w c_{vw} d v = ∑ w c v w . Fix a vector of nonnegative cost weights ω v \omega_v ω v . Its value is
T v ω = E v ∫ 0 τ A ω X t d t . T^\omega_v = \mathbb{E}_v \int_0^{\tau_A} \omega_{X_t}\,\mathrm{d}t. T v ω = E v ∫ 0 τ A ω X t d t .
The walk is fixed while these weights specify what is counted. Unit weights give mean passage time. The first-step equation is
T A ω = 0 , T v ω = η v ω v + ∑ w c v w d v T w ω , ( L F F T F ω ) v = d v η v ω v . (71) T^\omega_A = 0,\qquad T^\omega_v = \eta_v\omega_v + \sum_w \frac{c_{vw}}{d_v}T^\omega_w,\qquad(L_{FF}T^\omega_F)_v = d_v\eta_v\omega_v. \tag*{(71)} T A ω = 0 , T v ω = η v ω v + w ∑ d v c v w T w ω , ( L F F T F ω ) v = d v η v ω v . (71)
Choosing stopping vertices to reduce passage time is studied as absorbing random-walk centrality ([52 ] ). Its diminishing-returns property also holds for the nonnegative weighted costs used here.
Proposition 15 (Weighted passage costs and stopping). The values T ω T^\omega T ω are the unique minimizer, with u A = 0 u_A = 0 u A = 0 , of
Φ ω ( u ) = 1 2 u T L u − ∑ v ∈ F d v η v ω v u v . (72) \Phi_\omega(u) = \frac{1}{2}u^\mathsf{T}Lu - \sum_{v\in F}d_v\eta_v\omega_vu_v. \tag*{(72)} Φ ω ( u ) = 2 1 u T Lu − v ∈ F ∑ d v η v ω v u v . (72) Equivalently, each free vertex chooses u v u_v u v at cost 1 2 ∑ w c v w ( u v − u w ) 2 − d v η v ω v u v \frac{1}{2}\sum_w c_{vw}(u_v-u_w)^2-d_v\eta_v\omega_vu_v 2 1 ∑ w c v w ( u v − u w ) 2 − d v η v ω v u v . Its best response is (71), and Φ ω \Phi_\omega Φ ω is an exact potential. For G = L F F G = L_{FF} G = L F F and an additional absorbing vertex z ∈ F z\in F z ∈ F ,
T v ω − T v ω , A ∪ { z } = G ( v , z ) G ( z , z ) T z ω = P v ( τ z < τ A ) T z ω , v ∈ F . (73) T^\omega_v-T^{\omega,A\cup\{z\}}_v=\frac{G(v,z)}{G(z,z)}T^\omega_z=\mathbb{P}_v(\tau_z<\tau_A)T^\omega_z,\qquad v\in F. \tag*{(73)} T v ω − T v ω , A ∪ { z } = G ( z , z ) G ( v , z ) T z ω = P v ( τ z < τ A ) T z ω , v ∈ F . (73) For a fixed starting query q q q , the worth v q ω ( C ) = T q ω − T q ω , A ∪ C v^\omega_q(C)=T^\omega_q-T^{\omega,A\cup C}_q v q ω ( C ) = T q ω − T q ω , A ∪ C is normalized, monotone, and submodular in the additional absorbing members $C\subseteq F.
Proof. Differentiating (72) gives the first-step equation, and L F F L_{FF} L F F is positive definite. A unilateral change has the same cost in the player’s objective and in Φ ω \Phi_\omega Φ ω . Adding z z z is the clamp u z ↦ 0 u_z\mapsto0 u z ↦ 0 ; the Green response and the strong Markov property give (73) . For the last claim, couple all stopped walks to the original walk up to its hit of A A A . On a trajectory, let Y z = ∫ τ z τ A ω X t d t Y_z=\int_{\tau_z}^{\tau_A}\omega X_t\,\mathrm{d}t Y z = ∫ τ z τ A ω X t d t if z z z is visited before A A A , and zero otherwise. Since ω ≥ 0 \omega\geq0 ω ≥ 0 , the cost saved by stopping at the first member of C C C is max z ∈ C Y z \max_{z\in C}Y_z max z ∈ C Y z . A maximum of nonnegative single-member rewards is monotone and submodular. Taking expectations preserves both properties. □
The response is linear in the weights: T F ω = G diag ( ( d v η v ) v ∈ F ) ω F T_F^\omega=G\operatorname{diag}((d_v\eta_v)_{v\in F})\omega_F T F ω = G diag (( d v η v ) v ∈ F ) ω F . On a tree the same upward elimination and downward response compute it. Taking the stopping coalition to be a descendant set supplies concrete worths for the restricted-coalition construction below.
Which Coalitions Can Act
Let P P P be the players at the leaves of a finite rooted tree, and S u ⊆ P S_u\subseteq P S u ⊆ P the descendants of vertex u u u . The children of each internal vertex partition its players into at least two nonempty sets. A family of nested intervals with disjoint record sets at each fork is an example. Give each S u S_u S u a transferable worth ν ( S u ) ∈ R \nu(S_u)\in\mathbb{R} ν ( S u ) ∈ R and set ν ( ∅ ) = 0 \nu(\varnothing)=0 ν ( ∅ ) = 0 . At u u u , the increment for acting together is
δ u = ν ( S u ) − ∑ w c h i l d o f u ν ( S w ) . \delta_u=\nu(S_u)-\sum_{w\ \mathrm{child\ of}\ u}\nu(S_w). δ u = ν ( S u ) − w child of u ∑ ν ( S w ) .
It can have either sign. A coalition A A A formed from a subset of the leaves can act through the maximal descendant sets contained in A A A . With this convention its worth is
ν L ( A ) = ∑ u : S u ⊆ A δ u = ∑ u : S u m a x i m a l a m o n g d e s c e n d a n t s e t s i n A ν ( S u ) . (74) \nu_{\mathcal{L}}(A)=\sum_{u:S_u\subseteq A}\delta_u=\sum_{u:S_u\ \mathrm{maximal\ among\ descendant\ sets\ in}\ A}\nu(S_u). \tag*{(74)} ν L ( A ) = u : S u ⊆ A ∑ δ u = u : S u maximal among descendant sets in A ∑ ν ( S u ) . (74)
The equality follows by telescoping inside each maximal block. In particular ν L ( S u ) = ν ( S u ) \nu_{\mathcal{L}}(S_u)=\nu(S_u) ν L ( S u ) = ν ( S u ) . If a game specifies a separate worth for a set crossing branches, the extension agrees there exactly when that worth is the sum over its maximal blocks.
This is a restriction of communication. Take every nonsingleton descendant set as a conference in the sense of Myerson [55 ] , which can act only when all of its members participate. Inside any coalition the conferences that can meet join the players into its maximal descendant sets, so (74) is the laminar case of the hypergraph communication restriction (van den Nouweland et al., [70 ] ). Its first sum writes ν L \nu_{\mathcal{L}} ν L as a combination of unanimity games on the descendant sets, with the increments δ u \delta_u δ u as the Harsanyi dividends (Harsanyi [37 ] ). Paying each participant its marginal contribution to a common worth makes that worth an exact potential, a standard construction. The following construction puts the worths into preferred edge increments of one positive quadratic energy. The hierarchy supplies the admissible participation states and the recursion under partial clamps.
For a participation profile A A A , put z u ( A ) = 1 { S u ⊆ A } z_u(A)=\mathbf{1}\{S_u\subseteq A\} z u ( A ) = 1 { S u ⊆ A } . At a leaf this is its participation choice; at an internal vertex it is the product of its children’s activation values. These logical constraints specify the admissible states of the following quadratic energy.
Theorem 16 (A quadratic Hamiltonian for laminar participation). Give the edge entering each vertex u u u conductance c u > 0 c_u>0 c u > 0 . The parent of the root is an additional ground o o o , held at z o = 0 z_o=0 z o = 0 . Let Q Q Q be this tree’s grounded Laplacian, and put
β u = 1 2 + ν ( S u ) c u , H ν ( z ) = 1 2 ∑ u ∈ V ( T ) c u ( z u − z p ( u ) − β u ) 2 . (75) \beta_u=\frac{1}{2}+\frac{\nu(S_u)}{c_u},\qquad\mathcal{H}_{\nu}(z)=\frac{1}{2}\sum_{u\in V(T)}c_u\left(z_u-z_{p(u)}-\beta_u\right)^2. \tag*{(75)} β u = 2 1 + c u ν ( S u ) , H ν ( z ) = 2 1 u ∈ V ( T ) ∑ c u ( z u − z p ( u ) − β u ) 2 . (75) The worths ν ( S u ) \nu(S_u) ν ( S u ) may have either sign. Then:
(i) For ϑ > 0 \vartheta> 0 ϑ > 0 , the continuous density proportional to e − H ν / ϑ e^{-\mathcal{H}_{\nu}/\vartheta} e − H ν / ϑ is Gaussian, with covariance ϑ Q − 1 \vartheta Q^{-1} ϑ Q − 1 and mean given by μ ∂ = 0 \mu_{\partial}=0 μ ∂ = 0 , μ u − μ p ( u ) = β u \mu_u-\mu_{p(u)}=\beta_u μ u − μ p ( u ) = β u . Its quadratic part Q Q Q is independent of the worths.
(ii) For every coalition A ⊆ P A \subseteq P A ⊆ P ,
H ν ( 0 ) − H ν ( z ( A ) ) = ν L ( A ) . (76) \mathcal{H}_{\nu}(0)-\mathcal{H}_{\nu}(z(A))=\nu_{\mathcal{L}}(A). \tag*{(76)} H ν ( 0 ) − H ν ( z ( A )) = ν L ( A ) . (76)
(iii) Pay a participating leaf i i i its marginal ν L ( A ) − ν L ( A ∖ { i } ) \nu_{\mathcal{L}}(A)-\nu_{\mathcal{L}}(A\setminus\{i\}) ν L ( A ) − ν L ( A ∖ { i }) and a nonparticipant zero. This is an exact potential game with potential ν L = H ν ( 0 ) − H ν ∘ z \nu_{\mathcal{L}}=\mathcal{H}_{\nu}(0)-\mathcal{H}_{\nu}\circ z ν L = H ν ( 0 ) − H ν ∘ z . Under any partial assignment of participation, one upward and one downward pass find a pure Nash equilibrium maximizing this potential among profiles satisfying the clamps.
Proof. The change from vertex values to edge increments is invertible with the ground fixed. The increments are independent Gaussians of means β u \beta_u β u and variances ϑ / c u \vartheta/c_u ϑ / c u , proving (i). For an activation profile, z p ( u ) ≤ z u z_{p(u)}\le z_u z p ( u ) ≤ z u and both are Boolean. Thus z u − z p ( u ) z_u-z_{p(u)} z u − z p ( u ) is one exactly at the roots of the maximal participating descendant sets. On such an edge, changing the increment from zero to one reduces energy by c u ( β u − 1 / 2 ) = ν ( S u ) c_u(\beta_u-1/2)=\nu(S_u) c u ( β u − 1/2 ) = ν ( S u ) . Summing these reductions gives (74), proving (ii).
When i i i joins, its payoff increases by precisely the increase in ν L \nu_{\mathcal{L}} ν L . The marginal is the sum of the increments at the fully participating ancestors of i i i , including its singleton. Leaving reverses that change. This is the exact potential property [54 ] , also after any of the other choices have been fixed. A constrained global maximizer is therefore a Nash equilibrium among the free players.
To find it, let F u F_u F u be the best worth with all players of S u S_u S u participating, and Q u Q_u Q u the best with a proper subset participating. Each state respects the clamps below u u u , with value − ∞ -\infty − ∞ when infeasible. At a leaf, the two candidate values are its singleton worth and zero. At an internal node, F u = ν ( S u ) F_u=\nu(S_u) F u = ν ( S u ) if all its children can be full. For a proper selection at least one child must be proper. The worth then adds across the children by (74), so
Q u = max w c h i l d o f u [ Q w + ∑ z ≠ w c h i l d o f u max ( F z , Q z ) ] . (77) Q_u=\max_{w\ \mathrm{child\ of}\ u}\left[Q_w+\sum_{z\ne w\ \mathrm{child\ of}\ u}\max(F_z,Q_z)\right]. \tag*{(77)} Q u = w child of u max Q w + z = w child of u ∑ max ( F z , Q z ) . (77)
Sum the best child values once and use the loss of requiring each child to be proper to evaluate the maximum. The root takes the better of its two states. Following the stored choices downward recovers a maximizing participation profile. The work is linear in the number of vertices given the worth table.
The continuous field allows real vertex values. Participation uses the finite set Z T = { z ( A ) : A ⊆ P } \mathcal{Z}_T=\{z(A):A\subseteq P\} Z T = { z ( A ) : A ⊆ P } with its Boolean activation constraints; the two-pass recursion minimizes the same Hamiltonian on that set. With counting measure on participation profiles, its Gibbs weights are proportional to e ν L ( A ) / ϑ e^{\nu_{\mathcal{L}}(A)/\vartheta} e ν L ( A ) / ϑ . At each step choose a free player uniformly; its logit update chooses participation or departure with probabilities proportional to the exponentials of their payoffs divided by ϑ \vartheta ϑ [11 ] . For two profiles differing only in that player’s choice, the ratio of transition probabilities is e [ ν L ( A ′ ) − ν L ( A ) ] / ϑ e^{[\nu_{\mathcal{L}}(A')-\nu_{\mathcal{L}}(A)]/\vartheta} e [ ν L ( A ′ ) − ν L ( A )] / ϑ . Detailed balance therefore gives precisely these Gibbs weights, also after participation choices have been clamped. The finite participation law and the continuous Gaussian law use the same energy on different state spaces. Expanding the squares in (75) gives a linear forcing
b u = c u β u − ∑ w : p ( w ) = u c w β w = 1 2 ( c u − ∑ w : p ( w ) = u c w ) + δ u . b_u=c_u\beta_u-\sum_{w:p(w)=u}c_w\beta_w=\frac{1}{2}\left(c_u-\sum_{w:p(w)=u}c_w\right)+\delta_u. b u = c u β u − w : p ( w ) = u ∑ c w β w = 2 1 c u − w : p ( w ) = u ∑ c w + δ u .
The Gaussian mean solves Q μ = b Q\mu=b Q μ = b , the Poisson equation of the weighted passage calculation.
Dividends as sources. Let μ 0 \mu^0 μ 0 be the mean for zero worths, with forcing b u 0 = 1 2 ( c u − ∑ w : p ( w ) = u c w ) b^0_u = \frac{1}{2}\left(c_u-\sum_{w:p(w)=u}c_w\right) b u 0 = 2 1 ( c u − ∑ w : p ( w ) = u c w ) . The worth-dependent response is exactly
Q ( μ − μ 0 ) = δ , μ − μ 0 = Q − 1 δ . (78) Q(\mu-\mu^0)=\delta,\qquad\mu-\mu^0=Q^{-1}\delta. \tag*{(78)} Q ( μ − μ 0 ) = δ , μ − μ 0 = Q − 1 δ . (78)
Thus the Harsanyi dividends are sources and the worth-free Green’s function propagates them. Taking conductances that add over children, for example c u = ∣ S u ∣ c_u=|S_u| c u = ∣ S u ∣ , makes b u 0 = 0 b^0_u=0 b u 0 = 0 at every internal vertex. The leaves retain the baseline source c u / 2 c_u/2 c u /2 ; subtracting μ 0 \mu^0 μ 0 removes that baseline at all vertices.
Vector-valued fields. The same construction allows vector values: replace a squared edge increment by its quadratic form under a fixed W ≻ 0 W\succ0 W ≻ 0 . The precision becomes Q ⊗ W Q\otimes W Q ⊗ W , so Cov ( F u , F v ) = ϑ ( Q − 1 ) u v W − 1 \operatorname{Cov}(F_u,F_v)=\vartheta(Q^{-1})_{uv}W^{-1} Cov ( F u , F v ) = ϑ ( Q − 1 ) uv W − 1 . In particular a fixed linear measurement a ⊤ F a^\top F a ⊤ F has covariance ϑ ( a ⊤ W − 1 a ) Q − 1 \vartheta(a^\top W^{-1}a)Q^{-1} ϑ ( a ⊤ W − 1 a ) Q − 1 . These formulas follow from the independent Gaussian edge increments and persist under whole-vector clamps with the conditioned Green’s function.
To compare A ⊂ B A\subset B A ⊂ B , hold the members of A A A in and everyone outside B B B out. Pinning B ∖ A B\setminus A B ∖ A out or in reads the two prescribed coalition worths. Freeing those individual choices instead asks for the best participation profile between A A A and B B B , which the fold supplies. On a query path the change from one descendant set to its parent adds exactly the sibling’s records.
The worths and unilateral gains are the quantities preserved by this construction. Its maximizing equilibrium is selected by the recursion. Other equilibria can occur. For example, if two singletons are worth − 1 -1 − 1 each and their union is worth 1 1 1 , both staying out is an equilibrium, while both joining maximizes the potential. The marginal payoffs describe participation incentives. If a budget-balanced allocation is desired, the Shapley rule for ν L \nu_L ν L gives each i ∈ S u i\in S_u i ∈ S u the share δ u / ∣ S u ∣ \delta_u/|S_u| δ u /∣ S u ∣ of that node’s increment, as for any game with these dividends. In a random arrival order each member is equally likely to be the last to complete S u S_u S u , so summing these shares down each leaf path gives the allocation.
CoDisp as a Relative Change of the Field
The mass escape of Corollary 2 gives the expected normalized displacement. The same corollary writes the CoDisp cut ratio b / a b/a b / a as odds, where a a a is the query-side mass and b b b the sibling mass. There is also a variational reading on the mass-resistance path. Let v 0 , … , v ℓ v_0,\ldots,v_\ell v 0 , … , v ℓ be the path to the root and m i m_i m i the cumulative resistance from the query-side ground to v i v_i v i , including the query mass. Set all voltages at and above v i v_i v i to one and let u i u_i u i minimize the same quadratic Hamiltonian. Releasing the constraint at v i v_i v i extends the active prefix from a = m i a=m_i a = m i to a + b = m i + 1 a+b=m_{i+1} a + b = m i + 1 , so the minimum falls from 1 / ( 2 a ) 1/(2a) 1/ ( 2 a ) to 1 / ( 2 ( a + b ) ) 1/(2(a+b)) 1/ ( 2 ( a + b )) .
Since u i − u i + 1 u_i-u_{i+1} u i − u i + 1 is a feasible variation at the relaxed minimum, it is energy-orthogonal to u i + 1 u_{i+1} u i + 1 . Writing ∥ u ∥ L 2 = 2 H ( u ) \|u\|_L^2=2\mathcal{H}(u) ∥ u ∥ L 2 = 2 H ( u ) , we obtain
∥ u i − u i + 1 ∥ L 2 ∥ u i + 1 ∥ L 2 = H ( u i ) − H ( u i + 1 ) H ( u i + 1 ) = b a . (79) \frac{\|u_i-u_{i+1}\|_L^2}{\|u_{i+1}\|_L^2}
=
\frac{\mathcal{H}(u_i)-\mathcal{H}(u_{i+1})}{\mathcal{H}(u_{i+1})}
=
\frac{b}{a}. \tag*{(79)} ∥ u i + 1 ∥ L 2 ∥ u i − u i + 1 ∥ L 2 = H ( u i + 1 ) H ( u i ) − H ( u i + 1 ) = a b . (79)
Thus C T \sqrt{C_T} C T , with C T C_T C T the tree’s CoDisp, is the largest relative change of the minimizing field over one release. Equivalently log ( 1 + C T ) \log(1+C_T) log ( 1 + C T ) is the largest one-step decrease of the log minimum energy. These log decreases sum to log ( m ℓ / m 0 ) \log(m_\ell/m_0) log ( m ℓ / m 0 ) , so the maximum measures how much of that change can occur at a single cut. The mass chains 1 , 2 , 4 , 8 1,2,4,8 1 , 2 , 4 , 8 and 1 , 6 , 7 , 8 1,6,7,8 1 , 6 , 7 , 8 have the same total, and their largest one-step ratios are 1 1 1 and 5 5 5 . On a fixed chain, changing each log m i m_i m i by at most η \eta η moves log ( 1 + C T ) \log(1+C_T) log ( 1 + C T ) by at most 2 η 2\eta 2 η ; the bound concerns the value of the maximum and lets its location switch.
The sequence of clamps follows the exact descendant sets on the query’s path. § 6.6 compares this release with the reclamping of a class in a normalized cut.
Smoothed CoDisp from relative field responses. The response identity suggests an alternative ensemble reading while leaving the per-tree coalition move and Hamiltonian unchanged. Here T T T includes the sampled records, the cuts conditional on them, and any randomized query attachment, under the sampling law of § 3 . The calibration is applied before this ensemble average. For an increasing invertible calibration F F F , the score F ( E T [ F − 1 ( Q T ) ] ) F(\mathbb{E}_{T}[F^{-1}(Q_T)]) F ( E T [ F − 1 ( Q T )]) averages in the coordinate F − 1 F^{-1} F − 1 and reports the result on the scale of Q T Q_T Q T . With Q T = C T Q_T=C_T Q T = C T and F ( u ) = u 2 F(u)=u^2 F ( u ) = u 2 on nonnegative u u u , the field interpretation gives the score
S = ( E T C T ) 2 = E T C T − Var T ( C T ) . (80) S = \left(\mathbb{E}_{T}\sqrt{C_T}\right)^2 = \mathbb{E}_{T}C_T - \operatorname{Var}_{T}\left(\sqrt{C_T}\right). \tag*{(80)} S = ( E T C T ) 2 = E T C T − Var T ( C T ) . (80)
Ordinary forest CoDisp averages squared relative field-change magnitudes. Equation (80) squares the mean relative magnitude, so its discount is exactly the between-realization variance of that magnitude. Each tree has already selected its maximizing release; the mean is over nonnegative magnitudes, not over aligned field vectors. For instance two trees with scores 1 and 81 give ordinary CoDisp 41 and S = 25 S=25 S = 25 . A single large value has less effect in this example. The CoDisp comparisons in § 8.2 use the ordinary arithmetic mean; (80) specifies the alternative reporting rule.
Write R T = C T R_T=\sqrt{C_T} R T = C T . The variance removed in (80) has the exact decomposition
Var ( R T ) = Var S ∼ Q ( E [ R T ∣ S ] ) + E S ∼ Q [ Var ( R T ∣ S ) ] . \operatorname{Var}(R_T) = \operatorname{Var}_{S\sim Q}\left(\mathbb{E}[R_T\mid S]\right) + \mathbb{E}_{S\sim Q}\left[\operatorname{Var}(R_T\mid S)\right]. Var ( R T ) = Var S ∼ Q ( E [ R T ∣ S ] ) + E S ∼ Q [ Var ( R T ∣ S ) ] .
The first term comes from sampling the records; the second includes cut construction and query attachment with those records fixed. Quadratic energy identifies R T R_T R T as the response magnitude. Sampling then explains part of the difference between averaging squared magnitudes and squaring their average. If a common amplitude ρ \rho ρ is the target and R T = ρ + ε T R_T=\rho+\varepsilon_T R T = ρ + ε T has centred fluctuations, mean CoDisp is ρ 2 + Var ( ε T ) \rho^2+\operatorname{Var}(\varepsilon_T) ρ 2 + Var ( ε T ) . For N N N independent realizations, the expected square of the empirical mean amplitude is instead ρ 2 + Var ( ε T ) / N \rho^2+\operatorname{Var}(\varepsilon_T)/N ρ 2 + Var ( ε T ) / N . This gives a sampling interpretation of the smoothing, conditional on that choice of target. Variation between samples can also reflect meaningful geometry, and the two ensemble readings retain different amounts of it.
Departure or a declared mass. CoDisp can test the departure of a whole near-duplicate cloud when it is the query side of a split. The declared-mass escape instead retains the records and reads the query at an ancestor with enough support (§ 3.2 ). The former changes a boundary; the latter chooses where a private payoff is read. Their common resilience to near duplicates does not identify their payoffs.
Boundary Costs of a Fixed Partition
At a fixed partition the normalized cut can be attributed exactly: to the faces of the trees, to their merges, and to the references themselves. Each attribution answers the question of who pays for a boundary in its own way, and all of them add up to the same cut. Fix a partition A 1 , … , A k A_1,\ldots,A_k A 1 , … , A k of the routed reference set P P P , with pooled volumes V c = vol ( A c ) V_c=\operatorname{vol}(A_c) V c = vol ( A c ) , and an induced tree t t t with affinity K t K_t K t . Here S u ⊆ P S_u\subseteq P S u ⊆ P is the set of reference descendants of u u u , so masses in this calculation count the references. For a vertex u u u with children a a a and b b b write m a = ∣ S a ∣ m_a=|S_a| m a = ∣ S a ∣ and n a c = ∣ S a ∩ A c ∣ n_a^c=|S_a\cap A_c| n a c = ∣ S a ∩ A c ∣ .
The boundary game. For a descendant set S S S let
ν t ( S ) = ∑ c = 1 k 1 V c ∑ i ∈ S ∩ A c , j ∈ S \ A c K t ( i , j ) , (81) \nu_t(S)=\sum_{c=1}^{k}\frac{1}{V_c}\sum_{i\in S\cap A_c,\ j\in S\backslash A_c}K_t(i,j), \tag*{(81)} ν t ( S ) = c = 1 ∑ k V c 1 i ∈ S ∩ A c , j ∈ S \ A c ∑ K t ( i , j ) , (81)
the normalized cut that the partition draws inside S S S when only the tree’s affinities among the members of S S S count. It is a cost: the worth of a coalition is what its members pay for the boundaries among them.
Theorem 17 (Boundary prices).
(i) Two siblings that join at u u u add
δ u = ν t ( S u ) − ν t ( S a ) − ν t ( S b ) = φ t ( u ) ∑ c = 1 k n a c ( m b − n b c ) + n b c ( m a − n a c ) V c ≥ 0. (82) \delta_u=\nu_t(S_u)-\nu_t(S_a)-\nu_t(S_b)=\varphi_t(u)\sum_{c=1}^{k}\frac{n_a^c(m_b-n_b^c)+n_b^c(m_a-n_a^c)}{V_c}\ge0. \tag*{(82)} δ u = ν t ( S u ) − ν t ( S a ) − ν t ( S b ) = φ t ( u ) c = 1 ∑ k V c n a c ( m b − n b c ) + n b c ( m a − n a c ) ≥ 0. (82) A leaf ℓ \ell ℓ holding several references has worth ν t ( S ℓ ) = ∑ c n ℓ c ( m ℓ − n ℓ c ) / V c \nu_t(S_\ell)=\sum_c n_\ell^c(m_\ell-n_\ell^c)/V_c ν t ( S ℓ ) = ∑ c n ℓ c ( m ℓ − n ℓ c ) / V c , and
Ncut ( A ) = 1 N ∑ t = 1 N ν t ( P ) = 1 N ∑ t = 1 N [ ∑ u i n t e r n a l δ u t + ∑ ℓ l e a f ν t ( S ℓ ) ] . \operatorname{Ncut}(A)=\frac{1}{N}\sum_{t=1}^{N}\nu_t(P)=\frac{1}{N}\sum_{t=1}^{N}\left[\sum_{u\ \mathrm{internal}}\delta_u^t+\sum_{\ell\ \mathrm{leaf}}\nu_t(S_\ell)\right]. Ncut ( A ) = N 1 t = 1 ∑ N ν t ( P ) = N 1 t = 1 ∑ N [ u internal ∑ δ u t + ℓ leaf ∑ ν t ( S ℓ ) ] . (ii) For any z z z on P P P , the sibling sums
δ u ( z ) = φ t ( u ) ∑ i ∈ S a , j ∈ S b ( z i − z j ) 2 = φ t ( u ) [ m b ∑ S a z 2 + m a ∑ S b z 2 − 2 ∑ S a z ∑ S b z ] , \delta_u(z)=\varphi_t(u)\sum_{i\in S_a,j\in S_b}(z_i-z_j)^2=\varphi_t(u)\left[m_b\sum_{S_a}z^2+m_a\sum_{S_b}z^2-2\sum_{S_a}z\sum_{S_b}z\right], δ u ( z ) = φ t ( u ) i ∈ S a , j ∈ S b ∑ ( z i − z j ) 2 = φ t ( u ) [ m b S a ∑ z 2 + m a S b ∑ z 2 − 2 S a ∑ z S b ∑ z ] , with the leaves’ within variations m ℓ ∑ S ℓ ( z − z ˉ ℓ ) 2 m_\ell\sum_{S_\ell}(z-\bar z_\ell)^2 m ℓ ∑ S ℓ ( z − z ˉ ℓ ) 2 , add up to z ⊤ L t z z^\top L_tz z ⊤ L t z . For a slow eigenvector z l z_l z l of the pencil ( L ˉ , D ˉ ) (\bar L,\bar D) ( L ˉ , D ˉ ) with z l ⊤ D ˉ z l = 1 z_l^\top\bar D z_l=1 z l ⊤ D ˉ z l = 1 , their forest average is its relaxed cut λ l \lambda_l λ l .
Each takes one upward pass of class counts, or of sums of z z z and z 2 z^2 z 2 , per tree.
Proof. Two references in different children of u u u part at u u u , with affinity φ t ( u ) \varphi_t(u) φ t ( u ) ; pairs inside a child are counted in that child’s worth. A pair crosses the boundary of A c A_c A c exactly when one of its endpoints lies in A c A_c A c , which gives (82), and references that share a leaf have affinity one. Telescoping from the leaves gives ν t ( P ) = ∑ c cut ( A c ) / V c \nu_t(P)=\sum_c\operatorname{cut}(A_c)/V_c ν t ( P ) = ∑ c cut ( A c ) / V c , and since a cut is linear in the affinity its forest average is Ncut ( A ) \operatorname{Ncut}(A) Ncut ( A ) . For (ii) expand the square; the class indicators are the case z = 1 A c z=\mathbf{1}_{A_c} z = 1 A c , divided by V c V_c V c and summed over c c c .
Equation (54) distributes the same energy over the merges with the coefficients deg ( v ) \deg(v) deg ( v ) of a variance decomposition; the boundary game charges each pair to the split that separates it. The two differ already at a single merge. When the children of u u u hold the classes in the same proportions p c p_c p c , the contrast of their means vanishes and the merge contributes nothing to (54), yet its price is φ t ( u ) m a m b ∑ c 2 p c ( 1 − p c ) / V c \varphi_t(u)m_am_b\sum_c2p_c(1-p_c)/V_c φ t ( u ) m a m b ∑ c 2 p c ( 1 − p c ) / V c , positive unless S u S_u S u lies in one class: the split still separates pairs with different labels. By (67) on the tree induced on S S S , each worth is a clamped energy of a sparse network, with the class indicators as clamps. The price δ u \delta_u δ u is the interaction added when two siblings are joined at u u u : the cross affinities φ t ( u ) \varphi_t(u) φ t ( u ) between them enter the complete affinity graph, while the class indicators and pooled volumes stay fixed. In the sparse realization the join changes the degrees, and so the conductances, throughout both subtrees. It is not a release, which enlarges the feasible set on a fixed network and can only lower a clamped minimum (Section 6.1).
The value members give up at a cut. Consider the induced-subgraph game with worth B A B_A B A for coalition A A A [22 ] . Each self-affinity belongs to its own record. Each unordered pair contributes 2 K ˉ i j 2\bar{K}_{ij} 2 K ˉ ij , which the Shapley allocation divides equally between its endpoints. Indeed, in the random ordering that defines the allocation, either endpoint is equally likely to arrive second and complete the pair. Thus the share of i ∈ A i \in A i ∈ A is ϕ i ( A ) = K ˉ ( i , A ) \phi_i(A) = \bar{K}(i,A) ϕ i ( A ) = K ˉ ( i , A ) , compared with ϕ i ( P ) = d ˉ i \phi_i(P) = \bar{d}_i ϕ i ( P ) = d ˉ i in the grand coalition. In particular,
cut ( A ) V A = ∑ i ∈ A [ ϕ i ( P ) − ϕ i ( A ) ] ∑ i ∈ A ϕ i ( P ) . (83) \frac{\operatorname{cut}(A)}{V_A} = \frac{\sum_{i\in A}[\phi_i(P)-\phi_i(A)]}{\sum_{i\in A}\phi_i(P)}. \tag*{(83)} V A cut ( A ) = ∑ i ∈ A ϕ i ( P ) ∑ i ∈ A [ ϕ i ( P ) − ϕ i ( A )] . (83)
The cut term is the fraction of the members’ original allocation that they lose by forming their own coalition. Each loss is already available from the degree and the class-affinity product. The share identity uses the fixed pooled affinity and holds for any partition.
Laminar shares of the boundary. The boundary game extends from the descendant sets to every coalition as in (74) : a coalition acts through the maximal descendant sets it contains. In the language of Myerson [55 ] the descendant sets are the conferences that can meet, and the extension is the laminar case of the hypergraph communication restriction [70 ] . The prices (82) are its Harsanyi dividends, so its Shapley allocation (§ 6.3 ) charges i i i the share σ ( i ) = N − 1 ∑ u ⊇ i δ u t / m u \sigma(i) = N^{-1}\sum_{u\supseteq i}\delta_u^t/m_u σ ( i ) = N − 1 ∑ u ⊇ i δ u t / m u , over the vertices u u u above i i i and its leaf, with the leaf’s worth in place of δ \delta δ there. The shares sum to Ncut ( A ) \operatorname{Ncut}(A) Ncut ( A ) . The two allocations answer who pays differently. The pairwise rule charges each crossing pair to its two endpoints, as the induced-subgraph game above charges the affinity of a pair. The laminar rule shares the cost of the pair over the smallest descendant block that contains both, so a reference deep inside a group shares the price of the group’s boundary: collusion, restricted to siblings.
Contours. Each split u u u of each tree is an oriented face F u t F_u^t F u t : the cut on its coordinate at its threshold, with the two siblings on its sides. For routed references the faithful extent of the face is the box of the references routed to u u u , since the tree’s box comes from its own sample. The boundary prices make the faces a measure,
Γ A = 1 N ∑ t = 1 N ∑ u i n t e r n a l δ u t u F u t , (84) \Gamma_A = \frac{1}{N}\sum_{t=1}^{N}\sum_{u\ \mathrm{internal}}\delta_u^t\,\mathbf{u}_{F_u}^t, \tag*{(84)} Γ A = N 1 t = 1 ∑ N u internal ∑ δ u t u F u t , (84)
with u F \mathbf{u}_F u F the uniform probability on the face F F F . Its mass is Ncut ( A ) \operatorname{Ncut}(A) Ncut ( A ) less the price of references that share a leaf, which no face separates. Its density in space locates where the partition pays for its boundaries; each face records which coordinate separates the two sides, and the depth of u u u records the scale. With z i z_i z i in place of the class indicators and weight 1 / λ l 1/\lambda_l 1/ λ l , each slow mode contributes a contour of unit mass, less its leaf terms. This is the forest’s counterpart of the spectral boundary of Arbeláez et al. [5 ] , which differentiates the slow eigenvectors of a normalized cut, weights them by 1 / λ l 1/\sqrt{\lambda_l} 1/ λ l , and orders the resulting regions into an ultrametric contour map. Here each tree is already an ultrametric hierarchy, with box ranges as its heights. The boundary game puts the partition’s cut, or a slow mode’s relaxed cut, on its faces, and the forest averages N N N such hierarchies instead of selecting one.
Exact attribution on approximately nested trees. Each tree is exactly nested, so on each tree the prices are defined, nonnegative, and add up to that tree’s reading of the cut. The forest is only approximately nested: its trees hold different samples, and the pooled affinity K ˉ \bar{K} K ˉ belongs to no single hierarchy. The attribution is exact nonetheless. A cut is linear in the affinity and the worths share the pooled volumes, so the forest average of the per-tree attributions, to the faces and leaves in (84) or to the members in σ \sigma σ , is exactly Ncut ( A ) \operatorname{Ncut}(A) Ncut ( A ) . The approximation shows only in where the trees place the price, on which faces and at what depth, never as an error in the total. The spread of priced faces across trees along one stretch of a boundary is the boundary’s counterpart of the swing of a query (§ 3.3 ): small where the trees agree on where the partition pays, and large where each tree draws the boundary through its own boxes.
Collusive Cuts
In this subsection we use the collusion and coalition dynamics discussed in the context of subtree restricted games and apply them in the context of normalized cuts. Unlike the attributions of § 6.5 , which hold the partition fixed, a move changes the partition and with it the class volumes.
Evaluating a relocation. Write W ( C , A ) = ∑ i ∈ C , j ∈ A K ˉ i j W(C,A)=\sum_{i\in C,j\in A}\bar{K}_{ij} W ( C , A ) = ∑ i ∈ C , j ∈ A K ˉ ij , B A = W ( A , A ) B_A=W(A,A) B A = W ( A , A ) , and V A = vol ( A ) V_A=\operatorname{vol}(A) V A = vol ( A ) .
Proposition 18 (Exact evaluation and relocation). For a partition into k k k classes, the cuts, volumes, and class-affinity products K ˉ ( i , A c ) \bar{K}(i,A_c) K ˉ ( i , A c ) take O ( N k ∣ P ∣ ) O(Nk|\mathcal{P}|) O ( N k ∣ P ∣ ) work by tree passes on the induced trees. Moving a nonempty group C ⊂ A C\subset A C ⊂ A from class A A A to class B B B changes the normalized cut by
B A V A + B B V B − B A − 2 W ( C , A ) + B C V A − V C − B B + 2 W ( C , B ) + B C V B + V C . (85) \frac{B_A}{V_A}+\frac{B_B}{V_B}-\frac{B_A-2W(C,A)+B_C}{V_A-V_C}-\frac{B_B+2W(C,B)+B_C}{V_B+V_C}. \tag*{(85)} V A B A + V B B B − V A − V C B A − 2 W ( C , A ) + B C − V B + V C B B + 2 W ( C , B ) + B C . (85)
Proof. Apply Proposition 13 to the class counts and indicator vectors on each tree. Removing C C C subtracts the rows and columns of C C C from B A B_A B A , adding back B C B_C B C once. Adding C C C to B B B contributes both cross terms and B C B_C B C . The degrees of C C C transfer between the two volumes. Substitute these changes into k − ∑ c B A c / V A c k-\sum_c B_{A_c}/V_{A_c} k − ∑ c B A c / V A c .
For a singleton C = { i } C=\{i\} C = { i } , B C = 1 B_C=1 B C = 1 and V C = d ˉ i V_C=\bar{d}_i V C = d ˉ i .
A potential game of relocation. Potential games for normalized partition objectives have been studied in Avrachenkov et al. [6 ] . For the full-degree normalization of (66), let Φ ( A ) = B A / V A \Phi(A)=B_A/V_A Φ ( A ) = B A / V A on nonempty classes and Φ ( ∅ ) = 0 \Phi(\varnothing)=0 Φ ( ∅ ) = 0 . Give record i ∈ A i\in A i ∈ A the payoff u i ( A ) = Φ ( A ) − Φ ( A ∖ { i } ) u_i(A)=\Phi(A)-\Phi(A\setminus\{i\}) u i ( A ) = Φ ( A ) − Φ ( A ∖ { i }) . A unilateral change between classes, keeping all classes nonempty, changes this payoff by exactly the change in the partition potential ∑ A Φ ( A ) = k − Ncut \sum_A\Phi(A)=k-\operatorname{Ncut} ∑ A Φ ( A ) = k − Ncut . For a joint move the analogous gain is the group’s marginal value in the destination less its marginal value in the source. This is the payoff for changing membership. Equation (83) instead allocates affinity at a given partition.
A self-referential cut. To test a trial class for the query, let its row join the graph and prescribe that class. With k c = K ˉ ( q , A c ) k_c=\bar{K}(q,A_c) k c = K ˉ ( q , A c ) the total affinity to class c c c , and d q = 1 + ∑ c k c d_q=1+\sum_c k_c d q = 1 + ∑ c k c , the row adds k c k_c k c to each old volume vol ( A c ) \operatorname{vol}(A_c) vol ( A c ) . Joining class c c c adds d q d_q d q more to vol ( A c ) \operatorname{vol}(A_c) vol ( A c ) and 2 k c + 1 2k_c+1 2 k c + 1 to its within-class affinity. The normalized cut for each trial follows from the k k k sums k c k_c k c , one path per tree; the smallest chooses the class. The passive probe had support ∑ c k c \sum_c k_c ∑ c k c ; the self-referential test gives the query a degree including its unit diagonal, and changes the volumes it was previously read against. The stored tree is fixed. Actual sample insertion can also change the old affinities or the routing (§ 4.3 ).
A joint move when every singleton move fails. Multilevel graph clustering uses block moves on coarsened graphs (Dhillon et al., 2005). Here the range trees supply the candidate blocks: on one tree, every descendant set contained in a single current class can move together to any other class, and Theorem 20 below scans all of them in one pass. The example and the proposition that follow show why the blocks matter.
For an example, use the points 0 , … , 5 0,\ldots,5 0 , … , 5 on the line with successive pairs P 1 = { 0 , 1 } P_1=\{0,1\} P 1 = { 0 , 1 } , P 2 = { 2 , 3 } P_2=\{2,3\} P 2 = { 2 , 3 } , and P 3 = { 4 , 5 } P_3=\{4,5\} P 3 = { 4 , 5 } . The root splits off P 3 P_3 P 3 and its left child separates P 1 P_1 P 1 from P 2 P_2 P 2 . With h = 1 / log 2 h=1/\log2 h = 1/ log 2 , the affinities within a pair are 1 / 2 1/2 1/2 off the diagonal, between P 1 , P 2 P_1,P_2 P 1 , P 2 they are 1 / 8 1/8 1/8 , and to P 3 P_3 P 3 they are 1 / 32 1/32 1/32 . The partition ( P 1 ∪ P 3 , P 2 ) (P_1\cup P_3,P_2) ( P 1 ∪ P 3 , P 2 ) has normalized cut 84 / 319 ≈ 0.263 84/319\approx0.263 84/319 ≈ 0.263 . The best single-record move raises it to 280 / 783 ≈ 0.358 280/783\approx0.358 280/783 ≈ 0.358 . Moving the whole branch P 1 P_1 P 1 to P 2 P_2 P 2 lowers it to 42 / 377 ≈ 0.111 42/377\approx0.111 42/377 ≈ 0.111 .
Theorem 19 (An unbounded gap for small moves). For every s ≥ 2 s\ge2 s ≥ 2 , there are one-dimensional range trees on 3 s 3s 3 s distinct points with a two-class partition whose normalized cut can exceed optimal by an arbitrarily large factor. Every nonempty relabeling of fewer than s s s points increases its cut. Moving one descendant set of size s s s attains the optimum and strictly improves u i u_i u i for every record that moves.
Proof. Take three translated copies P 1 , P 2 , P 3 P_1,P_2,P_3 P 1 , P 2 , P 3 of s s s points in [ 0 , 1 ] [0,1] [ 0 , 1 ] including both endpoints, with shifts 0 , R − 1 , 2 R − 2 0,R-1,2R-2 0 , R − 1 , 2 R − 2 for R ≥ 3 R\ge3 R ≥ 3 . The root splits off P 3 P_3 P 3 , its other child separates P 1 P_1 P 1 and P 2 P_2 P 2 , and all three copies use the same internal splitting topology. With h = 1 / log 2 h=1/\log2 h = 1/ log 2 and ϵ = 2 − R \epsilon=2^{-R} ϵ = 2 − R , within-copy affinities are at least 1 / 2 1/2 1/2 , those between P 1 P_1 P 1 and P 2 P_2 P 2 are ϵ \epsilon ϵ , and those to P 3 P_3 P 3 are 2 ϵ 2 2\epsilon^2 2 ϵ 2 . The six-point example uses s = 2 , R = 3 s=2,R=3 s = 2 , R = 3 .
Any bipartition that splits a copy has crossing affinity at least ( s − 1 ) / 2 (s-1)/2 ( s − 1 ) /2 . Its normalized cut is at least 2 ( s − 1 ) / ( 9 s 2 ) 2(s-1)/(9s^2) 2 ( s − 1 ) / ( 9 s 2 ) since V A + V B ≤ 9 s 2 V_A+V_B\le9s^2 V A + V B ≤ 9 s 2 and 1 / V A + 1 / V B ≥ 4 / ( V A + V B ) 1/V_A+1/V_B\ge4/(V_A+V_B) 1/ V A + 1/ V B ≥ 4/ ( V A + V B ) . The partition ( P 1 ∪ P 3 , P 2 ) (P_1\cup P_3,P_2) ( P 1 ∪ P 3 , P 2 ) and the root partition ( P 3 , P 1 ∪ P 2 ) (P_3,P_1\cup P_2) ( P 3 , P 1 ∪ P 2 ) both have costs tending to zero as R → ∞ R\to\infty R → ∞ . Their crossing affinities are s ( ϵ + 2 ϵ 2 ) s(\epsilon+2\epsilon^2) s ( ϵ + 2 ϵ 2 ) and 4 s ϵ 2 4s\epsilon^2 4 s ϵ 2 , respectively. If M M M is the common within-copy affinity, their reciprocal-volume sums both tend to 3 / ( 2 M ) 3/(2M) 3/ ( 2 M ) . Their cost ratio is therefore asymptotic to 1 / ( 4 ϵ ) 1/(4\epsilon) 1/ ( 4 ϵ ) . The third whole-copy partition has the same cost as the starting one by symmetry of P 1 , P 2 P_1,P_2 P 1 , P 2 . Thus for large R R R the root partition is optimal. Every nonempty relabeling of fewer than s s s points splits a copy, while moving P 1 P_1 P 1 to P 2 P_2 P 2 gives the root partition.
For the payoff comparison, let A = P 1 ∪ P 3 A=P_1\cup P_3 A = P 1 ∪ P 3 , A ′ = P 1 ∪ P 2 A'=P_1\cup P_2 A ′ = P 1 ∪ P 2 , i ∈ P 1 i\in P_1 i ∈ P 1 , V = V A V=V_A V = V A , d = d ˉ i d=\bar d_i d = d ˉ i , and η = s 2 ( ϵ − 2 ϵ 2 ) \eta=s^2(\epsilon-2\epsilon^2) η = s 2 ( ϵ − 2 ϵ 2 ) . The volume and within-class affinity increase by η \eta η and 2 η 2\eta 2 η from A A A to A ′ A' A ′ . With i i i excluded the increases are η \eta η and 2 ( 1 − 1 / s ) η 2(1-1/s)\eta 2 ( 1 − 1/ s ) η . Taking the difference of the two marginal payoffs gives
u i ( A ′ ) − u i ( A ) η = 2 − Φ ( A ) V + η − 2 ( 1 − 1 / s ) − Φ ( A ∖ { i } ) V − d + η . \frac{u_i(A')-u_i(A)}{\eta}
=
\frac{2-\Phi(A)}{V+\eta}
-
\frac{2(1-1/s)-\Phi(A\setminus\{i\})}{V-d+\eta}. η u i ( A ′ ) − u i ( A ) = V + η 2 − Φ ( A ) − V − d + η 2 ( 1 − 1/ s ) − Φ ( A ∖ { i }) .
As R → ∞ R\to\infty R → ∞ , write d 0 = ∑ j ∈ P 1 K i j ≤ s d_0=\sum_{j\in P_1}K_{ij}\le s d 0 = ∑ j ∈ P 1 K ij ≤ s . Then V → 2 M V\to2M V → 2 M , d → d 0 d\to d_0 d → d 0 , Φ ( A ) → 1 \Phi(A)\to1 Φ ( A ) → 1 , and Φ ( A ∖ { i } ) → 1 − ( d 0 − 1 ) / ( 2 M − d 0 ) \Phi(A\setminus\{i\})\to1-(d_0-1)/(2M-d_0) Φ ( A ∖ { i }) → 1 − ( d 0 − 1 ) / ( 2 M − d 0 ) . Multiplying the limiting expression by 2 M − d 0 2M-d_0 2 M − d 0 gives
2 s − d 0 2 M − d 0 − 1 2 M − d 0 > 0 \frac{2}{s}-\frac{d_0}{2M}-\frac{d_0-1}{2M-d_0}>0 s 2 − 2 M d 0 − 2 M − d 0 d 0 − 1 > 0
because 2 M ≥ s ( s + 1 ) 2M\ge s(s+1) 2 M ≥ s ( s + 1 ) and each subtracted term is less than 1 / s 1/s 1/ s . Hence for large R R R every moving record gains. □
Moves as reclamping. For a class A A A on the affinity graph, ground A c A^c A c and then clamp all of A A A at one. The initial field is zero, and (69) gives twice the energy increase as 1 T ( G A A ) − 1 1 = cut ( A ) \mathbf{1}^{\mathsf{T}}(G_{AA})^{-1}\mathbf{1}=\operatorname{cut}(A) 1 T ( G AA ) − 1 1 = cut ( A ) . Dividing by V A V_A V A gives the class’s normalized boundary cost. A move of a block C C C from A A A to B B B reclamps it: in the field of A A A the block changes from one to zero, in the field of B B B from zero to one, and both class volumes are recomputed. The renormalization is why relocation is not a release and need not lower any one energy. The move in Theorem 19 preserves the branch’s internal affinity while exchanging a weak connection for a stronger one. It lowers the cut and benefits each mover even though each would lose by moving alone. The tree specifies which records can act together, and the two energies measure the consequences of that action.
Fix the partition, the induced tree, and the notation of § 6.5 .
The deviation game and the collusive cut. A descendant set C C C is pure if it lies inside one class A A A and is not all of it. Tree t t t reads the forest’s partition with its own affinities and the pooled volumes,
Ncut t ( A 1 , … , A k ) = ν t ( P ) = ∑ c = 1 k cut t ( A c ) V c . \operatorname{Ncut}_{t}(A_1,\ldots,A_k)=\nu_t(P)=\sum_{c=1}^{k}\frac{\operatorname{cut}_{t}(A_c)}{V_c}. Ncut t ( A 1 , … , A k ) = ν t ( P ) = c = 1 ∑ k V c cut t ( A c ) .
the worth (81) of the whole reference set. A cut is linear in the affinity, so for every partition the forest average of Ncut t \operatorname{Ncut}_{t} Ncut t is Ncut \operatorname{Ncut} Ncut . The gain of C C C from changing class together is
g t ( C ) = max B ≠ A [ Ncut t ( A 1 , … , A k ) − Ncut t ( the partition with C moved from A to B ) ] . g_t(C)=\max_{B\ne A}\left[\operatorname{Ncut}_{t}(A_1,\ldots,A_k)-\operatorname{Ncut}_{t}(\text{the partition with }C\text{ moved from }A\text{ to }B)\right]. g t ( C ) = B = A max [ Ncut t ( A 1 , … , A k ) − Ncut t ( the partition with C moved from A to B ) ] .
in which the moved partition has the pooled volumes V A − V C V_A-V_C V A − V C and V B + V C V_B+V_C V B + V C and the cuts
cut t ( A ∖ C ) = cut t ( A ) − cut t ( C ) + 2 W t ( C , A ∖ C ) , cut t ( B ∪ C ) = cut t ( B ) + cut t ( C ) − 2 W t ( C , B ) , \operatorname{cut}_{t}(A\setminus C)=\operatorname{cut}_{t}(A)-\operatorname{cut}_{t}(C)+2W_t(C,A\setminus C),\qquad\operatorname{cut}_{t}(B\cup C)=\operatorname{cut}_{t}(B)+\operatorname{cut}_{t}(C)-2W_t(C,B), cut t ( A ∖ C ) = cut t ( A ) − cut t ( C ) + 2 W t ( C , A ∖ C ) , cut t ( B ∪ C ) = cut t ( B ) + cut t ( C ) − 2 W t ( C , B ) ,
with W t W_t W t the cross affinity under K t K_t K t . Collusive displacement takes, over the descendant sets on a query’s path, the largest displacement per member ([36 ] ). In the same way put
CC t ( i ) = max { 0 , max C g t ( C ) ∣ C ∣ } , CC ( i ) = 1 N ∑ t = 1 N CC t ( i ) , (86) \operatorname{CC}_t(i)=\max\left\{0,\max_C\frac{g_t(C)}{|C|}\right\},\qquad\operatorname{CC}(i)=\frac{1}{N}\sum_{t=1}^{N}\operatorname{CC}_t(i), \tag*{(86)} CC t ( i ) = max { 0 , C max ∣ C ∣ g t ( C ) } , CC ( i ) = N 1 t = 1 ∑ N CC t ( i ) , (86)
the inner maximum over the pure descendant sets on the path of i i i in tree t t t , the singleton { i } \{i\} { i } included when it is pure. The zero is the option to stay put. It is also the value for the only member of a singleton class, which cannot move without emptying its class.
Theorem 20 (The collusive cut).
(i) On an induced tree, the class products K t ( i , A c ) K_t(i,A_c) K t ( i , A c ) summed upward give W t ( S u , A c ) W_t(S_u,A_c) W t ( S u , A c ) ; the within-coalition affinities satisfy B t ( S u ) = B t ( S a ) + B t ( S b ) + 2 φ t ( u ) m a m b B_t(S_u)=B_t(S_a)+B_t(S_b)+2\varphi_t(u)m_a m_b B t ( S u ) = B t ( S a ) + B t ( S b ) + 2 φ t ( u ) m a m b , with B t ( S ℓ ) = m a 2 B_t(S_\ell)=m_a^2 B t ( S ℓ ) = m a 2 at a leaf; and the tree’s degrees and the pooled degrees, summed upward, give volt ( S u ) \operatorname{volt}(S_u) volt ( S u ) and V ( S u ) V(S_u) V ( S u ) , hence cut t ( S u ) = volt ( S u ) − B t ( S u ) \operatorname{cut}_t(S_u)=\operatorname{volt}(S_u)-B_t(S_u) cut t ( S u ) = volt ( S u ) − B t ( S u ) . Hence CC t \operatorname{CC}_t CC t of every reference takes O ( k ∣ P ∣ ) O(k|P|) O ( k ∣ P ∣ ) work.
(ii) CC t ( i ) \operatorname{CC}_t(i) CC t ( i ) is at least the gain per member of every pure descendant set on the path of i i i , and in particular of the best single move of i i i . If CC t ( i ) = 0 \operatorname{CC}_t(i)=0 CC t ( i ) = 0 for every reference, no descendant set of tree t t t gains by changing class together. Stability under single moves does not imply this, and the gap in the objective can be unbounded (Theorem 19 ).
(iii) For one block C C C and one destination B B B , the forest average of the per-tree changes of Ncut \operatorname{Ncut} Ncut is the pooled change (85). A block proposed by one tree is approved by testing it with (85), which takes one more pooled product, for B C = 1 C T K 1 C B_C=\mathbf{1}_C^{\mathsf T}K\mathbf{1}_C B C = 1 C T K 1 C .
Proof. (i) Proposition 13 gives the class products in one pass, and cross pairs part at u u u , which gives the recursion; W t ( C , A ∖ C ) = W t ( C , A ) − B t ( C ) W_t(C,A\setminus C)=W_t(C,A)-B_t(C) W t ( C , A ∖ C ) = W t ( C , A ) − B t ( C ) . The cuts of the moved partition then cost O ( 1 ) O(1) O ( 1 ) per vertex and destination, hence O ( k ) O(k) O ( k ) per vertex. (ii) Every pure descendant set on the path of i i i is a candidate, and every pure descendant set is on the path of each of its members. Theorem 19 gives partitions in which every move of fewer than s s s records loses while a descendant set of size s s s reaches the optimum. (iii) For any set S S S the forest average of cut t ( S ) \operatorname{cut}_{t}(S) cut t ( S ) is the pooled cut, and the volumes are common to the trees, so the forest average of Ncut t \operatorname{Ncut}_{t} Ncut t is Ncut \operatorname{Ncut} Ncut for the partitions before and after the move.
As with collusive displacement, the maximum over nested coalitions answers masking: a group of near duplicates, none of which gains by moving alone, is tested as the descendant set that holds them. The collusive cut is quenched: each tree chooses its own block and destination and reads them with its own affinities, against the pooled volumes. By (iii) the per-tree readings of one common move average exactly to its pooled gain, so the per-tree diagnostic and the pooled approval differ only in which move they test, not in how they normalize it. A positive pooled gain for a candidate means that the forest would redraw that boundary too. A positive collusive cut without one marks references where single trees would redraw the boundary and the forest would not, much as the Kriging variance separates missing support from the incompatibility of the trees’ reconstructions [60 ] . The gains of pure descendant sets also define a laminar participation game (Theorem 16 ), whose fold (77) proposes disjoint blocks to move. Its extension adds the gains of those blocks and so ignores their interaction through shared volumes and cross affinities. The blocks are therefore only proposals: (85) tests one at a time, and after each accepted move the others are rescored against the updated partition.
Directional Audits and Perturbations
An answer can draw on different geometric directions, and a weak connection can make it sensitive to small changes. We want to know which directions supply its evidence and how strongly they hold the answer. Signed route lengths record where the evidence lies. Changing a directional scale tests its effect on the prediction after minimizing the updated Hamiltonian over the unknown values; prescribing another answer instead measures its energy cost.
We specify the directional units, then compute predictions together with their route evidence and audit stiffness. Differentiating the sparse realization connects geometric moments to the response of boundary energy. The stability theorem bounds finite changes in the resulting harmonic predictions.
Directional Lengths and Gauges
A gauge specifies the units in which the retained directional lengths are compared. It consists of positive scales on the signed half dimensions of § 2.1 . Dividing each length by its scale gives a dimensionless path length and hence an affinity. The same reference set P P P and the same pooling operation of § 5.1 are used after these scales are specified.
Lengths in local units. For directional interpretation, retain nonnegative signed length witnesses ℓ e , j ± ( x ) \ell_{e,j}^{\pm}(x) ℓ e , j ± ( x ) with sum ℓ e \ell_e ℓ e , recording the extent or gap assigned to each half dimension. Parent-box half extents, with declared positive floors, supply local scales s e , j ± s_{e,j}^{\pm} s e , j ± that can change with resolution along the traversal. With 2 d 2d 2 d independent positive multipliers, set
h e , j ± = λ j ± ( x ) s e , j ± , γ e , j ± = ℓ e , j ± ( x ) / h e , j ± , Γ e = ∑ j , ± γ e , j ± . (87) h_{e,j}^{\pm}=\lambda_j^{\pm}(x)s_{e,j}^{\pm}, \qquad\gamma_{e,j}^{\pm}=\ell_{e,j}^{\pm}(x)/h_{e,j}^{\pm}, \qquad\Gamma_e=\sum_{j,\pm}\gamma_{e,j}^{\pm}. \tag*{(87)} h e , j ± = λ j ± ( x ) s e , j ± , γ e , j ± = ℓ e , j ± ( x ) / h e , j ± , Γ e = j , ± ∑ γ e , j ± . (87)
Fix these annotations for the query before folding. The dimensionless lengths Γ e \Gamma_e Γ e define an additive tree metric d Γ d_{\Gamma} d Γ ; replacing ℓ e / h \ell_e/h ℓ e / h by Γ e \Gamma_e Γ e gives ρ e = e − Γ e \rho_e=e^{-\Gamma_e} ρ e = e − Γ e and K i j = e − d Γ ( i , j ) K_{ij}=e^{-d_{\Gamma}(i,j)} K ij = e − d Γ ( i , j ) . Theorem 12 applies to this symmetric metric with all 2 d 2d 2 d multipliers retained. Equal scales recover Γ e = ℓ e / h \Gamma_e = \ell_e/h Γ e = ℓ e / h . For edge-dependent scales the general path-product construction of Theorem 12 and the two passes (48) apply directly.
The affine comparison also quantifies a bounded change of these units. On the stored observations of Theorem 9, suppose 0 < h − ≤ h e , j ± ≤ h + 0 < h_- \le h_{e,j}^{\pm} \le h_+ 0 < h − ≤ h e , j ± ≤ h + uniformly over trees, edges, and faces. Nonnegativity of the witnesses and their sum ℓ e \ell_e ℓ e give d T / h + ≤ d Γ ≤ d T / h − d_T/h_+ \le d_{\Gamma} \le d_T/h_- d T / h + ≤ d Γ ≤ d T / h − . Let H Γ T \mathcal{H}^{T}_{\Gamma} H Γ T use affinities e − d Γ e^{-d_{\Gamma}} e − d Γ . Theorem 9, applied at h − h_- h − and h + h_+ h + , then gives
H h − / a ( 1 ) ( f ) ≤ E T H h + T ( f ) ≤ H h + / a ( 1 ) ( f ) . (88) \mathcal{H}^{(1)}_{h_-/a}(f) \le\mathbb{E}_T\mathcal{H}^{T}_{h_+}(f) \le\mathcal{H}^{(1)}_{h_+/a}(f). \tag*{(88)} H h − / a ( 1 ) ( f ) ≤ E T H h + T ( f ) ≤ H h + / a ( 1 ) ( f ) . (88)
The noisy version replaces a a a by a W a_W a W and multiplies the lower bound by e − b W / h − e^{-b_W/h_-} e − b W / h − . The comparison requires the stated uniform bounds on the gauges; the exact folds and directional derivatives below use the declared positive scales directly.
An edge decomposition from nested boxes. For a complete query x x x , resolve the occupied projection of B v B_v B v on each side of x x x :
o ˉ v j = [ min ( u v j , x j ) − ℓ v j ] + , o v j + = [ u v j − max ( ℓ v j , x j ) ] + . \bar{o}_{vj} = [\min(u_{vj},x_j)-\ell_{vj}]_+,\qquad o^+_{vj} = [u_{vj}-\max(\ell_{vj},x_j)]_+. o ˉ v j = [ min ( u v j , x j ) − ℓ v j ] + , o v j + = [ u v j − max ( ℓ v j , x j ) ] + .
Their sum is u v j − ℓ v j u_{vj}-\ell_{vj} u v j − ℓ v j . If B v ⊆ B p B_v \subseteq B_p B v ⊆ B p , each occupied half width is no larger than its parent’s. Set
ℓ { p , v } , j + ( x ) = 1 2 ( o p j + − o v j + ) . \ell^{+}_{\{p,v\},j}(x)=\frac{1}{2}(o^+_{pj}-o^+_{vj}). ℓ { p , v } , j + ( x ) = 2 1 ( o p j + − o v j + ) .
These entries are nonnegative and sum to H p − H v H_p-H_v H p − H v , where H v = R ( B v ) / 2 H_v=R(B_v)/2 H v = R ( B v ) /2 . In a tree of nonempty axis-aligned splits, the cut-coordinate width strictly decreases in each child, so H p − H v > 0 H_p-H_v>0 H p − H v > 0 . With singleton-location leaves, H v = 0 H_v=0 H v = 0 at the leaves and the induced tree distance between observations is R ( B lca ( i , j ) ) R(B_{\operatorname{lca}(i,j)}) R ( B lca ( i , j ) ) , the range-based tree distance used in Guha et al. [36 ] . Thus a maintained box tree supplies a concrete 2 d 2d 2 d decomposition without changing its cuts. Directional empty intervals between sibling boxes can be recorded separately within these length witnesses. The decomposition changes with x x x , while its unscaled total edge length does not.
Directional scales and operator response. With the tree, signed witnesses, and base scales fixed, let r i j r_{ij} r ij collect normalized path lengths in the 2 d 2d 2 d half dimensions and let θ α = 1 / λ α \theta_\alpha=1/\lambda_\alpha θ α = 1/ λ α be their inverse scale multipliers. The gauge (87) gives K i j = e − θ T r i j K_{ij}=e^{-\theta^\mathsf{T}r_{ij}} K ij = e − θ T r ij and L K = diag ( K 1 ) − K L_K=\operatorname{diag}(K\mathbf{1})-K L K = diag ( K 1 ) − K . Writing Lap ( W ) = diag ( W 1 ) − W \operatorname{Lap}(W)=\operatorname{diag}(W\mathbf{1})-W Lap ( W ) = diag ( W 1 ) − W , its directional response is
− ∂ θ α L K = Lap ( ( r i j , α K i j ) i j ) ⪰ 0. (89) -\partial_{\theta_\alpha}L_K=\operatorname{Lap}\left((r_{ij,\alpha}K_{ij})_{ij}\right)\succeq0. \tag*{(89)} − ∂ θ α L K = Lap ( ( r ij , α K ij ) ij ) ⪰ 0. (89)
Each pair contributes a positive exponential coefficient to L K L_K L K , making it convex in the matrix order as a function of the inverse scales; its energy at fixed potentials is convex as well. § 7.3 differentiates the conductances of the sparse network instead, edge by edge.
Reference support and direction. Write r q j r_{qj} r q j for the sum of the signed witness-to-scale vectors along the route from q q q to j j j before multiplication by the inverse scales θ \theta θ . The reference support and its weighted route moments are
b q ( θ ) = ∑ j ∈ P e − θ T r q j . (90) b_q(\theta)=\sum_{j\in P}e^{-\theta^\mathsf{T}r_{qj}}. \tag*{(90)} b q ( θ ) = j ∈ P ∑ e − θ T r q j . (90)
The negative gradient of log b q \log b_q log b q is the corresponding affinity-weighted mean route vector. The mass passes of (48) carry these first moments by adding the edge vector times the mass being passed.
Predictions and Their Directional Evidence
Harmonic prediction. On the sparse affinity network of Theorem 12 , the next observation terminal visited from i ′ i^\prime i ′ , including a return to i ′ i^\prime i ′ , is j ′ j^\prime j ′ with probability K i j / b i K_{ij}/b_i K ij / b i , where b i = ∑ j K i j b_i=\sum_j K_{ij} b i = ∑ j K ij . Indeed the pendant conductance is b i b_i b i , the trace conductance to j ′ ≠ i ′ j^\prime\ne i^\prime j ′ = i ′ is K i j K_{ij} K ij , and the remaining probability is 1 / b i 1/b_i 1/ b i [51 ] . Thus a harmonic prediction with colored observations as boundary values can be evaluated on the sparse tree [75 ] . For a passive probe it is f ( q ) = ∑ j ∈ P K q j f j / b q f(q)=\sum_{j\in P}K_{qj}f_j/b_q f ( q ) = ∑ j ∈ P K q j f j / b q , with b q = ∑ j ∈ P K q j b_q=\sum_{j\in P}K_{qj} b q = ∑ j ∈ P K q j and f j f_j f j the solved reference values. The support b q b_q b q distinguishes a strong reference connection from a well-defined voltage in a void.
Route evidence of a prediction. Let ω q ( a ′ ) \omega_q(a^\prime) ω q ( a ′ ) be the first-hit probability at colored terminal a ′ a^\prime a ′ , let y a , z y_{a,z} y a , z be its weight for color z z z , and let B ( q , a ′ ) B(q,a^\prime) B ( q , a ′ ) be the sum of the signed dimensionless lengths along the unique route. The value and its route evidence are
f z ( q ) = ∑ a ω q ( a ′ ) y a , z , J z ( q ) = ∑ a ω q ( a ′ ) y a , z B ( q , a ′ ) . (91) f_z(q)=\sum_a\omega_q(a^\prime)y_{a,z},\qquad J_z(q)=\sum_a\omega_q(a^\prime)y_{a,z}B(q,a^\prime). \tag*{(91)} f z ( q ) = a ∑ ω q ( a ′ ) y a , z , J z ( q ) = a ∑ ω q ( a ′ ) y a , z B ( q , a ′ ) . (91)
For f z ( q ) > 0 f_z(q)>0 f z ( q ) > 0 , J z / f z J_z/f_z J z / f z is the mean route length conditional on that color. A branch response a u − b au-b a u − b , with unnormalized route moment M z M_z M z , extends through an edge of conductance c c c and length vector γ e \gamma_e γ e as
( a ′ , b ′ ) = c c + a ( a , b ) , M z ′ = c c + a ( M z + γ e b z ) . (a^\prime,b^\prime)=\frac{c}{c+a}(a,b),\qquad M_z^\prime=\frac{c}{c+a}(M_z+\gamma_e b_z). ( a ′ , b ′ ) = c + a c ( a , b ) , M z ′ = c + a c ( M z + γ e b z ) .
The series factor multiplies every first-hit coefficient in that branch, and the new edge adds its length vector to every route. Parallel branches add their responses and moments. Dividing the resulting ( b z , M z ) (b_z,M_z) ( b z , M z ) by a a a gives (91). This evaluates the predictions and their route evidence together. If the forest prediction averages passive voltages with support weights b q b_q b q , its route length vector for color z z z uses the same weights, E [ b q J z ( q ) ] / E [ b q f z ( q ) ] E[b_qJ_z(q)]/E[b_qf_z(q)] E [ b q J z ( q )] / E [ b q f z ( q )] , when the denominator is positive. Changing the scale of a route moves a prediction through its voltage drops and the Green’s response to that change. For example, constant boundary values give zero response to a scale change even when the route lengths are large. § 7.3 gives the directional derivatives.
Auditing a value. Prescribe z z z at a free probe q q q and let the other free voltages minimize. If F F F is the free set and G = ( L F F ) − 1 G=(L_{FF})^{-1} G = ( L F F ) − 1 , the cost is
H q ( z ) − H q ( f ( q ) ) = 1 2 a q ∥ z − f ( q ) ∥ 2 , a q = 1 / G ( q , q ) . (92) H_q(z)-H_q(f(q))=\frac{1}{2}a_q\lVert z-f(q)\rVert^2,\qquad a_q=1/G(q,q). \tag*{(92)} H q ( z ) − H q ( f ( q )) = 2 1 a q ∥ z − f ( q ) ∥ 2 , a q = 1/ G ( q , q ) . (92)
The residual current is a q ( z − f ( q ) ) a_q(z-f(q)) a q ( z − f ( q )) . This follows by eliminating the other free voltages, leaving one scalar stiffness per component. For a regression audit the boundary values and the probe value are scalars. The support and the audit stiffness report separate aspects of the local geometry.
Response to a Change of Scale
The route moments describe the geometry used by a prediction. To ask how changing that geometry changes the answer, we also need the voltage drops of the field. Fix the hierarchy, references, signed witnesses, and base scales throughout the differentiation.
Boundary-energy response. Partition the sparse network into retained boundary B B B and interior I I I , with invertible L I I L_{II} L I I . Its response is Λ B = L B B − L B I L I I − 1 L I B \Lambda_B=L_{BB}-L_{BI}L_{II}^{-1}L_{IB} Λ B = L B B − L B I L I I − 1 L I B . For boundary potentials y y y and harmonic extension v y v_y v y , differentiating the minimized energy gives
∂ θ α ( y T Λ B y ) = ∑ e = { u , v } ∂ θ α c e ( v y ( u ) − v y ( v ) ) 2 . (93) \partial_{\theta_\alpha}\left(y^\mathsf{T}\Lambda_B y\right)=\sum_{e=\{u,v\}}\partial_{\theta_\alpha}c_e\left(v_y(u)-v_y(v)\right)^2. \tag*{(93)} ∂ θ α ( y T Λ B y ) = e = { u , v } ∑ ∂ θ α c e ( v y ( u ) − v y ( v ) ) 2 . (93)
The interior derivative vanishes by harmonicity. Finite single-edge updates follow from the standard rank-one Kron sensitivity formula [26 ] . A geometric scale change can alter many conductances, including the pendant edges, so all their terms enter (93).
From directional mass to conductance. Write u e , α = ℓ e , α / s e , α u_{e,\alpha}=\ell_{e,\alpha}/s_{e,\alpha} u e , α = ℓ e , α / s e , α and ρ e = e − θ T u e \rho_e=e^{-\theta^\mathsf{T}u_e} ρ e = e − θ T u e . For a tree vertex v v v , its normalized directional mass moment is
r ˉ v , α = ∑ j ∈ P K v j r v j , α b v = − ∂ θ α log b v . \bar r_{v,\alpha}=\frac{\sum_{j\in P}K_{vj}r_{vj,\alpha}}{b_v}=-\partial_{\theta_\alpha}\log b_v. r ˉ v , α = b v ∑ j ∈ P K v j r v j , α = − ∂ θ α log b v .
Proposition 21 (Directional response of the sparse realization). For the conductances of Theorem 12,
− ∂ θ α log c u v = r ˉ u , α + r ˉ v , α + 1 + ρ u v 2 1 − ρ u v 2 u { u , v } , α . (94) -\partial_{\theta_\alpha}\log c_{uv}=\bar r_{u,\alpha}+\bar r_{v,\alpha}+\frac{1+\rho_{uv}^{2}}{1-\rho_{uv}^{2}}u_{\{u,v\},\alpha}. \tag*{(94)} − ∂ θ α log c uv = r ˉ u , α + r ˉ v , α + 1 − ρ uv 2 1 + ρ uv 2 u { u , v } , α . (94) A pendant has c j , j ′ = b j c_{j,j'}=b_j c j , j ′ = b j and ∂ θ α log c j , j ′ = − r ˉ j , α \partial_{\theta_\alpha}\log c_{j,j'}=-\bar r_{j,\alpha} ∂ θ α log c j , j ′ = − r ˉ j , α . For fixed boundary-value dimension, the 2 d 2d 2 d directional derivatives of boundary energy in (93) take O ( d ∣ W ∣ ) O(d|W|) O ( d ∣ W ∣ ) work on the sparse tree.
Proof. Differentiate log c u v = log b u + log b v + log ρ u v − log ( 1 − ρ u v 2 ) \log c_{uv}=\log b_u+\log b_v+\log\rho_{uv}-\log(1-\rho_{uv}^{2}) log c uv = log b u + log b v + log ρ uv − log ( 1 − ρ uv 2 ) to obtain (94). The mass passes of (48) accumulate their directional moments by adding an edge vector times the mass passed across it. Tree elimination supplies the voltages; summing the edge contributions in (93) proves the work bound.
A void on one edge. For an edge e = { u , v } e=\{u,v\} e = { u , v } separating V L , V R V_L,V_R V L , V R , let Γ e \Gamma_e Γ e be its dimensionless length. Every cross-terminal path passes through that edge. Its affinity factors as
K x y = e − d Γ ( x , u ) ρ e − d Γ ( v , y ) , ρ = e − Γ e , K_{xy}=e^{-d_{\Gamma}(x,u)}\rho e^{-d_{\Gamma}(v,y)},\qquad\rho=e^{-\Gamma_e}, K x y = e − d Γ ( x , u ) ρ e − d Γ ( v , y ) , ρ = e − Γ e ,
so summing the cross affinities with
Z L = ∑ x ∈ V L e − d Γ ( x , u ) , Z R = ∑ y ∈ V R e − d Γ ( v , y ) Z_L=\sum_{x\in V_L}e^{-d_{\Gamma}(x,u)},\qquad Z_R=\sum_{y\in V_R}e^{-d_{\Gamma}(v,y)} Z L = x ∈ V L ∑ e − d Γ ( x , u ) , Z R = y ∈ V R ∑ e − d Γ ( v , y )
gives the total cross-cut affinity
C K ( V L ) = e − Γ e Z L Z R . (95) C_K(V_L)=e^{-\Gamma_e}Z_LZ_R. \tag*{(95)} C K ( V L ) = e − Γ e Z L Z R . (95)
A geometric gap of length G G G at common scale h h h is the special case Γ e = G / h \Gamma_e=G/h Γ e = G / h . The cross-cut affinity is also the current when the groups are clamped at zero and one, preserved by the sparse realization. Holding Z L , Z R Z_L,Z_R Z L , Z R fixed, an increase Δ Γ e \Delta\Gamma_e Δ Γ e reduces this coupling by e − Δ Γ e e^{-\Delta\Gamma_e} e − Δ Γ e . Because Γ e \Gamma_e Γ e retains all 2 d 2d 2 d signed length-to-scale contributions, the weak connection can be traced to a gap and the local scale at which it matters. These same contributions appear in the color-conditioned routes of (91).
The exact sparse realization makes this barrier visible on one electrical edge. On the two sides of the gap, (41) gives b u = Z L + ρ Z R b_u=Z_L+\rho Z_R b u = Z L + ρ Z R and b v = Z R + ρ Z L b_v=Z_R+\rho Z_L b v = Z R + ρ Z L . Therefore
c u v = ρ ( Z L + ρ Z R ) ( Z R + ρ Z L ) 1 − ρ 2 , c u v ρ Z L Z R ⟶ 1 (96) c_{uv}=\frac{\rho(Z_L+\rho Z_R)(Z_R+\rho Z_L)}{1-\rho^2},\qquad\frac{c_{uv}}{\rho Z_LZ_R}\longrightarrow1 \tag*{(96)} c uv = 1 − ρ 2 ρ ( Z L + ρ Z R ) ( Z R + ρ Z L ) , ρ Z L Z R c uv ⟶ 1 (96)
as Γ e → ∞ \Gamma_e\to\infty Γ e → ∞ with fixed positive Z L , Z R Z_L,Z_R Z L , Z R . This edge retains the leading exponential attenuation of the complete graph’s cross-cut capacity. Exact effective capacity between terminal sets also includes any resistance within the sides; it is preserved by the network elimination of Theorem 12.
Stability under Conductance Perturbations
Keep the edge support and supplied boundary values fixed. We first bound the change in the whole harmonic field, then in its hitting and directional evidence. These are perturbations of a specified Hamiltonian. The coordinate-noise result of Corollary 1 controls the geometry from which the trees are built, and Theorem 9 brackets their averaged energy. The following bounds compare two specified operators through relative errors in their conductances.
Theorem 22 (Stability under relative conductance errors). Let a finite connected network have positive edge conductances and a nonempty fixed boundary. Compare conductances c , c ′ > 0 c,c^{\prime}>0 c , c ′ > 0 on the same edge support, with the same prescribed boundary values. Write κ − = min e c e ′ / c e > 0 \kappa_{-}=\min_{e}c^{\prime}_{e}/c_{e}>0 κ − = min e c e ′ / c e > 0 , κ + = max e c e ′ / c e \kappa_{+}=\max_{e}c^{\prime}_{e}/c_{e} κ + = max e c e ′ / c e and η = 1 2 log ( κ + / κ − ) \eta=\frac{1}{2}\log(\kappa_{+}/\kappa_{-}) η = 2 1 log ( κ + / κ − ) , which is zero for a common rescaling of all conductances. If u u u and u ′ u^{\prime} u ′ minimize the old and the new energies, then at every interior vertex q q q
∥ u ′ − u ∥ L ≤ sinh ( η ) ∥ u ∥ L , ∣ u q ′ − u q ∣ ≤ sinh ( η ) ( L I I − 1 ) q q ∥ u ∥ L , \lVert u^{\prime}-u\rVert_{L}\le\sinh(\eta)\lVert u\rVert_{L},\qquad|u^{\prime}_{q}-u_{q}|\le\sinh(\eta)\sqrt{(L_{II}^{-1})_{qq}}\lVert u\rVert_{L}, ∥ u ′ − u ∥ L ≤ sinh ( η ) ∥ u ∥ L , ∣ u q ′ − u q ∣ ≤ sinh ( η ) ( L I I − 1 ) q q ∥ u ∥ L , where ∥ u ∥ L 2 = u T L u \lVert u\rVert_{L}^{2}=u^{\mathsf{T}}Lu ∥ u ∥ L 2 = u T Lu for the old Laplacian L L L , with interior block L I I L_{II} L I I . The relative bound is attained on a path of two edges.
Proof. Put w = u ′ − u w=u^{\prime}-u w = u ′ − u , k = ( κ + + κ − ) / 2 k=(\kappa_{+}+\kappa_{-})/2 k = ( κ + + κ − ) /2 and B 0 = L ′ / k − L B_{0}=L^{\prime}/k-L B 0 = L ′ / k − L . Edgewise, the weights of B 0 B_{0} B 0 are at most tanh ( η ) \tanh(\eta) tanh ( η ) times those of L L L in absolute value, so Cauchy–Schwarz gives ∣ w T B 0 u ′ ∣ ≤ tanh ( η ) ∥ w ∥ L ∥ u ′ ∥ L |w^{\mathsf{T}}B_{0}u^{\prime}|\le\tanh(\eta)\lVert w\rVert_{L}\lVert u^{\prime}\rVert_{L} ∣ w T B 0 u ′ ∣ ≤ tanh ( η ) ∥ w ∥ L ∥ u ′ ∥ L . Harmonicity implies w T L u = 0 w^{\mathsf{T}}Lu=0 w T Lu = 0 and w T L ′ u ′ = 0 w^{\mathsf{T}}L^{\prime}u^{\prime}=0 w T L ′ u ′ = 0 , so ∥ w ∥ L 2 = − w T B 0 u ′ \lVert w\rVert_{L}^{2}=-w^{\mathsf{T}}B_{0}u^{\prime} ∥ w ∥ L 2 = − w T B 0 u ′ and ∥ u ′ ∥ L 2 = ∥ u ∥ L 2 + ∥ w ∥ L 2 \lVert u^{\prime}\rVert_{L}^{2}=\lVert u\rVert_{L}^{2}+\lVert w\rVert_{L}^{2} ∥ u ′ ∥ L 2 = ∥ u ∥ L 2 + ∥ w ∥ L 2 . Solving the resulting inequality gives the first bound. The second follows from Cauchy–Schwarz on the grounded block applied to w q = e q T w I w_{q}=e_{q}^{\mathsf{T}}w_{I} w q = e q T w I . For the path, take resistances e 2 η e^{2\eta} e 2 η and 1 1 1 in series between boundary values zero and one, and multiply the two conductances by e η e^{\eta} e η and e − η e^{-\eta} e − η . The divider voltage becomes 1 / 2 1/2 1/2 , and substitution gives ∥ w ∥ L / ∥ u ∥ L = sinh ( η ) \lVert w\rVert_{L}/\lVert u\rVert_{L}=\sinh(\eta) ∥ w ∥ L / ∥ u ∥ L = sinh ( η ) .
The factor ( L I I − 1 ) q q \sqrt{(L_{II}^{-1})_{qq}} ( L I I − 1 ) q q is the square root of the effective resistance from q q q to the boundary, the Kriging standard deviation of § 4.1 when the boundary holds the references: the sensitivity bound is larger for a weakly supported probe.
Corollary 5 (Stability of hitting and directional evidence). On a fixed connected network, suppose ∣ log ( c e ′ / c e ) ∣ ≤ η |\log(c^{\prime}_{e}/c_{e})|\le\eta ∣ log ( c e ′ / c e ) ∣ ≤ η and let m m m be the number of free vertices. The first-hit laws on the same nonempty absorbing boundary satisfy
TV ( p q ′ , p q ) ≤ tanh ( m η / 2 ) . (97) \operatorname{TV}(p^{\prime}_{q},p_{q})\le\tanh(m\eta/2). \tag*{(97)} TV ( p q ′ , p q ) ≤ tanh ( m η /2 ) . (97) On a tree, use the same nonnegative boundary color weights in both models. If reference route lengths in half dimension α \alpha α lie in [ 0 , B α max ] [0,B_{\alpha}^{\max}] [ 0 , B α m a x ] and their changes are at most δ α \delta_{\alpha} δ α , the color-conditioned route moments satisfy
∣ J z , α ′ ( q ) f z ′ ( q ) − J z , α ( q ) f z ( q ) ∣ ≤ δ α + B α max tanh ( m η / 2 ) , (98) \left|\frac{J^{\prime}_{z,\alpha}(q)}{f^{\prime}_{z}(q)}-\frac{J_{z,\alpha}(q)}{f_{z}(q)}\right|\le\delta_{\alpha}+B_{\alpha}^{\max}\tanh(m\eta/2), \tag*{(98)} f z ′ ( q ) J z , α ′ ( q ) − f z ( q ) J z , α ( q ) ≤ δ α + B α m a x tanh ( m η /2 ) , (98) whenever the conditioning color has positive weight.
Proof. The harmonic forest formula represents the first-hit law by absorbing-root spanning forests, each with m m m edges (Pitman and Tang, 2018). Their weight ratios lie between e − m η e^{-m\eta} e − m η and e m η e^{m\eta} e m η . A probability law tilted by a weight in [ a , b ] [a,b] [ a , b ] changes in total variation by at most ( b − a ) / ( b + a ) (\sqrt{b}-\sqrt{a})/(\sqrt{b}+\sqrt{a}) ( b − a ) / ( b + a ) , which proves (97). Multiplying each forest weight by the same nonnegative endpoint color preserves these ratio bounds. Changing the route lengths costs at most δ α \delta_{\alpha} δ α ; changing their conditional law costs at most B α max tanh ( m η / 2 ) B_{\alpha}^{\max}\tanh(m\eta/2) B α m a x tanh ( m η /2 ) .
On the complete affinity graph, a uniform distance error δ \delta δ at fixed h h h gives log-conductance error at most δ / h \delta/h δ / h . For weighted ℓ 1 \ell_1 ℓ 1 distances, moving each endpoint by at most σ j \sigma_j σ j in coordinate j j j gives δ ≤ 2 ∑ j β j σ j \delta\le2\sum_j \beta_j\sigma_j δ ≤ 2 ∑ j β j σ j . There is a corresponding bound for the range tree when descendant memberships remain fixed. Each box width changes by at most 2 σ j 2\sigma_j 2 σ j , so its range, and hence every stored-pair tree distance, changes by at most 2 W 2W 2 W , where W = ∑ j σ j W=\sum_j\sigma_j W = ∑ j σ j . At fixed h h h this gives ∣ log ( K i j ′ / K i j ) ∣ ≤ 2 W / h \lvert\log(K'_{ij}/K_{ij})\rvert\le2W/h ∣ log ( K ij ′ / K ij )∣ ≤ 2 W / h . Theorem 22 therefore applies to the complete observation-affinity graph with η ≤ 2 W / h \eta\le2W/h η ≤ 2 W / h . Under log-conductance error η \eta η , every fixed cut capacity and volume changes by a factor in [ e − η , e η ] [e^{-\eta},e^\eta] [ e − η , e η ] , so its normalized cut changes by a factor in [ e − 2 η , e 2 η ] [e^{-2\eta},e^{2\eta}] [ e − 2 η , e 2 η ] .
These bounds condition on common support and boundaries. A stream can also change routes or topology. Section 5.1 then tests a reused field by its residual against the current operator; the fixed-support bounds do not supply that correspondence.
Measurements and Illustration
The preceding sections compute fields, their boundary costs, and their directional response. We now specify the query measurements used in the illustration. Local outlier factors compare a query’s density with its neighbors. We introduce a few more measures that capture anisotropy. The box calibration ALOFT compares the extent needed to include it at a declared reference mass. Face diffusion reads attachment to spatial boundaries. We then define uncertainty and support readings from Kriging variance and Nyström residuals. The rotating star uses the same maintained trees to compare these readings and to assign new observations to a reference clustering.
Local Outlier Factors and Calibration
Local outlier factors. For a reference set S S S , let N k , S ( x ) N_{k,S}(x) N k , S ( x ) contain up to k k k nearest points of S S S in Euclidean distance, using all available points if fewer than k k k are available and a fixed ordering to break ties. Exclude x x x itself when it belongs to S S S . Let d k , S ( o ) d_{k,S}(o) d k , S ( o ) be the last distance in the neighborhood of o o o . With this capped-neighborhood convention, the local reachability density and local outlier factor of Breunig et al. [16 ] , and their average over the trees, are
lrd k , S ( x ) = [ 1 ∣ N k , S ( x ) ∣ ∑ o ∈ N k , S ( x ) max { d k , S ( o ) , ∥ x − o ∥ 2 } ] − 1 , (99) \operatorname{lrd}_{k,S}(x)=\left[\frac{1}{\lvert N_{k,S}(x)\rvert}\sum_{o\in N_{k,S}(x)}\max\{d_{k,S}(o),\lVert x-o\rVert_2\}\right]^{-1}, \tag*{(99)} lrd k , S ( x ) = ∣ N k , S ( x )∣ 1 o ∈ N k , S ( x ) ∑ max { d k , S ( o ) , ∥ x − o ∥ 2 } − 1 , (99)
LOF k , S ( x ) = ∑ o ∈ N k , S ( x ) lrd k , S ( o ) ∣ N k , S ( x ) ∣ lrd k , S ( x ) , LOFT k ( x ) = 1 ∣ T ∗ ∣ ∑ t ∈ T ∗ LOF k , S t ( x ) . (100) \operatorname{LOF}_{k,S}(x)=\frac{\sum_{o\in N_{k,S}(x)}\operatorname{lrd}_{k,S}(o)}{\lvert N_{k,S}(x)\rvert\operatorname{lrd}_{k,S}(x)},\qquad\operatorname{LOFT}_k(x)=\frac{1}{\lvert T_*\rvert}\sum_{t\in T_*}\operatorname{LOF}_{k,S_t}(x). \tag*{(100)} LOF k , S ( x ) = ∣ N k , S ( x )∣ lrd k , S ( x ) ∑ o ∈ N k , S ( x ) lrd k , S ( o ) , LOFT k ( x ) = ∣ T ∗ ∣ 1 t ∈ T ∗ ∑ LOF k , S t ( x ) . (100)
where S t S_t S t is the set of distinct stored points of tree t t t and T ∗ T_* T ∗ contains the trees with at least two such points. Pooled LOF uses the distinct reference points across the forest; LOFT computes the ratio separately in each tree’s sample and averages those ratios [76 ] . The tie and cap conventions above are the ones used in the comparison.
Calibration at a declared mass. The reference neighborhood must be chosen before its expansion by the query is measured. Use the calibration box at a declared mass μ \mu μ in Definition 6. The default is min { ψ , 2 ψ } \min\{\psi,2\sqrt{\psi}\} min { ψ , 2 ψ } ; (28) parameterizes other mass choices. Tree ancestry can overshoot that requested mass. This controls the population being compared, just as k k k controls the neighborhood in LOF, SOD, and COF.
Ratios of extents. We introduce a new measure ALOFT, which asks how much a neighborhood of the declared mass must stretch to admit the query. It reads the stretch along each coordinate after averaging the widths across trees. Let E t j E_{tj} E t j be the coordinate extents of tree t t t ’s calibration box, and let A t j ≥ E t j A_{tj} \ge E_{tj} A t j ≥ E t j be its extents enlarged to include x x x . Average both over the trees first, to E ˉ j \bar{E}_j E ˉ j and A ˉ j \bar{A}_j A ˉ j ; then
ALOFT μ ( x ) = 1 d ∑ j log A ˉ j E ˉ j . (101) \operatorname{ALOFT}_{\mu}(x) = \frac{1}{d}\sum_j \log\frac{\bar{A}_j}{\bar{E}_j}. \tag*{(101)} ALOFT μ ( x ) = d 1 j ∑ log E ˉ j A ˉ j . (101)
summarizes the per-coordinate ratios, each coordinate in its own units, with no centre, covariance or inverse. A ratio of averages weights each tree by its extent. It is zero when x x x lies in the box in every tree, and it is the log of the geometric mean of the ratios, which reads a stretch on its own scale rather than as a small excess over one. A coordinate with E ˉ j = 0 \bar{E}_j = 0 E ˉ j = 0 contributes nothing when A ˉ j = 0 \bar{A}_j = 0 A ˉ j = 0 ; otherwise every tree’s box is flat there and x x x is off it, and ALOFT μ \operatorname{ALOFT}_{\mu} ALOFT μ is infinite. Per tree, the log volume ratio is the Kullback–Leibler divergence between the uniform laws on the box without and with the query; we note that other divergences can also be used [2 ] . At the passage box itself the ratio loses most of its contrast, since the passage vertex is chosen by the query’s own cut probability, the expansion it measures (§ 8.2 ).
Directional expansion of a reference box. Let P t P_t P t be the uniform law on a tree’s calibration box and Q t Q_t Q t the uniform law on that box enlarged to include x x x . For positive widths, their density ratio on the smaller box is the volume ratio. Consequently
KL ( P t ∥ Q t ) = ∑ j log A t j E t j = ∑ j log ( 1 + g t j − + g t j + E t j ) . (102) \operatorname{KL}(P_t \mathbin{\|} Q_t) = \sum_j \log\frac{A_{tj}}{E_{tj}} = \sum_j \log\left(1 + \frac{g_{tj}^{-} + g_{tj}^{+}}{E_{tj}}\right). \tag*{(102)} KL ( P t ∥ Q t ) = j ∑ log E t j A t j = j ∑ log ( 1 + E t j g t j − + g t j + ) . (102)
The coordinate term belongs to the one face beyond which x x x lies. It therefore gives a signed direction as well as a scalar expansion. A coordinate on which both boxes are flat contributes zero; leaving a flat coordinate gives an infinite term unless a positive regularization scale has been declared.
The order of averaging. (101) averages widths before taking ratios. The per-coordinate mean ( d N ) − 1 ∑ t KL ( P t ∥ Q t ) (dN)^{-1}\sum_t \operatorname{KL}(P_t\mathbin{\|}Q_t) ( d N ) − 1 ∑ t KL ( P t ∥ Q t ) instead measures expansion in each tree before averaging. These are the two orders already distinguished in § 5.2 . They need not have the same ranking and neither uniformly bounds the other. For example, the width pairs ( E , A ) = ( 1 , 2 ) , ( 3 , 3 ) (E,A)=(1,2),(3,3) ( E , A ) = ( 1 , 2 ) , ( 3 , 3 ) give log ( 5 / 4 ) \log(5/4) log ( 5/4 ) for the ratio of means and 1 2 log 2 \frac{1}{2}\log2 2 1 log 2 for the mean log ratio; ( 1 , 1 ) , ( 1 , 3 ) (1,1),(1,3) ( 1 , 1 ) , ( 1 , 3 ) give log 2 \log2 log 2 and 1 2 log 3 \frac{1}{2}\log3 2 1 log 3 . Pooling widths reduces the influence of a nearly flat tree, while the mean per-tree divergence retains that tree’s large relative change. The experiment reports both.
Face Diffusion and Boundary Payoffs. Face diffusion asks whether the query can wander to the outer face of the tree’s box before reaching the local data face. This measures escape separately on each face. The half dimensions of Definition 1 keep the two directions of each coordinate separate. A diffusion in each direction gives both the probability of reaching the outer boundary before the data box and the mean time to reach either boundary. We combine these probabilities in two ways: using the cut law to choose a face, or giving more weight to faces with shorter mean passage times. For face f = ( j , ± ) f=(j,\pm) f = ( j , ± ) at level v v v , orient the coordinate outward. Place the query at zero, the data face at − g v , f < 0 -g_{v,f}<0 − g v , f < 0 , and the outer face of the root box with x x x at ρ f > 0 \rho_f>0 ρ f > 0 . A drift-free diffusion with variance σ f 2 > 0 \sigma_f^2>0 σ f 2 > 0 per unit time has outward exit probability and mean exit time
p v , f o u t = g v , f g v , f + ρ f , m v , f = E 0 τ v , f = g v , f ρ f σ f 2 . (103) p_{v,f}^{\mathrm{out}} = \frac{g_{v,f}}{g_{v,f}+\rho_f}, \qquad m_{v,f} = \mathbb{E}_0\tau_{v,f} = \frac{g_{v,f}\rho_f}{\sigma_f^2}. \tag*{(103)} p v , f out = g v , f + ρ f g v , f , m v , f = E 0 τ v , f = σ f 2 g v , f ρ f . (103)
These follow from the generator A f = ( σ f 2 / 2 ) d 2 / d z 2 \mathcal{A}_f=(\sigma_f^2/2)\,d^2/dz^2 A f = ( σ f 2 /2 ) d 2 / d z 2 : the exit probability solves A f p o u t = 0 \mathcal{A}_f p^{\mathrm{out}}=0 A f p out = 0 with endpoint values 0, 1, and the mean duration solves A f m = − 1 \mathcal{A}_f m=-1 A f m = − 1 with zero endpoint values (Pitman and Yor, 2018). A symmetric walk with steps ± ϵ σ f \pm\epsilon\sigma_f ± ϵ σ f and step duration ϵ 2 \epsilon^2 ϵ 2 gives the same small-step limit. The probability depends on the two distances; the duration also depends on the directional scale σ f \sigma_f σ f .
First choose the face using the cut law. Conditional on separation at v v v , face f f f has probability g v , f / G v g_{v,f}/G_v g v , f / G v . Running its diffusion and recording outward exit gives, after averaging over separation events,
D ( x ) = ∑ v s v R ( B v ∪ { x } ) ∑ f g v , f 2 g v , f + ρ f . (104) D(x)=\sum_v\frac{s_v}{R(B_v\cup\{x\})}\sum_f\frac{g_{v,f}^2}{g_{v,f}+\rho_f}. \tag*{(104)} D ( x ) = v ∑ R ( B v ∪ { x }) s v f ∑ g v , f + ρ f g v , f 2 . (104)
The inner sum uses positive-gap faces. A level with zero separation probability contributes zero. The directional scales affect duration while the combined exit probability stays fixed.
The reported directional statistic weights each face by the reciprocal of its mean duration, ν v , f = 1 / m v , f \nu_{v,f}=1/m_{v,f} ν v , f = 1/ m v , f . Its outward probability is
D σ ( x ) = ∑ v s v G v R ( B v ∪ { x } ) ∑ f ν v , f p v , f o u t ∑ f ν v , f , ν v , f = σ f 2 g v , f ρ f . (105) D_{\sigma}(x)=\sum_v s_v\frac{G_v}{R(B_v\cup\{x\})}\frac{\sum_f\nu_{v,f}p^{\mathrm{out}}_{v,f}}{\sum_f\nu_{v,f}},\qquad\nu_{v,f}=\frac{\sigma_f^2}{g_{v,f}\rho_f}. \tag*{(105)} D σ ( x ) = v ∑ s v R ( B v ∪ { x }) G v ∑ f ν v , f ∑ f ν v , f p v , f out , ν v , f = g v , f ρ f σ f 2 . (105)
Each direction now influences the answer according to how quickly its two distances can be traversed at its own scale. Increasing one σ f \sigma_f σ f increases that face’s weight; multiplying all the σ f \sigma_f σ f by the same factor changes duration but leaves the weights unchanged. Section 8.2 uses the forest means of the cut-box face gaps as these scales. These are specified inverse-duration weights, rather than the joint exit law of simultaneous face diffusions. Eligible faces have positive gap and diffusivity. If an eligible face has ρ f = 0 \rho_f=0 ρ f = 0 , use its limiting outward probability one at that level. Otherwise use the finite rates in (105). With no eligible face, the level contributes zero.
Weighted boundary payoffs. Keep the face diffusions and the cut-law face selection in (104) fixed. Give outward exit on face f f f the declared payoff α f ∈ R \alpha_f\in\mathbb{R} α f ∈ R , with zero payoff at the data face. Collect the directional exit contributions as
d f ( x ) = ∑ v : g v , f > 0 s v R ( B v ∪ { x } ) g v , f 2 g v , f + ρ f , D α ( x ) = ∑ f α f d f ( x ) . (106) d_f(x)=\sum_{v:g_{v,f}>0}\frac{s_v}{R(B_v\cup\{x\})}\frac{g_{v,f}^2}{g_{v,f}+\rho_f},\qquad D_{\alpha}(x)=\sum_f\alpha_f d_f(x). \tag*{(106)} d f ( x ) = v : g v , f > 0 ∑ R ( B v ∪ { x }) s v g v , f + ρ f g v , f 2 , D α ( x ) = f ∑ α f d f ( x ) . (106)
Thus a vector of face contributions supports every fixed linear payoff; α f = 1 \alpha_f=1 α f = 1 recovers (104). The same payoff weighting applies to any fixed face-choice law. On each interval the harmonic field has boundary values zero and α f \alpha_f α f . Its discretizations are the local quadratic potential game of Section 4.1; their interval limit reads α f p v , f o u t \alpha_f p^{\mathrm{out}}_{v,f} α f p v , f out at the query. The interior field values are the players’ actions, while α \alpha α specifies the boundary rewards. Changing α \alpha α changes those rewards while keeping the process fixed.
The face statistic reads one query against fixed boundaries. The stopping worth in Proposition 15 measures the cost saved by adding absorbing members.
The Nyström residual. This asks how much of the query’s kernel feature the few modes of the reference spectral start leave unexplained. It evaluates the feature residual of (65) with the passive row k q k_q k q and the modes retained for the chosen reference model.
Kriging variance. This asks how much uncertainty about the query remains when the whole reference set is available. With unit prior query variance and nugget ϵ > 0 \epsilon> 0 ϵ > 0 , it is 1 − k q T ( K ˉ + ϵ I ) − 1 k q 1-k_q^{\mathsf{T}}(\bar{K}+\epsilon I)^{-1}k_q 1 − k q T ( K ˉ + ϵ I ) − 1 k q . Theorem 14 decomposes this variance using the placement’s first-contact scale. The illustration uses the capped placement below; identification with the original range escape requires uncapped placement and the stored sample as reference.
Rotating Star and Noise Coordinates
The rotating-star example compares two uses of the same forest: clustering a frame after insertion, and reading its queries before insertion. The former has generating blade identities against which assignments can be checked. The anomaly comparison measures agreement with the forest’s own flags.
Clustering protocol. The library example streams 2700 points per frame from five elongated blades, rotated one degree per frame, into 25 trees of 256 points with time decay 1/800 ([67 ] ). After insertion, each frame draws 300 distinct stored points without replacement in proportion to their copies and routes them through N N N trees drawn without replacement. The shared bandwidth is five percent of their median root range. Cluster count uses the largest ratio of successive positive relaxed cuts, up to twelve classes. Thus changing N N N can change both the ensemble and its bandwidth. Table 1 reports a full turn of 360 frames.
The normalized-cut procedure uses the spectral start and Lloyd refinement of § 5.3 . Its representatives come from recursive two-way normalized cuts of each piece ([65 ] ), bisecting the cheapest piece while its two-way cut is at most 0.5, and selecting each piece’s point of largest affinity. CURE uses the library’s multi-centroid summarizer with at most twelve clusters, five representatives per cluster, and shrinkage 0.05 ([34 ] ). The reference partitions are scored by their exact cut under the same K ˉ \bar{K} K ˉ . The frame’s points use the fixed-class mean rule after refinement, or the Nyström readout of the spectral start; CURE assigns by the nearest representative. All these assignments use routed rows.
Table 1 (PDF p. 60). Clustering on 360 frames with 300 reference points. Agreement matches the 2700 observations to their generating blades. Both partitions are scored under the same affinity on frames where both found five classes. Times are clustering plus assignment for the forest cut, and their combined time for CURE, on one core. The Nyström row extends the spectral start.
With N = 25 N = 25 N = 25 the forest cut found five classes on every frame. Where both methods found five, its cut was lower on 84 frames, equal on 254, and higher on 15. Nyström and fixed-class means assigned 99.8–99.9% of points alike; Nyström took about half the assignment time in these runs. Refinement moved references on only five frames. The reported Lloyd pass tests old class means; it does not certify the exact relocation test (85). The collusive-cut theorems describe additional tests, not changes made in this experiment.
Coordinate noise. Pad every point with eight coordinates drawn independently on [ − a , a ] [-a,a] [ − a , a ] each frame, against blade standard deviations 2 along and 0.2 across. This has the form of Corollary 1, with a two-dimensional affine part and W = 8 a W=8a W = 8 a . That corollary controls localization and stretch above 16 W 16W 16 W ; the measured behavior at smaller scales is empirical. Table 2 gives the clustering results and the coordinate selection of (68) . Up to a = 1 a=1 a = 1 the rule selected exactly the star’s two coordinates on 344–356 of 360 frames. When it missed, it retained extra noise coordinates and kept both star coordinates. Recomputing the cut on the selected coordinates restored five classes on every frame and raised agreement at a = 1 a=1 a = 1 from 0.883 to 0.996. At a = 11 a=11 a = 11 , the trees spent most cuts on noise, neither method recovered the blades, and the importance ratios followed the range shares.
Table 2. Eight independent noise coordinates uniform on [ − a , a ] [-a,a] [ − a , a ] , N = 25 N=25 N = 25 trees, and 360 frames. Importance per share (68) is averaged over frames and, in the noise row, over the eight coordinates. The last two rows recompute the cut and bandwidth on the same trees using the selected coordinates.
SOD: deviation in locally tight coordinates. SOD ([40 ] ) chooses a reference set R ( x ) R(x) R ( x ) by shared-neighbor similarity. Let μ R \mu_R μ R and v j v_j v j be its mean and coordinate variances, and set
I R = { j : v j < α d ∑ l v l } , SOD ( x ) = ∑ j ∈ I R ( x j − μ R , j ) 2 ∣ I R ∣ I_R=\left\{j:v_j<\frac{\alpha}{d}\sum_l v_l\right\},\qquad\operatorname{SOD}(x)=\frac{\sqrt{\sum_{j\in I_R}(x_j-\mu_{R,j})^2}}{|I_R|} I R = { j : v j < d α l ∑ v l } , SOD ( x ) = ∣ I R ∣ ∑ j ∈ I R ( x j − μ R , j ) 2
Thus it measures deviation from the reference mean along coordinates on which the references are comparatively tight. The retained coordinate set records which directions are relevant. Use zero when no coordinate is selected.
COF: the cost of connecting to neighbors. COF ([68 ] ) grows a connected set from x x x through its k k k neighbors, repeatedly adding the shortest link from the reached set to an unreached point. If those links have lengths ℓ 1 , … , ℓ k \ell_1,\ldots,\ell_k ℓ 1 , … , ℓ k , its average chaining distance is
a k ( x ) = ∑ i = 1 k 2 ( k − i + 1 ) k ( k + 1 ) ℓ i , COF k ( x ) = a k ( x ) k − 1 ∑ o ∈ N k ( x ) a k ( o ) a_k(x)=\sum_{i=1}^{k}\frac{2(k-i+1)}{k(k+1)}\ell_i,\qquad\operatorname{COF}_k(x)=\frac{a_k(x)}{k^{-1}\sum_{o\in N_k(x)}a_k(o)} a k ( x ) = i = 1 ∑ k k ( k + 1 ) 2 ( k − i + 1 ) ℓ i , COF k ( x ) = k − 1 ∑ o ∈ N k ( x ) a k ( o ) a k ( x )
The reference points’ chaining distances are computed in their own neighborhoods. The statistic compares connection cost, rather than the single-radius reachability density used by LOF.
Measurements before each frame. Each query is evaluated on the forest before its frame is inserted. The comparison reports displacement as a count of old records. For CoDisp, each possible virtual attachment supplies the largest sibling ratio on its augmented query path, including the new leaf. The reported score averages those maxima over attachments and trees. Range escape and gauged diffusion are the forest means of (22) and (105). A tree with zero augmented root range contributes zero to the range mean. The latter uses one set of directional scales per query, with each σ f \sigma_f σ f the mean cut-box gap on that face over trees that supply geometry (§ 3.3 ). Pooled LOF and LOFT are (100), ALOFTμ _\mu μ is (101), and the mean per-tree comparison is ( d N ) − 1 ∑ t KL ( P t ∥ Q t ) (dN)^{-1}\sum_t \operatorname{KL}(P_t\|Q_t) ( d N ) − 1 ∑ t KL ( P t ∥ Q t ) from (102).
The TF0 _0 0 and TF2 _2 2 readings simulate the power allocations (1) on the maintained length-weighted trees. They reweight surviving paths by (7) and use the target separation probabilities (6), with the same inverse-depth payoff (20) for an unseen query. They ask how readily the query would escape if cuts gave equal weight to nonflat coordinates (TF0 _0 0 ), or weight proportional to squared width (TF2 _2 2 ). The first uses the coordinate allocation of isolation forests [48 ] . § 2.2 discusses simulation under these cut laws.
Pre-frame reference diagnostics. The Nyström residual and the Kriging variance use the preceding frame’s reference set and its fixed, routed affinity matrix K ˉ \bar{K} K ˉ . Both use the same passive query row k q k_q k q , averaged over trees and randomized placements as in § 5.1 . In the example, the placement height is capped at the parent of the separation vertex v v v ,
2 H r u n = min { R ( B v ∪ { x } ) , R ( B par ( v ) ) } ( v ≠ root ) , (107) 2H_{\mathrm{run}} = \min\{R(B_v \cup\{x\}), R(B_{\operatorname{par}(v)})\} \qquad(v \ne\operatorname{root}), \tag*{(107)} 2 H run = min { R ( B v ∪ { x }) , R ( B par ( v ) )} ( v = root ) , (107)
and at the root the full augmented range is used. Each placement registers the query’s gap from its separation box and uses routed affinities for references that parted higher up. The Kriging calculation uses unit probe variance and nugget ϵ = 10 − 6 \epsilon= 10^{-6} ϵ = 1 0 − 6 ; both residuals are read before insertion.
Comparing the query readings. On frames with a preceding reference snapshot, every ninth point was read before insertion, giving 2346 flagged and 105354 other points. Table 3 collects agreement with those flags on the unpadded star and at a = 0.5 a = 0.5 a = 0.5 . These are agreement measurements, since the forest’s score supplies the flags.
Table 3. Area under the curve against the forest’s pre-insertion flags. Ordinary arithmetic-mean CoDisp is used. COF and SOD use k = 32 k = 32 k = 32 on the pooled references; SOD uses 16 shared neighbors and variance threshold 0.8. A dash denotes a value not reported here. The ALOFT row uses the reported default-mass run.
The neighborhood being compared matters. Pooled LOF and LOFT at the same k = 32 k = 32 k = 32 had rank correlation 0.38: 32 of a tree’s 256 records is a coarser neighborhood than 32 of roughly 2200 pooled points. LOFT and ALOFT at mass 32 had correlation 0.59. Using the passage box for ALOFT instead gave area 0.419, against 0.973 at the calibration box: the passage vertex is selected by the query’s own expansion, whereas calibration fixes the reference mass first.
The geometry also locates the contribution. In the first 120 frames, 99.6% of flagged points lay at blade tips beyond 1.5 standard deviations along the blade. Their signed expansion vectors had mean ∣ cos θ ∣ |\cos\theta| ∣ cos θ ∣ 0.942 with its long axis, against 0.733 inside a blade. At a = 0.5 a = 0.5 a = 0.5 , these vectors kept 91% of their mass for flagged tip points on the two star coordinates, against 26% inside a blade. The retained faces distinguish extension along a blade from deviation across its narrow side.
The two support residuals lose contrast under noise. At a = 0.5 a = 0.5 a = 0.5 the Kriging variance averaged 0.991 for flagged and 0.979 for other points: both were weakly supported under the chosen geometry. The noise coordinates widen the box at which the probe first meets the references. This is the missing-support term of Theorem 14 ; its initial-rate identification with range escape uses uncapped placement and the stored sample as reference, rather than the capped placement used here.
AI Use Statement
We used multiple generative AI tools for this work. We have reviewed all AI-assisted work.