Introduction
Learning a system’s state requires deciding which differences between its possible conditions matter. Knowing that state would let us reason beyond the aspects we happen to observe. For an animal, a neural recording offers one such view: a pattern of activity from which we seek to understand the condition that gives rise to it [7 , 5 ] . Giving this underlying condition a precise meaning is part of the problem of knowing it.
Figure 1. From observation to prediction. The encoder maps input observations to a learned state ; the predictor estimates observations at the target queries.
Predicting further observations offers a way to study this problem. Neural population models learn to predict held-out activity (Pei et al., 2021), and models trained across recordings use related objectives to support downstream decoding (Ye et al., 2023). A learned state is useful here because it carries information about observations beyond its input.
“When does learning to predict a system amount to learning its state?”
We make this question precise relative to a query universe. A query specifies what is measured, and an observation is the resulting value. Within that universe, the ontic state is the complete response profile: configurations agreeing on every query count as the same state. In a neural recording, a query specifies a unit and a trailing-window duration. An encoder maps input observations to a learned state, from which a predictor estimates observations at target queries (see Figure 1). Theoretical work relates auxiliary prediction and context learning to downstream performance (Lee et al., 2021; Zhai et al., 2025). We ask how much of the ontic state can be recovered from the learned state that supports these predictions.
Consider targets that all depend on one coordinate of a state. Exact prediction constrains that coordinate while leaving the others free. Query ambiguity measures the unresolved state differences. It connects the distinctions tested by prediction to the profile we seek to recover.
This connection motivates three contributions:
Ontic-state formulation. We formulate ontic states as complete response profiles over a query universe.
Recovery guarantees. We establish prediction–recovery guarantees in both general settings and the low-dimensional settings commonly studied in neuroscience.
Fixed-family sharpness. We prove matching budget bounds on a fixed state family, distinguishing targets shared across encoders from targets tailored to each encoder .
Ontic State
State, Query, and Observation
What does it mean for a system to be in a particular state, independently of how we describe it? We need a criterion of identity. Leibniz’s account of individual substance locates identity in a complete concept determining the predicates of its subject (Leibniz, 1686). For a scientific account of state, which properties should determine this identity?
We give these properties empirical meaning through the observations the system could produce [19 ] (Peirce 1878).
Each query specifies one such observation. Two configurations may agree in position and differ in velocity: whether we distinguish them depends on the queries admitted. We therefore fix a query universe before defining its states.
If no question in this universe can distinguish two putative states, we regard them as one ontic state: a difference in description alone does not constitute a difference in state. Their common response profile fixes this identity independently of the coordinates used to represent it (Poincaré, 1905). An experiment reveals part of the profile, while the state specifies responses throughout the universe. We thus model a state as a function from queries to observations.
Definition 2.1 (Ontic State). An ontic state is an admissible bounded function
f : U → R , (1) f:\mathcal{U}\to\mathbb{R}, \tag*{(1)} f : U → R , (1) where U \mathcal{U} U F \mathcal{F} F ontic state space .
Definition 2.2 (Query Universe). The common domain U \mathcal{U} U query universe , and its elements are called queries . For f ∈ F f \in\mathcal{F} f ∈ F q ∈ U q \in\mathcal{U} q ∈ U f ( q ) f(q) f ( q ) observation returned by query q q q f f f
An experiment records responses to a finite set of queries. We collect these readings into an observation vector.
Definition 2.3 (Observation). A query set is a nonempty finite subset Q ⊆ U Q \subseteq\mathcal{U} Q ⊆ U observation vector of a state f ∈ F f \in\mathcal{F} f ∈ F Q Q Q
f ( Q ) : = ( f ( q ) ) q ∈ Q ∈ R Q . (2) f(Q) := \left(f(q)\right)_{q\in Q} \in\mathbb{R}^{Q}. \tag*{(2)} f ( Q ) := ( f ( q ) ) q ∈ Q ∈ R Q . (2) Here R Q \mathbb{R}^{Q} R Q Q Q Q
To compare two profiles, we ask how far their responses can differ over the queries being considered. Taking the largest difference gives a common distance for full states and finite observation vectors.
Definition 2.4 (Ontic Distance). For a nonempty index set I I I u u u v v v I I I I I I
d I ( u , v ) ≔ sup i ∈ I ∣ u ( i ) − v ( i ) ∣ . (3) d_I(u,v) \coloneqq\sup_{i \in I} \lvert u(i)-v(i)\rvert. \tag*{(3)} d I ( u , v ) : = i ∈ I sup ∣ u ( i ) − v ( i )∣ . (3) This is the supremum metric between their restrictions to I I I I I I
The ontic distance is d U ( f , g ) d_{\mathcal{U}}(f,g) d U ( f , g ) observation distance on Q Q Q d Q ( f ( Q ) , g ( Q ) ) d_Q(f(Q),g(Q)) d Q ( f ( Q ) , g ( Q )) Q Q Q
Figure 2 shows how a state and query set determine an observation vector. The noisy extension treats the profile as a conditional mean response and measured activity as a noisy observation (section E).
Figure 2. State, query, and observation. The state f ∈ F f \in\mathcal{F} f ∈ F Q = { q 1 , q 2 } ⊆ U Q = \{q_1,q_2\} \subseteq\mathcal{U} Q = { q 1 , q 2 } ⊆ U f ( Q ) f(Q) f ( Q ) f ( q 1 ) f(q_1) f ( q 1 ) f ( q 2 ) f(q_2) f ( q 2 ) Q Q Q
Query Ambiguity
A complete response profile defines a state, but how much of it do our observations determine? Hume’s question about inference beyond present observation brings this gap into focus [18 sec. IV] . Within a specified model class, optimal recovery makes the uncertainty quantitative [3 ] (Micchelli and Rivlin 1977). We ask how different two full states can be when their recorded observations agree within a given tolerance.
Definition 2.5 (Query Ambiguity). Fix a query set Q ⊆ U Q \subseteq\mathcal{U} Q ⊆ U r ≥ 0 r \ge0 r ≥ 0 query ambiguity on Q Q Q
ω Q ( r ) ≔ sup f , g ∈ F d Q ( f ( Q ) , g ( Q ) ) ≤ r d U ( f , g ) . (4) \omega_Q(r) \coloneqq\sup_{\substack{f,g \in\mathcal{F}\\ d_Q(f(Q),g(Q)) \le r}} d_{\mathcal{U}}(f,g). \tag*{(4)} ω Q ( r ) : = f , g ∈ F d Q ( f ( Q ) , g ( Q )) ≤ r sup d U ( f , g ) . (4) At r = 0 r = 0 r = 0 Q Q Q observation decoder recovers a state from the observation vector.
Definition 2.6 (Observation Decoder). Fix a query set Q ⊆ U Q \subseteq\mathcal{U} Q ⊆ U observation decoder class by
A Q ≔ { a Q : R Q → F } . (5) \mathcal{A}_Q \coloneqq\{a_Q : \mathbb{R}^{Q} \to\mathcal{F}\}. \tag*{(5)} A Q : = { a Q : R Q → F } . (5) For an observation decoder a Q ∈ A Q a_Q \in\mathcal{A}_Q a Q ∈ A Q recovery error is
E Q ( a Q ) ≔ sup f ∈ F d U ( a Q ( f ( Q ) ) , f ) . (6) \mathcal{E}_Q(a_Q) \coloneqq\sup_{f \in\mathcal{F}} d_{\mathcal{U}}(a_Q(f(Q)),f). \tag*{(6)} E Q ( a Q ) : = f ∈ F sup d U ( a Q ( f ( Q )) , f ) . (6) The optimal recovery error from observations on Q Q Q
E Q ∗ ≔ inf a Q ∈ A Q E Q ( a Q ) . (7) \mathcal{E}_Q^* \coloneqq\inf_{a_Q \in\mathcal{A}_Q} \mathcal{E}_Q(a_Q). \tag*{(7)} E Q ∗ : = a Q ∈ A Q inf E Q ( a Q ) . (7) Additional queries can resolve distinctions left open by the current observations. We describe this accumulation of information by a nested query sequence.
Definition 2.7 (Query Sequence). A query sequence is a family ( Q p ) p ≥ 1 (Q_p)_{p \ge1} ( Q p ) p ≥ 1 Q p ⊆ Q p + 1 ⊆ U Q_p \subseteq Q_{p+1} \subseteq\mathcal{U} Q p ⊆ Q p + 1 ⊆ U p ≥ 1 p \ge1 p ≥ 1 p p p f ( Q p ) f(Q_p) f ( Q p )
Definition 2.8 (State Separation). A query sequence ( Q p ) p ≥ 1 (Q_p)_{p \ge1} ( Q p ) p ≥ 1 separate states in F \mathcal{F} F if
∀ f ≠ g ∈ F , ∃ p ≥ 1 : f ( Q p ) ≠ g ( Q p ) . (8) \forall f \ne g \in\mathcal{F},\quad\exists p \ge1:\quad f(Q_p) \ne g(Q_p). \tag*{(8)} ∀ f = g ∈ F , ∃ p ≥ 1 : f ( Q p ) = g ( Q p ) . (8) Separating queries progressively resolve observation ambiguity (figure 8). Compactness makes this resolution stable under vanishing tolerances.
Lemma 2.9 (Vanishing Ambiguity). Let ( F , d U ) (\mathcal{F},d_{\mathcal{U}}) ( F , d U ) ( P m ) m ≥ 1 (P_m)_{m \ge1} ( P m ) m ≥ 1 query sequence separating states. For nonnegative tolerances r m → 0 r_m \to0 r m → 0
ω P m ( r m ) → 0. (9) \omega_{P_m}(r_m) \to0. \tag*{(9)} ω P m ( r m ) → 0. (9) Prediction and Recovery
What must a learned representation preserve for us to recover the state it describes? Hertz’s account of representation centers on the correspondence between consequences [17 Introduction] . Here we test that correspondence by predicting further responses from encoded observations. The preceding lemma explains why the targets matter: agreement on sufficiently rich readings constrains the difference between full states. Prediction can thereby test whether a representation retains the distinctions defining the state.
Definition 2.10 (Prediction and Recovery). The learning pipeline uses two disjoint query sets: Q Q Q f ( Q ) f(Q) f ( Q ) P P P f ( P ) f(P) f ( P ) encoder e : R Q → Z e : \mathbb{R}^{Q} \to\mathcal{Z} e : R Q → Z f ( Q ) f(Q) f ( Q ) e ( f ( Q ) ) e(f(Q)) e ( f ( Q )) Z \mathcal{Z} Z predictor b : Z → R P b : \mathcal{Z} \to\mathbb{R}^{P} b : Z → R P f ( P ) f(P) f ( P ) state decoder a : Z → F a : \mathcal{Z} \to\mathcal{F} a : Z → F f f f
L Q , P ( b ; e ) ≔ sup f ∈ F d P ( ( b ∘ e ) ( f ( Q ) ) , f ( P ) ) , R Q ( a ; e ) ≔ sup f ∈ F d U ( ( a ∘ e ) ( f ( Q ) ) , f ) . (10) \begin{aligned}
\mathcal{L}_{Q,P}(b;e) &\coloneqq\sup_{f \in\mathcal{F}} d_P\bigl((b \circ e)(f(Q)), f(P)\bigr),\\
\mathcal{R}_{Q}(a;e) &\coloneqq\sup_{f \in\mathcal{F}} d_{\mathcal{U}}\bigl((a \circ e)(f(Q)), f\bigr). \tag*{(10)}
\end{aligned} L Q , P ( b ; e ) R Q ( a ; e ) : = f ∈ F sup d P ( ( b ∘ e ) ( f ( Q )) , f ( P ) ) , : = f ∈ F sup d U ( ( a ∘ e ) ( f ( Q )) , f ) . (10) Let B P \mathcal{B}_P B P A \mathcal{A} A encoder , define
L Q , P ⋆ ( e ) ≔ inf b ∈ B P L Q , P ( b ; e ) , R Q ⋆ ( e ) ≔ inf a ∈ A R Q ( a ; e ) . (11) \mathcal{L}^{\star}_{Q,P}(e) \coloneqq\inf_{b \in\mathcal{B}_P} \mathcal{L}_{Q,P}(b;e), \qquad\mathcal{R}^{\star}_{Q}(e) \coloneqq\inf_{a \in\mathcal{A}} \mathcal{R}_{Q}(a;e). \tag*{(11)} L Q , P ⋆ ( e ) : = b ∈ B P inf L Q , P ( b ; e ) , R Q ⋆ ( e ) : = a ∈ A inf R Q ( a ; e ) . (11) Both optimizations hold the encoder fixed; R Q ⋆ ( e ) \mathcal{R}^{\star}_{Q}(e) R Q ⋆ ( e )
Corollary 2.11 (Input Ambiguity). For a nonempty state family, query set Q Q Q e e e
R Q ⋆ ( e ) ≥ E Q ⋆ ≥ 1 2 ω Q ( 0 ) . (12) \mathcal{R}^{\star}_{Q}(e) \ge\mathcal{E}^{\star}_{Q} \ge\frac{1}{2}\omega_{Q}(0). \tag*{(12)} R Q ⋆ ( e ) ≥ E Q ⋆ ≥ 2 1 ω Q ( 0 ) . (12) Input queries supply distinctions to the encoder ; targets test their preservation in its learned state.
Low-Dimensional Geometry
Figure 3. From population activity to state geometry. The neural inset motivates a low-dimensional model for the ontic state space F \mathcal{F} F
A neural recording contains many coordinates, yet activity within a task can vary through a few latent variables [11 , 20 ] . This organization can constrain which patterns are learned (Sadtler et al., 2014) and take forms such as the toroidal geometry of grid-cell activity [12 ] . The resulting state family may curve through a much larger linear space [9 ] .
Recovering a state within such a family requires responses that distinguish its members. Classical population decoding illustrates how neural responses carry this information: movement direction can be read from motor-cortical ensembles [13 ] , and position reconstructed from hippocampal place-cell activity (Zhang et al., 1998). Models of population codes explain how neuronal tuning and background activity affect readout accuracy (Seung and Sompolinsky, 1993).
These observations lead to two quantitative requirements (figure 3). The state family should admit a controlled number of representatives at each resolution, and queries outside the inputs should distinguish its members. Covering numbers express the first requirement; observability expresses the second.
Assumption 2.12 (Low-Dimensional Observability). For a query set Q Q Q U ∖ Q ≠ ∅ \mathcal{U} \setminus Q \ne\varnothing U ∖ Q = ∅ C ≥ 1 C \ge1 C ≥ 1 d > 0 d > 0 d > 0 α ∈ ( 0 , 1 ] \alpha\in(0,1] α ∈ ( 0 , 1 ]
N F ( ε ) ≤ C ε − d , 0 < ε ≤ 1 , (13) \mathcal{N}_{\mathcal{F}}(\varepsilon) \le C\varepsilon^{-d}, \qquad0 < \varepsilon\le1, \tag*{(13)} N F ( ε ) ≤ C ε − d , 0 < ε ≤ 1 , (13) where N F ( ε ) \mathcal{N}_{\mathcal{F}}(\varepsilon) N F ( ε ) d U d_{\mathcal{U}} d U ε \varepsilon ε F \mathcal{F} F F \mathcal{F} F Q Q Q
sup p ∈ U ∖ Q ∣ f ( p ) − g ( p ) ∣ ≥ α d U ( f , g ) , f , g ∈ F . (14) \sup_{p\in\mathcal{U}\setminus Q} \lvert f(p)-g(p)\rvert\ge\alpha d_{\mathcal{U}}(f,g), \qquad f,g\in\mathcal{F}. \tag*{(14)} p ∈ U ∖ Q sup ∣ f ( p ) − g ( p )∣ ≥ α d U ( f , g ) , f , g ∈ F . (14) Main Results
Prediction tests the distinctions retained by a learned state. If an encoder maps two states to the same learned state, every predictor must return the same target observations for both. Their target difference therefore imposes an unavoidable prediction error.
Lemma 3.1 (Prediction within Fibers). Assume ( F , d U ) (\mathcal{F},d_{\mathcal{U}}) ( F , d U ) Q ⊆ U Q\subseteq\mathcal{U} Q ⊆ U P ⊆ U ∖ Q P\subseteq\mathcal{U}\setminus Q P ⊆ U ∖ Q e : R Q → Z e:\mathbb{R}^{Q}\to\mathcal{Z} e : R Q → Z
L Q , P ⋆ ( e ) = 1 2 sup f , g ∈ F e ( f ( Q ) ) = e ( g ( Q ) ) d P ( f ( P ) , g ( P ) ) . (15) \mathcal{L}^{\star}_{Q,P}(e)=\frac{1}{2}\sup_{\substack{f,g\in\mathcal{F}\\e(f(Q))=e(g(Q))}}d_{P}(f(P),g(P)). \tag*{(15)} L Q , P ⋆ ( e ) = 2 1 f , g ∈ F e ( f ( Q )) = e ( g ( Q )) sup d P ( f ( P ) , g ( P )) . (15) In the one-coordinate example, two states can have identical target readings while differing elsewhere. A predictor can be exact on those targets even if its encoder merges the states. Recovery depends on how much full-state difference those same readings permit.
General Case
Target ambiguity controls how prediction accuracy translates into state recovery.
Theorem 3.2 (General Prediction-Recovery Bound). Let ( F , d U ) (\mathcal{F},d_{\mathcal{U}}) ( F , d U ) For disjoint query sets Q Q Q P P P e : R Q → Z e:\mathbb{R}^{Q}\to\mathcal{Z} e : R Q → Z
L Q , P ⋆ ( e ) ≤ ⌈ R Q ⋆ ( e ) ⌉ ≤ ω P ( 2 L Q , P ⋆ ( e ) ) . (16) \mathcal{L}^{\star}_{Q,P}(e)\le\left\lceil\mathcal{R}^{\star}_{Q}(e)\right\rceil\le\omega_{P}\left(2\mathcal{L}^{\star}_{Q,P}(e)\right). \tag*{(16)} L Q , P ⋆ ( e ) ≤ ⌈ R Q ⋆ ( e ) ⌉ ≤ ω P ( 2 L Q , P ⋆ ( e ) ) . (16) The upper bound also holds with L Q , P ( b ; e ) \mathcal{L}_{Q,P}(b;e) L Q , P ( b ; e ) for any predictor b b b with finite error.
When the state space is compact, a growing set of separating targets resolves the remaining ambiguity. Prediction and recovery then converge together.
Theorem 3.3 (General Prediction-Recovery Equivalence). Let ( F , d U ) (\mathcal{F},d_{\mathcal{U}}) ( F , d U ) be nonempty and compact, and let ( P m ) m ≥ 1 (P_m)_{m\ge1} ( P m ) m ≥ 1 be a nested query sequence separating states. For each m m m choose a query set Q m ⊆ U ∖ P m Q_m\subseteq\mathcal{U}\setminus P_m Q m ⊆ U ∖ P m and an encoder e m : R Q m → Z m e_m:\mathbb{R}^{Q_m}\to\mathcal{Z}_m e m : R Q m → Z m Then
L Q m , P m ⋆ ( e m ) → 0 ⟺ R Q m ⋆ ( e m ) → 0. (17) \mathcal{L}^{\star}_{Q_m,P_m}(e_m)\to0 \quad\Longleftrightarrow\quad\mathcal{R}^{\star}_{Q_m}(e_m)\to0. \tag*{(17)} L Q m , P m ⋆ ( e m ) → 0 ⟺ R Q m ⋆ ( e m ) → 0. (17) For fixed input queries and a fixed encoder , vanishing prediction error along such a separating sequence leaves no distinct ontic states with the same learned state: an exact state decoder exists (corollary C.1). If the map to a Euclidean learned state is also continuous, compactness makes it a homeomorphism onto its image (corollary C.2). Under these conditions, preserving the distinctions that define states also preserves the topology of the state space.
Low-Dimensional Case
Assumption 2.12 combines a bound on the number of representative states with the ability to distinguish them through queries outside Q Q Q
Theorem 3.4 (Low-Dimensional Prediction-Recovery Bound). Let F \mathcal{F} F be nonempty and let ( F , Q ) (\mathcal{F},Q) ( F , Q ) satisfy assumption 2.12 with constants C C C d d d α \alpha α For every integer m ≥ C 2 / 2 m\ge C^{2}/2 m ≥ C 2 /2 there is a target set P ⊆ U ∖ Q P\subseteq\mathcal{U}\setminus Q P ⊆ U ∖ Q 1 ≤ ∣ P ∣ ≤ m 1\le\lvert P\rvert\le m 1 ≤ ∣ P ∣ ≤ m such that every encoder e e e satisfies
L Q , P ⋆ ( e ) ≤ ⌈ R Q ⋆ ( e ) ⌉ ≤ 2 α L Q , P ⋆ ( e ) + 2 + α α ( C 2 2 m ) 1 2 d . (18) \mathcal{L}^{\star}_{Q,P}(e)\le\left\lceil\mathcal{R}^{\star}_{Q}(e)\right\rceil\le\frac{2}{\alpha}\mathcal{L}^{\star}_{Q,P}(e)+\frac{2+\alpha}{\alpha}\left(\frac{C^{2}}{2m}\right)^{\frac{1}{2d}}. \tag*{(18)} L Q , P ⋆ ( e ) ≤ ⌈ R Q ⋆ ( e ) ⌉ ≤ α 2 L Q , P ⋆ ( e ) + α 2 + α ( 2 m C 2 ) 2 d 1 . (18) The upper bound also holds with L Q , P ( b ; e ) \mathcal{L}_{Q,P}(b;e) L Q , P ( b ; e ) for any predictor b b b with finite error.
The recovery allowance vanishes as the target budget grows, yielding convergence and rate equivalences.
Theorem 3.5 (Low-Dimensional Prediction-Recovery Equivalence). Let F \mathcal{F} F Assume ( F , Q m ) (\mathcal{F},Q_m) ( F , Q m ) assumption 2.12 with common C C C d d d α \alpha α m ≥ C d 2 / 2 m \ge C_d^2/2 m ≥ C d 2 /2 P m P_m P m theorem 3.4 . Then every encoder sequence ( e m ) (e_m) ( e m )
L Q m , P m ⋆ ( e m ) → 0 ⟺ R Q m ⋆ ( e m ) → 0. (19) \mathcal{L}^{\star}_{Q_m,P_m}(e_m) \to0 \quad\Longleftrightarrow\quad\mathcal{R}^{\star}_{Q_m}(e_m) \to0. \tag*{(19)} L Q m , P m ⋆ ( e m ) → 0 ⟺ R Q m ⋆ ( e m ) → 0. (19) Moreover,
L Q m , P m ⋆ ( e m ) = O ( m − 1 / ( 2 d ) ) ⟺ R Q m ⋆ ( e m ) = O ( m − 1 / ( 2 d ) ) . (20) \mathcal{L}^{\star}_{Q_m,P_m}(e_m) = O(m^{-1/(2d)}) \quad\Longleftrightarrow\quad\mathcal{R}^{\star}_{Q_m}(e_m) = O(m^{-1/(2d)}). \tag*{(20)} L Q m , P m ⋆ ( e m ) = O ( m − 1/ ( 2 d ) ) ⟺ R Q m ⋆ ( e m ) = O ( m − 1/ ( 2 d ) ) . (20) Sharpness on a Fixed State Family
We next examine whether the budget rate can be improved when one target set serves every encoder . For α = 1 \alpha= 1 α = 1 2 L Q , P ⋆ ( e ) 2\mathcal{L}^{\star}_{Q,P}(e) 2 L Q , P ⋆ ( e ) d d d
Definition 3.6 (Uniform Transfer Residual). Let E L i p \mathcal{E}_{\mathrm{Lip}} E Lip e : R Q → R 2 e:\mathbb{R}^{Q}\to\mathbb{R}^{2} e : R Q → R 2 m ≥ 1 m \ge1 m ≥ 1 uniform transfer residual is
T m ( F , Q ) : = inf P ⊆ U ∖ Q 1 ≤ ∣ P ∣ ≤ m sup e ∈ E L i p [ R Q ⋆ ( e ) − 2 L Q , P ⋆ ( e ) ] . (21) \mathcal{T}_m(\mathcal{F},Q) := \inf_{\substack{P\subseteq\mathcal{U}\setminus Q\\1\leq|P|\leq m}}\ \sup_{e\in\mathcal{E}_{\mathrm{Lip}}}\left[\mathcal{R}^{\star}_{Q}(e)-2\mathcal{L}^{\star}_{Q,P}(e)\right]. \tag*{(21)} T m ( F , Q ) := P ⊆ U ∖ Q 1 ≤ ∣ P ∣ ≤ m inf e ∈ E Lip sup [ R Q ⋆ ( e ) − 2 L Q , P ⋆ ( e ) ] . (21) The target set is selected before the encoder . The family below has identifying inputs and even a single query outside Q Q Q encoder can nevertheless merge distinct states. The residual measures the additive cost of controlling their recovery by twice the prediction error, uniformly across encoders .
Theorem 3.7 (Sharp Uniform Transfer). For every d > 1 d>1 d > 1 U \mathcal{U} U Q Q Q F \mathcal{F} F [ 0 , 1 ] [0,1] [ 0 , 1 ] The pair ( F , Q ) (\mathcal{F},Q) ( F , Q ) assumption 2.12 with C = C d C=C_d C = C d α = 1 \alpha=1 α = 1 m ≥ C d 2 / 2 m \ge C_d^2/2 m ≥ C d 2 /2
1 6 m − 1 / ( 2 d ) ≤ T m ( F , Q ) ≤ 3 ( C d 2 2 m ) 1 / ( 2 d ) . (22) \frac{1}{6}m^{-1/(2d)} \leq\mathcal{T}_m(\mathcal{F},Q) \leq3\left(\frac{C_d^2}{2m}\right)^{1/(2d)}. \tag*{(22)} 6 1 m − 1/ ( 2 d ) ≤ T m ( F , Q ) ≤ 3 ( 2 m C d 2 ) 1/ ( 2 d ) . (22) The supremum in equation (21) has the same value when restricted to 1-Lipschitz encoders.
The order of selection matters as well. On this same family, a target selected for one fixed 1-Lipschitz encoder can achieve R Q ⋆ ( e ) ≤ 2 L Q , { p } ⋆ ( e ) \mathcal{R}^{\star}_{Q}(e) \leq2\mathcal{L}^{\star}_{Q,\{p\}}(e) R Q ⋆ ( e ) ≤ 2 L Q , { p } ⋆ ( e ) encoder gives the sharp budget in theorem 3.7. Classical optimal recovery studies the radius left by fixed information [23 ] (Traub et al. 1983); our budget measures how many prediction targets suffice to control that radius uniformly across learned representations.
From Recorded Error to Recovery
The bounds above use prediction error over the entire state family. Training supplies finitely many observation pairs. To pass from recorded error to a recovery guarantee, we need to know both how these records cover possible observations and how predictions vary between them.
Definition 3.8 (Recorded Error and Coverage). Fix disjoint query sets Q Q Q P P P ( e , b ) (e,b) ( e , b ) f 1 , … , f n ∈ F f_1,\ldots,f_n\in\mathcal{F} f 1 , … , f n ∈ F n ≥ 1 n\geq1 n ≥ 1 ( f i ( Q ) , f i ( P ) ) (f_i(Q),f_i(P)) ( f i ( Q ) , f i ( P )) empirical prediction error and observation coverage radius are
L ^ Q , P , n ( b ; e ) : = max 1 ≤ i ≤ n d P ( ( b ∘ e ) ( f i ( Q ) ) , f i ( P ) ) , (23) \widehat{\mathcal{L}}_{Q,P,n}(b;e) := \max_{1\leq i\leq n} d_P\left((b\circ e)(f_i(Q)),f_i(P)\right), \tag*{(23)} L Q , P , n ( b ; e ) := 1 ≤ i ≤ n max d P ( ( b ∘ e ) ( f i ( Q )) , f i ( P ) ) , (23) h n : = sup f ∈ F min 1 ≤ i ≤ n d Q ∪ P ( f ( Q ∪ P ) , f i ( Q ∪ P ) ) . (24) h_n := \sup_{f\in\mathcal{F}}\min_{1\leq i\leq n} d_{Q\cup P}\left(f(Q\cup P),f_i(Q\cup P)\right). \tag*{(24)} h n := f ∈ F sup 1 ≤ i ≤ n min d Q ∪ P ( f ( Q ∪ P ) , f i ( Q ∪ P ) ) . (24) The radius measures coverage jointly in inputs and targets: every possible observation pair lies within h n h_n h n
Assumption 3.9 (Prediction Stability). For the pair ( e , b ) (e,b) ( e , b ) ρ : [ 0 , ∞ ) → [ 0 , ∞ ) \rho: [0,\infty) \to[0,\infty) ρ : [ 0 , ∞ ) → [ 0 , ∞ ) ρ ( 0 ) = 0 \rho(0)=0 ρ ( 0 ) = 0 ρ ( t ) → 0 \rho(t) \to0 ρ ( t ) → 0 t ↓ 0 t \downarrow0 t ↓ 0 f , g ∈ F f,g \in\mathcal{F} f , g ∈ F
d P ( ( b ∘ e ) ( f ( Q ) ) , ( b ∘ e ) ( g ( Q ) ) ) ≤ ρ ( d Q ( f ( Q ) , g ( Q ) ) ) . (25) d_P\left((b \circ e)(f(Q)),(b \circ e)(g(Q))\right) \leq\rho\left(d_Q(f(Q),g(Q))\right). \tag*{(25)} d P ( ( b ∘ e ) ( f ( Q )) , ( b ∘ e ) ( g ( Q )) ) ≤ ρ ( d Q ( f ( Q ) , g ( Q )) ) . (25) For a nearby record, the prediction changes by at most ρ ( h n ) \rho(h_n) ρ ( h n ) h n h_n h n ρ ( t ) = K t \rho(t)=Kt ρ ( t ) = K t
Proposition 3.10 (Recovery from Records). Let ( F , d U ) (\mathcal{F},d_U) ( F , d U ) ( e , b ) (e,b) ( e , b )
E n : = L ^ Q , P , n ( b ; e ) + ρ ( h n ) + h n , (26) E_n := \widehat{L}_{Q,P,n}(b;e) + \rho(h_n) + h_n, \tag*{(26)} E n := L Q , P , n ( b ; e ) + ρ ( h n ) + h n , (26) L ^ Q , P , n ( b ; e ) ≤ L Q , P ( b ; e ) ≤ E n , (27) \widehat{L}_{Q,P,n}(b;e) \leq L_{Q,P}(b;e) \leq E_n, \tag*{(27)} L Q , P , n ( b ; e ) ≤ L Q , P ( b ; e ) ≤ E n , (27) and the recovery error satisfies
R Q ⋆ ( e ) ≤ ω P ( 2 E n ) . (28) R_Q^\star(e) \leq\omega_P(2E_n). \tag*{(28)} R Q ⋆ ( e ) ≤ ω P ( 2 E n ) . (28) Under assumption 2.12, for targets selected by theorem 3.4,
R Q ⋆ ( e ) ≤ 2 α E n + 2 + α α ( C 2 2 m ) 1 / ( 2 d ) . (29) R_Q^\star(e) \leq\frac{2}{\alpha}E_n + \frac{2+\alpha}{\alpha}\left(\frac{C^2}{2m}\right)^{1/(2d)}. \tag*{(29)} R Q ⋆ ( e ) ≤ α 2 E n + α 2 + α ( 2 m C 2 ) 1/ ( 2 d ) . (29) The terms answer different questions. The empirical error measures fit on recorded pairs. Their coverage and the prediction modulus control what this fit implies between records. The target budget controls distinctions left unresolved even by a uniformly accurate predictor . More training records and more target queries address these two sources of uncertainty separately. The fine-tuning results in theorems theorems D.4 and D.5 include model-class error and optimization tolerance; section section E also treats measurement perturbations.
Experiment
The experiments examine prediction and recovery with two kinds of access to the state. On Sphere , Torus , and General families, the known response law makes the complete profile available for evaluation. Random target sampling tests separation and budget dependence; theorem 3.7 supplies the sharpness result analytically. Neural recordings then let us examine covering geometry, held-out prediction, and recovery of a fixed model’s hidden state as an ontic-state surrogate.
Synthetic Experiment
We use three synthetic state families (Figure 4 ): Sphere is uniform on a unit sphere; Torus consists of independent circle factors; General has independent uniform coordinates in a high-dimensional box. A common observation law embeds them into the same query universe. We jointly train an encoder and predictor on input–target observations, then recover the state through a deterministic ground-truth (GT) map, following theorem 3.2. Here the coordinate structure determines the random-sampling budget; the experiment tests how target separation transfers prediction accuracy to recovery.
Figure 4. Synthetic state families. Representative low-dimensional views of the sphere, product of circles and independent box.
For a latent vector x ∈ [ − 1 , 1 ] r x \in[-1,1]^r x ∈ [ − 1 , 1 ] r
f x ( j , k ) = tanh ( a j k s j x i ( j ) + c j k ) , (30) f_x(j,k) = \tanh(a_{jk}s_jx_{i(j)} + c_{jk}), \tag*{(30)} f x ( j , k ) = tanh ( a j k s j x i ( j ) + c j k ) , (30) where i ( j ) i(j) i ( j ) s j ∈ { − 1 , 1 } s_j \in\{-1,1\} s j ∈ { − 1 , 1 } a j k > 0 , c j k ∈ R a_{jk}>0,c_{jk} \in\mathbb{R} a j k > 0 , c j k ∈ R
Given predicted target observations y ∈ R P y \in\mathbb{R}^{P} y ∈ R P
f ^ ( y ) ∈ arg min g ∈ F d P ( g ( P ) , y ) . (31) \hat{f}(y) \in\arg\min_{g \in\mathcal{F}} d_{P}(g(P),y). \tag*{(31)} f ^ ( y ) ∈ arg g ∈ F min d P ( g ( P ) , y ) . (31) The known observation law converts target-error tolerances into coordinate intervals. A deterministic feasibility calculation approximates this map (section F) and supplies the complete recovered profile, including responses outside the target set.
Observability check. To apply theorem 3.2, we need to control target ambiguity. Complete coordinate coverage gives an explicit constant before training.
Proposition 4.1 (Coordinate Observability). For the family in equation (30) , suppose P P P ℓ i \ell_i ℓ i a j k sech 2 ( a j k + ∣ c j k ∣ ) a_{jk}\operatorname{sech}^{2}(a_{jk}+\lvert c_{jk}\rvert) a j k sech 2 ( a j k + ∣ c j k ∣) i i i A i A_i A i a j k a_{jk} a j k f , g ∈ F f,g \in\mathcal{F} f , g ∈ F
α P = min i ℓ i A i > 0 , d P ( f , g ) ≥ α P d U ( f , g ) . (32) \alpha_{P}=\min_{i}\frac{\ell_i}{A_i}>0,\qquad d_{P}(f,g)\geq\alpha_{P}d_{U}(f,g). \tag*{(32)} α P = i min A i ℓ i > 0 , d P ( f , g ) ≥ α P d U ( f , g ) . (32) This inequality gives ω P ( t ) ≤ t / α P \omega_{P}(t)\leq t/\alpha_{P} ω P ( t ) ≤ t / α P R Q ⋆ ( ϵ ) ≤ 2 L Q , P ( b ; ϵ ) / α P R^{\star}_{Q}(\epsilon)\leq2L_{Q,P}(b;\epsilon)/\alpha_{P} R Q ⋆ ( ϵ ) ≤ 2 L Q , P ( b ; ϵ ) / α P Table 7 and section F.
Table 7: Smallest budget for which equation (132) gives at least 95% probability of separation. The pool contains 32,768 queries; r r r d d d 4.18 × 10 − 6 4.18 \times10^{-6} 4.18 × 1 0 − 6
Table 7.
Error transfer. With target separation established, Figure 5 follows the prediction-to-recovery relation. The left panel compares finite-test errors with the exact-map benchmark. The middle panel gives the sufficient separation budget under the coordinate-sampling law. The right panel extends the recovery comparison to the full known family through the conditional numerical audit below.
Figure 5. Prediction-to-recovery transfer. Left: test errors and exact-map benchmark. Middle: budgets for 95% separation. Right: conditional whole-domain bounds. Dashed lines: independent-family budget and profile diameter. Bars: repetition standard deviations. See Table 1 and Table 7 .
Whole-domain audit. A test set samples prediction error; the theoretical quantity is a supremum over the state family. Where proposition 3.10 controls this supremum through observation coverage, the known coordinatewise law permits a scalar-domain audit. For each target, its error is a scalar function of the observed latent coordinate. A fixed grid and a sensitivity bound computed from the saved weights enclose its error between grid points. Under the arithmetic-error assumptions in section F.5, the grid audit gives a conditional upper bound throughout the known family. Applying the ideal GT map yields Table 1 .
Table 1. Conditional whole-domain bounds for the ideal GT map: mean ± \pm ±
Every paired run at the audited separating budgets yields an ideal-map bound below the full-profile diameter (Table 1 ). The bound is evaluated for each trained predictor using the protocol in section F.5.
For joint queries, section G derives a global prediction-to-recovery certificate from calibration records at known latent coordinates.
Real Experiment
In neural recordings, we examine covering complexity, held-out target prediction, and recovery of a fixed reference model’s hidden coordinates. These coordinates define the surrogate state whose preservation we evaluate.
Observation geometry. The HC-3 hippocampal–entorhinal recordings ((Mizuseki et al., 2014)) span animals and behavioral contexts. Chronological blocks separate preprocessing, reference geometry, diagnostic geometry, and prediction evaluation (section H.2).
We cover the diagnostic block with observed representatives in the maximum-coordinate metric (Figure 6 ). At a fixed resolution, every session needs a smaller greedy cover than its median time-shift control, with a median reduction of 15.2%. These controls preserve each unit’s marginal distribution and circular autocorrelation while changing alignment between units. The comparison holds within every selected animal and across shorter and longer counting windows (Table 2 , Table 3 and Table 13 ). Covering curves measure the finite-record analog of the approximation complexity in equation (13) . Variance-retention ranks measure concentration along reference-derived covariance directions.
Figure 6. HC-3 covering geometry. Left: fixed-radius cover sizes; bars span shift controls. Middle: session reductions relative to median shift controls, grouped by animal; bars mark animal medians. Right: median covering curves with session interquartile bands. See section H.2.
Table 2. Diagnostic-block greedy covers at the fixed radius in Table 12 . Variance-retention ranks use reference-derived directions.
Table 3. Cover-size reduction within each selected animal. Values are session medians relative to the matched time-shift median at the fixed radius. Every selected session contributes.
Table 13. Fixed-radius cover-size comparison over the full cohort. Each session has a smaller cover than its median time-shift control at every window.
Data and controls use matched radii and sample sizes (section H.2). Table 3 compares counting windows within each animal.
Target prediction. We next ask whether a learned representation predicts a disjoint target set. An encoder–predictor learns from the same sessions’ training blocks and is evaluated on later held-out observations. At the larger target budgets, prediction improves on a constant predictor derived from the training block (Figure 7 , left).
Figure 7. Prediction and reference-state recovery. Left: prediction gain. Middle: recovery gain with session (S) or animal (A) weighting. Right: animal gains. Repetition means and standard deviations; see Table 19 and Table 21 .
Table 19: Prediction gain, mean ± \pm ±
Table 19.
Reference state recovery. The trained encoder above supplies the reference model: its full hidden-coordinate vector defines each observation’s surrogate state. A smaller model learns the same prediction task independently, using the same observations and chronological split. We then recover the reference state by local interpolation in the smaller model’s learned space: reference states at nearby training inputs are averaged with distance-based weights. The reference states enter the recovery map after predictive training.
The recovery score measures reconstruction of the surrogate’s full coordinates on held-out observations. Predictive training improves reconstruction of these reference coordinates relative to the same small encoder at initialization, with a session-weighted gain of .0239 ± .0118 in R 2 R^{2} R 2 Figure 7 and Table 21 ). The mean gain remains positive within each animal and each paired seed group, and after leaving out any animal. The protocol and cohort summaries appear in section J.
Table 21 : Local-recovery gains, mean ± \pm ±
Table 21.
Related Work
Prediction and state learning. Predictive state models describe dynamics through action-conditioned future observations (Littman et al., 2001). Auxiliary prediction and context learning link predictive objectives to downstream performance (Lee et al., 2021; Zhai et al., 2025). Zhang et al. (2025) recover generating latent variables up to coordinate negations and permutations as tasks increase, under a low-degree task distribution and preference for low-degree realizations. Our target is the complete query-response profile; the fixed-family bound quantifies the cost of one target set serving all encoders .
Identifiability. Identifiability depends on the learning setup (Locatello et al., 2019). Auxiliary variables [21 ] , contrastive objectives with generative assumptions (Zimmermann et al., 2021), and multiple modalities [8 ] can identify factors up to transformations. Our recovery target is specified by query responses before choosing latent coordinates.
Optimal recovery and sampling. Optimal recovery studies uncertainty under partial observations (Micchelli and Rivlin, 1977). Here, states sharing an encoder output form a set whose diameter limits recovery; prediction constrains that set through its targets. Structured model classes also yield recovery and sampling guarantees [3 , 23 ] .
Geometry and stable approximation. Embeddings (Sauer et al., 1991) and random manifold projections [2 ] relate geometry to measurement requirements. Stable manifold widths link approximation to metric entropy [6 ] . Here, low-dimensional covering controls the distinctions to resolve; observability supplies the queries.
Neural geometry and computation. Population geometry relates to computation [4 , 20 ] , cross-task generalization (Wakhloo et al., 2026), and cross-animal decoding (Safaie et al., 2023). Intrinsic-dimension estimation characterizes the degrees of freedom in neural activity [10 ] . These studies motivate our geometric assumptions, which connect covering complexity to query budgets.
Neural state learning. LFADS infers population dynamics (Pandarinath et al., 2018); the Neural Latents Benchmark evaluates held-out activity and behavioral readouts (Pei et al., 2021). Multi-context, multi-task, and foundation models learn across recordings [1 ] (Ye et al. 2023; Wang et al. 2025). CEBRA uses behavioral or temporal context (Schneider et al., 2023), DPAD prioritizes behaviorally relevant dynamics (Sani et al., 2024), and MARBLE represents local manifold dynamics (Gosztolai et al., 2025). These objectives reward different distinctions; we ask when preserving them suffices to recover the complete response profile.
Conclusion
We studied when learning to predict a system enables recovery of its ontic state. Defining that state as a complete response profile lets us relate prediction to the distinctions a learned representation preserves. Query ambiguity makes this connection quantitative, yielding recovery guarantees in general and low-dimensional settings. In the latter, we establish query-budget bounds whose order is sharp on a fixed state family when one target set serves every encoder. These results give a precise sense in which predicting observations can become a way of knowing the state that produces them. We hope that our work offers theoretical support for using deep learning models as computational surrogates for complex systems, including those studied in neuroscience.
State, Query, and Observation
We begin with observation ambiguity, then derive the prediction–recovery bounds and their sharpness. The training and noisy-observation results extend this chain to finite records and perturbed measurements. Experimental details follow the main-text order: synthetic recovery, joint-query calibration, and neural observation geometry and surrogate-state recovery.
Distance and Restriction
Distances contract when we restrict the query set. For query sets Q ⊆ Q ′ ⊆ U Q \subseteq Q' \subseteq\mathcal{U} Q ⊆ Q ′ ⊆ U f , g ∈ F f,g \in\mathcal{F} f , g ∈ F
d Q ( f ( Q ) , g ( Q ) ) ≤ d Q ′ ( f ( Q ′ ) , g ( Q ′ ) ) ≤ d U ( f , g ) . (33) d_{Q}(f(Q),g(Q)) \leq d_{Q'}(f(Q'),g(Q')) \leq d_{\mathcal{U}}(f,g). \tag*{(33)} d Q ( f ( Q ) , g ( Q )) ≤ d Q ′ ( f ( Q ′ ) , g ( Q ′ )) ≤ d U ( f , g ) . (33) Each inequality follows by taking the supremum over a subset of the same observation coordinates. In particular, every observation map f ↦ f ( Q ) f \mapsto f(Q) f ↦ f ( Q )
Lemma A.1 (Ontic Metric). The function d U : F × F → [ 0 , ∞ ) d_{\mathcal{U}}:\mathcal{F} \times\mathcal{F} \to[0,\infty) d U : F × F → [ 0 , ∞ ) F \mathcal{F} F ( F , d U ) (\mathcal{F},d_{\mathcal{U}}) ( F , d U )
Proof. Boundedness of f , g ∈ F f,g \in\mathcal{F} f , g ∈ F
d U ( f , g ) ≤ sup q ∈ U ∣ f ( q ) ∣ + sup q ∈ U ∣ g ( q ) ∣ < ∞ . (34) d_{\mathcal{U}}(f,g) \leq\sup_{q\in\mathcal{U}} |f(q)| + \sup_{q\in\mathcal{U}} |g(q)| < \infty. \tag*{(34)} d U ( f , g ) ≤ q ∈ U sup ∣ f ( q ) ∣ + q ∈ U sup ∣ g ( q ) ∣ < ∞. (34) Nonnegativity and symmetry follow from absolute value. Moreover, d U ( f , g ) = 0 d_{\mathcal{U}}(f,g)=0 d U ( f , g ) = 0 f ( q ) = g ( q ) f(q)=g(q) f ( q ) = g ( q ) f = g f=g f = g q ∈ U q \in\mathcal{U} q ∈ U
∣ f ( q ) − g ( q ) ∣ ≤ ∣ f ( q ) − h ( q ) ∣ + ∣ h ( q ) − g ( q ) ∣ ≤ d U ( f , h ) + d U ( h , g ) . (35) |f(q)-g(q)| \leq|f(q)-h(q)| + |h(q)-g(q)| \leq d_{\mathcal{U}}(f,h) + d_{\mathcal{U}}(h,g). \tag*{(35)} ∣ f ( q ) − g ( q ) ∣ ≤ ∣ f ( q ) − h ( q ) ∣ + ∣ h ( q ) − g ( q ) ∣ ≤ d U ( f , h ) + d U ( h , g ) . (35) Taking the supremum over q q q
A Common Readout
States sharing the same input must receive the same output. Their target spread limits the accuracy of a common readout.
Lemma A.2 (Fiberwise Error Bound). Let F \mathcal{F} F ( V , d ) (\mathcal{V},d) ( V , d ) T : F → Y T:\mathcal{F}\to\mathcal{Y} T : F → Y G : F → V G:\mathcal{F}\to\mathcal{V} G : F → V
Δ : = sup f , g ∈ F T ( f ) = T ( g ) d ( G ( f ) , G ( g ) ) . (36) \Delta:= \sup_{\substack{f,g\in\mathcal{F}\\T(f)=T(g)}} d(G(f),G(g)). \tag*{(36)} Δ := f , g ∈ F T ( f ) = T ( g ) sup d ( G ( f ) , G ( g )) . (36) Then
Δ 2 ≤ inf c : Y → V sup f ∈ F d ( c ( T ( f ) ) , G ( f ) ) ≤ Δ . (37) \frac{\Delta}{2} \leq\inf_{c:\mathcal{Y}\to\mathcal{V}}\sup_{f\in\mathcal{F}} d(c(T(f)),G(f)) \leq\Delta. \tag*{(37)} 2 Δ ≤ c : Y → V inf f ∈ F sup d ( c ( T ( f )) , G ( f )) ≤ Δ. (37) If ( V , d ) = ( R P , d P ) (\mathcal{V},d)=(\mathbb{R}^{P},d_{P}) ( V , d ) = ( R P , d P ) P P P G ( F ) G(\mathcal{F}) G ( F ) Δ / 2 \Delta/2 Δ/2
Proof. Choose a common output. For each attainable input y ∈ T ( F ) y \in T(\mathcal{F}) y ∈ T ( F ) S y : = { f ∈ F : T ( f ) = y } S_y := \{f \in\mathcal{F}:T(f)=y\} S y := { f ∈ F : T ( f ) = y } h y ∈ S y h_y \in S_y h y ∈ S y c ( y ) = G ( h y ) c(y)=G(h_y) c ( y ) = G ( h y ) V \mathcal{V} V f ∈ S y f \in S_y f ∈ S y
d ( c ( T ( f ) ) , G ( f ) ) = d ( G ( h y ) , G ( f ) ) ≤ Δ . (38) d(c(T(f)),G(f)) = d(G(h_y),G(f)) \leq\Delta. \tag*{(38)} d ( c ( T ( f )) , G ( f )) = d ( G ( h y ) , G ( f )) ≤ Δ. (38) Taking the supremum over f ∈ F f \in\mathcal{F} f ∈ F
The state-valued case G ( f ) = f G(f)=f G ( f ) = f
Figure 1 (PDF p. 17).
Compare any common output. If T ( f ) = T ( g ) T(f)=T(g) T ( f ) = T ( g ) c c c
d ( G ( f ) , G ( g ) ) ≤ d ( G ( f ) , c ( T ( f ) ) ) + d ( c ( T ( g ) ) , G ( g ) ) ≤ 2 sup h ∈ F d ( c ( T ( h ) ) , G ( h ) ) . (39) d(G(f),G(g)) \le d(G(f),c(T(f))) + d(c(T(g)),G(g)) \le2\sup_{h\in\mathcal{F}} d(c(T(h)),G(h)). \tag*{(39)} d ( G ( f ) , G ( g )) ≤ d ( G ( f ) , c ( T ( f ))) + d ( c ( T ( g )) , G ( g )) ≤ 2 h ∈ F sup d ( c ( T ( h )) , G ( h )) . (39) Taking the supremum over pairs and then the infimum over readouts gives the lower bound Δ / 2 \Delta/2 Δ/2
Use coordinate midpoints. When ( V , d ) = ( R P , d P ) (\mathcal{V},d)=(\mathbb{R}^{P},d_{P}) ( V , d ) = ( R P , d P ) G ( F ) G(\mathcal{F}) G ( F )
[ c ⋆ ( y ) ] ( p ) : = 1 2 ( inf g ∈ S y [ G ( g ) ] ( p ) + sup g ∈ S y [ G ( g ) ] ( p ) ) , p ∈ P . (40) [c^\star(y)](p) := \frac{1}{2}\left(\inf_{g\in S_y}[G(g)](p)+\sup_{g\in S_y}[G(g)](p)\right),\qquad p\in P. \tag*{(40)} [ c ⋆ ( y )] ( p ) := 2 1 ( g ∈ S y inf [ G ( g )] ( p ) + g ∈ S y sup [ G ( g )] ( p ) ) , p ∈ P . (40) The endpoints are finite by boundedness. Set c ⋆ ( y ) = 0 ∈ R P c^\star(y)=0\in\mathbb{R}^{P} c ⋆ ( y ) = 0 ∈ R P f ∈ S y f\in S_y f ∈ S y p ∈ P p\in P p ∈ P
∣ [ c ⋆ ( y ) ] ( p ) − [ G ( f ) ] ( p ) ∣ ≤ 1 2 sup g , h ∈ S y ∣ [ G ( g ) ] ( p ) − [ G ( h ) ] ( p ) ∣ ≤ Δ 2 . (41) \left|[c^\star(y)](p)-[G(f)](p)\right|\le\frac{1}{2}\sup_{g,h\in S_y}\left|[G(g)](p)-[G(h)](p)\right|\le\frac{\Delta}{2}. \tag*{(41)} ∣ [ c ⋆ ( y )] ( p ) − [ G ( f )] ( p ) ∣ ≤ 2 1 g , h ∈ S y sup ∣ [ G ( g )] ( p ) − [ G ( h )] ( p ) ∣ ≤ 2 Δ . (41) Taking the maximum over p ∈ P p\in P p ∈ P f ∈ F f\in\mathcal{F} f ∈ F c ⋆ c^\star c ⋆
Lemma A.3 (Exact Decoding). For nonempty F \mathcal{F} F R Q ⋆ ( e ) = 0 R^\star_Q(e)=0 R Q ⋆ ( e ) = 0 f ↦ e ( f ( Q ) ) f\mapsto e(f(Q)) f ↦ e ( f ( Q )) ( a ∘ e ) ( f ( Q ) ) = f (a\circ e)(f(Q))=f ( a ∘ e ) ( f ( Q )) = f f ∈ F f\in\mathcal{F} f ∈ F
Proof. If R Q ⋆ ( e ) = 0 R^\star_Q(e)=0 R Q ⋆ ( e ) = 0 d U ( f , g ) = 0 d_{\mathcal{U}}(f,g)=0 d U ( f , g ) = 0 e ( f ( Q ) ) = e ( g ( Q ) ) e(f(Q))=e(g(Q)) e ( f ( Q )) = e ( g ( Q )) F \mathcal{F} F decoder . Conversely, an exact decoder has zero error, so R Q ⋆ ( e ) = 0 R^\star_Q(e)=0 R Q ⋆ ( e ) = 0
Ambiguity and Recovery
The common-readout construction now applies to all states compatible with an observation vector. Compactness will ensure that their ambiguity vanishes as separating queries accumulate.
Assumption A.4 (Compactness). The ontic state space ( F , d U ) (\mathcal{F},d_{\mathcal{U}}) ( F , d U )
Lemma A.5 (Ambiguity and Recovery). Assume F \mathcal{F} F Q ⊆ U Q\subseteq\mathcal{U} Q ⊆ U
1 2 ω Q ( 0 ) ≤ E Q ⋆ ≤ ω Q ( 0 ) . (42) \frac{1}{2}\omega_Q(0)\le E^\star_Q\le\omega_Q(0). \tag*{(42)} 2 1 ω Q ( 0 ) ≤ E Q ⋆ ≤ ω Q ( 0 ) . (42) Proof. By lemma A.1, ( F , d U ) (\mathcal{F},d_{\mathcal{U}}) ( F , d U ) T ( f ) = f ( Q ) T(f)=f(Q) T ( f ) = f ( Q ) G ( f ) = f G(f)=f G ( f ) = f ( V , d ) = ( F , d U ) (\mathcal{V},d)=(\mathcal{F},d_{\mathcal{U}}) ( V , d ) = ( F , d U ) Δ = ω Q ( 0 ) \Delta=\omega_Q(0) Δ = ω Q ( 0 ) (37) gives both bounds.
Proof of lemma 2.9. Suppose the conclusion fails. There are ε > 0 \varepsilon>0 ε > 0 m k m_k m k f k , g k ∈ F f_k,g_k\in\mathcal{F} f k , g k ∈ F
d P m k ( f k ( P m k ) , g k ( P m k ) ) ≤ r m k , d U ( f k , g k ) ≥ ε . (43) d_{P_{m_k}}\left(f_k(P_{m_k}),g_k(P_{m_k})\right)\le r_{m_k},\qquad d_{\mathcal{U}}(f_k,g_k)\ge\varepsilon. \tag*{(43)} d P m k ( f k ( P m k ) , g k ( P m k ) ) ≤ r m k , d U ( f k , g k ) ≥ ε . (43) Compactness gives a common subsequence with f k → f f_k\to f f k → f g k → g g_k\to g g k → g d U ( f , g ) ≥ ε d_{\mathcal{U}}(f,g)\ge\varepsilon d U ( f , g ) ≥ ε
Fix any stage m m m m k ≥ m m_k\ge m m k ≥ m (33) give
d P m ( f k ( P m ) , g k ( P m ) ) ≤ r m k → 0. (44) d_{P_m}\left(f_k(P_m),g_k(P_m)\right)\le r_{m_k}\to0. \tag*{(44)} d P m ( f k ( P m ) , g k ( P m ) ) ≤ r m k → 0. (44) Taking the limit yields f ( P m ) = g ( P m ) f(P_m)=g(P_m) f ( P m ) = g ( P m ) f = g f=g f = g d U ( f , g ) ≥ ε d_{\mathcal{U}}(f,g)\ge\varepsilon d U ( f , g ) ≥ ε
Before introducing a learned representation, we characterize when the observations themselves suffice for recovery.
Theorem A.6 (Separation and Recovery). Assume ( F , d U ) (\mathcal{F},d_{\mathcal{U}}) ( F , d U ) ( Q p ) p ≥ 1 (Q_p)_{p\ge1} ( Q p ) p ≥ 1
State separation ⟺ E Q p ⋆ → p → ∞ 0. (45) \text{State separation}\quad\Longleftrightarrow\quad E^\star_{Q_p}\xrightarrow[p\to\infty]{}0. \tag*{(45)} State separation ⟺ E Q p ⋆ p → ∞ 0. (45) Proof. If the queries separate states, apply lemma 2.9 with zero tolerance and then lemma A.5:
0 ≤ E Q p ⋆ ≤ ω Q p ( 0 ) → 0. (46) 0 \leq\mathcal{E}^{\star}_{Q_p} \leq\omega_{Q_p}(0) \to0. \tag*{(46)} 0 ≤ E Q p ⋆ ≤ ω Q p ( 0 ) → 0. (46) Conversely, if two distinct states f , g f,g f , g Q p Q_p Q p
E Q p ⋆ ≥ 1 2 ω Q p ( 0 ) ≥ 1 2 d U ( f , g ) > 0 (47) \mathcal{E}^{\star}_{Q_p} \geq\frac{1}{2}\omega_{Q_p}(0) \geq\frac{1}{2}d_U(f,g) > 0 \tag*{(47)} E Q p ⋆ ≥ 2 1 ω Q p ( 0 ) ≥ 2 1 d U ( f , g ) > 0 (47) at every stage. Thus vanishing recovery error requires state separation. □
The diagram shows candidate sets under state separation.
Figure 8. Schematic candidate sets for a fixed f f f r m r_m r m r m → 0 r_m \to0 r m → 0
Prediction Setup
We now compare prediction and recovery from the learned state e ( f ( Q ) ) e(f(Q)) e ( f ( Q )) z i = e ( f i ( Q ) ) z_i=e(f_i(Q)) z i = e ( f i ( Q ))
Proof of corollary 2.11. Any state decoder a a a e e e a a a
E Q ⋆ ≤ E Q ( a ∘ e ) = R Q ( a ; e ) . (48) \mathcal{E}^{\star}_{Q} \leq\mathcal{E}_{Q}(a \circ e)=\mathcal{R}_{Q}(a;e). \tag*{(48)} E Q ⋆ ≤ E Q ( a ∘ e ) = R Q ( a ; e ) . (48) Taking the infimum over a a a E Q ⋆ ≤ R Q ⋆ ( e ) \mathcal{E}^{\star}_{Q} \leq\mathcal{R}^{\star}_{Q}(e) E Q ⋆ ≤ R Q ⋆ ( e ) E Q ⋆ ≥ ω Q ( 0 ) / 2 \mathcal{E}^{\star}_{Q} \geq\omega_{Q}(0)/2 E Q ⋆ ≥ ω Q ( 0 ) /2
The recovery criterion is R Q m ⋆ ( e m ) → 0 \mathcal{R}^{\star}_{Q_m}(e_m) \to0 R Q m ⋆ ( e m ) → 0 encoder at a sufficiently late stage admits a decoder achieving that accuracy uniformly over states and ontic queries. The decoder may depend on the stage.
Proof of lemma 3.1. Apply lemma A.2 with T ( f ) = e ( f ( Q ) ) T(f)=e(f(Q)) T ( f ) = e ( f ( Q )) G ( f ) = f ( P ) G(f)=f(P) G ( f ) = f ( P ) ( V , d ) = ( R P , d P ) (\mathcal{V},d)=(\mathbb{R}^{P},d_P) ( V , d ) = ( R P , d P ) F \mathcal{F} F G ( F ) G(\mathcal{F}) G ( F )
Prediction and Recovery
We prove theorems 3.2 and 3.3 first, followed by theorems 3.4 and 3.5. In each case, fixing the encoder gives an exact-recovery corollary.
General Case
The predictor assigns a common target vector to states merged by the encoder . Their target spread controls their full-state spread through query ambiguity.
Proof of theorem 3.2. We apply lemma A.2 to ontic states mapped to the same learned state, taking T ( f ) = e ( f ( Q ) ) T(f)=e(f(Q)) T ( f ) = e ( f ( Q )) G ( f ) = f G(f)=f G ( f ) = f ( V , d ) = ( F , d U ) (\mathcal{V},d)=(\mathcal{F},d_U) ( V , d ) = ( F , d U )
Lower bound. Restricting a decoded state to the target queries gives a predictor , as illustrated below.
Figure 9 (PDF p. 20).
Target observations differ by at most the full state distance. Combining this with lemma 3.1 and the lower bound in lemma A.2 gives
L Q , P ⋆ ( e ) = 1 2 sup f , g ∈ F e ( f ( Q ) ) = e ( g ( Q ) ) d P ( f ( P ) , g ( P ) ) ≤ 1 2 sup f , g ∈ F e ( f ( Q ) ) = e ( g ( Q ) ) d U ( f , g ) ≤ R Q ⋆ ( e ) . (49) \mathcal{L}^{\star}_{Q,P}(e)=\frac{1}{2}\sup_{\substack{f,g\in\mathcal{F}\\e(f(Q))=e(g(Q))}}d_{P}(f(P),g(P))\leq\frac{1}{2}\sup_{\substack{f,g\in\mathcal{F}\\e(f(Q))=e(g(Q))}}d_{U}(f,g)\leq\mathcal{R}^{\star}_{Q}(e). \tag*{(49)} L Q , P ⋆ ( e ) = 2 1 f , g ∈ F e ( f ( Q )) = e ( g ( Q )) sup d P ( f ( P ) , g ( P )) ≤ 2 1 f , g ∈ F e ( f ( Q )) = e ( g ( Q )) sup d U ( f , g ) ≤ R Q ⋆ ( e ) . (49) Upper bound. By lemma 3.1, any pair with e ( f ( Q ) ) = e ( g ( Q ) ) e(f(Q))=e(g(Q)) e ( f ( Q )) = e ( g ( Q )) 2 L Q , P ⋆ ( e ) 2\mathcal{L}^{\star}_{Q,P}(e) 2 L Q , P ⋆ ( e )
R Q ⋆ ( e ) ≤ sup f , g ∈ F e ( f ( Q ) ) = e ( g ( Q ) ) d U ( f , g ) ≤ ω P ( 2 L Q , P ⋆ ( e ) ) . (50) \mathcal{R}^{\star}_{Q}(e)\leq\sup_{\substack{f,g\in\mathcal{F}\\e(f(Q))=e(g(Q))}}d_{U}(f,g)\leq\omega_{P}\left(2\mathcal{L}^{\star}_{Q,P}(e)\right). \tag*{(50)} R Q ⋆ ( e ) ≤ f , g ∈ F e ( f ( Q )) = e ( g ( Q )) sup d U ( f , g ) ≤ ω P ( 2 L Q , P ⋆ ( e ) ) . (50) For a predictor b b b L Q , P ⋆ ( e ) ≤ L Q , P ( b ; e ) \mathcal{L}^{\star}_{Q,P}(e)\leq\mathcal{L}_{Q,P}(b;e) L Q , P ⋆ ( e ) ≤ L Q , P ( b ; e ) ω P \omega_{P} ω P
Proof of theorem 3.3. If L Q m , P m ⋆ ( e m ) → 0 \mathcal{L}^{\star}_{Q_m,P_m}(e_m)\to0 L Q m , P m ⋆ ( e m ) → 0 2 L Q m , P m ⋆ ( e m ) 2\mathcal{L}^{\star}_{Q_m,P_m}(e_m) 2 L Q m , P m ⋆ ( e m )
0 ≤ R Q m ⋆ ( e m ) ≤ ω P m ( 2 L Q m , P m ⋆ ( e m ) ) → 0. (51) 0\leq\mathcal{R}^{\star}_{Q_m}(e_m)\leq\omega_{P_m}\left(2\mathcal{L}^{\star}_{Q_m,P_m}(e_m)\right)\to0. \tag*{(51)} 0 ≤ R Q m ⋆ ( e m ) ≤ ω P m ( 2 L Q m , P m ⋆ ( e m ) ) → 0. (51) Conversely, 0 ≤ L Q m , P m ⋆ ( e m ) ≤ R Q m ⋆ ( e m ) 0\leq\mathcal{L}^{\star}_{Q_m,P_m}(e_m)\leq\mathcal{R}^{\star}_{Q_m}(e_m) 0 ≤ L Q m , P m ⋆ ( e m ) ≤ R Q m ⋆ ( e m )
For a fixed encoder , vanishing prediction error gives exact state recovery.
Corollary C.1 (General Exact Recovery). Under assumption A.4 , fix a query set Q Q Q e : R Q → Z e:\mathbb{R}^{Q}\to\mathcal{Z} e : R Q → Z ( P m ) m ≥ 1 (P_m)_{m\geq1} ( P m ) m ≥ 1 P m ⊆ U ∖ Q P_m\subseteq U\setminus Q P m ⊆ U ∖ Q
L Q , P m ⋆ ( e ) → m → ∞ 0 , (52) \mathcal{L}^{\star}_{Q,P_m}(e)\xrightarrow[m\to\infty]{}0, \tag*{(52)} L Q , P m ⋆ ( e ) m → ∞ 0 , (52) then there exists a state decoder a ∈ A a\in\mathcal{A} a ∈ A
( a ∘ e ) ( f ( Q ) ) = f , ∀ f ∈ F . (53) (a\circ e)(f(Q))=f,\qquad\forall f\in\mathcal{F}. \tag*{(53)} ( a ∘ e ) ( f ( Q )) = f , ∀ f ∈ F . (53) Proof. By theorem 3.3, the recovery error tends to zero. Since Q Q Q e e e R Q ⋆ ( e ) \mathcal{R}^{\star}_{Q}(e) R Q ⋆ ( e ) decoder .
Embedding theory relates state distinguishability to the geometry preserved by observations (Sauer et al., 1991). Here, exact state recovery has a topological counterpart when the state space is compact and the map to learned states is continuous.
Corollary C.2 (Topology Preservation). Under assumption A.4 , fix a query set Q Q Q e : R Q → Z = R ℓ e:\mathbb{R}^{Q}\to\mathcal{Z}=\mathbb{R}^{\ell} e : R Q → Z = R ℓ f ↦ e ( f ( Q ) ) f\mapsto e(f(Q)) f ↦ e ( f ( Q )) a ∈ A a\in\mathcal{A} a ∈ A
( a ∘ e ) ( f ( Q ) ) = f , ∀ f ∈ F , (54) (a\circ e)(f(Q))=f,\qquad\forall f\in\mathcal{F}, \tag*{(54)} ( a ∘ e ) ( f ( Q )) = f , ∀ f ∈ F , (54) then this map is a homeomorphism onto its image. In particular, if F \mathcal{F} F
Proof. Injectivity. By lemma A.3, the map f ↦ e ( f ( Q ) ) f\mapsto e(f(Q)) f ↦ e ( f ( Q )) Continuity of the inverse. Every closed subset of the compact space F \mathcal{F} F R L \mathbb{R}^{L} R L
Manifold structure. Composing each local chart of F \mathcal{F} F
Figure 10 (PDF p. 21).
Low-Dimensional Case
The general case controls recovery through target ambiguity. In the low-dimensional case, covering and observability let us bound that ambiguity using finitely many queries, yielding the recovery and convergence results in theorems 3.4 and 3.5.
Lemma C.3 (Covering Queries). Let F \mathcal{F} F ( F , Q ) (\mathcal{F},Q) ( F , Q ) C C C d d d α \alpha α 0 < ε ≤ 1 0<\varepsilon\le1 0 < ε ≤ 1 τ > 0 \tau>0 τ > 0
There is a finite representative set F ε ⊆ F \mathcal{F}_{\varepsilon}\subseteq\mathcal{F} F ε ⊆ F N = ∣ F ε ∣ ≤ C ε − d N=|\mathcal{F}_{\varepsilon}|\le C\varepsilon^{-d} N = ∣ F ε ∣ ≤ C ε − d F \mathcal{F} F d U d_{\mathcal{U}} d U ε \varepsilon ε
There also exists a target set P ⊆ U ∖ Q P\subseteq\mathcal{U}\setminus Q P ⊆ U ∖ Q 1 ≤ ∣ P ∣ ≤ max { 1 , N ( N − 1 ) / 2 } 1\le|P|\le\max\{1,N(N-1)/2\} 1 ≤ ∣ P ∣ ≤ max { 1 , N ( N − 1 ) /2 } f , g ∈ F ε f,g\in\mathcal{F}_{\varepsilon} f , g ∈ F ε
d P ( f ( P ) , g ( P ) ) ≥ α d U ( f , g ) − τ . (55) d_P(f(P),g(P))\ge\alpha d_{\mathcal{U}}(f,g)-\tau. \tag*{(55)} d P ( f ( P ) , g ( P )) ≥ α d U ( f , g ) − τ . (55) Equivalently,
d U ( f , g ) ≤ 1 α d P ( f ( P ) , g ( P ) ) + τ α . (56) d_{\mathcal{U}}(f,g)\le\frac{1}{\alpha}d_P(f(P),g(P))+\frac{\tau}{\alpha}. \tag*{(56)} d U ( f , g ) ≤ α 1 d P ( f ( P ) , g ( P )) + α τ . (56) The representatives and targets are selected independently of the encoder.
Proof. Choose the representatives. By equation (13) , take an ε \varepsilon ε F ε ⊆ F \mathcal{F}_{\varepsilon}\subseteq\mathcal{F} F ε ⊆ F 1 ≤ N = ∣ F ε ∣ ≤ C ε − d 1\le N=|\mathcal{F}_{\varepsilon}|\le C\varepsilon^{-d} 1 ≤ N = ∣ F ε ∣ ≤ C ε − d
Distinguish each pair. For each unordered pair { f , g } ⊆ F ε \{f,g\}\subseteq\mathcal{F}_{\varepsilon} { f , g } ⊆ F ε (14) and τ > 0 \tau>0 τ > 0 p { f , g } ∈ U ∖ Q p_{\{f,g\}}\in\mathcal{U}\setminus Q p { f , g } ∈ U ∖ Q
∣ f ( p { f , g } ) − g ( p { f , g } ) ∣ ≥ α d U ( f , g ) − τ . (57) \left|f(p_{\{f,g\}})-g(p_{\{f,g\}})\right|\ge\alpha d_{\mathcal{U}}(f,g)-\tau. \tag*{(57)} f ( p { f , g } ) − g ( p { f , g } ) ≥ α d U ( f , g ) − τ . (57) The positive allowance permits this choice even if no query attains the supremum.
Collect the queries. If N ≥ 2 N\ge2 N ≥ 2 P = { p { f , g } : f , g ∈ F ε , f ≠ g } P=\{p_{\{f,g\}}:f,g\in\mathcal{F}_{\varepsilon},\ f\ne g\} P = { p { f , g } : f , g ∈ F ε , f = g }
1 ≤ ∣ P ∣ ≤ N ( N − 1 ) 2 . (58) 1\le|P|\le\frac{N(N-1)}{2}. \tag*{(58)} 1 ≤ ∣ P ∣ ≤ 2 N ( N − 1 ) . (58) For distinct f , g ∈ F ε f,g\in\mathcal{F}_{\varepsilon} f , g ∈ F ε P P P
d P ( f ( P ) , g ( P ) ) ≥ ∣ f ( p { f , g } ) − g ( p { f , g } ) ∣ ≥ α d U ( f , g ) − τ . (59) d_P(f(P),g(P))\ge\left|f(p_{\{f,g\}})-g(p_{\{f,g\}})\right|\ge\alpha d_{\mathcal{U}}(f,g)-\tau. \tag*{(59)} d P ( f ( P ) , g ( P )) ≥ f ( p { f , g } ) − g ( p { f , g } ) ≥ α d U ( f , g ) − τ . (59) For f = g f=g f = g (55) ; rearranging gives equation (56) .
If N = 1 N=1 N = 1 P ⊆ U ∖ Q P\subseteq\mathcal{U}\setminus Q P ⊆ U ∖ Q
We next extend the representative bound to arbitrary states.
Lemma C.4 (Decoding a Cover). Let F ε ⊆ F \mathcal{F}_{\varepsilon} \subseteq\mathcal{F} F ε ⊆ F ε \varepsilon ε P P P (56) with constants α > 0 \alpha> 0 α > 0 τ ≥ 0 \tau\ge0 τ ≥ 0 y ∈ R P y \in\mathbb{R}^{P} y ∈ R P g ∈ arg min h ∈ F ε d P ( y , h ( P ) ) g \in\arg\min_{h \in\mathcal{F}_{\varepsilon}} d_{P}(y,h(P)) g ∈ arg min h ∈ F ε d P ( y , h ( P )) f ∈ F f \in\mathcal{F} f ∈ F
d U ( f , g ) ≤ 2 α d P ( y , f ( P ) ) + ( 2 + α ) ε + τ α . (60) d_{U}(f,g) \leq\frac{2}{\alpha}d_{P}(y,f(P)) + \frac{(2+\alpha)\varepsilon+\tau}{\alpha}. \tag*{(60)} d U ( f , g ) ≤ α 2 d P ( y , f ( P )) + α ( 2 + α ) ε + τ . (60) Proof. Take f ε ∈ F ε f_{\varepsilon} \in\mathcal{F}_{\varepsilon} f ε ∈ F ε d U ( f , f ε ) ≤ ε d_{U}(f,f_{\varepsilon}) \leq\varepsilon d U ( f , f ε ) ≤ ε (56) , the minimizing property of g g g
d U ( f , g ) ≤ d U ( f , f ε ) + d U ( f ε , g ) ≤ ε + 1 α d P ( f ε ( P ) , g ( P ) ) + τ α ≤ ε + 2 α d P ( y , f ε ( P ) ) + τ α ≤ 2 α d P ( y , f ( P ) ) + ( 2 + α ) ε + τ α . (61) \begin{aligned}
d_{U}(f,g) &\leq d_{U}(f,f_{\varepsilon})+d_{U}(f_{\varepsilon},g) \leq\varepsilon+\frac{1}{\alpha}d_{P}(f_{\varepsilon}(P),g(P))+\frac{\tau}{\alpha}\\
&\leq\varepsilon+\frac{2}{\alpha}d_{P}(y,f_{\varepsilon}(P))+\frac{\tau}{\alpha}\leq\frac{2}{\alpha}d_{P}(y,f(P))+\frac{(2+\alpha)\varepsilon+\tau}{\alpha}. \tag*{(61)}
\end{aligned} d U ( f , g ) ≤ d U ( f , f ε ) + d U ( f ε , g ) ≤ ε + α 1 d P ( f ε ( P ) , g ( P )) + α τ ≤ ε + α 2 d P ( y , f ε ( P )) + α τ ≤ α 2 d P ( y , f ( P )) + α ( 2 + α ) ε + τ . (61) To prove theorem 3.4, we fit this construction to the prescribed query budget and decode to a representative whose target observations are closest to the prediction.
Proof of theorem 3.4. Decode to a representative. For radius ε \varepsilon ε τ \tau τ F ε \mathcal{F}_{\varepsilon} F ε P P P e e e b b b z z z b ( z ) b(z) b ( z ) g ( P ) g(P) g ( P )
a ( z ) ∈ arg min g ∈ F ε d P ( b ( z ) , g ( P ) ) , (62) a(z) \in\arg\min_{g \in\mathcal{F}_{\varepsilon}} d_{P}(b(z),g(P)), \tag*{(62)} a ( z ) ∈ arg g ∈ F ε min d P ( b ( z ) , g ( P )) , (62) resolving ties by a fixed ordering.
The diagram shows the returned representative g = a ( z ) g=a(z) g = a ( z ) b ( z ) b(z) b ( z )
Figure 11 (PDF p. 22).
Bound the recovery error. Apply lemma C.4 with y = ( b ∘ e ) ( f ( Q ) ) y=(b\circ e)(f(Q)) y = ( b ∘ e ) ( f ( Q )) g = ( a ∘ e ) ( f ( Q ) ) g=(a\circ e)(f(Q)) g = ( a ∘ e ) ( f ( Q ))
d U ( f , ( a ∘ e ) ( f ( Q ) ) ) ≤ 2 α d P ( ( b ∘ e ) ( f ( Q ) ) , f ( P ) ) + ( 2 + α ) ε + τ α . (63) d_{U}(f,(a\circ e)(f(Q))) \leq\frac{2}{\alpha}d_{P}((b\circ e)(f(Q)),f(P))+\frac{(2+\alpha)\varepsilon+\tau}{\alpha}. \tag*{(63)} d U ( f , ( a ∘ e ) ( f ( Q ))) ≤ α 2 d P (( b ∘ e ) ( f ( Q )) , f ( P )) + α ( 2 + α ) ε + τ . (63) Set the query budget. The radius controls the number of representatives; the allowance absorbs the approximation of separating queries. Let δ m = ( C 2 / ( 2 m ) ) 1 / ( 2 d ) ≤ 1 \delta_{m}=(C^{2}/(2m))^{1/(2d)}\leq1 δ m = ( C 2 / ( 2 m ) ) 1/ ( 2 d ) ≤ 1 0 < ε < δ m 0<\varepsilon<\delta_{m} 0 < ε < δ m δ m \delta_{m} δ m
N ≤ C ε − d < ⌊ C δ m − d ⌋ + 1 = ⌊ 2 m ⌋ + 1 , (64) N \leq C\varepsilon^{-d}<\lfloor C\delta_{m}^{-d}\rfloor+1=\lfloor\sqrt{2m}\rfloor+1, \tag*{(64)} N ≤ C ε − d < ⌊ C δ m − d ⌋ + 1 = ⌊ 2 m ⌋ + 1 , (64) hence N ≤ ⌊ 2 m ⌋ N\leq\lfloor\sqrt{2m}\rfloor N ≤ ⌊ 2 m ⌋ τ = ( 2 + α ) ( δ m − ε ) > 0 \tau=(2+\alpha)(\delta_{m}-\varepsilon)>0 τ = ( 2 + α ) ( δ m − ε ) > 0
∣ P ∣ ≤ max { 1 , N ( N − 1 ) 2 } ≤ max { 1 , N 2 2 } ≤ m . (65) |P|\leq\max\left\{1,\frac{N(N-1)}{2}\right\}\leq\max\left\{1,\frac{N^{2}}{2}\right\}\leq m. \tag*{(65)} ∣ P ∣ ≤ max { 1 , 2 N ( N − 1 ) } ≤ max { 1 , 2 N 2 } ≤ m . (65) These choices are independent of e e e
R Q ⋆ ( e ) ≤ R Q ( a ; e ) ≤ 2 α L Q , P ( b ; e ) + ( 2 + α ) ε + τ α = 2 α L Q , P ( b ; e ) + ( 2 + α ) ε + ( 2 + α ) ( δ m − ε ) α = 2 α L Q , P ( b ; e ) + 2 + α α δ m . (66) \begin{aligned}
\mathcal{R}^{\star}_{Q}(e) &\leq\mathcal{R}_{Q}(a;e)\leq\frac{2}{\alpha}\mathcal{L}_{Q,P}(b;e)+\frac{(2+\alpha)\varepsilon+\tau}{\alpha}\\
&=\frac{2}{\alpha}\mathcal{L}_{Q,P}(b;e)+\frac{(2+\alpha)\varepsilon+(2+\alpha)(\delta_{m}-\varepsilon)}{\alpha}
=\frac{2}{\alpha}\mathcal{L}_{Q,P}(b;e)+\frac{2+\alpha}{\alpha}\delta_{m}. \tag*{(66)}
\end{aligned} R Q ⋆ ( e ) ≤ R Q ( a ; e ) ≤ α 2 L Q , P ( b ; e ) + α ( 2 + α ) ε + τ = α 2 L Q , P ( b ; e ) + α ( 2 + α ) ε + ( 2 + α ) ( δ m − ε ) = α 2 L Q , P ( b ; e ) + α 2 + α δ m . (66) Taking the infimum over b b b F \mathcal{F} F
The bound in theorem 3.4 now gives the convergence and rate equivalences. These target sets need not be nested: each budget has its own uniform recovery bound.
Proof of theorem 3.5. By theorem 3.4, for every admissible budget,
0 ≤ L Q m , P m ⋆ ( e m ) ≤ R Q m ⋆ ( e m ) ≤ 2 α L Q m , P m ⋆ ( e m ) + 2 + α α ( C 2 2 m ) 1 / ( 2 d ) . (67) 0 \le L^\star_{Q_m,P_m}(e_m) \le R^\star_{Q_m}(e_m) \le\frac{2}{\alpha}L^\star_{Q_m,P_m}(e_m) + \frac{2+\alpha}{\alpha}\left(\frac{C^2}{2m}\right)^{1/(2d)}. \tag*{(67)} 0 ≤ L Q m , P m ⋆ ( e m ) ≤ R Q m ⋆ ( e m ) ≤ α 2 L Q m , P m ⋆ ( e m ) + α 2 + α ( 2 m C 2 ) 1/ ( 2 d ) . (67) Convergence.
Left-to-right. If L Q m , P m ⋆ ( e m ) → 0 L^\star_{Q_m,P_m}(e_m) \to0 L Q m , P m ⋆ ( e m ) → 0 R Q m ⋆ ( e m ) → 0 R^\star_{Q_m}(e_m) \to0 R Q m ⋆ ( e m ) → 0
Right-to-left. If R Q m ⋆ ( e m ) → 0 R^\star_{Q_m}(e_m) \to0 R Q m ⋆ ( e m ) → 0 L Q m , P m ⋆ ( e m ) → 0 L^\star_{Q_m,P_m}(e_m) \to0 L Q m , P m ⋆ ( e m ) → 0
Rate.
Left-to-right. Assume L Q m , P m ⋆ ( e m ) = O ( m − 1 / ( 2 d ) ) L^\star_{Q_m,P_m}(e_m) = O(m^{-1/(2d)}) L Q m , P m ⋆ ( e m ) = O ( m − 1/ ( 2 d ) ) c ≥ 0 c \ge0 c ≥ 0 m m m L Q m , P m ⋆ ( e m ) ≤ c m − 1 / ( 2 d ) L^\star_{Q_m,P_m}(e_m) \le c m^{-1/(2d)} L Q m , P m ⋆ ( e m ) ≤ c m − 1/ ( 2 d ) m m m (67) gives
R Q m ⋆ ( e m ) ≤ 2 α L Q m , P m ⋆ ( e m ) + 2 + α α ( C 2 2 m ) 1 / ( 2 d ) ≤ 2 c α m − 1 / ( 2 d ) + 2 + α α ( C 2 2 ) 1 / ( 2 d ) m − 1 / ( 2 d ) = [ 2 c α + 2 + α α ( C 2 2 ) 1 / ( 2 d ) ] ⋅ m − 1 / ( 2 d ) . (68) \begin{aligned}
R^\star_{Q_m}(e_m) &\le\frac{2}{\alpha}L^\star_{Q_m,P_m}(e_m) + \frac{2+\alpha}{\alpha}\left(\frac{C^2}{2m}\right)^{1/(2d)} \\
&\le\frac{2c}{\alpha}m^{-1/(2d)} + \frac{2+\alpha}{\alpha}\left(\frac{C^2}{2}\right)^{1/(2d)}m^{-1/(2d)} \\
&= \left[\frac{2c}{\alpha}+\frac{2+\alpha}{\alpha}\left(\frac{C^2}{2}\right)^{1/(2d)}\right]\cdot m^{-1/(2d)}. \tag*{(68)}
\end{aligned} R Q m ⋆ ( e m ) ≤ α 2 L Q m , P m ⋆ ( e m ) + α 2 + α ( 2 m C 2 ) 1/ ( 2 d ) ≤ α 2 c m − 1/ ( 2 d ) + α 2 + α ( 2 C 2 ) 1/ ( 2 d ) m − 1/ ( 2 d ) = [ α 2 c + α 2 + α ( 2 C 2 ) 1/ ( 2 d ) ] ⋅ m − 1/ ( 2 d ) . (68) The bracket is independent of m m m R Q m ⋆ ( e m ) = O ( m − 1 / ( 2 d ) ) R^\star_{Q_m}(e_m) = O(m^{-1/(2d)}) R Q m ⋆ ( e m ) = O ( m − 1/ ( 2 d ) )
Right-to-left. Assume R Q m ⋆ ( e m ) = O ( m − 1 / ( 2 d ) ) R^\star_{Q_m}(e_m) = O(m^{-1/(2d)}) R Q m ⋆ ( e m ) = O ( m − 1/ ( 2 d ) ) c ≥ 0 c \ge0 c ≥ 0 m m m m m m
0 ≤ L Q m , P m ⋆ ( e m ) ≤ R Q m ⋆ ( e m ) ≤ c m − 1 / ( 2 d ) . (69) 0 \le L^\star_{Q_m,P_m}(e_m) \le R^\star_{Q_m}(e_m) \le c m^{-1/(2d)}. \tag*{(69)} 0 ≤ L Q m , P m ⋆ ( e m ) ≤ R Q m ⋆ ( e m ) ≤ c m − 1/ ( 2 d ) . (69) Thus L Q m , P m ⋆ ( e m ) = O ( m − 1 / ( 2 d ) ) L^\star_{Q_m,P_m}(e_m) = O(m^{-1/(2d)}) L Q m , P m ⋆ ( e m ) = O ( m − 1/ ( 2 d ) ) (20) .
For a fixed encoder , the selected targets give the same recovery conclusion.
Corollary C.5 (Low-Dimensional Exact Recovery). Under the assumptions of theorem 3.4, fix an encoder e : R Q → Z e : \mathbb{R}^{Q} \to\mathcal{Z} e : R Q → Z m ≥ C 2 / 2 m \ge C^2/2 m ≥ C 2 /2 P m P_m P m Q Q Q
L Q , P m ⋆ ( e ) → m → ∞ 0 , (70) L^\star_{Q,P_m}(e) \xrightarrow{m\to\infty} 0, \tag*{(70)} L Q , P m ⋆ ( e ) m → ∞ 0 , (70) then there exists a state decoder a ∈ A a \in\mathcal{A} a ∈ A
( a ∘ e ) ( f ( Q ) ) = f , ∀ f ∈ F . (71) (a \circ e)(f(Q)) = f,\qquad\forall f \in\mathcal{F}. \tag*{(71)} ( a ∘ e ) ( f ( Q )) = f , ∀ f ∈ F . (71) Proof. By theorem 3.5, the recovery error tends to zero. Since Q Q Q e e e R Q ⋆ ( e ) R^\star_Q(e) R Q ⋆ ( e ) decoder .
Budget Sharpness
We first prove the fixed-family result supporting theorem 3.7, then give a finite-query construction for comparison.
A fixed family at every budget. A multiscale arc places the obstruction inside a fixed family: finer blocks contain more distinctions, while their response amplitudes decrease. The encoder merges a pair whose distinguishing query was omitted, and the radius of that pair gives the recovery cost. We construct the fixed family in lemma C.6, find a pair missed by the target set in lemma C.7, and merge that pair in lemma C.8. Combining their bounds proves lemma C.9.
Lemma C.6 (Multiscale Arc). For every d > 1 d>1 d > 1 there exist a countable query universe U U U a singleton input set Q Q Q and a compact state family F \mathcal{F} F homeomorphic to [ 0 , 1 ] [0,1] [ 0 , 1 ] of diameter 1 1 1 The input observation map is injective, and ( F , Q ) (\mathcal{F},Q) ( F , Q ) satisfies assumption 2.12 2.12 2.12 with C = C d C=C_d C = C d and α = 1 \alpha=1 α = 1 The covering constant is
C d : = 1 + 12 1 − 2 1 / d − 1 . (72) C_d := 1+\frac{12}{1-2^{1/d-1}}. \tag*{(72)} C d := 1 + 1 − 2 1/ d − 1 12 . (72) Proof. Construct the family. Fix d > 1 d>1 d > 1 j ≥ 1 j\geq1 j ≥ 1
N j = 8 ⋅ 2 j − 1 , s j = 1 2 2 − ( j − 1 ) / d , (73) N_j=8\cdot2^{j-1},\qquad s_j=\frac{1}{2}2^{-(j-1)/d}, \tag*{(73)} N j = 8 ⋅ 2 j − 1 , s j = 2 1 2 − ( j − 1 ) / d , (73) ℓ j = 2 − j − 1 , T j = 1 − 2 1 − j . (74) \ell_j=2^{-j-1},\qquad T_j=1-2^{1-j}. \tag*{(74)} ℓ j = 2 − j − 1 , T j = 1 − 2 1 − j . (74) Let v 1 , … , v N j v_1,\ldots,v_{N_j} v 1 , … , v N j R N j \mathbb{R}^{N_j} R N j [ T j , T j + ℓ j ] [T_j,T_j+\ell_j] [ T j , T j + ℓ j ] λ j ( t ) \lambda^j(t) λ j ( t ) v 1 → ⋯ → v N j v_1\to\cdots\to v_{N_j} v 1 → ⋯ → v N j [ T j + ℓ j , T j + 1 ] [T_j+\ell_j,T_{j+1}] [ T j + ℓ j , T j + 1 ] λ j ( t ) = w v N j \lambda^j(t)=wv_{N_j} λ j ( t ) = w v N j λ j + 1 ( t ) = ( 1 − w ) v 1 \lambda^{j+1}(t)=(1-w)v_1 λ j + 1 ( t ) = ( 1 − w ) v 1 w w w 1 1 1 0 0 0 t = 1 t=1 t = 1 [ 0 , 1 ) [0,1) [ 0 , 1 ) T j + 1 − T j = 2 ℓ j T_{j+1}-T_j=2\ell_j T j + 1 − T j = 2 ℓ j
Take Q = { q 0 } Q=\{q_0\} Q = { q 0 } p 0 p_0 p 0 p h j p_h^j p h j p a b j p_{ab}^j p ab j 1 ≤ a < b ≤ N j 1\leq a<b\leq N_j 1 ≤ a < b ≤ N j f t f_t f t 0 ≤ t ≤ 1 0\leq t\leq1 0 ≤ t ≤ 1
f t ( q 0 ) = f t ( p 0 ) = t / 2 , (75) f_t(q_0)=f_t(p_0)=t/2, \tag*{(75)} f t ( q 0 ) = f t ( p 0 ) = t /2 , (75) f t ( p a b j ) = s j ( λ a j ( t ) − λ b j ( t ) ) , (76) f_t(p_{ab}^j)=s_j(\lambda_a^j(t)-\lambda_b^j(t)), \tag*{(76)} f t ( p ab j ) = s j ( λ a j ( t ) − λ b j ( t )) , (76) f t ( p h j ) = 2 s j ( 1 − ∥ λ j ( t ) ∥ 1 ) . (77) f_t(p_h^j)=2s_j(1-\|\lambda^j(t)\|_1). \tag*{(77)} f t ( p h j ) = 2 s j ( 1 − ∥ λ j ( t ) ∥ 1 ) . (77) Let F = { f t : 0 ≤ t ≤ 1 } \mathcal{F}=\{f_t:0\leq t\leq1\} F = { f t : 0 ≤ t ≤ 1 }
Geometry and observability. All responses are bounded. The parameter query makes t ↦ f t t\mapsto f_t t ↦ f t α = 1 \alpha=1 α = 1 j j j f 1 f_1 f 1 2 s j 2s_j 2 s j s j s_j s j 2 s j 2s_j 2 s j 2 − j ≤ s j 2^{-j}\leq s_j 2 − j ≤ s j 1 1 1 F \mathcal{F} F 1 1 1 2 s 1 = 1 2s_1=1 2 s 1 = 1
The shrinking blocks accumulate at the endpoint.
Figure 1 (PDF p. 24).
Covering bound. Each edge within block j j j 2 s j 2s_j 2 s j p h j p_h^j p h j N j − 1 N_j-1 N j − 1 N j N_j N j 0 < ε < 1 0<\varepsilon<1 0 < ε < 1 J J J 2 s J ≥ ε 2s_J\geq\varepsilon 2 s J ≥ ε J + 1 J+1 J + 1 ε \varepsilon ε f 1 f_1 f 1 2 s j 2s_j 2 s j ⌈ s j / ε ⌉ \lceil s_j/\varepsilon\rceil ⌈ s j / ε ⌉
N F ( ε ) ≤ 1 + ∑ j ≤ J N j ( s j ε + 1 ) ≤ 1 + 3 ε ∑ j ≤ J N j s j , (78) \mathcal{N}_{\mathcal{F}}(\varepsilon)\leq1+\sum_{j\leq J}N_j\left(\frac{s_j}{\varepsilon}+1\right)\leq1+\frac{3}{\varepsilon}\sum_{j\leq J}N_js_j, \tag*{(78)} N F ( ε ) ≤ 1 + j ≤ J ∑ N j ( ε s j + 1 ) ≤ 1 + ε 3 j ≤ J ∑ N j s j , (78) ∑ j ≤ J N j s j ≤ N J s J 1 − 2 1 / d − 1 ≤ 4 ε 1 − d 1 − 2 1 / d − 1 . \sum_{j\leq J}N_js_j\leq\frac{N_Js_J}{1-2^{1/d-1}}\leq\frac{4\varepsilon^{1-d}}{1-2^{1/d-1}}. j ≤ J ∑ N j s j ≤ 1 − 2 1/ d − 1 N J s J ≤ 1 − 2 1/ d − 1 4 ε 1 − d . This gives the stated C d ε − d C_d\varepsilon^{-d} C d ε − d ε = 1 \varepsilon=1 ε = 1 N j N_j N j j j j 2 s j 2s_j 2 s j s j / 2 s_j/2 s j /2 N j = 8 ( 2 s j ) − d N_j=8(2s_j)^{-d} N j = 8 ( 2 s j ) − d d d d
The next two lemmas use the family and block notation constructed in the proof of lemma C.6.
Lemma C.7 (Omitted Pair). For every integer m ≥ 1 m \ge1 m ≥ 1 and nonempty P ⊆ U ∖ Q P \subseteq U \setminus Q P ⊆ U ∖ Q with ∣ P ∣ ≤ m \lvert P\rvert\le m ∣ P ∣ ≤ m some block j j j contains two nonadjacent vertices with parameters t i , t k t_i,t_k t i , t k such that
d P ( f t i ( P ) , f t k ( P ) ) ≤ s j , s j ≥ 1 2 m − 1 / ( 2 d ) . (79) d_P(f_{t_i}(P),f_{t_k}(P)) \le s_j,\qquad s_j \ge\frac{1}{2}m^{-1/(2d)}. \tag*{(79)} d P ( f t i ( P ) , f t k ( P )) ≤ s j , s j ≥ 2 1 m − 1/ ( 2 d ) . (79) Proof. Fix m m m P P P j j j ( N j − 1 ) ( N j − 2 ) / 2 > m (N_j-1)(N_j-2)/2 > m ( N j − 1 ) ( N j − 2 ) /2 > m j j j i < k i < k i < k k − i ≥ 2 k-i \ge2 k − i ≥ 2 p i k j ∉ P p_{ik}^{j} \notin P p ik j ∈ / P t i , t k t_i,t_k t i , t k p i k j p_{ik}^{j} p ik j 2 s j 2s_j 2 s j s j s_j s j ℓ j / 2 ≤ s j / 4 \ell_j/2 \le s_j/4 ℓ j /2 ≤ s j /4
d P ( f t i ( P ) , f t k ( P ) ) ≤ s j . (80) d_P(f_{t_i}(P),f_{t_k}(P)) \le s_j. \tag*{(80)} d P ( f t i ( P ) , f t k ( P )) ≤ s j . (80) When j = 1 j=1 j = 1 N j = 8 ≤ 8 m N_j=8 \le8\sqrt{m} N j = 8 ≤ 8 m j j j ( N j − 1 − 1 ) ( N j − 1 − 2 ) / 2 ≤ m (N_{j-1}-1)(N_{j-1}-2)/2 \le m ( N j − 1 − 1 ) ( N j − 1 − 2 ) /2 ≤ m N j − 1 ≤ 2 m + 2 N_{j-1} \le\sqrt{2m}+2 N j − 1 ≤ 2 m + 2 N j = 2 N j − 1 ≤ 8 m N_j=2N_{j-1} \le8\sqrt{m} N j = 2 N j − 1 ≤ 8 m
s j 3 = 1 6 ( N j / 8 ) − 1 / d ≥ 1 6 m − 1 / ( 2 d ) . (81) \frac{s_j}{3}=\frac{1}{6}(N_j/8)^{-1/d}\ge\frac{1}{6}m^{-1/(2d)}. \tag*{(81)} 3 s j = 6 1 ( N j /8 ) − 1/ d ≥ 6 1 m − 1/ ( 2 d ) . (81) :::
Lemma C.8 (Two-State Fiber). Fix two nonadjacent vertices f t i , f t k f_{t_i},f_{t_k} f t i , f t k with i < k i<k i < k in block j j j and a nonempty finite target set P ⊆ U ∖ Q P \subseteq U \setminus Q P ⊆ U ∖ Q There is a globally Lipschitz encoder e : R Q → R 2 e:\mathbb{R}^{Q}\to\mathbb{R}^{2} e : R Q → R 2 whose only non-singleton state fiber is this pair, with
L Q , P ⋆ ( e ) = 1 2 d P ( f t i ( P ) , f t k ( P ) ) , (82) \mathcal{L}^{\star}_{Q,P}(e)=\frac{1}{2}d_P(f_{t_i}(P),f_{t_k}(P)), \tag*{(82)} L Q , P ⋆ ( e ) = 2 1 d P ( f t i ( P ) , f t k ( P )) , (82) R Q ⋆ ( e ) ≥ 4 3 s j . (83) \mathcal{R}^{\star}_{Q}(e)\ge\frac{4}{3}s_j. \tag*{(83)} R Q ⋆ ( e ) ≥ 3 4 s j . (83) Proof. A two-state fiber. Put u i = t i / 2 u_i=t_i/2 u i = t i /2 u k = t k / 2 u_k=t_k/2 u k = t k /2
e ( u ) = { ( u − u i , 0 ) , u ≤ u i , ( sin 2 π ( u − u i ) u k − u i , 1 − cos 2 π ( u − u i ) u k − u i ) , u i ≤ u ≤ u k , ( u − u k , 0 ) , u ≥ u k . (84) e(u)=
\begin{cases}
(u-u_i,0), & u\le u_i,\\
\left(\sin\frac{2\pi(u-u_i)}{u_k-u_i},1-\cos\frac{2\pi(u-u_i)}{u_k-u_i}\right), & u_i\le u\le u_k,\\
(u-u_k,0), & u\ge u_k.
\end{cases} \tag*{(84)} e ( u ) = ⎩ ⎨ ⎧ ( u − u i , 0 ) , ( sin u k − u i 2 π ( u − u i ) , 1 − cos u k − u i 2 π ( u − u i ) ) , ( u − u k , 0 ) , u ≤ u i , u i ≤ u ≤ u k , u ≥ u k . (84) Figure 12 (PDF p. 25).
e ( u i ) = e ( u k ) e(u_i)=e(u_k) e ( u i ) = e ( u k ) u i , u k u_i,u_k u i , u k encoder is globally Lipschitz with constant at most max { 1 , 2 π / ( u k − u i ) } \max\{1,2\pi/(u_k-u_i)\} max { 1 , 2 π / ( u k − u i )}
L Q , P ⋆ ( e ) = 1 2 d P ( f t i ( P ) , f t k ( P ) ) . (85) \mathcal{L}^{\star}_{Q,P}(e)=\frac{1}{2}d_P(f_{t_i}(P),f_{t_k}(P)). \tag*{(85)} L Q , P ⋆ ( e ) = 2 1 d P ( f t i ( P ) , f t k ( P )) . (85) The admissible recovery center: Every singleton fiber can be decoded exactly. It remains to bound the radius of the pair about any g ∈ F g\in\mathcal{F} g ∈ F
For g = f t g=f_t g = f t j j j λ j ( t ) \lambda^{j}(t) λ j ( t ) i , k i,k i , k λ i j ( t ) = 0 \lambda_i^{j}(t)=0 λ i j ( t ) = 0 λ b j ( t ) ≥ 1 / 2 \lambda_b^{j}(t)\ge1/2 λ b j ( t ) ≥ 1/2 b ≠ i b\ne i b = i { i , b } \{i,b\} { i , b }
s j ∣ 1 − λ i j ( t ) + λ b j ( t ) ∣ = s j ( 1 + λ b j ( t ) ) ≥ 3 2 s j . (86) s_j\lvert1-\lambda_i^{j}(t)+\lambda_b^{j}(t)\rvert=s_j(1+\lambda_b^{j}(t))\ge\frac{3}{2}s_j. \tag*{(86)} s j ∣ 1 − λ i j ( t ) + λ b j ( t )∣ = s j ( 1 + λ b j ( t )) ≥ 2 3 s j . (86) On an adjacent connector, the block vector is w w w s j ( 1 + w ) s_j(1+w) s j ( 1 + w ) 2 s j ( 1 − w ) 2s_j(1-w) 2 s j ( 1 − w ) 2 s j 2s_j 2 s j
1 s j max { d U ( g , f t i ) , d U ( g , f t k ) } ≥ { 3 / 2 , in block j , max { 1 + w , 2 ( 1 − w ) } , on a connector , 2 , elsewhere ≥ 4 3 . (87) \frac{1}{s_j}\max\{d_U(g,f_{t_i}),d_U(g,f_{t_k})\}\ge
\begin{cases}
3/2, & \text{in block }j,\\
\max\{1+w,2(1-w)\}, & \text{on a connector},\\
2, & \text{elsewhere}
\end{cases}
\ge\frac{4}{3}. \tag*{(87)} s j 1 max { d U ( g , f t i ) , d U ( g , f t k )} ≥ ⎩ ⎨ ⎧ 3/2 , max { 1 + w , 2 ( 1 − w )} , 2 , in block j , on a connector , elsewhere ≥ 3 4 . (87) The connector bound follows by minimizing over 0 ≤ w ≤ 1 0\le w\le1 0 ≤ w ≤ 1 w = 1 / 3 w=1/3 w = 1/3
The three lemmas now give the fixed-family lower bound.
Lemma C.9 (Fixed-Family Sharpness). For every d > 1 d>1 d > 1 U \mathcal{U} U Q Q Q F \mathcal{F} F [ 0 , 1 ] [0,1] [ 0 , 1 ] ( F , Q ) (\mathcal{F},Q) ( F , Q ) C = C d C=C_d C = C d α = 1 \alpha=1 α = 1
For every integer m ≥ 1 m\ge1 m ≥ 1 P ⊆ U ∖ Q P\subseteq\mathcal{U}\setminus Q P ⊆ U ∖ Q ∣ P ∣ ≤ m |P|\le m ∣ P ∣ ≤ m e : R Q → R 2 e:\mathbb{R}^{Q}\to\mathbb{R}^{2} e : R Q → R 2
R Q ⋆ ( e ) − 2 L Q , P ⋆ ( e ) ≥ 1 6 m − 1 / ( 2 d ) . (88) \mathcal{R}^{\star}_{Q}(e)-2\mathcal{L}^{\star}_{Q,P}(e)\ge\frac{1}{6}m^{-1/(2d)}. \tag*{(88)} R Q ⋆ ( e ) − 2 L Q , P ⋆ ( e ) ≥ 6 1 m − 1/ ( 2 d ) . (88) Proof. Lemma C.6 supplies the fixed family and its geometric properties. For any admissible m , P m,P m , P
R Q ⋆ ( e ) − 2 L Q , P ⋆ ( e ) ≥ 4 3 s j − d P ( f t i ( P ) , f t k ( P ) ) ≥ s j 3 ≥ 1 6 m − 1 / ( 2 d ) . (89) \mathcal{R}^{\star}_{Q}(e)-2\mathcal{L}^{\star}_{Q,P}(e)\ge\frac{4}{3}s_j-d_P(f_{t_i}(P),f_{t_k}(P))\ge\frac{s_j}{3}\ge\frac{1}{6}m^{-1/(2d)}. \tag*{(89)} R Q ⋆ ( e ) − 2 L Q , P ⋆ ( e ) ≥ 3 4 s j − d P ( f t i ( P ) , f t k ( P )) ≥ 3 s j ≥ 6 1 m − 1/ ( 2 d ) . (89) The uniform transfer residual. For the fixed family above, the uniform transfer residual in definition 3.6 measures the recovery cost of choosing targets before the encoder. The lower bound is supplied by lemma C.9; the upper bound follows from theorem 3.4.
Proof of theorem 3.7. Use the fixed family in lemma C.9, and fix an integer m ≥ C d 2 / 2 m\ge C_d^2/2 m ≥ C d 2 /2
Lower bound. For every admissible target set P P P
sup e ∈ E L i p [ R Q ⋆ ( e ) − 2 L Q , P ⋆ ( e ) ] ≥ 1 6 m − 1 / ( 2 d ) . (90) \sup_{e\in\mathcal{E}_{\mathrm{Lip}}}\left[\mathcal{R}^{\star}_{Q}(e)-2\mathcal{L}^{\star}_{Q,P}(e)\right]\ge\frac{1}{6}m^{-1/(2d)}. \tag*{(90)} e ∈ E Lip sup [ R Q ⋆ ( e ) − 2 L Q , P ⋆ ( e ) ] ≥ 6 1 m − 1/ ( 2 d ) . (90) Taking the infimum over P P P T m ( F , Q ) T_m(\mathcal{F},Q) T m ( F , Q )
Upper bound. Apply theorem 3.4 with C = C d C=C_d C = C d α = 1 \alpha=1 α = 1 P P P
R Q ⋆ ( e ) − 2 L Q , P ⋆ ( e ) ≤ 3 ( C d 2 2 m ) 1 / ( 2 d ) . (91) \mathcal{R}^{\star}_{Q}(e)-2\mathcal{L}^{\star}_{Q,P}(e)\le3\left(\frac{C_d^2}{2m}\right)^{1/(2d)}. \tag*{(91)} R Q ⋆ ( e ) − 2 L Q , P ⋆ ( e ) ≤ 3 ( 2 m C d 2 ) 1/ ( 2 d ) . (91) Taking the supremum over E L i p \mathcal{E}_{\mathrm{Lip}} E Lip P P P
1-Lipschitz encoders. If e e e K ≥ 1 K\ge1 K ≥ 1 e / K e/K e / K z ↦ K z z\mapsto Kz z ↦ K z e e e e / K e/K e / K z ↦ z / K z\mapsto z/K z ↦ z / K
Recovery and sampling. Classical optimal recovery describes the best readout from fixed information by the radius of its consistency sets (Traub et al., 1983, Chapters 1 and 3). Here the information is e ( f ( Q ) ) e(f(Q)) e ( f ( Q ))
Optimal sampling instead chooses queries together with a reconstruction map (Krieg and Ullrich, [23 ] ). Write the arc as F = { f t : 0 ≤ t ≤ 1 } \mathcal{F}=\{f_t:0\le t\le1\} F = { f t : 0 ≤ t ≤ 1 } p 0 ∈ U ∖ Q p_0\in U\setminus Q p 0 ∈ U ∖ Q f t ( p 0 ) = t / 2 f_t(p_0)=t/2 f t ( p 0 ) = t /2 a ( y ) = f 2 y a(y)=f_{2y} a ( y ) = f 2 y 0 ≤ y ≤ 1 / 2 0\le y\le1/2 0 ≤ y ≤ 1/2 f 0 f_0 f 0
inf a : R → F sup 0 ≤ t ≤ 1 d U ( a ( f t ( p 0 ) ) , f t ) = 0. (92) \inf_{a:\mathbb{R}\to\mathcal{F}}\sup_{0\le t\le1}d_{\mathcal{U}}(a(f_t(p_0)),f_t)=0. \tag*{(92)} a : R → F inf 0 ≤ t ≤ 1 sup d U ( a ( f t ( p 0 )) , f t ) = 0. (92) The residual in equation (21) measures the budget needed to transfer prediction accuracy to recovery uniformly across encoders. On the constructed family, a residual tolerance ε \varepsilon ε ε − 2 d \varepsilon^{-2d} ε − 2 d
Uniform-norm discretization gives a related view of how finitely many queries preserve distinctions in a linear function space (Kosov and Temlyakov, [22 ] ). Its criterion is multiplicative norm preservation over that space. The criterion here couples prediction and state-valued recovery on a fixed nonlinear compact family. The covering bound controls the number of states needed at a given resolution; the pairwise witnesses in the construction determine the target budget.
Encoder regularity and target order. Theorem 3.7 gives a common encoder regularity bound. Keeping this bound fixed, we compare targets selected before and after the encoder .
Corollary C.10 (Target Selection Order). On the fixed state family of lemma C.9, let E 1 \mathcal{E}_1 E 1 1 1 1 e : R Q → R 2 e:\mathbb{R}^{Q}\to\mathbb{R}^2 e : R Q → R 2 e ∈ E 1 e\in\mathcal{E}_1 e ∈ E 1 p ∈ U ∖ Q p\in U\setminus Q p ∈ U ∖ Q R Q ⋆ ( e ) ≤ 2 L Q , { p } ⋆ ( e ) \mathcal{R}^{\star}_{Q}(e)\le2\mathcal{L}^{\star}_{Q,\{p\}}(e) R Q ⋆ ( e ) ≤ 2 L Q , { p } ⋆ ( e ) m ≥ 1 m\ge1 m ≥ 1
sup e ∈ E 1 inf P ⊆ U ∖ Q 1 ≤ ∣ P ∣ ≤ m [ R Q ⋆ ( e ) − 2 L Q , P ⋆ ( e ) ] = 0. (93) \sup_{e\in\mathcal{E}_1}\inf_{\substack{P\subseteq U\setminus Q\\1\le|P|\le m}}\left[\mathcal{R}^{\star}_{Q}(e)-2\mathcal{L}^{\star}_{Q,P}(e)\right]=0. \tag*{(93)} e ∈ E 1 sup P ⊆ U ∖ Q 1 ≤ ∣ P ∣ ≤ m inf [ R Q ⋆ ( e ) − 2 L Q , P ⋆ ( e ) ] = 0. (93) Proof. For a fixed e ∈ E 1 e\in\mathcal{E}_1 e ∈ E 1
Δ e : = max f , g ∈ F e ( f ( Q ) ) = e ( g ( Q ) ) d U ( f , g ) . (94) \Delta_e:=\max_{\substack{f,g\in\mathcal{F}\\e(f(Q))=e(g(Q))}}d_{\mathcal{U}}(f,g). \tag*{(94)} Δ e := f , g ∈ F e ( f ( Q )) = e ( g ( Q )) max d U ( f , g ) . (94) The maximum exists: the map f ↦ e ( f ( Q ) ) f\mapsto e(f(Q)) f ↦ e ( f ( Q )) F × F \mathcal{F}\times\mathcal{F} F × F R Q ⋆ ( e ) ≤ Δ e \mathcal{R}^{\star}_{Q}(e)\le\Delta_e R Q ⋆ ( e ) ≤ Δ e
Each state on the arc has nonzero block mass in at most two blocks. For any pair, differences in private and height responses are confined to the finite union of their active blocks. The remaining differing coordinates are q 0 , p 0 q_0,p_0 q 0 , p 0 Q Q Q Δ e \Delta_e Δ e p p p p p p Δ e \Delta_e Δ e
R Q ⋆ ( e ) ≤ Δ e = 2 L Q , { p } ⋆ ( e ) . (95) \mathcal{R}^{\star}_{Q}(e)\le\Delta_e=2\mathcal{L}^{\star}_{Q,\{p\}}(e). \tag*{(95)} R Q ⋆ ( e ) ≤ Δ e = 2 L Q , { p } ⋆ ( e ) . (95) When Δ e = 0 \Delta_e=0 Δ e = 0 encoder thus has a singleton target giving a nonpositive residual. Conversely, e ( u ) = ( u , 0 ) e(u)=(u,0) e ( u ) = ( u , 0 ) E 1 \mathcal{E}_1 E 1 encoder both optimal errors are zero for every target set. Taking the indicated infimum and supremum proves equation (93) .
A finite-query comparison. Allowing the family to depend on the budget yields a complementary construction: the encoder is linear, the query universe is finite, and the exponent range includes d = 1 d=1 d = 1 2 2 2 α = 1 \alpha=1 α = 1
Proposition C.11 (Budget Sharpness). For every d ≥ 1 d\ge1 d ≥ 1 c d > 0 c_d>0 c d > 0 m ≥ 1 m\ge1 m ≥ 1 U U U Q Q Q F \mathcal{F} F 1 1 1 ( F , Q ) (\mathcal{F},Q) ( F , Q ) C = 3 C=3 C = 3 α = 1 \alpha=1 α = 1 P ⊆ U ∖ Q P\subseteq U\setminus Q P ⊆ U ∖ Q ∣ P ∣ ≤ m |P|\le m ∣ P ∣ ≤ m
R Q ⋆ ( e ) ≥ 2 L Q , P ⋆ ( e ) + c d m − 1 / ( 2 d ) . (96) \mathcal{R}^{\star}_{Q}(e)\ge2\mathcal{L}^{\star}_{Q,P}(e)+c_d m^{-1/(2d)}. \tag*{(96)} R Q ⋆ ( e ) ≥ 2 L Q , P ⋆ ( e ) + c d m − 1/ ( 2 d ) . (96) :::`
Proof. State space. Fix d ≥ 1 d \ge1 d ≥ 1 m ≥ 1 m \ge1 m ≥ 1
N : = ⌈ 2 m ⌉ + 3 , s : = 1 2 N − 1 / d . (97) N := \lceil\sqrt{2m}\rceil+ 3,\qquad s := \frac{1}{2}N^{-1/d}. \tag*{(97)} N := ⌈ 2 m ⌉ + 3 , s := 2 1 N − 1/ d . (97) Let v 1 , … , v N v_1,\ldots,v_N v 1 , … , v N R N \mathbb{R}^N R N v N + 1 = v 1 v_{N+1}=v_1 v N + 1 = v 1 ⋃ i = 1 N [ v i , v i + 1 ] \bigcup_{i=1}^N[v_i,v_{i+1}] ⋃ i = 1 N [ v i , v i + 1 ] H ( λ ) = 2 min { λ 1 , λ 2 } H(\lambda)=2\min\{\lambda_1,\lambda_2\} H ( λ ) = 2 min { λ 1 , λ 2 } [ v 1 , v 2 ] [v_1,v_2] [ v 1 , v 2 ] 1 1 1 Q = { q 0 , q 1 , … , q N } Q=\{q_0,q_1,\ldots,q_N\} Q = { q 0 , q 1 , … , q N } U ∖ Q = { p 0 } ∪ { p i j : 1 ≤ i < j ≤ N } U\setminus Q=\{p_0\}\cup\{p_{ij}:1\le i<j\le N\} U ∖ Q = { p 0 } ∪ { p ij : 1 ≤ i < j ≤ N } λ \lambda λ 1 ≤ i ≤ N 1\le i\le N 1 ≤ i ≤ N 1 ≤ i < j ≤ N 1\le i<j\le N 1 ≤ i < j ≤ N
f λ ( q i ) : = s λ i , f λ ( p i j ) : = s ( λ i − λ j ) . (98) f_\lambda(q_i):=s\lambda_i,\qquad f_\lambda(p_{ij}):=s(\lambda_i-\lambda_j). \tag*{(98)} f λ ( q i ) := s λ i , f λ ( p ij ) := s ( λ i − λ j ) . (98) Also set f λ ( q 0 ) = f λ ( p 0 ) = H ( λ ) f_\lambda(q_0)=f_\lambda(p_0)=H(\lambda) f λ ( q 0 ) = f λ ( p 0 ) = H ( λ ) F \mathcal{F} F f i = f v i f_i=f_{v_i} f i = f v i λ ↦ f λ \lambda\mapsto f_\lambda λ ↦ f λ λ i = f λ ( q i ) / s \lambda_i=f_\lambda(q_i)/s λ i = f λ ( q i ) / s F \mathcal{F} F ∑ i λ i = ∑ i μ i = 1 \sum_i\lambda_i=\sum_i\mu_i=1 ∑ i λ i = ∑ i μ i = 1
d U ( f λ , f μ ) = max { s [ max k ( λ k − μ k ) − min k ( λ k − μ k ) ] , ∣ H ( λ ) − H ( μ ) ∣ } . (99) d_U(f_\lambda,f_\mu)=\max\left\{s\left[\max_k(\lambda_k-\mu_k)-\min_k(\lambda_k-\mu_k)\right],\lvert H(\lambda)-H(\mu)\rvert\right\}. \tag*{(99)} d U ( f λ , f μ ) = max { s [ k max ( λ k − μ k ) − k min ( λ k − μ k ) ] , ∣ H ( λ ) − H ( μ )∣ } . (99) The target queries capture both terms, so target observability holds with α = 1 \alpha=1 α = 1 2 s ≤ 1 2s\le1 2 s ≤ 1 H H H 0 0 0 1 1 1 1 1 1
Covering bound. Each unlifted edge has length 2 s 2s 2 s d U d_U d U 0 < ε < s 0<\varepsilon<s 0 < ε < s N ⌈ s / ε ⌉ N\lceil s/\varepsilon\rceil N ⌈ s / ε ⌉ d ≥ 1 d\ge1 d ≥ 1
N ⌈ s / ε ⌉ ≤ 2 N s ε ≤ 2 N s d ε − d = 2 1 − d ε − d ≤ ε − d . (100) N\lceil s/\varepsilon\rceil\le\frac{2Ns}{\varepsilon}\le2Ns^d\varepsilon^{-d}=2^{1-d}\varepsilon^{-d}\le\varepsilon^{-d}. \tag*{(100)} N ⌈ s / ε ⌉ ≤ ε 2 N s ≤ 2 N s d ε − d = 2 1 − d ε − d ≤ ε − d . (100) For s ≤ ε < 2 s s\le\varepsilon<2s s ≤ ε < 2 s N N N N = ( 2 s ) − d ≤ ε − d N=(2s)^{-d}\le\varepsilon^{-d} N = ( 2 s ) − d ≤ ε − d 2 s ≤ ε ≤ 1 2s\le\varepsilon\le1 2 s ≤ ε ≤ 1 1 ≤ ε − d 1\le\varepsilon^{-d} 1 ≤ ε − d 1 1 1 1 1 1 2 2 2 ⌈ 1 / ε ⌉ ≤ 2 ε − 1 \lceil1/\varepsilon\rceil\le2\varepsilon^{-1} ⌈ 1/ ε ⌉ ≤ 2 ε − 1
N F ( ε ) ≤ ε − d + 2 ε − 1 ≤ 3 ε − d , 0 < ε ≤ 1. (101) N_{\mathcal{F}}(\varepsilon)\le\varepsilon^{-d}+2\varepsilon^{-1}\le3\varepsilon^{-d},\qquad0<\varepsilon\le1. \tag*{(101)} N F ( ε ) ≤ ε − d + 2 ε − 1 ≤ 3 ε − d , 0 < ε ≤ 1. (101) Encoder construction. Fix any allowed P P P N ( N − 3 ) / 2 > m N(N-3)/2>m N ( N − 3 ) /2 > m i < j i<j i < j p i j ∉ P p_{ij}\notin P p ij ∈ / P e : R Q → Z = R N e:\mathbb{R}^Q\to Z=\mathbb{R}^N e : R Q → Z = R N
e ( x ) : = ( x ( q 0 ) , x ( q i ) + x ( q j ) , ( x ( q k ) ) k ∈ { 1 , … , N } ∖ { i , j } ) . (102) e(x):=\left(x(q_0),x(q_i)+x(q_j),(x(q_k))_{k\in\{1,\ldots,N\}\setminus\{i,j\}}\right). \tag*{(102)} e ( x ) := ( x ( q 0 ) , x ( q i ) + x ( q j ) , ( x ( q k ) ) k ∈ { 1 , … , N } ∖ { i , j } ) . (102) It combines the two corresponding input coordinates and retains all others. In particular, f i f_i f i f j f_j f j
Figure 13 (PDF p. 28).
Recovery lower bound. Any state decoder must give one output g = f λ ∈ F g=f_\lambda\in\mathcal{F} g = f λ ∈ F f i , f j f_i,f_j f i , f j i , j i,j i , j λ i , λ j \lambda_i,\lambda_j λ i , λ j max k λ k ≥ 1 / 2 \max_k\lambda_k\ge1/2 max k λ k ≥ 1/2
max { d U ( f i , g ) , d U ( f j , g ) } ≥ s ( 1 + max k λ k ) ≥ 3 s 2 . (103) \max\{d_U(f_i,g),d_U(f_j,g)\}\ge s(1+\max_k\lambda_k)\ge\frac{3s}{2}. \tag*{(103)} max { d U ( f i , g ) , d U ( f j , g )} ≥ s ( 1 + k max λ k ) ≥ 2 3 s . (103) Thus every decoder has recovery error at least $3s/2, s o , so , so
Prediction upper bound. If e ( f λ ( Q ) ) = e ( f μ ( Q ) ) e(f_\lambda(Q)) = e(f_\mu(Q)) e ( f λ ( Q )) = e ( f μ ( Q )) λ − μ = t ( v i − v j ) \lambda-\mu= t(v_i-v_j) λ − μ = t ( v i − v j ) ∣ t ∣ ≤ 1 |t| \le1 ∣ t ∣ ≤ 1 H ( λ ) = H ( μ ) H(\lambda)=H(\mu) H ( λ ) = H ( μ ) p i j p_{ij} p ij 2 s ∣ t ∣ 2s|t| 2 s ∣ t ∣ s ∣ t ∣ s|t| s ∣ t ∣ p 0 p_0 p 0 p i j ∉ P p_{ij} \notin P p ij ∈ / P
L Q , P ⋆ ( e ) ≤ s 2 . (104) \mathcal{L}^{\star}_{Q,P}(e) \le\frac{s}{2}. \tag*{(104)} L Q , P ⋆ ( e ) ≤ 2 s . (104) Combining the two error bounds and using N ≤ 2 m + 4 ≤ 6 m N \le\sqrt{2m+4} \le6\sqrt{m} N ≤ 2 m + 4 ≤ 6 m
R Q ⋆ ( e ) − 2 L Q , P ⋆ ( e ) ≥ s 2 = 1 4 N − 1 / d ≥ 1 4 ⋅ 6 1 / d m − 1 / ( 2 d ) . (105) \mathcal{R}^{\star}_{Q}(e)-2\mathcal{L}^{\star}_{Q,P}(e) \ge\frac{s}{2}=\frac{1}{4}N^{-1/d}\ge\frac{1}{4\cdot6^{1/d}}m^{-1/(2d)}. \tag*{(105)} R Q ⋆ ( e ) − 2 L Q , P ⋆ ( e ) ≥ 2 s = 4 1 N − 1/ d ≥ 4 ⋅ 6 1/ d 1 m − 1/ ( 2 d ) . (105) Taking c d = 1 / ( 4 ⋅ 6 1 / d ) c_d=1/(4\cdot6^{1/d}) c d = 1/ ( 4 ⋅ 6 1/ d )
Fine-Tuning and Recovery
The preceding results start from prediction error over the entire state family. Fine-tuning instead gives an error on training observations. Coverage and prediction stability connect these quantities: a nearby training record controls the prediction at an unobserved state. The general and low-dimensional bounds then transfer that control to recovery.
Fine-Tuning Setup
We work under assumption A.4. Fix disjoint query sets Q Q Q P P P e : R Q → Z e:\mathbb{R}^{Q}\to\mathcal{Z} e : R Q → Z b : Z → R P b:\mathcal{Z}\to\mathbb{R}^{P} b : Z → R P Z \mathcal{Z} Z
Use the recorded error and observation coverage radius from definition 3.8, together with assumption 3.9. By equation (33) , h n h_n h n
Lemma D.1 (Record Extension). Let ( F , d U ) (\mathcal{F},d_U) ( F , d U ) f 1 , … , f n ∈ F f_1,\ldots,f_n\in\mathcal{F} f 1 , … , f n ∈ F n ≥ 1 n\ge1 n ≥ 1 Q Q Q P P P ( e , b ) (e,b) ( e , b )
L ^ Q , P , n ( b ; e ) ≤ L Q , P ( b ; e ) ≤ L ^ Q , P , n ( b ; e ) + ρ ( h n ) + h n , (106) \widehat{\mathcal{L}}_{Q,P,n}(b;e)\le\mathcal{L}_{Q,P}(b;e)\le\widehat{\mathcal{L}}_{Q,P,n}(b;e)+\rho(h_n)+h_n, \tag*{(106)} L Q , P , n ( b ; e ) ≤ L Q , P ( b ; e ) ≤ L Q , P , n ( b ; e ) + ρ ( h n ) + h n , (106) where h n h_n h n (24) .
Proof. The empirical error is a maximum over states in F \mathcal{F} F f ∈ F f\in\mathcal{F} f ∈ F f i f_i f i d Q ∪ P ( f ( Q ∪ P ) , f i ( Q ∪ P ) ) ≤ h n d_{Q\cup P}(f(Q\cup P),f_i(Q\cup P))\le h_n d Q ∪ P ( f ( Q ∪ P ) , f i ( Q ∪ P )) ≤ h n
d P ( ( b ∘ e ) ( f ( Q ) ) , f ( P ) ) ≤ d P ( ( b ∘ e ) ( f ( Q ) ) , ( b ∘ e ) ( f i ( Q ) ) ) + d P ( ( b ∘ e ) ( f i ( Q ) ) , f i ( P ) ) + d P ( f i ( P ) , f ( P ) ) ≤ ρ ( h n ) + L ^ Q , P , n ( b ; e ) + h n . (107) \begin{aligned}
d_P((b\circ e)(f(Q)),f(P))
&\le d_P((b\circ e)(f(Q)),(b\circ e)(f_i(Q)))+d_P((b\circ e)(f_i(Q)),f_i(P))\\
&\quad+d_P(f_i(P),f(P))\le\rho(h_n)+\widehat{\mathcal{L}}_{Q,P,n}(b;e)+h_n. \tag*{(107)}
\end{aligned} d P (( b ∘ e ) ( f ( Q )) , f ( P )) ≤ d P (( b ∘ e ) ( f ( Q )) , ( b ∘ e ) ( f i ( Q ))) + d P (( b ∘ e ) ( f i ( Q )) , f i ( P )) + d P ( f i ( P ) , f ( P )) ≤ ρ ( h n ) + L Q , P , n ( b ; e ) + h n . (107) Taking the supremum over f f f
Proof of proposition 3.10. Lemma D.1 gives the prediction bound. Since the optimal prediction error is no larger, theorems 3.2 and 3.4 and monotonicity of ω P \omega_P ω P
We now choose the pair by fine-tuning. Let H \mathcal{H} H encoder–predictor pairs accessible from the initialization by changing the permitted parameters. The empirical error is defined in equation (23) .
Definition D.2 (Fine-Tuning). The optimal prediction error in the fine-tuning class is
L Q , P H , ⋆ : = inf ( e , b ) ∈ H L Q , P ( b ; e ) . (108) \mathcal{L}^{\mathcal{H},\star}_{Q,P}:=\inf_{(e,b)\in\mathcal{H}}\mathcal{L}_{Q,P}(b;e). \tag*{(108)} L Q , P H , ⋆ := ( e , b ) ∈ H inf L Q , P ( b ; e ) . (108) Given an optimization tolerance η ≥ 0 \eta\ge0 η ≥ 0 η \eta η ( e ^ , b ^ ) ∈ H (\hat{e},\hat{b})\in\mathcal{H} ( e ^ , b ^ ) ∈ H
L ^ Q , P , n ( b ^ ; e ^ ) ≤ inf ( e , b ) ∈ H L ^ Q , P , n ( b ; e ) + η . (109) \widehat{\mathcal{L}}_{Q,P,n}(\hat{b};\hat{e})\le\inf_{(e,b)\in\mathcal{H}}\widehat{\mathcal{L}}_{Q,P,n}(b;e)+\eta. \tag*{(109)} L Q , P , n ( b ^ ; e ^ ) ≤ ( e , b ) ∈ H inf L Q , P , n ( b ; e ) + η . (109) For every η > 0 \eta> 0 η > 0 η \eta η η = 0 \eta= 0 η = 0 L Q , P H , ⋆ \mathcal{L}^{\mathcal{H},\star}_{Q,P} L Q , P H , ⋆ encoder–predictor pairs, while L Q , P ⋆ ( e ) \mathcal{L}^{\star}_{Q,P}(e) L Q , P ⋆ ( e ) encoder fixed.
For a fine-tuning output, the same lemma combines with empirical near-optimality to relate its prediction error to the best error in the model class.
Corollary D.3 (Fine-Tuning Consistency). Every η \eta η
L Q , P ⋆ ≤ L Q , P ( b ^ ; e ^ ) ≤ L Q , P H , ⋆ + η + ρ ( h n ) + h n . (110) \mathcal{L}^{\star}_{Q,P} \le\mathcal{L}_{Q,P}(\hat{b};\hat{e}) \le\mathcal{L}^{\mathcal{H},\star}_{Q,P} + \eta+ \rho(h_n) + h_n. \tag*{(110)} L Q , P ⋆ ≤ L Q , P ( b ^ ; e ^ ) ≤ L Q , P H , ⋆ + η + ρ ( h n ) + h n . (110) For stages k ≥ 1 k \ge1 k ≥ 1 ρ \rho ρ ( e ^ k , b ^ k ) (\hat{e}_k,\hat{b}_k) ( e ^ k , b ^ k ) η k \eta_k η k h k h_k h k
L Q k , P k H k , ⋆ + η k + h k → k → ∞ 0 , (111) \mathcal{L}^{\mathcal{H}_k,\star}_{Q_k,P_k} + \eta_k + h_k \xrightarrow{k\to\infty} 0, \tag*{(111)} L Q k , P k H k , ⋆ + η k + h k k → ∞ 0 , (111) then
0 ≤ L Q k , P k ⋆ ( e ^ k ) ≤ L Q k , P k ( b ^ k ; e ^ k ) → k → ∞ 0. (112) 0 \le\mathcal{L}^{\star}_{Q_k,P_k}(\hat{e}_k) \le\mathcal{L}_{Q_k,P_k}(\hat{b}_k;\hat{e}_k) \xrightarrow{k\to\infty} 0. \tag*{(112)} 0 ≤ L Q k , P k ⋆ ( e ^ k ) ≤ L Q k , P k ( b ^ k ; e ^ k ) k → ∞ 0. (112) Proof. For every pair in H \mathcal{H} H
L ^ Q , P , n ( b ^ ; e ^ ) ≤ inf ( e , b ) ∈ H L ^ Q , P , n ( b ; e ) + η ≤ L Q , P H , ⋆ + η . (113) \widehat{\mathcal{L}}_{Q,P,n}(\hat{b};\hat{e}) \le\inf_{(e,b)\in\mathcal{H}} \widehat{\mathcal{L}}_{Q,P,n}(b;e) + \eta\le\mathcal{L}^{\mathcal{H},\star}_{Q,P} + \eta. \tag*{(113)} L Q , P , n ( b ^ ; e ^ ) ≤ ( e , b ) ∈ H inf L Q , P , n ( b ; e ) + η ≤ L Q , P H , ⋆ + η . (113) Apply lemma D.1 to the returned pair to obtain the upper bounds. The lower bound follows because ( e ^ , b ^ ) ∈ H (\hat{e},\hat{b}) \in\mathcal{H} ( e ^ , b ^ ) ∈ H
Under equation (111) , all three nonnegative terms tend to zero. In particular, h k → 0 h_k \to0 h k → 0 ρ ( h k ) → 0 \rho(h_k) \to0 ρ ( h k ) → 0 L Q k , P k ( b ^ k ; e ^ k ) → 0 \mathcal{L}_{Q_k,P_k}(\hat{b}_k;\hat{e}_k) \to0 L Q k , P k ( b ^ k ; e ^ k ) → 0
General Case
Theorem D.4 (General Fine-Tuning Bound). Every η \eta η
L Q , P ⋆ ( e ^ ) ≤ R Q ⋆ ( e ^ ) ≤ ω P ( 2 [ L Q , P H , ⋆ + η + ρ ( h n ) + h n ] ) . (114) \mathcal{L}^{\star}_{Q,P}(\hat{e}) \le\mathcal{R}^{\star}_{Q}(\hat{e}) \le\omega_P\left(2\left[\mathcal{L}^{\mathcal{H},\star}_{Q,P} + \eta+ \rho(h_n) + h_n\right]\right). \tag*{(114)} L Q , P ⋆ ( e ^ ) ≤ R Q ⋆ ( e ^ ) ≤ ω P ( 2 [ L Q , P H , ⋆ + η + ρ ( h n ) + h n ] ) . (114) For stages k ≥ 1 k \ge1 k ≥ 1 ρ \rho ρ ( e ^ k , b ^ k ) (\hat{e}_k,\hat{b}_k) ( e ^ k , b ^ k ) η k \eta_k η k ( P k ) k ≥ 1 (P_k)_{k\ge1} ( P k ) k ≥ 1 F \mathcal{F} F
L Q k , P k H k , ⋆ + η k + h k → k → ∞ 0 , (115) \mathcal{L}^{\mathcal{H}_k,\star}_{Q_k,P_k} + \eta_k + h_k \xrightarrow{k\to\infty} 0, \tag*{(115)} L Q k , P k H k , ⋆ + η k + h k k → ∞ 0 , (115) then R Q k ⋆ ( e ^ k ) → 0 \mathcal{R}^{\star}_{Q_k}(\hat{e}_k) \to0 R Q k ⋆ ( e ^ k ) → 0 h k h_k h k Q k ∪ P k Q_k \cup P_k Q k ∪ P k k k k (24) .
Proof. Substitute equation (110) into theorem 3.2, using L Q , P ⋆ ( e ^ ) ≤ L Q , P ( b ^ ; e ^ ) \mathcal{L}^{\star}_{Q,P}(\hat{e}) \le\mathcal{L}_{Q,P}(\hat{b};\hat{e}) L Q , P ⋆ ( e ^ ) ≤ L Q , P ( b ^ ; e ^ ) ω P \omega_P ω P
For the asymptotic claim, corollary D.3 gives vanishing optimal prediction error. State separation and theorem 3.3 then give vanishing recovery error. □
Low-Dimensional Case
Using the targets constructed in theorem 3.4 makes the query budget explicit under the same training rule and stability condition.
Theorem D.5 (Low-Dimensional Fine-Tuning Bound). Suppose ( F , Q ) (\mathcal{F}, Q) ( F , Q ) C , d , α C,d,\alpha C , d , α m ≥ C 2 / 2 m \ge C^2/2 m ≥ C 2 /2 P ⊆ U ∖ Q P \subseteq\mathcal{U} \setminus Q P ⊆ U ∖ Q ∣ P ∣ ≤ m |P| \le m ∣ P ∣ ≤ m η \eta η
L Q , P ⋆ ( e ^ ) ≤ R Q ⋆ ( e ^ ) ≤ 2 α [ L Q , P H Q , P ⋆ + η + ρ ( h n ) + h n ] + 2 + α α ( C 2 2 m ) 1 2 d . (116) \mathcal{L}^{\star}_{Q,P}(\hat{e}) \le\mathcal{R}^{\star}_{Q}(\hat{e}) \le\frac{2}{\alpha}\left[\mathcal{L}^{\mathcal{H}^{\star}_{Q,P}}_{Q,P}+\eta+\rho(h_n)+h_n\right]+\frac{2+\alpha}{\alpha}\left(\frac{C^2}{2m}\right)^{\frac{1}{2d}}. \tag*{(116)} L Q , P ⋆ ( e ^ ) ≤ R Q ⋆ ( e ^ ) ≤ α 2 [ L Q , P H Q , P ⋆ + η + ρ ( h n ) + h n ] + α 2 + α ( 2 m C 2 ) 2 d 1 . (116) For budgets m → ∞ m \to\infty m → ∞ P m P_m P m ( e ^ m , b ^ m ) (\hat{e}_m,\hat{b}_m) ( e ^ m , b ^ m ) η m \eta_m η m C , d , α C,d,\alpha C , d , α ρ \rho ρ
L Q m , P m H m , ⋆ + η m + h m → m → ∞ 0 , (117) \mathcal{L}^{\mathcal{H}_{m,\star}}_{Q_m,P_m}+\eta_m+h_m \xrightarrow[m\to\infty]{} 0, \tag*{(117)} L Q m , P m H m , ⋆ + η m + h m m → ∞ 0 , (117) then R Q m ⋆ ( e ^ m ) → 0 \mathcal{R}^{\star}_{Q_m}(\hat{e}_m) \to0 R Q m ⋆ ( e ^ m ) → 0 h m h_m h m Q m ∪ P m Q_m \cup P_m Q m ∪ P m m m m (24) .
Proof. Select P P P (110) into that theorem, using L Q , P ⋆ ( e ^ ) ≤ L Q , P ( b ^ ; e ^ ) \mathcal{L}^{\star}_{Q,P}(\hat{e}) \le\mathcal{L}_{Q,P}(\hat{b};\hat{e}) L Q , P ⋆ ( e ^ ) ≤ L Q , P ( b ^ ; e ^ )
Under the stated convergence condition, corollary D.3 gives vanishing optimal prediction error. Applying theorem 3.5 to the selected target sets gives vanishing recovery error.
The bounds identify the contributions of the model class, optimization tolerance, training coverage, and prediction stability. The low-dimensional result also makes the target-query budget explicit: n n n m m m
Recovery from Noisy Observations
The construction in theorem 3.4 also applies when a query’s response is a conditional mean and its observation is noisy. The state specifies the mean response profile; the measurement law describes how a recording is drawn from it. Prediction error and measurement perturbation then enter the same distance to a covering center. Restricting the construction to a frequently encountered state family gives a probability guarantee and an integrated recovery bound.
Let F \mathcal{F} F D D D G ⊆ F G \subseteq\mathcal{F} G ⊆ F ( G , Q ) (G,Q) ( G , Q ) C G , d , α C_G,d,\alpha C G , d , α m ≥ C G 2 / 2 m \ge C_G^2/2 m ≥ C G 2 /2 P P P G G G
Let Φ = b ∘ e \Phi=b\circ e Φ = b ∘ e K K K ℓ \ell ℓ G G G b ( z ) b(z) b ( z ) P P P a ( z ) a(z) a ( z ) a ∘ e a\circ e a ∘ e Φ \Phi Φ
Lemma E.1 (Recovery under Perturbation). Under this setup , if f ∈ G f \in G f ∈ G d Q ( x , f ( Q ) ) ≤ η d_Q(x,f(Q)) \le\eta d Q ( x , f ( Q )) ≤ η
d U ( ( a ∘ e ) ( x ) , f ) ≤ 2 α ( ℓ + K η ) + 2 + α α ( C G 2 2 m ) 1 2 d . (118) d_{\mathcal{U}}((a\circ e)(x),f) \le\frac{2}{\alpha}(\ell+K\eta)+\frac{2+\alpha}{\alpha}\left(\frac{C_G^2}{2m}\right)^{\frac{1}{2d}}. \tag*{(118)} d U (( a ∘ e ) ( x ) , f ) ≤ α 2 ( ℓ + K η ) + α 2 + α ( 2 m C G 2 ) 2 d 1 . (118) Proof. The triangle inequality and input sensitivity give
d P ( Φ ( x ) , f ( P ) ) ≤ d P ( Φ ( x ) , Φ ( f ( Q ) ) ) + d P ( Φ ( f ( Q ) ) , f ( P ) ) ≤ K η + ℓ . (119) d_P(\Phi(x),f(P)) \le d_P(\Phi(x),\Phi(f(Q)))+d_P(\Phi(f(Q)),f(P)) \le K\eta+\ell. \tag*{(119)} d P ( Φ ( x ) , f ( P )) ≤ d P ( Φ ( x ) , Φ ( f ( Q ))) + d P ( Φ ( f ( Q )) , f ( P )) ≤ K η + ℓ . (119) Apply lemma C.4 on G G G y = Φ ( x ) y=\Phi(x) y = Φ ( x ) (118) .
The pointwise bound becomes a probability guarantee when both the state and the measurement lie in their specified ranges.
Corollary E.2 (Recovery under Noise). Under the preceding setup , let F F F μ \mu μ μ ( G ) ≥ 1 − δ \mu(G) \ge1-\delta μ ( G ) ≥ 1 − δ δ ∈ [ 0 , 1 ) \delta\in[0,1) δ ∈ [ 0 , 1 ) X X X f ∈ G f \in G f ∈ G
Pr ( d Q ( X , f ( Q ) ) ≤ η ∣ F = f ) ≥ 1 − ζ , (120) \Pr(d_Q(X,f(Q)) \le\eta\mid F=f) \ge1-\zeta, \tag*{(120)} Pr ( d Q ( X , f ( Q )) ≤ η ∣ F = f ) ≥ 1 − ζ , (120) where η ≥ 0 \eta\ge0 η ≥ 0 ζ ∈ ( 0 , 1 ) \zeta\in(0,1) ζ ∈ ( 0 , 1 )
B : = min { D , 2 α ( ℓ + K η ) + 2 + α α ( C G 2 2 m ) 1 / ( 2 d ) } . (121) B := \min\left\{D,\frac{2}{\alpha}(\ell+K\eta)+\frac{2+\alpha}{\alpha}\left(\frac{C_G^2}{2m}\right)^{1/(2d)}\right\}. \tag*{(121)} B := min { D , α 2 ( ℓ + K η ) + α 2 + α ( 2 m C G 2 ) 1/ ( 2 d ) } . (121) the recovery error obeys
Pr ( d U ( ( a ∘ e ) ( X ) , F ) ≤ B ) ≥ 1 − δ − ζ , (122) \Pr\left(d_{\mathcal U}((a\circ e)(X),F)\le B\right)\ge1-\delta-\zeta, \tag*{(122)} Pr ( d U (( a ∘ e ) ( X ) , F ) ≤ B ) ≥ 1 − δ − ζ , (122) E d U ( ( a ∘ e ) ( X ) , F ) 2 ≤ B 2 + D 2 ( δ + ζ ) . (123) \mathbb{E}d_{\mathcal U}((a\circ e)(X),F)^2\le B^2+D^2(\delta+\zeta). \tag*{(123)} E d U (( a ∘ e ) ( X ) , F ) 2 ≤ B 2 + D 2 ( δ + ζ ) . (123) Proof. The event F ∈ G F\in G F ∈ G d Q ( X , F ( Q ) ) ≤ η d_Q(X,F(Q))\le\eta d Q ( X , F ( Q )) ≤ η
μ ( G ) ( 1 − ζ ) ≥ ( 1 − δ ) ( 1 − ζ ) ≥ 1 − δ − ζ . (124) \mu(G)(1-\zeta)\ge(1-\delta)(1-\zeta)\ge1-\delta-\zeta. \tag*{(124)} μ ( G ) ( 1 − ζ ) ≥ ( 1 − δ ) ( 1 − ζ ) ≥ 1 − δ − ζ . (124) On this event, lemma E.1 and the diameter bound give error at most B B B D D D F \mathcal F F
Count observations. For a conditional Poisson measurement law, take X q = N q / T X_q=N_q/T X q = N q / T N q ∣ F = f N_q\mid F=f N q ∣ F = f T f ( q ) Tf(q) T f ( q ) 0 ≤ f ( q ) ≤ Λ 0\le f(q)\le\Lambda 0 ≤ f ( q ) ≤ Λ G G G T T T
η ( T , ζ ) = 2 Λ log ( 2 ∣ Q ∣ / ζ ) T + 2 log ( 2 ∣ Q ∣ / ζ ) 3 T . (125) \eta(T,\zeta)=\sqrt{\frac{2\Lambda\log(2|Q|/\zeta)}{T}}+\frac{2\log(2|Q|/\zeta)}{3T}. \tag*{(125)} η ( T , ζ ) = T 2Λ log ( 2∣ Q ∣/ ζ ) + 3 T 2 log ( 2∣ Q ∣/ ζ ) . (125) Indeed, the centered Poisson moment-generating function obeys log E e u ( N q − T f ( q ) ) = T f ( q ) ( e u − 1 − u ) \log\mathbb{E}e^{u(N_q-Tf(q))}=Tf(q)(e^u-1-u) log E e u ( N q − T f ( q )) = T f ( q ) ( e u − 1 − u ) e u − 1 − u ≤ u 2 / [ 2 ( 1 − u / 3 ) ] e^u-1-u\le u^2/[2(1-u/3)] e u − 1 − u ≤ u 2 / [ 2 ( 1 − u /3 )] 0 ≤ u < 3 0\le u<3 0 ≤ u < 3
Pr ( ∣ N q − T f ( q ) ∣ > 2 T Λ v + 2 v 3 | F = f ) ≤ 2 e − v . (126) \Pr\left(\left|N_q-Tf(q)\right|>\sqrt{2T\Lambda v}+\frac{2v}{3}\ \middle|\ F=f\right)\le2e^{-v}. \tag*{(126)} Pr ( ∣ N q − T f ( q ) ∣ > 2 T Λ v + 3 2 v F = f ) ≤ 2 e − v . (126) Choose v = log ( 2 ∣ Q ∣ / ζ ) v=\log(2|Q|/\zeta) v = log ( 2∣ Q ∣/ ζ ) K η ( T , ζ ) K\eta(T,\zeta) K η ( T , ζ )
Prediction sensitivity. For a continuously differentiable predictor composition, a sufficient full-input constant is
K ≥ sup x ∈ R Q ∥ J Φ ( x ) ∥ ∞ → ∞ = sup x ∈ R Q max p ∈ R ∑ q ∈ Q ∣ ∂ Φ p ( x ) ∂ x q ∣ . (127) K\ge\sup_{x\in\mathbb{R}^Q}\left\|J_{\Phi}(x)\right\|_{\infty\to\infty}
=\sup_{x\in\mathbb{R}^Q}\max_{p\in R}\sum_{q\in Q}\left|\frac{\partial\Phi_p(x)}{\partial x_q}\right|. \tag*{(127)} K ≥ x ∈ R Q sup ∥ J Φ ( x ) ∥ ∞ → ∞ = x ∈ R Q sup p ∈ R max q ∈ Q ∑ ∂ x q ∂ Φ p ( x ) . (127) The derivative bound may instead be taken on a convex domain containing every input and intervening segment used in the guarantee.
Functional observability. Functional observers recover selected state coordinates in structured dynamical networks; algebraic and graph conditions characterize which targets are accessible ((Montanari et al., 2022; 2023)). Observability also guides the eigenstructure and sparsity of neural state-space architectures ([15 , 16 ] ). Here, the state is specified by its complete query-response profile, and quantitative target observability compares its full metric with the remaining responses. The query-budget construction uses this comparison to turn target prediction into a state-valued recovery map. Retracing adds cycle consistency to predictive state learning in reinforcement learning ((Yu et al., 2022)); our bounds concern the distinctions preserved by a fixed encoder and the error of recovering its response profile.
Synthetic Experiment Details
The generator specifies the full state family and every query response. This makes recovery error, target separation, and the constants in the error bound directly computable. We describe the generator and inverse map first, then examine the budget dependence and the assumptions responsible for it.
The coordinatewise construction is followed by a second, joint-query construction in section G, which supplies the calibration records for a global recovery certificate.
State Family and Query
The GT observation law is a deterministic function of a sampled latent vector x x x Sphere , x x x R d + 1 \mathbb{R}^{d+1} R d + 1 Torus , its coordinates are cosine–sine pairs of independent uniform angles. For General , x x x [ − 1 , 1 ] D [-1,1]^D [ − 1 , 1 ] D (30) . The coefficients are sampled once and shared across families and repetitions. The map i ( j ) i(j) i ( j ) s j s_j s j encoder and predictor . These are structured, separable observation laws; their geometry and full observation profiles can be evaluated exactly.
Table 4. Synthetic design. The circle is included once, as the sphere of intrinsic dimension one. The general family has independent ambient coordinates.
Throughout the paper, decimals are rounded to at most four places and omit the leading zero. Exact zeros in decimal summaries appear as .0000; nonzero values rounding to zero appear as .0001 in magnitude, retaining their sign. All calculations use unrounded values.
Prediction and Deterministic Recovery
A shared coordinatewise MLP encodes input observations and their query coefficients. A second shared MLP combines this learned state with each target channel’s coefficients to predict its observation. We minimize the mean, over training states, of the maximum absolute target error. Every configuration is evaluated at the final training epoch specified in advance. Matching pilot configurations enter the paired-repetition design once.
The ideal GT map in equation (31) minimizes target discrepancy; the implementation approximates it by bisection. Monotonicity of each observation channel converts an allowed discrepancy into a latent-coordinate interval. We intersect these intervals and bisect the discrepancy until the resulting box intersects the known latent domain. For a sphere, intersection is decided by the minimum and maximum attainable squared norms; for a torus, the same criterion is applied to each circle factor. A deterministic feasible point then defines the recovered state. The inverse uses the predicted responses and the known latent domain; the test state’s latent vector enters the evaluation of recovery error.
Table 6. Synthetic training and deterministic recovery settings. Gradient updates train the encoder and predictor .
For each test state f f f d P ( ( b ∘ e ) ( f ( Q ) ) , f ( P ) ) d_P((b\circ e)(f(Q)), f(P)) d P (( b ∘ e ) ( f ( Q )) , f ( P )) d U ( f ^ ( ( b ∘ e ) ( f ( Q ) ) ) , f ) d_U(\widehat{f}((b\circ e)(f(Q))), f) d U ( f (( b ∘ e ) ( f ( Q ))) , f ) ± \pm ± Table 5 . Bound checks instead use the maximum over the test states within each run. Seed pairing fixes initialization, state samples, and target order across budgets; seeds jointly change these sources of randomness.
Table 5. Random-repetition design. Reported dispersions are sample standard deviations across repetitions of the same condition, with the biological aggregation held fixed. A repetition changes the sources of randomness specified by its protocol.
Training choices and computation. Dimensions and target counts form geometric grids spanning the three state families. The shared MLP widths, batch size, and sample counts are fixed design parameters; every reported configuration uses the same optimizer and two-stage schedule. Preliminary convergence checks informed the 200-epoch endpoint, which is held fixed across the full suite. AdamW uses weight decay .01, moments (.9, .999), and numerical stabilizer 10 − 8 10^{-8} 1 0 − 8
With the network widths fixed, a batch of B B B O ( B ( D + m ) ) O(B(D + m)) O ( B ( D + m )) encoder processes D D D predictor evaluates m m m D D D Table 9 ; target and grid batching bound working memory without changing the evaluated domain.
Table 9. Whole-domain audit settings. Bounds use the final training checkpoints and their training target sets. Seed sets and grid sizes appear in Table 5 .
Reproduction materials and infrastructure. The accompanying archive ontic-reproducibility.zip supplies source code, case manifests, dependency versions, original numerical endpoints, and commands for preprocessing, training, evaluation, and aggregation. Figure and table sources are included with the manuscript source. Synthetic data are generated by the supplied response laws and seeds. HC-3 recordings are obtained from their provider under the terms in section H. The archive retains all endpoints of the included suites and separates within-condition seed variation from biological aggregation.
Experiments ran in a Linux batch-computing environment. The supplied training configuration uses one NVIDIA A100 (80 GB); observation-geometry calculations use CPU numerical linear algebra. The reproduction environment is Python 3.10.19, PyTorch 2.10.0, NumPy 1.26.4, SciPy 1.14.1, Matplotlib 3.8.4, and mpmath 1.3.0. The archive includes version-pinned dependencies. Checkpoints retain optimizer, sampler, and random-generator state, with separate latest and previous copies. Resumption uses the same device type and PyTorch version.
The main-text observability inequality and constant α P \alpha_P α P (32) . We check coordinate coverage and its budget requirement before applying the transfer bound.
Assumption Checks
Observability. Write r r r d + 1 d+1 d + 1 2 d 2d 2 d D D D I ( P ) = { i ( j ) : ( j , k ) ∈ P } I(P)=\{i(j):(j,k)\in P\} I ( P ) = { i ( j ) : ( j , k ) ∈ P } (30) ,
P separates F ⟺ I ( P ) = { 1 , … , r } . (128) P\text{ separates }\mathcal{F}\quad\Longleftrightarrow\quad I(P)=\{1,\ldots,r\}. \tag*{(128)} P separates F ⟺ I ( P ) = { 1 , … , r } . (128) Full coordinate coverage gives separation by proposition 4.1. Conversely, each latent domain is invariant under changing the sign of one coordinate and contains a state with that coordinate nonzero. Reflecting an unobserved coordinate therefore gives two distinct states with identical target responses. Their full profiles differ because the query universe observes every coordinate.
Proof of proposition 4.1. Bound each channel. Fix f = f x f=f_x f = f x g = f x ′ g=f_{x'} g = f x ′ F \mathcal{F} F ( j , k ) (j,k) ( j , k ) ∣ t ∣ ≤ 1 |t|\leq1 ∣ t ∣ ≤ 1
a j k sech 2 ( a j k + ∣ c j k ∣ ) ≤ ∣ d d t tanh ( a j k s j t + c j k ) ∣ ≤ a j k . (129) a_{jk}\operatorname{sech}^{2}(a_{jk}+|c_{jk}|)\leq\left|\frac{d}{dt}\tanh(a_{jk}s_jt+c_{jk})\right|\leq a_{jk}. \tag*{(129)} a j k sech 2 ( a j k + ∣ c j k ∣ ) ≤ d t d tanh ( a j k s j t + c j k ) ≤ a j k . (129) The mean value theorem, followed by maximization over the selected target channels and over all channels, respectively, gives
d P ( f , g ) ≥ max i ℓ i ∣ x i − x i ′ ∣ , d U ( f , g ) ≤ max i A i ∣ x i − x i ′ ∣ . (130) d_P(f,g)\geq\max_i \ell_i|x_i-x_i'|,\qquad d_U(f,g)\leq\max_i A_i|x_i-x_i'|. \tag*{(130)} d P ( f , g ) ≥ i max ℓ i ∣ x i − x i ′ ∣ , d U ( f , g ) ≤ i max A i ∣ x i − x i ′ ∣. (130) A single increasing channel turns coordinate separation into response separation:
Figure 1 (PDF p. 35).
Compare the distances. Every coordinate has a target channel with positive slope magnitude, so ℓ i > 0 \ell_i>0 ℓ i > 0 A i > 0 A_i>0 A i > 0 α P = min i ℓ i / A i > 0 \alpha_P=\min_i\ell_i/A_i>0 α P = min i ℓ i / A i > 0 ℓ i ≥ α P A i \ell_i\geq\alpha_PA_i ℓ i ≥ α P A i i i i
d P ( f , g ) ≥ max i ℓ i ∣ x i − x i ′ ∣ ≥ α P max i A i ∣ x i − x i ′ ∣ ≥ α P d U ( f , g ) . (131) d_P(f,g)\geq\max_i\ell_i|x_i-x_i'|\geq\alpha_P\max_i A_i|x_i-x_i'|\geq\alpha_Pd_U(f,g). \tag*{(131)} d P ( f , g ) ≥ i max ℓ i ∣ x i − x i ′ ∣ ≥ α P i max A i ∣ x i − x i ′ ∣ ≥ α P d U ( f , g ) . (131) This proves equation (32).
For each certified target set, we recompute this constant and the test-error transfer. The input and candidate-target pools cover all latent coordinates.
Query budget. The actual target sampler permits a finite-budget calculation. Let M M M n i n_i n i i i i m m m i i i ( M − n i m ) / ( M m ) \binom{M-n_i}{m}/\binom{M}{m} ( m M − n i ) / ( m M )
Pr { P fails to separate F } ≤ min { 1 , ∑ i = 1 r ( M − n i m ) ( M m ) } . (132) \Pr\{P\text{ fails to separate }\mathcal{F}\}\leq\min\left\{1,\sum_{i=1}^{r}\frac{\binom{M-n_i}{m}}{\binom{M}{m}}\right\}. \tag*{(132)} Pr { P fails to separate F } ≤ min { 1 , i = 1 ∑ r ( m M ) ( m M − n i ) } . (132) where a binomial coefficient is zero when its lower argument exceeds its upper argument. On the complementary event, the global error transfer holds simultaneously for every predictor and every state, using the corresponding α P \alpha_P α P exp ( − m min i n i / M ) \exp(-m\min_i n_i/M) exp ( − m min i n i / M ) m ≥ ( M / min i n i ) log ( r / η ) m\geq(M/\min_i n_i)\log(r/\eta) m ≥ ( M / min i n i ) log ( r / η ) η \eta η M / min i n i M/\min_i n_i M / min i n i r r r r log ( r / η ) r\log(r/\eta) r log ( r / η )
We evaluate the without-replacement bound using the actual channel counts (Table 7 ). The low-dimensional families repeat each latent coordinate across many channels, so their sufficient budgets are much smaller. These are family-specific sampling guarantees; the general covering theorem accommodates arbitrary admissible response laws.
Covering number. The same generator supplies explicit covering constants. Put A = max j k a j k A=\max_{jk}a_{jk} A = max j k a j k
N F ( ε ) ≤ ( 1 + 2 π d A ε ) d , N F ( ε ) ≤ ( 1 + 2 π A ε ) d . (133) \mathcal{N}_{\mathcal{F}}(\varepsilon)\leq\left(1+\frac{2\pi dA}{\varepsilon}\right)^d,\qquad\mathcal{N}_{\mathcal{F}}(\varepsilon)\leq\left(1+\frac{2\pi A}{\varepsilon}\right)^d. \tag*{(133)} N F ( ε ) ≤ ( 1 + ε 2 π d A ) d , N F ( ε ) ≤ ( 1 + ε 2 π A ) d . (133) Indeed, changing a sphere angle changes its unit vector by at most the angular displacement in Euclidean norm; summing over angles gives the first grid radius. Each sine–cosine factor of a torus is Lipschitz in its own angle, giving the second radius in the maximum-coordinate metric. The observation map is A A A ε ≤ 1 \varepsilon\le1 ε ≤ 1 C ε − d C\varepsilon^{-d} C ε − d N F ( ε ) ≤ ( 1 + 2 A / ε ) D \mathcal{N}_{\mathcal{F}}(\varepsilon) \le(1 + 2A/\varepsilon)^D N F ( ε ) ≤ ( 1 + 2 A / ε ) D
Error Transfer
When every latent coordinate has a target channel, equation (32) supplies the global observability constant α P \alpha_P α P g ∈ F g \in\mathcal{F} g ∈ F ξ ≥ 0 \xi\ge0 ξ ≥ 0
α P d U ( g , f ) ≤ d P ( g ( P ) , y ) + d P ( y , f ( P ) ) ≤ 2 d P ( y , f ( P ) ) + ξ . (134) \alpha_P d_U(g,f) \le d_P(g(P),y) + d_P(y,f(P)) \le2d_P(y,f(P)) + \xi. \tag*{(134)} α P d U ( g , f ) ≤ d P ( g ( P ) , y ) + d P ( y , f ( P )) ≤ 2 d P ( y , f ( P )) + ξ . (134) The exact map has ξ = 0 \xi= 0 ξ = 0 [ 0 , ∥ y ∥ ∞ + 1 ] [0,\lVert y\rVert_\infty+ 1] [ 0 , ∥ y ∥ ∞ + 1 ] 10 − 10 10^{-10} 1 0 − 10 ξ ≤ 2 − 44 ( ∥ y ∥ ∞ + 1 ) + 10 − 10 \xi\le2^{-44}(\lVert y\rVert_\infty+ 1) + 10^{-10} ξ ≤ 2 − 44 (∥ y ∥ ∞ + 1 ) + 1 0 − 10 ξ \xi ξ 2 L ^ / α P 2\widehat{L}/\alpha_P 2 L / α P L ^ \widehat{L} L (128) .
Table 8: Numerical recovery compared with the exact-map benchmark on separating target sets. All observed errors lie below the benchmark, computed from maximum test prediction errors and global derivative bounds.
Table 8.
The preceding transfer checks use test states. The coordinatewise architecture also permits a bound over the entire known family. For a target t = ( j , k ) t = (j,k) t = ( j , k ) v = s j x ( j ) v = s_jx_{(j)} v = s j x ( j ) h t ( v ) h_t(v) h t ( v ) tanh ( a j k v + c j k ) \tanh(a_{jk}v + c_{jk}) tanh ( a j k v + c j k ) e s h e_{\mathrm{sh}} e sh b s h b_{\mathrm{sh}} b sh
h t ( v ) = b s h ( e s h ( tanh ( a j 1 v + c j 1 ) , tanh ( a j 2 v + c j 2 ) , a j 1 , a j 2 , c j 1 , c j 2 ) , a j k , c j k ) − tanh ( a j k v + c j k ) . (135) h_t(v) = b_{\mathrm{sh}}\left(e_{\mathrm{sh}}\left(\tanh(a_{j1}v + c_{j1}), \tanh(a_{j2}v + c_{j2}), a_{j1}, a_{j2}, c_{j1}, c_{j2}\right), a_{jk}, c_{jk}\right)
- \tanh(a_{jk}v + c_{jk}). \tag*{(135)} h t ( v ) = b sh ( e sh ( tanh ( a j 1 v + c j 1 ) , tanh ( a j 2 v + c j 2 ) , a j 1 , a j 2 , c j 1 , c j 2 ) , a j k , c j k ) − tanh ( a j k v + c j k ) . (135) Thus each target prediction depends only on v v v [ − 1 , 1 ] [-1,1] [ − 1 , 1 ]
L Q , P ( b ; e ) = max t ∈ P sup v ∈ [ − 1 , 1 ] ∣ h t ( v ) ∣ . (136) \mathcal{L}_{Q,P}(b;e)=\max_{t\in P}\sup_{v\in[-1,1]}|h_t(v)|. \tag*{(136)} L Q , P ( b ; e ) = t ∈ P max v ∈ [ − 1 , 1 ] sup ∣ h t ( v ) ∣. (136) The supremum commutes with the finite maximum: each target’s extremum can be attained at a state of its own. Agreement between the network forward pass and its scalar form checks this reduction before the domain audit.
We bound prediction sensitivity in the full input-observation norm using the trained weights. Let W 1 , … , W 6 W_1,\ldots,W_6 W 1 , … , W 6 encoder and predictor matrices, w 1 , w 2 w_1,w_2 w 1 , w 2 W 1 W_1 W 1 W 4 , h W_{4,h} W 4 , h predictor matrix. Using the GELU derivative bound 1.13, a valid constant is
K = 1.13 4 ∥ W 6 ∥ 2 ∥ W 5 ∥ 2 ∥ W 4 , h ∥ 2 ⋅ ∥ W 3 ∥ 2 ∥ W 2 ∥ 2 max σ ∈ { − 1 , 1 } ∥ w 1 + σ w 2 ∥ 2 . (137) K=1.13^{4}\lVert W_6\rVert_2\lVert W_5\rVert_2\lVert W_{4,h}\rVert_2\cdot\lVert W_3\rVert_2\lVert W_2\rVert_2\max_{\sigma\in\{-1,1\}}\lVert w_1+\sigma w_2\rVert_2. \tag*{(137)} K = 1.1 3 4 ∥ W 6 ∥ 2 ∥ W 5 ∥ 2 ∥ W 4 , h ∥ 2 ⋅ ∥ W 3 ∥ 2 ∥ W 2 ∥ 2 σ ∈ { − 1 , 1 } max ∥ w 1 + σ w 2 ∥ 2 . (137) Every varying input observation enters this calculation. The encoder–predictor composition therefore satisfies assumption 3.9 with ρ ( t ) = K t \rho(t)=Kt ρ ( t ) = K t H t = K max ( a j 1 , a j 2 ) + a j k H_t=K\max(a_{j1},a_{j2})+a_{jk} H t = K max ( a j 1 , a j 2 ) + a j k
On a regular grid v ℓ = − 1 + 2 ℓ / N v_\ell=-1+2\ell/N v ℓ = − 1 + 2 ℓ / N M t = max 0 ≤ ℓ ≤ N ∣ h t ( v ℓ ) ∣ M_t=\max_{0\le\ell\le N}|h_t(v_\ell)| M t = max 0 ≤ ℓ ≤ N ∣ h t ( v ℓ ) ∣ 1 / N 1/N 1/ N
max t ∈ P M t ≤ L Q , P ( b ; e ) ≤ max t ∈ P ( M t + H t N ) . (138) \max_{t\in P}M_t\le\mathcal{L}_{Q,P}(b;e)\le\max_{t\in P}\left(M_t+\frac{H_t}{N}\right). \tag*{(138)} t ∈ P max M t ≤ L Q , P ( b ; e ) ≤ t ∈ P max ( M t + N H t ) . (138) For the reported numerical values, let M ~ t \widetilde{M}_t M t 10 − 6 10^{-6} 1 0 − 6
L ‾ = max { 0 , max t ∈ P M ~ t − 10 − 6 } , L ‾ = max t ∈ P ( M ~ t + H t / N ) + 10 − 6 . (139) \underline{L}=\max\left\{0,\max_{t\in P}\widetilde{M}_t-10^{-6}\right\},\qquad\overline{L}=\max_{t\in P}\left(\widetilde{M}_t+H_t/N\right)+10^{-6}. \tag*{(139)} L = max { 0 , t ∈ P max M t − 1 0 − 6 } , L = t ∈ P max ( M t + H t / N ) + 1 0 − 6 . (139) The numerical enclosure assumes that 10 − 6 10^{-6} 1 0 − 6 H t H_t H t α P > 0 \alpha_P>0 α P > 0 Table 9 records the allowance and checks.
The grid is an offline domain audit of the fixed predictor . Its resolution is determined by the weight bound and a preset interpolation tolerance. Under the arithmetic conditions above, the ideal GT map in equation (31) obeys
sup f ∈ F d U ( f ^ ( ( b ∘ e ) ( f ( Q ) ) ) , f ) ≤ 2 L ‾ α P . (140) \sup_{f\in\mathcal{F}}d_{\mathcal{U}}\left(\widehat{f}((b\circ e)(f(Q))),f\right)\le\frac{2\overline{L}}{\alpha_P}. \tag*{(140)} f ∈ F sup d U ( f (( b ∘ e ) ( f ( Q ))) , f ) ≤ α P 2 L . (140) where L ‾ \overline{L} L F \mathcal{F} F ξ \xi ξ ξ / α P \xi/\alpha_P ξ / α P (134) .
The audit includes all repetitions for the largest sphere and torus at budgets above their sufficient sampling thresholds, and for the independent family at its full budget. Each bound uses the final training checkpoint and its training target set. All reported ideal-map bounds improve on the full-profile diameter (Table 1 ).
Joint-Query Calibration
This joint-query family makes the mechanism in theorem 3.2 explicit: target separation converts prediction error into recovery of the full state. Its linear inverse link lets records at known latent coordinates control global prediction error directly.
For the calibration certificate below, each query depends on the whole latent configuration. For x ∈ S r − 1 x\in S^{r-1} x ∈ S r − 1 f x ( q ) = tanh ( a q T x ) f_x(q)=\tanh(a_q^{\mathsf{T}}x) f x ( q ) = tanh ( a q T x ) A Q A_Q A Q A P A_P A P A λ = A P diag ( 1 , … , 1 , λ ) A_\lambda=A_P\operatorname{diag}(1,\ldots,1,\lambda) A λ = A P diag ( 1 , … , 1 , λ ) 0 ≤ λ ≤ 1 , 0\le\lambda\le1, 0 ≤ λ ≤ 1 ,
Consider an encoder applying the inverse response link and a matrix, followed by a joint predictor :
( b ∘ e ) ( f x ( Q ) ) = tanh ( W P W E arctanh ( f x ( Q ) ) ) . (141) (b \circ e)(f_x(Q)) = \tanh(W_P W_E \operatorname{arctanh}(f_x(Q))). \tag*{(141)} ( b ∘ e ) ( f x ( Q )) = tanh ( W P W E arctanh ( f x ( Q ))) . (141) Inputs are clipped to the valid observation range before applying the inverse link. The resulting full-input prediction map is Lipschitz with constant at most cosh 2 ( 1 ) ∥ W P W E ∥ ∞ → ∞ \cosh^2(1)\lVert W_P W_E\rVert_{\infty\to\infty} cosh 2 ( 1 ) ∥ W P W E ∥ ∞ → ∞
We obtain prediction, observability and recovery bounds from the same calibration records below. Throughout this section, σ min ( A ) = inf ∥ v ∥ 2 = 1 ∥ A v ∥ 2 \sigma_{\min}(A) = \inf_{\lVert v\rVert_2=1} \lVert Av\rVert_2 σ m i n ( A ) = inf ∥ v ∥ 2 = 1 ∥ A v ∥ 2
From Training Residuals to a Global Bound
Calibration records connect prediction error to recovery in three steps: they bound the global prediction error, certify that the targets distinguish states, and determine a recovery map.
For the joint-query family, set A P = A λ A_P = A_\lambda A P = A λ M = W P W E M = W_P W_E M = W P W E X ∈ R n × r X \in\mathbb{R}^{n \times r} X ∈ R n × r σ = σ min ( X ) > 0 \sigma= \sigma_{\min}(X) > 0 σ = σ m i n ( X ) > 0 U ∈ R n × ∣ Q ∣ U \in\mathbb{R}^{n \times|Q|} U ∈ R n × ∣ Q ∣ V ∈ R n × ∣ P ∣ V \in\mathbb{R}^{n \times|P|} V ∈ R n × ∣ P ∣ X A Q T X A_Q^{\mathsf{T}} X A Q T X A P T X A_P^{\mathsf{T}} X A P T η Q \eta_Q η Q η P \eta_P η P
Lemma G.1 (Calibration Error). Under the calibration setup, let Y ∈ R n × k Y \in\mathbb{R}^{n \times k} Y ∈ R n × k B ∈ R r × k B \in\mathbb{R}^{r \times k} B ∈ R r × k ∣ ( Y − X B ) i j ∣ ≤ η |(Y-XB)_{ij}| \le\eta ∣ ( Y − X B ) ij ∣ ≤ η
max j ∥ ( X † Y − B ) ⋅ j ∥ 2 ≤ n η σ , ∥ X † Y − B ∥ 2 → 2 ≤ n k η σ . (142) \max_j \lVert(X^\dagger Y-B)_{\cdot j}\rVert_2 \le\frac{\sqrt{n}\eta}{\sigma}, \qquad\lVert X^\dagger Y-B\rVert_{2 \to2} \le\frac{\sqrt{nk}\eta}{\sigma}. \tag*{(142)} j max ∥( X † Y − B ) ⋅ j ∥ 2 ≤ σ n η , ∥ X † Y − B ∥ 2 → 2 ≤ σ nk η . (142) Proof. Since X † X = I X^\dagger X = I X † X = I
X † Y − B = X † ( Y − X B ) . (143) X^\dagger Y-B = X^\dagger(Y-XB). \tag*{(143)} X † Y − B = X † ( Y − X B ) . (143) Each column of Y − X B Y-XB Y − X B n η \sqrt{n}\eta n η n k η \sqrt{nk}\eta nk η X † X^\dagger X † 1 / σ 1/\sigma 1/ σ
Lemma G.2 (Calibrated Prediction). Define the residual and its global error bound by
R : = U M T − V , (144) R := UM^{\mathsf{T}}-V, \tag*{(144)} R := U M T − V , (144) Δ : = max p ∈ P ∥ X † R ⋅ p ∥ 2 + n σ ( ∥ M ∥ ∞ → ∞ η Q + η P ) . (145) \Delta:= \max_{p \in P}\lVert X^\dagger R_{\cdot p}\rVert_2+\frac{\sqrt{n}}{\sigma}\left(\lVert M\rVert_{\infty\to\infty}\eta_Q+\eta_P\right). \tag*{(145)} Δ := p ∈ P max ∥ X † R ⋅ p ∥ 2 + σ n ( ∥ M ∥ ∞ → ∞ η Q + η P ) . (145) Then
L Q , P ( b ; e ) ≤ max p ∈ P ∥ ( M A Q − A P ) p ∥ 2 ≤ Δ . (146) L_{Q,P}(b;e) \le\max_{p \in P}\lVert(MA_Q-A_P)_p\rVert_2 \le\Delta. \tag*{(146)} L Q , P ( b ; e ) ≤ p ∈ P max ∥( M A Q − A P ) p ∥ 2 ≤ Δ. (146) For exact records, η Q = η P = 0 \eta_Q=\eta_P=0 η Q = η P = 0
Proof. The residual satisfies
R − X ( M A Q − A P ) T = ( U − X A Q T ) M T − ( V − X A P T ) . (147) R-X(MA_Q-A_P)^{\mathsf T}=(U-XA_Q^{\mathsf T})M^{\mathsf T}-(V-XA_P^{\mathsf T}). \tag*{(147)} R − X ( M A Q − A P ) T = ( U − X A Q T ) M T − ( V − X A P T ) . (147) Its entries have magnitude at most ∥ M ∥ ∞ → ∞ η Q + η P \lVert M\rVert_{\infty\to\infty}\eta_Q+\eta_P ∥ M ∥ ∞ → ∞ η Q + η P Y = R Y=R Y = R B = ( M A Q − A P ) T B=(MA_Q-A_P)^{\mathsf T} B = ( M A Q − A P ) T Δ \Delta Δ tanh \tanh tanh R = X ( M A Q − A P ) T R=X(MA_Q-A_P)^{\mathsf T} R = X ( M A Q − A P ) T X † R = ( M A Q − A P ) T X^{\dagger}R=(MA_Q-A_P)^{\mathsf T} X † R = ( M A Q − A P ) T
The same records estimate the target matrix. Its smallest singular value measures whether the targets retain every latent direction.
Lemma G.3 (Calibrated Observability). Define
A ^ P : = ( X † V ) T , τ : = n ∣ P ∣ η P σ , (148) \widehat{A}_P:=(X^{\dagger}V)^{\mathsf T},\qquad\tau:=\frac{\sqrt{n|P|}\eta_P}{\sigma}, \tag*{(148)} A P := ( X † V ) T , τ := σ n ∣ P ∣ η P , (148) α ^ P : = sech 2 ( 1 ) ∣ P ∣ ( σ min ( A ^ P ) − τ ) + . (149) \widehat{\alpha}_P:=\frac{\operatorname{sech}^{2}(1)}{\sqrt{|P|}}\left(\sigma_{\min}(\widehat{A}_P)-\tau\right)_{+}. \tag*{(149)} α P := ∣ P ∣ sech 2 ( 1 ) ( σ m i n ( A P ) − τ ) + . (149) Then ∥ A ^ P − A P ∥ 2 → 2 ≤ τ \lVert\widehat{A}_P-A_P\rVert_{2\to2}\le\tau ∥ A P − A P ∥ 2 → 2 ≤ τ and
d P ( f , g ) ≥ α ^ P d U ( f , g ) , f , g ∈ F . (150) d_P(f,g)\ge\widehat{\alpha}_P d_U(f,g),\qquad f,g\in\mathcal{F}. \tag*{(150)} d P ( f , g ) ≥ α P d U ( f , g ) , f , g ∈ F . (150) Thus α ^ P > 0 \widehat{\alpha}_P>0 α P > 0 A ^ P = A P \widehat{A}_P=A_P A P = A P
Proof. Apply lemma G.1 with Y = V Y=V Y = V B = A P T B=A_P^{\mathsf T} B = A P T η = η P \eta=\eta_P η = η P ∥ A ^ P − A P ∥ 2 → 2 ≤ τ \lVert\widehat{A}_P-A_P\rVert_{2\to2}\le\tau ∥ A P − A P ∥ 2 → 2 ≤ τ
σ min ( A P ) ≥ ( σ min ( A ^ P ) − τ ) + . (151) \sigma_{\min}(A_P)\ge\left(\sigma_{\min}(\widehat{A}_P)-\tau\right)_{+}. \tag*{(151)} σ m i n ( A P ) ≥ ( σ m i n ( A P ) − τ ) + . (151) All query rows have norm at most one, so d U ( f x , f x ′ ) ≤ ∥ x − x ′ ∥ 2 d_U(f_x,f_{x'})\le\lVert x-x'\rVert_2 d U ( f x , f x ′ ) ≤ ∥ x − x ′ ∥ 2 x , x ′ x,x' x , x ′ tanh \tanh tanh [ − 1 , 1 ] [-1,1] [ − 1 , 1 ] sech 2 ( 1 ) \operatorname{sech}^{2}(1) sech 2 ( 1 )
d P ( f x , f x ′ ) ≥ sech 2 ( 1 ) ∥ A P ( x − x ′ ) ∥ ∞ ≥ sech 2 ( 1 ) σ min ( A P ) ∣ P ∣ ∥ x − x ′ ∥ 2 ≥ α ^ P d U ( f x , f x ′ ) . (152) \begin{aligned}
d_P(f_x,f_{x'})&\ge\operatorname{sech}^{2}(1)\lVert A_P(x-x')\rVert_{\infty}\\
&\ge\frac{\operatorname{sech}^{2}(1)\sigma_{\min}(A_P)}{\sqrt{|P|}}\lVert x-x'\rVert_2
\ge\widehat{\alpha}_P d_U(f_x,f_{x'}). \tag*{(152)}
\end{aligned} d P ( f x , f x ′ ) ≥ sech 2 ( 1 ) ∥ A P ( x − x ′ ) ∥ ∞ ≥ ∣ P ∣ sech 2 ( 1 ) σ m i n ( A P ) ∥ x − x ′ ∥ 2 ≥ α P d U ( f x , f x ′ ) . (152) This proves equation (150). For exact target records, V = X A P T V=XA_P^{\mathsf T} V = X A P T A ^ P = A P \widehat{A}_P=A_P A P = A P
When this certificate is positive, the calibrated inverse estimates the latent vector. Normalization returns that estimate to the sphere.
Proposition G.4 (Calibrated Recovery). Use Δ \Delta Δ A ^ P \widehat{A}_P A P τ \tau τ σ min ( A ^ P ) > τ \sigma_{\min}(\widehat{A}_P)>\tau σ m i n ( A P ) > τ ν \nu ν
c ( y ) : = arctanh ( clip [ − tanh ( 1 ) , tanh ( 1 ) ] ( y ) ) , (153) c(y):=\operatorname{arctanh}\left(\operatorname{clip}_{[-\tanh(1),\tanh(1)]}(y)\right), \tag*{(153)} c ( y ) := arctanh ( clip [ − t a n h ( 1 ) , t a n h ( 1 )] ( y ) ) , (153) x ^ ( y ) : = ν ( A ^ P † c ( y ) ) . (154) \widehat{x}(y):=\nu\left(\widehat{A}_P^{\dagger}c(y)\right). \tag*{(154)} x ( y ) := ν ( A P † c ( y ) ) . (154) This readout satisfies
sup x ∈ S r − 1 d U ( f x ^ ( ( b ∘ e ) ( f x ( Q ) ) ) , f x ) ≤ 2 ( ∣ P ∣ Δ + τ ) σ min ( A ^ P ) . (155) \sup_{x\in\mathbb{S}^{r-1}}d_U\left(f_{\widehat{x}((b\circ e)(f_x(Q)))},f_x\right)\le\frac{2\left(\sqrt{|P|}\Delta+\tau\right)}{\sigma_{\min}(\widehat{A}_P)}. \tag*{(155)} x ∈ S r − 1 sup d U ( f x (( b ∘ e ) ( f x ( Q ))) , f x ) ≤ σ m i n ( A P ) 2 ( ∣ P ∣ Δ + τ ) . (155) Proof. Fix a unit x x x y = ( b ∘ e ) ( f x ( Q ) ) = tanh ( M A Q x ) y=(b\circ e)(f_x(Q))=\tanh(MA_Qx) y = ( b ∘ e ) ( f x ( Q )) = tanh ( M A Q x ) c ( y ) = clip [ − 1 , 1 ] ( M A Q x ) c(y)=\operatorname{clip}_{[-1,1]}(MA_Qx) c ( y ) = clip [ − 1 , 1 ] ( M A Q x ) A P x ∈ [ − 1 , 1 ] ∣ P ∣ A_Px\in[-1,1]^{|P|} A P x ∈ [ − 1 , 1 ] ∣ P ∣
∥ c ( y ) − A ^ P x ∥ 2 ≤ ∥ c ( y ) − A P x ∥ 2 + ∥ ( A P − A ^ P ) x ∥ 2 ≤ ∣ P ∣ Δ + τ . (156) \lVert c(y)-\widehat{A}_Px\rVert_2\le\lVert c(y)-A_Px\rVert_2+\lVert(A_P-\widehat{A}_P)x\rVert_2\le\sqrt{|P|}\Delta+\tau. \tag*{(156)} ∥ c ( y ) − A P x ∥ 2 ≤ ∥ c ( y ) − A P x ∥ 2 + ∥( A P − A P ) x ∥ 2 ≤ ∣ P ∣ Δ + τ . (156) For a unit x x x v v v
∥ ν ( v ) − x ∥ 2 ≤ ∥ ν ( v ) − v ∥ 2 + ∥ v − x ∥ 2 ≤ 2 ∥ v − x ∥ 2 . (157) \lVert\nu(v)-x\rVert_2\le\lVert\nu(v)-v\rVert_2+\lVert v-x\rVert_2\le2\lVert v-x\rVert_2. \tag*{(157)} ∥ ν ( v ) − x ∥ 2 ≤ ∥ ν ( v ) − v ∥ 2 + ∥ v − x ∥ 2 ≤ 2 ∥ v − x ∥ 2 . (157) Figure 14 (PDF p. 40).
Since A ^ P \widehat{A}_P A P
d U ( f x ^ ( y ) , f x ) ≤ 2 ∥ A ^ P C ( y ) − x ∥ 2 ≤ 2 ∥ c ( y ) − A ^ P x ∥ 2 σ min ( A ^ P ) ≤ 2 ( ∣ P ∣ Δ + τ ) σ min ( A ^ P ) . (158) d_U(f_{\widehat{x}(y)}, f_x) \le2\lVert\widehat{A}_{P_C}(y)-x\rVert_2 \le\frac{2\lVert c(y)-\widehat{A}_P x\rVert_2}{\sigma_{\min}(\widehat{A}_P)} \le\frac{2(\sqrt{|P|}\Delta+\tau)}{\sigma_{\min}(\widehat{A}_P)}. \tag*{(158)} d U ( f x ( y ) , f x ) ≤ 2 ∥ A P C ( y ) − x ∥ 2 ≤ σ m i n ( A P ) 2 ∥ c ( y ) − A P x ∥ 2 ≤ σ m i n ( A P ) 2 ( ∣ P ∣ Δ + τ ) . (158) Taking the supremum over x x x
Observation Geometry
We examine how many representative observations cover a later diagnostic block, using the same geometry protocol and time-shift controls. Covariance statistics provide a complementary description. Prediction and model-state recovery use this cohort in § 15 and J.
Data access and reuse. The recordings were contributed by Mizuseki, Sirota, Pastalkova, Diba, and Buzsáki and distributed as HC-3 through CRCNS.org (Mizuseki et al., 2013; 2014). The provider makes the data available for scientific use under the CRCNS data-use terms, which require attribution and citation and restrict redistribution. The HC-3 dataset page identifies the source files and access procedure. Our analysis reuses these existing animal recordings under that published provision for scientific use. The reproduction archive supplies the session manifest and preprocessing code; readers obtain recordings directly from the provider. Original code and aggregate outputs are supplied for confidential peer-review reproduction, and dependencies retain their upstream licenses, listed in the archive.
Session and Observation
The HC-3 cohort (Mizuseki et al., 2014) spans open-field, linear-track, and wheel recordings (Table 10 ). We select sessions using metadata to balance data availability with breadth across recording groups. Eligible sessions have aligned position, recording metadata, and spike files. We allocate sessions across recording prefixes, cycling through the groups within each prefix. Within a group, priority follows processing availability, matched shanks, estimated unit counts, and recording length. This procedure supplies varied recording contexts for comparing observation geometry.
Table 10. HC-3 study cohort relative to the complete local catalog, with session counts by animal. The ec015 recording prefix belongs to ec014.
Every included session meets the unit-quality criteria in Table 11 . Electrode metadata identifies recording prefixes belonging to the same animal. All analyses use this fixed cohort; detailed session statistics appear in Table 16 .
Table 11. Observation construction and time blocks. Block percentages refer to recording duration.
Table 16: Complete primary-window results. Effective dimension and shift reduction refer to the diagnostic block. The 90% rank uses directions from the reference basis.
Table 16.
Each query specifies a unit’s spike count over a trailing window. At each sampled observation time, the window ends at that time; evaluating the fixed collection of queries gives the corresponding observation vector. Each query returns the count, logarithmically transformed and scaled using the earlier training block (Table 11 ). This normalization stays fixed throughout the recording. Each window lies fully inside its assigned block, with gaps at the boundaries. We derive geometric coordinates from a reference block, test their transfer to a later diagnostic block, and reserve the final block for subsequent experiments. Shorter and longer counting windows test sensitivity to temporal resolution. The calculations operate directly on recorded observations.
Covering Geometry
For each recording block, let X X X N ^ X ( ε ) \widehat{N}_X(\varepsilon) N X ( ε ) N ‾ X ( ε ) \underline{N}_X(\varepsilon) N X ( ε ) N ‾ X ( ε ) ≤ N X ( ε ) ≤ N ^ X ( ε ) \underline{N}_X(\varepsilon) \le N_X(\varepsilon) \le\widehat{N}_X(\varepsilon) N X ( ε ) ≤ N X ( ε ) ≤ N X ( ε ) X X X 2 ε 2\varepsilon 2 ε
The reference block fixes the distance scale. Real and shifted diagnostic observations use identical sample indices, sample sizes, and radii. The main comparison uses the grid radius nearest a prespecified fraction of the reference diameter. Curve bands show session interquartile ranges; control bars span the time-shift controls.
The fixed-resolution comparison shows that recorded population observations admit smaller constructed covers than time-shift controls. The animal and session summaries use the same radius and retain the entire cohort. These are covering statistics for the recorded observations in the specified maximum-coordinate metric.
Computational cost. For n n n u u u O ( n 2 u ) O(n^{2}u) O ( n 2 u ) O ( n 2 ) O(n^{2}) O ( n 2 ) O ( n 2 ) O(n^{2}) O ( n 2 ) O ( n u 2 + u 3 ) O(n u^{2} + u^{3}) O ( n u 2 + u 3 ) O ( u 2 ) O(u^{2}) O ( u 2 ) Table 12 and Table 15 .
Table 12. Covering protocol, using the observations and chronological blocks in table 11.
Table 15. Covariance geometry settings. All use the same diagnostic block.
Table 14. Complete primary-window covering results at the fixed radius. The ratio divides the real greedy-cover size by its median time-shift cover size.
Covariance Geometry
We compute the covariance spectrum and participation-ratio effective dimension in each block. Reference covariance eigenvectors define the projection applied to the later diagnostic block. The number of directions needed to retain a given fraction of variance measures covariance concentration; the covering exponent in equation (13) controls how many representatives are needed to approximate every state at a given accuracy.
The controls independently circularly shift each unit’s observation sequence, preserving its marginal distribution and circular autocorrelation while changing alignment between units. The comparison reflects shared temporal modulation and nonstationarity as well as coordination between units. We compare all controls within each session and assess uncertainty by resampling contiguous time blocks. Complete session results and window sensitivity appear in Table 16 and Table 17 .
Table 17: Window sensitivity. Entries are session medians in the diagnostic block; reduction is relative to the median shift control. Every eligible session lies below every shift control at each window length.
Table 17.
The reference subspace captures substantial variance in the later block; the covariance summaries retain the complete cohort and window comparisons.
HC-3 Prediction Details
Matched prediction task.
We use the same metadata-selected session panel and preprocessing as section H. Units are partitioned into input queries Q Q Q V V V encoder compresses the input observations into learned tokens, and a query-conditioned predictor estimates target observations. All parameters are trained jointly from scratch. The full design and fixed endpoint are listed in table 18.
Training choices. The reference architecture and 200-epoch endpoint were fixed after preliminary training checks and then shared by every session, target budget, and repetition. The small model reduces depth, width, and learned-token count while retaining the same observation task. These architecture settings are design choices, not quantities selected separately on the final test block. AdamW uses weight decay 10 − 4 10^{-4} 1 0 − 4 ( .9 , .999 ) (.9, .999) ( .9 , .999 ) 10 − 8 10^{-8} 1 0 − 8
Attention cost. Let q = ∣ Q ∣ q = \lvert Q\rvert q = ∣ Q ∣ p = ∣ P ∣ p = \lvert P\rvert p = ∣ P ∣ t t t h h h O ( ( q + p + t ) h 2 + ( t 2 + t q + p 2 + p t ) h ) O((q + p + t)h^2 + (t^2 + tq + p^2 + pt)h) O (( q + p + t ) h 2 + ( t 2 + tq + p 2 + pt ) h ) O ( t 2 + t q + p 2 + p t ) O(t^2 + tq + p^2 + pt) O ( t 2 + tq + p 2 + pt ) h h h
Prediction score. The prediction score averages the maximum absolute target-observation error across test windows. The constant baseline is the coordinatewise training median on the same target set. Relative gain is one minus the ratio of model error to baseline error. We average seed-level gains within each session, then weight sessions equally. Session-to-session spread is descriptive because recordings within an animal share biological variation. Different budgets evaluate different target sets, each with its own matched baseline.
Local State Recovery
Reference state. The ontic-state surrogate is the large encoder ’s complete post-normalization token state, flattened into a reference vector. For each session and seed, the full-target large model is held fixed. The smaller encoder–predictor learns the same input-to-target task from scratch, with identical input and target queries, preprocessing, training observations, and chronological validation and test blocks. Its training objective uses target observations. After predictive training, the reference states determine the recovery maps and their evaluation. We compare the final small encoder with its exact saved initialization on the same held-out inputs.
Local recovery. For a held-out input, we locate the nearest neighbors of its learned state in the small encoder ’s training-state library and average their reference states. Distances use Euclidean coordinates, centered by the training mean and divided by a common training root-mean-square scale. This scalar normalization preserves neighbor order. A neighbor’s weight is proportional to the exponential of its negative squared distance divided by the squared distance of the furthest selected neighbor, with a numerical floor. At test time, these training neighbors and their distance-based weights directly determine the recovered reference state.
Table 20: Matched prediction models. The common optimization schedule and chronological split are given in Table 18 . Recovery uses the full target pool, with 25 sessions and 3 paired seeds. Parameter counts include the encoder and predictor .
Table 20.
Table 18: HC-3 prediction protocol. Percentages denote within-session chronological blocks or unit partitions. Normalization uses the earlier training portion specified in the geometry protocol. All configurations use the fixed final epoch.
Table 18.
Selection and cost. The neighborhood grid spans individual neighbors through broader local averages. The count minimizing validation mean squared error is selected separately for the trained and initial representations; ties retain the first count in ascending order. Evaluation uses the fixed final test block. For a library of n n n s s s B B B O ( B n s ) O(Bns) O ( B n s ) k k k r r r O ( B k r ) O(Bkr) O ( B k r ) O ( 128 n ) O(128n) O ( 128 n )
Recovery score. We score all reference coordinates together using squared reconstruction error divided by the squared error of the training-mean reference state; the reported R 2 R^2 R 2
