Finite-Query Coherent Projection of Black-Box Conditional Predictions

Introduction

Black-box conditional predictors provide local probabilistic answers that may not agree with any single joint distribution. For example, one binary conditional table can make $X_1$ independent of $X_2$, while another makes the distribution of $X_2$ depend on $X_1$; these cannot describe the same positive joint law. A coherent joint supplies a common source for conditional and event probabilities used in combined reasoning.

Compatibility, joint construction, and compromises between incompatible conditionals have a substantial statistical literature [1, 2, 3, 4]. Probabilistic knowledge-base repair also studies consistency-restoring adjustments [5, 6]. We build on these questions rather than propose joint construction or consistency repair as new objectives.

Coherent reconciliation raises two questions: how accurately a specified projection can be estimated from limited access, and which conditional directions that projection fits well. A learner observes black-box probability answers and returns a joint in a chosen student class. Its conditional-Kullback–Leibler (KL) risk is evaluated against the same teacher, under a specified distribution over conditional queries. Figure 1 shows this projection task. Comparing with the best member of the student class separates finite-access error from disagreement that remains even at the projection optimum (§ 3).

Exact probability answers remove label-sampling noise at observed keys, but leave unobserved entries of an arbitrary table unknown. The first question is therefore whether, and at what risk, a limited transcript determines the best coherent projection. We study this question for a local pairwise joint class, whose undirected edge parameters are shared across conditional directions.

A matching lower bound must respect this design and compare with the optimum of the same realized teacher table.

Even an accurately computed projection need not fit every conditional direction better: its objective determines which discrepancies are penalized. We examine this second question in a computable small-joint setting that preserves direct black-box marginals. Frozen model predictions allow us to assess whether fitting one conditional direction also improves the other. The case study thus complements the finite-access analysis by examining directional fidelity under a chosen projection objective; its output class and observation protocol are different.

Contributions.

1. Finite-access characterization. We establish matching upper and lower bounds for the expected excess conditional-KL risk of projecting arbitrary deterministic binary conditional tables onto a local bounded pairwise joint class. Under uniform passive exact-table queries, the result characterizes the minimax dependence on budget, dimension, and student radius over the stated window. The main technical contribution is a matching lower-bound construction that jointly respects the legal shared-edge design, frozen exact-table responses, and comparison with the best projection of the same realized table.

2. Directional trade-offs under fixed marginals. In a preregistered frozen TabPFN-v2 study, we examine whether fitting one conditional direction improves the opposite direction while preserving direct marginals. A confirmatory Car anchor and six external dataset–split units provide independent numerical checks and reveal dataset-dependent, asymmetric transfer: improvement in the fitted objective does not ensure improvement in the omitted direction.

Supporting coverage, rank, and fixed-design results clarify these claims.

An independent restricted response-adaptive conditional-table task is retained in Appendices H–K. Its output is not a joint, and its real-structure weighting study includes a dataset-level reversal; it is not evidence for active coherent reconstruction.

§ 2 compares the nearest compatibility and statistical estimation results. § 3 defines the projection risk and observation protocols. § 4 states the full passive characterization and its limits. § 5 gives the hard-marginal formulation, and § 6 reports the frozen case study. § 7 discusses the scope before the conclusion.

Related Work

Two bodies of work are especially close: constructing coherent compromises from conditionals, and estimating a best-in-class model with limited information. Table 1 compares selected results by observation, output, and target; a broader comparison is retained in Appendix K.

Work	Observation and target	Relevant distinction
Arnold et al.; Kuo et al. [2, 3]	Discrete conditional specifications; compatible and near-compatible joints	Joint compatibility and compromise, versus finite-transcript excess risk.
Picado Muiño; Potyka–Thimm [5, 6]	Explicit probabilistic statements; repair and generalized models	Repair and generalized reasoning, versus partial-access estimation of the same-table projection.
Torroba Hennigen–Kim [15]	Masked-LM conditionals; joint construction by conditional closeness	Conditional-closeness objective; here the student class and access protocol are fixed.
Dereziński et al. [19]	Hidden fixed responses; selectable least-squares design	Squared loss and selectable design, versus shared-edge logistic projection.
Mourtada–Gaïffas [20]	Labels; misspecified log loss with improper output	Hard labels and improper output, versus exact probabilities and bounded proper output.
This work	Arbitrary binary tables; uniform iid exact responses; pairwise joint output	Matching same-table projection excess risk within the stated radius and budget window.

Compatibility and probabilistic repair. Classical conditional specification studies existence, uniqueness, and construction of joint laws [1, 4]. Arnold et al. [2] formulate finite discrete exact, near, and $\epsilon$-compatibility, while Chen [7] uses odds-ratio representations. Kuo et al. [3] introduce structural ratio matrices for compatible and almost-compatible high-dimensional discrete conditional models. These are direct precedents for joint construction, including the incompatible case; high dimension alone does not distinguish our objective.

Compromise criteria also predate this work. Gibbs ensembles [8] and random-scan pseudo-Gibbs compromises [9] treat inconsistent conditional specifications. Dependency networks [10] allow local inconsistency, and subset-conditional consistency is studied by Majid et al. [11]. Picado Muiño [5], Thimm [12], and Potyka and Thimm [13, 6] develop probabilistic inconsistency measurement, consolidation, and reasoning with generalized models. Divergence-based recovery and merging also appear in probabilistic knowledge-based systems [14].

Conditional-to-joint construction and conditional-closeness objectives also occur for masked language models [15]. Conditional probability queries can recover low-rank Bayesian-network structure under appropriate compatibility assumptions [16]. Tractable probabilistic-circuit distillation [17] addresses another student capability, while coordinate-conditional sampling can support identity testing [18]. These query units, outputs, and assumptions are not interchangeable with an exact frozen-table projection risk.

Finite information and best-in-class estimation. Dereziński et al. [19] study limited-response experimental design and least-squares estimation, including arbitrary fixed responses. Their selectable designs, squared-loss objective, and estimator criteria differ from our uniform passive legal rows and bounded proper logistic output.

Mourtada and Gaïffas [20] give an improper estimator and excess-risk bounds for misspecified density estimation and logistic regression. Their hard-label observation model and potentially out-of-class predictor differ from our exact probability responses and required coherent student. Loss similarities do not make the experiments equivalent: probability responses can generate labels, so applicable label-based upper bounds may transfer, but a hard-label lower bound does not automatically lower-bound the more informative probability oracle.

Moment and curvature arguments give our upper bound; stochastic averaging offers another route [21]. Matching concerns the shared-edge design, frozen observations, proper bounded output, and realized-table comparator.

Constrained fitting and the TFM case study. Information projection [22], Bregman methods [23, 24], and constrained marginal fitting [25] provide precedents for our hard-marginal formulation. § 5 specifies its forward conditional KL, cell weights, lower bounds, and numerical checks.

TabPFN [26] supplies the frozen case-study predictions. TFM consistency diagnostics [27] motivate the question, while the heterogeneity of tabular learning [28, 29] cautions against generalizing from one checkpoint.

Problem Formulation

Let $X=(X_1,\ldots,X_d)$ be finite categorical variables, with $X_j\in\mathcal{X}_j$ and product alphabet $\mathcal{X}:=\prod_{j=1}^{d}\mathcal{X}_j$. Write $x\in\mathcal{X}$ for a complete realization. For an index set $T\subseteq[d]:=\{1,\ldots,d\}$, $x_T$ denotes the coordinates indexed by $T$, and $-T:=[d]\setminus T$ (in particular, $-j=[d]\setminus\{j\}$). A black-box oracle $\mathcal{O}$ maps a query key $z=(j,S,\bar{x}_S)$, with $S\subseteq-j$, to a categorical probability vector $\tilde{p}_j(\cdot\mid\bar{x}_S)$ on $\mathcal{X}_j$, where $\bar{x}_S\in\prod_{k\in S}\mathcal{X}_k$ is the fixed conditioning assignment. The responses need not arise from a common joint distribution. We write $\sigma(t)=(1+\exp(-t))^{-1}$ for the logistic link used below, and $\operatorname{Ber}(\pi)$ for the Bernoulli distribution on $\{0,1\}$ with success probability $\pi$. We also write

for the strictly positive probability simplex. Each $q\in\Delta_+(\mathcal{X})$ induces conditionals by marginalization:

Both sums range over complete configurations; positivity makes the denominator nonzero. Global coherence means that every reported student conditional is induced by this same normalized joint.

For probability vectors $p$ and $r$ on the same finite alphabet, define the forward Kullback–Leibler divergence [30] by

with $0\log(0/r):=0$. All student probabilities below are positive, so no division by zero occurs.

For a fixed evaluation distribution $\mu$ over query keys, we measure teacher-to-student conditional disagreement by

Fixing $\mu$ is part of the task definition: a best coherent joint is not well-defined without an evaluation distribution. For a coherent student class $Q \subseteq\Delta_+(\mathcal{X})$ and a budget-$B$ output $\widehat{q}_B \in Q$, introduce the two population risk floors

We then define

These definitions give the accounting identity

This telescoping identity is not an additional statistical guarantee. The terms are nonnegative: irreducible disagreement, class approximation, and finite-query or optimization excess, respectively. Only the last depends on the learner’s transcript and can be random; we abbreviate it as $E_{B,Q}$ when the other arguments are fixed, and take expectations after the pointwise decomposition.

Crucially, $I_\mu= 0$ concerns only positive-$\mu$ keys; global oracle compatibility additionally requires sufficient coverage and, when relevant, attainment by a positive joint. Also, $A_{\mu,Q}$ includes all student restrictions, such as hard marginals, not just pairwise structure. The restricted optimum must not be relabelled as pure incompatibility.

Notation used in the theory. For vectors $u,v \in\mathbb{R}^m$, $\langle u,v\rangle= u^\top v$ and $\lVert u\rVert_2 = \left(\sum_{r=1}^{m} u_r^2\right)^{1/2}$. The indicator of an event $E$ is $\mathbf{1}_{\{E\}}$. Vector inequalities are componentwise unless a matrix order is explicitly stated.

Query protocols and budget semantics. The oracle model and budget convention are part of every claim. The main theory uses exact deterministic conditional-probability tables. Its passive-i.i.d. protocol samples query keys independently from a stated design distribution; repeated occurrences of a key consume design slots but return the same frozen table entry. They are not independent noisy observations and do not entitle the learner to draw a replacement key. Quantized and noisy response models have distinct information budgets and are not silently identified with this exact-table protocol.

An adaptive policy chooses keys using its transcript, whereas a static policy fixes its allocation before observing responses. The main matching theorem uses neither adaptive selection nor a guaranteed-distinct budget. The supplementary adaptive construction (Appendix H) instead reports a non-joint conditional table and has a different loss. Its guarantee does not transfer to coherent reconstruction.

Table 2 identifies the complementary roles of finite-access precision and directional fidelity, together with their distinct protocols and the supplementary studies.

Task	Observation, target, and evidence
Passive theory	Arbitrary deterministic binary full-conditional table; uniform iid exact slots, repeats charged. Same-table best bounded pairwise projection; matching expected excess conditional KL within Theorem 1’s radius and budget window.
TFM case study	Frozen direct marginals and both conditional families; small hard-marginal joint tables. Independent numerical checks and within-unit directional risk contrasts, not passive-rate or predictive-accuracy validation.
Supplementary tasks	Quantized fixed-pool recovery and noisy non-joint table prediction have separate losses and budgets (Appendices H–K). Their outcomes are not pooled with either main task.

Passive Finite-Query Characterization

How accurately can exact local probability answers determine a coherent projection when only a fraction of the conditional table is observed? The main result gives matching upper and lower bounds for arbitrary deterministic binary teacher tables and a local bounded pairwise student class. Shared undirected parameters enforce coherence of the output; the teacher need not be compatible with any joint.

Coherent pairwise output

For $X \in\{-1,+1\}^{d}$, set $m := d + d(d-1)/2$ and write

The edge block is ordered lexicographically by $(j,k)$, and $w_{jk}=w_{kj}$ when referenced in either order. The positive pairwise joint is

where $\mathcal{N}(\theta)$ is the finite normalizing constant. At a full-conditional key $z=(j,x_{-j})$, define $a(z)\in\mathbb{R}^{m}$ to have entry $1$ at the $j$th intercept, entry $x_k$ at each edge $\{j,k\}$ incident to $j$, and zero elsewhere. Then

Thus one shared edge parameter is used in both conditional directions. Write $\Theta_R:=\{\theta\in\mathbb{R}^{m}:\|\theta\|_2\le R\}$ and $Q_R:=\{q_\theta:\theta\in\Theta_R\}$.

Uniform passive exact-table protocol

For clarity, let $\mathcal{Z}:=\{(j,x_{-j}):j\in[d],x_{-j}\in\{-1,+1\}^{d-1}\}$ and let $\mathcal{P}_{\mathrm{all}}$ be the set of all deterministic tables $p:\mathcal{Z}\to[0,1]$ on the $N=d2^{d-1}$ full-conditional keys. Values zero and one are allowed; the student conditionals remain strictly positive, so the KL risk is finite under the convention in Section 3. Throughout this subsection, $\mu$ is uniform on $\mathcal{Z}$. Under $B$ independent uniform draws from $\mathcal{Z}$, let $\mathcal{O}_p$ denote the oracle returning $p(z)$ and define

where $\mathcal{A}$ ranges over randomized learners whose output lies in $\Theta_R$, and the expectation is over the passive design and learner randomization. Set $\kappa_0:=\sigma(M_0)(1-\sigma(M_0))$ for a fixed $M_0>0$. The following statement includes the radius and budget restrictions; Appendix A supplies its complete proof.

The upper bound holds uniformly over $\mathcal{P}_{\mathrm{all}}$ (Proposition 4). For the lower bound, it suffices to use the subclass with entries in $\{1/4,3/4\}$, obtained by fixing $\gamma=1/4$ in Appendix A’s frozen-table construction. Inclusion of this subclass in $\mathcal{P}_{\mathrm{all}}$ gives the same lower bound for the full teacher class. Thus the theorem does not require teacher probabilities to be close to $1/2$; the local radius restriction concerns the student.

Why exact answers leave finite-access uncertainty

Exact responses remove label-sampling noise at an observed key, but do not reveal unobserved entries of an arbitrary table. The projection depends on a moment over all keys, so a limited passive transcript can leave its optimum uncertain. The lower bound quantifies this uncertainty within the shared-edge design, rather than importing a hard-label lower bound into a more informative observation model.

Write $A(t) = \log(1 + e^t)$, $G(\theta) = \mathbb{E}_{\mu}A(a(Z)^\top\theta)$, and $b_p = \mathbb{E}_{\mu}[p(Z)a(Z)]$. Up to a teacher-dependent constant, the population risk is $G(\theta) - b_p^\top\theta$. Replacing $b_p$ by its sample average yields the moment estimator in Eq. (A.5). Its upper bound follows from a second-moment estimate and logistic curvature (Proposition 4). These are standard estimation tools: projected stochastic subgradient averaging also gives the familiar strongly convex rate [21]. The contribution is the matching characterization for the specified projection experiment, rather than a new general optimizer.

The lower bound must additionally respect the fixed legal feature rows and the exact-response protocol. Its edge-code construction generates a deterministic table before observation; querying a repeated key reveals no new sign. Concentration of the realized table moment then connects code separation to the optimum of that actual table. Interior and boundary arguments, with a transition step, give the two regimes. The proof uses standard information lower-bound tools [31, 32]; its problem-specific work is the legal frozen-table construction and the realized-table comparison.

The estimator treats the population design term $G$ as known. Evaluating its exponential sum is not asserted to take polynomial time. The theorem is therefore an information result, not computational minimax optimality.

Rate interpretation and student-class scope

Let $S = R\sqrt{d}$ and $m = d(d + 1)/2$. Since $m$ is of order $d^2$, the rate can equivalently be written, up to constants, as $\min\{S/\sqrt{B}, m/B\}$. The two displayed branches meet at $B = d^4/S^2$. These are familiar norm- and dimension-dependent scales; the theorem identifies their matching role for the present observation and output constraints. For a given radius, the crossover need not lie inside the stated budget window.

The radius restriction has a substantive modeling effect. The uniform design covariance in Lemma 3 implies

Thus the root-mean-square conditional logit decreases with $d$ under this evaluation design, although the worst-key logit can remain bounded away from zero. A single fixed nonzero edge weight eventually violates the radius condition, even though its conditional logits remain uniformly bounded. The theorem does not cover general fixed-strength sparse interactions. This is a local restriction on the student, not on the arbitrary teacher table. Vanishing excess risk approaches the selected projection optimum; it need not eliminate the total teacher–student disagreement. The matching guarantee covers the stated budget window, whose lower endpoint is of order $d^3$, rather than all query budgets.

Supporting comparisons

Two simpler comparisons clarify why observation protocol and comparator must be kept separate. The first is an unstructured coverage barrier; the second is exact identification of an already compatible teacher. Neither comparison measures the price of incompatibility by subtracting risks from different protocols.

Unstructured coverage

This comparison illustrates the coverage cost of an unstructured family. Let $X=(Y,Z_1,\ldots,Z_{d-1})\in\{0,1\}^{d}$, query only the full conditional of $Y$, and evaluate risk uniformly over the $N_{\mathrm{ctx}}=2^{d-1}$ contexts of $Z$. For each sign vector $\omega\in\{-1,+1\}^{N_{\mathrm{ctx}}}$ and $0<\gamma<1/2$, consider the compatible teacher family

Each member is induced by a valid joint distribution, so the irreducible incompatibility term is zero in this subproblem.

This is a standard independent-sign coverage argument using Yao’s principle [31], not a universal oracle lower bound. Appendix B checks compatibility and the unqueried-context Bayes loss for this concrete family.

Compatible exact-response rank boundary

For a compatible teacher $p_{\theta}(z)=\sigma(a(z)^{\mathsf{T}}\theta)$, taking an exact response’s logit supplies one linear measurement of $m$ parameters. This equation-solving mechanism is established in logistic model extraction [33, Section 4.1.1]. Here uniform full-key evaluation and output in $\Theta_R$ give the adaptive risk lower bound

where $\kappa_0=\sigma(M_0)(1-\sigma(M_0))$ and $M_0>0$ is a fixed logit envelope satisfying $R\sqrt{d}\leq M_0$; hence $0<\kappa_0\leq1/4$. Appendix D verifies the uniform-ball residual uncertainty, the curvature conversion, and the spanning property of our legal rows. Consequently $m$ linearly independent exact queries recover a compatible teacher. That identification boundary is distinct from passive projection of an arbitrary frozen table; the cited model-extraction result is not being used as a proof of Eq. (14).

A fixed-design high-probability upper bound and a residual approximation bound are retained in Appendix K. They are supporting inequalities, not additional main contributions.

Hard-Marginal Reconciliation

Having characterized finite-access precision for one student class, we turn to directional fidelity: what does a chosen coherent projection fit well? We use a computable small joint table whose direct marginals are fixed to black-box predictions. Varying the weight on the two conditional directions lets us examine whether improvement in the fitted objective transfers to the other direction. This is a complementary instance of the conditional-risk framework, with a different output class and observation design; it is not an implementation or empirical test of Theorem 1.

Let the two categorical targets be $A \in\mathcal{A}$ and $B \in\mathcal{B}$, with $n_A \coloneqq|\mathcal{A}|$ and $n_B \coloneqq|\mathcal{B}|$. For each test covariate vector $x$, four black-box prediction tasks provide the two direct marginals and both conditional directions. Fix $x$ and write $q_{ab} \coloneqq q_x(A=a,B=b)$ and $\mathbf{q} \coloneqq\operatorname{vec}((q_{ab})_{a,b})$. Define the two uniformly weighted directional risks

and, for $\omega\in[0,1]$,

The confirmatory direction-balanced estimand is

The extension additionally uses $\omega=1$ for an $A\mid B$-only objective and $\omega=0$ for a $B\mid A$-only objective. The opposite conditional direction is not included in either endpoint objective.

The direct predictions $p_A(\cdot\mid x)$ and $p_B(\cdot\mid x)$ constrain $q_x$ and are not extra risk terms. On the fixed-marginal feasible set, $q_{A\mid B}(a \mid b,x)=q_{ab}/p_B(b \mid x)$ and $q_{B\mid A}(b \mid a,x)=q_{ab}/p_A(a \mid x)$. Hence, up to a constant that does not depend on $q$,

All reported probabilities are floored at $10^{-9}$ and renormalized once before risk evaluation. Let $\mathrm{M}$ be the full-row-rank marginalization matrix retaining every row equation and all but the last column equation, and let $\mathbf{r}$ contain the corresponding direct marginals. With $\underline{q}_{ab}:=10^{-8}p_A(a \mid x)p_B(b \mid x)$, define the feasible set by

The direction-weighted objective and its optimizer are

The independent joint $q_x^{\mathrm{ind}}(a,b)=p_A(a \mid x)p_B(b \mid x)$ is strictly feasible. Compactness, strict convexity, and standard convex duality [34 Chapter 5] give a unique optimum and a primal–dual suboptimality bound. The Frank–Wolfe gap is another standard certificate [35 Section 2], valid for exactly feasible points regardless of which optimizer produced them. Proposition 6 in Appendix § 15 collects the cell response, both certificates, and the checks specific to our constraints. These are applications of established optimality tools, not new general algorithms. Since the independent joint is feasible,

This ordering concerns only the optimized objective. When the two frozen conditional directions are incompatible, it gives no ordering for the direction omitted at $\omega=0$ or $1$. Thus the observed magnitude and certificate coverage are empirical findings, whereas the sign of exact optimized-objective non-inferiority is structural.

We solve Eq. (19) in dual coordinates with a damped semismooth Newton method, following the established nonsmooth Newton framework [36]. Every returned floating-point solution is checked independently at registered tolerances for positivity, normalization, marginal feasibility, the lower bound, a stable Fenchel primal–dual gap, and a projected-gradient Frank–Wolfe gap. Solver termination alone is not accepted as a certificate, and the checks remain numerical rather than proofs that stored floating-point rows are exactly feasible. No general convergence rate for the Newton implementation is claimed.

The relation to classical probability fitting is discussed in § 2. Here the variable-dependent objective is $-\sum c_{ab}^{(\omega)} \log q_{ab}$, equivalently the $q$-dependent part of $D_{\mathrm{KL}}(c^{(\omega)}\mathbin{\|}q)$. It is not a regularized transport objective with the reverse KL direction.

Frozen TFM Case Study

The case study examines directional fidelity under the hard-marginal projection in § 5: does optimizing one conditional direction improve the direction omitted from its objective? Independent numerical checks assess the computed solutions across the registered units. A TabPFN-v2 [26]/OpenML Car split [37] is the confirmatory anchor. A separately frozen extension uses two additional categorical datasets and three splits per dataset. Together, these layers assess the formulation’s numerical reliability and directional consequences, rather than estimate the passive minimax rate.

Across both layers, all four prediction tasks use the same TabPFN-v2 checkpoint through tabpfn 8.1.0, with n_estimators=1, low_memory fit mode, and float32 inference. Probability vectors are frozen before optimization; reconciliation neither refits nor updates the TFM.

AI-assisted research workflow. OpenAI Codex (OpenAI), using GPT-5.6 Sol (model identifier: gpt-5.6-sol), assisted in preparing and revising research scripts, zero-model-output preflight checks, offline verifiers, and tests. Formal runs followed frozen protocols with author authorization. Dataset and checkpoint choices and protocol changes remained under author control. Reported values came from registered computational outputs checked by protocol-defined verifiers, not generated textual answers. The authors reviewed and edited AI-assisted material, checked the results, and take responsibility for the research decisions and conclusions.

Confirmatory Car anchor

The method was reached through three non-interchangeable registered stages. T0 reduced conditional risk without preserving the directly predicted marginals. T1 imposed exact marginals, but a generic SLSQP implementation passed the main numerical gates on only 8/64 rows. T2 changed the numerical method, not the estimand or scientific thresholds, and used a new split that excluded all T0/T1 test positions. The full failure sequence and the T2 row-wise distribution are retained in Appendix F.

The T2 split used OpenML Car dataset 40975, version 3 [38], containing 1,728 complete rows. Target $A$ = class has four classes, target $B$ = safety has three classes, and the five covariates are buying, maint, doors, persons, and lug_boot. With random state 20260924, the disjoint confirmatory split contains 1,024 training rows and 64 test rows. All 64 transport problems passed the independent numerical checks. The mean direction-balanced conditional risk decreased from 0.5040 for the independent joint to 0.03992 for the anchored joint, an aggregate relative reduction of 92.08% against the preregistered 20% threshold. Direct and reconciled marginal accuracies were identical, as expected because the direct marginals were fixed. Table 3 summarizes the confirmatory results.

Measure	Recorded value
Independent-joint mean conditional risk	0.5040015
Reconciled-joint mean conditional risk	0.0399169
Aggregate relative conditional-risk reduction	92.08%
Rows passing independent numerical checks	64/64
Maximum direct-marginal probability change	$9.09 \times10^{-14}$

Frozen external-validity extension

The extension retains the same TabPFN-v2 checkpoint, hard-marginal solver, lower bound, and certificate tolerances. It uses OpenML Mushroom (ID 24, version 1) [39] and Bank Marketing (ID 1461, version 1) [40], with three fixed splits per dataset, 1,024 training rows and 64 test rows per split. The three test sets within each dataset are disjoint; training sets may overlap but exclude all three registered test sets. Dataset eligibility, target identities, class order, and split positions were frozen by zero-model-output screening before any extension model fit or prediction. Four TabPFN tasks per unit produce the two direct marginals and the two conditional families, for 24 fits and 384 test rows in total.

After the registered complete-case rule, Mushroom contains 5,644 rows; target $A$ = class has two classes, target $B$ = cap-color has eight classes, and the remaining 21 variables are covariates. Bank Marketing contains 45,211 rows; target $A$ = Class has two classes, target $B$ = V9 (contact communication type) has three classes, and the remaining 15 variables are covariates.

For a unit $u$ with test covariates $x_1,\ldots,x_{64}$, the bidirectional gain and its aggregate relative reduction are

The two direction-held-out contrasts are

Here “direction-held-out” means that the evaluated opposite-direction conditional prediction was omitted from the optimization objective for the same test row. It does not mean evaluation against new labels, a new model, or a population of datasets. Negative $\widehat{H}$ favors one-direction reconciliation; positive $\widehat{H}$ favors the independent joint.

The dataset–split unit, not an individual row or solver call, is the primary generalization unit. We report all six unit estimates. Each interval is a two-sided percentile interval from 10,000 paired-row bootstrap resamples within that unit. It quantifies within-unit row variation and is not a population interval over datasets or splits. Three-split dataset means are descriptive; no cross-dataset population test is performed.

Extension results

All six units passed their numerical status gate. Every one-direction and bidirectional method passed on every row: 192/192 certificates per unit and 1,152/1,152 overall. Table 4 reports both absolute and relative bidirectional reductions. The absolute gains are much larger on Mushroom than on Bank Marketing, so the percentages should not be read without their risk scales.

Dataset / split	$\overline{R}_{\mathrm{ind}}$	$\overline{R}_{\mathrm{bi}}$	$\widehat{G}^{\mathrm{bi}}$	95% interval	Reduction
Mushroom / 0	0.203929	0.004938	0.198991	[0.143780, 0.260780]	97.58%
Mushroom / 1	0.712894	0.009236	0.703658	[0.531375, 0.881553]	98.70%
Mushroom / 2	0.588489	0.014333	0.574155	[0.425745, 0.732887]	97.56%
Bank Marketing / 0	0.013765	0.006852	0.006913	[0.002849, 0.012826]	50.22%
Bank Marketing / 1	0.014419	0.006153	0.008266	[0.004651, 0.013017]	57.33%
Bank Marketing / 2	0.016563	0.005444	0.011119	[0.004792, 0.019215]	67.13%

Figure 2 shows the two direction-held-out contrasts. Optimizing $A \mid B$ improves the omitted $B \mid A$ direction on all three Mushroom splits, with intervals below zero, but the three Bank Marketing intervals cross zero. In the reverse analysis, optimizing $B \mid A$ worsens the omitted $A \mid B$ direction on average in all six units, with every interval above zero.

Interpretation

The direction-held-out analysis distinguishes an accurately solved projection objective from improved fit in every conditional direction. On the registered units, the independent checks support the computed solutions, yet the opposite-direction contrasts depend on both the dataset and which direction was optimized. This complements the theory’s finite-access question: approaching a selected projection optimum and obtaining improvement outside its fitted objective are different goals. The interpretation remains conditional on the frozen checkpoint, selected units, and within-unit uncertainty assessment; § 7 collects the resulting scope limits.

Discussion and Scope

Finite-access precision and directional fidelity are distinct aspects of coherent reconciliation. The theory quantifies the excess risk of estimating a selected projection from partial access; the case study examines whether fitting one direction also improves another under fixed marginals. Their common conditional-risk framework separates accurate approximation of an optimum from the consequences of choosing that optimum. The output classes and observation protocols differ, so the case study complements rather than empirically validates the minimax rate.

Statistical and computational scope. The matching rate allows arbitrary deterministic binary teacher tables, including probabilities zero and one, under uniform passive-iid exact-table slots. Its local bounded proper pairwise student class and budget window remain substantive restrictions. The radius restriction yields the decreasing average signal in Eq. (11); the result is not a general fixed-strength high-dimensional reconstruction theorem. It does not establish a new rate shape, the cost of teacher incompatibility, an advantage over directed noncoherent estimation, or a joint-KL/TV guarantee. Approaching the same-class optimum reduces excess risk, not necessarily the incompatibility or approximation floors. The known population design term in the upper-bound estimator also prevents interpreting query optimality as a polynomial-time computational guarantee.

Meaning of the empirical findings. The hard-marginal case study provides complete numerical check coverage within one checkpoint and the registered units. Feasibility of the independent baseline implies exact optimized-objective non-inferiority; the observed gain magnitude and opposite-direction contrasts are the empirical questions. Direction-held-out means omission from a row’s optimization objective, not prediction against new labels. Intervals describe paired row variation within a unit, not uncertainty over a population of datasets or models. Numerical checks at registered tolerances are not exact-feasibility proofs for stored floating-point rows. No predictive-accuracy or general-TFM reliability conclusion follows.

Separate auxiliary evidence and future questions. The supplementary response-adaptive construction has a non-joint output and a finite specified comparator class. Its public-data weighting study fails the all-dataset gate because Mushroom reverses sign (Appendix K); the failure is retained, not pooled with the real-TFM case study. It does not supply an active coherent query theorem. Joint-level finite-query precision, or the cost of coherent rather than directed output under one common observation model, remain distinct open questions. Answering either would require additional theory, not extrapolation of the current rate.

Conclusion

We characterize finite-access precision and examine directional fidelity in coherent projection of black-box conditional predictions. For arbitrary deterministic binary teacher tables, matching expected excess-risk bounds identify the budget, dimension, and radius dependence of the best local bounded pairwise projection under uniform passive exact-table queries. The characterization retains its stated radius and budget window.

The fixed-marginal TabPFN-v2 case study examines the complementary question of what a selected projection fits well. Independent numerical checks support the computed solutions, while the asymmetric directional contrasts show that improvement in a fitted objective need not extend to the omitted direction. Together, the results separate estimating a projection accurately from improving every conditional direction. Joint-level precision, coherent active design, and guarantees of predictive-accuracy improvement remain questions for further study.

Data availability

The original datasets used in this study are publicly available through OpenML [37]. The case study uses Car [38], Mushroom [39], and Bank Marketing [40]. The semisynthetic study also uses Letter Recognition [41], the morphological feature subset of Multiple Features [42], Nursery [43], and Page Blocks Classification [44]. The dataset references provide original-source DOIs and links to the OpenML records, with the versions actually used. The case-study data handling and split design are described in § 6; the semisynthetic use of public data is described in Appendix K.§ 4.

Declaration of generative AI and AI-assisted technologies in the manuscript preparation process

During the preparation of this work, the authors used OpenAI Codex (OpenAI), using GPT-5.6 Sol (model identifier: gpt-5.6-sol), to assist with manuscript organization, language and clarity revision, LaTeX preparation, consistency checks, and the preparation and revision of research scripts and protocol-verification artifacts, including preflight, offline verification, and tests. This supported the research workflow, not development of a standalone software product. The tool was not a source of experimental observations; it did not independently select datasets, alter frozen protocols or checkpoints, authorize executions, or replace stored outputs. Reported values were derived from frozen execution artifacts and checked by protocol-defined verifiers. The authors reviewed and edited the AI-assisted output as needed, verified the final manuscript and reported results against the frozen source files and protocol-defined checks, and take full responsibility for the content of the published article.

A passive exact-table minimax characterization

We isolate a setting in which the statistical cost of projecting an arbitrary conditional table onto a coherent pairwise model can be characterized up to constants. The result is deliberately protocol-specific: it concerns a uniform passive design with exact deterministic table responses. It is not an active-query or computational minimax theorem.

Model and risk

Let $X \in\{-1,+1\}^{d}$. A query key is $z=(j,x_{-j})$, where $j \in[d]$ is the target coordinate and $x_{-j} \in\{-1,+1\}^{d-1}$ is the context. The set of all keys is $\mathcal{Z}$, with

and $\mu$ denotes the uniform distribution on $\mathcal{Z}$.

The coherent pairwise student has

parameters: one intercept per target and one shared parameter per unordered pair. We encode its conditional logit as $a(z)^{\mathsf{T}}\theta$, where the feature row $a(z) \in\mathbb{R}^{m}$ has the target intercept equal to one, the $d-1$ edges incident to the target equal to the corresponding context signs, and every other coordinate equal to zero. Thus

where $\sigma(u)=(1+e^{-u})^{-1}$.

The teacher is an arbitrary deterministic table

Endpoint values use the KL convention from § 3; the student probabilities are strictly positive. For $B$ passive design slots, the learner observes

If the same key appears more than once, its response is the same frozen table entry. In particular, a collision is not a fresh noisy label. The population risk is

We write

where the infimum ranges over randomized learners under the preceding passive protocol whose output $\hat{\theta}$ belongs to $\Theta_R$ almost surely, the supremum ranges over the deterministic teacher tables specified above, and the expectation is over the design and learner randomization.

Let $A(u)=\log(1+e^u)$ and define

Up to a term $C_p$ independent of $\theta$,

Thus the teacher affects its best pairwise projection only through the $m$-dimensional moment $b_p$.

Proof. Each row has one nonzero intercept and $d-1$ nonzero incident-edge entries. The intercept coordinate for target $j$ is active with probability $1/d$, and an edge is active when either endpoint is queried, with probability $2/d$. Every cross moment vanishes by the independent Rademacher context signs, proving (A.3). Moreover, abbreviating $a=a(Z)$ inside the expectation,

Since $\lVert a(z)\rVert_2=\sqrt{d}$ and $\lVert\theta\rVert_2\le R$, the logit has magnitude at most $M_0$. Hence its logistic derivative lies in $[\kappa_0,1/4]$. Combining this with (A.3) gives the claimed bounds.

Moment estimator: the matching upper bound

Given the $B$ exact responses, set

and output any minimizer

The population design term $G$ is treated as known. This is an information or query result; it does not assert that the exponential sum defining $G$ can be evaluated in polynomial time.

Matching lower bound

The full statement of Theorem 1 appears in § 4.2. Proposition 4 already bounds the risk uniformly over $\mathcal{P}_{\mathrm{all}}$. To prove the lower bound, fix $\gamma:= 1/4$ throughout the construction below and restrict to

With the same learner class and observation protocol, enlarging the teacher class cannot decrease the minimax risk: for every learner its worst-case excess risk over $\mathcal{P}_{\mathrm{all}}$ is at least that over $\mathcal{P}_{1/4}$, and taking the infimum preserves the inequality. The two-valued tables below lie in this subclass. Hence their lower bound, together with the uniform upper bound, proves the full-class statement without changing the student radius or budget window. Since $\gamma$ is fixed, the theorem constants depend only on $M_0$.

The same construction also retains the restricted-class result: for any fixed $\gamma\in(0,1/4]$, replacing the fixed amplitude by $\gamma$ gives matching bounds over $\mathcal{P}_{\gamma} := \{p : \mathcal{Z} \to[1/2-\gamma,1/2+\gamma]\}$, with constants depending on $M_0$ and $\gamma$. The full-class theorem needs only the fixed choice $\gamma= 1/4$.

Proof. The upper bound is Proposition 4. We prove the full-class lower bound using the preceding fixed-amplitude subclass in seven steps.

Step 1: Edge code and frozen deterministic tables. Restrict the hard family to the $r = d(d-1)/2$ edge coordinates and set every intercept perturbation to zero. A Varshamov–Gilbert code [32] gives $\Omega\subset\{-1,+1\}^{r}$ such that

Draw $\omega$ uniformly from $\Omega$. Unless stated otherwise, probabilities in the lower-bound argument are over this codeword prior, the conditionally drawn frozen table, the i.i.d. design slots, and learner randomization. Let

where $c_1 > 0$ will be chosen sufficiently small. Whenever $t_\omega$ is combined with a full $m$-dimensional parameter or moment vector, it denotes its zero-intercept embedding in $\mathbb{R}^{m}$. Let $a_E(z)$ denote the edge subvector of $a(z)$. Conditional on $\omega$, independently generate one sign for every key,

for a fixed absolute $c_2 > 0$, and then freeze the exact table

Since a key activates at most $d-1$ edge coordinates and $B \ge Cd^3$, choosing $c_1/\sqrt{C}$ sufficiently small ensures $\lvert c_2 a_E(z)^{\top}t_\omega\rvert\le1/2$. Thus (A.7) is valid, and every realization of (A.8) is a deterministic teacher in $\mathcal{P}_{1/4} \subset\mathcal{P}_{\mathrm{all}}$. The randomness is a prior over frozen tables, not response noise at query time.

Step 2: Collision-aware transcript information. Let $P_\omega^{(B)}$ be the transcript law after mixing over $S$. Given the key sequence, observing the exact responses is equivalent to observing $S_z$ once for each unique queried key. Hence

where $\rho=1/2$ is the preceding bias envelope. Repeated keys can only remove terms from this upper bound. By (A.3),

Choose $c_1$ so that this is at most $\frac{1}{8}\log|\Omega|$. Fano’s inequality [32] then implies that every transcript decoder satisfies, for all sufficiently large $r$,

Step 3: The realized table stays near its code moment. Set

For the realized table, define $b_{\omega,S}:=b_{p_{\omega,S}}=\mathbb{E}_\mu[p_{\omega,S}(Z)a(Z)]$. The prior-mean moment shift is edge-only and equals

Write $e_{\omega,S}:=b_{\omega,S}-b_0-h_\omega$. Independence across the frozen table entries and $\lVert a(z)\rVert_2^2=d$ give

Fix a sufficiently small $\varepsilon_0 > 0$. Since $\delta= c_1\sqrt{d/B}$ and $B \le N^{1/3}$, Markov’s inequality implies, after increasing $d_0$ if needed,

This is the good-table event.

Step 4: Common ideal geometry. Define

Then $\Phi(0) = 0$, $\nabla\Phi(0) = 0$, and its Hessian obeys Lemma 3. Up to a table-dependent additive constant, the realized objective is

The ideal objective is $F^0_\omega(\theta) = \Phi(\theta) - h_\omega^{\mathsf T}\theta$.

We first treat the interior regime. Let $\theta^0_\omega$ be the unique minimizer of $F^0_\omega$ over $\Theta_R$. For any boundary point $\lVert\theta\rVert_2 = R$, integrating the lower Hessian bound along the segment $s\theta$ gives

A constrained boundary minimizer would instead require this radial derivative to be nonpositive. Therefore, if $\lVert h_\omega\rVert_2 < \kappa_0R/d$, the ideal minimizer is interior. Symmetry of the Rademacher contexts makes every intercept objective strictly convex and stationary at zero, so $\theta^0_\omega$ has zero intercept coordinates. The edge-subspace lower curvature and its first-order condition then yield

Consequently there is a constant $c_{\mathrm{in}} > 0$ such that

places every ideal center inside the ball of radius $R/2$.

The upper Hessian bound gives, for distinct codewords,

Let $\theta^*_{\omega,S}$ minimize the realized objective. Strong convex variational inequalities for two objectives differing by a linear term imply

On the good-table event, choosing $\varepsilon_0$ sufficiently small makes this at most $\Delta_{\mathrm{in}}/8$.

Step 5: Interior decoding implies $d^2/B$ excess risk. Decode a learner output by its nearest ideal center. A decoding error implies $\lVert\widehat{\theta}-\theta^0_\omega\rVert_2 \ge\Delta_{\mathrm{in}}/2$ and, by (A.17),

on the good-table event. Since the realized objective is $\kappa_0/d$-strongly convex on the ball,

Step 6: Boundary decoding implies $R\sqrt{d/B}$ excess risk. Let

Thus $H\asymp K\delta$, where the comparison constants are absolute (and in particular do not depend on $d$, $B$, or $R$). Code separation gives $u_\omega^\mathsf{T}u_{\omega'}\le3/4$ for $\omega\ne\omega'$. Define the decoder by $\widehat{\omega}\in\arg\max_{\nu\in\Omega}u_\nu^\mathsf{T}\widehat{\theta}$, with a fixed lexical tie rule. If this decoder is wrong, elementary halfspace geometry gives

Convexity and $\nabla\Phi(0)=0$ imply $\Phi\ge0$, while smoothness gives $\Phi(Ru_\omega)\le R^2/(4d)$. Using $Ru_\omega$ as a feasible comparator in (A.12), every decoder-error output satisfies

Hence there is a constant $C_{\mathrm{bd}}>0$ such that, on the good-table event,

Step 7: Fano, Yao, and the transition band. For either decoder, (A.9) and (A.11) imply

Combining this event with (A.18) or (A.21) yields the corresponding Bayes expected excess-risk lower bound. The prior is supported entirely on deterministic exact tables in $\mathcal{P}_{1/4} \subset\mathcal{P}_{\mathrm{all}}$, so Yao’s principle [31] transfers the Bayes lower bound to randomized learners in the minimax problem (A.1).

It remains to connect the two sufficient regimes. Write $\delta_B=c_1\sqrt{d/B}$, let $U=N^{1/3}$, and enlarge $C_{\mathrm{bd}}$ if necessary so that $C_{\mathrm{bd}}>c_{\mathrm{in}}$. The boundary and interior conditions respectively hold for

These thresholds differ by the fixed factor $(C_{\mathrm{bd}}/c_{\mathrm{in}})^2$. If $B_{\mathrm{in}}\le Cd^3$, the interior argument already covers the declared window. Otherwise set $B_*:=\lceil B_{\mathrm{in}}\rceil$. For integer slot budgets $B\in[B_{\mathrm{bd}},B_*]$, the target minimum is at most $(C_{\mathrm{bd}}/c_1)R^2/d$, while the interior lower bound at $B_*$ is a constant multiple of $R^2/d$; rounding changes only that constant for sufficiently large $d$. Since the minimax risk is nonincreasing in the number of passive slots, the bound at $B_*$ transfers to every smaller budget in this transition band.

For completeness, fix any constant $L>4(C_{\mathrm{bd}}/c_{\mathrm{in}})^2$. If $B_*\le LU$, the interior construction is valid at $B_*$ even if $B_*>U$: its Fano and bias bounds require only $B_*\ge Cd^3$, and its good-table probability requires $\sqrt{B_*/N}$ to be sufficiently small, which follows from $B_*\le LN^{1/3}$ by increasing $d_0$. If $B_*>LU$, then for all sufficiently large $d$ one has $B_{\mathrm{in}}>LU-1$, hence $\delta_Ud>C_{\mathrm{bd}}R$; the boundary argument therefore covers every $B\le U$. This exhausts the transition and window-endpoint cases without changing the stated theorem window or rate, and proves the lower bound in (10).

Proof of the unstructured query barrier

Proof of Proposition 2. Put the uniform prior on the $N_{\mathrm{ctx}} = 2^{d-1}$ independent signs in Eq. (12). Give the context vector $Z$ a uniform marginal. For each sign assignment, combining this marginal with the specified conditional of $Y$ constructs a compatible joint law.

Fix any adaptive querying algorithm and condition on its transcript, including its internal randomization. An exact response determines the sign at the queried context. The signs at all unqueried contexts remain independent and uniform under the prior. If the algorithm predicts $u \in(0,1)$ at one such context, its conditional Bayes loss is

The average log loss is minimized at $u = 1/2$, where it equals $\kappa(\gamma)$. This lower bound applies to the conditionals induced by any coherent joint output; requiring coherence cannot reduce it.

At most $B$ distinct contexts can have been queried, so at least $(N_{\mathrm{ctx}}-B)_{+}$ remain unqueried. Their contribution to the uniform-context risk is at least $(1-B/N_{\mathrm{ctx}})_{+}\kappa(\gamma)$ on average under the prior. Every other contribution is nonnegative. Averaging over transcripts and then taking the worst sign assignment proves the first inequality for every randomized algorithm. Finally, Pinsker’s inequality for Bernoulli distributions [32] gives $\kappa(\gamma) \ge2\gamma^{2}$; multiplication by the nonnegative unqueried fraction proves the second inequality and the stated query threshold.

Fixed-design structured bound

Equation (K.1) is a standard bounded-loss uniform convergence bound. We give the specialization to fix its constants, using symmetrization, contraction, and bounded differences [45 Sections 2–3].

For $L_{\theta}(z) := D_{\mathrm{KL}}(\operatorname{Ber}(p(z))\mathbin{\|}\operatorname{Ber}(\sigma(a(z)^{\mathsf{T}}\theta)))$, write $v = a(z)^{\mathsf{T}}\theta$ and $A(v) = \log(1+e^{v})$. Then $L_{\theta} = A(v)-p(z)v-H(p(z))$, $|v| \le M$, $0 \le L_{\theta} \le C_{M}$, and $|\partial L_{\theta}/\partial v| \le1$. In the one-sided Rademacher convention (no absolute value inside the supremum), centering and scalar contraction give

Here $\epsilon_i$ are independent Rademacher signs and $\lVert a(Z_i)\rVert_2^2=d$. Each one-sided uniform deviation between $R_\mu$ and empirical risk $\widehat{R}_B$ therefore has mean at most $2R\sqrt{d/B}$ and bounded differences $C_M/B$. McDiarmid’s inequality and a union bound make both deviations at most $2R\sqrt{d/B}+C_M\sqrt{\log(2/\delta)/(2B)}$ with probability $1-\delta$. Empirical optimality adds the two deviations, proving Eq. (K.1).

For Eq. (K.2), put $v_0=a(z)^\top\theta_0$. The Bernoulli KL is the Bregman remainder $A(v_0)-A(v_0+r)+A'(v_0+r)r\le r^2/8$, since $A''\le1/4$. Average and compare with $\theta_0$. This is a smoothness calculation, not an additional reconstruction theorem.

Compatible rank boundary

To verify (14), put the uniform prior on the Euclidean ball $\Theta_R\subset\mathbb{R}^m$. An exact compatible response reveals $a(z)^\top\theta$ after taking its logit. Conditional on any transcript whose measured rows have rank $k$, rotate coordinates so that the first $k$ directions span those rows. The unmeasured component remains uniform on an $(m-k)$-dimensional ball whose squared radius is $R^2$ minus the squared norm of the observed projection. The same statement remains true when each new row is chosen from previous observations: a new independent row can reveal only one direction of this rotationally symmetric residual ball.

For a uniform ball in $n$ dimensions with radius $s$, each coordinate has second moment $s^2/(n+2)$ and the total second moment is $ns^2/(n+2)$. After $k$ independent measurements, iterating this identity shows that the prior-averaged posterior squared error is $R^2(m-k)/(m+2)$. Repeated or dependent measurements can only leave more unmeasured directions. Equivalently, pad a shorter transcript with independent measurement directions before taking the Bayes average; additional information can only reduce posterior error. Thus $k\le B$ yields the lower bound $R^2(m-B)_+/(m+2)$ for any estimator, including randomized estimators. The uniform full-key covariance obeys $\Sigma\succeq I_m/d$ and the Bernoulli log-partition has curvature at least $\kappa_0$ on the bounded student ball. Therefore conditional KL from a compatible teacher $\theta$ to an output $\widehat{\theta}$ is at least $\kappa_0\lVert\widehat{\theta}-\theta\rVert_2^2/(2d)$. The Bayes bound implies the stated worst-case bound. Finally, the legal rows span $\mathbb{R}^m$: a vector orthogonal to every row must have zero intercept and edge coefficients by averaging each queried target over all Rademacher contexts. Hence $m$ linearly independent rows exist and their exact logits determine $\theta$ uniquely.

Hard-marginal optimality and certificates

Proof. Use standard convex duality [34 Chapter 5] and the Frank–Wolfe gap [35 Section 2], with $x,\omega$ suppressed and cells vectorized. Equation (18) already gives the objective reduction; the probability floor keeps $c_i > 0$ also at $\omega= 0,1$. The independent joint is strictly above $\mathbf{q}$ and has the required marginals. Thus $\mathcal{P}_x$ is nonempty and compact, and $\nabla^2 \mathcal{J} = \operatorname{diag}(c_i/q_i^2) \succ0$ gives uniqueness.

For $\mathbf{y} = \mathbf{M}^{\mathsf{T}}\nu> 0$, scalar minimization gives the stated clipped response. With $f_i(u) := -c_i \log u$, define

Weak duality gives $D(\nu) \le\mathcal{J}(\mathbf{q}^{\star})$. The strictly feasible independent joint supplies Slater’s condition; KKT stationarity is $y_i = c_i/q_i^{\star} + \lambda_i > 0$, with $\lambda_i \ge0$ for the lower bounds. The dual gradient is $\mathbf{M}\mathbf{q}(\nu) - \mathbf{r}$. Away from cell switches its Hessian is

the active-set limits give generalized Hessian elements at switches.

For exactly feasible $\mathbf{q}$, define

The first equality uses $\mathbf{y}^{\mathsf{T}}\mathbf{q} = \boldsymbol{\nu}^{\mathsf{T}}\mathbf{r}$. Weak duality and the standard Frank–Wolfe bound give Eq. (E.1). Finally, if $\mathbf{g} - \mathbf{g}_{\perp} = \mathbf{M}^{\mathsf{T}}\mathbf{v}$, then every feasible pair satisfies

Thus removing the equality-gradient component leaves the gap unchanged.

TFM development sequence and Car row-wise distribution

Table F.1 retains the registered development failures that preceded the confirmatory Car anchor. T2 changed the numerical method rather than retrospectively changing either earlier decision. Figure F.1 shows the individual conditional-risk reductions for the 64 confirmatory test rows.

Stage	Intervention	Diagnostic and registered outcome
T0	Conditional-only projection	Mean direct-marginal TV drift 0.207; failed marginal-fidelity gate
T1	Hard direct marginals with SLSQP	Only 8/64 rows passed the main numerical gates; failed numerical certificate
T2	Hard direct marginals with transport solver	64/64 rows passed independent numerical checks on a new split; passed confirmatory gates

Protocol and estimand distinctions

Table G.1 records the observation laws, charged budgets, and evaluation targets. It expands the main-task summary in Table 2. Exact frozen responses, fresh noisy responses, and the TFM’s fixed predictions define different information structures; results are interpreted within their own rows.

Study	Observation and charged budget	Output, target, and scope
Unstructured coverage	Exact probabilities; at most $B$ adaptive or nonadaptive context queries.	Coherent joint; uniform-context conditional KL for a compatible teacher (Proposition 2).
Passive pairwise projection	Exact frozen probabilities; $B$ uniform iid slots, with repeats charged.	Bounded pairwise joint; expected same-table best-projection excess KL within Theorem 1’s radius and budget window.
Fixed-pool recovery	Quantized logits: prefixes 128, 256, 512; 4,096 reference calls per target charged separately.	Least-squares parameters; distance to the frozen full-pool target in Eq. (K.3).
Study	Observation and charged budget	Output, target, and scope
Restricted auxiliary theorem	Fresh Rademacher logit noise; 18 diagnostic and two local calls.	Non-joint table; expected full-key loss and E-AUC; 16 static allocations and response-independent mixtures (Theorem 7).
Gaussian comparison	Fresh Gaussian logit noise, standard deviation 0.05; 18+2 calls.	Non-joint table E-AUC; two frozen families and two endpoints, with a different noise law from Theorem 7.
Semisynthetic bridge	Public-data context structure and weights; generated oracle with fresh Gaussian query noise.	Weighted table E-AUC and minimum static-minus-adaptive contrasts; the all-dataset gate fails.
Hard-marginal TFM study	Frozen direct marginals and both conditional families; four prediction tasks per unit; reconciliation adds no model calls.	Hard-marginal joint; balanced and opposite-direction risks against the same rows’ frozen predictions, not new labels.

Auxiliary conditional-table task and construction

This supplementary task compares response-adaptive and response-independent allocation under a shared learner with a conditional-table output. Classical active data selection provides background [46, 47]. The finite construction below fixes its families, noisy observations, and 16-policy static comparator; Theorem 7 states the separation and Appendix J proves it.

Task and loss

For each full-conditional key $k$ of five binary variables, let $\ell(k)$ denote the teacher logit and $\hat{\ell}_{B}(k)$ the learner’s reported logit at budget $B$. Define the auxiliary full-key table loss

where the 80 keys comprise five targets and 16 contexts per target. For a fixed family $F$ and scalar amplitude $\tau$, set

where the superscript indicates that the expectation includes any policy randomization, in addition to the uniform hidden regime $Z$ and fresh query noise $\xi$. E-AUC is therefore the registered two-budget, equal-weight average of expected full-key table losses at 18 and 20 charged calls; smaller values indicate better conditional-table prediction. This table loss does not require the reported conditionals to be compatible and is not the coherent-joint risk $R_{\mu}(q;\mathcal{O})$ from Eq. (2).

Restricted construction and shared learner

Let $X_0,\ldots,X_4 \in\{-1,+1\}$ and evaluate uniformly over all five single-target full-conditional keys and all 16 contexts per target. Fix $L=1/2$, $a=\delta=\nu=1/20$, and $\tau\in[19/100,1/5]$. The known family $F$ is either $A$ or $B$; within a family the hidden regime $Z$ is uniform on $\{-1,+1\}$. Put $\psi(x)=x_1x_2x_3x_4$ and define the nonzero teacher logits by

with every other target logit zero and

All logits are in $[-L,L]$. These are genuinely incompatible conditional families: at a context where the local feature vanishes, the cross difference of the target-0 logit in the reciprocal pair is $2a+2\delta Z$, whereas the reverse target logit gives $2a$.

The diagnostic key is target 0 at context $(1,1,1,1)$. Its mean logit is $\mu_Z=a+\delta Z$ and its local feature is zero. Each charged query returns the teacher logit plus a fresh independent noise $\xi\in\{-\nu,+\nu\}$ with equal probability. Thus the 18 repeated diagnostic calls are 18 noisy observations, unlike repeated keys in the exact-table protocol. The common selector chooses the nearer of $\mu_{-1}$ and $\mu_{+1}$ to their sample mean, with a fixed lexical tie rule.

For each regime, the two registered local keys are target-0 contexts

Their local features in the correct regime are $+1$ and $-1$. The adaptive policy chooses $D^F_Z$ after diagnosis. Every deterministic static policy fixes an ordered pair from $D^{F}_{-1} \cup D^{F}_{+1}$ before seeing responses, giving $4^2 = 16$ possibilities; response-independent random mixtures are also admitted. Both kinds of policy spend the same 18 diagnostic and two local calls.

The common learner uses diagnostic calls only for $\widehat{Z}$. Given two local responses $Y_i$ at keys $k_i$, it subtracts the public known-logit map $h_{F,\widehat{Z}}(k_i)$ and fits the scalar coefficient

The strictly convex objective has the unique minimizer

Here $h_{F,z}$ is the corresponding $ax_1$ or $ax_2$ pairwise term, $\delta_z\psi$ at target 0, the reciprocal $ax_0$ term at target 1 or 2, and zero elsewhere; $\phi^{F}_{z}$ is zero off target 0. The reported full-key logit is $\operatorname{clip}_{[-L,L]}(h_{F,\widehat{Z}}+\phi^{F}_{\widehat{Z}}\widehat{\tau})$. This output is a conditional table, not a coherent joint. Indeed, in family $A$ fix $x_2=x_3=x_4=1$ and vary $x_1$. The local feature vanishes and clipping is inactive; the reported target-0 logit changes by $2a+2\delta_{\widehat{Z}}$, while the reciprocal target-1 logit changes by $2a$ when $x_0$ varies. A positive joint requires these cross-differences to agree. In family $B$ the same mismatch uses targets 0 and 2, fixing $x_1=x_3=x_4=1$ and varying $x_2$. Thus no joint can in general induce the reported table, irrespective of the fitted $\widehat{\tau}$. At budget 18, no local coefficient is fitted and the common default is $\widehat{\tau}=0$, giving identical outputs for all policies.

Restricted response-adaptive separation

Proof of the restricted separation

Proof of Theorem 7. The two diagnostic means are separated by $2\delta= 0.1$. A sub-Gaussian tail bound for the average of 18 independent noises gives, for each $Z$ and either family,

The local noises are independent of the diagnostic transcript. On correct diagnosis, let $n \in\{0,1,2\}$ be the number of local calls whose feature is nonzero for the true regime. From Eq. (H.6),

The diagnostic rows do not enter this two-row ridge normalization. For every $\tau$ in the stated interval, $\mathcal{E}_{\tau}(n)$ is decreasing and convex on $[0,2]$: direct differentiation gives a negative first derivative because $\nu^{2}<4\tau^{2}$, and a nonnegative second derivative because $\nu^{2}<3\tau^{2}$.

The local feature is nonzero at four of the 16 target-0 contexts and zero for the other four targets. Thus the full-key risk weight of the scalar error on correct diagnosis is $w=4/(5\cdot16)=1/20$. The true and reported logits lie in $[-L,L]$, and the clipping is inactive on correct diagnosis. Logistic curvature therefore bounds the risk there between $(m_L/2)w\mathcal{E}_{\tau}(n)$ and $(1/8)w\mathcal{E}_{\tau}(n)$, where $m_L=\sigma(L)(1-\sigma(L))$. On wrong diagnosis, clipping bounds the entire full-key risk by $L^2/2$.

For any deterministic static ordered pair, let $n_{+}$ and $n_{-}$ count its calls in the two regime blocks. Since $n_{+}+n_{-}\leq2$ and $Z$ is uniform, $\mathbb{E}_{nZ}\leq1$. This remains true under response-independent randomization. It would be incorrect to apply this count after conditioning on correct diagnosis, because the selector’s lexical tie rule can change the conditional regime weights. Instead, fix $Z=z$ and the static allocation randomness $U$, and define the success probability $s_z:=\Pr(\widehat{Z}=z\mid Z=z)\geq1-\epsilon$. Independence of local noise gives

Now average over the unconditioned uniform $Z$ and independent $U$. Jensen’s inequality and monotonicity of $\mathcal{E}_{\tau}$ yield

The adaptive policy has $n=2$ on correct diagnosis. Its unconditional risk therefore satisfies

Here replacing the actual error probability by $\epsilon$ is valid because the bad-event bound exceeds the correct-event bound.

Both bounds hold for each fixed family and every $\tau\in[0.19,0.20]$. The static lower expression is minimized at $\tau=0.19$, while the active upper expression is maximized at $\tau=0.20$. Direct substitution gives

Their difference exceeds $1.60 \times10^{-5}$. At budget 18 the shared diagnostic transcript, selector, and default coefficient yield exactly equal risks. Half-weighting budgets 18 and 20 therefore gives an E-AUC difference greater than $8.0 \times10^{-6} > 7 \times10^{-6}$ for each of the 16 static policies and any response-independent mixture. $\square$

Supplementary bounds and query studies

The fixed-pool study measures quantized parameter recovery; the Gaussian and semisynthetic studies assess the non-joint conditional-table task in Appendix H. Their protocols and estimands are compared in Table G.1.

Fixed-design upper bound and approximation

Use the pairwise joint and feature rows of Eqs. (7)–(8). In this subsection, let $\mu$ be any fixed, sampleable distribution over such keys, let the oracle return a deterministic exact probability $p(z) \in[0,1]$, and draw $B$ keys independently from $\mu$. If $\widehat{\theta}_B$ minimizes empirical conditional-KL risk over $\Theta_R$, write $M = R\sqrt{d}$ and $C_M = \log(1+e^M)$. For every $\delta\in(0,1)$, the structured oracle inequality is

This applies standard Rademacher uniform convergence [45]; Appendix C checks the loss envelope, feature norm, and constants. The comparator remains the best coherent pairwise projection for the given table.

If, additionally, the oracle logit is $a(z)^\top\theta_0+r(z)$ for some $\theta_0\in\Theta_R$ and $\mathbb{E}_{\mu}r(Z)^2\leq\rho^2$, then the logistic log-partition’s $1/4$ smoothness gives

This controls the sum of irreducible incompatibility and structural approximation; it does not identify either term separately.

Budget trend in a frozen structured family

The structured study uses a frozen family and a fixed pool of candidate queries. Checks of held-out joint risk, support, and parameter reporting covered 450 eligible rows across dimensions 16, 32, and 64. The independently frozen quantized study uses resolution $\eta= 10^{-3}$. For each target, the complete frozen candidate pool $C$ contains 4,096 keys, its design matrix is $A_C \in\mathbb{R}^{4096 \times d}$, and $\ell_{\eta,C} \in\mathbb{R}^{4096}$ is the vector of quantized oracle logits. The evaluation reference is the unique full-column-rank least-squares target

The stored design audit verifies that $A_C$ has full column rank, so the displayed minimizer is single-valued. The learner instead uses the first $B \in\{128, 256, 512\}$ rows of the fixed ordering and reports the corresponding prefix least-squares estimate $\hat{\beta}_B$. The 4,096 reference calls per target serve to define the evaluation target and are accounted for separately from the learner’s $B$ calls. Table K.1 reports the median parameter distance $\left\|\hat{\beta}_B-\beta_{\eta,C}\right\|_2$ among eligible targets at each budget. The median support F1 and the support-certificate-stable fraction were both 1 in all nine dimension–budget cells.

$d$	$B = 128$	$B = 256$	$B = 512$	log–log slope	fit $R^2$
16	1.93670	1.17407	0.853482	−0.591	0.984
32	3.03878	2.00064	1.26002	−0.635	0.999
64	4.93385	2.85877	1.80692	−0.725	0.998

The log–log slopes in Table K.1 fit exactly the three registered budget points for each dimension. Their negative signs show a within-family decrease in this particular fixed-pool error measure. The earlier development track did not meet its frozen all-dimension stability criterion; the independently frozen study reported here passed it. Neither the slopes nor the support scores establish a general finite-bit or $\varepsilon$-dependent rate, an active-design advantage, or the expected conditional-KL minimax rate of the exact-table theorem.

Restricted Gaussian query-design comparison

The Gaussian comparison keeps the two frozen conditional-logit families, two parameter endpoints, common learner, and all 16 response-independent two-local-call static policies from the restricted separation, but uses fresh Gaussian logit noise with standard deviation 0.05 rather than the theorem’s bounded Rademacher noise. Its registered core consists of eight anchors and 60 replications per anchor. Every one of the 3,840 recorded paired static-minus-active E-AUC gaps is positive, as is the worst-policy score in each of the 60 replication indices. The mean paired gap is $3.305842798 \times10^{-5}$ and the smallest recorded gap is $5.560061386 \times10^{-7}$. The pooled active mean E-AUC is about 18.1% below the pooled static mean under the study’s descriptive denominator. These observations describe conditional-table prediction in the frozen Gaussian construction. They establish neither a coherent-joint active advantage nor an expanded comparator class or observation law.

Public-data structural weighting and transfer boundary

The semisynthetic bridge uses five public OpenML datasets: letter (ID 6) [41], mfeat-morphological (ID 18) [42], mushroom (ID 24) [39], nursery (ID 26) [43], and page-blocks (ID 30) [44], all version 1. Three fixed splits and four generator families produce 60 reference units. The public data determine the selected binary context structure and weights; the conditional-logit oracle in this comparison is generated. This is a real-structure stress test of the restricted auxiliary conditional-table result, not a five-dataset evaluation of a real tabular foundation model or of active coherent reconstruction.

For each reference unit and replication, the recorded contrast is the minimum static-minus-adaptive E-AUC gap over the frozen family/parameter anchors and 16 static policies. A reference score averages its 60 replication minima; a dataset score averages its 12 reference scores (three splits and four generators). Thus the plotted statistic is a nested worst-comparator contrast, not an average-static-policy gap.

The registered global score, all four generator scores, and 53 of 60 reference scores favor the response-adaptive policy. The stronger registered all-dataset requirement fails: only four of five dataset scores favor it, and mushroom is negative. In mushroom split 0, all four generator references are negative and 234 of 240 replication minima are nonpositive. The registered decision remains bridge_not_passed; positive aggregate results do not reverse that decision. Table K.2 displays the positive core evidence beside the observed transfer failure, while Figure K.1 shows the five dataset-level signs.

Table K.2: Restricted query-design evidence and its transfer boundary. Positive auxiliary conditional-table contrasts favor the response-adaptive policy over the frozen response-independent static comparator. The semisynthetic all-dataset gate was not met.

Study	Positive evidence	Registered boundary
Gaussian core	3,840/3,840 paired gaps positive; mean gap 3.3058 $\times10^{-5}$	Two families, two endpoints, 16 static policies
Semisynthetic bridge	Global and 4/4 generator scores positive; 53/60 references positive	4/5 dataset scores positive; mushroom negative; all-dataset gate failed

A post-hoc within-design accounting attributes 98.42% of the mushroom split-0 negative contribution mass to three high-weight contexts. This locates the arithmetic concentration of the reversal; it does not identify a causal feature or establish how to prevent the failure. The permitted conclusion is an aggregate and majority-reference advantage with a verified dataset/split heterogeneity boundary.

Broader prior-work comparison

Table K.3 retains additional related results. Each row describes the cited result only; an absence of the present finite-query statement in a row is not a literature-wide novelty claim.

Work	Input/oracle assumption	Output and objective	Finite-query result
Majid et al. [11]	Family of subset conditionals; joint compatibility at issue	Conditions for consistent joint recovery	Not analyzed
Arnold et al. [2]	Two finite discrete conditional families	Exact, near, and $\epsilon$-compatibility; compatible-table adjustment	Not analyzed
Muré [9]	Potentially incompatible conditionals	Pseudo-Gibbs stationary compromise; mean-squared misfit	Not analyzed
Chen et al. [8]	Nearly compatible or incompatible conditional models	Gibbs-ensemble joint compromise	Not analyzed
Heckerman et al. [10]	Locally specified dependency network; possible inconsistency	Gibbs-like inference and joint behavior	Not for this oracle
Picado Muiño [5]	Explicit probabilistic knowledge base	Consistency-restoring probability adjustments	Not analyzed
Thimm [12]	Explicit probabilistic conditional logic	Distance-based inconsistency measures	Not analyzed
Potyka–Thimm [13]	Explicit probabilistic conditional knowledge base	Consolidation through generalized models	Not analyzed
Nguyen et al. [14]	Explicit probabilistic knowledge bases, possibly inconsistent and structurally diverse	Divergence-based consistency recovery and merging	Not analyzed
Barik–Honorio [16]	Conditional queries; low-rank BN tables and compatible teacher	Exact directed BN structure	Polynomial under stated assumptions
Torroba Hennigen–Kim [15]	Masked-LM conditionals; two-token analysis	Explicit joint; conditional closeness	Not analyzed
Liu et al. [17]	Deep generative teacher	Tractable probabilistic circuit distillation	Not for this conditional oracle
Blanca et al. [18]	Coordinate-conditional samples from a hidden joint	Identity test	Bounds for their sampling oracle
Klötergens et al. [27]	TFM predictive conditionals	Consistency diagnostics	Not a repair guarantee
This work	Possibly incompatible, probability-vector conditional oracle	Coherent-joint projection; conditional-KL excess risk	Matching minimax only for the frozen passive-i.i.d. exact-table protocol and window