betaFeedback

WorksComputer Science

Observable-Specific Validity Tests for Scientific AI: Controlled Approximations and Intervention Response

Abstract

An approximation can reproduce an energy while missing its response, or satisfy an error tolerance without being certified by a given bound. We formulate observable-specific validity records for scientific artificial intelligence (AI) that link a reduction to its reference model, error measure, tolerance, parameter domain, and transformations or interventions. Explicit displacement and velocity certificates for a singularly perturbed oscillator recover distinct convergence orders and provide sufficient validity guarantees. Exact Hubbard dimer relations resolve the different coupling domains accepted by energy and response criteria. Across 16 repeated neural fits, imposing dimensional homogeneity reduces energy scale transfer error from 52.3% to 0.0768% and raises the median fraction jointly satisfying gap and response tolerances from zero to 100%. Paired transformations verify the protected scaling relation, while a coupling crossover identifies the additional information needed for response transfer. A causal model recovers a positive intervention response beneath a strong negative observational association. For two mechanisms observed with Gaussian noise whose responses first differ at order xkx^k, we derive an exact fixed control query requirement scaling as x−2kx^{-2k} and verify its full success curve numerically. Together with a 256-task evaluation interface, these results provide independently checkable tests of controlled approximation, physical transfer, and mechanism discrimination in scientific AI.

Introduction

A useful physical model often becomes more explanatory by retaining fewer details. In a strongly interacting electron system, virtual charge excitations can be replaced by a low energy spin exchange. In a strongly damped mechanical system, a rapidly relaxing momentum can be eliminated from the slow displacement dynamics. The resulting description succeeds when the retained variables and neglected processes have the right hierarchy for the observable being studied. Establishing this hierarchy gives numerical predictions their physical interpretation.

Scientific artificial intelligence (AI) increasingly supports this process through prediction, equation discovery, and the organization of scientific evidence [1, 2]. Recent perspectives place large language models (LLMs) within the broader cycle of hypothesis formulation, modeling, and experimental assessment [3]. In physics, language models can execute multistep Hartree–Fock calculations when supplied with an appropriate decomposition [4]. PhysReason, PHYBench, and the condensed matter benchmark CMPhysBench assess demanding physical problems [5, 6, 7], while CritPt and PRL-Bench extend evaluation toward research calculations [8, 9]. These capabilities motivate quantitative tests of the physical commitments that accompany a successful prediction.

The connection between prediction and understanding also has a substantial philosophical foundation. Contextual accounts relate understanding to knowledge that a scientific community can use and manipulate [10]. Analyses of machine learning emphasize the evidential connection between a model and its target [11, 12]. We give this connection an operational form. A system demonstrates a specified physical capability when its explanation makes correct, testable commitments about reductions, limits, and interventions. This definition ties the evaluated capability to explicit scientific consequences.

Several recent studies directly motivate such tests. PhySense evaluates principle-based reasoning, including symmetry, dimensional analysis, power counting, and limiting cases [13]. PhysGym controls the prior information supplied during interactive discovery. Newton-Bench changes physical laws and experimental environments, and LLM-SRBench tests equation discovery beyond common formula recall [14, 15, 16]. DiscoverPhysics separately evaluates predictive error and explanatory quality in noncanonical simulated worlds [17]. On the methodological side, KeplerAgent uses inferred physical structure to constrain symbolic regression, while explicit physical modeling has been separated from execution in physics reasoning [18, 19]. Our focus is the validity ity of an approximation for each observable. Reference quantities, acceptance tolerances, validity domains, and sufficient certificates become explicit evaluation objects. This adds an independently checkable validity decision to principle-based reasoning and interactive discovery.

Physical structure provides a productive basis for such commitments. Reviews of physics informed machine learning describe how equations, symmetries, and domain knowledge can guide learning[20]. Dimensional analysis constrains admissible functional forms[21, 22], and sparse discovery identifies compact dynamics within a chosen library[23]. Hamiltonian, Lagrangian, and equivariant neural architectures encode conservation or transformation relations[24, 25, 26]. Physics informed neural networks (PINNs) enforce differential equations through residual losses[27]; studies of their optimization identify conditions under which these constraints are successfully learned[28]. These approaches show the value of making a physical requirement explicit. Our framework extends this logic to the validity domain of a proposed reduction and to the observables it is intended to describe.

Our contribution is an evaluation record for individual observables linking a proposed reduction to its reference model, target quantities, validity domain, acceptance tolerance, and intervention consequences. We make the record executable through distinct tests for certified validity, measured error, transformation consistency, and joint observable acceptance. Explicit fast–slow certificates, exact dimer relations, paired neural experiments, and a known causal model instantiate these tests. An exact Gaussian measurement calculation then connects mechanism separation to query cost. The reported evidence consists of analytical models, neural surrogates, and controlled simulations; it supplies reference tests for subsequent language model evaluations.

Observable-specific approximation certificates

The object being evaluated

Let M(p)M(\mathbf{p}) denote a reference physical model with parameters p\mathbf{p}. A proposed reduction AA specifies its retained variables or projection, effective equations, neglected terms, and controlling dimensionless quantities. We represent its claimed scope by

C=(M,A,Q,P,Ω,s,δ,b,T),(1)\mathcal{C}=(M,A,Q,\mathcal{P},\Omega,s,\delta,b,\mathcal{T}), \tag*{(1)}

where QQ is the set of target observables, P\mathcal{P} the parameter domain, Ω\Omega the observation window, ss the normalization rules, δ\delta the acceptance tolerances, bb the proposed error bounds, and T\mathcal{T} the specified transformations or interventions. For observable QQ, define

eQ(p;Ω)=∥QA(p)−QM(p)∥ΩsQ(p).(2)e_Q(\mathbf{p};\Omega)=\frac{\lVert Q_A(\mathbf{p})-Q_M(\mathbf{p})\rVert_{\Omega}}{s_Q(\mathbf{p})}. \tag*{(2)}

The positive scale function sQ(p)s_Q(\mathbf{p}) is declared before evaluation. The rule is fixed; its numerical value may depend on the parameters. Here displacement and dimensionless velocity use unit reference scales, gap error uses sJ=JM(p)>0s_J=J_M(\mathbf{p})>0, and response-exponent error uses sn=1s_n=1. An intervention difference uses a declared nonzero response scale; the dimensionless causal example below uses δΔY=1\delta_{\Delta Y}=1. A certificate establishes eQ≤bQe_Q\leq b_Q by analysis or a justified bound. For domain bookkeeping set bQ=+∞b_Q=+\infty outside its proof scope, while retaining an explicit out-of-scope label. Its certified domain satisfies

DQcert={p∈P:bQ(p)≤δQ}⊆DQvalid={p∈P:eQ(p)≤δQ}.(3)D_Q^{\mathrm{cert}}=\{\mathbf{p}\in\mathcal{P}:b_Q(\mathbf{p})\leq\delta_Q\} \subseteq D_Q^{\mathrm{valid}}=\{\mathbf{p}\in\mathcal{P}:e_Q(\mathbf{p})\leq\delta_Q\}. \tag*{(3)}

Equation (3) gives two complementary outputs. The certified domain provides a sufficient guarantee, while the reference error determines actual acceptance whenever it can be evaluated. Points with bQ>δQb_Q>\delta_Q remain eligible for reference based verification. Physical tolerances such as δJ=0.05\delta_J=0.05 describe the approximation itself; the numerical answer matching tolerance used by the evaluation interface measures the accuracy of a reported answer.

The record organizes the physical content of an explanation into the obligations in Table 1. An observation window specifies which fast processes must be retained, a transformation specifies which relations must be preserved, and an intervention specifies which response must change. Each successful test then supports a concrete part of the explanation. Alternative reductions are evaluated through the same declared quantities and tolerances.

ObligationTest objectSupported physical capability
Observable validityDefined norm, scale, and observation windowResolving displacement and initial velocity on their relevant time scales
Controlled reductionSmall parameter, retained variables, omitted termsLocating the regime in which eliminated processes remain perturbative
Physical transformationCovariance under the specified changePreserving the physical relation under the specified transformation
Intervention responseA changed equation, coupling, or structural assignmentPredicting the response to a changed physical assignment
Mechanism discriminationInformation from allowed measurements at fixed costSelecting measurements that separate competing physical responses

Table 1. Physical commitments made explicit by the evaluation record. Each row connects a scientific claim to a quantitative test and the capability it supports.

A residual certificate with an explicit physical scope

Residual-based error control and goal oriented estimates have a substantial numerical-analysis foundation[29]. Green function bounds also support validated computations for singularly perturbed boundary value problems[30]. We use the same residual-to-error principle in a known initial value problem, obtaining explicit constants for two observables and separating initial mismatch from fast residual layers. These constants make the approximation record directly evaluable.

Consider the dimensionless damped oscillator

εx′′+τx′+x=0,x(0)=1,x′(0)=0,(4)\varepsilon x''+\tau x'+x=0,\qquad x(0)=1,\qquad x'(0)=0, \tag*{(4)}

where ε=mk/γ2\varepsilon=mk/\gamma^2 and τ=kt/γ\tau=kt/\gamma for mass mm, stiffness kk, damping γ\gamma, and physical time tt. Primes denote derivatives with respect to τ\tau. For 0<ε≤1/40<\varepsilon\leq1/4, the causal Green function GεG_\varepsilon of the differential operator is nonnegative and has integral one on [0,∞)[0,\infty). Write Mε=sup⁡τ≥0Gε(τ)M_\varepsilon=\sup_{\tau\geq0}G_\varepsilon(\tau). For ε<1/4\varepsilon<1/4,

a=21+1−4ε,b=1+1−4ε2ε,a=\frac{2}{1+\sqrt{1-4\varepsilon}},\qquad b=\frac{1+\sqrt{1-4\varepsilon}}{2\varepsilon},
Mε=aexp⁡[−alog⁡(b/a)b−a].(5)M_\varepsilon=a\exp\left[-\frac{a\log(b/a)}{b-a}\right]. \tag*{(5)}

with M1/4=2/eM_{1/4}=2/e by continuity.

Proposition 1 (Displacement certificate). For 0<ε≤1/40 < \varepsilon\le1/4 and T>0T > 0, let y∈C2([0,T])y \in C^2([0,T]) be a candidate trajectory for Eq. (4), and let R=εy′′+y′+y=∑jRjR = \varepsilon y'' + y' + y = \sum_j R_j. On [0,T][0,T], with e0=y(0)−1e_0 = y(0)-1 and e1=y′(0)e_1 = y'(0),

∥y−x∥∞≤∣e0∣+εMε∣e1∣+∑jmin⁡{∥Rj∥∞,Mε∥Rj∥1}.(6)\lVert y-x\rVert_\infty\le|e_0|+\varepsilon M_\varepsilon|e_1| +\sum_j \min\{\lVert R_j\rVert_\infty,M_\varepsilon\lVert R_j\rVert_1\}. \tag*{(6)}

The norms of each residual component are taken over [0,T][0,T].

The proof is given in Appendix A. The bound separates initial data mismatch, persistent residuals, and short residual layers. A rapidly decaying residual can have a large instantaneous amplitude and a small integrated effect on displacement. The maximum norm controls persistent forcing, whereas the integrated norm captures a brief impulse. Choosing between them for each residual component retains the time scale of the eliminated process within an explicit displacement guarantee.

For the leading reduction y0=e−τy_0=e^{-\tau}, Eq. (6) gives

b0=ε(1+Mε).(7)b_0=\varepsilon(1+M_\varepsilon). \tag*{(7)}

A matched first order displacement is

y1=(1+ε)e−(1+ε)τ−εe−τ/ε.(8)y_1=(1+\varepsilon)e^{-(1+\varepsilon)\tau}-\varepsilon e^{-\tau/\varepsilon}. \tag*{(8)}

Its initial derivative mismatch is −2ε−ε2-2\varepsilon-\varepsilon^2. Splitting the slow and fast residuals yields

b1=(2ε+ε2)εMε+ε2(1+ε)(2+ε)+Mεε2.(9)b_1=(2\varepsilon+\varepsilon^2)\varepsilon M_\varepsilon+\varepsilon^2(1+\varepsilon)(2+\varepsilon)+M_\varepsilon\varepsilon^2. \tag*{(9)}

Thus the certificate itself recovers b0=O(ε)b_0=O(\varepsilon) and b1=O(ε2)b_1=O(\varepsilon^2) directly from the equation and the residual structure.

Proposition 2 (Velocity certificate). Under the assumptions of Proposition 1,

∥y′−x′∥∞≤Mε∣e0∣+∣e1∣+∑jmin⁡{2Mε∥Rj∥∞,ε−1∥Rj∥1}.(10)\lVert y'-x'\rVert_\infty\le M_\varepsilon|e_0|+|e_1| +\sum_j\min\{2M_\varepsilon\lVert R_j\rVert_\infty,\varepsilon^{-1}\lVert R_j\rVert_1\}. \tag*{(10)}

The derivative of the Green function weights the residual differently from the displacement kernel. For the two reductions, this gives

bv,0=1+2Mεε,b_{v,0}=1+2M_\varepsilon\varepsilon,
bv,1=3ε+ε2+2Mεε2(1+ε)(2+ε).(11)b_{v,1}=3\varepsilon+\varepsilon^2+\frac{2M_\varepsilon\varepsilon^2}{(1+\varepsilon)(2+\varepsilon)}. \tag*{(11)}

The first remains order unity, while the second is O(ε)O(\varepsilon). Appendix A establishes the kernel identities and the bounds.

Transformations and interventions require different responses

For a physical transformation gg, the observable obeys the corresponding map Q(gp)=TgQ(p)Q(gp)=T_gQ(p). Unit changes, label permutations, and removable gauge phases belong in this class when their assumptions hold. An intervention changes a coupling, boundary condition, or structural assignment and specifies a corresponding response. We evaluate paired correctness alongside transformation consistency, so that the score records both the preserved relation and agreement with the physical reference.

For a specified intervention pair, we test

ϵint=∣ΔQ^−ΔQM∣sΔQ,(12)\epsilon_{\mathrm{int}}=\frac{\left|\Delta\widehat{Q}-\Delta Q_M\right|}{s_{\Delta Q}}, \tag*{(12)}

with a prespecified nonzero response scale. When observations admit several intervention responses, the record identifies the compatible mechanisms and an informative measurement that separates them[31, 32, 33]. In this way the explanation leads either to a verified response or to a concrete experimental question.

Physical models and computational methods

Fast–slow dynamics

The ratio ε\varepsilon in Eq. (4) compares the inertial relaxation time m/γm/\gamma with the restoring time γ/k\gamma/k. We compare the exact solution, y0y_0, and y1y_1 over ε∈[10−4,2]\varepsilon\in[10^{-4},2]. A grid of 220 logarithmically spaced parameters and 5000 time points combining logarithmic and uniform sampling on [0,8][0,8] resolves the initial layer. Velocity is evaluated over both [0,8][0,8] and the delayed window [0.1,8][0.1,8]. Continuous displacement boundaries at tolerance 0.05 are located by optimizing over temporal error extrema and applying a bracketed root solver in ε\varepsilon. Independent Radau integration verifies the exact trajectories, including critical damping. Numerical integration and root finding use SciPy[34].

Effective exchange and learned representations

We use the symmetric two-site Hubbard model with two electrons,

H=−t∑σ(c1σ†c2σ+h.c.)+U∑i=12ni↑ni↓,U>0.(13)H=-t\sum_{\sigma}\left(c^{\dagger}_{1\sigma}c_{2\sigma}+\mathrm{h.c.}\right)+U\sum_{i=1}^{2}n_{i\uparrow}n_{i\downarrow},\qquad U>0. \tag*{(13)}

There is one undirected bond, ℏ=1\hbar=1, and the spins are 1/21/2. The exact dimer provides a transparent reference for effective-model reasoning[35]. Virtual high-energy excitations motivate a low-energy transformation of the type formalized by Schrieffer and Wolff[36]. The triplet energy is zero and the relevant singlet block is

HS=(0−2t−2tU).(14)H_{\mathrm{S}}=\begin{pmatrix}0&-2t\\-2t&U\end{pmatrix}. \tag*{(14)}

Here h.c. denotes the Hermitian conjugate. The positive singlet–triplet gap is

J=U2+16t2−U2,r=∣t∣U,(15)J=\frac{\sqrt{U^{2}+16t^{2}}-U}{2},\qquad r=\frac{\lvert t\rvert}{U}, \tag*{(15)}

corresponding to Heff=J(S1⋅S2−1/4)H_{\mathrm{eff}}=J(\mathbf{S}_{1}\cdot\mathbf{S}_{2}-1/4). We also evaluate J2=4t2/UJ_{2}=4t^{2}/U and J4=4t2/U−16t4/U3J_{4}=4t^{2}/U-16t^{4}/U^{3}. Numerical evaluation uses the rationalized expression J=8t2/(U2+16t2+U)J=8t^{2}/(\sqrt{U^{2}+16t^{2}}+U) to avoid cancellation.

We compare a raw multilayer perceptron (MLP) with a dimensionless multilayer perceptron (Π\Pi-MLP), where Π\Pi denotes dimensionless groups. Both networks have two hidden layers of 32 hyperbolic tangent units and a linear output. They use the limited memory Broyden–Fletcher–Goldfarb–Shanno (L-BFGS) optimizer[37], implemented in scikit-learn[38]. The raw MLP maps (log⁡(∣t∣/E0),log⁡(U/E0))(\log(\lvert t\rvert/E_{0}),\log(U/E_{0})) to log⁡(J/E0)\log(J/E_{0}), where E0E_{0} is a fixed reference energy. The Π\Pi-MLP maps log⁡r\log r to log⁡(J/U)\log(J/U) and reconstructs JJ by multiplication by UU, imposing the dimensional homogeneity exactly. This comparison isolates the effect of a supplied physical representation within a common network architecture.

Each of 16 independent fits uses 240 physical training points sampled uniformly in logarithmic coordinates from U/E0∈[0.5,2]U/E_{0}\in[0.5,2] and r∈[0.02,0.2]r\in[0.02,0.2]. Each fit has 512 test points in each of three distributions comprising the training range, an energy scale shift with U/E0∈[20,200]U/E_{0}\in[20,200], and a coupling ratio shift with r∈[0.3,1]r\in[0.3,1]. One range changes at a time, and the two representations share the physical samples. Training uses exact gap labels, with test points reserved for evaluation. Inputs are standardized using training statistics, and the natural logarithmic targets retain their original scale.

Besides relative gap error, we measure the response exponent nt=∂log⁡J/∂log⁡tn_{t}=\partial\log J/\partial\log t at fixed UU, using centered differences with logarithmic step 10−310^{-3}. Central values are medians of the per fit test medians; percentile ranges describe variation across the repeated fits.

A known observational and interventional model

To isolate causal response, we use centered, dimensionless structural equations driven by independent standard normal exogenous noises[31],

P=ηP,Z=P+0.1ηZ,Y=2Z−3P+0.1ηY,(16)P=\eta_{P},\qquad Z=P+0.1\eta_{Z},\qquad Y=2Z-3P+0.1\eta_{Y}, \tag*{(16)}

Here PP is a measured common cause, ZZ an intermediate quantity, and YY the response. Each of 400 independent repetitions contains 400 observational samples and 400 randomized samples. The randomized assignment is Zdo∼N(0,1)Z^{\mathrm{do}}\sim\mathcal{N}(0,1), independent of PP and both outcome noises. It produces Ydo=2Zdo−3P+0.1ηYdoY^{\mathrm{do}}=2Z^{\mathrm{do}}-3P+0.1\eta_{Y}^{\mathrm{do}}, with a fresh independent standard-normal ηYdo\eta_{Y}^{\mathrm{do}}. Observational and randomized conditions share the same 400 draws of PP, but use independent ZZ-assignment and outcome noises. All ordinary least-squares regressions include an intercept; unadjusted and PP-adjusted estimates use the same observational samples. The coordinates are dimensionless and have zero population means, with no distribution-specific rescaling.

Two-mechanism measurement design

Consider equal-prior hypotheses

H1: μ1(x)=x,H2: μ2(x)=x1+x,(17)H_{1}:\ \mu_{1}(x)=x,\qquad H_{2}:\ \mu_{2}(x)=\frac{x}{1+x}, \tag*{(17)}

with independent observations y=μH(x)+ξy=\mu_{H}(x)+\xi, ξ∼N(0,σ2)\xi\sim\mathcal{N}(0,\sigma^{2}), σ=0.3\sigma=0.3, and x∈[0.02,1]x\in[0.02,1]. Gains, saturation scale, and noise are known, and all queries have equal cost. Bayesian design selects a query by expected posterior information gain[39]. We use 40-point Gauss–Hermite quadrature on 48 logarithmically spaced candidate controls. Under these equal-variance known hypotheses, the best query is x=1x=1 for every nondegenerate posterior, so the information strategy repeats that control. The known hypotheses give an analytically tractable reference for measurement selection.

We compare it with uniform controls on [0.02,1][0.02,1], logarithmically uniform controls, and repeated x=0.05x=0.05 measurements. The eight-query experiment has 500 noise repetitions, evenly divided between the two true hypotheses, with common noise sequences paired across policies. The outcome is the fraction with posterior probability at least 0.95 on the true hypothesis at each budget. Wilson intervals quantify finite repetition uncertainty[40]. The comparison isolates measurement selection for these two known physical mechanisms.

Results

Initial layer matching improves displacement and velocity at different orders

As shown in Figure 1(a,b), the exact dynamics separate slow displacement from the initial velocity layer. At ε=0.03\varepsilon= 0.03, both approximate displacements are close to the exact trajectory. The leading solution nevertheless starts with y0(0)=−1y_0(0) = -1, whereas the reference velocity is zero. For the full window,

Figure 1. Observable-dependent validity of the damped-oscillator reduction. (a) Displacement and (b) early velocity at ε=0.03\varepsilon= 0.03. (c) Solid displacement-error curves cover the full plotted parameter range; dashed residual certificates apply only for ε≤1/4\varepsilon\le1/4. The horizontal line is tolerance 0.05 and the vertical line marks the onset of oscillatory exact motion. (d) Velocity errors on full (solid) and delayed (dashed) windows, with full window certificates (dash-dot) restricted to ε≤1/4\varepsilon\le1/4. All errors are deterministic, using the initial displacement and dimensionless time unit as reference scales.

∥y0′−x′∥[0,8],∞≥1(18)\lVert y'_0-x' \rVert_{[0,8],\infty} \ge1 \tag*{(18)}

for every positive ε\varepsilon. This persistent initial mismatch identifies the fast momentum relaxation that must be restored for a full window velocity approximation. Panel (d) shows how a delayed observation window instead selects the slow dynamics.

The mismatch originates in eliminating momentum before its initial relaxation. The fast displacement component has amplitude O(ε)O(\varepsilon) and duration O(ε)O(\varepsilon), so its derivative contributes at order unity even as its displacement vanishes. Consequently, the zero-inertia and initial-time limits do not commute for velocity. The matched solution restores this component, leaving displacement and full window velocity errors of orders ε2\varepsilon^2 and ε\varepsilon, respectively. Fitted exponents over ε≤0.003\varepsilon\le0.003 are 0.993 and 1.993 for the leading and matched displacement errors, and 0 and 1.000 for the respective full window velocity errors. The observed slopes confirm this separation of scales.

Figure 1(c,d) connects this hierarchy to usable parameter domains. At displacement tolerance 0.05, the continuously located validity boundaries are ε=0.06498\varepsilon= 0.06498 for the leading reduction and 0.16390 for the matched reduction. Their sufficient certified boundaries are 0.02593 and 0.10103. Matching therefore enlarges both the directly accepted displacement interval and the analytically guaranteed interval. The matched velocity has its own sufficient boundary, ε=0.01624\varepsilon= 0.01624, because differentiating the fast layer amplifies its contribution. The gap between each bound and the measured error quantifies the conservatism introduced by separately bounding residual components.

These accuracy boundaries all precede the critical value ε=1/4\varepsilon= 1/4, where the two real decay rates coalesce. Beyond this value, they form a complex conjugate pair. Consequently, an overdamped trajectory can already require a more accurate reduction before any oscillatory motion appears. The record distinguishes a physical change of motion from an observable dependent accuracy requirement. It also shows how the required measurement window determines the variables that an effective description must retain.

Energy agreement and response agreement have different domains

The dimer gap has the controlled expansion

J=4t2U−16t4U3+O(t6/U5).(19)J = \frac{4t^2}{U} - \frac{16t^4}{U^3} + O(t^6/U^5). \tag*{(19)}

The second order term reflects a virtual intermediate state with a doubly occupied site. Two hopping events connect the low-energy spin configurations, with an energy cost UU for the intermediate charge excitation. Singlet mixing lowers the energy, whereas Pauli exclusion prevents this lowering for the triplet, giving antiferromagnetic J>0J > 0. The exact response exponent is

nt=1+11+16r2.(20)n_t = 1 + \frac{1}{\sqrt{1+16r^2}}. \tag*{(20)}

As illustrated in Figure 2(b), ntn_t decreases from 2 in the superexchange regime toward 1 as charge hybridization becomes strong. For small rr, the two hopping events and their charge energy denominator give J∝t2/UJ \propto t^2/U. For large rr, the exact singlet–triplet gap approaches 2∣t∣2\lvert t\rvert, so the dominant energy dependence is linear in hopping. The exponent is a logarithmic susceptibility to hopping and tests the response dependence predicted by the proposed exchange mechanism.

Figure 2. Effective exchange and the scope of a superexchange explanation. (a) Exact gap and truncated expansions. The fourth order curve is shown only where positive; it becomes negative for r>1/2r > 1/2. (b) Exact hopping-response exponent. (c) Relative gap error and absolute exponent error of the second order reduction, each divided by its declared tolerance 0.05. The shaded interval 0.08217<r≤0.11460.08217 < r \le0.1146 passes the gap criterion but fails the exponent criterion. Curves are exact analytical relations of the dimer model.

For J2J_2, the exact relative gap error and absolute exponent error are

eJ=1+16r2−12,e_J = \frac{\sqrt{1+16r^2}-1}{2},
en=1−11+16r2.(21)e_n = 1 - \frac{1}{\sqrt{1+16r^2}}. \tag*{(21)}

Consequently, eJ≤δe_J \le\delta permits

r≤δ(1+δ)2,(22)r \le\frac{\sqrt{\delta(1+\delta)}}{2}, \tag*{(22)}

whereas en≤η<1e_n \le\eta< 1 requires

r≤14(1−η)−2−1.(23)r \le\frac{1}{4}\sqrt{(1-\eta)^{-2}-1}. \tag*{(23)}

At δ=η=0.05\delta= \eta= 0.05, the respective limits are 0.1146 and 0.08217. The shaded interval in Figure 2(c) therefore isolates points accepted by the gap criterion that require a more accurate response description. The tolerances represent 5% relative gap error and 0.05 absolute exponent error. Expanding Eq. (21) gives eJ=4r2+O(r4)e_J = 4r^2 + O(r^4) and en=8r2+O(r4)e_n = 8r^2 + O(r^4), explaining why response is the more sensitive diagnostic near the controlled limit. A small correction to the energy can contribute more strongly to its logarithmic derivative.

The fourth order curve in Figure 2(a) further identifies the role of expansion control. The Taylor series has radius of convergence 1/41/4 in rr, and its fourth order truncation becomes negative at r>1/2r > 1/2 while the exact gap remains positive. In the coupling shift test its median relative error is about 130%, compared with 69.6% at second order. These analytical controls provide a physical interpretation of the crossover results. Resumming the hybridization or using the exact reduced function is appropriate when the expansion parameter reaches this regime.

The completed records in Table 2 translate these curves into individual decisions. At ε=0.05\varepsilon= 0.05, direct verification accepts the leading displacement while the residual bound supplies a more conservative guaranteed domain. At r=0.1r = 0.1, the same second order expression supports the gap value under its tolerance and calls for refinement of the hopping response. Recording these outcomes separately identifies both the usable prediction and the physical quantity that benefits from an improved approximation.

Record fieldLeading oscillator, ε=0.05\varepsilon= 0.05Second-order dimer, r=0.1r = 0.1
Targets and scalesDisplacement, sx=1s_x = 1Gap sJ=Js_J = J; exponent sn=1s_n = 1
Actual errorsex=0.04008e_x = 0.04008eJ=0.03852e_J = 0.03852; en=0.07152e_n = 0.07152
Bound or referenceb0=0.09452b_0 = 0.09452Exact error expressions, Eq. (21)
Validity decisionsDisplacement passesGap passes; response fails
Certificate decisionNot guaranteed by this boundExact expressions resolve both decisions

Table 2. Filled evaluation records. All listed acceptance tolerances are 0.05. The oscillator uses [0,8][0, 8]; the dimer quantities are static, with the derivative at fixed UU.

Homogeneity protects energy scale transfer

The exact relation J(λt,λU)=λJ(t,U)J(\lambda t, \lambda U) = \lambda J(t, U), for λ>0\lambda> 0, identifies the energy scale transformation. It leaves rr and the competition between hopping and repulsion unchanged. Table 3 and Figure 3 show how enforcing that relation changes the learned model. The raw network has a median same range gap error of 0.139%, but its error rises to 52.3% when the overall energy scale changes. The dimensionless network has errors of 0.0733% and 0.0768%, respectively. The improvement follows from a supplied physical prior: scale transfer requires no extrapolation of the dimensionless function when rr remains in its training range.

DistributionRepresentationGap error (%)Exponent error
Same rangeRaw MLP0.13880.006155
Same rangeΠ-MLP0.073320.004088
Energy-scale shiftRaw MLP52.330.5163
Energy-scale shiftΠ-MLP0.076750.004289
Coupling-ratio shiftRaw MLP9.8130.1356
Coupling-ratio shiftΠ-MLP8.710.1492

Table 3. Neural fits evaluated against the exact dimer. Each value is the median across 16 independent fits of a 512-point test median. Gap errors are percentages; response-exponent errors are absolute dimensionless differences.

Figure 3. Gap errors (top), hopping-response exponent errors (middle), and joint acceptance fractions (bottom) in 16 repeated network fits. A bottom-row point is the fraction of 512 test points simultaneously satisfying relative gap error ≤0.05\le0.05 and absolute exponent error ≤0.05\le0.05; other points are per fit error medians. Gray segments connect fits using the same physical samples. Black bars are medians across fits and colored vertical lines span their 2.5–97.5 percentiles, describing training and sampling variation.

The middle row of Figure 3 shows that the protected transfer extends to a physical derivative. The raw network’s median exponent error rises from 0.00616 to 0.516, whereas the dimensionless network changes from 0.00409 to 0.00429. Writing J^=Uf^(r)\hat{J} = U\hat{f}(r) makes the reason explicit. Its hopping exponent is rf^′(r)/f^(r)r\hat{f}'(r)/\hat{f}(r), independent of the overall energy scale. The same representation thus preserves the relation between energy and response along each scaling orbit. The Euler homogeneity identity provides a complementary check of this imposed structure.

Changing rr probes the shape of the crossover function. In r∈[0.3,1]r \in[0.3,1], gap errors are 8.71% for the dimensionless network and 9.81% for the raw network, with corresponding exponent errors of 0.149 and 0.136. These results locate the additional learning problem within f^(r)\hat{f}(r) itself. In this range the exact exponent moves away from the quadratic superexchange limit toward linear hybridization, so data that resolve the crossover provide information beyond the scaling relation. Appendix § 11 makes this distinction precise through a family of functions sharing the same homogeneity, training values, and limiting laws while varying within an unobserved interval. Energy scale transfer rises from zero for the raw representation to 100% for the dimensionless representation, whose 2.5–97.5 percentile range is 89.4–100%. The bottom row of Figure 3 displays the paired changes directly. Under coupling ratio transfer, the dimensionless model has a median gap acceptance of 30.7% and joint acceptance of zero. The derivative criterion identifies a concrete target for additional training, namely the change in the physical response through hybridization.

Joint acceptance makes this distinction directly operational. For each trained fit and the declared test distribution, define

pJ=Pr⁡(eJ≤0.05),pn=Pr⁡(en≤0.05),p_J = \Pr(e_J \le0.05), \qquad p_n = \Pr(e_n \le0.05),
p∩=Pr⁡(eJ≤0.05∧en≤0.05),(24)p_{\cap} = \Pr(e_J \le0.05 \land e_n \le0.05), \tag*{(24)}

where probabilities are empirical fractions in each fit’s test set, summarized by their median across fits. As reported in Table 4, the median joint fraction under

DistributionRepresentationpJp_J (%)pnp_n (%)p∩p_{\cap} (%)
Same rangeRaw MLP10098.998.9
Same rangeΠ\Pi-MLP100100100
Energy-scale shiftRaw MLP1.1700
Energy-scale shiftΠ\Pi-MLP100100100
Coupling-ratio shiftRaw MLP20.80.8790.879
Coupling-ratio shiftΠ\Pi-MLP30.700

Table 4. Median acceptance fractions for the two physical tolerances in Eq. (24). Each fit uses 512 test points shared by the two representations. Variation across fits is shown in Figure 3.

Joint acceptance also gives a practical interpretation to the error summaries. A median prediction error describes a typical point, whereas pnp_n estimates the probability mass, under the declared test distribution, on which both physical requirements are satisfied. Reporting pJp_J, pnp_n, and p∩p_{\cap} together therefore connects surrogate accuracy to its usable domain. The paired fits in the figure show how strongly this domain changes when the representation incorporates the correct scale relation.

We additionally transform each of the 8192 same range test points per representation by λ∈{0.01,0.1,10,100,1000}\lambda\in\{0.01, 0.1, 10, 100, 1000\}, keeping rr fixed. The maximum dimensionless-network defect ∣Jb(λt,λU)/(λJb(t,U))−1∣\left|J_b(\lambda t,\lambda U)/(\lambda J_b(t,U))-1\right| is 2.45×10−152.45 \times10^{-15} across all five transformations. At λ=100\lambda= 100, the raw-network defect has a pooled median of 0.622. These strictly paired tests isolate covariance from the independent test sampling used in Table 3, explaining why the two dimensionless gap-error summaries can differ slightly while the covariance relation is exact to rounding precision.

Causal structure recovers the intervention response

For Eq. (16), Var⁡(Z)=1.01\operatorname{Var}(Z) = 1.01, Var⁡(Y)=1.05\operatorname{Var}(Y) = 1.05, and Cov⁡(Z,Y)=−0.98\operatorname{Cov}(Z,Y) = -0.98. The unadjusted population slope and explained variance are

βobs=2−3/1.01=−0.9703,R2=0.9821.01×1.05=0.9056.(25)\begin{aligned} \beta_{\mathrm{obs}} &= 2 - 3/1.01 = -0.9703,\\ R^2 &= \frac{0.98^2}{1.01 \times1.05} = 0.9056. \tag*{(25)} \end{aligned}

An intervention on ZZ instead has

∂E[Y∣do⁡(Z=z)]∂z=2.(26)\frac{\partial\mathbb{E}[Y \mid\operatorname{do}(Z=z)]}{\partial z} = 2. \tag*{(26)}

In the observational data, an increase of ZZ is usually accompanied by an increase of PP. The negative direct effect of PP on YY exceeds the positive ZZ pathway, producing the negative association. Randomization replaces the assignment of ZZ and cuts the link from PP, while preserving the response equation for YY. It therefore isolates the positive response coefficient +2+2.

Figure 4(c) confirms the population analysis over 400 repetitions. The observational regression has median R2=0.9046R^2 = 0.9046 and mean slope −0.9699-0.9699 with standard deviation 0.0160. Adjustment for the measured common cause recovers a mean slope of 2.0017 with standard deviation 0.0504, while randomization gives 1.9941 with standard deviation 0.1532. For a fixed exogenous state and Δz=2\Delta z = 2, the structural response is ΔY=4\Delta Y = 4, compared with approximately −1.94-1.94 from observational regression. Specifying the intervention thus changes both the sign and the physical meaning of the predicted response.

Figure 4. Observational and intervention response in a known synthetic system. (a) Structural graph and coefficients. (b) One repetition with 400 samples per condition and population regression lines; randomized Zdo∼N(0,1)Z^{\mathrm{do}} \sim\mathcal{N}(0,1) is independent of PP. (c) Slopes over 400 repetitions; points are medians, segments are 2.5–97.5 percentiles, and the dashed vertical line is the true intervention slope. Variables use the dimensionless coordinates of the structural equations, with no distribution-specific rescaling.

The scatter and slope distributions clarify why precision and causal accuracy are distinct. The observational slope is tightly concentrated around the association generated by the common cause. Adjustment uses the measured PP to separate its direct contribution from the ZZ response, while the unadjusted randomized regression averages over variation in PP. This explains the smaller spread of the adjusted estimator in this supplied structural model. Both procedures recover the positive response through their respective identification conditions.

An identifiability test specifies which additional observation would determine the response. With independent standard Gaussian noises and 0<∣ρ∣<10 < |\rho| < 1, the two models

Z=ηZ,Y=ρZ+1−ρ2ηY,Y=ηY,Z=ρY+1−ρ2ηZ(27)\begin{aligned} Z &= \eta_Z,\quad Y = \rho Z + \sqrt{1-\rho^2}\eta_Y,\\ Y &= \eta_Y,\quad Z = \rho Y + \sqrt{1-\rho^2}\eta_Z \tag*{(27)} \end{aligned}

generate the same observational distribution and have ZZ-intervention slopes ρ\rho and zero, respectively. Temporal information, a justified physical direction, or an intervention can separate these mechanisms. Reporting this equivalence class together with a discriminating measurement turns observational ambiguity into an actionable experimental prediction.

Mechanism separation sets the measurement cost

For two known Gaussian hypotheses with mean difference Δ(x)=∣μ1(x)−μ2(x)∣\Delta(x) = |\mu_1(x)-\mu_2(x)| and equal variance σ2\sigma^2, a single measurement at fixed control has Kullback–Leibler (KL) divergence

DKL(H1∥H2∣x)=Δ(x)22σ2.(28)D_{\mathrm{KL}}(H_1\Vert H_2 \mid x) = \frac{\Delta(x)^2}{2\sigma^2}. \tag*{(28)}

After BB independent measurements at that control, the log posterior odds in favor of the true hypothesis are Gaussian with mean BΔ2/(2σ2)B\Delta^2/(2\sigma^2) and variance BΔ2/σ2B\Delta^2/\sigma^2. Equal prior odds are assumed.

Proposition 3 (Fixed-control query requirement). For 0<α,β<1/20 < \alpha,\beta< 1/2 and Δ>0\Delta> 0, the smallest integer budget ensuring posterior probability at least 1−α1-\alpha on the true hypothesis with probability at least 1−β1-\beta is

Bmin⁡(x)=⌈σ2Δ(x)2[z1−β+z1−β2+2Lα]2⌉,(29)B_{\min}(x) = \left\lceil\frac{\sigma^2}{\Delta(x)^2}\left[z_{1-\beta}+\sqrt{z_{1-\beta}^2+2L_\alpha}\right]^2 \right\rceil, \tag*{(29)}

where Lα=log⁡[(1−α)/α]L_\alpha= \log[(1-\alpha)/\alpha] and z1−β=Φ−1(1−β)z_{1-\beta} = \Phi^{-1}(1-\beta).

Appendix § 12 derives this expression. If the first difference between mechanisms is Δ(x)=cxk+o(xk)\Delta(x)=c x^k+o(x^k) with c>0c>0 and k>0k>0, then at fixed σ\sigma, α\alpha, and β\beta,

Bmin⁡(x)=Θ(x−2k)(x→0).(30)B_{\min}(x)=\Theta(x^{-2k}) \quad(x\to0). \tag*{(30)}

The exponent 2k2k connects the first distinguishing physical correction to the measurement cost for the specified Gaussian hypotheses and fixed control.

For Eq. (17), Δ=x2/(1+x)\Delta= x^2/(1+x) and k=2k=2. Both mechanisms share the linear response, with their first difference arising from the quadratic saturation correction. As shown in Figure 5(a,Figure 5b), this physical degeneracy produces an x4x^4 information scale and an x−4x^{-4} discrimination cost. With α=β=0.05\alpha=\beta=0.05 and σ=0.3\sigma=0.3, Eq. (29) requires eight measurements at x=1x=1 and 332,510 at x=0.05x=0.05. Moving the control into the nonlinear regime separates the mechanisms relative to the same observation noise.

Figure 5. Mechanism separation and query cost. (a) Exact per-query KL divergence and its small-control fourth order asymptote. (b) Integer query requirement from Eq. (29) for α=β=0.05\alpha= \beta= 0.05, at noise standard deviation 0.3. (c) Equal-budget measurement strategies over 500 balanced noise repetitions; shaded regions are pointwise Wilson 95% intervals. The black dashed curve is the analytical success probability at x=1x = 1 from Eq. (38). The known hypotheses make repeated x=1x = 1 measurement the maximum-information policy.

The policy comparison in Figure 5(c) gives the corresponding finite budget result. Eight measurements produce correct and sufficiently confident fractions of 97.2% for maximum information selection, 43.6% for uniform controls, and 7.8% for controls sampled uniformly in logarithmic coordinates. Repeated low control measurements remain below the posterior threshold. The exact success probability at x=1x=1 rises from 17.53% at one query to 95.84% at eight; the simulated eight query fraction differs by 1.52 Monte Carlo standard errors. The analytical curve therefore provides a quantitative check across the full budget range.

The ordering of these strategies follows their access to the distinguishing saturation term. Uniform sampling allocates some measurements near the informative upper control, whereas logarithmic sampling concentrates more strongly where the shared linear response dominates. The maximum information policy uses the largest allowed separation at every measurement. In this known two mechanism problem, the result gives a direct physical reason for the gain from experimental design[39]. It also identifies a useful evaluation target for a scientific agent, namely selecting a control that reveals the relevant correction within a fixed measurement budget.

Implications for evaluating language models

The controlled examples define evaluation targets for approximation validity, transformation consistency, intervention response, and measurement selection. The development interface implements structured questions about prescribed reductions and physical models, with numerical and categorical outputs scored against analytical references.

The evaluation set contains 128 base systems and 256 paired prompts in four families. Oscillator tasks return actual displacement validity and certificate status separately; dimer tasks return distinct gap and response decisions. The distinction in Eq. (3) thus becomes a scored scientific output. Field correctness, strict correctness, paired joint correctness, and equivalence consistency quantify complementary aspects of the responses. Both variants of each system are presented independently and remain in the same data split. Missing or unparseable responses remain in the score denominators.

These questions support comparisons between structured physical prompts and general analysis prompts under the same response format and tool budget. Supplying the governing equations isolates the use of physical structure in validity and response calculations. A further evaluation condition can ask a model to construct equations from scenarios and data, following the control of prior information studied in PhysGym[14]. Separating these conditions identifies the contributions of model construction and reasoning within a supplied model.

Interactive evaluations can extend these tasks by allowing a model to choose successive measurements and tracking the information gained within a fixed query budget. The Gaussian calculation supplies an analytical reference for measurement selection with known candidate mechanisms. Analogous extensions can ask a model to propose a reduction and then test its accepted observables and domain. Appendix § 14 defines the current task families, data splits, scoring rules, and representative analytical answers. The task bank and scoring implementation are available from the corresponding author on request.

Discussion

Validity changes with the observable and time window because eliminating a fast variable affects derivatives more strongly than slowly varying coordinates. In the oscillator, the leading displacement converges uniformly as O(ε)O(\varepsilon) while the initial velocity mismatch remains unity. Matching the initial layer improves displacement to O(ε2)O(\varepsilon^2) and velocity to O(ε)O(\varepsilon), whereas delaying the observation window targets the response after fast relaxation. Propositions 1 and 2 express this distinction through the Green function norms appropriate to each observable. The dimer gives a corresponding static example. Near the exchange limit, the relative gap error is 4r2+O(r4)4r^2 + O(r^4) and the absolute response-exponent error is 8r2+O(r4)8r^2 + O(r^4), producing different acceptance boundaries at the same numerical tolerance. These calculations identify which variables and corrections must be retained for a specified physical measurement.

The neural comparison separates energy scale transfer from changes in coupling. Writing the predicted gap as UU times a learned function of rr enforces homogeneity and preserves the response exponent along a scaling orbit. Under the energy scale shift, the dimensionless network has a median joint acceptance fraction of 100%. Under the coupling ratio shift, the corresponding fraction is zero, although the median gap acceptance fraction is 30.7%. The second test therefore identifies where additional training information is needed, namely the shape and derivative of the crossover function. As shown by the function family in Appendix § 11, crossover observations determine information independent of homogeneity and the limiting laws. New data in that range can be selected to improve both the gap and its hopping response.

Mechanism discrimination depends on the measured variables, allowed interventions, and observation noise. In the causal example, measuring the common cause or randomizing the intermediate variable recovers the positive response; the observationally equivalent models in Eq. (27) specify an intervention that distinguishes their responses. In the measurement example, the candidate mechanisms share a linear response and first separate through the quadratic saturation term. Their per-query information consequently scales as x4x^4, giving an x−4x^{-4} query requirement at fixed noise. Increasing the control from 0.05 to 1 reduces the required budget from 332,510 to eight measurements for the stated confidence targets. The gain follows from exposing the distinguishing nonlinear correction. Specifying the accessible control range, noise, and candidate responses thus connects a proposed mechanism to a quantitative measurement decision.

Conclusion

Validity records for individual observables make the scope of a physical explanation quantitatively testable. We linked reductions to declared scales and tolerances, separated certified domains from actual validity, and derived complementary displacement and velocity certificates. Exact dimer relations and paired neural tests show why energy homogeneity protects scale transfer while joint acceptance of gap and response identifies the coupling crossover as a separate learning target. A known causal model separates observational association from intervention response. The order at which two mechanisms separate then determines an exact fixed control query cost under Gaussian noise.

These results support evaluation that keeps prediction, approximation, and mechanism identification individually accountable. The analytical references and structured development interface connect each validity decision to a physical quantity, an acceptance criterion, and a reproducible calculation, providing a basis for evaluating physical reasoning in scientific AI.

Data and code availability

The data, code, trained model weights, task bank, and scoring implementation supporting this study are available from the corresponding author on request.

AI assistance disclosure

Generative AI assisted literature discovery, analytical development, code implementation, figure preparation, and manuscript drafting. Analytical expressions, explicit structural equations, and independently checked numerical calculations define the reference results. The authors take responsibility for the analysis, numerical results, references, and final manuscript.

Proof and scope of the residual certificate

For ε<1/4\varepsilon< 1/4, the Green function is

Gε(τ)=e−aτ−e−bτε(b−a),τ≥0,(31)G_{\varepsilon}(\tau)=\frac{e^{-a\tau}-e^{-b\tau}}{\varepsilon(b-a)},\qquad\tau\ge0, \tag*{(31)}

where aa, bb are defined in Eq. (5) and ab=1/εab=1/\varepsilon. It is nonnegative and its integral is one. Its maximum occurs at τ∗=log⁡(b/a)/(b−a)\tau_*=\log(b/a)/(b-a), giving Eq. (5). At critical damping, G1/4=4τe−2τG_{1/4}=4\tau e^{-2\tau} and the same statements hold by continuity.

Let ϕ0\phi_0 be the exact solution with initial displacement one and initial velocity zero. It decreases from one to zero in this parameter range. The error satisfies

y−x=e0ϕ0+e1εGε+∑jGε∗Rj.(32)y-x=e_0\phi_0+e_1\varepsilon G_\varepsilon+\sum_j G_\varepsilon* R_j . \tag*{(32)}

The two convolution bounds ∥G∗Rj∥∞≤∥Rj∥∞\lVert G*R_j\rVert_\infty\le\lVert R_j\rVert_\infty and ∥G∗Rj∥∞≤Mε∥Rj∥1\lVert G*R_j\rVert_\infty\le M_\varepsilon\lVert R_j\rVert_1 establish Proposition 1. The nonnegative kernel on 0<ε≤1/40<\varepsilon\le1/4 establishes the stated scope of this certificate.

For y1y_1, the residual separates as

Rs=ε2(1+ε)(2+ε)e−(1+ε)τ,R_s=\varepsilon^2(1+\varepsilon)(2+\varepsilon)e^{-(1+\varepsilon)\tau},
Rf=−εe−τ/ε.(33)R_f=-\varepsilon e^{-\tau/\varepsilon}. \tag*{(33)}

Bounding RsR_s in maximum norm and RfR_f in integrated norm, together with the initial velocity error, gives Eq. (9). The fast residual has amplitude O(ε)O(\varepsilon) but integrated magnitude O(ε2)O(\varepsilon^2), explaining the importance of matching the residual norm to the eliminated process.

For velocity, ϕ0′=−Gε\phi_0'=-G_\varepsilon and differentiation of the convolution gives

y′−x′=−e0Gε+e1εGε′+∑jGε′∗Rj.(34)y'-x'=-e_0G_\varepsilon+e_1\varepsilon G_\varepsilon'+\sum_j G_\varepsilon' * R_j . \tag*{(34)}

The kernel is unimodal and vanishes at both endpoints. The following kernel norms are taken over [0,∞)[0,\infty); residual norms remain over [0,T][0,T].

∥Gε′∥1=2Mε,∥Gε′∥∞=ε−1.(35)\lVert G_\varepsilon'\rVert_1=2M_\varepsilon,\qquad\lVert G_\varepsilon'\rVert_\infty=\varepsilon^{-1}. \tag*{(35)}

The positive derivative maximum is Gε′(0)=1/εG_\varepsilon'(0)=1/\varepsilon. Its negative extremum occurs at 2log⁡(b/a)/(b−a)2\log(b/a)/(b-a) and has magnitude smaller than 1/ε1/\varepsilon; the critical case follows by continuity. Young’s convolution inequalities now establish Proposition 2. For y1y_1, use the maximum norm for RsR_s and the integrated norm for RfR_f. Their contributions are 2Mεε2(1+ε)(2+ε)2M_\varepsilon\varepsilon^2(1+\varepsilon)(2+\varepsilon) and ε\varepsilon, respectively. Together with ∣e1∣=2ε+ε2\lvert e_1\rvert=2\varepsilon+\varepsilon^2, these give Eq. (11).

Independent information in the crossover function

Write J(t,U)=Uf(r)J(t,U)=Uf(r), r=∣t∣/Ur=\lvert t\rvert/U, with f(r)>0f(r)>0. Let a smooth nonnegative bump h(r)h(r) have compact support in a test interval disjoint from the training interval. Then

J~(t,U)=U[f(r)+ch(r)],c>0,(36)\widetilde{J}(t,U)=U[f(r)+ch(r)],\qquad c>0, \tag*{(36)}

has the same homogeneity, hopping sign invariance on the single bond, and training values as JJ. Since the bump vanishes near zero and infinity, both asymptotic regimes also remain unchanged. Varying cc changes the value and derivative within its support while preserving these constraints. This family explicitly isolates the information supplied by crossover observations from that supplied by symmetry, training values, and limiting laws.

Derivation of the Gaussian query requirement

At a fixed xx, the log likelihood ratio from one observation is

ℓ=(y−μ2)2−(y−μ1)22σ2.(37)\ell=\frac{(y-\mu_2)^2-(y-\mu_1)^2}{2\sigma^2}. \tag*{(37)}

Under H1H_1, it is normal with mean Δ2/(2σ2)\Delta^2/(2\sigma^2) and variance Δ2/σ2\Delta^2/\sigma^2; under H2H_2, reverse its sign. The distributions of evidence in favor of the true hypothesis are identical. Summing BB independent observations and using equal priors gives

P{p(Htrue∣DB)≥1−α}=Φ(BΔ2σ−σLαBΔ).(38)\mathbb{P}\{p(H_{\mathrm{true}}\mid D_B)\ge1-\alpha\}=\Phi\left(\frac{\sqrt{B}\Delta}{2\sigma}-\frac{\sigma L_\alpha}{\sqrt{B}\Delta}\right). \tag*{(38)}

Set the argument at least z1−βz_{1-\beta} and solve the resulting quadratic in BΔ/σ\sqrt{B}\Delta/\sigma to obtain Eq. (29). The result quantifies the sampling probability of attaining the declared posterior threshold on the true mechanism.

Training and uncertainty

Both networks use double precision and natural logarithms. Input centering and scaling are fitted on the training samples, and logarithmic targets retain their original scale. Training uses an L2L_2 penalty of 10−510^{-5}, a gradient tolerance of 10−910^{-9}, and at most 2000 iterations. Each repeated fit varies the physical samples and initialization, with both representations using the same training and test points. This pairing compares the two representations under matched physical conditions.

The oscillator and dimer curves describe deterministic changes of physical parameters. The network percentiles describe training and sampling variation across repeated fits. Causal intervals summarize finite samples from the stated structural model, and measurement intervals summarize independent observation noise under the two hypotheses. Independent simulation with 50,000 balanced noise sequences verifies the analytical success curve over budgets 1–8, with maximum probability discrepancy 0.00389. These uncertainty descriptions give each plotted range a specific statistical interpretation.

Executable development interface

Each of four families has 32 base systems divided into 16 training, eight development, and eight test groups. Across families this gives 64, 32, and 32 groups, corresponding to 128, 64, and 64 prompts. Paired variants remain in the same split. Oscillator variants rescale all equation coefficients, and dimer variants rescale energies and reverse the hopping sign on the single bond. Causal and measurement variants test consistency under equivalent wording and parameter ordering. The oscillator is stratified into certified points, accepted points with a bound above tolerance, rejected points with a bound above tolerance, and points outside the proof domain. The dimer is stratified by joint gap and response acceptance, gap acceptance alone, and rejection of both. Each split contains every realizable stratum. For each sampled parameter pp, the boundary margin satisfies

min⁡b∈B∣p−b∣b≥10−3,(39)\min_{b\in\mathcal{B}}\frac{\lvert p-b\rvert}{b}\ge10^{-3}, \tag*{(39)}

where B\mathcal{B} contains the relevant positive boundaries. For the oscillator, p=εp=\varepsilon and the boundaries are the leading displacement certificate threshold, the actual displacement threshold on [0,8][0,8], and 1/41/4. For the dimer, p=rp=r and the boundaries are the gap and response acceptance thresholds. The generator uses unrounded boundary values; the sampling configuration specifies ranges, logarithmic sampling, boundary adjacent points, and randomization.

As summarized in Table 5, 16 oscillator prompts distinguish actual acceptance from a sufficient guarantee, and 22 dimer prompts distinguish energy acceptance from response acceptance. These cases make the two central validity decisions directly testable within every data split.

FamilyPhysical regionTrainingDevelopmentTest
OscillatorCertified422
OscillatorValid, bound above tolerance422
OscillatorInvalid, bound above tolerance422
OscillatorOutside proof domain422
DimerGap and response accepted632
DimerGap accepted, response rejected533
DimerGap and response rejected523

Table 5. Stratified coverage of the oscillator and dimer task families, counted by base system. Each system contributes two equivalent prompts.

Let task ii have mim_i required fields and cif∈{0,1}c_{if}\in\{0,1\} indicate a correct field. Let ai=∏fcifa_i=\prod_f c_{if} be strict task correctness, and let N=256N=256, G=128G=128. The evaluation scores are defined as

Sfield=∑i,fcif∑imi,Sstrict=1N∑iai,S_{\mathrm{field}}=\frac{\sum_{i,f}c_{if}}{\sum_i m_i},\qquad S_{\mathrm{strict}}=\frac{1}{N}\sum_i a_i,
Sjoint=1G∑g=1Gag0ag1,Sequiv=1G∑g=1Gqg,(40)S_{\mathrm{joint}}=\frac{1}{G}\sum_{g=1}^{G}a_{g0}a_{g1},\qquad S_{\mathrm{equiv}}=\frac{1}{G}\sum_{g=1}^{G}q_g, \tag*{(40)}

where qg=1q_g=1 when all paired answers agree after mapping to the common representation. The first score weights the 1280 required fields equally, the second requires every field of a task to be correct, and the third requires both variants of a system to be correct. The fourth measures equivalence consistency after validating field types and permitted certificate states. Reporting the last two together separates preserved relations from agreement with the analytical reference. Field scores are also grouped into validity decisions (256 fields), numerical inference (384), qualitative regime identification (64), and structural rules (576), giving each physical obligation an explicit contribution to the evaluation.

A numerical answer aa matches reference bb when ∣a−b∣≤10−5+0.01∣b∣\lvert a-b\rvert\le10^{-5}+0.01\lvert b\rvert. Paired equivalence uses the symmetric rule ∣a−b∣≤10−5+0.01max⁡(∣a∣,∣b∣)\lvert a-b\rvert\le10^{-5}+0.01\max(\lvert a\rvert,\lvert b\rvert). Boolean answers use either true or false values. Certificate status distinguishes a sufficient guarantee, a bound above tolerance, and parameters outside the proof domain. Missing answers, invalid types, and nonfinite numbers receive zero for the corresponding fields, and all tasks remain in the score denominators. Different derivations are accepted when they produce the requested quantities and decisions. Explanatory reasoning and proposed alternative reductions can be assessed alongside these structured outputs.

Representative physical tasks

Oscillator. For m=0.055222103m=0.055222103, γ=1.2826983\gamma=1.2826983, k=1.4897249k=1.4897249, evaluate the leading and matched reductions on [0,8][0,8] at tolerance 0.05. The reference gives ε=0.05\varepsilon=0.05, nonoscillatory motion, and displacement error powers 1 and 2. The leading displacement is accepted with ex=0.04008e_x=0.04008, while its bound b0=0.09452b_0=0.09452 is above tolerance.

Hubbard dimer. For t=0.077362401t = 0.077362401, U=0.77362401U = 0.77362401, the reference gives r=0.1r = 0.1, J/U=0.038516481J/U = 0.038516481, leading hopping power 2, and invariance under hopping sign reversal. The second order gap error eJ=0.03852e_J = 0.03852 satisfies tolerance 0.05; its response error en=0.07152e_n = 0.07152 requires refinement.

Causal response. In Z=aP+σZηZZ = aP + \sigma_Z\eta_Z and Y=bZ+cP+0.1ηYY = bZ + cP + 0.1\eta_Y, take independent standard normal exogenous variables and a=1.1302027a = 1.1302027, b=1.3579449b = 1.3579449, c=−2.8713394c = -2.8713394, σZ=0.15652024\sigma_Z = 0.15652024. The observational slope is −1.1347991-1.1347991 and the intervention slope is 1.35794491.3579449. Adjustment for measured PP recovers the positive response.

Measurement design. Compare y=xy = x and y=x/(1+x/s)y = x/(1 + x/s) with equal priors and s=1.837815s = 1.837815, σ=0.37099509\sigma= 0.37099509, x∈[0,1.1757526]x \in[0, 1.1757526]. The maximum information control is the upper endpoint. The saturated response lies below the linear response for x>0x > 0, and their difference vanishes as x→0x \to0. At x=0x = 0 the likelihoods coincide and the posterior odds remain unchanged.

These analytical answers satisfy all required fields and receive full field and strict correctness scores. Paired variants yield the same answers after the specified physical mapping.

References

References

  1. [1]G. Carleo, I. Cirac, K. Cranmer, et al. “Machine learning and the physical sciences”. In: Reviews of Modern Physics 91.4, 045002 (2019). DOI: 10.1103/RevModPhys.91.045002.DOI
  2. [2]H. Wang, T. Fu, Y. Du, et al. “Scientific discovery in the age of artificial intelligence”. In: Nature 620.7972 (2023), pp. 47–60. DOI: 10.1038/s41586–023–06221–2.DOI
  3. [3]Y. Zhang, S. A. Khan, A. Mahmud, et al. “Exploring the role of large language models in the scientific method: from hypothesis to discovery”. In: npj Artificial Intelligence 1.1, 14 (2025). DOI: 10.1038/s44387–025–00019–5.
  4. [4]H. Pan, N. Mudur, W. Taranto, et al. “Quantum many-body physics calculations with large language models”. In: Communications Physics 8.1 (2025), p. 49. DOI: 10.1038/s42005–025–01956–y.
  5. [5]X. Zhang, Y. Dong, Y. Wu, et al. “PhysReason: A Comprehensive Benchmark towards Physics-Based Reasoning”. In: Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2025, pp. 16593–16615. DOI: 10.18653/v1/2025.acl-long.811.DOI
  6. [6]S. Qiu, S. Guo, Z.-Y. Song, et al. PHYBench: Holistic Evaluation of Physical Perception and Reasoning in Large Language Models. 2025. arXiv: 2504.16074.arxiv.org/abs/2504.16074
  7. [7]W. Wang, D. Huang, J. Li, et al. CMPhysBench: A Benchmark for Evaluating Large Language Models in Condensed Matter Physics. 2025. arXiv: 2508.18124.arxiv.org/abs/2508.18124
  8. [8]M. Zhu, M. Tian, X. Yang, et al. Probing the Critical Point (CritPt) of AI Reasoning: a Frontier Physics Research Benchmark. 2025. arXiv: 2509.26574.arxiv.org/abs/2509.26574
  9. [9]T. Miao, W. Jin, M. Zhang, et al. PRL-Bench: A Comprehensive Benchmark Evaluating LLMs’ Capabilities in Frontier Physics Research. 2026. arXiv: 2604.15411.arxiv.org/abs/2604.15411
  10. [10]H. W. de Regt and D. Dieks. “A Contextual Approach to Scientific Understanding”. In: Synthese 144.1 (2005), pp. 137–170. DOI: 10.1007/s11229–005–5000–4.DOI
  11. [11]E. Sullivan. “Understanding from Machine Learning Models”. In: The British Journal for the Philosophy of Science 73.1 (2022), pp. 109–133. DOI: 10.1093/bjps/axz035.
  12. [12]E. Duede. “Deep Learning Opacity in Scientific Discovery”. In: Philosophy of Science 90.5 (2023), pp. 1089–1099. DOI: 10.1017/psa.2023.8.DOI
  13. [13]Y. Xu, Y. Liu, Z. Gao, et al. PhySense: Principle-Based Physics Reasoning Benchmarking for Large Language Models. 2025. arXiv: 2505.24823.arxiv.org/abs/2505.24823
  14. [14]Y. Chen, P. Pikos, M. Ostaszewski, et al. PhysGym: Benchmarking LLMs in Interactive Physics Discovery with Controlled Priors. 2025. arXiv: 2507.15550.arxiv.org/abs/2507.15550
  15. [15]T. Zheng, K. K.-W. Tam, N. H.-N. K. Nguyen, et al. NewtonBench: Benchmarking Generalizable Scientific Law Discovery in LLM Agents. 2025. arXiv: 2510.07172.arxiv.org/abs/2510.07172
  16. [16]P. Shojaee, N.-H. Nguyen, K. Meidani, et al. LLM-SRBench: A New Benchmark for Scientific Equation Discovery with Large Language Models. 2025. arXiv: 2504.10415.arxiv.org/abs/2504.10415
  17. [17]M. L. Wiemann, L. M. Smith, P. Melchior, et al. DiscoverPhysics: Benchmarking LLMs for Out-of-the-Box Scientific Thinking. 2026. arXiv: 2605.26087.arxiv.org/abs/2605.26087
  18. [18]J. Yang, O. Venkatachalam, M. Kianezhad, et al. Think like a Scientist: Physics-guided LLM Agent for Equation Discovery. 2026. arXiv: 2602.12259.arxiv.org/abs/2602.12259
  19. [19]Y. Zhang, X. Guo, R. Pan, et al. “Decoupled Physical Modeling and Execution for Physics Reasoning”. In: Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2026, pp. 32112–32131. G. E. Karniadakis, I. G. Kevrekidis, L. Lu, et al. “Physics-informed machine learning”. In: Nature Reviews Physics 3.6 (2021), pp. 422–440. DOI: 10.1038/s42254-021-00314-5.
  20. [21]E. Buckingham. “On Physically Similar Systems; Illustrations of the Use of Dimensional Equations”. In: Physical Review 4.4 (1914), pp. 345–376. DOI: 10.1103/PhysRev.4.345.DOI
  21. [22]S.-M. Udrescu and M. Tegmark. “AI Feynman: A physics-inspired method for symbolic regression”. In: Science Advances 6.16 (2020), eaay2631. DOI: 10.1126/sciadv.aay2631.DOI
  22. [23]S. L. Brunton, J. L. Proctor, and J. N. Kutz. “Discovering governing equations from data by sparse identification of nonlinear dynamical systems”. In: Proceedings of the National Academy of Sciences 113.15 (2016), pp. 3932–3937. DOI: 10.1073/pnas.1517384113.DOI
  23. [24]S. Greydanus, M. Dzamba, and J. Yosinski. “Hamiltonian Neural Networks”. In: Advances in Neural Information Processing Systems. Vol. 32. Curran Associates, Inc., 2019. URL: https://proceedings.neurips.cc/paper/2019/hash/26cd8ecadce0d4efd6cc8a8725cbd1f8 - Abstract.html.arxiv.org/abs/1906.01563
  24. [25]M. Cramer, S. Greydanus, S. Hoyer, et al. Lagrangian Neural Networks. 2020. arXiv: 2003.04630.
  25. [26]V. G. Satorras, E. Hoogeboom, and M. Welling. “E(n) Equivariant Graph Neural Networks”. In: Proceedings of the 38th International Conference on Machine Learning. Vol. 139. Proceedings of Machine Learning Research. PMLR, 2021, pp. 9323–9332. URL: https://proceedings.mlr.press/v139/satorras21a.html.
  26. [27]M. Raissi, P. Perdikaris, and G. Karniadakis. “Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations”. In: Journal of Computational Physics 378 (2019), pp. 686–707. DOI: 10.1016/j.jcp.2018.10.045.DOI
  27. [28]A. Krishnapriyan, A. Gholami, S. Zhe, et al. “Characterizing possible failure modes in physics-informed neural networks”. In: Advances in Neural Information Processing Systems. Vol. 34. Curran Associates, Inc., 2021, pp. 26548–26560. URL: https://proceedings.neurips.cc/paper/2021/hash/df438e5206f31600e6ae4af72f2725f1 - Abstract.html.arxiv.org/abs/2109.01050
  28. [29]Y. Cao and L. Petzold. “A Posteriori Error Estimation and Global Error Control for Ordinary Differential Equations by the Adjoint Method”. In: SIAM Journal on Scientific Computing 26.2 (2004), pp. 359–374. DOI: 10.1137/S1064827503420969.DOI
  29. [30]J. Birrell. “A Posteriori Error Bounds for Two Point Boundary Value Problems: A Green’s Function Approach”. In: Journal of Computational Dynamics 2.2 (2015), pp. 143–164. DOI: 10.3934/jcd.2015001.DOI
  30. [31]J. PEARL. “Causal diagrams for empirical research”. In: Biometrika 82.4 (1995), pp. 669–688. DOI: 10.1093/biomet/82.4.669.DOI
  31. [32]URL: https://aclanthology.org/2026.emnlp-main.1405/. the IEEE 109.5 (2021), pp. 612–634. DOI: 10.1109/JPROC.2021.3058954.DOI
  32. [33]J. Peters, P. Bühlmann, and N. Meinshausen. “Causal Inference by using Invariant Prediction: Identification and Confidence Intervals”. In: Journal of the Royal Statistical Society Series B: Statistical Methodology 78.5 (2016), pp. 947–1012. DOI: 10.1111/rssb.12167.
  33. [34]P. Virtanen, R. Gommers, T. E. Oliphant, et al. “SciPy 1.0: fundamental algorithms for scientific computing in Python”. In: Nature Methods 17.3 (2020), pp. 261–272. DOI: 10.1038/s41592-019-0686-2.DOI
  34. [35]D. J. Carrascal, J. Ferrer, J. C. Smith, et al. “The Hubbard dimer: a density functional case study of a many-body problem”. In: Journal of Physics: Condensed Matter 27.39 (2015), p. 393001. DOI: 10.1088/0953-8984/27/39/393001.DOI
  35. [36]J. R. Schrieffer and P. A. Wolff. “Relation between the Anderson and Kondo Hamiltonians”. In: Physical Review 149.2 (1966), pp. 491–492. DOI: 10.1103/PhysRev.149.491.DOI
  36. [37]D. C. Liu and J. Nocedal. “On the limited memory BFGS method for large scale optimization”. In: Mathematical Programming 45.1-3 (1989), pp. 503–528. DOI: 10.1007/bf01589116.
  37. [38]F. Pedregosa, G. Varoquaux, A. Gramfort, et al. “Scikit-learn: Machine Learning in Python”. In: Journal of Machine Learning Research 12.85 (2011), pp. 2825–2830. URL: https://jmlr.org/papers/v12/pedregosa11a.html.arxiv.org/abs/1201.0490
  38. [39]K. Chaloner and I. Verdinelli. “Bayesian Experimental Design: A Review”. In: Statistical Science 10.3 (1995), pp. 273–304. DOI: 10.1214/ss/1177009939.DOI
  39. [40]E. B. Wilson. “Probable Inference, the Law of Succession, and Statistical Inference”. In: Journal of the American Statistical Association 22.158 (1927), pp. 209–212. DOI: 10.1080/01621459.1927.10502953.DOI

Paper details

vlnc:W_cf6tkmzg
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 10

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Generative AI

View statement
Looking for related papers…
Contents