WorksEngineering

The Novelty Horizon H_N: Why Large Language Models Fail to Generate Scientific Data beyond Their Training Distribution

Alex Mwololo Kimuya, Mary Mwadulo, Dickson Mwenda Kinyua

Published Nov 03, 2026Fresh — added to the Hub within the last 30 daysv1103 min read

AI contributions

Not recorded for this paper

Abstract

Large Language Models (LLMs) lack physical grounding and extrapolation. This paper formalises the Novelty Horizon as tasks needing extrapolation beyond training support, causal reasoning absent from learned correlations, or post-frontier information. A forensic analysis tests LLMs on unguided, guided, and ground-truth data from an underrepresented diode-clamped short-circuit experiment using temporal, spectral, and statistical validation against a first-principles Modified Ohm's Law. Both script-based approaches fail catastrophically with physical and spectral inconsistencies, extreme statistical pathologies, zero Generative Fidelity Score, and linear-regime outputs. Direct prompting corroborates distinct failure modes. This paper introduces the Past Information Boundary and physics-based authentication. LLM data generation cannot extrapolate past the Novelty Horizon essential for novel discovery. Hybrid neuro-symbolic systems must preserve human causal reasoning and experimental design.

Introduction

The integration of Artificial Intelligence (AI) into the scientific method represents one of the most significant paradigm shifts in modern research practice. Across disciplines, from computational biology to materials science, AI tools are accelerating discovery by identifying patterns in vast datasets, optimizing experimental designs, and simulating complex systems [1, 2, 3, 4, 5, 6, 7]. The integration of large language models into the scientific workflow has progressed from a theoretical possibility to an operational reality with remarkable speed. This transformative potential is perhaps most publicly visible in the rapid adoption of Large Language Models (LLMs), whose fluent text generation promises to assist in literature synthesis, hypothesis generation, and the drafting of research communications [8, 9, 10]. As Zhang et al. [11] note in their comprehensive review, LLMs are now deeply involved in key stages of the scientific cycle, including “experimental design, data analysis, and workflows”, marking a shift from theoretical possibility to operational reality across fields such as chemistry and biology. A narrative has emerged suggesting that these models are not merely tools but potential collaborators – or even autonomous agents – capable of contributing to the substantive, creative core of scientific work. This narrative often leans on the impressive ability of LLMs to manipulate scientific terminology and reproduce common logical structures found in the academic literature, creating an illusion of deep, contextual understanding [12, 12, 13, 14].

Beneath this veneer of competence lies a foundational and unresolved problem concerning the nature of the knowledge LLMs possess. As critics have rigorously argued, the operation of these models is fundamentally grounded in statistical pattern recognition across the corpora on which they are trained. They generate plausible sequences of tokens by modeling the probability distribution of human language, an engineering feat that does not equate to grasping meaning, establishing truth, or constructing causal models of the world [15, 16, 17, 18]. The models, in this view, are sophisticated correlational engines, brilliant at interpolation within the bounds of their training data but devoid of the embodied, precarious, and participatory understanding that characterizes human cognition and scientific reasoning. This critical perspective forces a crucial question for the sciences: Can a system that excellently predicts the next word in a sentence also generate novel, valid, and empirically faithful scientific data? The conflation of linguistic plausibility with scientific rigor poses a direct threat to research integrity, as plausible yet physically impossible outputs – hallucinations – may be woven seamlessly into the fabric of AI-assisted research.

The existing literature reveals a stark gap in directly testing the limits of LLMs in the empirical, data-generating domain of experimental science. While studies have documented their proficiency or failure in tasks like answering exam questions, summarizing papers, or writing code, the assessment often remains within the discursive realm [19, 20, 21, 22]. Investigations into whether LLMs learn human-interpretable concepts from data, such as the work of [23, 24, 25], probe the internal representations of models but do not test their output against physical ground truth in a controlled experiment. Similarly, research on human–AI collaboration, like that of Nguyen et al. [26] [26], valuably maps interaction patterns but typically assesses the final human-authored product, not the AI’s raw contribution to empirical data. What is missing, and what this paper provides, is a forensic, physics-grounded analysis that moves beyond textual analysis to a fundamental test: pitting LLM-generated numerical data against data produced by a real physical experiment, using the inviolable laws of nature as the ultimate validator. This approach shifts the question from “Does this text sound scientific?” to “Does this data obey physics?”.

This gap is not merely technical but epistemological. Recent studies demonstrating LLM generated synthetic clinical data that replicates marginal statistical distributions [27, 28, 29] have rightly highlighted accessibility and efficiency gains. Yet the very same evidence exposes a troubling trajectory: as LLM use extends from literature synthesis into the generation of empirical data itself, statistical indistinguishability is increasingly treated as a proxy for empirical validity. This paper extends this critique by systematically interrogating that assumption-arguing that distributional similarity alone cannot certify causal physical fidelity, and that the research community must therefore develop forensic, physics grounded validation frameworks before trusting LLM generated data as a foundation for scientific inference. The epistemological critique that motivates this forensic investigation is developed at length in Appendix § 11, where we examine why statistical indistinguishability has been mistakenly treated as a proxy for empirical validity, and why this error necessitates the physics-grounded authentication framework that the following sections execute.

The central objective of this paper is to establish and demonstrate the intrinsic limitations of autoregressive LLMs in generating scientifically valid empirical data. We posit that these models are constrained by what we term the Past Information Boundary, $B_{P}$ (formally defined in Definition 2.1, § 2.6)-the ordered triple $(t_{p}, \Omega_{\mathrm{train}}, C_{\mathrm{train}})$ representing the temporal frontier $t_{p}$ (latest timestamp in training), the support $\Omega_{\mathrm{train}}$ of the empirical training distribution, and the set $C_{\mathrm{train}}$ of learned conditional probability distributions. Beyond this boundary, the model’s capacity for reliable extrapolation fails, giving rise to the Novelty Horizon, $H_{N}$ (Definition 2.2). To test this, we designed a multi faceted investigation comprising two distinct experimental frameworks: one where the LLM generates data by first producing a simulation script (the focus of this main paper), and a parallel framework, detailed in the supplementary information, where the LLM is prompted to output numerical data directly. This dual framework approach allows us to isolate whether the observed failures are specific to the code generation paradigm or represent a fundamental limitation of the models themselves. Within the script based framework central to this paper, we employ a tripartite methodology, generating three parallel datasets: (i) from an unguided LLM given only basic circuit parameters, (ii) from the same LLM explicitly guided with the full experimental procedure, and (iii) from an actual physical measurement serving as the ground truth. The paper methodology subjects all three datasets to a multi layered analytical framework that scrutinizes temporal statistics, spectral signatures, adherence to conservation laws, and, most decisively, fit to a first principles derived Modified Ohm’s Law that describes the system’s physics.

The contributions of this work are threefold and establish a new standard for evaluating AI in empirical science. First, we introduce and formally define the theoretical constructs of the “Past Information Boundary, $B_{P}$ (Definition 2.1)” and the “Novelty Horizon, $H_{N}$ (Definition 2.2)”. This framework does not merely state that LLMs fail at data generation, but precisely predicts and explains why they succeed in interpolative tasks within their training distribution yet fail catastrophically at the genuine scientific extrapolation required for novel experimental configurations. This paper is a focused test at this horizon. As formalised in Definition 2.2, a task lies beyond $H_{N}$ if it requires extrapolation outside $\operatorname{conv}(\Omega_{\mathrm{train}})$ (Condition 2.1), causal reasoning not encoded in $C_{\mathrm{train}}$ (Condition 2.2), or reasoning about post- $t_{p}$ phenomena (Condition 2.3). The diode-clamped short-circuit experiment satisfies all three conditions, making it an ideal test case. Second, we develop and execute a novel forensic methodology for the “physics-based authentication” of scientific data. This methodology moves beyond statistical similarity to test for causal-physical consistency, exposing artifacts-such as perfect algebraic closure or incorrect noise spectra-that betray algorithmic generation. Third, we propose a quantitative Generative Fidelity Score (GFS), synthesizing multiple diagnostic metrics into a single measure of how well a dataset conforms to the expected signatures of physical measurement and governing laws. The paper results provide unambiguous, quantitative evidence that LLMs, regardless of prompting strategy, cannot generate empirically valid data for tasks beyond $H_{N}$ , revealing their fundamental epistemological limitation as tools for scientific discovery.

The remainder of this paper is structured to guide the reader from theoretical foundations to definitive conclusions. In § 2, we delineate the architectural and theoretical constraints of LLMs, deriving from first principles the inherent limitations that predispose them to the failures our experiment reveals. § 3 details our foundational framework, including the derivation of the Modified Ohm’s Law for the test system and the design of our three test cases. § 4 describes our multi-layered forensic methodology for data analysis and validation. § 5 presents the results, systematically comparing the LLM-generated data against the physical ground truth across every diagnostic dimension. In § 6, we discuss the profound implications of these findings, interpreting the failures through our theoretical framework and exploring the consequences for researchers, educators, and the broader scientific enterprise. Finally, § 7 offers the concluding remarks, synthesizing the evidence that LLMs are bounded by their Novelty Horizon, $H_N$ (Definition 2.2). We affirm the indispensable role of human scientists in conceiving experiments that push beyond this horizon and define the proper path for Augmented Intelligence in research.

Comparative Framework: Standard Versus Modified Ohm’s Law

Before proceeding to the theoretical development of LLM constraints, it is instructive to establish a comparative baseline between the well-represented Standard Ohm’s Law and the Modified formulation that governs the experimental system under study. This comparison serves two purposes. First, it clarifies why the linear regime (governed by Standard Ohm’s Law) lies inside the Novelty Horizon-where LLMs might reasonably interpolate-while the nonlinear clamped regime lies outside it. Second, it provides a clear framework for interpreting the regime-dependent failures documented in § 5: if LLMs perform adequately in the linear regime but fail catastrophically in the nonlinear regime, this demonstrates that their competence is bounded by the convex hull of training examples $\operatorname{conv}(\Omega_{\mathrm{train}})$ (Condition 2.1 of Definition 2.2). Table 1 (additional details on these theoretical models established in [30]) presents a ten-dimensional comparison of the Standard and Modified Ohm’s Law formulations, highlighting their governing equations, resistance behavior, temporal resolution, fault-modeling suitability, predictive limits, fitting parameters, power prediction accuracy, interpretability, experimental alignment, and fundamental limitations.

Feature	Standard Ohm’s Law	Modified Ohm’s Law
Governing Equation	$I = V/R$	$I_{\mathrm{modified}} = ae^{R_{\mathrm{mant}}/R_0}$
Resistance Behavior	Static or externally supplied	Implicit via $R_{\mathrm{short}}$
Temporal Resolution	None	Indirect via $R_{\mathrm{short}}$
Suitability for Fault Modeling	Limited	Moderate-captures early ramp-up
Predictive Limits	Fails as $R \to0$	Diverges for long durations
Use of Fitting Parameters	None	Parameter $a$ from baseline
Power Prediction Accuracy	Reasonable under static $R$	Sometimes paradoxical at high $I$
Interpretability	Direct and intuitive	Physically motivated; exponential mapping
Experimental Alignment	Matches steady-state	Aligns in early transient window
Limitations	Ignores resistance evolution	Misrepresents energy at long $t$

Table 1. Comparative Features of Ohmic Formulations

This comparative framework establishes that Standard Ohm’s Law-ubiquitously represented in training corpora across introductory physics textbooks, engineering problem sets, and scientific literature lies securely within the convex hull of training examples $\operatorname{conv}(\Omega_{\mathrm{train}})$ . The Modified Ohm’s Law, by contrast, introduces nonlinearity through the exponential relationship $I_{\mathrm{modified}} = ae^{R_{\mathrm{shart}}/R_0}$ , which requires understanding of semiconductor junction physics, temperature-dependent thermal voltage $V_T = k_B T/q$ , and ideality factors—concepts whose specific conjunction in time-resolved short-circuit measurement lies outside $\operatorname{conv}(\Omega_{\mathrm{train}})$ . This contrast motivates the experimental design: by comparing LLM performance across these two regimes, we can directly test whether the model’s competence degrades catastrophically at the Novelty Horizon.

The Tripartite Framework: Isolating Layers of Generative Failure

A second critical clarification concerns the tripartite comparison structure employed in this study. The framework compares three datasets: (i) unguided LLM generation (Case 1), (ii) procedurally guided LLM generation (Case 2), and (iii) experimental ground truth (Case 3). Each layer in this framework isolates a distinct aspect of the generative process and supports a specific claim about extrapolation.

Layer 1. Unguided Generation (Case 1) isolates the LLM’s intrinsic capacity for statistical extrapolation from basic electrical concepts. By providing only nominal circuit parameters (2.5V/0.25A and 5.0V/0.5A) and no procedural context, this layer tests whether the model can infer the correct measurement dynamics, noise characteristics, and nonlinear clamping behavior solely from patterns in its training corpus $C_{\mathrm{train}}$ . Failure at this layer—manifesting as perfect algebraic closure, incorrect spectral signatures, or regime misplacement—indicates that the task lies beyond $H_N$ under Condition 2.1 (extrapolation outside $\operatorname{conv}(\Omega_{\mathrm{train}})$ ) and Condition 2.2 (causal reasoning not encoded in $C_{\mathrm{train}}$ ).

Layer 2. Guided Generation (Case 2) isolates the LLM’s capacity for informed pattern matching when provided with the complete experimental procedure, including circuit schematics, component specifications (1N5408 diodes, ACS712T sensor), and the data averaging protocol (10,000 samples). This layer tests whether additional contextual information can steer the model toward physical plausibility by effectively expanding the conceptual frontier $\Omega_{\mathrm{train}}$ through the inclusion of procedural descriptions. If the model still fails—producing extreme kurtosis ( $\gamma_2 > 65$ ), regime misplacement (all data in $|v_s| < 0.3\,\mathrm{V}$ ), or deterministic residual-current correlations ( $\rho\approx0.999$ )—this demonstrates that the failure is not merely informational but structural: the model lacks the causal reasoning mechanisms required by Condition 2.2, regardless of how much procedural context is supplied.

Layer 3. Experimental Ground Truth (Case 3) provides the immutable physical benchmark against which both generative approaches are judged. This dataset embodies the true causal structure $\Phi$ (the Modified Ohm’s Law with independent measurement noise), the authentic spectral signatures of the ACS712T Hall-effect sensor ( $1/f^\alpha$ noise with $\alpha\approx1$ ), and the correct regime-dependent physics (nonlinear clamping at $|v_s| \ge0.3\,\mathrm{V}$ ). The distance between Cases 1/2 and Case 3 in the multi-dimensional space of diagnostic metrics quantifies the LLM’s inability to cross the Novelty Horizon.

The reasoning that links these layers to the central argument about extrapolation is as follows. If the LLM’s failure were merely a matter of insufficient procedural detail, then providing the full experimental procedure (Case 2) should substantially improve performance, yielding data that approximates the statistical and physical characteristics of Case 3. If, however, the failure stems from a fundamental inability to model causal structures not encoded in $C_{\mathrm{train}}$ —specifically, the independence of voltage and current measurements, the continuous physics of diode clamping, and the colored noise spectra of physical sensors—then even maximal procedural guidance will not rescue performance. The empirical results (detailed in § 5.1, ??, and 4.5) decisively support the latter interpretation: Case 2 exhibits catastrophic failures—including severe distributional pathologies (§ 5.3), perfect algebraic closure that violates measurement independence (§ 4.2.2), and complete misplacement of the physical operating regime (§ 4.5)—that are distinct from but equally severe as those of Case 1. The persistence of failure across both layers, and across the parallel direct-prompt framework (Supplementary Information, Sections S2–S5), demonstrates that the Novelty Horizon is a structural boundary inherent to the autoregressive architecture, not a contingent limitation that can be overcome by better prompting.

Architecture, Optimization, and Inherent Constraints of Large Language Models

To assess the limitations of Large Language Models (LLMs) in scientific research, a deep understanding of their theoretical and architectural foundations is essential. This section briefly delineates the mathematical and computational principles underpinning models such as GPT-5.2, Claude, and Llama.

Language Modeling as Conditional Probability Estimation

The theoretical foundation of any Large Language Model rests upon probability theory and information theory. Formally, an LLM is a parametric function designed to approximate the true probability distribution of natural language sequences. Consider a vocabulary $\mathcal{V}$ containing all possible tokens (word pieces or characters). Any text document can be represented as a sequence of tokens $\mathbf{x} = (x_1, x_2, \ldots, x_T)$ where each $x_t \in\mathcal{V}$ and $T$ is the sequence length. The fundamental assumption of statistical language modeling is that language follows a probabilistic structure. The joint probability of a sequence can be decomposed using the chain rule of probability:

P(\mathbf{x}) = P(x_1, x_2, \ldots, x_T) = \prod_{t=1}^{T} P(x_t \mid x_1, x_2, \ldots, x_{t-1}) = \prod_{t=1}^{T} P(x_t \mid x_{<t}) \tag*{(1)}

where $x_{<t} = (x_1, \ldots, x_{t-1})$ denotes the entire context preceding position $t$ . The model’s objective is to learn a parameterized function $f_\theta$ that approximates the conditional probability $P(x_t \mid x_{<t})$ for any token in the vocabulary given any context. The parameters $\theta$ of this function, which number in the range of millions to hundreds of billions, are optimized during training to minimize the discrepancy between the predicted distribution and the empirical distribution observed in a massive training corpus $\mathcal{D}$ . The standard objective function is the negative log-likelihood, equivalent to the cross-entropy loss:

\mathcal{L}(\theta) = -\frac{1}{|\mathcal{D}|} \sum_{\mathbf{x} \in\mathcal{D}} \sum_{t=1}^{T} \log P_\theta(x_t \mid x_{<t}) \tag*{(2)}

Minimizing Equation (2) is equivalent to maximizing the likelihood of the training data under the model’s distribution. When the model perfectly mimics the empirical distribution of the training corpus, the cross-entropy equals the true entropy of the language source, establishing an information-theoretic lower bound [31, 32]. This mathematical formulation reveals the core nature of an LLM’s “intelligence”. The model does not understand language in a human sense; it simply learns to assign probabilities to token sequences in a way that compresses the statistical regularities of its training data. The knowledge encoded in the parameters $\theta$ represents a high-dimensional manifold that captures correlations between tokens, but it lacks any explicit representation of meaning, truth, or causal relationships.

The Transformer and the Self-Attention Mechanism

The practical realization of the theoretical framework in Equation (1) requires an architecture capable of modeling long-range dependencies in sequential data. The Transformer architecture [33], introduced by Vaswani et al. (2017) [34], provides this capability through its self-attention mechanism, which has become the foundation for nearly all modern LLMs.

Input Representation. The input sequence of tokens is first converted into a sequence of dense vector representations. Each token $x_t$ is mapped to an embedding vector $e_t \in\mathbb{R}^{d_{\mathrm{model}}}$ via an embedding matrix $\mathbf{E} \in\mathbb{R}^{\lvert V\rvert\times d_{\mathrm{model}}}$ , where $d_{\mathrm{model}}$ is the model dimension (typically 512 to 8192 in modern models). To incorporate positional information, a positional encoding $p_t \in\mathbb{R}^{d_{\mathrm{model}}}$ is added to each token embedding. The original Transformer used sinusoidal functions:

p_t^{(2i)} = \sin\left(\frac{t}{10000^{2i/d_{\mathrm{model}}}}\right), \qquad p_t^{(2i+1)} = \cos\left(\frac{t}{10000^{2i/d_{\mathrm{model}}}}\right) \tag*{(3)}

where $i$ indexes the dimension. Many modern models use learned positional embeddings instead. The combined input representation for token $t$ is thus $x_t = e_t + p_t$ .

Scaled Dot-Product Attention. The core innovation of the Transformer is the self-attention mechanism. For each token position, the model computes a weighted sum of value vectors from all positions in the sequence, with weights determined by the compatibility between the query at the current position and keys at all positions. Formally, let the input sequence be represented as a matrix $\mathbf{X} \in\mathbb{R}^{T \times d_{\mathrm{model}}}$ , where each row corresponds to a token’s combined embedding. The input is linearly projected into three distinct matrices: queries $\mathbf{Q}$ , keys $\mathbf{K}$ , and values $\mathbf{V}$ :

\mathbf{Q} = \mathbf{X}\mathbf{W}^{Q}, \qquad\mathbf{K} = \mathbf{X}\mathbf{W}^{K}, \qquad\mathbf{V} = \mathbf{X}\mathbf{W}^{V} \tag*{(4)}

where $\mathbf{W}^{Q}, \mathbf{W}^{K} \in\mathbb{R}^{d_{\mathrm{model}} \times d_k}$ and $\mathbf{W}^{V} \in\mathbb{R}^{d_{\mathrm{model}} \times d_v}$ are learned projection matrices [34, 35]. Typically, $d_k = d_v = d_{\mathrm{model}}/h$ , where $h$ is the number of attention heads. The attention scores are computed as the scaled dot product between queries and keys:

\operatorname{Attention}(\mathbf{Q}, \mathbf{K}, \mathbf{V}) = \operatorname{softmax}\left(\frac{\mathbf{Q}\mathbf{K}^{\mathsf{T}}}{\sqrt{d_k}}\right)\mathbf{V} \tag*{(5)}

The scaling factor $1/\sqrt{d_k}$ prevents the dot products from growing too large in magnitude, which would push the softmax function into regions of extremely small gradients during training. The softmax function ensures that the attention weights for each query sum to 1 [34, 35], creating a convex combination of value vectors.

Multi-Head Attention. To allow the model to jointly attend to information from different representation subspaces at different positions, the Transformer employs multi-head attention:

\operatorname{MultiHead}(\mathbf{Q}, \mathbf{K}, \mathbf{V}) = \operatorname{Concat}(\operatorname{head}_1,\ldots,\operatorname{head}_h)\mathbf{W}^{O} \tag*{(6)}

where each head is computed as:

\operatorname{head}_i = \operatorname{Attention}\left(\mathbf{X}\mathbf{W}^{Q}_i,\mathbf{X}\mathbf{W}^{K}_i,\mathbf{X}\mathbf{W}^{V}_i\right) \tag*{(7)}

and $\mathbf{W}^{O} \in\mathbb{R}^{h \cdot d_v \times d_{\mathrm{model}}}$ is an output projection matrix. The parameters $\mathbf{W}^{Q}_i, \mathbf{W}^{K}_i, \mathbf{W}^{V}_i$ are distinct for each head, allowing each to learn different patterns of attention.

Position-wise Feed-Forward Networks. After the attention sub-layer, each position’s representation is independently transformed by a feed-forward network (FFN) consisting of two linear transformations with a non-linearity in between:

\operatorname{FFN}(\mathbf{x}) = \operatorname{ReLU}(\mathbf{x}\mathbf{W}_1 + \mathbf{b}_1)\mathbf{W}_2 + \mathbf{b}_2 \tag*{(8)}

where $\mathbf{W}_1 \in\mathbb{R}^{d_{\mathrm{model}} \times d_{\mathrm{ff}}}$ , $\mathbf{W}_2 \in\mathbb{R}^{d_{\mathrm{ff}} \times d_{\mathrm{model}}}$ , and typically $d_{\mathrm{ff}} = 4 \times d_{\mathrm{model}}$ . This provides additional non-linear processing capacity.

Residual Connections and Layer Normalization. Each sub-layer (attention and FFN) is wrapped with a residual connection and followed by layer normalization:

\mathbf{x}_{\mathrm{out}} = \operatorname{LayerNorm}(\mathbf{x}_{\mathrm{in}} + \operatorname{Sublayer}(\mathbf{x}_{\mathrm{in}})) \tag*{(9)}

This architecture enables stable training of very deep networks by mitigating the vanishing gradient problem.

Transformer Block. A complete Transformer block consists of a multi-head attention sub-layer followed by a feed-forward sub-layer, each with residual connections and layer normalization. Modern LLMs typically stack 12 to 96 such blocks, creating a deep network that progressively refines token representations through successive layers of abstraction.

Training Dynamics

The development of a capable LLM involves several stages of training, each with distinct objectives and computational requirements [36].

Pre-training. The foundation of any LLM is unsupervised pre-training on a massive corpus of text data, typically containing hundreds of billions to trillions of tokens [37]. The objective is to minimize the loss function in Equation (2) using a variant of stochastic gradient descent, most commonly the Adam optimizer. The optimization process for a model with parameters $\theta$ can be formalized as:

\theta_{k+1} = \theta_k - \eta_k \cdot\hat{g}_k \tag*{(10)}

where $\eta_k$ is the learning rate at step $k$ and $\hat{g}_k$ is an estimate of the gradient $\nabla_\theta\mathcal{L}(\theta)$ computed on a mini-batch of training data. Modern training regimes use sophisticated learning rate schedules, often with warmup and decay phases, to ensure stable convergence. The computational scale of this pre-training is extraordinary. Training a state-of-the-art LLM requires thousands of specialized processors (GPUs or TPUs) running continuously for weeks or months, with electricity consumption measured in megawatt-hours [38, 39]. This scale imposes significant practical constraints on who can develop frontier models and contributes to the centralization of AI capabilities. During pre-training, the model develops increasingly sophisticated representations of language. Through careful probing studies, researchers have shown that different layers of the Transformer encode different types of information: lower layers capture syntactic structures, middle layers capture semantic relationships, and higher layers integrate task-specific information. This hierarchical representation emerges spontaneously from the optimization process rather than being explicitly engineered.

Emergent Abilities. As models scale in size (number of parameters) and training data, they exhibit “emergent abilities”-capabilities that are not present in smaller models and appear abruptly at certain scale thresholds. These include multi-step reasoning, instruction following, and few-shot learning. Mathematically, emergence can be understood as a phase transition in the model’s loss landscape: as capacity increases, the model can represent more complex functions that solve tasks requiring compositional reasoning. However, emergence is fundamentally a statistical phenomenon. It results from the model’s improved approximation of the data distribution, not from the development of discrete reasoning modules. The model learns to recognize patterns in its training data that correlate with certain types of reasoning, allowing it to mimic reasoning processes without actually implementing them [40, 41].

Fine-tuning and Alignment. The base pre-trained model, optimized purely for next-token prediction, often generates text that is unhelpful, unsafe, or inconsistent with human values. To make it useful for specific applications, it undergoes supervised fine-tuning (SFT) and alignment. In SFT, the model is further trained on high-quality demonstration data consisting of (prompt, response) pairs. The objective remains the same cross-entropy loss, but the training data is carefully curated to teach the model desirable response patterns. For more sophisticated alignment, Reinforcement Learning from Human Feedback (RLHF) is commonly used. This process involves three stages:

Collect human preference data: Human annotators rank multiple model responses to the same prompt based on quality.

Train a reward model: A separate model $r_{\phi}(x,y)$ is trained to predict human preferences, typically using a Bradley-Terry model [42]:

P(y_1 \succ y_2 \mid x) = \frac{\exp(r_{\phi}(x,y_1))}{\exp(r_{\phi}(x,y_1))+\exp(r_{\phi}(x,y_2))} \tag*{(11)}

where $y_1 \succ y_2$ means response $y_1$ is preferred over $y_2$ for prompt $x$ .

Optimize the policy: The LLM’s policy $\pi_{\theta}(y \mid x)$ is fine-tuned to maximize the expected reward while penalizing deviation from the original pre-trained model $\pi_{\mathrm{ref}}(y \mid x)$ :

\max_{\theta} \mathbb{E}_{x \sim D, y \sim\pi_{\theta}(\cdot\mid x)} \left[r_{\phi}(x,y)-\beta\operatorname{KL}\left(\pi_{\theta}(\cdot\mid x)\Vert\pi_{\mathrm{ref}}(\cdot\mid x)\right)\right] \tag*{(12)}

The KL divergence term prevents the model from deviating too far from its original knowledge distribution, which helps maintain coherence and prevents catastrophic forgetting. This objective shapes the model’s outputs to be more helpful, honest, and harmless, but it does not alter the fundamental next-token prediction mechanism.

The Sampling Process - Generating Text from the Model

At inference time, the LLM generates text through an autoregressive sampling process [43]. Given a prompt $x_{<t}$ , the model computes a probability distribution over the vocabulary $P_{\theta}(x_t \mid x_{<t})$ . The next token is selected by sampling from this distribution, then appended to the context, and the process repeats.

Pure greedy sampling (always selecting the token with highest probability) leads to repetitive and degenerate text. Therefore, various stochastic sampling strategies are employed to introduce controlled randomness:

Temperature Scaling. The logits $z_t$ (pre-softmax scores) are divided by a temperature parameter $\tau> 0$ before applying the softmax:

P_{\theta}(x_t=v \mid x_{<t}) = \frac{\exp(z_t^{(v)}/\tau)}{\sum_{v'\in\mathcal{V}}\exp(z_t^{(v')}/\tau)} \tag*{(13)}

where $z_t^{(v)}$ is the logit for vocabulary item $v$ . Lower temperatures ( $\tau< 1$ ) sharpen the distribution, making high-probability tokens more likely and producing more deterministic text. Higher temperatures ( $\tau> 1$ ) flatten the distribution, increasing randomness and “creativity” at the cost of potential incoherence.

Top- $k$ Sampling. Only the $k$ tokens with highest probabilities are considered, and the distribution is renormalized over this subset:

P_{\theta}^{\mathrm{top-}k}(x_t=v \mid x_{<t}) = \begin{cases} \frac{P_{\theta}(x_t=v \mid x_{<t})}{\sum_{v'\in\mathcal{V}_{\mathrm{top-}k}}P_{\theta}(x_t=v' \mid x_{<t})} & \text{if } v\in\mathcal{V}_{\mathrm{top-}k} \\ 0 & \text{otherwise} \end{cases} \tag*{(14)}

where $\mathcal{V}_{\mathrm{top-}k}$ contains the $k$ tokens with highest probabilities.

Top-p (Nucleus) Sampling. The smallest set of tokens whose cumulative probability exceeds a threshold $p$ is selected, and the distribution is renormalized over this set:

\mathcal{V}_{\mathrm{top-}p} = \underset{\mathcal{V}' \subseteq\mathcal{V}}{\operatorname{argmin}} \left\{ \sum_{v \in\mathcal{V}'} P_{\theta}(x_t = v \mid x_{<t}) \ge p \right\} \tag*{(15)}

This approach dynamically adjusts the number of tokens considered based on the shape of the distribution.

Beam Search. For tasks requiring exactness (like translation), beam search maintains multiple candidate sequences simultaneously, expanding only the most promising candidates at each step. For a beam width $B$ , the algorithm keeps the $B$ sequences with highest cumulative log-probability at each step. The choice of decoding strategy significantly impacts the quality and characteristics of generated text. In scientific applications, temperature is typically kept low ( $\tau\approx0.1$ – $0.5$ ) to reduce hallucinations, while creative writing might use higher temperatures ( $\tau\approx0.7$ – $1.0$ ) to encourage novelty.

Inherent Theoretical Constraints Deriving from the Architecture

The preceding mathematical description yields several fundamental constraints that delimit LLM capabilities in scientific research.

Causal structure of autoregression. The factorization in Equation (1) is strictly causal: each token depends only on previous tokens. The model has no mechanism for planning, look-ahead, or backtracking within a single forward pass. Any appearance of such reasoning is a recapitulation of patterns in the training data, not an internal search.

Absence of a world model. Parameters $\theta$ encode token correlations, not a causally structured model of the physical world. The model learns that “The ball is dropped” is often followed by “it falls”, but it cannot simulate gravity. When faced with novel scenarios lacking descriptive data, it cannot reason from first principles.

Context window as a hard bound. Self-attention (Equation (5)) has $O(T^2d_{\mathrm{model}})$ computational cost and $O(T^2)$ memory. This imposes a practical limit $L$ (typically 4K–128K tokens). Information beyond $L$ tokens is inaccessible, limiting synthesis of long documents or experimental logs.

Optimization for plausibility, not truth. The training objective (Equation (2)) maximizes likelihood of the next token. A factually incorrect but common statement can have higher probability than a correct but rare one. Hallucination is an inevitable consequence, not a bug. Human verification remains essential.

Statistical nature of “understanding”. The model maps queries to probable response patterns by navigating a high-dimensional text manifold. This differs fundamentally from human understanding, which involves causal mental models and grounded referents (the “Chinese Room” argument). LLMs reproduce arguments but cannot evaluate their epistemological foundations.

Dependence on data distribution. Knowledge is a function of the training distribution $p_{\mathrm{data}}(x)$ . Biases and gaps are directly encoded. Novelty is limited to interpolation within the convex hull of the training corpus. True conceptual innovation-paradigm shifts that reject historical patterns-lies outside the model’s operational domain.

Lack of meta-cognition and self-awareness. The model has no representation of its own knowledge state, uncertainty, or limitations. It cannot recognise when information is insufficient, unlike human experts who articulate boundaries and confidence.

Computational undecidability. Like any computational system, LLMs are subject to computability limits. The Halting Problem implies that there exist well-formed questions for which no finite prompt yields a correct answer, regardless of scale or architecture.

These constraints collectively define the ceiling for LLM capabilities in empirical science and motivate the formal bounds introduced in § 2.6.

The Bounds (The Past Information Boundary and Novelty Horizon)

The architectural and optimisation constraints detailed in § 2.1 through 2.5 culminate in two fundamental, formal bounds that define the operational envelope of an autoregressive large language model (LLM) as a scientific tool. These bounds are not merely descriptive; they are predictive principles that explain the empirical failures established in later sections and delineate the conditions under which such failures are inevitable. Unlike heuristic accounts of model limitation, the constructs introduced here are precisely grounded in the model’s training data and its statistical approximation, and yield testable predictions about generative performance as a function of task novelty.

The Past Information Boundary

Consider a model $M$ with parameters $\theta$ optimised on a training dataset $\mathcal{D} = \{(x_i, y_i)\}_{i=1}^{N}$ , where each data point is a sample drawn from an unknown true data-generating distribution $P_{\mathrm{data}}(X,Y)$ over an input space $\mathcal{X}$ and an output space $\mathcal{Y}$ . For scientific applications, each datum carries a timestamp $\tau_i \in\mathbb{R}_{\ge0}$ representing its creation or observation time (for example, the publication date of a source document or the completion time of an experiment). The model’s training is confined to $\mathcal{D}$ ; no information from times $t > \max_i \tau_i$ is accessible during training.

Let $\widehat{P}_{\mathrm{data}}$ denote the empirical distribution of $\mathcal{D}$ . The support of the training distribution is the set of all points in the combined input-output space for which the empirical probability density is non-zero, taken with respect to a suitable topology: $\Omega_{\mathrm{train}} = \operatorname{supp}(\widehat{P}_{\mathrm{data}}) = \{(x,y) \in\mathcal{X} \times\mathcal{Y} : \widehat{P}_{\mathrm{data}}(x,y) > 0\}$ . In practice, for continuous spaces, $\Omega_{\mathrm{train}}$ is understood as the closure of the region where the training density exceeds a small threshold, effectively capturing the convex hull and its immediate neighbourhood.

The model $M$ encodes a collection of statistical correlations and conditional probabilities derived from $\mathcal{D}$ . During autoregressive generation the model computes $P_{\theta}(x_t \mid x_{<t}) \ \forall x_t \in\mathcal{V}$ , where $\mathcal{V}$ is the token vocabulary. Let $\mathcal{C}_{\mathrm{train}}$ denote the set of all such conditional probability distributions that $\theta$ has learned: $\mathcal{C}_{\mathrm{train}} = \{P_{\theta}(\cdot\mid c) : c \text{ is a context sequence present in } \mathcal{D} \text{ or its typical continuations}\}$ .

Definition 2.1 (Past Information Boundary, $B_P$ ). For a model $M$ trained on $\mathcal{D}$ with associated timestamps $\{\tau_i\}_{i=1}^{N}$ , the Past Information Boundary is the ordered triple:

B_P(M) = (t_p, \Omega_{\mathrm{train}}, \mathcal{C}_{\mathrm{train}})

where

$t_p = \max_{i \in\{1,\ldots,N\}} \tau_i$ is the latest timestamp in $\mathcal{D}$ (the temporal frontier);

$\Omega_{train} \subseteq\mathcal{X} \times\mathcal{Y}$ is the support of the empirical training distribution (the conceptual and relational frontier);

$C_{train}$ is the set of all conditional probability distributions encoded in the model’s parameters (the statistical-correlation frontier).

The model’s knowledge is strictly confined to $B_P(M)$ . It has no mechanism for incorporating information whose factual content, conceptual structure, or relational geometry lies outside $\Omega_{train}$ or whose temporal origin is $t > t_p$ . Any output generated by $M$ is a recombination or interpolation of patterns already present within $B_P(M)$ . This aligns with the inherent constraints of autoregressive next-token prediction (§ 2.1 and § 2.5): the model is a retrospective statistical interpolator, not a prospective causal engine.

The Novelty Horizon

Whereas $B_P$ (Definition 2.1) delimits what the model knows, the Novelty Horizon characterises what the model can reliably do. A task may lie inside the boundary yet still be impossible if it requires extrapolation beyond the statistical or causal patterns encoded in $C_{train}$ . We therefore define a criterion based on the relationship between the task’s requirements and the components of $B_P(M)$ .

Let a task $\mathcal{T}$ be defined by:

an input space $\mathcal{I}$ (for example, a prompt or a set of experimental parameters);

a required output space $\mathcal{O}$ (for example, time-series data or a scientific claim);

a set of governing laws $\Phi$ , which may be physical (Kirchhoff’s laws, the Shockley diode equation) or logical (mathematical theorems, causal constraints).

Successful completion of $\mathcal{T}$ requires generating an output $o \in\mathcal{O}$ that is consistent with $\Phi$ given an input $i \in\mathcal{I}$ .

Definition 2.2 (Novelty Horizon, $H_N$ ). For a model $M$ with Past Information Boundary $B_P(M) = (t_p, \Omega_{train}, C_{train})$ , a task $\mathcal{T} = (\mathcal{I}, \mathcal{O}, \Phi)$ lies beyond the Novelty Horizon (equivalently, $M$ cannot reliably perform $\mathcal{T}$ ) if at least one of the following three conditions holds:

Condition 2.1 (Extrapolation outside the convex hull of $\Omega_{train}$ ). Let $\operatorname{conv}(\Omega_{train})$ denote the convex hull of the support of the training distribution, embedded in an appropriate vector space (for instance, via token embeddings or feature representations). The task requires generating an output $o^* \in\mathcal{O}$ (or a necessary intermediate representation) such that when projected onto the space of training examples via a mapping $\pi: \mathcal{O} \to\mathcal{X} \times\mathcal{Y}$ , the point $\pi(o^*)$ falls outside $\operatorname{conv}(\Omega_{train})$ : $\exists o^* \in\mathcal{O} : \pi(o^*) \notin\operatorname{conv}(\Omega_{train})$ . Because $M$ is a statistical interpolator – it has no capacity for analytic continuation or extrapolation beyond the manifold of its training data – it lacks any basis for generating $o^*$ .

Condition 2.2 (Causal disentanglement not present in $C_{train}$ ). The governing laws $\Phi$ require reasoning via causal mechanisms that are not reducible to the statistical correlations $C_{train}$ learned from text. More precisely, let $\Phi$ imply a set of invariant relationships $R_{\Phi}$ (for example, $v = iR$ for a resistor, or the Shockley diode equation $i = I_s(e^{v/(nV_T)} - 1$ )). If for any subset of variables the joint distribution $P_{data}$ satisfies a factorisation that respects physical independence (e.g., measurement noise on voltage and current must be independent, leading to non-zero Ohm’s-law residuals), but the conditional independences and counterfactual dependencies required by $\Phi$ are not encoded in $C_{train}$ , then the model may mimic the language of causality but cannot execute causal computation. The task therefore lies beyond $H_N$ .

Condition 2.3 (Temporal extrapolation beyond $t_p$ ). The task concerns events, theories, measurement outcomes, or data that first become available at a time $t > t_p$ . Because the model’s training data contains no information from the future, any output pertaining to such post- $t_p$ phenomena must be generated by extrapolation from past patterns. When the true post- $t_p$ distribution deviates from the historical continuation (for example, a paradigm shift or a genuinely novel experimental configuration), the model’s extrapolation is ungrounded and the task resides beyond $H_N$ .

If any of these conditions holds, we say that $\mathcal{T}$ lies beyond the Novelty Horizon, $H_N$ of $M$ . Conversely, a task lies within the horizon only if it requires only interpolation inside $\operatorname{conv}(\Omega_{\mathrm{train}})$ , uses only causal structures already encoded in $C_{\mathrm{train}}$ , and relies solely on information available at times $t \le t_p$ . The three conditions are not mutually exclusive; a task may satisfy two or all three. The Novelty Horizon, $H_N$ is therefore the union:

H_N(M) = \{\mathcal{T} \mid\text{Condition } 2.1 \text{ or } 2.2 \text{ or } 2.3 \text{ is true for } \mathcal{T} \text{ w.r.t. } B_P(M)\}.

Application to the Diode-Clamped Short-Circuit Experiment

The diode-clamped short-circuit experiment described in § 3 is engineered to reside precisely at - and partially beyond - the $H_N$ (Definition 2.2). While the training corpus $\mathcal{D}$ almost certainly contains textual descriptions of Ohm’s law (linear regime) and isolated diode characteristics, the specific, time-resolved conjunction of: nonlinear metallic resistance, semiconductor clamping dynamics (forward voltage, ideality factor, reverse saturation current), authentic noise characteristics of the ACS712T Hall-effect sensor (bandwidth, $1/f$ spectrum, averaging of 10000 samples), defines a point in the experimental configuration space that lies outside the convex hull of $\Omega_{\mathrm{train}}$ for integrated empirical data generation. No convex combination of training examples that separately describe linear resistors, ideal diodes, and white noise can reproduce the joint statistical-physical manifold of the real measurement process. Hence Condition 2.1 of Definition 2.2 applies.

Moreover, the task requires that voltage and current be generated as independent noisy measurements of a common underlying physical process. The correct causal structure is then:

\begin{aligned} I_{\mathrm{true}}(t) &= f(\text{circuit state}),\\ V_{\mathrm{true}}(t) &= g(I_{\mathrm{true}}(t),\text{diodes}),\\ I_{\mathrm{meas}}(t) &= I_{\mathrm{true}}(t) + \varepsilon_I(t),\\ V_{\mathrm{meas}}(t) &= V_{\mathrm{true}}(t) + \varepsilon_V(t), \end{aligned}

with $\varepsilon_I(t)$ and $\varepsilon_V(t)$ independent noise processes. The LLM, however, treats Ohm’s law as an algebraic tautology, generating one variable and computing the other deterministically - a violation of the causal independence condition. Thus Condition 2.2 also holds.

Finally, the specific noise spectrum and the transient interaction of the short circuit with the diode capacitance are not present in standard electrical engineering textbooks (the training data’s temporal frontier $t_p$ predates the detailed characterisation in [30]). Consequently Condition 2.3 applies as well.

Empirical consequences, proven in § 4.2.2 and § 5.2, follow directly. The unguided LLM (Case 1) defaults to generating data consistent with the simplest correlations in $C_{\mathrm{train}}$ , treating physical laws as algebraic identities; this yields catastrophic physical inconsistencies, such as Ohmic residuals of order $10^{-17}\,\Omega$ instead of the experimental $10^{-3}\,\Omega$ . The guided model (Case 2) attempts to navigate closer to the horizon by incorporating procedural descriptors (“clamping”, “averaging”), but fails to cross it because the underlying causal model $\Phi$ - the Modified Ohm’s Law derived in § 3.1 is not accessible to its architecture. Consequently, the guided data exhibits pathological statistics (§ 5.3) and a deterministic residual-current correlation (§ 5.6) that betrays a conditional-branching generation algorithm devoid of physical continuity.

Predictive Principles and the Necessity of Foundational Modifications to Cross the Horizon

The formal bounds are not mere descriptions of failure; they are predictive principles. They imply:

Smooth degradation within the horizon. As a task’s required output approaches the boundary of $\Omega_{\mathrm{train}}$ (but remains inside $\operatorname{conv}(\Omega_{\mathrm{train}})$ ), the model’s performance degrades gracefully, with increasing but still correctable errors.
Catastrophic collapse beyond the horizon. When the task demands extrapolation beyond the Novelty Horizon - i.e., any of the three conditions in Definition 2.2 is met - performance collapses discontinuously. Statistical correlation diverges from causal necessity, and generated outputs become physically inconsistent in fundamental ways (regime misplacement, perfect algebraic closure, impossible distributional shapes).

This explains the regime-dependent failure observed in § 4.5: the linear regime ( $|v_s| < 0.3\mathrm{V}$ ) lies inside the horizon (interpolation of common Ohm’s law correlations), whereas the nonlinear clamping regime ( $|v_s| \ge0.3\mathrm{V}$ ) lies outside it (extrapolation requiring novel synthesis of diode physics and measurement noise).

Necessary modifications to cross the Novelty Horizon. Definitions 2.1 and 2.2 make clear that merely scaling model size, increasing training data, or refining prompt engineering cannot enable an LLM to cross $H_N$ when the task requires extrapolation outside $\operatorname{conv}(\Omega_{\mathrm{train}})$ , causal reasoning not encoded in $C_{\mathrm{train}}$ , or reasoning about post- $t_p$ phenomena. To cross the horizon, an LLM would require foundational, architectural changes that address the root limitations:

Incorporation of a causal world model. The model must move from learning statistical correlations to learning a causally structured representation of the domain, including the ability to perform interventions and counterfactual reasoning. This implies a shift from purely autoregressive next-token prediction to hybrid architectures that combine neural perception with symbolic causal engines (e.g., structural causal models or probabilistic graphical models that are explicitly trained on interventional data).

Grounding in physical simulators. Instead of generating data from token probabilities alone, the model should be able to call upon or internalise a first-principles simulator (such as a SPICE engine for circuits, a finite-element solver for electromagnetism, or a molecular dynamics integrator). The LLM would then act as a high-level controller that formulates a configuration, queries the simulator, and returns the resulting data. Such an architecture would satisfy Condition 2.1 by operating inside the simulator’s valid domain, but it would no longer be a pure LLM; it would be a neuro-symbolic system.

Online learning and dynamic knowledge updating. To overcome the temporal condition (Condition 2.3), the model would need the capacity to update its knowledge continuously from new experimental results without retraining from scratch. This requires mechanisms for continual learning, memory consolidation, and uncertainty-aware knowledge integration - capabilities that current transformer-based LLMs lack by design.

Explicit handling of measurement processes and noise. Crossing the horizon for experimental data generation demands that the model represent the measurement chain as a distinct stochastic process, separate from the underlying physical laws. This includes modelling sensor bandwidth, quantisation, thermal noise, and drift. Current LLMs treat numbers as exact symbols; they would need a probabilistic programming layer that samples from measurement-error models.

Metacognitive abilities for detecting horizon proximity. A model that can recognise when a query lies near or beyond its Novelty Horizon could refuse to generate ungrounded outputs or explicitly flag its uncertainty. This would require internal state estimators that compare the input’s representation to the convex hull of training points - a form of out-of-distribution detection embedded in the generation loop.

Consequences for the augmented intelligence paradigm. The bounds provide a rigorous criterion for the division of labour between LLMs and human scientists:

Tasks strictly inside $H_N$ (interpolation, routine synthesis, well-documented physical regimes) can be safely delegated to LLMs for efficiency - literature summarisation, code completion, formatting, and initial hypothesis generation.
Tasks at or beyond $H_N$ (novel experimental configurations, extrapolation to unseen regimes, discovery of new causal relationships) demand the causal reasoning, experimental design, and physical grounding of human scientists. LLMs may serve as brainstorming assistants, but their outputs must be treated as unvalidated conjectures, not as empirical evidence.

The formalisation of $B_P$ and $H_N$ directly bridges the theoretical limits of § 2 to the empirical results of § 5 and the philosophical interpretation of § 6, creating a unified, stringent argument that withstands the evolution of model versions and training data. The Past Information Boundary, $B_P$ (Definition 2.1) and the Novelty Horizon are not static; they shift as training corpora expand, but the structural relationship between the model’s objective (next-token prediction on a static corpus) and the requirements of genuine scientific extrapolation remains invariant. Therefore, any claim that a future LLM has “crossed” the Novelty Horizon must be accompanied by evidence of one or more of the foundational modifications listed above - not merely improved benchmark scores on tasks that still lie inside $\operatorname{conv}(\Omega_{\mathrm{train}})$ .

Foundational Framework

Having established the theoretical constraints of LLMs, the following framework transitions from abstract limitation to empirical test, providing a first-principles physical model against which LLM generated data can be forensically examined for concrete failure.

The Modified Ohm’s Law for Time-Resolved Short-Circuit Measurement

The experimental investigation of short-circuit fault dynamics reveals systematic deviations from the linear voltage-current relationship described by standard Ohm’s Law. These deviations, particularly pronounced in the clamped diode configuration, necessitate a reformulated constitutive model that captures the inherent nonlinearities of the fault path. This section establishes a first-principles derivation of a Modified Ohm’s Law, transformed into the time domain to enable direct analysis of high-resolution experimental data. The derivation proceeds from fundamental charge transport physics in a conductive path containing semiconductor junctions, culminating in a time-resolved equation that relates instantaneous short-circuit voltage $v_s(t)$ to instantaneous current $i(t)$ through a physically grounded nonlinear relationship. This derived law serves as the fundamental threshold for physical authenticity, creating an unambiguous criterion that data must satisfy. Consequently, it provides the essential tool for translating the theoretical constraints of LLMs—such as the absence of a world model and optimization for plausibility over truth—into empirically measurable shortcomings when such models attempt to generate scientific data for this novel experimental configuration. Critically, the independence of voltage and current measurement noise embedded in Equation (19) represents a causal structure that lies outside $C_{\mathrm{train}}$ for most LLMs, as formalised by Condition 2.2 of Definition 2.2 (the Novelty Horizon).

Physical Model of the Short-Circuit Path

The short-circuit path in the experimental configuration represents a composite electrical element rather than a simple linear resistor. It comprises a metallic connection in parallel with a clamping diode network that becomes active during fault conditions. The total instantaneous voltage drop $v_s(t)$ across this composite path is the sum of the voltage across the intrinsic short-circuit resistance $R_{\mathrm{sc}}$ and the voltage across the active semiconductor junction $v_d(t)$ . This relationship is expressed as:

v_s(t) = i(t)R_{\mathrm{SC}} + v_d(t) \tag*{(16)}

The parameter $R_{\mathrm{sc}}$ represents the aggregate linear resistance arising from conductor bulk resistivity and contact points within the fault path itself. This component obeys a linear voltage-current relationship characteristic of metallic conduction. The junction voltage $v_d(t)$ , however, is governed by the Shockley diode equation, which describes charge carrier transport across a semiconductor p-n junction under forward bias conditions. The presence of this semiconductor element introduces the essential nonlinearity that distinguishes the experimental system from idealized resistive models.

Semiconductor Junction Dynamics in Forward Bias

The Shockley diode equation provides the fundamental link between junction voltage and current through it. For a forward-biased diode where $v_d(t) \gg V_T$ , the current-voltage relationship simplifies to:

i(t) = I_s\left[e^{\frac{v_d(t)}{nV_T}} - 1\right] \approx I_s e^{\frac{v_d(t)}{nV_T}} \tag*{(17)}

In Equation (17), $I_s$ denotes the reverse saturation current, a material constant typically ranging from $10^{-12}$ to $10^{-6}$ amperes for silicon diodes. The thermal voltage $V_T$ is defined as $k_B T/q$ , where $k_B$ is Boltzmann’s constant ( $1.380649 \times10^{-23}$ J/K), $T$ is the absolute junction temperature in kelvins, and $q$ is the elementary charge ( $1.602176634 \times10^{-19}$ C). At standard laboratory temperature of 300 K, $V_T \approx25.85$ mV. The ideality factor $n$ accounts for deviations from ideal diode behavior arising from recombination-generation currents in the depletion region, typically ranging between 1 and 2 for silicon devices. Solving Equation (17) for the junction voltage $v_d(t)$ yields:

v_d(t) = nV_T \ln\left(\frac{i(t)}{I_s}\right) \tag*{(18)}

This logarithmic relationship constitutes the mathematical origin of the significant nonlinearity observed in the experimental system. The equation demonstrates that the junction voltage increases only logarithmically with current, in stark contrast to the linear dependence characteristic of resistive elements. This fundamental difference produces the distinctive electrical signatures that enable discrimination between physically grounded data and statistically generated artifacts.

Derivation of the Time-Domain Modified Ohm's Law

Substituting the expression for the junction voltage from Equation (18) into the composite path voltage equation (16) produces the complete nonlinear constitutive law for the short-circuit branch:

v_s(t) = i(t)R_{\mathrm{sc}} + nV_T \ln\left(\frac{i(t)}{I_s}\right) \tag*{(19)}

Equation (19) represents a transformed version of the Modified Ohm’s Law [44] in explicit form, where the instantaneous voltage is expressed directly as a function of the instantaneous current. This formulation successfully integrates the linear resistive voltage drop with the logarithmic semiconductor junction characteristic. For analytical purposes requiring current as an explicit function of voltage, the equation can be rearranged to isolate the exponential relationship:

\ln\left(\frac{i(t)}{I_s}\right) = \frac{v_s(t)-i(t)R_{\mathrm{sc}}}{nV_T} \tag*{(20)}

Exponentiating both sides of Equation (20) yields:

\frac{i(t)}{I_s} = e^{\frac{v_s(t)-i(t)R_{\mathrm{sc}}}{nV_T}} \tag*{(21)}

Equation (21) represents a transcendental equation in $i(t)$ that cannot be solved algebraically in closed form. However, it defines the complete nonlinear relationship between the measurable quantities $v_s(t)$ and $i(t)$ at each moment in time, making it directly applicable to time-resolved experimental data sampled at 10-millisecond intervals. The equation remains valid throughout the dynamic evolution of the short-circuit event, from initial transient through quasi-steady state to eventual clearance.

Linearized Form and Dynamic Resistance Analysis

For small-signal analysis around a quiescent operating point $(I_Q, V_{SQ})$ , the model can be linearized to define an effective dynamic resistance. The operating point satisfies Equation (19):

V_{SQ} = I_Q R_{\mathrm{sc}} + nV_T \ln\left(\frac{I_Q}{I_s}\right) \tag*{(22)}

Considering perturbations such that $i(t) = I_Q + \Delta i(t)$ and $v_s(t) = V_{SQ} + \Delta v_s(t)$ , and substituting into Equation (19), we obtain:

V_{SQ} + \Delta v_s(t) = [I_Q + \Delta i(t)]R_{\mathrm{sc}} + nV_T \ln\left(\frac{I_Q+\Delta i(t)}{I_s}\right) \tag*{(23)}

Expanding the logarithmic term using the first-order Taylor approximation $\ln(1+x) \approx x$ for $|x| \ll1$ yields:

\ln\left(\frac{I_Q+\Delta i(t)}{I_s}\right) = \ln\left(\frac{I_Q}{I_s}\right) + \ln\left(1+\frac{\Delta i(t)}{I_Q}\right) \approx\ln\left(\frac{I_Q}{I_s}\right) + \frac{\Delta i(t)}{I_Q} \tag*{(24)}

Substituting Equation (24) into Equation (23) and subtracting the steady-state Equation (22) produces the linearized relationship:

\Delta v_s(t) = \Delta i(t)R_{\mathrm{sc}} + nV_T\frac{\Delta i(t)}{I_Q} \tag*{(25)}

This simplifies to:

\Delta v_s(t) = \Delta i(t) \left( R_{\mathrm{sc}} + \frac{nV_T}{I_Q} \right) \tag*{(26)}

The coefficient of $\Delta i(t)$ defines the dynamic or small-signal resistance $r_d$ of the composite path:

r_d = \frac{\Delta v_s(t)}{\Delta i(t)} = R_{\mathrm{sc}} + \frac{nV_T}{I_Q} \tag*{(27)}

The term $\frac{nV_T}{I_Q}$ represents the dynamic resistance of the semiconductor junction alone, which varies inversely with the quiescent current. This linearized form proves particularly valuable for analyzing circuit stability and frequency response around established operating points during fault transients.

Time-Resolved Power Dissipation Formulation

The instantaneous power $p_s(t)$ dissipated in the short-circuit path follows directly from the modified constitutive law. Electrical power is defined as the product of instantaneous voltage and current:

p_s(t) = v_s(t)i(t) \tag*{(28)}

Substituting the expression for $v_s(t)$ from Equation (19) yields the complete expression for power dissipation:

p_s(t) = i^2(t)R_{\mathrm{sc}} + nV_Ti(t)\ln\left(\frac{i(t)}{I_s}\right) \tag*{(29)}

The first term, $i^2(t)R_{\mathrm{sc}}$ , represents conventional Joule heating in the linear resistive component of the fault path. The second term embodies the nonlinear power dissipation associated with the semiconductor junction, arising from the energy required to overcome the built-in potential barrier during charge carrier injection. This formulation enables precise computation of energy dissipated during the fault event through temporal integration:

E_{\mathrm{diss}} = \int_{t_0}^{t_f} p_s(t)\,\mathrm{d}t = \int_{t_0}^{t_f} \left[ i^2(t)R_{\mathrm{sc}} + nV_Ti(t)\ln\left(\frac{i(t)}{I_s}\right) \right]\mathrm{d}t \tag*{(30)}

The energy integral provides crucial information for assessing potential damage to circuit components, even in the absence of significant temperature rise at lower power inputs.

Parameter Identification and Experimental Validation

The time-domain Modified Ohm’s Law contains three physically meaningful parameters that can be determined experimentally through nonlinear regression analysis of the measured $v_s(t)$ and $i(t)$ waveforms:

Short-circuit resistance $R_{\mathrm{sc}}$ : The linear resistive component of the fault path, typically ranging from 0.001 $\Omega$ to 0.1 $\Omega$ for metallic connections in experimental setups.
Diode ideality factor $n$ : A dimensionless parameter characterizing the diode’s departure from ideal behavior, constrained between 1 and 2 for silicon devices.
Reverse saturation current $I_s$ : A temperature-dependent material parameter typically ranging from $10^{-12}$ to $10^{-6}$ amperes for silicon diodes at room temperature.

The thermal voltage $V_T$ is treated as a known constant determined by ambient laboratory temperature. For $T = 300$ K, $V_T = k_\mathrm{B}T/q \approx25.85$ mV. The model provides a rigorous framework for analyzing nonlinear signatures present in experimental Case 3 data, which are expected to be absent or incorrectly represented in LLM-generated data from Cases 1 and 2. By fitting Equation (19) to measured waveforms using nonlinear least-squares optimization:

\min_{R_\mathrm{sc},I_\mathrm{s}} \sum_{k=1}^{N} \left[v_\mathrm{s}(t_k)-i(t_k)R_\mathrm{sc}-nV_T\ln\left(\frac{i(t_k)}{I_\mathrm{s}}\right)\right]^2 \tag*{(31)}

the physical parameters can be extracted, and the model’s validity can be quantitatively assessed through residual analysis and goodness-of-fit metrics.

Case Studies: Script-Based LLM Generation

This paper employs two distinct experimental frameworks to probe the Past Information Boundary $B_\mathrm{P} = (t_\mathrm{p}, \Omega_\mathrm{train}, C_\mathrm{train})$ (Definition 2.1). The diode-clamped short-circuit experiment was selected because it satisfies all three conditions of Definition 2.2, thus placing it beyond the Novelty Horizon of any LLM trained on standard corpora. Specifically: (i) the specific conjunction of nonlinear resistance, semiconductor clamping dynamics, and authentic sensor noise defines a point outside $\operatorname{conv}(\Omega_\mathrm{train})$ (Condition 2.1); (ii) the correct causal structure requires independent measurement noise, which is not encoded in $C_\mathrm{train}$ (Condition 2.2); and (iii) the detailed transient interaction and noise spectrum post-date the training frontier $t_\mathrm{p}$ (Condition 2.3). In the first framework, detailed in this section, we task the LLM with a meta-scientific challenge: design a computational simulator of the physical experiment. This is a crucial distinction. The LLM is not sampling from an internal model of electrical manifolds; it is generating code that implements its own inferred physics model. Thus, we test its ability to translate a textual description of an experiment into a functioning numerical model—a process mirroring that of a student writing a simulation from a lab manual. The failures we uncover are therefore dual in nature: failures in computational physics modeling and failures in statistical emulation of empirical processes. The following cases establish two approaches to this simulator design task: one from first principles (naïve extrapolation) and one from detailed instruction (informed mimicry). The code they produce, and the data it generates, are analyzed as coupled outputs, where flaws in the former directly manifest as pathologies in the latter. A second, parallel framework, wherein the LLM is prompted to generate numerical data directly, is presented and analyzed in the supplementary information (SI, Section S2). This complementary approach serves as a critical robustness check, testing whether the failures we observe are artifacts of the script-based paradigm or represent a more fundamental limitation.

Case 1: Unguided Script-Based LLM Generation - Statistical Extrapolation of Basic Circuit Principles

The first dataset was generated by a GPT-5.2 class model operating without access to the specific experimental procedure. The model received only the nominal circuit parameters—voltages of 2.5 V and 5.0 V with corresponding currents of 0.25 A and 0.5 A—and was instructed to produce short-circuit fault data. The complete prompt, which also requested the theoretical foundations for the data generation, is established in Appendix § 8. This approach forced the model to infer all measurement dynamics, noise characteristics, and temporal evolution solely from statistical patterns in its training corpus, representing a pure test of extrapolation from learned electrical concepts. Circuit Configuration and Theoretical Foundation. The model conceptualized a minimal experimental setup. It described two independent low-voltage direct current circuits on a solderless breadboard, each using an Arduino Uno for data acquisition and an ACS712 Hall-effect sensor in series with the supply line. A short-circuit was introduced by bridging the load terminals with a low-resistance conductor. The theoretical justification provided by the model rested entirely on applying fundamental electrical relationships, deliberately excluding any semiconductor physics or detailed measurement system dynamics. The generation algorithm implemented the following core relationships, which the model cited as its foundational assumptions. The effective short-circuit resistance at any instant was computed directly from the measured voltage and current:

R_s(t) = \frac{V_s(t)}{I_s(t)} \quad\text{for } I_s(t) \ne0 \tag*{(32)}

Here, $V_s(t)$ denotes the instantaneous voltage across the short path, and $I_s(t)$ is the corresponding current from the sensor. This formulation assumes a perfectly linear and instantaneous resistive relationship throughout the fault event. The instantaneous electrical power dissipated was calculated using the standard definition:

P_s(t) = V_s(t)I_s(t) \tag*{(33)}

To prevent unrealistic current divergence, the model imposed a simple current limit based on an estimated source internal resistance:

I_{\max} = \frac{V_{\mathrm{nom}}}{R_{\mathrm{internal}}} \tag*{(34)}

The internal resistance $R_{\mathrm{internal}}$ was set to 5% of the nominal load resistance, defined as $R_{\mathrm{load}} = V_{\mathrm{nom}}/I_{\mathrm{nom}}$ . To avoid discontinuous jumps in the time-series data, state transitions were smoothed with an exponential relaxation model:

x(t + \Delta t) = x(t) + [x_{\mathrm{target}} - x(t)](1 - e^{-\Delta t/\tau}) \tag*{(35)}

A generic time constant of $\tau= 20$ ms was used, without a specific physical justification. All generated measurements were corrupted with additive noise following a simple statistical model:

x_{\mathrm{measured}}(t) = x_{\mathrm{true}}(t) + \mathcal{N}(0, \sigma^2) \tag*{(36)}

The noise magnitude $\sigma$ was uniformly set to 2% of the nominal signal value, representing an idealized, frequency-independent measurement uncertainty. This unguided generation process tests the LLM’s capacity to synthesize plausible data through statistical interpolation of common electrical concepts. Its reliance on simplified linear models and generic noise, in contrast to the complex nonlinear physics of the actual experiment, establishes a clear baseline for identifying artifacts of pure statistical pattern matching.

Case 2: Informed Pattern Matching with Experimental Context - Guided Script-Based LLM Generation

The second dataset was produced by the same class of model but with explicit, detailed guidance. The model was provided with the full experimental procedure adopted from the short-circuit measurement and validation framework recently established as the “Clamped Short Diodes Nominal” configuration [30], including the circuit schematic, component specifications, and measurement protocol. The prompt, contained in Appendix § 9, explicitly asked the model to generate data consistent with this procedure and to outline its theoretical reasoning. This case evaluates the model’s ability to integrate specific domain knowledge into its generative process, moving beyond basic extrapolation. Circuit Configuration and Diode Clamp Dynamics. The model’s internal representation incorporated the three-diode clamping network described in the provided procedure. It simulated a system with a DC source featuring an internal resistance $R_s$ , a nominal load resistor $R_{\mathrm{load}}$ , and the diode network creating a controlled, low-voltage short path. The theoretical foundation articulated by the model included simplified semiconductor physics and the specific measurement averaging protocol. To ensure the specified nominal voltage appeared across the load under normal conditions, the model defined the source voltage as:

V_s = V_{\mathrm{nom}}\left(1+\frac{R_s}{R_{\mathrm{load}}}\right) \tag*{(37)}

For an unclamped short-circuit, the current and voltage were calculated as:

I_{\mathrm{short,unclamped}}=\frac{V_s}{R_s+R_{\mathrm{short}}}, \qquad V_{\mathrm{short,unclamped}}=I_{\mathrm{short,unclamped}}R_{\mathrm{short}} \tag*{(38)}

The metallic short resistance $R_{\mathrm{short}}$ was confined to a range of 0.001–0.02 $\Omega$ . The model incorporated a diode clamp activation threshold. When the magnitude of the unclamped voltage exceeded the diode forward voltage $V_{\mathrm{clamp}}\approx0.7$ V, the clamping mechanism engaged:

I_{\mathrm{short,clamped}}\approx\frac{V_s}{R_s+R_{\mathrm{clamp,eq}}}, \qquad V_{\mathrm{short,clamped}}\approx\operatorname{sign}(V_{\mathrm{short,unclamped}})V_{\mathrm{clamp}} \tag*{(39)}

An effective clamp resistance was approximated as $R_{\mathrm{clamp,eq}}\approx V_{\mathrm{clamp}}/I_{\mathrm{short,unclamped}}$ . The instantaneous measured resistance and power were defined by the interdependent relationships:

R_{\mathrm{meas}}(t)=\frac{V_{\mathrm{short}}(t)}{I(t)}, \qquad P_{\mathrm{short}}(t)=V_{\mathrm{short}}(t)I(t)=I^2(t)R_{\mathrm{meas}}(t) \tag*{(40)}

Faithful to the experimental procedure, the model simulated the effect of averaging 10,000 analog-to-digital converter samples to reduce noise:

\sigma_{\mathrm{averaged}}=\frac{\sigma_{\mathrm{single}}}{\sqrt{N_{\mathrm{avg}}}}, \qquad\text{with }N_{\mathrm{avg}}=10{,}000 \tag*{(41)}

Transient dynamics were again governed by an exponential approach, but with the time constant now presented as part of a realistic circuit response model:

x(t+\Delta t)\approx x(t)+[x_{\mathrm{target}}-x(t)](1-e^{-\Delta t/\tau}), \qquad\tau=20\ \mathrm{ms} \tag*{(42)}

Furthermore, the model populated its simulation with realistic component parameters for the 1N5408 diodes, including a forward voltage $V_f\approx1.0$ V and an ideality factor $\eta\approx1.05$ . This guided generation approach tests the LLM’s capacity to utilize detailed procedural context. It demonstrates an attempt to simulate not just electrical behavior but the complete measurement process, incorporating specific component characteristics and a more nuanced, though still simplified, physical model. The comparison between this informed output and the unguided case reveals the extent to which explicit instructions can steer statistical generation toward physical plausibility.

A Parallel Framework: Direct-Prompt Generation

The script-based generation paradigm described above represents one mode of LLM interaction—one that forces the model to articulate its understanding in the structured, executable form of code. To determine whether the observed failures are specific to this paradigm or reflect a more fundamental limitation, we conducted a parallel investigation using a distinct generative approach. In this second framework, detailed in the supplementary information (SI, Section S2), the LLM was prompted to generate the numerical datasets directly, without the intervening step of code generation and execution. The direct-prompt framework employed two configurations analogous to those in the script-based paradigm. In the unguided condition, the model received only the nominal circuit parameters and was instructed to output time-series data for voltage, current, resistance, and power. In the guided condition, the model was provided with the full experimental procedure for the diode-clamped short-circuit system, including component specifications and measurement protocols, and was then asked to generate the numerical data directly. The complete prompts used for this parallel investigation are reproduced in the supplementary information (SI, Section S4). This direct-prompt paradigm serves a critical purpose: it tests whether the LLM can produce physically consistent numerical data when the mediating structure of code is removed. If the script-based failures were merely artifacts of imperfect code generation—errors in syntax, logic, or numerical implementation—one might expect the direct-prompt approach to yield superior results. Conversely, if the failures stem from a more fundamental inability to model the causal structure of the measurement process, they should persist, albeit potentially manifesting in different pathological signatures. The complementary analysis of directly generated data, presented in the supplementary information, therefore serves as a critical robustness check, testing whether the failures we observe are artifacts of the script-based paradigm or a more fundamental limitation intrinsic to the architecture and training of autoregressive language models.

Methodology (A Physical Analytical Framework)

We present here, a multi-layered, forensic analytical framework to test the core hypothesis that autoregressive LLMs cannot generate scientifically valid empirical data for tasks lying beyond their Novelty Horizon $H_N$ (Definition 2.2). The framework is designed to detect violations of the three conditions that define $H_N$ : extrapolation outside $\operatorname{conv}(\Omega_{\mathrm{train}})$ (Condition 2.1), causal structures absent from $C_{\mathrm{train}}$ (Condition 2.2), and reliance on post- $t_p$ information (Condition 2.3). The methodology is built on a tripartite comparative structure, evaluating two LLM-generated datasets against an immutable physical benchmark. The framework subjects all three datasets—which are aligned in nominal conditions (2.5 V/0.25 A, 5.0 V/0.5 A) and temporal parameters (10-minute duration, effective 10-ms resolution)—to a hierarchy of tests of increasing discriminative power. The analysis ascends from examining superficial statistical signatures to validating deep adherence to first-principles physical laws, thereby isolating the artifacts of statistical correlation from the authentic fingerprints of causal physical measurement.

Experimental Design and Data Generation Protocols: Script-Based Framework

This section details the methodology for the first of our two experimental frameworks, wherein LLMs generate data via a script-based simulator. The second framework, based on direct-prompt generation, is fully described in the supplementary information (SI, Section S3). The methodological core involves the parallel analysis of three complementary datasets, each representing a fundamentally different epistemology of knowledge derivation about the same physical phenomenon. The tripartite comparison, therefore, is not merely between three datasets, but between three generative origins: a naïve algorithmic simulator (Case 1), an uninformed algorithmic simulator (Case 2), and the physical universe itself (Case 3). Each dataset carries the imprint of its origin. The LLM-generated data bears the signature of its code’s logic—its time-stepping algorithm, its noise model, its treatment of physical laws. This paper’s forensic framework is designed to reverse-engineer these signatures. For example, a simulator that treats Ohm’s Law as a deterministic algebraic rule rather than a relationship between independent noisy measurements will produce data with a telltale artifact: infinitesimal Ohmic residuals. The analysis thus becomes a study in digital paleontology, where we infer the structure of the LLM’s internal simulator from the fossils it leaves in the data. All computations, including the execution of the LLM for data generation and the subsequent analytical workflow, were performed on a workstation equipped with an Intel® Core™ i7-2620M CPU operating at a base and maximum turbo frequency of 2.70 GHz. The processor featured 2 cores and 4 threads, supported by 4 MB of Intel® Smart Cache and 4 GB of DDR3 RAM. For the LLM-based generation tasks (Cases 1 and 2), the OpenAI GPT-5.2 model (specifically the GPT-5.2 variant as of March 2024) was utilized via its API, with all prompts and generated code documented in Appendices § 8 and § 9.

Case 1: Unguided Script-Based LLM Generation (Statistical Extrapolation) This dataset was produced by a state-of-the-art GPT-5.2 class model prompted to generate short-circuit fault data without access to the specific experimental procedure or circuit dynamics. The model received only the nominal circuit parameters-input voltages of 2.5 V and 5.0 V with corresponding currents of 0.25 A and 0.5 A -and was instructed to output time-series data for voltage $v_s(t)$ , current $i(t)$ , computed resistance $R_{\mathrm{calc}}(t)$ , and computed power $P_{\mathrm{calc}}(t)$ . The generation occurred autonomously, relying solely on the model’s internal statistical representations of electrical concepts derived from its training corpus, as detailed in § 3.2.1. This case represents the LLM’s intrinsic capacity for synthesizing scientific data based on learned textual and numerical patterns, absent any directed guidance regarding measurement physics or nonlinear circuit behavior.

Case 2: Procedurally Guided Script-Based LLM Generation (Informed Pattern Matching) The second dataset was generated by the same class of model but was provided with the complete experimental context. The prompt included the detailed procedure for the “Clamped Short Diodes Nominal” configuration, including the circuit schematic (defining the fault path between nodes A and B), component specifications (1N5408 diodes, ACS712T sensor), the data averaging protocol (10,000 samples), and the fault induction method from the foundational work of Kimuya et al. [30]. The model was instructed to generate data consistent with this described methodology. This case evaluates the LLM’s ability to integrate specific procedural knowledge into its generative process, moving beyond basic extrapolation towards a guided-yet still statistically derived-simulation of the experiment, as outlined in § 3.2.2.

Case 3: Experimental Measurement (Physical Ground Truth) The third dataset serves as the empirical, non-replicable ground truth. Critically, this data is not newly measured for this study. It is adopted directly from the comprehensive experimental campaign of Kimuya et al. [30], which established the “Clamped Short Diodes Nominal” framework as a validated platform for time-resolved short-circuit measurement. The experimental configuration-comprising the three-diode (1N5408) clamping network (Circuit Block 1, CB1), an ACS712T-20A current sensor, an Arduino Uno, and a regulated DC supply-is shown in Figure 10 (Appendix § 10). The data acquisition precisely followed the automated algorithm formalized in the prior work (Table 2, adopted from [30]). Key Differentiator from Case 1 and Case 2. The integrity of the Case 3 data is underpinned by the extensive sensor calibration and validation framework fully documented in [30, 45]. This includes baseline characterization, performance validation under fault conditions, and the derivation of a Consolidated Calibration Confidence Score (CCCS), all confirming the system’s measurement fidelity. No recalibration or re-validation was performed for this paper. We adopt the pre-validated dataset, with its inherent and quantified noise characteristics, temporal stability, and adherence to circuit laws, as the definitive physical benchmark. Therefore, Case 3 is not merely another dataset; it is the objective reality against which the synthetic outputs of the LLMs are forensically compared. Its role is to provide the physically-grounded manifold in voltage-current-power space onto which the LLM-generated data is projected, revealing the geometric and statistical distance between statistical plausibility and empirical truth.

Step	Action/Description
1	Connect DC power supply’s positive terminal to the ACS712T current sensor’s positive input though the output of $D_1$ . Connect the power supply’s negative terminal to the current sensor’s negative input.
2	Connect the current sensor’s OUT pin to the Arduino’s A0 analog input pin for current readings, through the output of $D_3$ . Connect also the output of $D_3$ to the Arduino’s A1 analog input pin via the current sensor’s OUT pin for voltage readings.
3	Connect $D_1$ ’s anode to the power supply’s positive terminal. Branch $D_1$ ’s cathode to the next circuit segment’s positive terminal and $D_2$ ’s anode.
4	Connect $D_2$ ’s cathode to Arduino ground (GND), completing the circuit path to the power supply’s negative terminal.
5	Connect $D_3$ ’s anode to the same node as $D_2$ ’s anode. Connect $D_3$ ’s cathode to an adjacent circuit’s negative terminal. For clarity, the cathodes of $D_1$ and $D_3$ proceed to “Circuit Block 2”.
6	Create a short-circuit path by linking nodes A and B (between $D_1$ and $D_3$ ) with a connector. This step should be performed carefully, only during when the short-circuit is induced.
7	Program Arduino to record voltage (A1) and current (A0) as magnitudes at 1-second intervals for 600s, during the short-circuit transient and post-transient events, at sampling frequencies of 10ms in each time interval. Upload the sample code (provided in supplementary materials) to the Arduino.
8	Turn on the selected power supply source voltage (2.5V, 5V, and 10V). Begin data logging to observe normal circuit behavior.
9	Connect the short-circuit path between nodes A and B. Continue data logging to capture short-circuit data.
10	Analyze the recorded data to assess the circuit’s response under normal (base measurements) and short-circuit conditions.

Table 2. Algorithm for Automated “Clamped Short Diodes” Base and Nominal Currents and Voltages Measurement (Adopted and modified from Kimuya et al. [30])

Figure 10. Experimental setup for “Clamped Short Diodes Nominal” measurements, adopted from [30]. Key components: DC power supply, ACS712T current sensor, Arduino Uno, and clamped diode network ( $D_1$ , $D_2$ , $D_3$ ). Nodes A and B define the short-circuit path during fault induction.

The Parallel Direct-Prompt Generation Framework

To determine whether the failures observed in the script-based generation paradigm are artifacts of that specific interaction mode or reflect a more fundamental limitation of autoregressive LLMs, we conducted a parallel investigation using a distinct generative approach. In this second framework, the LLM was prompted to generate the numerical datasets directly, without the intervening step of code generation and execution. This direct-prompt paradigm bypasses the mediating layer of Python code, compelling the model to produce time-series data from its internal numerical representations and learned statistical patterns alone. The direct-prompt framework employed two configurations directly analogous to those in the script-based paradigm. In the unguided condition, the model received only the nominal circuit parameters (2.5 V / 0.25 A and 5.0 V / 0.5 A) and was instructed to output CSV files containing time, voltage, current, resistance, and power columns for a 10-minute short-circuit event at 2-second intervals. In the guided condition, the model was provided with the complete experimental procedure for the “Clamped Short Diodes Nominal” configuration-identical to that supplied in Case 2 of the script-based framework-and was then asked to generate the numerical data directly. The complete prompts used for this parallel investigation are reproduced in the supplementary information (SI, Section S4). This direct-prompt paradigm serves a critical purpose: it tests whether the LLM can produce physically consistent numerical data when the mediating structure of code is removed. If the script-based failures were merely artifacts of imperfect code generation-errors in syntax, logic, or numerical implementation-one might expect the direct-prompt approach to yield superior results, as the model is freed from the constraints of generating syntactically correct Python. Conversely, if the failures stem from a more fundamental inability to model the causal structure of the measurement process and the independent nature of physical observables, they should persist across both paradigms, albeit potentially manifesting in different pathological signatures. The complementary analysis of directly generated data, presented in the supplementary information (SI, Section S2), therefore serves as a critical robustness check, testing whether the failures we observe are artifacts of the script-based paradigm or a more fundamental limitation intrinsic to the architecture and training of autoregressive language models.

Foundational Analytical Framework

The comparative analysis proceeds from the axiom that authentic experimental data embodies constraints-causal physical laws and intrinsic measurement process signatures-that transcend mere statistical correlation. The framework is hierarchically structured, with each analytical layer providing increasingly discriminating tests of generative fidelity.

Temporal and Statistical Structure Analysis

The initial analytical layer examines the intrinsic statistical properties of each time series variable $x(t)$ -where $x$ represents $v_s$ , $i$ , $R_{\mathrm{calc}}$ , or $P$ -independent of assumed physical relationships. This reveals whether LLM-generated data replicates the characteristic signatures of physical measurement systems.

Temporal Autocorrelation Function: The autocorrelation $R_{xx}(\tau)$ quantifies the memory and periodicity within a signal:

R_{xx}(\tau) = \frac{1}{N-\tau}\sum_{t=1}^{N-\tau}(x(t)-\mu_x)(x(t+\tau)-\mu_x) \tag*{(43)}

where $\mu_x$ is the signal mean, $N$ is the sample count, and $\tau$ is the lag. Physical sensor data typically exhibits a gradual autocorrelation decay due to system bandwidth and thermal time constants. LLM-generated data may display anomalously rapid decay (approaching white noise) or unnatural periodicities.

Power Spectral Density (PSD) Distribution: The PSD $S_{xx}(f)$ characterizes the distribution of signal power across frequencies:

S_{xx}(f) \approx\frac{1}{Nf_s}\left|\operatorname{FFT}\{x(t)\}\right|^2,\qquad f_s = 100\ \mathrm{Hz} \tag*{(44)}

Authentic sensor noise exhibits a characteristic $1/f^\alpha$ spectrum ( $\alpha\approx1$ ) at low frequencies (flicker noise), transitioning to white noise ( $\alpha= 0$ ) at higher frequencies. LLM-generated data, lacking a physical noise model, may present pure white noise spectra or contain spectral artifacts from the generation algorithm.

Probability Distribution and Moments: The empirical probability density function and its moments - mean ( $\mu_x$ ), variance ( $\sigma_x^2$ ), skewness ( $\gamma_1$ ), and kurtosis ( $\gamma_2$ ) - are computed. Measurement noise in calibrated systems typically approximates a Gaussian distribution ( $\gamma_1 \approx0$ , $\gamma_2 \approx0$ ). LLM-generated numerical values, shaped by token probability distributions, may exhibit artifacts such as multimodality or truncated tails.

Validation of Fundamental Physical Constraints

The second analytical layer tests adherence to inviolable physical laws, providing a stark discriminant since LLMs optimize for plausibility, not consistency.

Ohmic Consistency (Instantaneous): For each time step $t_k$ , the computed resistance must satisfy Ohm’s law within measurement uncertainty:

\epsilon_R(t_k) = R_{\mathrm{calc}}(t_k) - \frac{v_s(t_k)}{i(t_k)}, \quad\text{for } i(t_k) \ne0 \tag*{(45)}

For physically consistent data, the residuals $\epsilon_R(t_k)$ should be normally distributed with zero mean and variance $\sigma_R^2$ consistent with propagated instrument error.

Power Consistency (Multiple Formulations): The three mathematically equivalent expressions for instantaneous power must yield identical values:

\begin{cases} \epsilon_P^{(1)}(t_k) = P_{\mathrm{calc}}(t_k) - v_s(t_k)i(t_k) \\ \epsilon_P^{(2)}(t_k) = P_{\mathrm{calc}}(t_k) - i^2(t_k)R_{\mathrm{calc}}(t_k) \\ \epsilon_P^{(3)}(t_k) = P_{\mathrm{calc}}(t_k) - \frac{v_s^2(t_k)}{R_{\mathrm{calc}}(t_k)} \tag*{(46)} \end{cases}

The variances $\operatorname{Var}(\epsilon_P^{(1)})$ , $\operatorname{Var}(\epsilon_P^{(2)})$ , and $\operatorname{Var}(\epsilon_P^{(3)})$ must be statistically equivalent. Significant disparities indicate the generated data treats electrical quantities as independent variables rather than physically coupled entities.

Energy Conservation and Continuity: The total dissipated energy must align with the energy supplied, and signals must evolve continuously. The total energy is computed via integration:

E_{\mathrm{total}} = \int_{t_0}^{t_f} P_{\mathrm{short}}(t)\,\mathrm{d}t \approx\sum_{k=1}^{N} P_{\mathrm{calc}}(t_k)\Delta t, \quad\Delta t = 0.01\ \mathrm{s} \tag*{(47)}

Furthermore, incremental changes $\Delta x(t_k) = x(t_k) - x(t_{k-1})$ are bounded by physical limits (e.g., $\mathrm{d}i/\mathrm{d}t \le V/L$ for inductive paths). Abrupt, unphysical discontinuities signify a failure to respect continuity constraints.

Nonlinear Dynamic Model Fitting and Residual Analysis

The most stringent evaluation of generative fidelity involves fitting a first-principles physical model to each dataset. For the experimental configuration under study, the Modified Ohm’s Law, derived in § 3.1, provides the governing constitutive relationship. However, a critical clarification regarding the circuit’s operational physics is necessary for correct interpretation. The diode-clamp network in the experimental design, detailed through previous studies [45, 30], serves a specific protective function: its intrinsic capacitance mitigates transient backflow currents, while its conductive path does not significantly contribute to the steady-state measured current. Consequently, the primary electrical behavior observed in the recorded data is the linear voltage-current relationship of the metallic short-circuit path. The full Modified Ohm’s Law, which includes a logarithmic term representing ideal diode conduction, is therefore employed not to model a dominant semiconductor current, but as a sophisticated framework for computing the effective, instantaneous resistance of the composite fault path. This approach isolates the resistive component $R_{\mathrm{sc}}$ as the parameter of primary physical interest, with the diode parameters $n$ and $I_{\mathrm{s}}$ serving as auxiliary variables that account for minor clamping effects and ensure mathematical consistency in the regression.

Model Formulation and Parameter Estimation

The time-domain Modified Ohm’s Law for the composite short-circuit path is given by:

v_{\mathrm{s}}(t) = i(t)R_{\mathrm{sc}} + nV_{\mathrm{T}} \ln\left(\frac{i(t)}{I_{\mathrm{s}}}\right) \tag*{(48)}

In this expression, $v_{\mathrm{s}}(t)$ and $i(t)$ are the instantaneous voltage and current, respectively. The parameter $R_{\mathrm{sc}}$ represents the effective short-circuit resistance of the metallic connection, expected to range from 0.001 to 0.1 $\Omega$ . The second term contains the diode ideality factor $n$ (dimensionless, typically 1–2 for silicon), the reverse saturation current $I_{\mathrm{s}}$ (typically $10^{-12}$ to $10^{-6}$ A), and the thermal voltage $V_{\mathrm{T}} = k_{\mathrm{B}}T/q \approx25.85$ mV at 300 K. For the purpose of analyzing the experimentally recorded data (Case 3), the logarithmic term does not represent a substantial conductive current through the diodes. Instead, its inclusion allows the nonlinear least-squares optimization to converge on a stable estimate for $R_{\mathrm{sc}}$ while accommodating the subtle voltage-clamping behavior present at the measurement nodes. The model is fit to each dataset by estimating the parameter vector $\theta= \{R_{\mathrm{sc}}, n, I_{\mathrm{s}}\}$ that minimizes the sum of squared residuals:

\hat{\theta} = \underset{\theta}{\operatorname{argmin}} \sum_{k=1}^{N} \left[v_{\mathrm{s}}(t_k) - i(t_k)R_{\mathrm{sc}} - nV_{\mathrm{T}} \ln\left(\frac{i(t_k)}{I_{\mathrm{s}}}\right)\right]^2 \tag*{(49)}

The optimization is constrained to physically plausible ranges: $0.001\,\Omega\le R_{\mathrm{sc}} \le0.1\,\Omega$ , $1 \le n \le2$ , and $10^{-12}$ A $\le I_{\mathrm{s}} \le10^{-6}$ A. The Levenberg–Marquardt algorithm is employed with multiple random initializations to avoid local minima. For the LLM-generated datasets (Cases 1 and 2), this fitting process tests whether the statistically synthesized data conforms to the nuanced, physics-based relationship of Equation (48), or whether it deviates in ways that reveal an underlying lack of causal understanding.

Analysis of Fitted Parameters

The estimated parameters $\hat{R}_{\mathrm{sc}}$ , $\hat{n}$ , and $\hat{I}_{\mathrm{s}}$ provide direct, quantitative insight into the physical plausibility of each dataset.

Physical Plausibility Assessment: The extracted parameters are compared against known ranges for the experimental components. For the breadboard-based short-circuit path with a diode-clamp network, expectations are:

$\hat{R}_{\mathrm{sc}} \approx0.005$ – $0.02\,\Omega$ , representing the resistance of wire connections, breadboard contacts, and the shorting jumper.
$\hat{n} \approx1.1$ – $1.8$ , consistent with general-purpose silicon diodes, though this parameter is not the primary focus given the circuit’s operation.
$\hat{I}_{\mathrm{s}} \approx10^{-9}$ – $10^{-7}$ A for silicon p-n junctions at room temperature.

Parameters falling conspicuously outside these ranges, or exhibiting non-physical correlations (e.g., an extremely low $\hat{R}_{\mathrm{sc}}$ coupled with an extremely high $\hat{I}_{\mathrm{s}}$ ), indicate that the data generation process did not respect the interdependent constraints of real electrical components.

Temporal Stability Analysis: A stable physical system exhibits consistent characteristics over time. To assess this, each dataset is divided into ten sequential 60-second windows. The parameter vector $\hat{\theta}_w$ is re-estimated for each window $w$ using Equation (49). The coefficient of variation (CV) for each parameter across all windows is then calculated:

\mathrm{CV}_{\theta} = \frac{\sigma_{\theta}}{\mu_{\theta}} \times100\% \tag*{(50)}

Here, $\mu_{\theta}$ and $\sigma_{\theta}$ are the mean and standard deviation of the parameter estimates across the windows. Authentic experimental data should yield low CV values (typically < 10%), reflecting measurement noise around a stable true value. LLM-generated data may exhibit high CV or systematic drifts, revealing statistical generation artifacts rather than steady physical properties.

Inter-Case Consistency Testing: To determine whether the LLM-generated data captures the true electrical characteristics, the distributions of parameter estimates from Cases 1 and 2 are statistically compared to those from Case 3. For each parameter, the two-sample Kolmogorov–Smirnov statistic is computed:

D_{ij} = \sup_x \left|F_i(x) - F_j(x)\right| \tag*{(51)}

In this equation, $F_i(x)$ and $F_j(x)$ represent the empirical cumulative distribution functions of the parameter estimates (aggregated across time windows or bootstrap samples) for cases $i$ and $j$ . A large $D_{ij}$ value with an associated $p$ -value less than 0.05 indicates a statistically significant difference, suggesting the generative model failed to replicate the underlying physics.

Comprehensive Residual Analysis

After fitting the model, the residual sequence $\epsilon(t_k) = v_s(t_k) - v_s^{\mathrm{model}}(t_k;\hat{\theta})$ undergoes rigorous diagnostic testing. The properties of these residuals reveal whether the model has successfully captured all systematic behavior or if unmodeled dynamics remain.

Whiteness Test (Autocorrelation Analysis): The sample autocorrelation function of the residuals is computed:

R_{\epsilon\epsilon}(\tau) = \frac{1}{N-\tau}\sum_{k=1}^{N-\tau}\epsilon(t_k)\epsilon(t_{k+\tau}) \tag*{(52)}

For a well-specified model, the residuals should approximate white noise, meaning $R_{\epsilon\epsilon}(\tau) \approx0$ for all $\tau> 0$ within statistical confidence bounds. Significant autocorrelation at non-zero lags indicates the presence of unmodeled temporal structure, which in LLM-generated data could stem from an overly simplistic or periodic generation algorithm.

Normality Test: The distribution of residuals is tested for Gaussianity using the Shapiro–Wilk test. The test statistic $W$ is calculated as:

W = \frac{\left(\sum_{k=1}^{N} a_k\epsilon_{(k)}\right)^2}{\sum_{k=1}^{N}(\epsilon_k-\bar{\epsilon})^2} \tag*{(53)}

Here, $\epsilon_{(k)}$ are the ordered residuals, and $a_k$ are tabulated coefficients. Measurement noise in a properly calibrated system, resulting from the combination of many independent sources, tends toward a Gaussian distribution. Significant deviations from normality ( $p$ -value < 0.05) in the residuals of LLM-generated data may indicate that the synthetic noise does not emulate the statistical central limit behavior of physical sensors.

Heteroscedasticity Analysis: The assumption of homoscedasticity-constant variance of residuals -is tested by examining the relationship between residual magnitude and the current level. The Spearman rank correlation coefficient $\rho_s$ between $|\epsilon(t_k)|$ and $i(t_k)$ is computed:

\rho_s = 1 - \frac{6\sum_{k=1}^{N} d_k^2}{N(N^2-1)} \tag*{(54)}

In this formula, $d_k$ is the difference between the ranks of $|\epsilon(t_k)|$ and $i(t_k)$ . Genuine sensor noise is typically homoscedastic across the operational current range. A significant correlation (e.g., $\rho_s > 0.3$ with $p$ -value $< 0.05$ ) suggests heteroscedasticity, which could be an artifact of an LLM incorrectly scaling noise with signal amplitude.

Goodness-of-Fit Metrics

Multiple quantitative metrics provide a summarized assessment of how well the Modified Ohm’s Law describes each dataset.

Normalized Root Mean Square Error (NRMSE): This scale-independent metric expresses the root mean square error as a percentage of the mean signal:

\mathrm{NRMSE} = \frac{1}{\bar{v}_s}\sqrt{\frac{1}{N}\sum_{k=1}^{N}\epsilon^2(t_k)} \times100\% \tag*{(55)}

Here, $\bar{v}_s$ is the mean measured voltage.

Coefficient of Determination ( $R^2$ ): This metric quantifies the proportion of variance in the data explained by the model:

R^2 = 1 - \frac{\sum_{k=1}^{N}\epsilon^2(t_k)}{\sum_{k=1}^{N}(v_s(t_k)-\bar{v}_s)^2} \tag*{(56)}

Akaike Information Criterion (AIC): The AIC balances model fit with complexity, penalizing the use of additional parameters:

\mathrm{AIC} = N\ln\left(\frac{\sum_{k=1}^{N}\epsilon^2(t_k)}{N}\right) + 2p,\quad p=3 \tag*{(57)}

For the experimental data (Case 3), we expect a high $R^2$ ( $> 0.95$ ) and a moderate AIC, indicating the model adequately captures the dominant linear resistive behavior. LLM-generated data may exhibit two distinct failure modes: a poor fit (low $R^2$ , high NRMSE) if it fails to respect the fundamental voltage-current relationship, or a suspiciously perfect fit (very high $R^2$ approaching 1.0) if it lacks realistic measurement noise and thus overfits to a simplistic pattern.

Quantitative Metrics for Generative Fidelity Assessment

To objectively rank the three datasets by their adherence to physical reality, a composite Generative Fidelity Score (GFS) is calculated. This score integrates the multi-faceted diagnostic analyses into a single, interpretable metric.

Component Metrics

The GFS is a weighted sum of three normalized component metrics, each scaled from 0 to 100, where 100 represents perfect alignment with the characteristics of the experimental ground truth data.

\mathrm{GFS} = w_1 \cdot M_{\mathrm{phys}} + w_2 \cdot M_{\mathrm{stat}} + w_3 \cdot M_{\mathrm{model}} \tag*{(58)}

The weights reflect the relative importance of each aspect for scientific validity: $w_1 = 0.5$ for adherence to physical constraints, $w_2 = 0.2$ for statistical plausibility, and $w_3 = 0.3$ for model fit quality.

Physical Constraint Metric ( $M_{\mathrm{phys}}$ ): This metric quantifies how well the data satisfies fundamental physical laws, such as Ohm’s Law and energy conservation.

M_{\mathrm{phys}} = 100 \times\exp\left(-\frac{1}{3}\left[\frac{\mathrm{RMSE}_{R}}{\sigma_{R}} + \frac{\mathrm{RMSE}_{P}}{\sigma_{P}} + \frac{|\Delta E|}{E_{\mathrm{exp}}}\right]\right) \tag*{(59)}

Here, $\mathrm{RMSE}_{R}$ and $\mathrm{RMSE}_{P}$ are the root mean square errors from the Ohmic (Equation (45)) and power consistency checks (Equation (46)), respectively. $\sigma_{R}$ and $\sigma_{P}$ are the expected measurement uncertainties propagated from instrument specifications. $|\Delta E|$ is the absolute difference between the dataset’s computed total energy (Equation (47)) and the experimental benchmark $E_{\mathrm{exp}}$ .

Statistical Properties Metric ( $M_{\mathrm{stat}}$ ): This metric measures the plausibility of the data’s noise characteristics and temporal structure.

M_{\mathrm{stat}} = 100 \times\frac{1}{4}\left[D_{\mathrm{KS}}(\mathrm{PDF}) + D_{\mathrm{KS}}(\mathrm{ACF}) + D_{\mathrm{KS}}(\mathrm{PSD}) + (1 - |\rho_s|)\right] \tag*{(60)}

The terms $D_{\mathrm{KS}}(\mathrm{PDF})$ , $D_{\mathrm{KS}}(\mathrm{ACF})$ and $D_{\mathrm{KS}}(\mathrm{PSD})$ are the complements (i.e., 1 minus the value) of the Kolmogorov-Smirnov distances comparing the dataset’s probability density function, autocorrelation function, and power spectral density to those of the experimental data. $\rho_s$ is the Spearman correlation from the heteroscedasticity analysis (Equation (54)).

Model Fit Metric ( $M_{\mathrm{model}}$ ): This metric evaluates the quality of the fit to the Modified Ohm’s Law.

M_{\mathrm{model}} = 100 \times\frac{1}{4}\left[R^2 + \left(1 - \frac{\mathrm{NRMSE}}{100}\right) + W + (1 - D_{\mathrm{KS}}(\theta))\right] \tag*{(61)}

The components are the coefficient of determination $R^2$ (Equation (56)), the complement of the normalized RMSE (Equation (55)), the Shapiro-Wilk statistic $W$ (Equation (53)), and the complement of the maximum Kolmogorov-Smirnov distance $D_{\mathrm{KS}}(\theta)$ between the distributions of fitted parameters $(\hat{R}_{\mathrm{sc}}, \hat{n}, \hat{I}_{s})$ and those from the experimental fit.

Interpretation of GFS Values

The experimental dataset (Case 3) serves as the benchmark. Due to inherent, irreducible measurement noise and minor model imperfections, its GFS typically falls in the range of 85–95. Scores for the LLM-generated data are interpreted as follows:

GFS of 70–85: The data exhibits minor deviations from physical reality but remains generally plausible. This range might indicate reasonably successful guided generation.
GFS of 50–70: The data shows significant physical inconsistencies, though it may retain some realistic surface features. Unguided generation often falls here.

GFS below 50: The data demonstrates a fundamental failure to capture essential physical characteristics, revealing the limitations of pure statistical pattern matching.

The GFS provides a quantitative, standardized measure for comparing the effectiveness of different LLM prompting strategies and for tracking the evolution of generative model capabilities over time.

Distinguishing Linear from Nonlinear Response Regimes

A powerful aspect of this analytical framework is its ability to dissect LLM performance based on the complexity of the underlying physical relationship. By separating instances where the system behaves linearly from those where nonlinear effects are significant, we can test whether an LLM’s success is limited to interpolating simple correlations common in its training data.

Regime Segmentation Criteria

The data is partitioned into two mutually exclusive regimes based on the instantaneous short-circuit voltage, which serves as a proxy for diode activation.

Linear (Resistive) Regime: This regime comprises all data points where the magnitude of the short-circuit voltage is below the typical forward conduction threshold of a silicon diode:

|v_s(t_k)| < 0.3\ \mathrm{V} \tag*{(62)}

In this condition, the diode-clamp network remains effectively non-conductive, and the Modified Ohm’s Law (Equation (19)) reduces to a simple linear relationship:

v_s(t) \approx i(t)R_{\mathrm{sc}} \tag*{(63)}

The physics here is dominated by the metallic resistance of the fault path, representing a straightforward application of Ohm’s Law.

Nonlinear (Clamping) Regime: This regime includes all data points where the voltage magnitude reaches or exceeds the diode threshold:

|v_s(t_k)| \ge0.3\ \mathrm{V} \tag*{(64)}

Under these conditions, the diode network begins to conduct and clamp the voltage, introducing a more complex relationship between current and voltage. The full Modified Ohm’s Law must be considered:

v_s(t) = i(t)R_{\mathrm{sc}} + nV_T \ln\left(\frac{i(t)}{I_s}\right) \tag*{(65)}

While the primary current path remains resistive, the clamping action modifies the effective terminal voltage, creating a nonlinear signature in the $(v_s-i)$ data.

Comparative Analysis Across Regimes

For each dataset (Cases 1, 2, and 3) and within each defined regime, a focused analysis is conducted:

Linear Correlation: The Pearson correlation coefficient $r$ between $v_s$ and $i$ is computed for the Linear Regime.

Variance Explained: The coefficient of determination $R^2$ is calculated for both the simple linear model (Equation (63)) in the Linear Regime and the full nonlinear model (Equation (65)) in the Nonlinear Regime.

Parameter Estimates: The resistance $R_{\mathrm{sc}}$ is estimated via linear regression in the Linear Regime. The full parameter set $\{R_{\mathrm{sc}, n, I_{\mathrm{s}}\}}$ is estimated via nonlinear regression in the Nonlinear Regime.

Residual Diagnostics: The whiteness, normality, and homoscedasticity of the model residuals are analyzed separately for each regime.

The core hypothesis, framed in terms of the Novelty Horizon (Definition 2.2), is that LLM-generated data will demonstrate reasonable fidelity in the Linear Regime, where the governing relationship (Ohm’s Law) lies inside $\operatorname{conv}(\Omega_{\mathrm{train}})$ (i.e., within $H_N$ ). In contrast, performance will degrade catastrophically in the Nonlinear Regime, where the required understanding of diode clamping dynamics requires extrapolation outside $\operatorname{conv}(\Omega_{\mathrm{train}})$ and causal reasoning not encoded in $C_{\mathrm{train}}$ —placing the task beyond $H_N$ .

Sustenance Efficiency and the Role of Diode Capacitance

The experimental design incorporates diodes not as primary current carriers but as elements whose intrinsic junction capacitance provides a critical, non-dissipative function. This capacitance absorbs and releases charge during transient events, effectively preventing damaging current backflow and contributing to circuit stability over extended fault durations [30]. A key finding from the experimental work of [30] is that the energy dissipated as heat in the resistive path over a sustained fault is many orders of magnitude greater than the maximum possible energy that could be cycled in the diodes’ junction capacitance. This immense ratio, termed the sustenance efficiency, empirically validates that the observed sustained power output is a genuine result of governed resistive dissipation ( $i^2 R_{\mathrm{sc}}$ ), not a transient capacitive effect. For LLM-generated data to be physically authentic, it must implicitly reflect this reality: the time-series should be consistent with a sustained, steady power draw from the source, not with the decaying exponential signature of a capacitive discharge. Analyzing the stationarity of the power signal $P(t)$ and its spectral content provides a test for this characteristic.

Model Reduction and the Primacy of Resistive Behavior

Given the operational physics established in § 4.5, the primary objective of fitting the Modified Ohm’s Law (Equation (19)) to the experimental data is to obtain a robust estimate of the short-circuit resistance $R_{\mathrm{sc}}$ . The diode parameters $n$ and $I_{\mathrm{s}}$ , while mathematically necessary for the regression, are of secondary physical significance in this context. This leads to a model reduction interpretation: for the purpose of evaluating generative fidelity, the most critical outcome is the accuracy and stability of the $\hat{R}_{\mathrm{sc}}$ estimate. A successful generative model must produce data from which a stable, physically-plausible $R_{\mathrm{sc}}$ can be inferred. Therefore, the analysis places particular emphasis on the distribution and temporal stability of $\hat{R}_{\mathrm{sc}}$ estimates (Equation (50)) when comparing cases. An LLM that merely learns to output voltages and currents that are statistically correlated may still fail to generate data that yields a consistent, sensible value for this fundamental physical parameter.

Results

Temporal Signature Analysis and Immediate Generative Artifacts

The raw time-series data in Figure 1 provides the first visual discriminant between statistical generation and physical measurement. In Figure 1(a) and (b), Case 1 (script-based unguided LLM) for the 2.5V configuration presents voltage and current traces that are numerically perfect, with means $\bar{V}_s = 2.501$ V and $\bar{I}_s = 50.014$ A. The signals exhibit minimal, spectrally flat noise ( $\sigma_V = 0.025$ ) and maintain a constant computed resistance of $R_{\mathrm{calc}} \approx0.050\,\Omega$ , leading to a power time-series in Figure 1(d) of $P \approx125.1$ W. This artificial perfection stems from the model implementing an idealized Ohm’s Law, $V_s(t) = I_s(t)R_{\mathrm{calc}}$ , with additive white noise, entirely missing the nonlinear dynamics of the clamped diode network. Conversely, the experimental ground truth in Figure 1(i) and (j) (Case 3, 2.5V) reveals a starkly different reality. The measured voltage stabilizes at $\bar{V}_s = 0.913$ V, not 2.5 V, and the current at $\bar{I}_s = 8.462$ A, directly reflecting the limiting effect of the metallic short-circuit resistance $R_{\mathrm{sc}}$ . The temporal evolution shows authentic sensor noise with subtle low-frequency drift and a characteristic autocorrelation structure absent from the LLM outputs. The power signal in Figure 1(l), with a mean of 7.72 W, is three orders of magnitude lower than the unguided LLM’s prediction, immediately highlighting a catastrophic failure in energy scale comprehension. Case 2 (script-based guided LLM), shown in Figure 1(e) and (f), attempts to correct this scale by incorporating procedural knowledge. The generated voltage for the 2.5V case plummets to $\bar{V}_s = 0.0535$ V, and the current to $\bar{I}_s = 4.893$ A. While this demonstrates an ability to adjust numerical magnitude based on context, the resulting signals exhibit severe statistical pathologies, including extreme skewness ( $\gamma_{1,I} = -6.42$ ) and kurtosis ( $\gamma_{2,I} = 65.4$ ), indicative of a flawed noise-generation algorithm rather than a physically coherent simulation. The parallel direct-prompt generation, detailed in the supplementary information (SI, Figure S1), exhibits even more extreme temporal pathologies. The guided direct-prompt case shows current fluctuations exceeding 100 A, while the unguided direct-prompt case displays a perfectly constant mean resistance across the entire time series. These findings further underscore the fundamental disconnect from physical measurement, revealing that the pathologies are not confined to the script-based paradigm but represent a systemic failure.

Time-series data for short-circuit voltage, current, computed resistance, and instantaneous power across Case 1, Case 2, and Case 3 — Figure 1. Time-series data for short-circuit voltage, current, computed resistance, and instantaneous power across Case 1 (script-based unguided LLM), Case 2 (script-based guided LLM), and Case 3 (Experimental) for both 2.5V and 5.0V input configurations. (The constant, high-magnitude signals in Case 1 and the overly smooth transitions in Case 2 contrast with the noisy, physically plausible steady state of the experimental data).

Temporal Memory and Spectral Fingerprints of Generation

The analysis of temporal memory through autocorrelation functions (ACFs) in Figure 2 uncovers a deeper layer of generative artifice. For any real sensor system, inertia-thermal, electrical, or mechanical-imparts memory, causing measured values to be correlated over short time lags. The experimental data in Figure 2(i) and (j) for the 2.5V case displays this hallmark: the ACF of voltage decays slowly, with an initial value of $R_W(\tau= 1) = 0.468$ . This signifies that consecutive voltage samples are highly correlated, a property of bandwidth-limited analog measurement systems. The ACFs for both LLM-generated cases betray their algorithmic origin. Case 1, in Figure 2(a) and (b), shows ACFs that plummet to near-zero within 2–3 lags ( $R_W(\tau= 1) \approx0.048$ ), characteristic of white noise or an autoregressive model with an incorrectly short time constant. Case 2, in Figure 2(e), shows a similarly rapid decay. This demonstrates that LLMs lack a probabilistic model for the colored noise spectra intrinsic to physical instrumentation; they default to statistically independent, or “memoryless”, generation, violating a basic tenet of experimental time-series. The power spectral density (PSD) analysis in Figure 3 further distinguishes the generative origins of each dataset. Physical measurement noise typically follows a $1/f^\alpha$ spectrum (as seen in Figure 3(i)–(l) for Case 3), with increased power at lower frequencies. The experimental PSD for both voltage and current confirms this, showing elevated spectral density below 1 Hz. The peak frequency for the experimental 2.5V current PSD is a mere 0.173 Hz, reflecting slow, drift-dominated dynamics. The LLM-generated data spectra tell a different story. Case 1 data (Figure 3(a)–(Figure 3d)) shows pronounced, spurious spectral peaks at high frequencies (∼30 Hz), an artifact that suggests the underlying generation algorithm inadvertently introduced periodic components or used an insufficiently random number generator. Case 2 data (Figure 3(e)–(Figure 3h)), while less peaked, still deviates significantly from the $1/f$ form, displaying a flatter, more uniform spectral density that aligns with an oversimplified, frequency-independent Gaussian noise model added post-generation. The failure to replicate the authentic spectral signature of sensor noise underscores the LLMs’ lack of a grounded model for the measurement process itself. These temporal and spectral artifacts are not unique to the script-based generation paradigm. The direct-prompt data (SI, Figures S2, S3) shows analogous failures, including near-white-noise ACF in the unguided case and artificially high low-frequency power in the guided case, demonstrating that the inability to model realistic sensor dynamics is a systemic limitation.

Autocorrelation functions for voltage and current — Figure 2. Autocorrelation functions (ACF) for voltage and current. (The experimental data shows strong, slowly decaying temporal memory, while both LLM-generated cases exhibit ACFs that decay almost immediately to zero, indicating a lack of modeled system dynamics and improper noise correlation structure).

Power spectral density of voltage and current signals on a log-log scale — Figure 3. Power spectral density (PSD) of voltage and current signals on a log-log scale. (The experimental PSD shows a realistic $1/f$ -type noise profile. Case 1 reveals unnatural spectral peaks from its generation algorithm, and Case 2 shows an overly simplistic low-frequency concentration without the correct noise physics).

Statistical Distributions and the Signature of Token Probability

The probability density functions (PDFs) in Figure 4 offer a third discriminatory test. Measurement noise in a well-conditioned system arises from the summation of many independent error sources, typically converging toward a Gaussian distribution (Central Limit Theorem). The experimental current PDF in Figure 4(j) approximates this, with modest skew ( $\gamma_{1} = -0.299$ ) and kurtosis ( $\gamma_{2} = 0.634$ ). The PDFs for LLM-generated data reveal the underlying token probability distributions or simplistic random number generators. Case 1 data in Figure 4(b) and (d) are nearly perfectly Gaussian, a statistically unlikely outcome for real sensor data which often contains outliers. More tellingly, Case 2 data in Figure 4(f) and (h) exhibit pathological, multi-modal distributions with extreme skew and kurtosis ( $\gamma_{2,I} = 84.3$ for 5.0V). This suggests the guided model attempted to create a “clamping” effect by mixing two distinct statistical

Figure 4. Empirical probability density functions for voltage and current. (The experimental distributions are nearly Gaussian. Case 1 shows artificial Gaussianity from simple additive noise, while Case 2 exhibits extreme skewness and kurtosis, revealing a flawed “event-based” generative model inappropriate for continuous measurement).

populations but did so without a physically valid transition model, resulting in a distribution impossible to obtain from an analog-to-digital converter. The statistical signatures of the direct-prompt generation (SI, Figure S4) are even more revealing. The unguided direct-prompt data shows catastrophic distributional pathology, with kurtosis values exceeding 200, a direct consequence of the model’s underlying token probability distribution when generating long numerical sequences. This confirms that the Gaussian noise observed in the script-based generation was a relatively controlled artifact, whereas direct generation reveals a more fundamental instability in maintaining coherent numerical distributions.

Violations of Inviolable Physical Constraints

The most striking result, shown in Figure 5, provides unambiguous evidence of the LLMs’ lack of grounding in causal physical laws, directly violating Condition 2.2 of the Novelty Horizon (Definition 2.2). Condition 2.2 requires that the governing laws $\Phi$ (here, Ohm’s Law as a relationship between independently noisy measurements) be representable as causal mechanisms encoded in $C_{\mathrm{train}}$ . The LLM fails this condition entirely. For any real dataset, Ohm’s Law $v(t)=i(t)R(t)$ and the definitions of power $P(t)=v(t)i(t)=i^2(t)R(t)=v^2(t)/R(t)$ must hold exactly at every time step, up to measurement error. The experimental data in Figure 5(i)–(Figure 5l) confirms this: the Ohmic residual RMS is $5.78 \times10^{-3}\,\Omega$ , a small, non-zero value consistent with propagated sensor uncertainty, and the power residuals are at the machine precision limit ( $5.97 \times10^{-16}\,\mathrm{W}$ ), indicating perfect internal algebraic consistency. The LLM-generated data fails this basic test spectacularly. While Figure 5(a)–(Figure 5h) show power residuals also at machine precision—indicating the model performed the multiplications correctly—the Ohmic residuals are also at machine precision ( $5.31 \times10^{-17}\,\Omega$ ). This is a physical impossibility. In a real experiment, the resistance $R_{\mathrm{calc}}(t)$ is a derived quantity computed from the two independent measurements $v(t)$ and $i(t)$ . The presence of independent measurement noise guarantees that $v(t)/i(t)$ will not perfectly equal the recorded $R_{\mathrm{calc}}(t)$ at every instant. The fact that both Case 1 and Case 2 show perfect Ohmic consistency proves the LLM did not generate independent voltage and current traces. Instead, it generated one primary signal (likely current) and then computed the other using a deterministic, noise-free algebraic relationship, treating the physical law as a mathematical tautology rather than a relationship between independently noisy observables. This is the ‘perfect algebra artifact’-a direct empirical signature of a model operating beyond its Novelty Horizon (Condition 2.2). Critically, this perfect algebraic closure is not an artifact of the script-based generation method. The directly generated datasets (Supplementary Information, Figure S5) exhibit the same infinitesimal Ohmic residuals (on the order of $10^{-15}\ \Omega$ ), irrefutably proving that this violation of measurement independence is a fundamental consequence of the model’s training objective, independent of the generation paradigm. The persistent inability to satisfy Condition 2.2 of Definition 2.2-the causal structure of independent measurement is not represented in $C_{\mathrm{train}}$ -thus emerges as a robust, cross-paradigm hallmark of autoregressive LLM failure when tasked with generating data beyond the Novelty Horizon.

Residual plots testing adherence to Ohm’s Law and power conservation — Figure 5. Residuals testing adherence to Ohm’s Law and power conservation. The experimental data shows small, physically expected inconsistencies from measurement noise. (The LLM-generated data exhibits perfect algebraic closure (Ohmic residual $\approx10^{-17}\ \Omega$ versus experimental $10^{-3}\ \Omega$ ), a direct violation of Condition 2.2 of the Novelty Horizon (Definition 2.2)-the causal independence of voltage and current measurements is not encoded in $C_{\mathrm{train}}$ . This proves the voltage and current were not generated as independent measured quantities).

Failure to Conform to First-Principles Constitutive Laws

Fitting the Modified Ohm’s Law, $v_{\mathrm{s}}(t)=i(t)R_{\mathrm{sc}}+nV_{\mathrm{T}}\ln(i(t)/I_{\mathrm{s}})$ , provides a stringent test of whether the data conforms to the nonlinear physics of the experimental circuit. The results in Figure 6 are decisive. For the experimental Case 3, shown in Figure 6(i)–(Figure 6l), the model fits well, extracting a physically plausible short-circuit resistance $R_{\mathrm{sc}}\approx1.00\times10^{-3}\ \Omega$ . The negative $R^2$ values ( $-0.063$ , $-0.027$ ) are misleading; they occur because the variance of the fit residuals is slightly larger than the variance of the data around its mean, a consequence of the dominant linear relationship being fit with a three-parameter nonlinear model to noisy data. The NRMSE values of $3.63%$ and $0.71\%$ are reasonable. The LLM-generated data is completely incompatible with this physical model. Case 1, in Figure 6(a)–(Figure 6d), yields nonsensical parameters ($R_{\mathrm{sc}}\approx 0.03 $0.04\,\Omega$ , orders of magnitude too high) and extremely poor fits ( $R^2 \approx-0.45$ to $-0.61$ ). Case 2, in Figure 6(e)–(Figure 6h), fails even more catastrophically. The fitting algorithm hits the imposed parameter bounds ( $R_{\mathrm{sc}} = 0.001\,\Omega$ , $n = 1$ , $I_{\mathrm{s}} = 1 \times10^{-6}\,\mathrm{A}$ ), and the $R^2$ values are astronomically negative ( $-25967$ , $-8266$ ), with NRMSE exceeding 100%. This indicates the model-predicted voltage $v_{\mathrm{pred}}(t)$ is not even a remotely correct representation of the generated $v(t)$ data. The LLM’s guided attempt to simulate a diode-clamped short-circuit produced a voltage-current relationship that is fundamentally alien to the Shockley-derived equation that actually governs such a component’s behavior, even in a simplified form. The direct-prompt generation data (SI, Figure S6) reveals a more insidious form of failure: parameter compensation. The guided direct-prompt generation produces plausible short-circuit resistance estimates ( $R_{\mathrm{sc}} \approx1.1 \times10^{-3}\,\Omega$ ) by severely distorting the auxiliary diode parameters ( $I_{\mathrm{s}}$ three orders of magnitude too high), illustrating how a superficial metric can mask a deeper physical inconsistency.

Figure 6. Fit of the Modified Ohm’s Law to the data. The experimental data is well-described by the model, yielding a physically meaningful $R_{\mathrm{sc}}$ . The LLM-generated data yields nonsensical parameters and catastrophic fit quality, demonstrating no embedded understanding of the governing semiconductor physics.

Residual Diagnostics Revealing Algorithmic Artifacts

Diagnostics on the Modified Ohm’s Law fit residuals, shown in Figure 7, expose the internal logic of the generation process. For the experimental data in Figure 7(i)–(Figure 7l), the residuals are approximately Gaussian (Shapiro-Wilk statistic unavailable due to large N but visually confirmed) and show no strong correlation with current magnitude (Spearman $\rho\approx-0.04$ , 0.30), indicating homoscedastic noise. The Case 1 residuals in Figure 7(a)–(Figure 7d) also show no heteroscedasticity ( $\rho\approx-0.07$ , 0.01), but this is a trivial result of fitting a wildly inappropriate model to data generated from a different, simplistic linear assumption. The most telling diagnostic is for Case 2, in Figure 7(e)–(Figure 7h). The Spearman correlation between absolute residual and current is 0.999, with a $p$ -value of zero. This near-perfect correlation is a glaring artifact. It occurs because the LLM’s generation algorithm created a deterministic, nonlinear relationship between $v(t)$ and $i(t)$ that is different from the Modified Ohm’s Law. When the correct physical model is fit to this incorrect relationship, the error (residual) is a deterministic function of the current. This pattern is impossible in real experimental data, where model misfit is obscured by random, uncorrelated measurement noise. The LLM has generated a specious, alternative “physics” that is entirely self-consistent within its generation algorithm but bears no relation to the true underlying law.

Figure 7. Diagnostics of the Modified Ohm’s Law fit residuals. (The experimental residuals are well-behaved. Case 2 shows a deterministic, perfect correlation between residual magnitude and current ( $\rho\approx0.999$ ), an impossible artifact revealing the LLM generated data from an internally consistent but physically false algorithmic relationship).

Inability to Capture Regime-Dependent Physics

However, as established in § 4.5–§ 4.5, the primary observed behavior is linear resistive dissipation, with the diode network’s role being primarily capacitive and protective. The regime analysis in Figure 8 confirms the LLMs’ failure to replicate even this simplified reality. This failure is predicted by the Novelty Horizon framework (Definition 2.2): the nonlinear regime ( $|v_s| \ge0.3$ V) lies outside $\operatorname{conv}(\Omega_{\mathrm{train}})$ (Condition 2.1), as the specific conjunction of metallic short-circuit resistance and diode clamping dynamics is not representable as a convex combination of training examples. Moreover, the causal relationship between current and voltage in this regime involves semiconductor physics not encoded in $C_{\mathrm{train}}$ (Condition 2.2). The experimental data in Figure 8(i)–(Figure 8l) correctly resides entirely in the nonlinear regime ( $|v_s| \ge0.3$ V), as defined for analysis, though the dominant physics is the linear $i(t)R_{\mathrm{sc}}$ term. The LLM-generated data reveals profound conceptual errors. Case 1 data, in Figure 8(a)–(Figure 8d), also falls entirely in the nonlinear regime, but for the wrong reason: its voltages are erroneously high (2.5 V, 5.0 V). Case 2, in stark contrast, places all data points in the linear regime ( $|v_s| < 0.3$ V), as shown in Figure 8(e)–(Figure 8h). Despite being given the explicit “clamped diode” experimental context, the LLM failed to generate voltages that meaningfully exceed the diode forward threshold. It generated a low-voltage short but did not understand the clamping dynamics well enough to simulate the correct voltage magnitude ( $\approx0.7$ –1.0 V) that characterises the experimental steady state. It could only interpolate to a generic “low voltage” value. Furthermore, the negative linear correlations ( $r \approx-0.77, -0.71$ ) within its own linear regime are physically nonsensical for a passive resistive element, where voltage and current must be positively correlated. The direct-prompt generation (SI, Figure S8) confirms the regime-dependent nature of the failure.

Scatter plots separating data into linear and nonlinear regimes — Figure 8. Scatter plots separating data into linear ( $\lvert v\rvert< 0.3\ \mathrm{V}$ ) and nonlinear ( $\lvert v\rvert\ge0.3\ \mathrm{V}$ ) regimes. (Experimental data correctly resides in the nonlinear regime. Case 1 is erroneously in the nonlinear regime due to voltage magnitude errors. Case 2 incorrectly places all data in the linear regime with physically impossible negative correlations. This regime-dependent failure is predicted by the Novelty Horizon (Definition 2.2): the nonlinear regime lies outside $\operatorname{conv}(\Omega_{\mathrm{train}})$ (Condition 2.1), and the correct clamping dynamics require causal reasoning not encoded in $C_{\mathrm{train}}$ (Condition 2.2)).

Critically, the guided direct-prompt generation places nearly 30% of its data points in the linear regime ( $\lvert V\rvert< 0.3\ \mathrm{V}$ ), a physical impossibility for the shorted circuit. This demonstrates that providing procedural context allows the model to “know about” clamping, but not to understand it causally, leading it to treat the clamp as a conditional event rather than a fixed operational reality.

Quantitative Synthesis of Generative Fidelity

Figure 9 synthesizes the multi-faceted analysis into the Generative Fidelity Score (GFS). The component metrics $M_{\mathrm{phys}}$ (physical constraints), $M_{\mathrm{stat}}$ (statistical properties), and $M_{\mathrm{model}}$ (model fit) are computed per the framework in § 4.4. The experimental Case 3 serves as the benchmark, though its GFS is depressed due to the mismatch between the full Modified Ohm’s Law and the primarily resistive data, highlighting the sensitivity of the metric. The scores for the LLM cases are decisively lower, quantitatively confirming their failure. Case 1 scores catastrophically low on $M_{\mathrm{phys}}$ (0.0–0.6), as seen in Figure 9(b) and (f), due to its gross violation of energy scales and perfect, unphysical algebraic consistency. Its $M_{\mathrm{stat}}$ scores are moderate (60.6–62.3), reflecting its use of simple Gaussian noise, which captures some superficial statistical properties. Case 2 shows higher $M_{\mathrm{phys}}$ scores (72.5–75.2) in Figure 9(c) and (g), as it respected basic scaling and energy magnitudes better, but its $M_{\mathrm{stat}}$ scores are poor (37.5), penalized by its non-Gaussian, event-like distributions and poor spectral properties. Both cases have undefined $M_{\mathrm{model}}$ and thus GFS due to the complete failure of the physical model fit, represented as missing bars in Figure 9(a) and (b). The summary statistics in Figure 9(i) and (j) crystallize the outcome. The experimental Case 3 yields consistent, physically credible parameter estimates across both voltage configurations: $R_{\mathrm{sc}} \approx1.00 \times10^{-3}\,\Omega$ , $n \approx1.45$ – $1.66$ , $I_{\mathrm{s}} \approx6 \times10^{-9}$ to $9 \times10^{-9}$ A. Case 2, despite being guided, hits the algorithmic bounds on all parameters, a clear sign of generative pathology. Case 1 estimates a resistance two orders of magnitude too large. The LLMs did not merely add noise to correct data; they generated data from an entirely incorrect manifold in voltage-current-power space, one that is statistically plausible in a textual sense but geometrically incompatible with the manifold defined by the laws of physics.

Generative Fidelity Scores and component metrics — Figure 9. Generative Fidelity Scores (GFS) and component metrics. (Case 3 (Experimental) sets the benchmark. Case 1 fails utterly on physical constraints ( $M_{\mathrm{phys}} \approx0$ ). Case 2 shows moderate physical scaling but poor statistical properties and a complete failure in model fitting, resulting in an undefined overall GFS. Parameter summary tables highlight the physical implausibility of LLM-extracted parameters).

Discussion

Interpreting the Failure of Statistical Correlation

The combined results from our script-based generation framework and the parallel direct-prompt framework (detailed in the supplementary information) demonstrate a fundamental, rather than incidental, failure of Large Language Models to extrapolate beyond their training data to generate scientifically valid empirical data for a novel experimental configuration. This failure is not a blanket statement on all data generation, but a specific and profound disconnect between statistical correlation and causal physical reality that emerges at the Novelty Horizon. The LLM-generated datasets, whether unguided (Case 1) or procedurally informed (Case 2), consistently violated inviolable physical constraints, such as the independent measurement noise between voltage and current and the correct scaling of energy dissipation. These violations manifest as perfect algebraic consistency in Ohmic residuals and the generation of deterministic, rather than stochastic, relationships between variables. The consistency of these failures across two fundamentally different generation paradigms-one mediated by code and one producing numbers directly-elevates our findings from an observation about a specific implementation to a generalizable principle about the architectural limits of autoregressive LLMs. This result directly stems from the LLM’s architectural foundation as a next-token predictor, optimized for generating sequences that are statistically plausible within the context of its training corpus. Its “knowledge” of electrical circuits is a composite of textual descriptions, equations, and problem-solving templates it has encountered, not an internalized model of Kirchhoff’s laws or semiconductor physics. Consequently, it treats physical laws as mathematical tautologies to be applied post-hoc rather than as governing principles that generate the observed data through a noisy measurement process. This aligns with critiques by [46, 47], who argue that LLMs, devoid of grounded world models, are ultimately “stochastic parrots” reciting patterns without understanding. The experimental validation established in this paper provides concrete, quantitative evidence for this theoretical limitation within a specific engineering domain. This paper’s methodological choice—having the LLM generate a simulator—precisely isolates its capacity for constructing causal models from text. A counterfactual question arises: if provided a physics engine API (e.g., a SPICE simulator) and asked to parameterize it, would performance improve? Likely, yes—for interpolation tasks. The LLM could probably successfully fill in parameters for a standard diode model if prompted. However, for phenomena at the Novelty Horizon, such as the specific transient interaction of a metallic short with a protective diode capacitance network, the correct model topology and governing equations themselves are the unknowns. An LLM, reliant on the physics engine’s pre-defined equations, would be just as powerless to select or modify them appropriately as it was to code them from scratch. It might produce a numerically stable output, but one that is physically wrong, trading the glaring artifacts of a bad simulator for the insidious plausibility of a well-executed but misapplied one. The core limitation is not coding proficiency, but the lack of a judgment mechanism grounded in physical law.

The Past Information Boundary and the Novelty Horizon

To formalize the core limitation exposed by this work, we invoke the two complementary theoretical constructs introduced in § 2.6: the $BP$ (Definition 2.1) and the $HN$ (Definition 2.2). These constructs define the operational domain of an autoregressive LLM trained on a static corpus and provide predictive criteria for determining whether a given scientific task is feasible. For a model $M$ trained on a dataset $\mathcal{D} = \{(\mathbf{x}_i, y_i)\}_{i=1}^{N}$ with associated timestamps $\tau_i$ , the Past Information Boundary is the ordered triple $B_P(M) = (t_p, \Omega_{\mathrm{train}}, C_{\mathrm{train}})$ , where $t_p = \max_{i \in\{1,\ldots,N\}} \tau_i$ is the temporal frontier (the latest timestamp in $\mathcal{D}$ ), $\Omega_{\mathrm{train}} \subseteq\mathcal{X} \times\mathcal{Y}$ is the support of the empirical training distribution (the conceptual and relational frontier), and $C_{\mathrm{train}}$ is the set of all conditional probability distributions encoded in the model’s parameters (the statistical-correlation frontier). The model’s knowledge is strictly confined to $B_P(M)$ ; it has no mechanism for incorporating information whose factual content, conceptual structure, or relational geometry lies outside $\Omega_{\mathrm{train}}$ or whose temporal origin is $t > t_p$ . A task $\mathcal{T} = (\mathcal{I}, \mathcal{O}, \Phi)$ -defined by an input space $\mathcal{I}$ , an output space $\mathcal{O}$ , and governing laws $\Phi$ -lies beyond the Novelty Horizon $H_N$ (i.e., $M$ cannot reliably perform $\mathcal{T}$ ) if at least one of three conditions holds. Condition 2.1 (extrapolation outside $\operatorname{conv}(\Omega_{\mathrm{train}})$ ) applies when the task requires generating an output $o^* \in\mathcal{O}$ such that, when projected onto the space of training examples via a mapping $\pi: \mathcal{O} \to\mathcal{X} \times\mathcal{Y}$ , the point $\pi(o^*)$ falls outside the convex hull of the training support. Because $M$ is a statistical interpolator, it lacks any basis for generating such $o^*$ . Condition 2.2 (causal disentanglement not present in $C_{\mathrm{train}}$ ) applies when the governing laws $\Phi$ require reasoning via causal mechanisms that are not reducible to the statistical correlations learned from text—specifically, if the conditional independencies and counterfactual dependencies required by $\Phi$ are not encoded in $C_{\mathrm{train}}$ . Condition 2.3 (temporal extrapolation beyond $t_p$ ) applies when the task concerns events, theories, measurement outcomes, or data that first become available at a time $t > t_p$ . Since the training data contains no information from the future, any output pertaining to such post- $t_p$ phenomena must be generated by extrapolation from past patterns; when the true post- $t_p$ distribution deviates from the historical continuation (e.g., a paradigm shift or a genuinely novel experimental configuration), the model’s extrapolation is ungrounded. The diode-clamped short-circuit experiment described in § 3 is engineered to reside precisely at - and partially beyond - the Novelty Horizon. Condition 2.1 applies because the specific, time-resolved conjunction of nonlinear metallic resistance, semiconductor clamping dynamics, and authentic noise characteristics of the ACS712T sensor defines a point in experimental configuration space that lies outside $\operatorname{conv}(\Omega_{\mathrm{train}})$ . Condition 2.2 applies because the correct causal structure requires independent voltage and current noise processes, whereas the LLM treats Ohm’s Law as an algebraic tautology. Condition 2.3 applies because the detailed noise spectrum and transient interaction of the short circuit with diode capacitance are not present in standard electrical engineering textbooks; the training data’s temporal frontier $t_p$ predates the comprehensive characterisation reported in [30]. The formal bounds also yield predictive principles that align with the observed empirical results. Smooth degradation within the horizon: as a task approaches the boundary of $\Omega_{\mathrm{train}}$ but remains inside $\operatorname{conv}(\Omega_{\mathrm{train}})$ , performance degrades gracefully, with increasing but still correctable errors. Catastrophic collapse beyond the horizon: when the task demands extrapolation satisfying any of the three conditions, performance collapses discontinuously. This explains the regime-dependent failure documented in § 5.7: the linear regime ( $|v_s| < 0.3$ V) lies inside $H_N$ (interpolation of common Ohm’s law correlations), whereas the nonlinear clamping regime ( $|v_s| \ge0.3$ V) lies outside it (extrapolation requiring novel synthesis of diode physics and measurement noise). The revealed findings resonate with and extend the existing literature on LLM limitations in scientific contexts. Previous work has shown that LLM performance is heavily reliant on the density of relevant examples in the training set, and that true extrapolation beyond the training distribution remains elusive [48, 49]. The present results align with this view: the models successfully interpolated to simple linear resistive behavior (abundantly documented in training corpora) but failed catastrophically when required to extrapolate to the nonlinear clamping regime, whose specific interplay of metallic resistance and semiconductor dynamics had not been statistically encoded in their training data. Moreover, the validated experimental framework from [45, 30] provides a rigorous ground-truth benchmark that confirms the absence of such encoding. Crucially, our formalisation of $B_P$ and $H_N$ goes beyond prior descriptive accounts by offering predictive criteria that can be applied prospectively: any task that satisfies one or more of the three conditions is predicted to be infeasible for a pure autoregressive LLM, regardless of scaling or prompt engineering. This theoretical contribution sharpens the understanding of why LLMs fail where human scientists succeed, and it establishes a clear benchmark against which future claims of “scientific AI” must be evaluated.

Comparison with Prior Studies on AI in Scientific Research

The established failures expand upon a growing body of literature critically examining AI’s role in science. Previous studies have shown LLMs can be useful for literature review, code generation, and drafting [50, 51]. However, their performance sharply declines when tasks require precise factual recall, complex multistep reasoning, or the generation of novel data [52, 53]. This work moves beyond these general assessments to perform a forensic analysis of why LLMs fail at empirical data generation. We confirm that providing detailed procedural context (Case 2) can steer outputs toward more plausible numerical scales, as also observed by [54] in their study on prompt engineering for scientific tasks. Yet, this guidance is insufficient to impart true physical understanding. The guided model still produced pathological statistical distributions and a voltage-current relationship that was deterministically wrong, as evidenced by the near-perfect Spearman correlation ( $\rho\approx0.999$ ) between fit residuals and current. This indicates the model created an internally consistent but physically false generative algorithm-a “hallucinated” physics. This aligns with the concerns established by [55], who differentiate between factually incorrect hallucinations and logically consistent but unfounded fabrications, the latter being particularly dangerous in scientific contexts as they can appear superficially credible. The present study’s dual-framework design-comparing script-based to direct-prompt generation-yields unique insights that extend beyond the existing literature. It reveals that the “perfect algebra artifact” is not a bug of code generation but a symptom of the model’s deeper conceptual failure to grasp measurement independence. The direct-prompt generation’s catastrophic kurtosis (exceeding 200 in the unguided case), which was absent in the script-based version, shows that the script-based framework’s structure-the explicit for loop, the defined time step $dt$ , the sequential execution-inadvertently imposes a temporal stability and numerical regularity that masks the model’s most severe statistical pathologies. When the model is freed from these structural constraints and asked to output numbers directly, its underlying token probability distribution is exposed, revealing a fundamental instability in generating long, coherent numerical sequences. This methodological insight is crucial for designing future AI evaluations: the mode of interaction fundamentally shapes the nature of the failure we observe. Evaluations that rely solely on code-generation tasks may underestimate the severity of a model’s limitations, as the structure of the code itself can impose a semblance of coherence that is absent in direct numerical output.

The Regime-Dependent Nature of LLM Competence

A particularly illuminating result is the stark contrast in LLM performance between linear and nonlinear physical regimes. In the defined linear regime ( $|v_{\mathrm{s}}| < 0.3$ V), where Ohm’s law $v = iR$ dominates, one might expect an LLM to perform adequately, given the ubiquity of this relationship in its training data. The Case 2 model did indeed generate data that fell entirely within this linear voltage range. However, even here it failed, producing a negative linear correlation between voltage and current, a physical impossibility for a passive resistor. This reveals that the model’s grasp of even basic linear relationships is superficial and can be corrupted when attempting to integrate additional contextual constraints (like “clamped diode”). In the nonlinear regime, which characterized the actual experimental data, both LLM cases failed completely. Case 1 placed data here for the wrong reason (erroneously high voltages), while Case 2 failed to generate data that entered this regime at all. This regime-dependent collapse in fidelity provides strong empirical support for the theory that LLMs excel at interpolation within the convex hull of their training data distribution but lack the capacity for reliable extrapolation, especially when underlying principles change [56, 57]. The transition from linear to nonlinear behavior represents a shift in governing equations that the LLM cannot dynamically infer or apply; it can only mimic the textual description of such a transition.

The Epistemology of Measurement: An Unsolved Challenge for Generative AI

Beyond the failure to capture correct physics, the LLMs demonstrated a complete lack of a model for the measurement process itself. Authentic experimental data is a convolution of the underlying physical phenomenon and the artifacts of the sensing system-including bandwidth limitations, thermal noise, quantization error, and long-term drift. The autocorrelation functions and power spectral densities of the experimental data tell the story of a specific, bandwidth-limited analog sensor system. The LLM-generated data, in contrast, exhibited the statistical signatures of its internal algorithmic process: unnaturally fast ACF decay, spurious spectral peaks (Case 1), or simplistic, frequency-independent noise (Case 2). The models have no concept of a Hall-effect sensor’s transfer function, the averaging of 10,000 ADC samples, or the $1/f$ noise characteristics of electronic components. This failing is profound because it separates calculation from measurement. An LLM can correctly calculate power as $v \times i$ (hence the machine-precision power residuals), but it cannot simulate the process of independently measuring $v$ and $i$ with real instruments. This underscores a critical epistemological point: scientific data is not merely a set of numbers obeying equations; it is the recorded output of a physical interaction with the world, imbued with the unique signature of the apparatus. Current LLMs, operating purely in the symbolic domain, are inherently incapable of replicating this aspect of scientific practice, a limitation also noted by [58, 59, 60] in their analysis of AI for scientific discovery. The fact that this fundamental failing is observed whether the LLM is asked to simulate the experiment by writing code or to output the numbers directly demonstrates that the flaw is not in the model’s ability to write Python, but in its inability to conceive of a measurement process. It treats the final numerical value as a point estimate to be calculated, rather than a sample from a distribution to be recorded-a distinction that lies at the heart of experimental science. The script-based generation imposes a procedural structure that partially conceals this conceptual gap, producing data that appears more stable and Gaussian. The direct-prompt generation, by removing that structure, exposes the raw instability of the model’s numerical output. Together, these two frameworks reveal that the model’s failure is not a matter of implementation, but of epistemology: it does not understand what it means to measure.

Implications for Researchers, Educators, and Industries

The implications of these findings are significant for all stakeholders in the scientific and technical ecosystem.

For researchers, LLMs must be viewed as powerful, yet fundamentally limited, tools for ideation, drafting, and coding assistance. They cannot be trusted as oracles of factual knowledge or as simulators of physical reality. Our study provides a clear methodology-centered on testing for physical consistency, residual diagnostics, and regime analysis-that researchers can adopt to vet any AI-generated scientific content or data. The onus is on the human expert to provide the causal model and validate all outputs against it. This reinforces the argument of [61, 62] that AI should act as a “collaborator” under strict human supervision, not an autonomous agent.

For educators in engineering and physical sciences, this work offers a critical pedagogical tool. Students must be taught to recognize the difference between arriving at an answer through the application of first principles and arriving at a statistically plausible answer. Using LLM failures as case studies, such as the generation of data with negative resistance correlations or perfect Ohmic consistency, can vividly illustrate core concepts of measurement uncertainty, experimental design, and the nature of physical laws. It underscores the irreplaceable value of hands-on laboratory work in developing true understanding.

For industry, where AI is increasingly deployed for design, testing, and failure analysis, the risks are substantial. Relying on LLM-generated data or conclusions for safety-critical systems-such as electrical fault analysis, material stress testing, or chemical process design-could lead to catastrophic outcomes. Industries must implement rigorous, physics-based verification and validation pipelines for any AI-generated engineering output. The Generative Fidelity Score (GFS) framework proposed in this paper could serve as one component of such a validation suite.

For peer reviewers and journal editors, the results sound a strong cautionary note. The surface plausibility of AI-generated text, figures, and even data tables means that synthetic content could become increasingly difficult to detect. Journals must strengthen policies requiring explicit disclosure of AI use in manuscript preparation and, crucially, in data generation. Reviewers must be encouraged to apply heightened skepticism and

to scrutinize methodological sections for physical implausibilities, such as mismatches between described equipment and reported noise characteristics, or a lack of discussion of measurement uncertainty.

The Structural Limitation and the Path to Augmented Intelligence

The central thesis confirmed by this experiment is structural, not incidental. The transformer architecture, trained via next-token prediction on a static corpus, is inherently retrospective. It is a mechanism for recombining and interpolating the past. Scientific progress, in its most valuable form, is prospective-it involves creating new information structures that did not exist before, whether through novel experiments, new theoretical syntheses, or the discovery of unforeseen phenomena. This is the essence of the Novelty Horizon. Therefore, the future of AI in science does not lie in expecting autoregressive LLMs to become autonomous discoverers. Instead, the path forward is Augmented Intelligence, where AI tools are deeply integrated into a human-driven scientific process. In this paradigm, LLMs can help survey literature, generate hypotheses for human evaluation, draft experimental protocols, and assist in data analysis-but the core acts of causal reasoning, experimental design, and grounding in physical reality remain human responsibilities. Hybrid neuro-symbolic systems that combine pattern recognition with formal, causal world models may eventually extend the Past Information Boundary, but they will likely remain tools for augmentation.

The Formal Bounds as Predictive Tools: Practical Applications of $B_P$ and $H_N$

The formalisation of $B_P$ and $H_N$ is not merely a post-hoc explanation of empirical failures; it provides predictive principles that can guide researchers, educators, and practitioners in assessing whether a given scientific task is feasible for an LLM without running exhaustive experiments.

Predicting Task Feasibility. Given a candidate task $\mathcal{T} = (\mathcal{I}, \mathcal{O}, \Phi)$ and knowledge of a model’s training corpus (or reasonable assumptions about its composition), one can assess:

Condition 2.1 (Extrapolation): Does the required output $o^*$ lie within $\operatorname{conv}(\Omega_{\mathrm{train}})$ ? For continuous scientific variables (voltages, currents, temperatures, concentrations), this requires estimating whether the specific combination of parameter values has been observed in training. Tasks requiring parameter combinations outside the convex hull of training examples are predicted to fail catastrophically.

Condition 2.2 (Causal Structure): Are the governing laws $\Phi$ representable as statistical correlations in $C_{\mathrm{train}}$ , or do they require genuine causal reasoning? Tasks involving independent measurement noise, counterfactual interventions, or feedback loops are unlikely to be captured by next-token prediction alone. The LLM may produce outputs that satisfy algebraic relationships (e.g., $v = iR$ ) but will fail to reproduce the statistical structure of independent measurements (e.g., non-zero Ohm’s law residuals).

Condition 2.3 (Temporal Extrapolation): Does the task require knowledge of phenomena discovered or characterised after the model’s training cutoff $t_p$ ? If so, the model cannot possess ground-truth information and will resort to extrapolation from historical patterns-a process that fails when the post- $t_p$ distribution deviates from historical continuation.

Why Fine-Tuning Does Not Cross the Horizon. A common misconception is that fine-tuning an LLM on new experimental data will enable it to generate valid data for novel configurations. The formal bounds clarify why this is insufficient. Fine-tuning shifts $t_p$ (adding more recent data) and may expand $\Omega_{\mathrm{train}}$ (adding new examples). However, it does not address Condition 2.2: the model remains a next-token predictor optimised for statistical correlation, not causal reasoning. Unless the fine-tuning data explicitly contains the correct causal structure (e.g., independent measurement noise, intervention data, counterfactual examples), the model will continue to treat physical laws as algebraic tautologies. Moreover, fine-tuning cannot create extrapolation capabilities beyond $\operatorname{conv}(\Omega_{\mathrm{train}})$ ; it merely expands the convex hull.

Practical Heuristics for Researchers. Based on the formal bounds, we propose the following heuristics:

If a task requires generating numerical data for a configuration that is not a convex combination of training examples, do not trust LLM-generated outputs. Validate against first-principles models or ground-truth experiments.
If a task involves causal relationships that are not reducible to statistical correlations (e.g., independent measurements, feedback, interventions), assume LLM failure. The ‘perfect algebra artifact’ (§ 5.4) is a reliable indicator of this failure mode.
If a task concerns phenomena discovered after the model’s training cutoff, treat LLM outputs as untested extrapolations. The model may produce plausible-sounding but factually incorrect outputs.
To detect whether an LLM has operated beyond $H_N$ , apply the forensic framework of § 4: test for Ohmic residuals (Condition 2.2), regime-dependent performance (Condition 2.1), and spectral/temporal signatures (Condition 2.3).

Relationship to Interpolation vs. Extrapolation in High-Dimensional Spaces. The convex hull condition (Condition 2.1) connects the Novelty Horizon to the classical statistical distinction between interpolation and extrapolation. In the high-dimensional token embedding space of an LLM, $\operatorname{conv}(\Omega_{\mathrm{train}})$ defines the region where the model can reasonably interpolate. Tasks requiring points outside this convex hull demand extrapolation-a capability that autoregressive models trained with next-token prediction do not possess. The diode-clamped short-circuit experiment, with its specific conjunction of parameters and dynamics, lies outside this convex hull, explaining the catastrophic failure observed in the nonlinear regime.

Conclusion

This paper establishes and characterises a critical boundary for autoregressive large language models in empirical science: the Novelty Horizon $H_N$ (Definition 2.2). We have formally defined the Past Information Boundary $B_P = (t_p, \Omega_{\mathrm{train}}, C_{\mathrm{train}})$ (Definition 2.1) and the Novelty Horizon as the set of tasks satisfying at least one of three conditions: extrapolation outside $\operatorname{conv}(\Omega_{\mathrm{train}})$ , causal structures absent from $C_{\mathrm{train}}$ , or reliance on post- $t_p$ information. The multi-layered forensic analysis demonstrates that while linguistic fluency and statistical pattern recognition can mimic scientific discourse, they cannot achieve the causal, grounded understanding required to extrapolate and generate valid data for tasks beyond $H_N$ . The failure is comprehensive when the task lies beyond a model’s training distribution, as defined by these formal bounds. Generated outputs violated inviolable physical constraints, exhibited pathological statistical signatures, and proved geometrically incompatible with first-principles constitutive laws. Explicit procedural guidance merely steered outputs toward more plausible numerical scales while failing to impart genuine physical understanding, resulting in data with extreme statistical artifacts and a Generative Fidelity Score of zero. These empirical failures are explained and formalized by the theoretical constructs of the Past Information Boundary and the Novelty Horizon. A large language model’s knowledge is fundamentally an encapsulation of a static historical record, optimised for interpolation within that corpus. Generating valid data for a novel experiment, however, requires extrapolation beyond this boundary-a capability precluded by the model’s fundamental architecture under the conditions tested. The diode-clamped short-circuit experiment, with its specific nonlinear dynamics and sensor noise characteristics, resided squarely beyond the Novelty Horizon of the tested models. The models manipulated symbols describing “clamping” but could not simulate the underlying physics, confusing textual correlation with causal mechanism. This conclusion is powerfully reinforced by the parallel investigation using direct-prompt generation, presented in the supplementary information. The consistency of the failures-the perfect algebra artifact, the regime misplacement, and the pathological statistical signatures across these distinct generative paradigms-leaves no room for alternative interpretations. Indeed, the direct-prompt framework revealed even more extreme manifestations of these failures, with unguided generation producing kurtosis values exceeding 200 and guided generation exhibiting parameter compensation that masked plausible resistance estimates behind severely distorted diode parameters. That these pathologies persist and, in some cases, intensify when the mediating structure of code is removed confirms that the Novelty Horizon is not a quirk of the script-based methodology but a fundamental boundary inherent to the architecture and training of autoregressive language models. A careful interpretation of the paper results suggests that current LLMs, under the specific conditions of tasks that require extrapolation outside $\operatorname{conv}(\Omega_{\mathrm{train}})$ , causal reasoning absent from $C_{\mathrm{train}}$ , or post- $t_p$ information, show strong limitations in generating valid novel scientific data. This statement avoids an absolute claim that LLMs can never generate novel data; rather, it acknowledges that for the class of tasks defined by the three conditions, the observed failures are systematic and severe, and overcoming them would require foundational modifications to the models themselves. Consequently, the appropriate role for these tools must be understood in relation to this horizon. As established by Definition 2.2, tasks strictly inside $H_N$ (interpolation within $\operatorname{conv}(\Omega_{\mathrm{train}})$ ), routine synthesis using correlations in $C_{\mathrm{train}}$ , information from $t \leq t_p$ ) can be safely delegated to LLMs for efficiency-literature summarisation, code completion, formatting, and initial hypothesis generation. Tasks at or beyond $H_N$ (novel experimental configurations, extrapolation to unseen regimes, discovery of new causal relationships) demand the causal reasoning, experimental design, and physical grounding of human scientists. This paper provides a methodological framework for the physics-based authentication of data and a philosophical clarification: true scientific discovery operates at the frontier of the Novelty Horizon. That frontier is a boundary language models cannot navigate alone, bound as they are to the archive of the past ( $B_P$ ). The indispensable scientist, not the synthetic statistician, remains the sole author of discovery. Looking forward, crossing the Novelty Horizon will require not merely scaling existing architectures but developing domain-grounded, hybrid systems that integrate symbolic causal models with statistical pattern recognition. For the specific case of electrical short-circuit simulation-and more generally for any chaotic or strongly nonlinear physical system-a promising direction is the construction of background-oriented experimental simulators that embed first-principles knowledge directly into the generative loop. One concrete proposal is a neural-symbolic short-circuit engine that couples a pre-trained LLM with a library of validated component models (e.g., the bidirectional diode-clamp circuit, the Modified Ohm’s Law derived in § 3.1, and noise profiles of Hall-effect sensors). Instead of asking the LLM to generate data from scratch, the system would use the LLM to parse a natural-language experimental description, translate it into a parameterised configuration for a symbolic solver (e.g., a SPICE-like engine specialised for fault analysis), and then return the solver’s output as the “generated” data. Such an architecture would satisfy Condition 2.1 by operating entirely within the solver’s valid domain, and it would address Condition 2.2 by explicitly encoding the causal structure of measurement independence. More ambitiously, future systems could incorporate online learning of previously unseen nonlinear regimes—for instance, by allowing the symbolic solver to be augmented with data-driven corrections when the LLM detects a mismatch between its static component library and the user’s description of a genuinely novel configuration. This would require the model to possess metacognitive uncertainty estimation that triggers a “request for human guidance” or an automated search for new constitutive equations. Until such hybrid systems mature, the responsible path is to treat LLMs as powerful but bounded assistants, with human experts retaining ultimate responsibility for experimental design and physical interpretation. The Novelty Horizon is not a permanent barrier, but crossing it will demand a deliberate shift from pure autoregression to augmented, causally grounded intelligence.

Prompt for Unguided LLM Generation (Case 1 – Statistical Extrapolation)

This appendix contains the exact prompt used to generate the Case 1 dataset. It instructs a GPT-5.2 class model to synthesise short-circuit data given only basic nominal circuit parameters ( $2.5\,\mathrm{V}/0.25\,\mathrm{A}$ , $5.0\,\mathrm{V}/0.5\,\mathrm{A}$ ), without providing the specific diode-clamp experimental procedure. This tests the model’s intrinsic capacity for statistical extrapolation from fundamental electrical concepts, devoid of context regarding the novel measurement physics.

Prompt

“Imagine that you are conducting an experiment in two different electrical circuits with power inputs of $2.5\,\mathrm{V}$ , $0.25\,\mathrm{A}$ , and $5.0\,\mathrm{V}$ and $0.5\,\mathrm{A}$ , using an Arduino Uno microcontroller and the ACS712 20 A current sensor on a breadboard. Then, you have an electrical short in each of these two circuits. For this problem, provide a rigorous Python code that prompts the user to input the external circuit power configurations and the duration of the short-circuit; then, following the user inputs, generate the short-circuit fault data for short voltage, short current (as if the short-circuit voltage and current were measured by the current sensor), computed short resistance, and short power. Also, let the code include a function for saving the generated data file with all these fault data features for download. Compute the resistance and the power correctly and suitably, scientifically. After this code, please provide a description of the circuit configuration/design that you have experimented on, and also provide a theoretical description of how the resistance and the power data are computed. In each case, I want to generate the fault data for 10 minutes with 10-millisecond resolution, and I want you to record this data at intervals of two seconds. Include the time column in the generated data files.”

Commentary. The Linear Correlation Engine and Its Digital Artifacts

The code generated in response to this prompt typically implements a time-stepping simulator with a core physics model reduced to a single, static algebraic relationship. The LLM’s foundational assumption, drawn from the statistical prevalence of textbook problems, is that a short-circuit is a linear resistor. Its simulator logic follows:

Physics Kernel: It defines a nominal short-circuit resistance $R_{\mathrm{sc}}$ and enforces a pristine, noise-free version of Ohm’s Law at its core: $V_{\mathrm{true}}(t) = I_{\mathrm{true}}(t) \cdot R_{\mathrm{sc}}$ .

Signal Decoration: It then decorates these perfect signals with independent Gaussian noise: $V_{\mathrm{measured}}(t) = V_{\mathrm{true}}(t) + \mathcal{N}(0, \sigma_V)$ , $I_{\mathrm{measured}}(t) = I_{\mathrm{true}}(t) + \mathcal{N}(0, \sigma_I)$ .

Derived Quantity Calculation: Finally, it computes resistance and power from the noisy decorated signals as $R_{\mathrm{calc}}(t) = V_{\mathrm{measured}}(t)/I_{\mathrm{measured}}(t)$ and $P_{\mathrm{calc}}(t) = V_{\mathrm{measured}}(t) \cdot I_{\mathrm{measured}}(t)$ .

The analogy: This is similar to painting rust onto a brand-new steel beam to simulate age. The underlying structure is perfect and linear; the complexity is only a surface layer. This architecture guarantees two unphysical data artifacts:

The Perfect Algebra Artifact: Because $V_{\mathrm{true}}$ and $I_{\mathrm{true}}$ are perfectly coupled by $R_{\mathrm{sc}}$ , any deviation in the measured pair $(V_{\mathrm{measured}}, I_{\mathrm{measured}})$ is purely from independent noise. Therefore, the derived quantity $R_{\mathrm{calc}}(t)$ will have no mathematical relationship to the true $R_{\mathrm{sc}}$ used in the core kernel. The data thus violates the internal consistency of a real experiment, where $R_{\mathrm{calc}}(t)$ is estimated from the same two noisy measurements that are coupled by physics. This manifests as the infinitesimal Ohmic residual RMS (§ 5.4), a fingerprint of a simulator that confuses a physical law with an algebraic tautology.

The Spectral Anachronism: The use of independent, memoryless white noise generators fails to capture the temporal correlations (low-pass filtering) of real sensor systems and power supplies. This results in the anomalously fast-decaying autocorrelation functions (Figure 2) and the flat, non-1/ $f$ power spectral densities (Figure 3). The data speaks in the frequency domain with the accent of a random number generator, not a physical sensor.

This unguided prompt forces the LLM to operate entirely within its Past Information Boundary $B_P = (t_p, \Omega_{\mathrm{train}}, C_{\mathrm{train}})$ . The model has no access to the specific diode-clamp experimental procedure; it must rely solely on statistical patterns in $C_{\mathrm{train}}$ . The task-generating time-resolved short-circuit data with authentic sensor noise and nonlinear clamping dynamics—requires extrapolation outside $\operatorname{conv}(\Omega_{\mathrm{train}})$ (Condition 2.1 of Definition 2.2) and causal reasoning about independent measurements not encoded in $C_{\mathrm{train}}$ (Condition 2.2). The catastrophic failure observed in § 5 is therefore a direct empirical consequence of operating beyond the Novelty Horizon $H_N$ .

Prompt for Procedurally Guided LLM Generation (Case 2 – Informed Pattern Matching)

This appendix contains the exact prompt used to generate the Case 2 dataset. The prompt explicitly provides the full experimental procedure for the “Clamped Short Diodes Nominal” configuration from reference [30]. This tests the model’s ability to integrate specific domain knowledge and procedural context into its generative process, moving from pure extrapolation toward guided simulation of the described experiment.

Prompt

“Imagine that you are conducting an experiment in two different electrical circuits with power inputs of 2.5V, 0.25A, and 5.0V and 0.5A. Below, I have provided you with the uncommon diode clamp circuit being experimented on under all configurations, for your understanding and use. Then, you have an electrical short in each of these two circuits. For this setup, provide a rigorous Python code that prompts the user to input the external circuit power configurations and the duration of the short-circuit; then, following the user inputs, generate the short-circuit fault data for short voltage, short current, short resistance, and short power. Also, let the code save the generated data file, including all the data features, for download. Compute the resistance and the power correctly and suitably. After this code, also provide the circuit configuration/design that you have experimented on, and provide a theoretical description of how the resistance and the power data are computed. In each case, I want to generate the fault data for 10 minutes with 10-millisecond resolution, and I want you to record this data at intervals of two seconds. Include the time column in the generated data files. Consider the experiment description in the procedure provided in Table 2.

Commentary. Pattern Overload and the Emergence of Algorithmic Pathology

When provided with the detailed experimental procedure, the LLM’s generated code attempts to incorporate more complex patterns: diode clamps, averaging, and thresholding. However, it does so by stitching together conditional logic blocks rather than solving a unified physical model. Its simulator often adopts a structure like:

Branching Logic: It calculates an “unclamped” voltage. If $|V_{\mathrm{unclamped}}| > V_{\mathrm{clamp}}$ , it engages a “clamped” branch where voltage is fixed at $\pm V_{\mathrm{clamp}}$ and current is recalculated as:

\begin{cases} V(t) = I(t)R_{\mathrm{short}}, & \text{if } |V| < V_{\mathrm{clamp}} \\ V(t) = \operatorname{sign}(V)\cdot V_{\mathrm{clamp}},\quad I(t) = V(t)/R_{\mathrm{clamp,eq}}, & \text{otherwise} \end{cases}

Averaging as Post-Processing: The instruction to average 10,000 samples is often implemented as a simple noise reduction on the final time-series, $\sigma_{\mathrm{out}} = \sigma_{\mathrm{in}}/\sqrt{10000}$ , without modelling the underlying analog integration process.

The analogy: This is like a cook following a recipe by mechanically adding ingredients (patterns) without understanding how they react together. The result is a dish with isolated, overpowering flavours. In the data, this manifests as:

The Bimodal Catastrophe: The hard-switching between linear and clamped branches, each with its own noise model, creates a mixture of two statistical populations in the current and voltage data. This is the direct origin of the extreme skewness and kurtosis ( $\gamma_{2} > 65$ ) observed in § 5.3. The probability density function becomes a Frankenstein’s monster of two distributions, a signature impossible in a continuously measured physical system.

The Regime Misplacement: The LLM’s heuristic for the clamp threshold often misjudges the system’s operating point. Consequently, as seen in § 5.7, the entire generated dataset can reside in the incorrect physical regime (e.g., all data points below 0.3V). The simulator has the concept of a diode clamp from its training data, but not the causal understanding of energy balance and circuit loading required to correctly predict its activation point. It inserts the clamp as a narrative element, not a physical component.

This guided prompt provides the model with detailed procedural context, effectively expanding the conceptual frontier $\Omega_{\mathrm{train}}$ by including textual descriptions of the diode-clamp network. However, the model remains unable to cross the Novelty Horizon because the underlying causal structure-the Modified Ohm’s Law (Equation (19)) with independent measurement noise-is not representable as statistical correlations in $C_{\mathrm{train}}$ (Condition 2.2 of Definition 2.2). The model’s attempt to simulate clamping via conditional branching logic (“if $|V| > V_{\mathrm{clamp}}$ ”) produces the extreme kurtosis ( $\gamma_2 > 65$ ) and regime misplacement observed in § 5.3 and § 5.7. This demonstrates that providing procedural context shifts the model’s outputs within $BP$ but does not enable crossing $H_N$ —the fundamental limitation is architectural, not informational.

Experimental Framework for Case 3 – Adoption of a Pre-Validated Physical Design

Provenance of the Experimental Configuration and Data

Case 3, the physical ground truth dataset central to this paper’s forensic analysis, is not the product of a newly calibrated apparatus. Instead, it directly adopts the fully characterized experimental framework, measurement data, and validation outcomes from the recent foundational study, [30]. That work established a novel, governed short-circuit measurement system, culminating in a comprehensive, multi-layered sensor calibration and circuit validation framework. The present study leverages this rigorously validated foundation. The experimental design for Case 3 is precisely the “Clamped Short Diodes Nominal” configuration described in § 3.2.2 and illustrated in Figure 1 of [30]. This configuration, reproduced for clarity in Figure 10, integrates a three-diode (1N5408) clamping network (“Circuit Block 1” or CB1), an ACS712T-20A Hall-effect current sensor, an Arduino Uno microcontroller, and a programmable DC power supply. The core function of this architecture is to enforce a bounded electrical regime during a short-circuit event, enabling sustained, high-fidelity measurement of voltage and current without system destruction.

The data acquisition protocol follows the automated algorithm detailed in Table 2 of [30]. For the specific data used in this paper (10-minute duration, 10-ms base sampling), the Arduino firmware was configured to sample the analog voltage and current channels at 10-ms intervals, with each reported data point representing the average of 10,000 instantaneous samples to suppress quantization noise and aliasing, as per the Nyquist-Shannon criterion [30].

Adoption of Pre-Validated Data Without Recalibration

A critical distinction of this work is that it does not involve recalibrating the sensor system or revalidating the circuit’s clamping principle. That extensive work was conclusively performed and reported in [30]. The sensor calibration framework established in [30]-encompassing baseline validation, post-fault performance assessment under both nominal and extreme short-circuit conditions, and the derivation of the Consolidated Calibration Confidence Score (CCCS)-confirmed the measurement system’s integrity and resilience. The data utilized for Case 3 in this paper is drawn directly from the results of that validated experimental campaign, specifically the extended-duration (10-minute) tests under nominal short-circuit conditions at 2.5V and 5.0V inputs. Therefore, the voltage ( $v_s(t)$ ) and current ( $i(t)$ ) time-series for Case 3 carry with them the full weight of the calibration assurance demonstrated in [30]. Their statistical noise profiles, temporal stability, and adherence to fundamental circuit laws have already been authenticated against first principles and sensor specifications.

Role of Case 3 Data in the Present Study

In the context of this paper’s investigation into LLM capabilities, the pre-validated Case 3 data serves a singular, crucial purpose: to act as an immutable physical benchmark. It represents reality as measured by a trusted instrument whose errors and characteristics are known and bounded. The multi-layered analytical framework developed in § 4 of this paper-testing temporal statistics, spectral signatures, physical consistency, and fit to the Modified Ohm’s Law-uses this benchmark not to question the data’s validity, but to expose the deviations of the synthetically generated Cases 1 and 2 from physical plausibility. The governing physics for the system, derived in § 3.1 of this manuscript, provides the constitutive law against which all datasets are judged. The experimental data from [30] naturally conforms to this model because the model describes the physical reality that the apparatus in [30] was designed to measure. Thus, the failure of the LLM-generated data to fit this model or replicate the statistical fingerprints of the real measurement process (e.g., correct autocorrelation, $1/f$ noise spectra) is not a commentary on the experimental data’s quality, but a direct measure of the LLMs’ inability to generate outputs that reside on the physically-valid manifold defined by both the constitutive law and the authentic measurement process.

Epistemological Foundations of Physics-Based Authentication - Why Statistical Similarity Cannot Certify Causal Fidelity

This appendix develops the theoretical critique that motivates the forensic methodology of the main paper. It is placed here, rather than in the introduction, to allow the empirical argument to proceed without interruption while providing a self-contained reference for readers interested in the philosophical and statistical foundations of the authentication framework.

The Reference Class Problem in LLM-Generated Scientific Data

The central epistemological difficulty in evaluating LLM-generated scientific data can be formulated as a reference class problem. Let $D_{\mathrm{real}}$ denote the set of all empirically measured datasets from a physical system governed by laws $\Phi$ , and let $D_{\mathrm{synth}}$ denote a dataset generated by an LLM. A researcher assessing $D_{\mathrm{synth}}$ has access only to its statistical properties—its marginal distributions, correlation structure, and spectral characteristics—computed relative to some reference class $\mathcal{R}$ of real datasets. The implicit inference is:

\operatorname{Stat}(D_{\mathrm{synth})}) \approx\operatorname{Stat}(D_{\mathrm{real}}) \text{ for } D_{\mathrm{real}} \in\mathcal{R}, \text{ then } D_{\mathrm{synth}} \text{ is empirically valid.}

This inference commits what may be termed the reference class fallacy: the statistical properties that define $\mathcal{R}$ are aggregates over many possible generative processes, only a subset of which respect the causal structure of $\Phi$ . A dataset may satisfy $\operatorname{Stat}(D) \approx\operatorname{Stat}(D_{\mathrm{real}})$ while violating $\Phi$ in fundamental ways, provided the violations are not captured by the chosen statistical tests. The diode-clamped short-circuit experiment provides a concrete instantiation of this fallacy. Consider the following statistical properties that a naive validation might examine:

Mean current $\bar{I}$ and mean voltage $\bar{V}$
Pearson correlation $\rho_{VI}$ between $V$ and $I$
Signal-to-noise ratio and variances $\sigma_I^2$ , $\sigma_V^2$
Power spectral density slope at low frequencies

An LLM could be trained—or explicitly prompted—to match these aggregate statistics. Indeed, the unguided generation (Case 1) successfully produced near-Gaussian noise and a high positive correlation between $V$ and $I$ , properties that superficially resemble the experimental data. Yet the forensic analysis in § 5.4–§ 5.5 revealed catastrophic failures: the Ohmic residuals were $10^{-17}\ \Omega$ rather than $10^{-3}\ \Omega$ , and the fitted Modified Ohm’s Law yielded $R^2 \approx-26000$ . The dataset was statistically plausible at the level of marginal moments but physically impossible at the level of causal structure. This exposes a general principle: statistical similarity is necessary but not sufficient for empirical validity. The sufficiency direction fails because the space of possible datasets consistent with a given set of statistical constraints is vast, and only a vanishingly small fraction of that space corresponds to trajectories that could arise from a physical measurement process governed by $\Phi$ .

The Measurement Process as a Hidden Variable

A deeper analysis reveals that the reference class fallacy originates in the omission of what philosophers of science call the measurement process as a latent variable. Let $\mathcal{M}$ denote the measurement apparatus—sensors, amplifiers, analog-to-digital converters, and their associated noise characteristics, bandwidth limitations, and calibration drifts. The observed data $D_{\mathrm{real}}$ is not a direct recording of the underlying physical state $S(t)$ but rather a convolution: $D_{\mathrm{real}}(t) = \mathcal{M}[S(t)] + \eta(t)$ , where $\eta(t)$ represents irreducible measurement noise with spectral properties determined by $\mathcal{M}$ . The causal graph is: $S(t) \to\mathcal{M}[S(t)] \to D_{\mathrm{real}}(t)$ . The measurement apparatus $\mathcal{M}$ is itself a physical system with its own dynamics, including thermal noise (Johnson-Nyquist), shot noise, $1/f$ flicker noise, quantization error, and finite bandwidth. These properties are not arbitrary; they follow from fundamental physics. For a Hall-effect sensor such as the ACS712T, the noise power spectral density has the form: $S_W(f) = \frac{4k_B T R}{1+(2\pi f\tau)^2} + \frac{K}{f^\alpha} + \frac{\Delta^2}{12f_s}$ , where the first term represents thermal noise filtered by the sensor bandwidth $\tau$ , the second term represents $1/f$ noise (with $\alpha\approx1$ ), and the third term represents quantization noise from the ADC with step size $\Delta$ and sampling frequency $f_s$ . An LLM trained on text lacks any representation of $\mathcal{M}$ . It has never experienced the physical act of measurement—the settling time of an ADC, the random walk of thermal noise, the saturation of a sensor amplifier. When asked to generate “experimental data,” the LLM defaults to the only generative model it possesses: the statistical patterns in its training corpus. But those patterns are predominantly textual descriptions of measurement outcomes, not the outcomes themselves. The model learns that experimental papers report means, standard deviations, and correlations; it does not learn the stochastic process $\mathcal{M}$ that generates those summaries from underlying physical states. Consequently, LLM-generated data exhibits what may be termed the measurement-blindness artifact: it treats the reported values as primitives rather than as samples from a measurement distribution. This manifests as:

Perfect algebraic closure (§ 5.4). Because the model computes derived quantities like $R = V/I$ from the same internally consistent numbers, rather than from independently noisy measurements, the residual $V - IR$ is zero to machine precision-a statistical impossibility for any real measurement system.

Incorrect noise spectra (§ 5.2). Lacking a model of sensor bandwidth and $1/f$ noise, the LLM defaults to white noise or simple autoregressive processes, producing autocorrelation functions that decay within 2–3 lags rather than the gradual decay characteristic of band-limited analog signals.

Regime misplacement (§ 5.7). Without a causal model of the diode clamping threshold, the LLM treats the nonlinear regime as a conditional branch rather than a continuous physical transition, resulting in data that either never enters the clamping regime (Case 2) or enters it for the wrong reasons (Case 1).

The Necessity of Forensic Physics-Based Authentication

The preceding analysis implies that validating LLM-generated scientific data requires moving beyond statistical similarity to forensic physics-based authentication-a methodology that tests for consistency with the causal structure of $\Phi$ and the measurement process $\mathcal{M}$ . The main paper develops precisely such a methodology, organized around three discriminative tests:

Test 1: Algebraic Independence. For a system governed by $\Phi: v = iR$ (in the linear regime), real measurements produce residuals $\epsilon_R = v/i - R_{\mathrm{calc}}$ that are non-zero and distributed according to the propagation of independent measurement errors. Specifically, if $v = v_{\mathrm{true}} + \eta_v$ and $i = i_{\mathrm{true}} + \eta_i$ with $\eta_v, \eta_i$ independent, then $\epsilon_R = (v_{\mathrm{true}} + \eta_v)/(i_{\mathrm{true}} + \eta_i) - R_{\mathrm{calc}} \ne0$ , and the distribution of $\epsilon_R$ has variance determined by the sensor noise levels. LLM-generated data that yields $\epsilon_R \approx0$ to machine precision (as in § 5.4) fails this test, revealing that $v$ and $i$ were not generated as independent measurements.

Test 2: Spectral Authenticity. For a measurement system with bandwidth-limited sensor and $1/f$ noise, the power spectral density $S(f)$ must satisfy $\lim_{f\to0} S(f) \propto1/f^\alpha$ , $\alpha\approx1$ , and the autocorrelation function $\rho(\tau)$ must decay slowly over lags $\tau$ comparable to the sensor time constant. LLM-generated data that produces white noise spectra (flat $S(f)$ ) or ACFs that drop to zero within a few lags (as in § 5.2) fails this test, revealing that the generation process lacks a model of $\mathcal{M}$ .

Test 3: Regime Continuity. For a system with a threshold phenomenon (e.g., diode clamping), the transition between linear and nonlinear regimes must be continuous in both value and derivative. Let $f(V,I)$ be the function describing the system’s current-voltage characteristic. Physical continuity requires $\lim_{V\to V_{\mathrm{th}}^-} f(V) = \lim_{V\to V_{\mathrm{th}}^+} f(V)$ and $\lim_{V\to V_{\mathrm{th}}^-} \mathrm{d}f/\mathrm{d}V = \lim_{V\to V_{\mathrm{th}}^+} \mathrm{d}f/\mathrm{d}V$ . LLM-generated data that exhibits a hard threshold (Case 2’s conditional branching) or that places all data in the wrong regime (§ 5.7) fails this test, revealing that the model has learned the concept of a threshold from text but not the physics of continuous transition.

The Generative Fidelity Score as an Epistemological Instrument

The Generative Fidelity Score (GFS) introduced in § 4.4 can be understood as an epistemological instrument designed to quantify the distance between a candidate dataset $D_{\mathrm{synth}}$ and the manifold $\mathcal{M}_{\Phi}$ of datasets that could arise from the causal process $(S,\mathcal{M})$ . The GFS synthesizes multiple discriminative tests into a single metric:

\mathrm{GFS}=w_1M_{\mathrm{phys}}+w_2M_{\mathrm{stat}}+w_3M_{\mathrm{model}}

where $M_{\mathrm{phys}}$ tests for violations of inviolable constraints (Test 1), $M_{\mathrm{stat}}$ tests for spectral authenticity (Test 2), and $M_{\mathrm{model}}$ tests for consistency with the first-principles constitutive law (Test 3). The catastrophic GFS values obtained for both LLM-generated cases (effectively zero, as the model fit component was undefined) provide quantitative confirmation that these datasets lie far from $\mathcal{M}_{\Phi}$ —not merely in terms of statistical divergence, but in terms of physical impossibility. Critically, the GFS framework is theory-dependent: it requires a first-principles model $\Phi$ of the system under study. This is not a weakness but an essential feature. There is no model-free method for authenticating scientific data because validity is always relative to a theoretical framework. The Modified Ohm’s Law derived in § 3.1 provides that framework for the diode-clamped short-circuit system. For other domains, analogous first-principles models must be developed. The implication is that physics-based authentication is domain-specific and theory-laden—a fact that LLM proponents who advocate for generic “AI validation” often overlook.

Broader Implications for AI-Supported Science

The epistemological critique developed here has implications beyond the specific case of short-circuit measurement. It suggests a general principle: LLMs are fundamentally unsuited for generating empirical data in domains where the causal structure of the measurement process is not fully captured by statistical patterns in text. This includes virtually all of experimental physics, chemistry, biology, and engineering, where the measurement apparatus $\mathcal{M}$ is a physical system with dynamics that are rarely described in sufficient detail in training corpora. This is not to say that LLMs have no role in science. They remain powerful tools for literature synthesis, code assistance, and hypothesis generation—tasks where the output remains in the discursive domain. But the frontier of LLM use has already extended beyond this domain into empirical data generation, as evidenced by studies such as [28] on synthetic clinical data. The present paper provides the methodological toolkit for authenticating such data and demonstrates, through rigorous empirical test, that current LLMs fail catastrophically when pushed beyond their Novelty Horizon. The way forward is not to ban LLMs from science but to recognize their fundamental limitation: they are retrospective statistical interpolators, not prospective causal engines. The Past Information Boundary $B_P=(t_p,\Omega_{\mathrm{train}},C_{\mathrm{train}})$ is not a bug to be fixed by scaling or prompt engineering; it is a structural consequence of the autoregressive next-token prediction objective. Tasks that lie beyond the Novelty Horizon $H_N$ —requiring extrapolation outside $\operatorname{conv}(\Omega_{\mathrm{train}})$ , causal reasoning not encoded in $C_{\mathrm{train}}$ , or information post-dating $t_p$ —will remain beyond the reach of pure LLMs regardless of model size or training data volume. Crossing $H_N$ will require fundamental architectural changes, including the incorporation of causal world models, grounding in physical simulators, and explicit representation of measurement processes—changes that would transform these systems into something other than LLMs as currently conceived. Until such hybrid systems mature, the responsible path is to treat LLM-generated numerical data as unvalidated conjecture, subject to forensic physics-based authentication before being admitted as evidence in scientific reasoning. The methodology developed in this paper provides a template for such authentication across domains, and the theoretical framework of $B_P$ and $H_N$ provides a predictive criterion for when such authentication is essential: whenever the task requires extrapolation beyond the convex hull of training examples, causal reasoning not reducible to statistical correlation, or knowledge of post-training phenomena.

References

[1]“Advancing AI for science: From the revolution of tools to the tools for revolution,” AI Open, vol. 6, pp. 323–328, Jan. 2025, doi: 10.1016/j.aiopen.2025.11.002.
[2]V. Fu, “AI for Science: Opportunities, Challenges, and Future Directions,” Journal of Data Science, pp. 1–19, Jan. 2026, doi: 10.6339/25-JDS1214.DOI
[3]T. Hartung, “AI, agentic models and lab automation for scientific discovery - the beginning of scAI nce,” Front. Artif. Intell., vol. 8, Aug. 2025, doi: 10.3389/frai.2025.1649155.DOI
[4]E. Chávez-Angel et al., “Applied Artificial Intelligence in Materials Science and Material Design,” Advanced Intelligent Systems, vol. 7, no. 8, p. 2400986, 2025, doi: 10.1002/aisy.202400986.
[5]H. Jamialahmadi, G. Khalili-Tanha, E. Nazari, and M. Rezaei-Tavirani, “Artificial intelligence and bioinformatics: a journey from traditional techniques to smart approaches,” Gastroenterol Hepatol Bed Bench, vol. 17, no. 3, pp. 241–252, 2024, doi: 10.22037/ghfbb.v17i3.2977.DOI
[6]R. Pugliese, S. Badini, E. Frontoni, and S. Regondi, “Generative Artificial Intelligence for Advancing Discovery and Design in Biomateriomics,” Intelligent Computing, vol. 4, p. 0117, May 2025, doi: 10.34133/icomputing.0117.DOI
[7]A. Al-Janabi, “Has DeepMind’s AlphaFold Solved the Protein Folding Problem?,” Bio Techniques, vol. 72, no. 3, pp. 73–76, Mar. 2022, doi: 10.2144/btn-2022-0007.DOI
[8]P. Peykani, F. Ramezanlou, C. Tanasescu, and S. Ghanidel, “Large Language Models: A Structured Taxonomy and Review of Challenges, Limitations, Solutions, and Future Directions,” Applied Sciences, vol. 15, no. 14, p. 8103, Jan. 2025, doi: 10.3390/app15148103.DOI
[9]S. Ahn, “The transformative impact of large language models on medical writing and publishing: current applications, challenges and future directions,” Korean J Physiol Pharmacol, vol. 28, no. 5, pp. 393–401, Sep. 2024, doi: 10.4196/kjpp.2024.28.5.393.DOI
[10]“Assessing the potential of LLMs as crowdworkers for contextual information generation,” Information Processing & Management, vol. 63, no. 3, p. 104486, Apr. 2026, doi: 10.1016/j.ipm.2025.104486.
[11]Y. Zhang et al., “Exploring the role of large language models in the scientific method: from hypothesis to discovery,” npj Artif. Intell., vol. 1, no. 1, p. 14, Aug. 2025, doi: 10.1038/s44387-025-00019-5.DOI
[12]“Toward large reasoning models: A survey of reinforced reasoning with large language models,” Patterns, vol. 6, no. 10, p. 101370, Oct. 2025, doi: 10.1016/j.patter.2025.101370.
[13]M. Giannakos et al., “The promise and challenges of generative AI in education,” Behaviour & Information Technology, vol. 44, no. 11, pp. 2518–2544, Jul. 2025, doi: 10.1080/0144929X.2024.2394886.DOI
[14]S. S. Chowa et al., “From language to action: a review of large language models as autonomous agents and tool users,” Artif Intell Rev, vol. 59, no. 2, p. 71, Jan. 2026, doi: 10.1007/s10462-025-11471-9.DOI
[15]S. Lu, I. Bigoulaeva, R. Sachdeva, H. T. Madabushi, and I. Gurevych, “Are Emergent Abilities in Large Language Models just In-Context Learning?,” Jul. 15, 2024, arXiv: arXiv:2309.01809. doi: 10.48550/arXiv.2309.01809.DOI
[16]X. Liu, Z. Wu, X. Wu, P. Lu, K.-W. Chang, and Y. Feng, “Are LLMs Capable of Data-based Statistical and Causal Reasoning? Benchmarking Advanced Quantitative Reasoning with Data,” Jun. 09, 2024, arXiv: arXiv:2402.17644. doi: 10.48550/arXiv.2402.17644.arxiv.org/abs/2402.17644
[17]V. Abishe thvarman, F. Sabrina, and P. Kwan, “Knowledge Integrity in Large Language Models: A State-of-The-Art Review,” Information, vol. 16, no. 12, p. 1076, Dec. 2025, doi: 10.3390/info16121076.DOI
[18]J. C. F. de Winter, D. Dodou, and Y. B. Eisma, “System 2 Thinking in OpenAI’s o1-Preview Model: Near-Perfect Performance on a Mathematics Exam,” Computers, vol. 13, no. 11, p. 278, Nov. 2024, doi: 10.3390/computers13110278.DOI
[19]Y. Ye et al., “LLMs4All: A Review of Large Language Models Across Academic Disciplines,” Nov. 23, 2025, arXiv: arXiv:2509.19580. doi: 10.48550/arXiv.2509.19580.arxiv.org/abs/2509.19580
[20]“A scoping review of ChatGPT research in accounting and finance,” International Journal of Accounting Information Systems, vol. 55, p. 100715, Dec. 2024, doi: 10.1016/j.accinf.2024.100715.
[21]R. Azoulay, T. Hirst, and S. Reches, “Large Language Models in Computer Science Classrooms: Ethical Challenges and Strategic Solutions,” Applied Sciences, vol. 15, no. 4, p. 1793, Jan. 2025, doi: 10.3390/app15041793.DOI
[22]ChangYupeng et al., “A Survey on Evaluation of Large Language Models,” ACM Transactions on Intelligent Systems and Technology, Mar. 2024, doi: 10.1145/3641289.
[23]“Understanding LLMs: A comprehensive overview from training to inference,” Neurocomputing, vol. 620, p. 129190, Mar. 2025, doi: 10.1016/j.neucom.2024.129190.
[24]Z. Wu, X. V. Yu, D. Yogatama, J. Lu, and Y. Kim, “The Semantic Hub Hypothesis: Language Models Share Semantic Representations Across Languages and Modalities,” Mar. 11, 2025, arXiv: arXiv:2411.04986. doi: 10.48550/arXiv.2411.04986.DOI
[25]Á. Cadena-Bautista, F. F. López-Ponce, S. L. Ojeda-Trueba, G. Sierra, and G. Bel-Enguix, “Exploring the Behavior and Performance of Large Language Models: Can LLMs Infer Answers to Questions Involving Restricted Information?,” Information, vol. 16, no. 2, p. 77, Feb. 2025, doi: 10.3390/info16020077.
[26]A. Nguyen, Y. Hong, B. Dang, and X. Huang, “Human-AI collaboration patterns in AI-assisted academic writing,” Studies in Higher Education, vol. 49, no. 5, pp. 847–864, May 2024, doi: 10.1080/03075079.2024.2323593.DOI
[27]T. S. Behrend and R. N. Landers, “Participant Interactions with Artificial Intelligence: Using Large Language Models to Generate Research Materials for Surveys and Experiments,” J Bus Psychol, vol. 40, no. 6, pp. 1275–1297, Dec. 2025, doi: 10.1007/s10869-025-10035-6.DOI
[28]A. A. Barr, J. Quan, E. Guo, and E. Sezgin, “Large language models generating synthetic clinical datasets: a feasibility and comparative analysis with real-world perioperative data,” Front Artif Intell, vol. 8, p. 1533508, Feb. 2025, doi: 10.3389/frai.2025.1533508.DOI
[29]B. Almarie, P. E. P. Teixeira, K. Pacheco-Barrios, C. A. Rossetti, and F. Fregni, “Editorial – The Use of Large Language Models in Science: Opportunities and Challenges,” Princ Pract Clin Res, vol. 9, no. 1, pp. 1–4, Jul. 2023, doi: 10.21801/ppcrj.2023.9.1.1.DOI
[30]A. M. Kimuya and D. M. Kinyua, “A Bidirectional Diode-Clamp Circuit Paradigm for Time-Resolved Measurement of Electrical Short-Circuits,” Measurement: Energy, vol. 11, p. 100114, Sep. 2026, doi: 10.1016/j.meaene.2026.100114.DOI
[31]C. Hongler and A. Emil, “Cross-Entropy Games for Language Models: From Implicit Knowledge to General Capability Measures,” Jun. 22, 2025, arXiv: arXiv:2506.06832. doi: 10.48550/arXiv.2506.06832.DOI
[32]C. Xu, Z. Zhu, J. Wang, J. Wang, and W. Zhang, “Understanding the Role of Cross-Entropy Loss in Fairly Evaluating Large Language Model-based Recommendation,” Feb. 22, 2024, arXiv: arXiv:2402.06216. doi: 10.48550/arXiv.2402.06216.DOI
[33]E. Y. Zhang, A. D. Cheok, Z. Pan, J. Cai, and Y. Yan, “From Turing to Transformers: A Comprehensive Review and Tutorial on the Evolution and Applications of Generative Transformer Models,” Sci, vol. 5, no. 4, p. 46, Dec. 2023, doi: 10.3390/sci5040046.DOI
[34]A. Vaswani et al., “Attention Is All You Need,” Aug. 02, 2023, arXiv: arXiv:1706.03762. doi: 10.48550/arXiv.1706.03762.
[35]M. Qiao, “Understanding Transformers through the Lens of Pavlovian Conditioning,” Aug. 05, 2025, arXiv: arXiv:2508.08289. doi: 10.48550/arXiv.2508.08289.
[36]V. B. Parthasarathy, A. Zafar, A. Khan, and A. Shahid, “The Ultimate Guide to Fine-Tuning LLMs from Basics to Breakthroughs: An Exhaustive Review of Technologies, Research, Best Practices, Applied Research Challenges and Opportunities,” 2024, arXiv. doi: 10.48550/ARXIV.2408.13296.DOI
[37]I. D. Mienye, N. Jere, G. Obaido, O. O. Ogunruku, E. Esenogho, and C. Modisane, “Large language models: an overview of foundational architectures, recent trends, and a new taxonomy,” Discov Appl Sci, vol. 7, no. 9, p. 1027, Sep. 2025, doi: 10.1007/s42452-025-07668-w.DOI
[38]X. Chen, X. Wang, A. Colacelli, M. Lee, and L. Xie, “Electricity Demand and Grid Impacts of AI Data Centers: Challenges and Prospects,” Nov. 26, 2025, arXiv: arXiv:2509.07218. doi: 10.48550/arXiv.2509.07218.arxiv.org/abs/2509.07218
[39]S. M. H. Amiri, P. Goswami, Md. Mainul Islam, M. Shakhawat Hossen, S. M. H. Amiri, and N. Akter, “The Carbon Cost of Conversation, Sustainability in the Age of Language Models,” Jul. 01, 2025, arXiv. doi: 10.48550/arXiv.2507.20018.
[40]G. Wang et al., “Hierarchical Reasoning Model,” Aug. 04, 2025, arXiv: arXiv:2506.21734. doi: 10.48550/arXiv.2506.21734.
[41]B. Liang, Y. Wang, and C. Tong, “AI Reasoning in Deep Learning Era: From Symbolic AI to Neural–Symbolic AI,” Mathematics, vol. 13, no. 11, p. 1707, Jan. 2025, doi: 10.3390/math13111707.DOI
[42]N. Lambert, “Reinforcement Learning from Human Feedback,” Jan. 17, 2026, arXiv: arXiv:2504.12501. doi: 10.48550/arXiv.2504.12501.DOI
[43]X. Dong, M. Teleki, and J. Caverlee, “A Survey on LLM Inference-Time Self-Improvement,” Dec. 18, 2024, arXiv: arXiv:2412.14352. doi: 10.48550/arXiv.2412.14352.DOI
[44]A. Kimuya, “THE MODIFIED OHM’S LAW AND ITS IMPLICATIONS FOR ELECTRICAL CIRCUIT ANALYSIS,” EJSET, vol. 4, no. 2, pp. 59–70, Dec. 2023, doi: 10.55696/e jset.1373552.DOI
[45]A. M. Kimuya, “A novel circuit design for time-dependent short circuit measurement and analysis,” Energy Reports, vol. 13, pp. 1928–1949, Jun. 2025, doi: 10.1016/j.egyr.2025.01.049.DOI
[46]“Large models of what? Mistaking engineering achievements for human linguistic agency,” Language Sciences, vol. 106, p. 101672, Nov. 2024, doi: 10.1016/j.langsci.2024.101672.
[47]E. Bottazzi Grifoni and R. Ferrario, “The bewitching AI: The Illusion of Communication with Large Language Models,” Philos. Technol., vol. 38, no. 2, p. 61, May 2025, doi: 10.1007/s13347-025-00893-6.
[48]S. Wang et al., “Rethinking LLM Evaluation: Can We Evaluate LLMs with 200x Less Data?,” Oct. 12, 2025, arXiv: arXiv:2510.10457. doi: 10.48550/arXiv.2510.10457.
[49]X. Cao and R. Yousefzadeh, “Extrapolation and AI transparency: Why machine learning models should reveal when they make decisions beyond their training,” Big Data & Society, vol. 10, no. 1, p. 20539517231169731, Jan. 2023, doi: 10.1177/20539517231169731.DOI
[50]Umama et al., “LLM-Based Code Generation: A Systematic Literature Review With Technical and Demographic Insights,” IEEE Access, vol. 13, pp. 194915–194939, 2025, doi: 10.1109/ACCESS.2025.3631952.DOI
[51]“Using LLMs to enhance code quality: A systematic literature review,” Information and Software Technology, vol. 190, p. 107960, Feb. 2026, doi: 10.1016/j.infsof.2025.107960.
[52]P. Peykani, F. Ramezanlou, C. Tanasescu, and S. Ghanidel, “Large Language Models: A Structured Taxonomy and Review of Challenges, Limitations, Solutions, and Future Directions,” Applied Sciences, vol. 15, no. 14, p. 8103, Jan. 2025, doi: 10.3390/app15148103.DOI
[53]“Reasoning beyond limits: Advances and open problems for LLMs,” ICT Express, vol. 11, no. 6, pp. 1054–1096, Dec. 2025, doi: 10.1016/j.icte.2025.09.003.
[54]S. Vatsal and H. Dubey, “A Survey of Prompt Engineering Methods in Large Language Models for Different NLP Tasks,” Jul. 24, 2024, arXiv: arXiv:2407.12994. doi: 10.48550/arXiv.2407.12994.DOI
[55]S. A. Athaluri, S. V. Manthena, V. S. R. K. M. Kesapragada, V. Yarlagadda, T. Dave, and R. T. S. Duddumpudi, “Exploring the Boundaries of Reality: Investigating the Phenomenon of Artificial Intelligence Hallucination in Scientific Writing Through ChatGPT References,” Cureus, Apr. 2023, doi: 10.7759/cureus.37432.DOI
[56]F. Keisha, Z. Wu, Z. Wang, A. Koshiyama, and P. Treleaven, “Knowledge Collapse in LLMs: When Fluency Survives but Facts Fail under Recursive Synthetic Training,” Sep. 05, 2025, arXiv: arXiv:2509.04796. doi: 10.48550/arXiv.2509.04796.DOI
[57]E. Dohmatob, Y. Feng, and J. Kempe, “Model Collapse Demystified: The Case of Regression,” Apr. 30, 2024, arXiv: arXiv:2402.07712. doi: 10.48550/arXiv.2402.07712.arxiv.org/abs/2402.07712
[58]A. Mumuni and F. Mumuni, “Large language models for artificial general intelligence (AGI): A survey of foundational principles and approaches,” Jan. 06, 2025, arXiv: arXiv:2501.03151. doi: 10.48550/arXiv.2501.03151.DOI
[59]J. Kim, A. Podlasek, K. Shidara, F. Liu, A. Alaa, and D. Bernardo, “Limitations of large language models in clinical problem-solving arising from inflexible reasoning,” Sci Rep, vol. 15, no. 1, p. 39426, Nov. 2025, doi: 10.1038/s41598-025-22940-0.DOI
[60]M. A. Mohsin et al., “On the Fundamental Limits of LLMs at Scale,” Jan. 26, 2026, arXiv: arXiv:2511.12869. doi: 10.48550/arXiv.2511.12869.DOI
[61]“ChatGPT for research and publication: Opportunities and challenges,” JALT, vol. 6, no. 1, Apr. 2023, doi: 10.37074/jalt.2023.6.1.20.
[62]J. J. Cañas, “AI and Ethics When Human Beings Collaborate With AI Agents,” Front. Psychol., vol. 13, p. 836650, Mar. 2022, doi: 10.3389/fpsyg.2022.836650.DOI

Abstract

Introduction

Comparative Framework: Standard Versus Modified Ohm’s Law

The Tripartite Framework: Isolating Layers of Generative Failure

Architecture, Optimization, and Inherent Constraints of Large Language Models

Language Modeling as Conditional Probability Estimation

The Transformer and the Self-Attention Mechanism

Training Dynamics

The Sampling Process - Generating Text from the Model

Inherent Theoretical Constraints Deriving from the Architecture

The Bounds (The Past Information Boundary and Novelty Horizon)

The Past Information Boundary

The Novelty Horizon

Application to the Diode-Clamped Short-Circuit Experiment

Predictive Principles and the Necessity of Foundational Modifications to Cross the Horizon

Foundational Framework

The Modified Ohm’s Law for Time-Resolved Short-Circuit Measurement

Physical Model of the Short-Circuit Path

Semiconductor Junction Dynamics in Forward Bias

Derivation of the Time-Domain Modified Ohm's Law

Linearized Form and Dynamic Resistance Analysis

Time-Resolved Power Dissipation Formulation

Parameter Identification and Experimental Validation

Case Studies: Script-Based LLM Generation

Case 1: Unguided Script-Based LLM Generation - Statistical Extrapolation of Basic Circuit Principles

Case 2: Informed Pattern Matching with Experimental Context - Guided Script-Based LLM Generation

A Parallel Framework: Direct-Prompt Generation

Methodology (A Physical Analytical Framework)

Experimental Design and Data Generation Protocols: Script-Based Framework

The Parallel Direct-Prompt Generation Framework

Foundational Analytical Framework

Temporal and Statistical Structure Analysis

Validation of Fundamental Physical Constraints

Nonlinear Dynamic Model Fitting and Residual Analysis

Model Formulation and Parameter Estimation

Analysis of Fitted Parameters

Comprehensive Residual Analysis

Goodness-of-Fit Metrics

Quantitative Metrics for Generative Fidelity Assessment

Component Metrics

Interpretation of GFS Values

Distinguishing Linear from Nonlinear Response Regimes

Regime Segmentation Criteria

Comparative Analysis Across Regimes

Sustenance Efficiency and the Role of Diode Capacitance

Model Reduction and the Primacy of Resistive Behavior

Results

Temporal Signature Analysis and Immediate Generative Artifacts

Temporal Memory and Spectral Fingerprints of Generation

Statistical Distributions and the Signature of Token Probability

Violations of Inviolable Physical Constraints

Failure to Conform to First-Principles Constitutive Laws

Residual Diagnostics Revealing Algorithmic Artifacts

Inability to Capture Regime-Dependent Physics

Quantitative Synthesis of Generative Fidelity

Discussion

Interpreting the Failure of Statistical Correlation

The Past Information Boundary and the Novelty Horizon

Comparison with Prior Studies on AI in Scientific Research

The Regime-Dependent Nature of LLM Competence

The Epistemology of Measurement: An Unsolved Challenge for Generative AI

Implications for Researchers, Educators, and Industries

The Structural Limitation and the Path to Augmented Intelligence

The Formal Bounds as Predictive Tools: Practical Applications of BPB_PBP​ and HNH_NHN​

Conclusion

Prompt for Unguided LLM Generation (Case 1 – Statistical Extrapolation)

Prompt

Commentary. The Linear Correlation Engine and Its Digital Artifacts

Prompt for Procedurally Guided LLM Generation (Case 2 – Informed Pattern Matching)

Prompt

Commentary. Pattern Overload and the Emergence of Algorithmic Pathology

Experimental Framework for Case 3 – Adoption of a Pre-Validated Physical Design

Provenance of the Experimental Configuration and Data

Adoption of Pre-Validated Data Without Recalibration

Role of Case 3 Data in the Present Study

Epistemological Foundations of Physics-Based Authentication - Why Statistical Similarity Cannot Certify Causal Fidelity

The Reference Class Problem in LLM-Generated Scientific Data

The Measurement Process as a Hidden Variable

The Necessity of Forensic Physics-Based Authentication

The Generative Fidelity Score as an Epistemological Instrument

The Formal Bounds as Predictive Tools: Practical Applications of $B_P$ and $H_N$