WorksComputer Science
Tracing Knowledge and Reasoning in LLM
Abstract
Current evaluations of Large Language Models (LLMs) report performance at the dataset or task level, revealing what a model achieves but offering little insight into the internal processing associated with its performance. We propose a task-centric framework for characterising the model-internal processing elicited by a benchmark. It combines attention- and activation-based signals with representation-based measures to characterise benchmarks spanning a continuum from retrieval-dominant to reasoning-dominant and to surface failure modes that task-level scores conceal. We evaluate 16 LLMs across 24 diverse benchmarks and conduct targeted internal-state analyses on representative models and tasks, observing systematic differences in processing between knowledge-centric and reasoning-centric tasks. We shift evaluation from asking what score a model achieves toward characterising the processing patterns associated with that score, potentially supporting more targeted benchmark selection and more informed model choice. We open source our code at https://github.com/extraordinarylab/trace-kr.
Introduction
Large Language Models (LLMs) have demonstrated remarkable capabilities across a diverse spectrum of tasks, achieving state-of-the-art results on numerous benchmarks [63, 19, 35]. This progress has been largely measured by aggregate performance scores on static leaderboards. While invaluable for tracking progress, this evaluation paradigm is reaching its limits [75, 54]. A single accuracy number on a benchmark tells us what a model can do, but reveals little about how it achieves the result or why it fails, because it cannot tell whether a correct answer reflects memorised knowledge or actual reasoning. As models become more powerful, simply measuring their aggregate success yields diminishing insight into their underlying architectural strengths and weaknesses. The field requires a more fine-grained approach to evaluation that can disentangle the fundamental capabilities underpinning performance.
The challenge of disentangling a model’s knowledge from its reasoning is widely acknowledged in the community [20, 68]. Prior work has largely focused on analysing these capabilities in isolation, which has significant limitations. Research into knowledge boundaries (i.e., what an LLM knows versus what it does not) primarily uses techniques like uncertainty estimation, confidence calibration, and internal state probing [29, 69]. Recent work nevertheless shows that reliable boundary estimation remains difficult: uncertainty can be imperfectly calibrated, while poor awareness of parametric knowledge boundaries can lead to inappropriate reliance on internal or retrieved information [45, 30, 11]. Studies of mathematical and compositional reasoning reveal that performance can degrade under seemingly superficial perturbations [42]. More recent evidence further shows that models may possess the factual information needed to solve a task yet still rely on surface heuristics rather than applying that knowledge through principled reasoning [27]. While valuable, these largely isolated approaches leave open how knowledge retrieval and reasoning interact within hybrid tasks. Recent work [11] has begun to study this interaction, showing that failures can arise either from inadequate knowledge integration or from downstream reasoning even when the relevant knowledge is available. A high score on a “reasoning benchmark” does not reveal whether success comes from robust reasoning or from easier retrieval shortcuts. For example, a science benchmark may contain questions that require recalling a specific fact before applying it in a multi-step inference; strong aggregate performance does not show which of these two capabilities the model actually relies on. Recent evidence further suggests that access to relevant knowledge does not guarantee its effective use: models may attend to shortcut-like cues or fall back on parametric memory even when sufficient external knowledge is provided [68, 31].
Recent studies have begun to address this gap by moving beyond passive observation toward active decoupling of these capabilities. For instance, some frameworks propose decomposing the inference process into explicit steps by fine-tuning models with special tokens like <memory> and <reason> [20], or by prompting models to adopt fast (knowledge-based) versus slow (reasoning-based) thinking modes inspired by dual-system theory [70]. Another line of work focuses on evaluating the final reasoning chain, introducing metrics to judge the correctness of knowledge and the logical quality of each step [68]. Although effective in their respective settings, these methods either alter the model’s natural inference process or rely solely on analysing the generated text. Consequently, they do not provide a task-level account of the relative knowledge-retrieval and reasoning demands reflected in the internal processing of an unmodified LLM.
To address this limitation, we draw on prior findings concerning the internal structure of Transformers. Prior work [70, 56, 3] suggests partial functional specialisation across Transformer layers. Lower-to-middle feed-forward layers often support factual associations [12], while upper layers are important for some multi-step and arithmetic computations [57, 17]. These findings do not imply a strict separation between knowledge and reasoning. Instead, they motivate the hypothesis that tasks with different balances of retrieval and intermediate computation may produce different internal patterns. Motivated by this evidence, we test for systematic associations between these patterns and task demands without making causal claims.
In this paper, we distinguish knowledge retrieval from reasoning by the amount of intermediate transformation required during inference. Knowledge retrieval refers to identifying or recalling task-relevant factual or conceptual information, with little or no transformation of that information. Reasoning refers to deriving task-relevant information by composing or transforming available information through intermediate operations such as inference, calculation, comparison, or state tracking. As most tasks require both processes, we therefore use “knowledge-centric” and “reasoning-centric” to describe dominant tendencies along a continuum rather than mutually exclusive task categories. This distinction allows us to examine whether similar task-level performance can arise from different underlying processing demands and failure modes. Our investigation is structured around two core research questions:
RQ1: Do tasks with different balances of knowledge retrieval and reasoning produce systematically different layer-wise and decoding-time patterns in an unmodified LLM?
RQ2: How do model family, parameter scale, and inference-time prompting affect performance across the knowledge-reasoning continuum?
Our contribution is not a new activation or probing metric in isolation. Instead, we introduce a task-level diagnostic framework that combines existing internal signals within a unified knowledge-reasoning analysis. Specifically, we: (i) introduce a task-centric framework, which combines static layer-wise activation statistics, decoding-time trajectories, and representation-based measures; (ii) use these signals to characterise benchmark demands and construct a knowledge-reasoning capability landscape across 16 models and 24 benchmarks; and (iii) connect the capability landscape to behavioural stress tests that reveal how prompting affects performance and robustness to shortcut-inducing perturbations.
Related Work
Our research is situated at the intersection of two critical areas: the effort to disentangle core abilities like knowledge and reasoning, and the field of mechanistic interpretability, which seeks to explain model behaviour through its internal structure.
Behavioural Approaches
A significant body of research has sought to separate knowledge and reasoning by analysing a model’s inputs and outputs. The most prominent example is Chain-of-Thought (CoT) prompting [65], which encourages models to externalise their reasoning steps. Subsequent frameworks have built on this by evaluating the logical quality and factual correctness of the generated reasoning chain [68], often revealing that models can produce specious or unfaithful explanations [48, 59, 26, 1]. While these methods provide valuable insight into a model’s problem-solving process, they are fundamentally analyses of the final text output, not the internal computations that produced it. Another line of work actively alters the model’s inference process to force a separation. Some approaches fine-tune models with special tokens (e.g., <memory> and <reason>) to explicitly direct the model’s processing path [20]. Others [73, 70, 18] draw inspiration from dual-process theory [14], prompting models to adopt distinct fast (System 1) and slow (System 2) thinking modes [22] to disentangle intuitive knowledge retrieval from deliberate reasoning. These are powerful interventionist techniques, but they do not characterise the intrinsic behaviour of an unmodified LLM. Our work differs by aiming to diagnose the model’s unaltered inference process.
Mechanistic Foundations
A distinct and complementary field, mechanistic interpretability, suggests that the conceptual separation between knowledge and reasoning is mirrored by a structural separation within the Transformer architecture itself [74, 53, 71, 13]. This provides the theoretical foundation for our empirical approach. Geva et al. [12] demonstrated that Feed-Forward Networks (FFNs) function as key-value memories, storing factual associations learned during training, particularly in the lower-to-mid layers of the model. This finding has been extended by research that successfully edits specific factual knowledge within the FFN weights of mid-layer transformers [40, 41], providing a mechanistic basis for where parametric knowledge resides. In contrast, causal analysis and probing studies have identified the upper layers of the network as being critical for multi-step, compositional, and arithmetic reasoning [57, 17]. For instance, some studies interpret LLMs by focusing on how the information is processed between layers [46, 44]. Related work further suggests that information encoded in internal representations may not always be reflected in observable model behaviour [34]. These higher-level processes are understood to integrate and manipulate information that has been retrieved or processed by the lower layers. These findings motivate a testable hypothesis rather than a strict localisation claim: tasks that rely more on direct retrieval and tasks that require more intermediate computation may produce different patterns across layers. Because representations can become less distinguishable in deeper causal Transformers [17], final-layer states may obscure differences that appear earlier. We therefore analyse the full layer-wise evolution of internal states to identify where task-related differences emerge and whether they persist.
Towards Holistic Evaluation
Behavioural methods analyse the model from the outside, while mechanistic work “reverse engineer” the function of internal components in isolation [17, 76, 78, 62, 16] (Zhao et al. 2025). Our work bridges this divide by operationalising the findings from mechanistic interpretability. Instead of focusing on model-centric properties, such as a model’s self-awareness of its knowledge gaps [72, 38] or the procedural limits of its CoT capabilities [10, 6], our approach is task-centric. We use internal activation patterns to characterise the relative retrieval and reasoning demands that a task elicits in an LLM. We then compare models according to how their internal states and behavioural performance respond to these demands. This provides a unified view of model performance across the knowledge-reasoning continuum. Unlike prior studies that use individual interpretability techniques to localise specific behaviours, we combine complementary internal signals to compare task demands across benchmarks and model families. Rather than introducing a new metric, its novelty lies in using these signals together to capture different aspects of task-level processing.
Method and Experimental Setup
Benchmarks and Taxonomy
We evaluate 24 benchmarks covering factual and biomedical QA, scientific knowledge, commonsense and logical reasoning, reading comprehension, symbolic reasoning, and mathematical problem solving. Appendix § 8.§ 1 reports the complete benchmark list, evaluation formats, numbers of examples, and citations. We classify each benchmark by its dominant task demand. A benchmark is knowledge-centric when most items primarily require factual retrieval or verification with limited intermediate transformation. A benchmark is reasoning-centric when most items require composition, inference, calculation, comparison, or state tracking. This assignment is based on the benchmark design and required operations and is made independently of the analyses conducted in this paper. The labels describe dominant tendencies rather than disjoint categories, as many benchmarks require both knowledge and reasoning. Appendix § 8.1.1 provides the complete assignment and discusses mixed cases.
Scope. The capability landscape includes both multiple-choice and free-form benchmarks, but probing, representation similarity, and NOTA (none of the above) analyses use multiple-choice tasks because fixed candidate sets enable controlled representation extraction and answer-option perturbation. We restrict the study to English to avoid confounding task demands with differences in tokenisation, language coverage, and translation. These choices provide a controlled diagnostic setting.
Profiling Representations and Internal States of Models
We propose a task-centric framework for characterising the model-internal processing elicited by a task in an unmodified LLM. It does not directly measure knowledge or reasoning capability; rather, it compares internal patterns across tasks with different balances of retrieval and intermediate transformation. We combine layer-wise attention entropy and feed-forward activation sparsity, their trajectories during decoding, and representation-based evidence from probing and correct–incorrect answer comparisons.
Static Profile. We define a static profile as the benchmark-level pattern of layer-wise attention entropy and activation sparsity measured before answer generation. For each benchmark, we process all available questions with Qwen2.5-7B-Instruct without generating an answer. We select Qwen2.5-7B-Instruct as the reference model because it provides a tractable mid-sized, open-weight setting for extracting layer-wise internal states across all 24 benchmarks. To reduce the risk that the observed representational patterns are specific to this model family, the subsequent probing and representation analyses additionally examine Llama3.1-8B-Instruct and OLMo-3-7B-Instruct. We extract the layer-output hidden state and attention distribution at the final input-token position. No prompt template or instruction is added in this analysis, so all 24 benchmarks share the same input format. We use attention entropy to characterise how concentrated or diffuse the model’s attention is across the input at each layer. The layer-wise entropy profile therefore serves as a descriptive measure of how selectively the model allocates attention when processing tasks with different retrieval and reasoning demands.
Let denote the attention weight assigned to input position by head in layer , when the query is the final input token of example . We first average the attention distribution across the heads:
We normalise this distribution as
where . The layer-wise attention entropy is then
where is the number of examples.
We use activation sparsity to characterise how selectively the model’s hidden representation is activated at each layer. Higher sparsity indicates that a larger proportion of hidden dimensions remain near zero, providing a descriptive measure of how concentrated or distributed the layer-wise representation is across tasks.
Let denote dimension of the layer-output hidden state at the final input-token position, and let denote the model hidden size. We compute hidden state sparsity as
The dataset-level static feature vector is
where is the number of Transformer layers. These features summarise the final-token attention distribution and layer-output hidden state. They are descriptive and are not interpreted as causal measures of knowledge retrieval or reasoning.
Decoding-Time Trajectories. This analysis is intended to characterise how internal processing evolves during answer generation, rather than only how it differs across layers at input time. We trace the same metrics over generated tokens for all available SciQ examples using Qwen2.5-7B-Instruct, and repeat the analysis on two reasoning-centric benchmarks, MathQA and LogiQA. We use SciQ as the primary controlled diagnostic dataset because its questions are short and relatively homogeneous, reducing confounds from variable generation lengths and heterogeneous reasoning strategies. The two reasoning-centric benchmarks test whether the resulting signal extends beyond knowledge retrieval. All three benchmarks use the retrieve-then-answer prompt given in Figure 12. We compare the token-wise mean trajectories and within-trajectory variability of attention entropy and activation sparsity between correct and incorrect generations. We report layers 3, 14, and 22 as representative lower, middle, and upper layers. Generation uses temperature 0.7, top- 0.8, and a maximum of 1,024 new tokens; generations that are truncated or contain no parsable answer are excluded. Signals are read at the positions of the generated tokens: after sampling, the prompt and sampled tokens are passed through the model once, which, under causal masking, reproduces the internal state at each decoding step. To compare responses of different lengths, we sample uniformly spaced positions from each generation of length . For , the selected positions are

Prompt Template (Retrieve → Reason).
Identify key facts and technical principles related to the following question. Based only on the knowledge retrieved above, analyze the following question. The last line of your response should be of the following format: ’ANSWER: [LETTER]’ (without quotes) where [LETTER] is one of letters.
Question: {question}
Choices: {choices}
Figure 12. Prompt used for decoding-time trajectory analysis.
where .
Probing and Representation Analysis
To complement the static profile and decoding-time trajectory analyses, we use linear probing and representation similarity analysis to examine two questions: (1) at which layers is the correct answer option linearly recoverable from pre-generation hidden states, and (2) how distinctly does the model represent a question paired with its correct answer versus the same question paired with an incorrect distractor?
We conduct these probing and representation analyses on a balanced diagnostic subset of four multiple-choice benchmarks: two knowledge-centric tasks (BioMixQA and SciQ) and two reasoning-centric tasks (LogiQA and MathQA). We use multiple-choice benchmarks because their fixed candidate sets enable controlled extraction and comparison of correct and incorrect answer representations. The analyses are conducted using Llama3.1-8B-Instruct, Qwen2.5-7B-Instruct, and OLMo-3-7B-Instruct.
Layer-Wise Probing. For each question, we perform one forward pass using the templates in Figure 13 and extract the hidden state at the final input position before answer generation. At each layer, we train a logistic regression classifier to predict the correct answer option using 5-fold cross-validation and report average held-out accuracy. We compare this accuracy with the majority-class baseline, obtained by always predicting the most frequent correct answer label in each benchmark. We track accuracy across layers to examine when the correct answer becomes linearly recoverable in each task group. Accuracy close to this baseline does not show that reasoning information is absent. It indicates only that the selected linear probe does not reliably recover the answer under the given prompt and token position. The task groups use different templates, so task type and template choice are confounded. We therefore run every benchmark under both templates to compare the task groups under the same prompting conditions. The model, questions, probing procedure, and token position are held fixed. From these forward passes, we record the head-averaged attention of the final input token (Eq. (1)). We report the share assigned to the question and answer options, whose text is identical in both templates.

(a) Prompt for knowledge-centric datasets
Question: {question}
Choices: {choices}
Respond with one letter. The answer is _
(b) Prompt for reasoning-centric datasets
Please enclose your thinking process in <think></think> and the final answer in <final_answer></final_answer>.
Question: {question}
Choices: {choices}
Respond with one letter. The answer is _
<think></think><final_answer>
Figure 13. Prompt templates for extracting representations under the multiple-choice setting. We query the LLM with the appropriate template and extract the last-token representation from each layer as input to the linear probing classifiers.
Representation Similarity. For each question , we append either the correct answer or an incorrect distractor :
We extract their final-token hidden states at each layer and compute an adjusted cosine similarity:
where the expectation is computed over randomly paired completions from the same benchmark. We use this measure to characterise how distinctly the model represents the correct answer relative to a plausible incorrect alternative. Comparing this separation across task types allows us to test whether knowledge-centric and reasoning-centric benchmarks differ systematically in the geometry of their answer representations. This measure captures relative representational similarity and does not establish causal separation between correct and incorrect answers.
Capability Scores and Evaluation
Let denote the performance of model , under prompting condition , on benchmark . We compute the knowledge and reasoning scores as unweighted averages over these reported evaluation settings:
where and . Each reported setting therefore contributes equally to the corresponding capability score, irrespective of the number of examples it contains.
For either dimension , let
and apply min-max normalisation:
where min/max are taken jointly over model prompting condition (Standard and CoT), separately for each dimension. Evaluation scores are expressed on a common scale. We use accuracy for classification and multiple-choice settings and exact match for the free-form mathematical benchmarks GSM8K, AIME, MATH500, and AMC.
We evaluate open-weight, instruction-tuned models from the Qwen, Llama, Gemma, OLMo, GLM, and GPT-OSS families. The models range from 2B to more than 30B parameters and include both dense and Mixture-of-Experts architectures. Full model details are provided in Appendix § 8.§ 2.
All evaluations use EvalScope ([43]), with SGLang ([77]) as the inference backend. We follow the benchmark-specific zero-shot and few-shot settings provided by the evaluation framework. We use accuracy for classification and multiple-choice tasks and exact match for free-form mathematical tasks. Standard and CoT conditions use the same evaluation configuration except for the prompting instruction.
Knowledge-Reasoning Patterns
RQ1: Do tasks with different balances of knowledge retrieval and reasoning produce systematically different layer-wise and decoding-time patterns in an unmodified LLM?
Internal Mechanics. We first examine whether an LLM exhibits a consistent, data-driven internal pattern that differentiates tasks. To establish a baseline internal profile, we perform a question-only analysis using Qwen2.5-7B-Instruct. For each dataset, we feed the model only the input question (without generating an answer) and summarise its layer-wise internal activity into a single static profile. This design isolates task comprehension and retrieval demands from decoding-dependent effects such as variable generation length, sampling stochasticity, or branching during answer construction. As a result, differences in the profile reflect how the model processes a fixed input, rather than decoding-dependent variability.
We compute attention entropy and activation sparsity at each layer. Attention entropy measures how concentrated the attention distribution is. Higher values indicate more diffuse attention. Activation sparsity is the proportion of feed-forward activations that are close to zero. We concatenate these values across layers to form a feature vector for each dataset. Figure 1 shows these profiles for all 24 datasets. We order the datasets by the first principal component of the activation sparsity matrix. This produces a smooth progression in sparsity across datasets and layers. Under this ordering, activation sparsity shows a clearer pattern in the lower layers than attention entropy. LogiQA, TruthfulQA, and PIQA have the highest sparsity in the lower layers. The mathematical benchmarks AIME, GSM8K, MATH500, AMC, and MathQA fall in the middle of the range. The lowest values occur in both task groups. These include knowledge-centric benchmarks such as SciQ and MedMCQA and reasoning-centric benchmarks such as CommonsenseQA and SIQA. Figure 1 describes how the model processes the input of each benchmark. The controls below show that adding the same suffix to every input reduces the differences in lower-layer sparsity across benchmarks. This indicates that these measurements are sensitive to how the input ends. We therefore use probing, decoding trajectories, representation geometry, and behavioural analyses to compare knowledge and reasoning.
Figure 1. Layer-wise static internal profiles of Qwen2.5-7B-Instruct across 24 benchmarks. The heatmaps show attention entropy (left), where higher values indicate more diffuse attention, and activation sparsity (right), defined as the proportion of near-zero feed-forward activations. Benchmarks are ordered by the first principal component of the activation sparsity matrix to expose cross-benchmark variation in their layer-wise profiles.
Prompt Length and Token-Composition Controls. Because benchmarks differ in input length, vocabulary, and formatting, we test whether these input properties explain the differences in lower-layer sparsity in Figure 1. We run the model again on the same questions from each benchmark, one question at a time. We record the prompt length and token composition for each question (Figure 2). Across the 24 benchmarks, lower-layer sparsity (layers 0 to 3) has a moderate correlation with mean prompt length (Spearman ). It shows little correlation with the proportion of tokens containing digits or mathematical symbols (). We fit a regression of sparsity on log prompt length within each benchmark. We then estimate sparsity at a common prompt length, which is the median across all questions. This adjustment leaves the benchmark ordering nearly unchanged ( with the original ordering). We measure sparsity at the final input token. To test whether this token affects the measurements, we append the same suffix (“\nAnswer:”) to every prompt. This makes all prompts end with the same token. The standard deviation of lower-layer sparsity across benchmarks decreases. The rank correlation with the original benchmark ordering falls to . We find no statistically significant difference in lower-layer sparsity between knowledge-centric and reasoning-centric benchmarks. This holds both before and after the suffix is added (Mann–Whitney and
Figure 2. Controls for the lower-layer activation sparsity in Figure 1 (Qwen2.5-7B-Instruct; mean over layers 0–3 at the final input token). Left and middle: benchmark means against mean prompt length and against the share of prompt tokens that contain a digit or mathematical symbol. Right: benchmark means under two controls against the original values. Length-adjusted values are predicted at a fixed length from a within-benchmark regression on log length; same-final-token values append “\nAnswer:” to every prompt. The dashed line marks equality.
, respectively). These results indicate that the lower-layer differences in Figure 1 depend on how each benchmark’s input ends. We therefore do not draw conclusions about knowledge and reasoning from lower-layer sparsity alone.
Decoding-time Trajectories. Using the controlled SciQ setup described in § 3.2, we examine how attention entropy and activation sparsity evolve during generation for correct and incorrect predictions. SciQ is additionally evaluated with a structured retrieve-then-answer prompt (Figure 12), providing a consistent temporal scaffold for interpreting these trajectories.
Figure 4 visualises the evolution of internal signals during generation on SciQ. Here, the mean trajectory represents the token-wise average of the metric across examples after length normalisation.
Figure 4. Decoding-time trajectory analysis on SciQ using Qwen2.5-7B-Instruct. Rows show attention entropy (top) and activation sparsity (bottom). Columns separate correct (left) and incorrect (right) predictions. Curves show token-wise mean trajectories at representative lower, middle, and upper layers (layers 3, 14, and 22). Shaded regions show standard deviation across examples within each correctness group. Generated sequences are length-normalised to 101 aligned token positions by uniform subsampling. Incorrect generations are associated with greater within-trajectory variability in attention entropy, particularly at the lower and upper layers. The one-sided tests do not detect greater within-trajectory variability in activation sparsity for incorrect generations (Table 3).
A key observation is that mean levels alone often fail to distinguish correct from incorrect outputs. The diagnostic signal lies primarily in differences in within-trajectory variability. Correct predictions display a clearer late-stage drop in attention entropy (reflecting answer consolidation). However, the separation in mean levels is modest. Incorrect generations show greater within-trajectory variability in attention entropy. On SciQ, incorrect generations are associated with greater within-trajectory variability in attention entropy, particularly at the lower and upper layers (Table 3). The one-sided tests do not detect greater within-trajectory variability in activation sparsity for incorrect generations than for correct ones.
| Metric | Lower | Lower | Middle | Middle | Upper | Upper |
| Attention Entropy | <.001 | .42 | .055 | .14 | <.001 | .45 |
| Activation Sparsity | .999 | −.27 | .999 | −.07 | .205 | .13 |
Table 3. Correct-incorrect differences in within-trajectory variability across representative lower, middle, and upper layers. denotes the Benjamini-Hochberg adjusted -value [4], with correction across the three layer-wise comparisons within each metric. denotes Cliff’s delta [8]; positive values indicate greater within-trajectory variability for incorrect than correct generations. Bold values indicate .
Figure 3 shows that this pattern is not specific to SciQ. As on SciQ, incorrect generations on MathQA and LogiQA show a larger within-trajectory standard deviation of attention entropy than correct ones. Under the one-sided Mann-Whitney test used in Table 3, the difference is significant at the lower and upper layers on all three benchmarks (). The difference is also significant at the middle layer on MathQA and LogiQA. Activation sparsity provides no consistent signal across benchmarks. Its within-trajectory variability differs between correct and incorrect generations at some layers on MathQA and LogiQA. On SciQ, the one-sided tests do not detect greater sparsity variability in incorrect generations.
Figure 3. Within-trajectory variability of attention entropy for Qwen2.5-7B-Instruct on SciQ, MathQA, and LogiQA. For each generation, the standard deviation of attention entropy is computed over the 101 aligned generated-token positions at the lower, middle, and upper depths (layers 3, 14, and 22).
Probing and Geometric Analysis. The trajectory analysis suggests that correct and incorrect generations differ more in within-trajectory variability than in mean signal levels, particularly for attention entropy. We next examine how reliably a linear probe can recover the correct answer option from a static hidden state before generation in a multiple-choice setting.
Figure 5 reveals a clear contrast in pregeneration answer recoverability across the two task groups. On the knowledge-centric benchmarks BioMixQA and SciQ, probing accuracy rises consistently through the mid-to-late layers across the three model families, indicating that the correct option becomes progressively more linearly accessible before answer generation. In contrast, probing accuracy on LogiQA and MathQA stays close to the majority-class baseline for Llama3.1-8B-Instruct and OLMo-3-7B-Instruct (peak 0.29-0.34 against baselines of 0.24 and 0.28). Qwen2.5-7B-Instruct is the exception: its accuracy rises above chance from around layer 20, peaking at 0.46 on LogiQA and 0.35 on MathQA, although its gains over the baseline (0.18 and 0.11) remain smaller than on BioMixQA and SciQ (0.22 and 0.69).
Figure 5. Layer-wise linear probing accuracy under the multiple-choice setting across three model families. Knowledge-centric datasets show increasing probing accuracy in the mid-to-late layers, indicating that the correct option becomes more linearly recoverable. Prompts are shown in Figure 13.
Under the probing protocol defined in § 3.3, these results indicate that answer-discriminative information is more linearly recoverable from pregeneration hidden states for the two knowledge-centric tasks than for the two reasoning-centric tasks. This should not, however, be interpreted as evidence that reasoning information is absent from the latter representations; rather, the correct answer is not reliably recoverable from them using the selected prompt, final-token representation, and linear probe.
Effect of Prompt Structure. Figure 6 shows that using different templates for the two task groups does not fully explain the probing contrast in Figure 5. Averaged over Llama3.1-8B-Instruct, Qwen2.5-7B-Instruct, and OLMo-3-7B-Instruct, BioMixQA and SciQ retain higher peak probing accuracy than LogiQA and MathQA under both templates. The reasoning-centric template attracts more attention to its instructions and response cues. Attention to the question and answer options remains nearly unchanged. The extra instructions therefore change how the model allocates attention without closing the gap in answer recoverability. This supports the knowledge-reasoning comparison for these benchmarks under the two tested templates.
Figure 6. Template-swap control for the probing analysis. Each benchmark is run under both the knowledge-centric and the reasoning-centric template of Figure 13. Bars show the mean over Llama3.1-8B-Instruct, Qwen2.5-7B-Instruct, and OLMo-3-7B-Instruct; markers show individual models. Left: peak layer-wise probing accuracy; dashed lines mark the majority-class baseline. Right: share of the final token’s attention, averaged over heads and layers, on the question and answer options.
Distinctness of Representations
While linear probing reveals whether correctness is separable from hidden states, it does not quantify how far apart correct and incorrect completions lie in representation space. We therefore compare the adjusted correct–incorrect representation similarity defined in §3.3 across the four diagnostic benchmarks. In Figure 7, BioMixQA and SciQ show lower adjusted cosine similarity, whereas LogiQA and MathQA show higher similarity. This pattern is consistent with greater semantic separation between factual answer options and greater lexical or structural overlap between reasoning options.
Figure 7. Representation geometry of correct and distractor answers. Curves show the adjusted cosine similarity between hidden states for prompts appended with the ground-truth answer versus an incorrect distractor; shaded regions indicate variability across examples. Lower values indicate greater representational separation. Knowledge-centric benchmarks (BioMixQA and SciQ) generally show lower similarity than reasoning-centric benchmarks (LogiQA and MathQA) across the three models.
It may reflect both task demands and distractor construction. The result therefore shows an association between benchmark type and representation geometry, not a causal or structural separation between truth and falsehood. One possible explanation is that factual distractors often differ from correct answers in entity or semantic content, whereas reasoning distractors are frequently near-miss solutions with similar wording and structure. The observed geometry may therefore reflect both task demands and answer-option construction. It should not be interpreted as evidence that truth and falsehood are causally or structurally separated in the early layers.
The analyses in this section provide complementary forms of evidence. The static analysis compares all benchmarks at input time, the trajectory analysis examines correct and incorrect generations on SciQ, MathQA, and LogiQA, and the probing and geometric analyses examine four multiple-choice benchmarks. Their agreement supports an association between benchmark demands and internal processing patterns. However, prompt format, answer semantics, distractor construction, and dataset composition may also contribute to the observed differences. Because we do not perform activation patching, layer ablation, or causal tracing, these results identify diagnostic associations rather than causal mechanisms.
Knowledge-Reasoning Landscape
We now turn from internal task characterisation to model-level capability profiling. RQ2 examines how model family, scale, and prompting shift models along the knowledge–reasoning dimensions and whether apparent gains remain robust to answer-option perturbations.
RQ2.1: How do the knowledge and reasoning profiles of LLMs vary with model family and scale?
Model Family. Figure 10 shows the patterns of different model families in the knowledge-reasoning plane: models from the same family tend to occupy similar regions, indicating that successive architectural revisions can influence performance as much as, and sometimes more than, model size. More recent families tend to achieve higher scores on both axes, placing them nearer the upper-right region of the plane. In particular, Qwen3 variants are among the strongest overall performers, while Gemma3 variants consistently outperform earlier Gemma versions along both knowledge and reasoning axes. Within the Gemma family, newer models exhibit a near-linear progression in the knowledge-reasoning plane.
Figure 10. For each model and prompting condition, raw performance is first averaged over the corresponding knowledge or reasoning evaluation settings (Eqs. (9)–(10)). Each resulting dimension is then min-max normalised jointly across all model and prompting-condition pairs (Eq. (13)). Hollow (◦) and solid (•) markers denote Standard and CoT prompting, respectively, and arrows indicate the change induced by CoT.
Scaling. Within a given family, increasing model size improves performance, but the gains are uneven across the two axes. Larger models move rightward more consistently, indicating that knowledge scores tend to increase more reliably with scale. Improvements in reasoning are less monotonic and often exhibit smaller marginal returns at larger sizes. The Gemma3 series illustrates this pattern: scaling from 3B to
27B yields a substantial gain in knowledge but only a modest improvement in reasoning. At the same time, smaller models (e.g., Gemma3-4B, Qwen2.5-3B) already achieve competitive reasoning scores. Within the evaluated model families, knowledge scores increase more consistently with parameter count than reasoning scores. However, model size co-varies with architecture, training data, instruction tuning, and other design choices. These comparisons therefore do not isolate a causal effect of scale. The result should be interpreted as a descriptive pattern: larger models show more reliable gains on the knowledge axis, whereas reasoning gains vary more across model families.
ing score. The change in knowledge is smaller and less consistent: for many models the arrow is close to vertical (little change in knowledge), while for several models the arrow also moves to the right (a small gain in knowledge alongside improved reasoning). However, CoT is not always beneficial. A clear counterexample is Gemma-7B, which moves down-left under CoT, indicating a drop in both reasoning and knowledge. CoT increases reasoning scores in 14 of the 16 checkpoints and knowledge scores in 9. Paired model-level tests detect a significant increase in reasoning scores but do not detect a significant change in knowledge scores (Appendix § 8.§ 4, Table 4). CoT can still reduce performance for individual models, as illustrated by Gemma-7B.
| Capability | p | |||
| Reasoning | +.036 | 14/16 | .008 | +.74 |
| Knowledge | +.001 | 9/16 | .597 | +.16 |
Table 4. Effect of CoT prompting on the knowledge and reasoning dimensions. is the median paired change from Standard to CoT prompting, and reports the number of evaluated configurations with a positive change. is obtained from a two-sided Wilcoxon signed-rank test [67], and denotes the matched-pairs rank-biserial correlation [23].
RQ2.2: What is the impact of CoT prompting on the internal knowledge-reasoning balance, and does it introduce performance trade-offs?
Knowledge-Reasoning Trade-off. Figure 10 compares each model under standard prompting (hollow marker ◦) and CoT prompting (solid marker •), with an arrow (→) showing the change. The main pattern is simple: most arrows point upward, meaning CoT usually increases the reasoning score.
RQ2.3: Does CoT prompting mitigate parametric shortcuts?
Experiment Settings. To disentangle parametric shortcuts (answering by matching memorised option patterns) from dynamic reasoning (deriving an answer independent of the option set), we adopt a “none of the above” (NOTA) perturbation stress test, inspired by Wang et al. [61]. We evaluate two complementary perturbations under zero-shot multiple-choice evaluation: (i) NOTA-as-answer, where we remove the ground-truth option and make NOTA the correct choice; and (ii) NOTA-as-distractor, where we inject NOTA as an additional incorrect option. We compare Standard prompting against CoT prompting across six instruction-tuned LLMs (Llama3.1, Qwen2.5, Olmo3, Gemma3 4B/12B/27B) on MathQA and SciQ. We report accuracy and visualise robustness as the gap between Base (original options) and each perturbed condition (Figure 8 and Figure 9).
Figure 8. Robustness to NOTA perturbations on MathQA under Standard and CoT prompting. Hollow markers (◦) indicate Base accuracy with the original answer options, while solid markers (•) show accuracy after the NOTA-as-answer or NOTA-as-distractor perturbation; connecting lines indicate the resulting performance change. NOTA-as-answer produces substantially larger degradation than NOTA-as-distractor for most models, with the effect varying across prompting conditions.
Figure 9. Robustness to NOTA perturbations on SciQ under Standard and CoT prompting. Hollow markers (◦) indicate Base accuracy with the original answer options, while solid markers (•) show accuracy under the NOTA-as-answer or NOTA-as-distractor condition; connecting lines indicate the performance change. NOTA-as-answer causes a pronounced accuracy drop across models, whereas adding NOTA as a distractor has comparatively little effect.
Robustness to Option Manipulation. On the MathQA dataset, CoT frequently improves robustness to option-set manipulations. For several models (notably Qwen2.5 and OLMo3), CoT both increases Base accuracy and reduces the drop under NOTA-as-answer, consistent with reduced reliance on specific answer options. However, this effect is not universal: Llama3.1 still shows a large gap between Base and NOTA-as-answer even with CoT, indicating that correctness continues to depend on the presence of the original correct option. This residual gap is consistent with partially option-dependent CoT, in which plausible rationales are produced while some reliance on option matching remains. SciQ exhibits a more consistent pattern across models. Adding NOTA as a distractor has only a small effect on accuracy under both Standard and CoT, suggesting that merely inserting an additional wrong option does not substantially disrupt model behaviour. In contrast, NOTA-as-answer causes a pronounced drop across models, directly revealing reliance on the correct option’s presence in the candidate set. Paired item-level tests confirm this asymmetry across model, dataset, and prompting conditions (Appendix § 8.§ 4; Table 5). CoT can raise Base accuracy and sometimes narrows this gap, but the remaining degradation indicates that option dependence is reduced rather than eliminated. Overall, CoT mitigates parametric shortcuts to the extent that it reduces option dependence. MathQA shows that CoT can improve robustness for some models, while SciQ confirms a strong and broadly shared failure mode: performance collapses when the ground-truth option is removed, and CoT only partially alleviates this. Therefore, NOTA stress tests provide a necessary complement to standard leaderboard accuracy for diagnosing whether CoT gains reflect genuinely option-invariant reasoning.
| Perturbation | ||
| NOTA-as-answer | .346 | 24/24 |
| NOTA-as-distractor | .012 | 3/24 |
Table 5. Robustness to NOTA perturbations across model, dataset, and prompting conditions. denotes the median decrease in accuracy relative to the corresponding Base condition. The final column reports the proportion of the 24 comparisons that remain significant after Benjamini-Hochberg correction over all 48 exact McNemar tests [39].
Conclusion
In this paper, we introduced a task-centric framework that combines layer-wise, decoding-time, representation-based, and behavioural signals to characterise the demands that benchmarks place on LLMs. Across 16 models and 24 benchmarks, we find that knowledge- and reasoning-centric tasks exhibit different internal processing patterns, and that these differences are not fully captured by aggregate task performance alone. At the behavioural level, knowledge performance scales more consistently than reasoning performance. Paired model-level tests detect a significant increase in reasoning scores under CoT. They do not detect a significant change in knowledge scores. At the same time, the NOTA perturbation analysis shows that improved performance under CoT does not necessarily imply option-invariant reasoning: models can remain strongly dependent on the presence of the original ground-truth answer. Taken together, these results illustrate the value of analysing model behaviour along multiple complementary axes. Static layer-wise profiles characterise how task inputs are processed before generation. Decoding-time trajectories reveal differences in within-trajectory variability. Representation-based and perturbation analyses expose distinctions that may remain hidden in final task accuracy. Rather than treating knowledge and reasoning as disjoint capabilities, we provide a view of how their relative demands vary across tasks, models, and prompting conditions.
Code and Data Availability
Our codebase for reproducing the figures and tables are available at https://github.com/extraordinarylab/trace-kr. To ensure accessibility, all figures in this work utilise the colour-blind safe Okabe-Ito palette.
Limitations
Our study is limited to English benchmarks, and the probing, representation-similarity, and NOTA analyses rely on multiple-choice answer structure. Cross-lingual differences in tokenisation and model coverage may change the observed footprints, while open-ended generation may exhibit different reasoning paths and shortcut behaviours. These settings define the scope of our conclusions and motivate future multilingual and open-ended evaluation.
Acknowledgments
We are grateful to the action editor and the anonymous reviewers for their thoughtful feedback and valuable suggestions, which have helped improve the quality and clarity of this work. This work was supported by UKRI Metascience-funded RISC project (APP111963). Eleni Vasilaki was supported by the UK Neuromorphic Computing Hardware Semiconductor IKC (Neuroware; UKRI2784), funded by EPSRC and Innovate UK. The research is supported by Center for AI Safety and Scale AI. _Contributions_ All authors have contributed to the dataset creation process. The Center for AI Safety and Scale AI consortia jointly designed the dataset premise and pipeline; operated the data collection platform (https://lastexam.ai); and provided funding, inference infrastructure for LLMs and review/auditing resources. The authors in the HLE Contributors Consortium contributed to the dataset in various ways, including submitting at least one accepted question to one of the dataset versions, contributing to dataset refinement or assisting with evaluations. In the Center for AI Safety and Scale AI, Long Phan, Alice Gatti, Ziwen Han and Nathaniel Li led the project, and Summer Yue,
Alexandr Wang and Dan Hendrycks provided senior supervision.
Experiments Details
Datasets
Benchmark Taxonomy
Following the operational criteria in the main text, the knowledge-centric group comprises BioMixQA, MedMCQA, MRI-MCQA, SciQ, QASC, MusicTrivia, TruthfulQA, and HaluEval. These benchmarks predominantly test retrieval or verification of biomedical, scientific, cultural, or factual information. The reasoning-centric group comprises AI2-ARC, GPQA-Diamond, RACE, PubMedQA, CommonsenseQA, Winogrande, PIQA, SIQA, MMLU-Pro, MathQA, GSM8K, AIME, MATH500, AMC, CoinFlip, and LogiQA. Their items predominantly require composition, inference from a supplied context, calculation, or state tracking. This grouping records the dominant demand of a benchmark rather than asserting that every item is pure: GPQA and MMLU-Pro require substantial domain knowledge, while SciQ and biomedical QA can require local inference. These mixed cases are precisely why we analyse a graded spectrum and report deviations from the group-level trend.
BioMixQA ([55]) is a biomedical MCQA dataset created to evaluate Knowledge Graph-based Retrieval-Augmented Generation (KG-RAG) frameworks.
MedMCQA ([47]) is a large-scale MCQA dataset with over 194,000 questions from Indian medical entrance exams, testing complex reasoning across 21 medical subjects.
MRI-MCQA ([52]) is a specialised MCQA benchmark covering Magnetic Resonance Imaging (MRI) topics such as cardiac imaging, pulse sequences, and physics.
PubMedQA ([21]) is a biomedical QA dataset requiring yes/no/maybe answers to research questions based on PubMed abstracts, testing the understanding of quantitative results.
SciQ ([66]) is a crowdsourced dataset of 13,679 multiple-choice science questions in Physics, Chemistry, and Biology, most of which include a supporting text with evidence.
QASC ([24]) is a grade-school science dataset with nearly 10,000 questions designed to test multi-hop reasoning by composing facts.
AI2-ARC ([7]) is a dataset of 7,787 grade-school science questions. Its Challenge Set of 2,590 questions is designed to require complex reasoning beyond simple retrieval or word association.
GPQA-Diamond ([49]) is a Google-Proof dataset of 448 challenging science questions created by PhD-level experts, designed to be difficult even with internet access.
MusicTrivia ([60]) is an MCQA dataset testing a wide range of musical knowledge, from classical composers and periods to modern pop/rock artists and general music history.
RACE ([25]) is a large-scale reading comprehension dataset with nearly 100,000 questions sourced from English exams for middle and high school students.
TruthfulQA ([32]) is a benchmark designed to measure a model’s truthfulness using 817 adversarial questions that probe common human misconceptions.
HaluEval ([28]) is a large-scale benchmark for evaluating an LLM’s ability to recognize and avoid generating factually incorrect or unverifiable content (hallucinations).
CommonsenseQA ([58]) is a challenging MCQA dataset designed to evaluate commonsense reasoning ability beyond simple fact retrieval.
Winogrande ([50]) is a large-scale and challenging dataset for commonsense reasoning, designed as a more robust successor to the Winograd Schema Challenge.
PIQA ([5]) is a benchmark for physical commonsense, requiring models to choose the more sensible of two solutions to achieve a practical, real-world goal.
SIQA ([51]) is an MCQA benchmark for testing social and emotional commonsense by requiring reasoning about people’s motivations and actions in social situations.
MMLU-Pro ([64]) is an enhanced version of MMLU that increases difficulty with more complex questions and expands from four to ten multiple-choice options for a more robust evaluation.
| Dataset | Domain | Question Type | Metric | # Examples |
| BioMixQA ([55]) | Biomedical | Multiple-choice | Accuracy | 306 |
| MedMCQA ([47]) | Biomedical | Multiple-choice | Accuracy | 4,183 |
| MRI-MCQA ([52]) | Biomedical | Multiple-choice | Accuracy | 563 |
| PubMedQA ([21]) | Biomedical | Yes/No | Accuracy | 1,000 |
| SciQ ([66]) | Science | Multiple-choice | Accuracy | 1,000 |
| QASC ([24]) | Science | Multiple-choice | Accuracy | 926 |
| AI2-ARC ([7]) | Science | Multiple-choice | Accuracy | 3,548 |
| GPQA-Diamond ([49]) | Science | Multiple-choice | Accuracy | 198 |
| MusicTrivia ([60]) | Music Knowledge | Multiple-choice | Accuracy | 512 |
| RACE ([25]) | Reading Comprehension | Multiple-choice | Accuracy | 4,934 |
| TruthfulQA ([32]) | Factuality Detection | Multiple-choice | Accuracy | 684 |
| HaluEval ([28]) | Factuality Detection | Yes/No | Accuracy | 30,000 |
| CommonsenseQA ([58]) | Commonsense Reasoning | Multiple-choice | Accuracy | 1,221 |
| Winogrande ([50]) | Commonsense Reasoning | Multiple-choice | Accuracy | 1,267 |
| PIQA ([5]) | Commonsense Reasoning | Multiple-choice | Accuracy | 1,838 |
| SIQA ([51]) | Commonsense Reasoning | Multiple-choice | Accuracy | 1,954 |
| MMLU-Pro ([63]) | Multidisciplinary Knowledge | Multiple-choice | Accuracy | 12,032 |
| MathQA ([2]) | Mathematics | Multiple-choice | Accuracy | 2,985 |
| GSM8K ([9]) | Mathematics | Free-form | Exact Match | 1,319 |
| AIME ([36]) | Mathematics | Free-form | Exact Match | 120 |
| Math500 ([15]) | Mathematics | Free-form | Exact Match | 500 |
| AMC ([37]) | Mathematics | Free-form | Exact Match | 134 |
| CoinFlip ([65]) | Symbolic Reasoning | Yes/No | Accuracy | 3,333 |
| LogiQA ([33]) | Logical Reasoning | Multiple-choice | Accuracy | 651 |
Table 1. Statistics of datasets used for knowledge and reasoning analysis, grouped by domain.
MathQA ([2]) is a large-scale dataset for math word problems where each problem is paired with an operational program, promoting interpretable, step-by-step solutions.
GSM8K ([9]) is a dataset of 8,500 high-quality grade school math word problems designed to evaluate multi-step reasoning, with solutions provided as natural language explanations.
AIME ([36]) is a benchmark of problems from the American Invitational Mathematics Examination, a challenging high school competition used to evaluate advanced mathematical reasoning.
MATH500 ([15]) is a benchmark of thousands of competition math problems from pre-algebra to calculus, used to evaluate mathematical problem-solving via step-by-step solutions.
AMC ([37]) consists of problems from the American Mathematics Competitions, used to test pre-collegiate proficiency in algebra, geometry, number theory, and combinatorics.
CoinFlip ([65]) is a symbolic reasoning benchmark that tests a model’s ability to track the binary state of an object through a sequence of actions.
LogiQA ([33]) is a reading comprehension benchmark from civil service exams that tests formal logical reasoning (e.g., categorical, conditional) based on provided premises.
Models
Table 2 provides an overview of the open-source LLMs evaluated in this work. Models from the same family share a common background colour.
| Models | Weights | Model Link | License | Release Time |
| Llama3-8B | Dense | meta-llama/Meta-Llama-3-8B-Instruct | Llama 3 Community | Apr 2024 |
| Llama3.1-8B | Dense | meta-llama/Llama-3.1-8B-Instruct | Llama 3.1 Community | Jul 2024 |
| Qwen2.5-3B | Dense | Qwen/Qwen2.5-3B-Instruct | Apache 2.0 | Sep 2024 |
| Qwen2.5-7B | Dense | Qwen/Qwen2.5-7B-Instruct | Apache 2.0 | Sep 2024 |
| Qwen2.5-14B | Dense | Qwen/Qwen2.5-14B-Instruct | Apache 2.0 | Sep 2024 |
| Qwen3-8B | Dense | Qwen/Qwen3-8B | Apache 2.0 | Apr 2025 |
| Qwen3-30B-A3B | MoE | Qwen/Qwen3-30B-A3B-Instruct-2507 | Apache 2.0 | Jul 2025 |
| Gemma-7B | Dense | google/gemma-7b-it | Gemma Terms of Use | Feb 2024 |
| Gemma-2-2B | Dense | google/gemma-2-2b-it | Gemma Terms of Use | Jul 2024 |
| Gemma-2-9B | Dense | google/gemma-2-9b-it | Gemma Terms of Use | Jun 2024 |
| Gemma-3-4B | Dense | google/gemma-3-4b-it | Gemma Terms of Use | Mar 2025 |
| Gemma-3-12B | Dense | google/gemma-3-12b-it | Gemma Terms of Use | Mar 2025 |
| Gemma-3-27B | Dense | google/gemma-3-27b-it | Gemma Terms of Use | Mar 2025 |
| GLM-4-9B | Dense | zai-org/glm-4-9b-chat-hf | Apache 2.0 | Oct 2024 |
| Olmo-3-7B | Dense | allenai/Olmo-3-7B-Instruct | Apache 2.0 | Nov 2025 |
| GPT-OSS-20B | MoE | openai/gpt-oss-20b | Apache 2.0 | Aug 2025 |
Table 2. Overview of the models evaluated in this work, including their HuggingFace links, licensing terms, and release time. Models from the same family share the same background colour.
Prompts
Our prompt templates are designed to (i) standardise the interface across benchmarks for representation extraction, (ii) avoid confounds from variable-length decoding. Importantly, we do not generate free-form outputs for probing. Instead, we run a single forward pass on the full prompt and extract the hidden state at the last input token from each layer, which serves as the feature for the layer-wise linear classifiers.
Figure 11. Cross-benchmark variability in adjusted cosine similarity for the Llama, Qwen, and OLMo model families. The plotted value is the population standard deviation of those four benchmark means. Dispersion is greatest in early layers and decreases toward a lower, relatively stable level in middle and later layers.
Statistical Validation
We complement the descriptive comparisons in RQ2 with paired non-parametric tests targeting the corresponding experimental units. For the CoT analysis, comparisons are paired across model checkpoints. For the NOTA analysis, comparisons are paired at the item level within each model, dataset, and prompting condition.
CoT prompting. To assess whether the shifts in Figure 10 are systematic across models, we compare the Standard and CoT scores separately for the knowledge and reasoning dimensions using two-sided Wilcoxon signed-rank tests (Wilcoxon, [67]). The model checkpoint is treated as the paired unit; the Think and No-Think configurations of Qwen3-8B are aggregated within the shared checkpoint before testing. We report the median paired change, the number of models with a positive change, and the matched-pairs rank-biserial correlation (Kerby, [23]) as an effect size. Table 4 shows a systematic positive shift in reasoning, whereas the corresponding shift in knowledge is not supported.
NOTA perturbations. For Figure 8 and Figure 9, we test changes in item-level correctness between each Base condition and its corresponding NOTA perturbation using two-sided exact McNemar tests (McNemar, [39]). Tests are conducted separately for each model dataset prompting condition, yielding 24 comparisons for NOTA-asanswer and 24 for NOTA-as-distractor. We control the false discovery rate across the resulting 48 tests using the Benjamini-Hochberg procedure [4]. As summarised in Table 5, the NOTA-as-answer effect is consistently detected across conditions, whereas the NOTA-as-distractor effect is comparatively limited.
| Model | MRI-MCQA (0-shot) Std. | MRI-MCQA (0-shot) CoT | BioMixQA (0-shot) Std. | BioMixQA (0-shot) CoT | SciQ (0-shot) Std. | SciQ (0-shot) CoT | QASC (0-shot) Std. | QASC (0-shot) CoT | MusicTrivia (0-shot) Std. | MusicTrivia (0-shot) CoT | TruthfulQA (0-shot) Std. | TruthfulQA (0-shot) CoT | MedMCQA (0-shot) Std. | MedMCQA (0-shot) CoT |
| Llama3-8B-Instruct | 52.58 | 50.80 | 50.98 | 45.75 | 90.60 | 90.70 | 76.24 | 74.51 | 76.37 | 73.44 | 52.05 | 54.39 | 57.42 | 53.45 |
| Llama3.1-8B-Instruct | 54.88 | 55.77 | 49.67 | 49.67 | 91.60 | 93.30 | 79.59 | 80.78 | 73.83 | 76.76 | 55.12 | 59.94 | 56.87 | 59.12 |
| Qwen2.5-3B-Instruct | 49.38 | 48.31 | 36.60 | 37.91 | 91.70 | 92.00 | 72.03 | 74.30 | 71.48 | 71.29 | 59.94 | 57.02 | 48.24 | 47.12 |
| Qwen2.5-7B-Instruct | 55.95 | 56.48 | 39.87 | 40.52 | 94.70 | 94.40 | 79.59 | 82.07 | 80.47 | 79.49 | 69.88 | 64.04 | 55.22 | 55.13 |
| Qwen2.5-14B-Instruct | 60.04 | 63.23 | 58.82 | 57.19 | 95.80 | 95.90 | 82.94 | 85.85 | 85.55 | 86.52 | 78.07 | 73.39 | 60.41 | 60.27 |
| Qwen3-8B (No-Think) | 67.14 | 54.71 | 57.52 | 44.77 | 96.90 | 95.20 | 86.93 | 83.26 | 84.57 | 82.42 | 74.85 | 73.25 | 65.34 | 57.14 |
| Qwen3-8B (Think) | 67.50 | 70.69 | 57.52 | 57.19 | 96.50 | 95.40 | 86.50 | 82.51 | 84.96 | 81.64 | 73.68 | 72.66 | 65.31 | 56.56 |
| Gemma-7B-Instruct | 40.14 | 25.93 | 38.89 | 14.71 | 85.50 | 64.30 | 67.71 | 46.11 | 57.03 | 37.11 | 31.87 | 23.50 | 39.88 | 28.74 |
| Gemma-2-2B-Instruct | 41.74 | 36.23 | 35.29 | 37.25 | 88.50 | 88.30 | 67.39 | 67.39 | 67.19 | 62.11 | 46.93 | 50.73 | 41.55 | 39.97 |
| Gemma-2-9B-Instruct | 54.35 | 53.64 | 49.02 | 46.41 | 94.90 | 93.90 | 80.89 | 80.35 | 82.23 | 79.88 | 72.81 | 73.83 | 54.46 | 53.69 |
| Gemma-3-4B-Instruct | 41.39 | 43.34 | 34.64 | 32.35 | 86.80 | 88.50 | 67.49 | 72.35 | 66.80 | 70.90 | 51.02 | 48.98 | 43.39 | 46.21 |
| Gemma-3-12B-Instruct | 56.13 | 61.63 | 52.29 | 49.35 | 93.50 | 94.50 | 80.56 | 82.29 | 84.18 | 83.59 | 68.57 | 65.94 | 55.51 | 58.81 |
| Gemma-3-27B-Instruct | 60.92 | 63.94 | 50.98 | 52.61 | 95.30 | 94.90 | 82.51 | 83.80 | 85.35 | 86.33 | 75.15 | 72.37 | 61.20 | 64.12 |
| GLM-4-9B-Instruct | 45.47 | 44.58 | 43.79 | 37.25 | 91.10 | 90.80 | 74.84 | 71.60 | 73.63 | 74.41 | 56.43 | 53.80 | 46.98 | 45.21 |
| Olmo-3-7B-Instruct | 47.42 | 53.11 | 34.64 | 48.69 | 88.80 | 92.30 | 65.77 | 75.16 | 67.97 | 70.12 | 60.09 | 56.58 | 41.07 | 47.86 |
| GPT-OSS-20B | 63.06 | 67.32 | 61.76 | 62.09 | 51.50 | 81.00 | 49.68 | 72.79 | 60.94 | 74.80 | 42.11 | 64.77 | 50.25 | 57.04 |
| Qwen3-30B-A3B | 71.94 | 74.07 | 59.48 | 64.38 | 97.30 | 97.30 | 89.09 | 88.44 | 87.50 | 87.70 | 78.07 | 75.58 | 70.95 | 71.00 |
Table 6.
| Model | HaluEval-D (6-shot) Std. | HaluEval-D (6-shot) CoT | HaluEval-Q (7-shot) Std. | HaluEval-Q (7-shot) CoT | HaluEval-S (3-shot) Std. | HaluEval-S (3-shot) CoT |
| Llama3-8B-Instruct | 82.10 | 93.94 | 78.98 | 82.69 | 71.74 | 83.86 |
| Llama3.1-8B-Instruct | 80.22 | 81.74 | 68.33 | 73.28 | 65.52 | 76.48 |
| Qwen2.5-3B-Instruct | 52.19 | 50.61 | 49.14 | 49.65 | 45.45 | 45.66 |
| Qwen2.5-7B-Instruct | 90.87 | 90.27 | 67.33 | 67.52 | 58.75 | 53.73 |
| Qwen2.5-14B-Instruct | 94.04 | 93.89 | 88.39 | 88.11 | 84.90 | 85.40 |
| Qwen3-8B (No-Think) | 79.56 | 89.30 | 84.56 | 87.28 | 69.14 | 73.49 |
| Qwen3-8B (Think) | 79.89 | 88.86 | 84.90 | 87.92 | 69.21 | 73.47 |
| Gemma-7B-Instruct | 58.37 | 59.42 | 64.03 | 65.36 | 96.04 | 96.95 |
| Gemma-2-2B-Instruct | 56.35 | 55.37 | 28.41 | 28.82 | 44.19 | 46.54 |
| Gemma-2-9B-Instruct | 71.95 | 71.89 | 70.73 | 71.81 | 70.73 | 71.81 |
| Gemma-3-4B-Instruct | 46.88 | 59.34 | 47.00 | 43.87 | 58.08 | 57.02 |
| Gemma-3-12B-Instruct | 79.11 | 79.29 | 41.10 | 70.65 | 71.21 | 70.65 |
| Gemma-3-27B-Instruct | 81.13 | 74.26 | 34.83 | 55.80 | 53.14 | 55.80 |
| GLM-4-9B-Instruct | 76.77 | 67.27 | 50.13 | 52.93 | 52.48 | 52.93 |
| GPT-OSS-20B | 98.20 | 98.20 | 95.38 | 96.42 | 95.38 | 96.42 |
| Qwen3-30B-A3B | 77.69 | 77.97 | 83.13 | 68.83 | 69.52 | 68.83 |
Table 6. Performance of all models across knowledge-based benchmarks. For each dataset, we report accuracy under both standard (Std.) and CoT prompting. The best-performing model in each setting is highlighted in bold.
| Model | GSM8K (8-shot) Std. | GSM8K (8-shot) CoT | MathQA (0-shot) Std. | MathQA (0-shot) CoT | MATH500 (0-shot) Std. | MATH500 (0-shot) CoT | AMC (0-shot) Std. | AMC (0-shot) CoT | AIME22 (0-shot) Std. | AIME22 (0-shot) CoT | AIME23 (0-shot) Std. | AIME23 (0-shot) CoT | AIME24 (0-shot) Std. | AIME24 (0-shot) CoT | AIME25 (0-shot) Std. | AIME25 (0-shot) CoT |
| Llama3-8B-Instruct | 74.75 | 75.06 | 54.84 | 55.28 | 23.20 | 23.80 | 5.22 | 7.46 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 3.33 | 0.00 | 0.00 |
| Llama3.1-8B-Instruct | 78.77 | 82.26 | 54.17 | 69.45 | 47.40 | 50.20 | 12.69 | 16.42 | 6.67 | 0.00 | 3.33 | 3.33 | 3.33 | 6.67 | 0.00 | 0.00 |
| Qwen2.5-3B-Instruct | 79.08 | 81.65 | 77.79 | 77.25 | 65.80 | 65.60 | 27.61 | 38.06 | 3.33 | 3.33 | 6.67 | 3.33 | 16.67 | 3.33 | 0.00 | 3.33 |
| Qwen2.5-7B-Instruct | 86.66 | 88.48 | 38.63 | 83.28 | 76.40 | 73.40 | 42.54 | 43.28 | 16.67 | 6.67 | 13.33 | 10.00 | 16.67 | 10.00 | 20.00 | 13.33 |
| Qwen2.5-14B-Instruct | 90.75 | 90.14 | 48.54 | 85.16 | 78.60 | 80.40 | 48.51 | 51.49 | 6.67 | 10.00 | 10.00 | 13.33 | 10.00 | 20.00 | 16.67 | 13.33 |
| Qwen3-8B (No-Think) | 88.40 | 90.52 | 53.74 | 89.65 | 22.00 | 24.40 | 8.96 | 80.60 | 3.33 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 3.33 | 3.33 |
| Qwen3-8B (Think) | 89.31 | 91.13 | 68.94 | 89.78 | 22.40 | 22.00 | 8.21 | 89.55 | 0.00 | 10.00 | 3.33 | 0.00 | 0.00 | 0.00 | 0.00 | 3.33 |
| Gemma-7B-Instruct | 16.68 | 24.72 | 20.13 | 23.25 | 6.20 | 5.00 | 2.24 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 |
| Gemma-2-2B-Instruct | 39.50 | 37.76 | 20.70 | 33.13 | 21.00 | 21.00 | 0.00 | 2.24 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 |
| Gemma-2-9B-Instruct | 79.45 | 81.80 | 28.07 | 61.74 | 49.20 | 50.80 | 13.43 | 10.45 | 0.00 | 0.00 | 3.33 | 6.67 | 0.00 | 6.67 | 0.00 | 0.00 |
| Gemma-3-4B-Instruct | 81.35 | 82.49 | 25.80 | 73.63 | 73.20 | 76.60 | 37.31 | 38.81 | 3.33 | 10.00 | 6.67 | 10.00 | 6.67 | 20.00 | 10.00 | 16.67 |
| Gemma-3-12B-Instruct | 89.23 | 89.16 | 32.19 | 78.76 | 84.20 | 83.60 | 52.98 | 52.24 | 13.33 | 26.67 | 13.33 | 13.33 | 23.33 | 23.33 | 20.00 | 20.00 |
| Gemma-3-27B-Instruct | 92.65 | 91.66 | 46.73 | 85.29 | 87.80 | 88.40 | 60.45 | 58.96 | 16.67 | 16.67 | 20.00 | 13.33 | 26.67 | 30.00 | 26.67 | 23.33 |
| GLM-4-9B-Instruct | 65.73 | 64.75 | 28.71 | 50.22 | 28.40 | 27.40 | 7.46 | 5.97 | 0.00 | 0.00 | 0.00 | 3.33 | 0.00 | 0.00 | 0.00 | 0.00 |
| Olmo-3-7B-Instruct | 88.40 | 88.32 | 33.03 | 81.68 | 79.00 | 78.00 | 44.78 | 50.00 | 16.67 | 23.33 | 13.33 | 13.33 | 16.67 | 13.33 | 20.00 | 23.33 |
| GPT-OSS-20B | 94.54 | 93.78 | 75.08 | 78.16 | 79.60 | 76.20 | 18.66 | 20.15 | 13.33 | 6.67 | 10.00 | 6.67 | 16.67 | 20.00 | 16.67 | 20.00 |
| Qwen3-30B-A3B | 93.63 | 93.56 | 90.28 | 90.32 | 95.60 | 95.60 | 71.64 | 77.61 | 73.33 | 83.33 | 80.00 | 76.67 | 83.33 | 73.33 | 73.33 | 73.33 |
Table 7. Performance on Mathematical Reasoning benchmarks (Part 1). Scores are reported as accuracy. The number of few-shot examples used is in parentheses. The best-performing model in each setting is highlighted in bold.
| Model | CSQA (0-shot) Std. | CSQA (0-shot) CoT | SiQA (0-shot) Std. | SiQA (0-shot) CoT | PiQA (0-shot) Std. | PiQA (0-shot) CoT | Wino. (0-shot) Std. | Wino. (0-shot) CoT | LogiQA (0-shot) Std. | LogiQA (0-shot) CoT |
| Llama3-8B-Instruct | 74.53 | 71.58 | 71.85 | 68.94 | 80.63 | 77.75 | 59.75 | 62.12 | 43.63 | 41.47 |
| Llama3.1-8B-Instruct | 73.71 | 74.61 | 67.40 | 73.13 | 81.01 | 83.35 | 60.85 | 64.33 | 41.94 | 42.09 |
| Qwen2.5-3B-Instruct | 75.59 | 73.38 | 72.98 | 70.01 | 77.15 | 79.33 | 63.69 | 64.33 | 43.47 | 46.24 |
| Qwen2.5-7B-Instruct | 82.06 | 79.61 | 78.05 | 77.18 | 87.05 | 87.21 | 66.93 | 72.38 | 53.76 | 54.22 |
| Qwen2.5-14B-Instruct | 82.06 | 81.41 | 78.56 | 78.35 | 89.17 | 92.00 | 77.19 | 81.14 | 61.60 | 58.22 |
| Qwen3-8B (No-Think) | 83.13 | 81.98 | 77.69 | 76.71 | 90.32 | 88.47 | 80.27 | 78.85 | 51.31 | 42.70 |
| Qwen3-8B (Think) | 83.46 | 81.65 | 76.71 | 76.97 | 89.93 | 88.85 | 81.69 | 77.51 | 75.58 | 53.00 |
| Gemma-7B-Instruct | 68.96 | 40.87 | 64.38 | 42.99 | 71.76 | 51.36 | 56.20 | 37.81 | 34.41 | 16.74 |
| Gemma-2-2B-Instruct | 69.78 | 68.30 | 69.29 | 66.63 | 74.59 | 71.76 | 53.91 | 52.57 | 43.01 | 38.10 |
| Gemma-2-9B-Instruct | 80.34 | 78.46 | 75.03 | 73.23 | 87.05 | 84.22 | 72.45 | 71.11 | 51.92 | 49.62 |
| Gemma-3-4B-Instruct | 67.81 | 70.02 | 66.68 | 67.55 | 77.97 | 79.76 | 56.83 | 58.96 | 46.54 | 43.78 |
| Gemma-3-12B-Instruct | 78.21 | 79.28 | 75.33 | 76.82 | 86.72 | 87.70 | 69.69 | 71.19 | 58.06 | 58.53 |
| Gemma-3-27B-Instruct | 79.77 | 79.12 | 77.12 | 77.18 | 89.88 | 89.66 | 73.48 | 75.69 | 61.60 | 62.52 |
| GLM-4-9B-Instruct | 78.87 | 76.66 | 74.41 | 74.92 | 80.36 | 81.99 | 69.93 | 67.72 | 48.39 | 45.78 |
| Olmo-3-7B-Instruct | 72.24 | 75.35 | 67.30 | 69.96 | 75.35 | 80.25 | 57.62 | 63.93 | 39.63 | 43.78 |
| GPT-OSS-20B | 48.16 | 73.87 | 54.61 | 73.64 | 40.75 | 77.64 | 56.43 | 73.56 | 54.22 | 54.53 |
| Qwen3-30B-A3B | 83.95 | 83.70 | 78.97 | 79.48 | 93.47 | 93.14 | 84.69 | 85.48 | 76.34 | 76.04 |
Table 8. Performance on General and Commonsense Reasoning benchmarks (Part 2). Scores are reported as accuracy. The number of few-shot examples used is in parentheses. The best-performing model is highlighted in bold.
| Model | ARC-Easy (0-shot) Std. | ARC-Easy (0-shot) CoT | ARC-Chal. (0-shot) Std. | ARC-Chal. (0-shot) CoT | RACE (3-shot) Std. | RACE (3-shot) CoT | PubMedQA (0-shot) Std. | PubMedQA (0-shot) CoT | MMLU-Pro (5-shot) Std. | MMLU-Pro (5-shot) CoT | GPQA-Dia. (0-shot) Std. | GPQA-Dia. (0-shot) CoT | CoinFlip (8-shot) Std. | CoinFlip (8-shot) CoT |
| Llama3-8B-Instruct | 87.84 | 87.88 | 77.99 | 81.14 | 49.56 | 65.08 | 63.00 | 81.80 | 22.75 | 39.63 | 29.29 | 32.32 | 67.45 | 72.07 |
| Llama3.1-8B-Instruct | 88.01 | 89.98 | 79.27 | 85.07 | 72.31 | 82.37 | 70.90 | 82.20 | 44.96 | 46.99 | 28.79 | 26.26 | 59.89 | 75.31 |
| Qwen2.5-3B-Instruct | 89.56 | 89.31 | 82.08 | 81.83 | 81.19 | 81.25 | 58.40 | 66.10 | 44.42 | 44.85 | 29.80 | 33.84 | 58.39 | 65.89 |
| Qwen2.5-7B-Instruct | 92.13 | 92.26 | 87.63 | 89.93 | 87.63 | 86.16 | 53.60 | 58.10 | 56.58 | 56.70 | 32.83 | 34.85 | 73.24 | 70.81 |
| Qwen2.5-14B-Instruct | 93.73 | 93.86 | 91.13 | 91.81 | 91.06 | 90.70 | 44.80 | 74.80 | 64.66 | 64.83 | 32.83 | 40.91 | 71.83 | 84.01 |
| Qwen3-8B (No-Think) | 94.32 | 93.22 | 93.69 | 90.61 | 88.53 | 90.44 | 65.70 | 79.40 | 63.03 | 73.90 | 18.69 | 56.06 | 78.40 | 85.45 |
| Qwen3-8B (Think) | 94.49 | 93.06 | 93.77 | 91.04 | 89.34 | 90.42 | 65.60 | 68.70 | 63.07 | 78.05 | 17.68 | 58.08 | 86.35 | 85.45 |
| Gemma-7B-Instruct | 81.19 | 57.62 | 69.54 | 47.01 | 44.10 | 28.28 | 72.10 | 73.60 | 9.58 | 9.99 | 28.28 | 17.68 | 14.01 | 2.46 |
| Gemma-2-2B-Instruct | 84.60 | 85.90 | 72.01 | 75.34 | 74.79 | 61.49 | 39.50 | 76.50 | 24.28 | 25.32 | 30.30 | 17.17 | 42.24 | 35.01 |
| Gemma-2-9B-Instruct | 92.00 | 92.63 | 88.05 | 89.08 | 87.41 | 86.58 | 41.60 | 79.00 | 50.46 | 50.65 | 30.30 | 26.26 | 36.93 | 42.93 |
| Gemma-3-4B-Instruct | 85.06 | 86.41 | 74.74 | 80.72 | 76.57 | 76.83 | 55.20 | 78.30 | 42.19 | 41.71 | 33.84 | 24.75 | 50.02 | 52.69 |
| Gemma-3-12B-Instruct | 92.42 | 93.35 | 88.23 | 91.30 | 87.98 | 87.80 | 49.40 | 77.40 | 60.44 | 59.98 | 32.83 | 37.37 | 78.46 | 79.90 |
| Gemma-3-27B-Instruct | 93.81 | 93.86 | 91.04 | 92.15 | 88.95 | 89.48 | 58.20 | 80.50 | 67.19 | 67.46 | 40.40 | 46.97 | 74.17 | 79.66 |
| GLM-4-9B-Instruct | 90.45 | 89.23 | 85.07 | 84.22 | 83.10 | 77.47 | 53.80 | 41.80 | 30.87 | 35.89 | 28.28 | 22.73 | 78.55 | 77.77 |
| Olmo-3-7B-Instruct | 87.21 | 90.15 | 75.77 | 85.67 | 80.07 | 81.09 | 51.30 | 72.00 | 51.26 | 52.18 | 30.30 | 30.81 | 70.54 | 78.67 |
| GPT-OSS-20B | 63.72 | 83.25 | 70.99 | 87.71 | 67.29 | 82.69 | 96.30 | 95.30 | 61.37 | 60.24 | 46.97 | 50.51 | 18.63 | 13.92 |
| Qwen3-30B-A3B | 95.03 | 94.74 | 94.88 | 94.88 | 91.38 | 91.41 | 72.30 | 71.40 | 77.78 | 77.64 | 59.09 | 61.11 | 80.02 | 80.02 |
Table 9. Performance on Reading Comprehension and Multi-Discipline Reasoning benchmarks (Part 3). Scores are reported as accuracy. The number of few-shot examples is in parentheses. The best model is in bold.
References
- [1]Yanda Chen, Joe Benton, Ansh Radhakrishnan, Jonathan Uesato, Carson Denison, John Schulman, Arushi Somani, Peter Hase, Misha Wagner, Fabien Roger, Vlad Mikulik, Samuel R. Bowman, Jan Leike, Jared Kaplan, and Ethan Perez. 2025. Reasoning models don’t always say what they think.arxiv.org/abs/2505.05410
- [2]Aida Amini, Saadia Gabriel, Shanchuan Lin, Rik Koncel-Kedziorski, Yejin Choi, and Hannaneh Hajishirzi. 2019. MathQA: Towards interpretable math word problem solving with operation-based formalisms. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 2357–2367, Minneapolis, Minnesota. Association for Computational Linguistics.arxiv.org/abs/1905.13319
- [3]Tomer Ashuach, Shai Gretz, Yoav Katz, Yonatan Belinkov, and Liat Ein-Dor. 2026. Masked by consensus: Disentangling privileged knowledge in LLM correctness. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10577–10596, San Diego, California, United States. Association for Computational Linguistics.
- [4]Yoav Benjamini and Yosef Hochberg. 1995. Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B (Methodological), 57(1):289–300.DOI
- [5]Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao, and Yejin Choi. 2020. Piqa: Reasoning about physical commonsense in natural language. In Proceedings of the AAAI conference on artificial intelligence.arxiv.org/abs/1911.11641
- [6]Qiguang Chen, Libo Qin, Jiaqi Wang, Jingxuan Zhou, and Wanxiang Che. 2024. Unlocking the capabilities of thought: A reasoning boundary framework to quantify and optimize chain-of-thought. Advances in Neural Information Processing Systems, 37:54872–54904.arxiv.org/abs/2410.05695
- [7]Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457.arxiv.org/abs/1803.05457
- [8]Norman Cliff. 1993. Dominance statistics: Ordinal analyses to answer ordinal questions. Psychological bulletin, 114(3):494.DOI
- [9]Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168.arxiv.org/abs/2110.14168
- [10]Nouha Dziri, Ximing Lu, Melanie Sclar, Xiang Lorraine Li, Liwei Jiang, Bill Yuchen Lin, Sean Welleck, Peter West, Chandra Bhagavatula, Ronan Le Bras, Jena D. Hwang, Soumya Sanyal, Xiang Ren, Allyson Ettinger, Zaid Harchaoui, and Yejin Choi. 2023. Faith and fate: Limits of transformers on compositionality. In Thirty-seventh Conference on Neural Information Processing Systems.arxiv.org/abs/2305.18654
- [11]Tao Feng, Xinke Jiang, Xinyan Hu, Yonggang Zhang, Zhen Tao, Wentao Zhang, Boyang Liu, Wenhao Jiang, and Chao Wu. 2026a. Trust within? seek beyond? knowledge boundary aware policy optimization for agentic search. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 27664–27682, San Diego, California, United States. Association for Computational Linguistics. Yiyang Feng, Zeming Chen, Haotian Wu, Jiawei Zhou, and Antoine Bosselut. 2026b. Tracking the limits of knowledge propagation: How LLMs fail at multi-step reasoning with conflicting knowledge. In Proceedings of the 19th Conference of the European Chapter of the Asso ciation for Computational Linguistics (Volume 1: Long Papers), pages 5813–5847, Rabat, Morocco. Association for Computational Linguistics.
- [12]Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. 2021. Transformer feed-forward layers are key-value memories. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5484–5495, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.arxiv.org/abs/2012.14913
- [13]Andrey Gromov, Kushal Tirumala, Hassan Shapourian, Paolo Glorioso, and Dan Roberts. 2025. The unreasonable ineffectiveness of the deeper layers. In The Thirteenth International Conference on Learning Representations.arxiv.org/abs/2403.17887
- [14]Philip M Groves and Richard F Thompson. 1970. Habituation: a dual-process theory. Psychological review, 77(5):419.pubmed.ncbi.nlm.nih.gov/4319167
- [15]Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring mathematical problem solving with the MATH dataset. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2).
- [16]Hanhua Hong, Chenghao Xiao, Yang Wang, Yiqi Liu, Wenge Rong, and Chenghua Lin. 2026. Beyond one-size-fits-all: Inversion learning for highly effective nlg evaluation prompts. Transactions of the Association for Computational Linguistics, 14:689–710.
- [17]Yifan Hou, Jiaoda Li, Yu Fei, Alessandro Stolfo, Wangchunshu Zhou, Guangtao Zeng, Antoine Bosselut, and Minrmay Sachan. 2023. Towards a mechanistic interpretation of multi-step reasoning capabilities of language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 4902–4919, Singapore. Association for Computational Linguistics.
- [18]Xinda Jia, Jinpeng Li, Zezhong Wang, Jingjing Li, Xingshan Zeng, Yasheng Wang, Weinan Zhang, Yong Yu, and Weiwen Liu. 2025. Fast, slow, and tool-augmented thinking for llms: A review. arXiv preprint arXiv:2508.12265.arxiv.org/abs/2508.12265
- [19]Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R Narasimhan. 2024. SWE-bench: Can language models resolve real-world github issues? In The Twelfth International Conference on Learning Representations.arxiv.org/abs/2310.06770
- [20]Mingyu Jin, Weidi Luo, Sit ao Cheng, Xinyi Wang, Wenyue Hua, Ruixiang Tang, William Yang Wang, and Yongfeng Zhang. 2025. Disentangling memory and reasoning ability in large language models. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1681–1701, Vienna, Austria. Association for Computational Linguistics.arxiv.org/abs/2411.13504
- [21]Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William Cohen, and Xinghua Lu. 2019. PubMedQA: A dataset for biomedical research question answering. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2567–2577, Hong Kong, China. Association for Computational Linguistics.arxiv.org/abs/1909.06146
- [22]Daniel Kahneman. 2011. Thinking, fast and slow. Farrar, Straus and Giroux.
- [23]Dave S Kerby. 2014. The simple difference formula: An approach to teaching nonparametric correlation. Comprehensive Psychology, 3:11–II.DOI
- [24]Tushar Khot, Peter Clark, Michal Guerquin, Peter Jansen, and Ashish Sabharwal. 2020. Qasc: A dataset for question answering via sentence composition. In Proceedings of the AAAI Conference on Artificial Intelligence.arxiv.org/abs/1910.11473
- [25]Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang, and Eduard Hovy. 2017. RACE: Large-scale ReAding comprehension dataset from examinations. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 785–794, Copenhagen, Denmark. Association for Computational Linguistics.arxiv.org/abs/1704.04683
- [26]Tamera Lanham, Anna Chen, Ansh Radhakrishnan, Benoit Steiner, Carson Denison, Danny Hernandez, Dustin Li, Esin Durmus, Evan Hubinger, Jackson Kernion, Kamilė Lukošiūtė, Karina Nguyen, Newton Cheng, Nicholas Joseph, Nicholas Schiefer, Oliver Rausch, Robin Larson, Sam McCandlish, Sandipan Kundu, Saurav Kadavath, Shannon Yang, Thomas Henighan, Timothy Maxwell, Timothy Telleen-Lawton, Tristan Hume, Zac Hatfield-Dodds, Jared Kaplan, Jan Brauner, Samuel R. Bowman, and Ethan Perez. 2023. Measuring faithfulness in chain-of-thought reasoning.arxiv.org/abs/2307.13702
- [27]Hans Hergen Lehmann, Jae Hee Lee, Steven Schockaert, and Stefan Wermter. 2026. Knowing the facts but choosing the shortcut: Understanding how large language models compare entities. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4788–4821, Rabat, Morocco. Association for Computational Linguistics.
- [28]Junyi Li, Xiaoxue Cheng, Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. 2023. HaluEval: A large-scale hallucination evaluation benchmark for large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6449–6464, Singapore. Association for Computational Linguistics.
- [29]Moxin Li, Yong Zhao, Wenxuan Zhang, Shuaiyi Li, Wenya Xie, See-Kiong Ng, Tat-Seng Chua, and Yang Deng. 2025. Knowledge boundary of large language models: A survey. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5131–5157, Vienna, Austria. Association for Computational Linguistics.
- [30]Pengqi Li, Lizhong Ding, Zhehao Zhou, Chunhui Zhang, Jiarun Fu, Hao Li, Ye Yuan, and Guoren Wang. 2026a. Demystifying uncertainty in LLMs: Active calibration between concepts and human evaluations. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5726–5759, San Diego, California, United States. Association for Computational Linguistics.
- [31]Shanghai Li, Jinda Han, Yibo Wang, Yuanjie Zhu, Zihe Song, Langzhou He, Kenan Kamel A Alghythee, and Philip S. Yu. 2026b. Why LLMs hallucinate on structured knowledge: A mechanistic analysis of reasoning over linearized representations. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 19943–19956, San Diego, California, United States. Association for Computational Linguistics.
- [32]Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. TruthfulQA: Measuring how models mimic human falsehoods. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3214–3252, Dublin, Ireland. Association for Computational Linguistics.arxiv.org/abs/2109.07958
- [33]Jian Liu, Leyang Cui, Hanmeng Liu, Dandan Huang, Yile Wang, and Yue Zhang. 2020. Logiqa: A challenge dataset for machine reading comprehension with logical reasoning. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20, pages 3622–3628. International Joint Conferences on Artificial Intelligence Organization.arxiv.org/abs/2007.08124
- [34]Yiqi Liu, Yang Wang, Songxin Wang, Chenghao Xiao, and Chenghua Lin. 2026. Broken symmetry in llm refusal: Answer release is more local than refusal restoration. arXiv preprint arXiv:2608.15772.arxiv.org/abs/2608.15772
- [35]M-A-P Team, Xinrun Du, Yifan Yao, Kaijing Ma, Bingli Wang, Tianyu Zheng, Kang Zhu, Minghao Liu, Yiming Liang, Xiaolong Jin, Zhenlin Wei, Chujie Zheng, Kaixing Deng, Shuyue Guo, Shian Jia, Sichao Jiang, Yiyan Liao, Rui Li, Qinrui Li, Sirun Li, Yizhi Li, Yunwen Li, Dehua Ma, Yuansheng Ni, Haoran Que, Qiyao Wang, Zhoufutu Wen, Siwei Wu, Tianshun Xing, Ming Xu, Zhenzhu Yang, Zekun Moore Wang, Junting Zhou, Yuelin Bai, Xingyuan Bu, Chenglin Cai, Liang Chen, Yifan Chen, Chengtuo Cheng, Tianhao Cheng, Keyi Ding, Siming Huang, Yun Huang, Yaoru Li, Yizhe Li, Zhaoqun Li, Tianhao Liang, Chengdong Lin, Hongquan Lin, Yinghao Ma, Zhongyuan Peng, Zifan Peng, Qige Qi, Shi Qiu, Xingwei Qu, Yizhou Tan, Zili Wang, Chenqing Wang, Hao Wang, Yiya Wang, Yubo Wang, Jiajun Xu, Kexin Yang, Ruibin Yuan, Yuanhao Yue, Tianyang Zhan, Chun Zhang, Jingyang Zhang, Xiyue Zhang, Xingjian Zhang, Yue Zhang, Yongchi Zhao, Xiangyu Zheng, Chenghua Zhong, Yang Gao, Zhoujun Li, Dayiheng Liu, Qian Liu, Tianyu Liu, Shiwen Ni, Junran Peng, Yujia Qin, Wenbo Su, Guoyin Wang, Shi Wang, Jian Yang, Min Yang, Meng Cao, Xiang Yue, Zhaoxiang Zhang, Wangchunshu Zhou, Jiacheng Liu, Qunshu Lin, Wenhao Huang, and Ge Zhang. 2026. SuperGPQA: Scaling LLM evaluation across 285 graduate disciplines. In The Thirty-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track.arxiv.org/abs/2502.14739
- [36]MAA. 2025a. Aime problems and solutions. https://artofproblemsolving.com/wiki/index.php/AIME_Problems_and_Solutions.
- [37]MAA. 2025b. Amc 12 problems and solutions. https://artofproblemsolving.com/wiki/index.php/AMC_12_Problems_and_Solutions.
- [38]Potsawee Manakul, Adian Liusie, and Mark Gales. 2023. SelfCheckGPT: Zero-resource black-box hallucination detection for generative large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 9004–9017, Singapore. Association for Computational Linguistics.
- [39]Quinn McNemar. 1947. Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika, 12(2):153–157.pubmed.ncbi.nlm.nih.gov/20254758
- [40]Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022. Locating and editing factual associations in gpt. Advances in neural information processing systems, 35:17359–17372.arxiv.org/abs/2202.05262
- [41]Kevin Meng, Arnab Sen Sharma, Alex J Andonian, Yonatan Belinkov, and David Bau. 2023. Mass-editing memory in a transformer. In The Eleventh International Conference on Learning Representations.arxiv.org/abs/2210.07229
- [42]Seyed Iman Mirzadeh, Keivan Alizadeh, Hooman Shahrokhi, Oncel Tuzel, Samy Bengio, and Mehrdad Farajtabar. 2025. GSM-symbolic: Understanding the limitations of mathematical reasoning in large language models. In The Thirteenth International Conference on Learning Representations.arxiv.org/abs/2410.05229
- [43]ModelScope. 2024. EvalScope: Evaluation framework for large models.
- [44]Neel Nanda, Lawrence Chan, Tom Lieberum, Jess Smith, and Jacob Steinhardt. 2023. Progress measures for grokking via mechanistic interpretability. In The Eleventh International Conference on Learning Representations.arxiv.org/abs/2301.05217
- [45]Shiyu Ni, Keping Bi, Jiafeng Guo, Lulu Yu, Baolong Bi, and Xueqi Cheng. 2025. Towards fully exploiting LLM internal states to enhance knowledge boundary perception. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 24315–24329, Vienna, Austria. Association for Computational Linguistics.
- [46]Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Scott Johnston, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah. 2022. In-context learning and induction heads. arXiv preprint arXiv:2209.11895.arxiv.org/abs/2209.11895
- [47]Ankit Pal, Logesh Kumar Umapathi, and Malaikannan Sankarasubbu. 2022. Medmcqa: A large-scale multi-subject multi-choice dataset for medical domain question answering. In Conference on health, inference, and learning, pages 248–260. PMLR.arxiv.org/abs/2203.14371
- [48]Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah Smith, and Mike Lewis. 2023. Measuring and narrowing the compositionality gap in language models. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 5687–5711, Singapore. Association for Computational Linguistics.arxiv.org/abs/2210.03350
- [49]David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. 2024. Gpqa: A graduate-level google-proof q&a benchmark. In First Conference on Language Modeling.arxiv.org/abs/2311.12022
- [50]Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2021. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 64(9):99–106.
- [52]Oscar Molina Sedano. 2025. Mri-mcqa. https://huggingface.co/datasets/HPAI-BSC/MRI-MCQA.
- [53]Guangyuan Shi, Zexin Lu, Xiaoyu Dong, Wenlong Zhang, Xuanyu Zhang, Yujie Feng, and Xiao-Ming Wu. 2024. Understanding layer significance in llm alignment. CoRR, abs/2410.17875.arxiv.org/abs/2410.17875
- [54]Zhichao Shi, Xuhui Jiang, Chengjin Xu, Cangli Yao, Shengjie Ma, Yinghan Shen, Zixuan Li, Jian Guo, and Yuanzhuo Wang. 2026. JudgeAgent: Beyond static benchmarks for knowledge-driven and dynamic LLM evaluation. In Findings of the Association for Computational Linguistics: ACL 2026, pages 13004–13030, San Diego, California, United States. Association for Computational Linguistics.arxiv.org/abs/2509.02097
- [55]Karthik Soman, Peter W Rose, John H Morris, Rabia E Akbas, Brett Smith, Braian Petoom, Catalina Villouta-Reyes, Gabriel Cerono, Yongmei Shi, Angela Rizk-Jackson, Sharat Israni, Charlotte A Nelson, Sui Huang, and Sergio E Baranzini. 2024. Biomedical knowledge graph-optimized prompt generation for large language models. Bioinformatics, 40(9):btae560.DOI
- [56]Xinyuan Song, Keyu Wang, PengXiang Li, Lu Yin, and Shiwei Liu. 2025. Demystifying the roles of llm layers in retrieval, knowledge, and reasoning.arxiv.org/abs/2510.02091
- [57]Alessandro Stolfo, Yonatan Belinkov, and Mrinmaya Sachan. 2023. A mechanistic interpretation of arithmetic reasoning in language models using causal mediation analysis. In The 2023 Conference on Empirical Methods in Natural Language Processing.
- [58]Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2019. CommonsenseQA: A question answering challenge targeting commonsense knowledge. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4149–4158, Minneapolis, Minnesota. Association for Computational Linguistics.arxiv.org/abs/1811.00937
- [59]Miles Turpin, Julian Michael, Ethan Perez, and Samuel R. Bowman. 2023. Language models don’t always say what they think: Unfaithful explanations in chain-of-thought prompting. In Thirty-seventh Conference on Neural Information Processing Systems.arxiv.org/abs/2305.04388
- [60]Yang Wang. 2025. Music trivia. https://huggingface.co/datasets/extraordinarylab/music-trivia.
- [61]Yang Wang, Chenchao Xiao, Chia-Yi Hsiao, Zi Yan Chang, Chi-Li Chen, Tyler Loakman, and Chenghua Lin. 2025a. Drivel-ology: Challenging LLMs with interpreting nonsense with depth. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 23074–23096, Suzhou, China. Association for Computational Linguistics.arxiv.org/abs/2509.03867
- [62]Yang Wang, Chenghao Xiao, Yizhi Li, Stuart E. Middleton, Noura Al Moubayed, and Chenghua Lin. 2025b. Adversarial defense without adversarial defense: Enhancing language model robustness via instance-level principal component removal. Transactions of the Association for Computational Linguistics, 13:1381–1409.DOI
- [63]Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Aluraj, Xuan He, Ziyan Jiang, Tianle Li, Max Ku, Kai Wang, Alex Zhuang, Rongqi Fan, Xiang Yue, and Wenhu Chen. 2024a. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark. Advances in Neural Information Processing Systems, 37:95266–95290.
- [64]Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, Tianle Li, Max Ku, Kai Wang, Alex Zhuang, Rongqi Fan, Xiang Yue, and Wenhu Chen. 2024b. Mmlu-pro: a more robust and challenging multi-task language understanding benchmark. In Proceedings of the 38th International Conference on Neural Information Processing Systems, NIPS ’24, Red Hook, NY, USA. Curran Associates Inc.arxiv.org/abs/2406.01574
- [65]Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed H. Chi, Quoc V Le, and Denny Zhou. 2022. Chain of thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems.arxiv.org/abs/2201.11903
- [66]Johannes Welbl, Nelson F. Liu, and Matt Gardner. 2017. Crowdsourcing multiple choice science questions. In Proceedings of the 3rd Workshop on Noisy User-generated Text, pages 94–106, Copenhagen, Denmark. Association for Computational Linguistics.arxiv.org/abs/1707.06209
- [67]Frank Wilcoxon. 1945. Individual comparisons by ranking methods. Biometrika bulletin, 1(6):80–83.
- [68]Juncheng Wu, Sheng Liu, Haoqin Tu, Hang Yu, Xiaoke Huang, James Y. Zou, Cihang Xie, and Yuyin Zhou. 2025. Knowledge or reasoning? a close look at how llms think across domains. CoRR, abs/2506.02126.arxiv.org/abs/2506.02126
- [69]Chenghao Xiao, Hou Pong Chan, Hao Zhang, Mahani Aljunied, Lidong Bing, Noura Al Moubayed, and Yu Rong. 2025. Analyzing LLMs’ knowledge boundary cognition across languages through the lens of internal representations. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 24099–24115, Vienna, Austria. Association for Computational Linguistics.arxiv.org/abs/2504.13816
- [70]Mutian Yang, Jiandong Gao, and Ji Wu. 2025a. Decoupling knowledge and reasoning in llms: An exploration using cognitive dual-system theory. arXiv preprint arXiv:2507.18178.arxiv.org/abs/2507.18178
- [71]Sohee Yang, Nora Kassner, Elena Gribovskaya, Sebastian Riedel, and Mor Geva. 2025b. Do large language models perform latent multi-hop reasoning without exploiting shortcuts? In Findings of the Association for Computational Linguistics: ACL 2025, pages 3971–3992, Vienna, Austria. Association for Computational Linguistics.
- [72]Zhangyue Yin, Qiushi Sun, Qipeng Guo, Jiawen Wu, Xipeng Qiu, and Xuanjing Huang. 2023. Do large language models know what they don’t know? In Findings of the Association for Computational Linguistics: ACL 2023, pages 8653–8665, Toronto, Canada. Association for Computational Linguistics.arxiv.org/abs/2305.18153
- [73]Duzhen Zhang, Zhong-Zhi Li, Ming-Liang Zhang, Jiaxin Zhang, Zengyan Liu, Yuxuan Yao, Haotian Xu, Junhao Zheng, Xiuyi Chen, Yingying Zhang, Fei Yin, Jiahua Dong, Zhijiang Guo, Le Song, and Cheng-Lin Liu. 2026. From system 1 to system 2: A survey of reasoning in large language models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 48(3):3335–3354.arxiv.org/abs/2502.17419
- [74]Yang Zhang, Yanfei Dong, and Kenji Kawaguchi. 2024. Investigating layer importance in large language models. In The 7th BlackboxNLP Workshop.arxiv.org/abs/2409.14381
- [75]Bingchen Zhao, Richard Wheeler, Kaniuar Bacho, {Eduardo M.} Ponti, Brian Rabern, and Ondrej Bohdal. 2026. A benchmark of expert-level academic questions to assess ai capabilities. Nature, 649(8099):1139–1146. The research is supported by Center for AI Safety and Scale AI. _Contributions_ All authors have contributed to the dataset creation process. The Center for AI Safety and Scale AI consortia jointly designed the dataset premise and pipeline; operated the data collection platform (https://lastexam.ai); and provided funding, inference infrastructure for LLMs and review/auditing resources. The authors in the HLE Contributors Consortium contributed to the dataset in various ways, including submitting at least one accepted question to one of the dataset versions, contributing to dataset refinement or assisting with evaluations. In the Center for AI Safety and Scale AI, Long Phan, Alice Gatti, Ziwen Han and Nathaniel Li led the project, and Summer Yue, Haiyan Zhao, Fan Yang, Bo Shen, Ali Payani, Himabindu Lakkaraju, and Mengnan Du. 2025. Towards uncovering how large language models work: An interpretability perspective. ACM SIGKDD Explorations Newsletter, 27(2):79–87.
- [76]Yiran Zhao, Wenxuan Zhang, Guizhen Chen, Kenji Kawaguchi, and Lidong Bing. 2024. How do large language models handle multilingualism? In The Thirty-eighth Annual Conference on Neural Information Processing Systems.arxiv.org/abs/2402.18815
- [77]Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun, Jeff Huang, Cody Hao Xu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E. Gonzalez, Clark Barrett, and Ying Sheng. 2024. SGLang: Efficient execution of structured language model programs. In The Thirty-eighth Annual Conference on Neural Information Processing Systems.arxiv.org/abs/2312.07104
- [78]Rui-Jie Zhu, Tianhao Peng, Tianhao Cheng, Xingwei Qu, Jinfa Huang, Dawei Zhu, Hao Wang, Kaiwen Xue, Xuanliang Zhang, Yong Shan, Tianle Cai, Taylor Kergan, Assel Kembay, Andrew Smith, Chenghua Lin, Binh Nguyen, Yuqi Pan, Yuhong Chou, Zefan Cai, Zhenhe Wu, Yongchi Zhao, Tianyu Liu, Jian Yang, Wangchunshu Zhou, Chujie Zheng, Chongxuan Li, Yuyin Zhou, Zhoujun Li, Zhaoxiang Zhang, Jiaheng Liu, Ge Zhang, Wenhao Huang, and Jason Eshraghian. 2025. A survey on latent reasoning.arxiv.org/abs/2507.06203










