betaFeedback

WorksPhysics and Astronomy

LLMs Can Tackle High Energy Theoretical Physics, yet Gaps Remain: Part I, Quantum Field Theory

Abstract

Beyond math and code, Large language models (LLMs) are beginning to tackle problems that have traditionally required substantial training for theoretical physicists, but it remains unclear how far this capability extends and where it might break. As a first step toward pushing LLMs in high-energy theoretical physics, we introduce QFT-Bench, a benchmark of 75 graduate- to research-level problems requiring extensive reasoning and derivation across core Quantum Field Theory (QFT) subareas. Each question has a well-defined target and closed-form reference answer. Evaluation combines symbolic verification with LLM-based judge. Across 39 proprietary and open-source models, frontier models already demonstrate substantial capabilities on nontrivial QFT calculations, with the strongest models exceeding 95% accuracy. At the same time, clear gaps remain. Even strong models still make errors in calculations, conventions, and instruction following, while weaker models often fail at the level of physical concepts or derivations. We further monitor models' CoTs and show that current models exhibit nontrivial reasoning unfaithfulness, i.e., they could reach correct answers but the reasoning itself does not justify and logically support them. These results provide a quantitative assessment of current LLM capabilities in QFT and a first step toward broader evaluation in high-energy theoretical physics.

Resources: 🤗 Data 🌐 Page

LLM Scaling Law on QFT-Bench

Figure 1. LLM Scaling Law on QFT-Bench.

Introduction

Large language models (LLMs) have advanced rapidly in mathematical reasoning: moving from solving Olympiad-level problems to making substantive contributions to open mathematical questions [3 b,c] (OpenAI 2026a) within a remarkably short period. One plausible reason behind this progress is that mathematics, as well as domains such as code, often comes with cheap, mechanical and binary verification, providing unusually reliable signals for model training. For example, formal systems such as Lean [18] break down proofs into smaller and verifiable components, enabling models to learn through iterative exploration and refinement. This helps explain why LLMs remain substantially less capable on discovery-oriented natural science such as theoretical physics, where (1). problems are less formally specified; (2). breakthroughs require fundamentally new ways of understanding nature; (3). the validity of a theoretical result may depend on physical plausibility, consistency with established theory, and ultimately the test of experimental evidence.

Quantum field theory (QFT) makes an especially demanding test case within this gap. It is the mathematical language of the Standard Model and the core of much of modern theoretical physics [54, 67]. The reasoning itself is hard: QFT calculations often involve long, multi-step derivations in which a small error can change the result, and they require more than correct algebra, since one must identify the physical assumptions and approximations that apply. Moreover, QFT is underrepresented in training data: compared with mathematics and code, it offers much fewer verifiable training samples, and the available text is highly technical and fragmented across subfields.

We introduce QFT-Bench, a set of 75 hard-graduate- to easy-research-level quantum field theory problems spanning eight subareas, from classical field theory, quantum electrodynamics and chromodynamics to supersymmetric and conformal field theory. We manually construct each problem to have a closed-form reference answer, allowing correctness to be determined by checking symbolic equivalence using SymPy. We iteratively modify and reformulate problems to minimize cases where a physically or mathematically alternative answer fails to match the ground truth due to ambiguity. Nevertheless, some problems still admit equivalent answers hard to symbolically verify, such as different conventions and gauge choices. Therefore, when symbolic verification is inconclusive, we fall back to an LLM judge (Claude Opus-5).

We evaluate 22 representative frontier proprietary reasoning models, including Opus-5.5, GPT-6.1 and Gemini-3.8, and 17 open-source ones of families such as Kimi, DeepSeek, Llama, GLM, and Qwen, each with 64k max tokens. The strongest proprietary LLMs solve a substantial fraction of problems, yet still make non-trivial errors. In contrast, proprietary models released only a few months earlier, as well as recent open-source models, exhibit a performance gap. This gap widens further for smaller open-source models, e.g., Llama-4-109B and Qwen-8B/27B. Beyond final-answer accuracy, we also monitor models’ CoTs, and examine reasoning faithfulness, defined by whether a model’s CoT derivations justify and logically support its final answer. We find that current models exhibit non-trivial unfaithfulness in theoretical physics reasoning, and that faithfulness scales (almost) linearly with model capability. Together, these results expose limitations of current reasoning models, and our benchmark fills a gap in tracking progress in theoretical physics reasoning.

Our contributions are summarized:

  • We present QFT-Bench, to our knowledge the first systematic benchmark of LLM reasoning in QFT, containing 75 hard graduate- to easy research-level problems across eight subareas. Each question has a closed-form reference answer, evaluated with a symbolic grader and an LLM judge.

  • We perform a comprehensive evaluation of 39 proprietary and open-source reasoning models, with an error analysis identifying the failure mode.

  • We go beyond final-answer accuracy with a faithfulness analysis, showing that unfaithfulness persists

even in strong models and decreases with capability.

LLM Reasoning for Theoretical Physics. Frontier LLMs have begun to contribute to theoretical physics research. Case studies with GPT-5 [13] show that the model can independently derive intermediate steps underlying results in published theoretical physics papers. Similarly, recent work has explored using LLMs to accelerate the theoretical physics research by conjecturing formulas and generating hypotheses [1, 29], or by speeding up calculations [28, 33, 34, 64]. To improve such capabilities, previous papers either fine-tunes language models for specialized tasks in high-energy physics [61, 69], or complementarily trains transformer models from scratch specialized for symbolic QFT calculations [2, 14, 16]. Yet we find that current general-purpose frontier language models themselves can already solve challenging QFT problems.

Existing Theoretical Physics Benchmarks. Reliable evaluation is essential for measuring progress in AI reasoning. Broad expert-level benchmarks such as GPQA-Diamond [60], Humanity’s Last Exam [55], and FrontierScience [65] include challenging physics problems alongside other scientific disciplines, but consequently provide limited coverage of individual subfield such as QFT. Physics-specific benchmarks such as PHYSICS [21] and UGPhysics [71] primarily focus on undergraduate topics. More specialized benchmarks extend evaluation to selected high energy theory and research-level problems [17, 74]. Despite this progress, systematic evaluation of QFT reasoning itself remains relatively limited, motivating a benchmark that provides broad coverage of core subareas of QFT while providing both verifiable answers and reliable CoT solutions.

Benchmarks# of HEP questionsLevelPhysics-specializedHEP-specializedEvaluation
GPQA-diamond [60]-undergraduate, graduatenonorule
HLE [55]-graduate, researchnonoLLM
SciBench [66]0undergraduatenonorule
PHYSICS [21]0undergraduateyesnorule + LLM
TPBench [17]18undergraduate, graduate, researchyesnorule + LLM
CritPt [74]10researchyesnorule
QFT-Bench (ours)75hard graduate, researchyesyesrule + LLM

Table 1. Comparison of representative benchmarks across High-Energy Physics (HEP) scale, difficulty level, and physics specialization.

QFT-Bench

Overview and statistics

QFT-Bench comprises 75 problems, ranging from graduate-level material to early research topics. The problems are grouped into the eight topic categories shown in Table 2. For problems spanning multiple areas of QFT, we assign each to the category that best reflects its main focus.

AreasProblemsPercentage
Classical Field Theory (Classical)79.3%
Quantization of Scalar, Spinor and Vector Fields (Quant.)810.7%
Quantum Electrodynamics (QED)79.3%
Regularization and Renormalization (Renorm.)68.0%
Non-Abelian Gauge Theory and QCD (Non-Ab.)68.0%
Electroweak and Higgs Mechanism (EW.)810.7%
Supersymmetric Field Theories (SUSY)1824.0%
Conformal Field Theories (CFT)1520.0%

Table 2. Distribution of questions in QFT-Bench across eight topic categories.

Standard graduate courses in QFT typically focus on field quantization, perturbation theory, renormalization, and gauge theories [54, 62]. Supersymmetry and conformal field theory are often covered separately in more advanced or specialized treatments [20, 68]. QFT-Bench also places emphasis on both topics, with around 20% questions on both supersymmetric field theory and conformal field theory. The remaining questions cover six core areas of QFT.

Dataset Construction

Design and problem construction. QFT-Bench is designed to test whether models can solve QFT problems rather than simply recall known formulas. We use standard QFT textbooks and publicly available lecture notes to guide our selection of core graduate-level topics, but finally adapt problems so that answers cannot be easily retrieved from these sources.

Development. We develop the problems in two ways. First, we design original problems around the selected topics: for each, we identify the central physical concept, the required calculation or derivation, and the reference answer before formulating the problem statement. Second, we substantially adapt existing problems that rely on context from particular course materials or textbook chapters. The original versions often omit necessary assumptions, leave notation or conventions implicit, or do not clearly specify the target quantity. We rewrite them as self-contained problems while preserving the core physics they are intended to test.

To be included in QFT-Bench, a problem must test a core QFT concept or method, require substantive reasoning or calculation, and have a closed-form verifiable reference answer. These requirements enable the two-stage verification procedure described in § 3.3.

Iterative refinement. After assembling the initial problem set, we run preliminary evaluations with Sonnet-5 and manually reviewed their responses. This reveals ambiguities in some problems, such as missing assumptions, undefined notation, unspecified conventions, or an unclear target quantity. We resolve each accordingly and, where necessary, specify the form of the answer.

For example, in the Seiberg–Witten monodromy problem, we specify the analytic continuation path, the contour orientation, and the ordering of period basis on which the monodromy matrix acts. In the free-fermion CFT problem, we state the Neveu–Schwarz sector, oscillator normalization, and radial-ordering convention.

Reference CoT and quality control. For each problem, we also prepare a reference CoT solution. We check that the solution follows from the stated assumption; notation and conventions are consistent throughout.

Verification

To reliably evaluate model-generated solutions, we employ a two-stage verification pipeline. First, we extract the final answer from each model response and use a SymPy-based ([38]) grader to check its symbolic equivalence to the reference answer. If symbolic verification is inconclusive or fails, we fall back to an LLM-as-a-judge using Opus 5, which compares the model’s solution against the reference answer and the corresponding step-by-step solution. See Appx. A for the prompt. The judge assigns a binary correctness score, determining whether the model’s final answer is mathematically and physically equivalent to the reference answer.

Experimental Setup

Language Models and API Setup

We select 39 representative state-of-the-art LLMs, including 22 proprietary closed-source models and 17 open-source ones. Proprietary LLMs include:

  • Claude family: Opus-5.5 [9], Opus-5 [8], Opus-4.6 [7], Sonnet-5.5 [11], Sonnet-5 [10], and Haiku-4.5 [5].

  • GPT family: GPT-6.1-Sol [48], GPT-6-Sol [49], GPT-5.6-Sol/Terra/Luna [47], GPT-5.2 [46], GPT-5 [45], GPT-4.1 [44], and GPT-4o [43].

  • Gemini family: Gemini-3.8-Flash [26], Gemini-3.6-Flash [25], Gemini-3-Flash-Preview [23], Gemini-3.5-Flash-Lite [25], and Gemini-3.1-Flash-Lite [24].

  • Muse family: Muse-Spark-1.3 [37] and Muse-Spark-1.1 [36].

Open-source LLMs include:

  • Kimi family: Kimi-K3 [40], and Kimi-K2.5 [39].

  • DeepSeek family: DeepSeek-v4.1-Flash [70], DeepSeek-v3.1 [19] and DeepSeek-v3 [32].

  • Llama family: Llama-4-Maverick [35], Llama-4-Scout [35], Llama-3.1-Instruct and Llama-3.3-Instruct [27].

  • GLM family: GLM-5.3 [73], and GLM-5 [72].

  • Qwen family: Qwen-3.8 (Max (2.4T-A95B) and 27B) [58], Qwen-3.5 (397B-A17B and 35B-A3B) [57], Qwen-3 (235B-A22B-Thinking-2507 and 8B) [56].

For Claude, GPT and Gemini, we use the official API from Anthropic, OpenAI and Google platforms (Anthropic; Google; OpenAI). For the other models, we use OpenRouter platform (OpenRouter). We set max_tokens to 65,536 for all inference runs and leave all other parameters (e.g., temperature, top-p, and top-k) at their default values. For reproducibility, we provide exact API names and release dates in Table 3.

ModelsAPI NamesRelease Date
ClaudeOpus 5.5claude-opus-5-52026-09
ClaudeOpus 5claude-opus-52026-07
ClaudeOpus 4.6claude-opus-4-62026-02
ClaudeSonnet 5.5claude-sonnet-5-52026-09
ClaudeSonnet 5claude-sonnet-52026-06
ClaudeHaiku 4.5claude-haiku-4-52025-10
GPTGPT-6.1-Solgpt-6.1-sol2026-09
GPTGPT-6-Solgpt-6-sol2026-09
GPTGPT-5.6-Solgpt-5.6-sol2026-07
GPTGPT-5.6-Terragpt-5.6-terra2026-07
GPTGPT-5.6-Lunagpt-5.6-luna2026-07
GPTGPT-5.2gpt-5.22025-12
GPTGPT-5gpt-52025-08
GPTGPT-4.1gpt-4.12025-04
GPTGPT-4ogpt-4o2024-05
GeminiGemini-3.8-Flashgemini-3.8-flash2026-09
GeminiGemini-3.6-Flashgemini-3.6-flash2026-07
GeminiGemini-3-Flash-Previewgemini-3-flash-preview2025-12
GeminiGemini-3.5-Flash-Litegemini-3.5-flash-lite2026-07
GeminiGemini-3.1-Flash-Litegemini-3.1-flash-lite2026-03
MuseMuse-Spark-1.3meta/muse-spark-1.32026-09
MuseMuse-Spark-1.1meta/muse-spark-1.12026-07
KimiKimi-K3-104B-2.8Tmoonshotai/kimi-k32026-07
KimiKimi-K2.5-32B-1Tmoonshotai/kimi-k2.52026-01
DeepseekDeepSeek-v4.1-Flash-16B-552Bdeepseek/deepseek-v4.1-flash2026-09
DeepseekDeepseek-v3.1-37B-671Bdeepseek/deepseek-chat-v3.12025-08
DeepseekDeepseek-v3-37B-671Bdeepseek/deepseek-chat2024-12
LlamaLlama-4-Maverick-17B-400Bmeta-llama/llama-4-maverick2025-04
LlamaLlama-4-Scout-17B-109Bmeta-llama/llama-4-scout2025-04
LlamaLlama-3.3-Instruct-70Bmeta-llama/llama-3.3-70b-instruct2024-12
LlamaLlama-3.1-Instruct-70Bmeta-llama/llama-3.1-70b-instruct2024-07
GLMGLM-5.3-40B-744Bz-ai/glm-5.32026-08
GLMGLM-5-40B-744Bz-ai/glm-52026-02
QwenQwen3.8-2.4T-A95Bqwen/qwen3.8-2.4t-a95b2026-08
QwenQwen3.8-27Bqwen/qwen3.8-27b2026-08
QwenQwen3.5-397B-A17Bqwen/qwen3.5-397b-a17b2026-02
QwenQwen3.5-35B-A3Bqwen/qwen3.5-35b-a3b2026-02
QwenQwen3-235B-A22B-Thinking-2507qwen/qwen3-235b-a22b-thinking-25072025-07
QwenQwen3-8Bqwen/qwen3-8b2025-04

Table 3. Evaluated models, their API names, and release dates.

Metric and Verification

We report average accuracy across all problems as our primary metric, since we observe high difference across models. Future work may extend to accuracy@k. Each problem is assigned a binary score, with 1 indicating the correct answer and 0 for incorrect. All models are evaluated with thinking mode enabled when available.

Results

Overall performance

We present main results in Table 4. First, the strongest frontier models demonstrate substantial proficiency in QFT calculations, with twelve models achieving over 95% accuracy. This finding is consistent with broader advances in applying LLMs to QFT calculations, with a recent process computing an exceptionally complex nine-loop scattering amplitude in planar N=4\mathcal{N}=4 super Yang–Mills theory (Anthropic, 2026). Second, within the same model family, newer generation models generally outperform their predecessors. Finally, proprietary models in general perform better than open-sourced ones.

SymbolicClas.EW.Non-Ab.QEDQuant.Renorm.SUSYCFTAvg.
Closed-source proprietary LLMs
Opus-5.572.0100.0100.0100.085.7100.0100.0100.0100.098.7
Sonnet-5.574.7100.0100.0100.0100.0100.083.3100.093.397.3
AnthropicOpus-564.0100.0100.0100.085.7100.083.3100.0100.097.3
Opus-4.673.385.775.0100.085.775.066.788.993.385.3
Sonnet-566.785.787.583.385.787.566.772.293.382.7
Haiku-4.538.757.162.550.042.962.550.050.040.050.7
GPT-6.1-Sol68.0100.0100.0100.0100.0100.0100.0100.0100.0100.0
GPT-6-Sol66.7100.0100.0100.085.7100.083.3100.0100.097.3
GPT-5.6-Sol68.0100.0100.0100.0100.0100.083.3100.0100.098.7
GPT-5.6-Terra61.3100.087.5100.085.7100.083.3100.0100.096.0
OpenaiGPT-5.6-Luna61.3100.0100.0100.085.7100.066.7100.093.394.7
GPT-5.256.071.475.066.771.4100.066.777.886.778.7
GPT-562.785.787.5100.0100.087.566.794.4100.092.0
GPT-4.150.742.987.583.371.475.066.772.273.372.0
GPT-4o44.071.475.066.742.962.533.338.973.357.3
Gemini-3.8-Flash85.3100.0100.0100.0100.0100.083.3100.0100.098.7
GeminiGemini-3.6-Flash81.385.7100.0100.0100.087.583.3100.0100.096.0
Gemini-3-Flash-Preview80.0100.0100.0100.0100.0100.083.3100.0100.098.7
Gemini-3.5-Flash-Lite65.385.7100.066.771.487.566.777.880.080.0
Gemini-3.1-Flash-Lite56.071.475.050.057.187.550.055.673.365.3
MuseMuse-Spark-1.381.3100.0100.0100.085.7100.0100.0100.0100.098.7
Muse-Spark-1.170.785.787.583.371.487.583.394.4100.089.3
Open-source LLMs
KimiKimi-K3-104B-2.8T64.0100.087.5100.0100.0100.066.794.4100.094.7
Kimi-K2.5-32B-1T68.085.7100.083.3100.087.550.088.986.786.7
DeepseekDeepSeek-v4.1-Flash-16B-552B64.085.787.5100.0100.0100.083.383.3100.092.0
DeepSeek-v3.1-37B-671B64.085.750.050.085.775.066.788.986.777.3
DeepSeek-v3-37B-671B45.357.150.050.057.175.033.333.373.353.3
LlamaLlama-4-Maverick-17B-400B64.071.487.583.357.162.566.772.286.774.7
Llama-4-Scout-17B-109B33.342.962.516.757.150.033.333.340.041.3
Llama-3.3-Instruct-70B42.742.962.550.042.950.033.350.053.349.3
Llama-3.1-Instruct-70B34.728.650.033.342.962.533.338.940.041.3
GLMGLM-5.3-40B-744B58.7100.087.583.385.7100.033.383.386.784.0
GLM-5-40B-744B68.085.787.583.3100.087.550.077.8100.085.3
QwenQwen-3.8-2.4T-A95B66.7100.0100.083.385.7100.083.3100.0100.096.0
Qwen-3.8-27B53.385.787.566.785.787.550.066.7100.080.0
Qwen-3.5-397B-A17B77.3100.0100.0100.0100.0100.066.7100.073.392.0
Qwen3.5-35B-A3B62.771.4100.066.785.775.066.777.893.381.3
Qwen3-235B-A22B-Thinking-250762.785.787.566.785.775.066.766.7100.080.0
Qwen3-8B41.371.425.033.371.462.516.744.446.746.7

Table 4. Table 4 Accuracy (%) on QFT-Bench. “Symbolic” reports symbolic verification scores. “Avg.” report final accuracy. LLMs are evaluated with 64k max tokens.

Error analysis

To understand where the models fail, we examine their response and classify each incorrect response to one of four categories: conceptual, calculations, strong-prior, or instruction-following errors. Table 5 in Appx. C shows how these failures are distributed across models.

Failure categoriesTop halfOthers
Conceptual error23.4% ± 2.0%47.2% ± 0.9%
Calculation error21.6% ± 0.0%18.6% ± 0.7%
Strong-prior error23.4% ± 0.8%13.4% ± 0.9%
Instruction following31.5% ± 2.0%20.8% ± 0.4%

Table 5. Distribution of failure categories (% of failures within each group). Top half denotes the half highest-scoring models in Table 4; Others denotes the remaining models.

Failure mode changes with model performance. Conceptual errors dominate among weaker models but become uncommon among the stronger ones. The stronger models usually apply the underlying physics correctly, yet can still fail for reasons such as involving incorrect calculations, substitute a familiar textbook result for the actual one in the question, or answer a different quantity from requested. We examine an example of each category below.

Case study 1: conceptual errors. These responses apply the wrong physics.

Question: Consider a theory in dd spacetime dimensions with a Dirac fermion ψ\psi coupled to a massive vector field AμA_\mu through the interaction term −eAμψˉγμψ-eA_\mu\bar{\psi}\gamma^\mu\psi. At large momentum, the massive vector propagator scales as O(1)\mathcal{O}(1). Derive the superficial degree of divergence DD for a connected Feynman diagram with EeE_e external fermion lines, EAE_A external AμA_\mu lines, and VV vertices.

GPT-5.6-Luna:

… (model’s reasoning) …

For a theory with the vertex AμψˉγμψA_\mu\bar{\psi}\gamma^\mu\psi,

2Ie+Ee=2V,IA+EA=V,2I_e + E_e = 2V, \qquad I_A + E_A = V,

… (model’s reasoning) …

D=d+(d−1)V−dEA−d−12EeD = d + (d-1)V - dE_A - \frac{d-1}{2}E_e

(wrong)

The correct relation for the vector field should be 2IA+EA=V2I_A+E_A=V, since each vertex contributes one vector leg, while each internal vector line accounts for two, whereas GPT-5.6-Luna instead uses IA+EA=VI_A+E_A=V, counting each internal vector line only once. This mistake makes its inferred value of IAI_A twice as large and leads to the incorrect final result.

Case study 2: calculation errors. These responses use the appropriate method but make an error in carrying it out, such as losing a factor or a sign, or failing to account for field normalization.

Question: Consider the Abelian Higgs Lagrangian L[Aμ,H]=−14FμνFμν+(DμH)†DμH−λ22(H†H−v2)2\mathcal{L}[A_\mu,H]=-\frac{1}{4}F_{\mu\nu}F^{\mu\nu}+(D_\mu H)^\dagger D^\mu H-\frac{\lambda^2}{2}(H^\dagger H-v^2)^2, where Dμ=∂μ−igAμD_\mu=\partial_\mu-igA_\mu, Fμν=∂μAν−∂νAμF_{\mu\nu}=\partial_\mu A_\nu-\partial_\nu A_\mu and H=ρ(x)eiθ(x)/vH=\rho(x)e^{i\theta(x)/v}. Assume that the Higgs is heavier than the vector, that is mA≲E≪mhm_A\lesssim E\ll m_h. One can integrate the former out, and in this limit one obtains the Proca Lagrangian LProca=−14F~μνF~μν+mA22A~μA~μ+ξ24(A~μA~μ)2\mathcal{L}_{\mathrm{Proca}}=-\frac{1}{4}\widetilde{F}_{\mu\nu}\widetilde{F}^{\mu\nu}+\frac{m_A^2}{2}\widetilde{A}_\mu\widetilde{A}^\mu+\frac{\xi^2}{4}(\widetilde{A}_\mu\widetilde{A}^\mu)^2. Identify the coupling ξ2\xi^2 in terms of the massive mode mAm_A, the Higgs mass mhm_h, and the coupling constant.

GPT-5.6-Terra:

⋯\cdots (model’s reasoning) ⋯\cdots

H=(v+h)eiθ/v,A~μ=Aμ−1gv∂μθ.H=(v+h)e^{i\theta/v},\qquad\widetilde{A}_\mu=A_\mu-\frac{1}{gv}\partial_\mu\theta.

Expanding about the vacuum gives

mA2=2g2v2,mh2=4λ2v2.m_A^2=2g^2v^2,\qquad m_h^2=4\lambda^2v^2.

⋯\cdots (model’s reasoning) ⋯\cdots

ξ2=2gmAmh\xi^2=\frac{2gm_A}{m_h}

(wrong)

References

  1. [1]Prateek Agrawal, Nathaniel Craig, Amalia Madden, and Iñigo Valenzuela Lombera. The fermiacc: Agents for particle theory. arXiv preprint arXiv:2603.22538, 2026.arxiv.org/abs/2603.22538
  2. [2]Abdulhakim Alnuqaydan, Sergei Gleyzer, and Harrison Prosper. Symba: symbolic computation of squared amplitudes in high energy physics with machine learning. Machine Learning: Science and Technology, 4(1):015007, 2023.
  3. [3]L Alpöge and R Furman. More than two thirds of the zeros of the riemann zeta function are simple and on the critical line. arXiv preprint arXiv:2608.13637, 2026.
  4. [5]Anthropic. Introducing claude haiku 4.5, 2025. URL https://www.anthropic.com/news/claude-haiku-4-5.
  5. [7]Anthropic. Introducing claude opus 4.6, 2026a. URL https://www.anthropic.com/news/claude-opus-4-6.
  6. [8]Anthropic. Introducing claude opus 5, 2026b. URL https://www.anthropic.com/news/claude-opus-5.
  7. [9]Anthropic. Introducing claude opus 5.5, 2026c. URL https://www.anthropic.com/claude-opus-5-5.
  8. [10]Anthropic. Introducing claude sonnet 5, 2026d. URL https://www.anthropic.com/news/claude-sonnet-5.
  9. [11]Anthropic. Introducing claude sonnet 5.5, 2026e. URL https://www.anthropic.com/claude-sonnet-5-5.
  10. [12]Bowen Baker, Joost Huizinga, Leo Gao, Zehao Dou, Melody Y Guan, Aleksander Madry, Wojciech Zaremba, Jakub Pachocki, and David Farhi. Monitoring reasoning models for misbehavior and the risks of promoting obfuscation. arXiv preprint arXiv:2503.11926, 2025.arxiv.org/abs/2503.11926
  11. [13]Sébastien Bubeck, Christian Coester, Ronen Eldan, Timothy Gowers, Yin Tat Lee, Alexandru Lupascu, Mehtaab Sawhney, Robert Scherrer, Mark Sellke, Brian K Spears, et al. Early science acceleration experiments with gpt-5. arXiv preprint arXiv:2511.16072, 2025.arxiv.org/abs/2511.16072
  12. [14]Tianji Cai, Garrett W Merz, François Charton, Niklas Nolte, Matthias Wilhelm, Kyle Cranmer, and Lance J Dixon. Transforming the bootstrap: using transformers to compute scattering amplitudes in planar n= 4 super yang–mills theory. Machine Learning: Science and Technology, 5(3):035073, 2024.DOI
  13. [15]Yanda Chen, Joe Benton, Ansh Radhakrishnan, Jonathan Uesato, Carson Denison, John Schulman, Arushi Somani, Peter Hase, Misha Wagner, Fabien Roger, et al. Reasoning models don’t always say what they think. arXiv preprint arXiv:2505.05410, 2025.arxiv.org/abs/2505.05410
  14. [16]Clifford Cheung, Aurélien Dersy, and Matthew D Schwartz. Learning the simplicity of scattering amplitudes. arXiv preprint arXiv:2408.04720, 2024.DOI
  15. [17]Daniel JH Chung, Zhiqi Gao, Yurii Kvasik, Tianyi Li, Moritz Münchmeyer, Maja Rudolph, Frederic Sala, and Sai Chaitanya Tadapelli. Theoretical physics benchmark (tpbench)—a dataset and study of ai reasoning capabilities in theoretical physics. Machine Learning: Science and Technology, 6(3):030505, 2025.arxiv.org/abs/2502.15815
  16. [18]Leonardo De Moura, Soonho Kong, Jeremy Avigad, Floris Van Doorn, and Jakob von Raumer. The lean theorem prover (system description). In International conference on automated deduction, pages 378–388. Springer, 2015.DOI
  17. [19]DeepSeek-AI. Deepseek-v3.1 release, 2025. URL https://www.deepseek.com/en/news/deepseek-v3-1/.
  18. [20]Philippe Di Francesco, Pierre Mathieu, and David Sénéchal. Conformal Field Theory. Graduate Texts in Contemporary Physics. Springer, New York, 1997. doi: 10.1007/978-1-4612-2256-9.DOI
  19. [21]Kaiyue Feng, Yilun Zhao, Yixin Liu, Tianyu Yang, Chen Zhao, John Sous, and Arman Cohan. Physics: Benchmarking foundation models on university-level physics problem solving. In Findings of the Association for Computational Linguistics: ACL 2025, pages 11717–11743, 2025.arxiv.org/abs/2503.21821
  20. [23]Google. Gemini 3 flash: frontier intelligence built for speed, 2025. URL https://blog.google/products-and-platforms/products/gemini/gemini-3-flash/.
  21. [24]Google. Gemini 3.1 flash-lite: Built for intelligence at scale, 2026a. URL https://blog.google/innovation-and-ai/models-and-research/gemini-models/gemini-3-1-flash-lite/.
  22. [25]Google. Introducing gemini 3.6 flash, 3.5 flash-lite, and 3.5 flash cyber, 2026b. URL https://blog.google/innovation-and-ai/models-and-research/gemini-models/gemini-3-6-flash-3-5-flash-lite-3-5-flash-cyber/.
  23. [26]Google. Introducing gemini 3.8 flash and 3.8 flash cyber, 2026c. URL https://blog.google/innovation-and-ai/models-and-research/gemini-models/3-8-flash-and-3-8-flash-cyber/.
  24. [27]Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024.arxiv.org/abs/2407.21783
  25. [28]Yi Gu and Sven Krippendorf. Scattering amplitudes as programs: Self-evolving search for theory and event generation. arXiv preprint arXiv:2607.21629, 2026.arxiv.org/abs/2607.21629
  26. [29]Alfredo Guevara, Alexandru Lupascca, David Skinner, Andrew Strominger, and Kevin Weil. Single-minus gluon tree amplitudes are nonzero. arXiv preprint arXiv:2602.12176, 2026.arxiv.org/abs/2602.12176
  27. [30]Alon Jacovi and Yoav Goldberg. Towards faithfully interpretable nlp systems: How should we define and evaluate faithfulness? In Proceedings of the 58th annual meeting of the association for computational linguistics, pages 4198–4205, 2020.arxiv.org/abs/2004.03685
  28. [31]Tamera Lanham, Anna Chen, Ansh Radhakrishnan, Benoit Steiner, Carson Denison, Danny Hernandez, Dustin Li, Esin Durmus, Evan Hubinger, Jackson Kernion, et al. Measuring faithfulness in chain-of-thought reasoning. arXiv preprint arXiv:2307.13702, 2023.arxiv.org/abs/2307.13702
  29. [32]Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024.arxiv.org/abs/2412.19437
  30. [33]Tony Menzo, Alexander Roman, Sergei Gleyzer, Konstantin Matchev, George T Fleming, Stefan Höche, Stephen Mrenna, and Prasanth Shyamsundar. Heptapod: Orchestrating high energy physics workflows towards autonomous agency. arXiv preprint arXiv:2512.15867, 2025.arxiv.org/abs/2512.15867
  31. [34]Tony Menzo, Alexander Roman, George T Fleming, Sergei Gleyzer, Konstantin T Matchev, and Stephen Mrenna. Agentic diagrammatica: Towards autonomous symbolic computation in high energy physics. arXiv preprint arXiv:2603.26990, 2026.arxiv.org/abs/2603.26990
  32. [35]Meta. The llama 4 herd: The beginning of a new era of natively multimodal ai innovation, 2026a. URL https://ai.meta.com/blog/llama-4-multimodal-intelligence/.
  33. [36]Meta. Introducing muse spark 1.1, 2026b. URL https://ai.meta.com/blog/introducing-muse-spark-meta-model-api/.
  34. [37]Meta. Introducing muse spark 1.3, 2026c. URL https://research.meta.ai/blog/introducing-muse-spark-1-3.
  35. [38]Aaron Meurer, Christopher P Smith, Mateusz Paprocki, Ondřej Čertík, Sergey B Kirpichev, Matthew Rocklin, AMiT Kumar, Sergiu Ivanov, Jason K Moore, Bartaj Singh, et al. Sympy: symbolic computing in python. PeerJ Computer Science, 3:e103, 2017.
  36. [39]MoonshotAI. Kimi k2.5: Ai that sees, codes, and works like an expert, 2026a. URL https://www.kimi.ai/ai-models/kimi-k2-5.
  37. [40]MoonshotAI. Kimi k3: Open frontier intelligence, 2026b. URL https://www.kimi.com/en/blog/kimi-k3.
  38. [41]VA Novikov, Mikhail A Shifman, AI Vainshtein, and Valentin I Zakharov. Exact gell-mann-low function of supersymmetric yang-mills theories from instanton calculus. Nuclear Physics B, 229(2):381–393, 1983.DOI
  39. [43]Openai. Introducing gpt-4o, 2024. URL https://openai.com/index/hello-gpt-4o/.
  40. [44]Openai. Introducing gpt-4.1 in the api, 2025a. URL https://openai.com/index/gpt-4-1/.
  41. [45]Openai. Introducing gpt-5, 2025b. URL https://openai.com/index/introducing-gpt-5/.
  42. [46]Openai. Introducing gpt-5.2, 2025c. URL https://openai.com/index/introducing-gpt-5-2/.
  43. [47]Openai. Introducing gpt-5.6, 2026a. URL https://openai.com/index/gpt-5-6/.
  44. [48]Openai. Introducing gpt-6.1 sol, 2026b. URL https://openai.com/index/introducing-gpt-6-1-sol/.
  45. [49]Openai. Introducing gpt-6 sol and luna, 2026c. URL https://openai.com/index/introducing-gpt-6-sol-and-luna/.
  46. [54]Michael E. Peskin and Daniel V. Schroeder. An Introduction to Quantum Field Theory. Addison-Wesley, Reading, Massachusetts, 1995.
  47. [55]Long Phan, Alice Gatti, Ziwen Han, Nathaniel Li, Josephina Hu, Hugh Zhang, Chen Bo Calvin Zhang, Mohamed Shaaban, John Ling, Sean Shi, et al. Humanity’s last exam. arXiv preprint arXiv:2501.14249, 2025.
  48. [56]Qwen Team. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025.
  49. [57]Qwen Team. Qwen3.5: Towards native multimodal agents, February 2026a. URL https://qwen.ai/blog?id=qwen3.5.
  50. [58]Qwen Team. Qwen3.8-Max: A new bar for coding and cowork, August 2026b. URL https://qwen.ai/blog?id=qwen3.8.
  51. [59]Salman Rahman, Jingyan Shen, Anna Mordovina, Hamid Palangi, Saadia Gabriel, and Pavel Izmailov. When can llms learn to reason with weak supervision? arXiv preprint arXiv:2604.18574, 2026.arxiv.org/abs/2604.18574
  52. [60]David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. Gpqa: A graduate-level google-proof q&a benchmark. arXiv preprint arXiv:2311.12022, 2023.arxiv.org/abs/2311.12022
  53. [61]Paul Richmond, Constantinos Papageorgakis, Vasilis Niarchos, Borun Chowdhury, and Prarit Agarwal. Feyntune: large language models for high-energy theory. Machine Learning: Science and Technology, 7(2):025012, 2026.DOI
  54. [62]David Tong. Lectures on quantum field theory. University of Cambridge Part III Mathematical Tripos, 2006. URL https://www.damtp.cam.ac.uk/user/tong/qft.htm.
  55. [63]Miles Turpin, Julian Michael, Ethan Perez, and Samuel Bowman. Language models don’t always say what they think: Unfaithful explanations in chain-of-thought prompting. Advances in Neural Information Processing Systems, 36:74952–74965, 2023.arxiv.org/abs/2305.04388
  56. [64]Matt von Hippel and Matthias Wilhelm. Refining integration-by-parts reduction of feynman integrals with machine learning. Journal of High Energy Physics, 2025(5):185, 2025.arxiv.org/abs/2502.05121
  57. [65]Miles Wang, Robi Lin, Kat Hu, Joy Jiao, Neil Chowdhury, Ethan Chang, and Tejal Patwardhan. Frontierscience: Evaluating ai’s ability to perform expert-level scientific tasks. arXiv preprint arXiv:2601.21165, 2026.arxiv.org/abs/2601.21165
  58. [66]Xiaoxuan Wang, Ziniu Hu, Pan Lu, Yangqiao Zhu, Jieyu Zhang, Satyen Subramaniam, Arjun R Loomba, Shichang Zhang, Yizhou Sun, and Wei Wang. Scibench: Evaluating college-level scientific problem-solving abilities of large language models. arXiv preprint arXiv:2307.10635, 2023.arxiv.org/abs/2307.10635
  59. [67]Steven Weinberg. The quantum theory of fields, volume 2. Cambridge university press, 1995.
  60. [68]Steven Weinberg. The Quantum Theory of Fields, Volume 3: Supersymmetry. Cambridge University Press, Cambridge, 2000. doi: 10.1017/CBO9781139644198.
  61. [69]Nathaniel Sherlock Woodward, Zhiqi Gao, Yurii Kvasiuk, Kendrick M Smith, Frederic Sala, and Moritz Münchmeyer. Fine-tuning small reasoning models for quantum field theory. Machine Learning: Science and Technology, 2026.DOI
  62. [70]Anyí Xu, B Li, Bangcai Lin, Bing Xue, BingCheng Xian, Bingzheng Xu, Bochao Wu, Boweì Zhang, Boyi Deng, CC Yu, et al. Deepseek-v4. 1-flash: Pushing the limits of kv cache compression. arXiv preprint arXiv:2609.19969, 2026.arxiv.org/abs/2609.19969
  63. [71]Xin Xu, Qiyun Xu, Tong Xiao, Tianhao Chen, Yuchen Yan, Jiaxin Zhang, Shizhe Diao, Can Yang, and Yang Wang. Uglyphysics: A comprehensive benchmark for undergraduate physics reasoning with large language models. arXiv preprint arXiv:2502.00334, 2025.arxiv.org/abs/2502.00334
  64. [72]Z.ai. Glm-5: From vibe coding to agentic engineering, 2026a. URL https://z.ai/blog/glm-5.
  65. [73]Z.ai. Glm-5.3: Frontier coding with emergent cyber capabilities, 2026b. URL https://z.ai/blog/glm-5.3.
  66. [74]Minhui Zhu, Minyang Tian, Xiaocheng Yang, Tianci Zhou, Lifan Yuan, Penghao Zhu, Eli Chertkov, Shengyan Liu, Yufeng Du, Ziming Ji, et al. Probing the critical point (critpt) of ai reasoning: a frontier physics research benchmark. arXiv preprint arXiv:2509.26574, 2025.arxiv.org/abs/2509.26574

Paper details

vlnc:W_33w5zpt4
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 20

downloads · 2

A view counts once per browser per day, a download each time the PDF is requested.

  • Physics and Astronomy

Not recorded for this paper

View statement
Looking for related papers…
Contents