betaFeedback

WorksComputer Science

daVinci-Prover: Agent-Native Mid-Training for Formal Theorem Proving

Abstract

Advances in large language models (LLMs) are rapidly transforming formal theorem proving. Yet further progress remains constrained by the lack of a transparent and effective data recipe for systematically improving theorem-proving models. In this work, we argue that effective agentic provers require strong mathematical skill priors to ground theorem proving and agentic trajectories to enable long-horizon, interactive proof construction. To this end, we synthesize an agent-native 33.7B-token mid-training corpus covering decomposition, search, self-verification, and revision, together with verifier-grounded agentic trajectories. We further introduce the daVinci-Prover harness, which enables models to construct proofs and interact with a Lean verifier in an isolated workspace, and use it to collect approximately 36K high-quality agent trajectories for supervised fine-tuning. Our mid-training recipe yields an average improvement of 13.7 points over the matched Qwen3-8B SFT-only baseline across MiniF2F, FATE-M, and ProofNet. When applied to the Qwen3.5 series, our approach also yields strong performance across theorem-proving benchmarks and achieves near-saturating MiniF2F performance with memory compaction. The results highlight agent-native mid-training as an effective foundation for developing stronger interactive theorem-proving agents.

§ github.com/GAIR-NLP/daVinci-prover

Introduction

Formal theorem proving turns mathematical definitions, statements, and proofs into computational objects that can be checked, reused, and extended. Proof assistants such as Lean (De Moura et al., [6]; Moura & Ullrich, [11]) provide a particularly attractive interface for language models: they combine the reasoning trajectories required by mathematics with machine-checkable feedback. Recent large language models (LLMs) have made rapid progress in formal proof generation, demonstrating increasingly strong performance across problems ranging from high-school olympiads to research-level mathematics (Chen et al., [4]; Ren et al., [14]; Lin et al., [8]).

Existing LLM-based theorem provers broadly follow three paradigms. (i) Whole-proof generation produces a complete proof in a single pass, offering a simple and efficient inference procedure (Lin et al., [8]). (ii) Agentic proving places the model in an executable environment, where it can inspect the workspace, query libraries, run Lean, interpret verifier feedback, and iteratively revise its proof (Ma et al., [10]; Wang et al., [22]). (iii) Orchestrated systems coordinate multiple reasoning or proving components within structured workflows (Chung et al., [5]). These paradigms are complementary rather than mutually exclusive. Across all three, success depends heavily on the underlying model’s ability to decompose the problem effectively, explore alternative proof steps, diagnose failures, and integrate feedback over long reasoning horizons.

Despite this progress, existing formal-reasoning corpora are still dominated by whole-proof trajectories, where models primarily observe completed theorem–proof pairs, sometimes augmented with verifier feedback or revised proofs (Lin et al., [8]; Ma et al., [10]). In contrast, many behaviors essential to effective proving are exposed more explicitly in post-training or inference-time procedures, including decomposing goals into useful sub-theorems [4], exploring alternative proof steps [14, 24], verifying intermediate reasoning [16], and localizing and repairing failures. Rather than leaving these capabilities to be acquired primarily during post-training, we argue that they can be injected into mid-training data as reusable mathematical skill priors, an idea supported by recent findings on self-improving reasoners [7]. Beyond these, agentic provers also require a distinct form of training data that captures executable interactions with Lean, including observing proof states, interpreting verifier feedback, and revising failed attempts. These agentic trajectories during mid-training can reduce the distribution gap between static model training and agentic inference, where the model operates within an executable proving environment [28, 26].

In this work, we introduce davinci-Prover, a family of formal theorem-proving models trained on our curated mid-training corpus and high-quality supervised fine-tuning (SFT) data collected through a lightweight prover harness. As shown in Figure 1, our mid-training corpus covers informal mathematical reasoning, proof decomposition and assembly, verifier-grounded subgoal search, and error diagnosis and revision. We further incorporate agent-native trajectories that expose models to authentic tool calls, execution results, intermediate proof states, and Lean feedback. Together, these data teach the model not only to generate valid proofs, but also to develop the reasoning and interaction patterns required for effective proving in an executable environment. We instantiate this recipe with a 33.7B-token mid-training corpus. For post-training, we build a lightweight prover harness that provides a persistent workspace, typed tools, Lean verification, and trusted completion checks. Using a strong teacher model within this harness, we collect and filter approximately 36K domain-diverse agent trajectories for SFT. We evaluate the resulting models on various Lean 4 benchmarks. Our main contributions are as follows:

Mid-training data design and benchmark performance

Figure 1. Mid-training data design and benchmark performance. Left: The mid-training data incorporates mathematical skill priors and iterative agent trajectories. Right: Mid-training leads to consistent performance gains on benchmarks across different model families.

  • We construct a 33.7B-token corpus combining mathematical skill priors with environment-native agent trajectories to prepare models for interactive theorem proving.

  • We develop the daVinci-Prover harness, a lightweight prover harness, and curate approximately 36K high-quality, domain-diverse agent trajectories for supervised fine-tuning.

  • Our mid-training improves Qwen3-8B pass@1 by an average of 13.7 points over the matched Base + SFT baseline across MiniF2F, FATE-M, and ProofNet, while consistently improving both Qwen3.5-9B and Qwen3.5-35B-A3B over their corresponding Base + SFT baselines.

Background

Publicly available Lean provers for academic use remain relatively weak [8, 14], while stronger systems are either unreleased [4] or prohibitively expensive to run [20, 15]. Existing approaches can be broadly categorized into three directions as follows [5]:

(1) Whole-proof generation and refinement. These methods generate a complete Lean proof directly from the theorem statement. Representative approaches include DeepSeek-Prover [14], Goed el Prover [8], and Kimina-Prover [21]. Most recent systems augment this paradigm with iterative repair. Goedel-Prover-V2 [8] introduces verifier-guided self-correction, where a complete proof is revised using feedback from the Lean compiler. OProver [10] further combines whole-proof refinement with retrieval and compiler feedback, and trains the prover directly on such multi-round repair trajectories.

(2) Agentic provers. Agentic provers are driven by a single LLM agent that interacts with external tools and adapts its proof strategy based on their outputs. AxProverBase [15] augments off-the-shelf frontier LLMs with iterative proof refinement, memory, and search capabilities. Numina-Lean Agent [9] similarly builds on Claude Code, exposing Lean verification, theorem retrieval, and auxiliary reasoning tools through MCP, thereby enabling a general-purpose coding agent to carry out formal proof development through iterative tool use.

(3) Orchestrated prover systems. In contrast to single-agent approaches, orchestrated systems coordinate multiple reasoning or proving components within a predefined workflow, often separating stages such as decomposition, proof sketching, verification, and refinement. Representative systems include Seed-Prover 1.5 [4], Hilbert [20], and LongCat-Flash-Prover [22], which combine multiple stages or specialized components to guide formal proof construction. Goedel-Architect [5] constructs and iteratively refines a dependency-graph blueprint of formal lemmas, optionally seeded by a natural-language proof, and dispatches the resulting subgoals to tool-equipped Lean provers in parallel.

However, sophisticated orchestrated systems rely on capable agentic provers as their core building blocks; in this work, we focus on advancing the capability of the agentic prover itself. More related work can be found in Appendix § 12.

Mid-training Data Construction

This section presents our mid-training data construction framework for developing prover-oriented foundation models. The corpus comprises two complementary components. The first is designed to instill core theorem-proving skill priors, including (1) proof decomposition and assembly, (2) proof search, (3) error diagnosis and revision, and (4) self-verification. The second consists of environment-native agent trajectories that capture authentic interactions with tools, execution outcomes, and verifier feedback.

Theorem-proving Skill Priors

Construction of Decomposition and Assembly Trajectories. We construct decomposition and assembly trajectories [14, 30, 4] to develop lemma-based proof planning and composition. As shown in Figure 2(a), a teacher LLM receives a Lean theorem and its verified reference proof, and generates a natural-language proof plan, auxiliary lemmas with dependency information, and a complete assembled Lean proof. The teacher is instructed to specify each lemma’s required variables and hypotheses and preserve the original theorem statement. These trajectories are used to identify useful intermediate results, formulate self-contained lemmas, and combine them into a coherent proof.

Overview of the midtrain data generation pipeline

Figure 2. Overview of our midtrain data generation pipeline.

Construction of Verified Subgoal Search Data. We replay verified Lean proofs to extract intermediate states and executable snapshots, as shown in Figure 2(b). For each state, a proposal model generates diverse tactics, which we combine with the reference tactic, anonymize, shuffle, and execute independently from the same snapshot. We retain successful closures, accepted open states, and Lean errors, while excluding timeouts and infrastructure failures. Each trajectory pair links candidate tactics and their search intents with the corresponding execution outcomes, teaching the model to compare alternatives, interpret formal feedback, and distinguish valid intermediate transitions from completed proofs.

Construction of Revision Trajectories. We construct revision trajectories that explicitly connect error diagnosis to proof repair. Given failed proof attempts, diagnostic feedback, and reference corrections, a teacher LLM synthesizes the diagnosis. Figure 2(c) shows a structured diagnosis with four fields: location identifies the affected lemma or proof step; symptom describes the observed failure; cause explains the underlying issue; and fix proposes a targeted correction. Each diagnosis is followed by an explicit revision plan and a revised lemma or proof. These trajectories train the model to localize errors, explain their causes, and formulate targeted corrections.

Construction of Informal Reasoning and Verification Trajectories. We incorporate informal trajectories from Nemotron-Math-Proofs-v2 using DeepSeekMath-V2-style prompts [16]. The data covers proof generation with self-evaluation, proof verification, and meta-verification. Proof verification identifies gaps and assesses proof quality, while meta-verification reviews either the generator’s self-evaluation or an independent verifier’s critique, checking whether the reported defects and assigned scores are justified. These complementary trajectories aim to develop rigorous natural-language reasoning, calibrated self-assessment, and evidence-based critique of both proofs and their evaluations.

daVinci-Prover harness

The harness formulates theorem proving as an iterative edit–check–revise loop with Lean4. The input theorem is copied verbatim into scratch/solution.lean, the agent’s main proof file. At each turn, the agent selects from seven tools (see Appendix § 16) to edit candidates, inspect files, retrieve Lean declarations, and run focused checks. The writable scratch/ directory stores proof candidates and auxiliary files, while sketch/ holds optional lemma decomposition drafts. The read-only target/ directory provides the Lean/Mathlib project used for verification. Every Lean-file edit automatically triggers verification. Draft checks expose local goals and diagnostics, while placeholder-free candidates undergo strict verification. Tool results are added to the conversation to guide subsequent revisions. The episode terminates successfully only when a candidate passes strict Lean verification and checks confirming that its declaration and elaborated type match the original theorem.

daVinci-Prover agent harness

Figure 3. daVinci-Prover agent harness

The Final Dataset: Davinci-prover 33.7B Collection

Data Source. We use a curated corpus drawn from NuminaMath-LEAN [21], AoPS [1], Lean Workbook [27], Goedel LM/SFT dataset v2 [8], and OProofs [10]. To mitigate data contamination [32], we remove duplicate problems and exclude examples that overlap with our evaluation benchmarks. These filtering steps remove 0.03427% of the 33.7B-token mid-training corpus, with token counts computed using the Qwen3.5 tokenizer; see Appendix § 14.

Data Cleaning and Reconstruction. Following § 3, we use DeepSeek-V4-Flash and DeepSeek-V4-Flash-0731 [25] to construct an approximately 29B-token mid-training corpus, whose composition is shown in Table 1. We remove repetitive system prompts, tool schemas, and source-specific boilerplate while retaining the problems, reasoning traces, tool interactions, and outcomes. Each example is then reconstructed into a unified training document and semantically chunked to at most 32K tokens at natural boundaries.

Category# Samples (M)# Toks. (B)Share (%)
Pre-Replay Data8.13128.684.8
Self-verification0.0835.7016.9
Theorem Decomposition4.6628.4325.0
Proof Revision1.0273.8411.4
Proof Search1.9623.4510.2
Environment-Aware Agent Trajectories0.3977.1721.3
Replay Data1.4545.1115.2
Total9.58533.70100.0

Table 1. Categories and statistics of the mid-training corpus.

Data Replay. We define a parent example as one complete logical trajectory before 32K semantic chunking. We assign each source a replay budget according to the desired data mixture and select high-quality parent examples using source-specific scores. To preserve diversity, we cover distinct problems before selecting additional trajectories from the same problem. Once a parent example is selected, all of its semantic chunks are replayed together and processed using the same reconstruction and packing pipeline as the original corpus. This adds 5.11B replay tokens, resulting in a final corpus of 33.7B tokens.

Supervised Fine-Tuning

Data Source. Our SFT training corpus consists of trajectories generated through harness rollouts. We sample problems from the Goedel-LM/RL dataset V2 dataset [8], which contains approximately 99K unverified problems, and use Goedel-LM/SFT dataset V2 for domain supplementation. We limit our interaction turns to 30 turns with a maximum of 200 tool calls. The trajectories are collected using DeepSeek-V4-Flash-0731.

SFT Data Overview. We ultimately collect approximately 36K high-quality SFT trajectories from successful theorem-proving runs. The resulting dataset spans a broad range of mathematical domains, including algebra, number theory, geometry, analysis, combinatorics, topology, probability and statistics, linear algebra, abstract algebra, and logic and set theory, as shown in Figure 4. Following the filtering strategy of Raoof et al. [13], we remove low-quality trajectories exhibiting undesirable behaviors, such as excessive repetition or unnecessary tool calls, as well as trajectories containing fewer than 5 turns. We further retain only trajectories that fit within a 144K-token context window.

SFT data overview

Figure 4. SFT Data Overview

Experiments

Models. We select Qwen3.5-9B-Base and Qwen3.5-35B-A3B-Base for experiments. We also perform ablation on Qwen3-8B-Base, see § 6.5.

Benchmarks. We evaluate on five Lean 4 theorem-proving benchmarks spanning diverse levels of mathematical difficulty and scope: MiniF2F [31], which contains 244 high-school olympiad problems; MathOlympiadBench [8], consisting of 360 recent competition problems; ProofNet [2], featuring 186 undergraduate-level theorems; ProverBench [14], which includes 325 problems covering both olympiad and undergraduate mathematics; and PutnamBench [18], a collection of 672 Putnam competition problems representing the most challenging benchmark in the suite. We additionally evaluate on FATE-M and FATE-H [17], which consist of standard textbook exercises and challenging problems from abstract and commutative algebra.

Evaluation. We evaluate all models using the daVinci-Prover harness with a temperature of 1.0, the context length of 200K, and up to 30 interactions. We report pass@1 averaged over three runs where applicable, and pass@8 otherwise. All experiments use Lean v4.26 and Mathlib at commit 2df2f0150c275ad53cb3c90f7c98ec15a56a1a67. We reject any proof attempt that introduces axioms or local instances, as these could circumvent formal verification, including through native_decide.

Effectiveness of daVinci-Prover Harness

We select DeepSeek-V4-Flash-0731 and Qwen3.6-35B-A3B to evaluate the harness’s effectiveness. As shown in Table 3, we evaluate our lightweight agent harness across different model backbones, with high reasoning effort enabled for DeepSeek-V4-Flash-0731. Despite its simple design, augmenting the harness with a mathlib retrieval tool consistently improves performance across benchmarks of varying difficulty. The gains are particularly pronounced on the more challenging Fate-H and Putnam

Table 3: Effectiveness of the daVinci-Prover harness. We report pass@1 averaged over three runs. “w/ ret.” denotes the use of the mathlib retrieval tool.

ModelMiniF2FFATE-MFATE-HPutnam
Qwen3.6-35B-A3B68.048.74.03.0
Qwen3.6-35B-A3B (w/ ret.)73.057.35.03.4
DeepSeek-V4-Flash90.294.719.018.9
DeepSeek-V4-Flash (w/ ret.)91.399.329.022.8

Table 3.

benchmarks. We observe a similar improvement trend with Qwen3.6-35B-A3B, indicating that the benefit of retrieval is not specific to a single model backbone.

Agentic Prover Performance

The overall performance results are presented in Table 2. We provide the following analysis:

Table 2: Theorem-proving performance. Best results are in bold, and second-best results are _underlined_. † marks scores not re-evaluated by us under our protocol.

Model#ParamsMathOlympiadMiniF2FProofNetProverBenchPutnam
Open-Weights Reasoning Models (pass@32)
DeepSeek-V3.2671B-A37B14.7†77.9†20.4†42.8†5.8†
Kimi-K2.51T-A32B7.5†76.6†19.9†44.3†1.2†
Open-Weights Prover Models (pass@32)
DeepSeek-Prover-V2-7B7B7.9†73.9†18.1†45.7†1.5†
DeepSeek-Prover-V2671B-A37B13.9†82.4†30.5†52.9†3.3†
Goedel-Prover-V2-8B8B9.9†79.4†17.3†48.6†0.7†
Goedel-Prover-V2-32B32B16.0†85.8†22.0†51.0†5.0†
LongCat-Flash-Prover560B-A27B16.9†84.4†19.9†49.9†4.9†
whole-proof w/ TIR560B-A27B27.5†90.2†36.1†57.9†10.4†
sketch-proof w/ TIR560B-A27B35.8†93.9†47.3†66.5†28.9†
OProver-8B8B21.7†91.8†31.9†56.0†9.0†
OProver-32B32B22.8†93.3†33.2†58.2†11.3†
daVinci-Prover Harness (pass@8)
DeepSeek-V4-Flash-0731284B-A13B52.295.956.568.939.7
Qwen3.5-9B-Base + SFT9B32.392.651.166.222.3
Qwen3.5-9B-Midtrain + SFT9B35.393.952.766.524.9
Qwen3.5-35B-Base + SFT35B-A3B35.893.052.766.524.3
Qwen3.5-35B-Midtrain + SFT35B-A3B_36.1__94.3__53.8__67.1__26.5_

Table 2.

Agentic prover outperforms existing whole-proof baselines. DeepSeek-V4-Flash-0731, evaluated with the daVinci-Prover harness, achieves the highest reported scores across all five benchmarks. Its interactive pass@8 results exceed the reported pass@32 results of whole-proof generation and refinement methods. The results highlight the potential of agentic proving to enable strong theorem proving with whole proof.

High-quality SFT trajectories effectively elicit theorem-proving capabilities. We further use the proof trajectories generated by the daVinci-Prover harness as SFT data to train Qwen3.5 models. Despite their substantially smaller model sizes, the resulting models achieve strong performance across all benchmarks, indicating that trajectories collected by the harness provide effective supervision for transferring its theorem-proving behavior to smaller models.

Mid-training consistently improves performance over the base models. Across both model scales, the Midtrain + SFT variants consistently outperform their corresponding Base + SFT counterparts. The gains are particularly pronounced on more challenging benchmarks such as MathOlympiad and Putnam. For Qwen3.5-9B, mid-training improves MathOlympiad from 32.3 to 35.3 and Putnam from 22.3 to 24.9, while maintaining consistent improvements on the other benchmarks. A similar trend is observed for Qwen3.5-35B. These results suggest that mid-training provides a stronger initialization for subsequent theorem-proving SFT, with particularly clear benefits on more difficult mathematical reasoning tasks.

IN-DEPTH ANALYSES

Training Process

Figure 5. Training dynamics and miniF2F performance of models with and without mid-training.

Setup. We compare base and mid-trained checkpoints over five epochs of SFT. Each checkpoint is evaluated on miniF2F-test problems using five runs and reports avg@5 and pass@5.

Mid-training bridges the gap to agentic learning. fig:5a shows that mid-trained checkpoints start SFT with lower loss and remain better aligned with the SFT distribution throughout training. As shown in fig:5b and fig:5c, after one epoch, mid-training improves avg@5 by 5.33 and 5.08 points for the 9B and 35B models, respectively, and already matches the base model’s epoch-5 pass@5. These gains persist through epoch 5, indicating faster adaptation and a better initialization for downstream SFT.

Scaling with model size. The 35B-A3B MoE model generally outperforms the 9B model, although the gap is relatively small on MiniF2F, possibly due to limited headroom on this benchmark. As shown in Table 2, the 35B-A3B midtrain model consistently outperforms the 9B midtrain model across all benchmarks. Notably, the 9B midtrain model surpasses the 35B-A3B base model on both MiniF2F metrics despite having a higher final SFT loss, suggesting that the downstream gains potentially come from mid-training.

Analysis of Agent Behavior

Common patterns. As shown in Figure 6, across all four settings, the aggregate tool-use distribution shifts from retrieval towards verification and revision as the interaction progresses. lean_retrieve accounts for over half of all tool calls during the first five turns, whereas lean_verify and file_edit together account for approximately 55–57% during turns 21–30. This shift remains visible after normalizing by the number of active trajectories, suggesting a common workflow of gathering relevant information followed by checking and revising candidate proofs. The declining bar heights also reflect the shrinking population of active trajectories. For a relatively easier benchmark such as MiniF2F, we provide a separate analysis in Appendix § 13.§ 1, which reveals distinct behavioral patterns.

Tool call analysis of PutnamBench across four model settings

Figure 6. Tool call analysis of PutnamBench. Tool colours: file_edit, lean_retrieve, lean_verify, file_write, content_search, file_read, and file_search. The dashed line indicates the number of active trajectories.

Midtraining changes model behaviour. As shown in Figure 6, midtrained models exhibit stronger tool engagement at the beginning of an interaction and increase the share of file_write calls, particularly in the early turns, indicating earlier file construction. In Figure 7, explicit auxiliary-check passing improves consistently after midtraining, from 11.8% to 12.7% at 9B and from 12.6% to 13.4% at 35B. Moreover, mid-training increases the number of local steps in the first target-candidate proof on medium- and high-difficulty problems, while having little effect on easy problems. These results suggest that mid-training promotes more decomposition and verification as problem difficulty increases, with the decomposition behavior on harder problems becoming closer to that of DeepSeek-V4-Flash-0731.

Behavior comparison showing decomposition and self-check

Figure 7. Behavior comparison.

Scaling with Data Volume

Setup. We evaluate mid-training checkpoints of Qwen3.5-9B at 7.5B, 15B, 22.5B, 30B, and 33.7B tokens. We apply the same two-epoch SFT procedure to each checkpoint and the base model. We measure performance by average pass@1 across three runs.

Analysis. Agentic theorem-proving performance exhibits an approximately log-linear trend with mid-training data scale. Much of the improvement is achieved within the first 15B tokens, yielding gains of 3.96 and 11.11 percentage points over the base model on miniF2F and FATE-M, respectively. Further mid-training generally improves performance, although gains shrink at larger data scales. These results suggest that targeted mid-training can effectively develop agentic theorem proving capabilities.

Scaling law of agent-native mid-training

Figure 8. Scaling law of agent-native mid-training.

Ablation on daVinci-Prover

Setup. We explore the contribution of each data component on the Qwen3.5-9B models.

Analysis. As shown in Table 4, the ablation highlights the importance of agent-native supervision beyond generic theorem-proving skill acquisition. Despite using only 8.4B tokens, Dagent\mathcal{D}_{agent} produces larger gains than the 25.3B-token Dskill\mathcal{D}_{skill} on MiniF2F and Fate-M. A likely explanation is distribution alignment: Dagent\mathcal{D}_{agent} exposes the model to the same interaction primitives encountered at inference time, thereby bridging the gap between static next-token training and interactive proving. In contrast, Dskill\mathcal{D}_{skill} primarily strengthens reusable reasoning priors such as decomposition, search, diagnosis, and self-verification. Importantly, these two forms of supervision are complementary rather than substitutable. Combining them consistently outperforms either component alone, indicating that agentic proving benefits from both knowing how to reason about a proof and knowing how to realize that reasoning through environment interaction.

ComponentTokensMiniF2FFate-MProofNet
Qwen3.5-9B Base–87.488.442.9
Dskill\mathcal{D}_{skill}25.3B87.889.843.6
Dagent\mathcal{D}_{agent}8.4B88.892.243.5
Dskill+Dagent\mathcal{D}_{skill} + \mathcal{D}_{agent}33.7B90.193.344.5

Table 4. Data component ablation.

Comparison with Existing Corpora

Setup. We perform mid-training on Qwen3-8B-Base using 33B tokens, followed by agentic SFT for 5 epochs. We compare the resulting model against Qwen3-8B-Base without mid-training and Oprover-8B-Base [10], which is trained with 65B mid-training tokens. For all base models, we use YaRN [12] to extend the context length from 32K to 256K.

Analysis. As shown in Table 5, our mid-training substantially improves performance over Qwen3-8B-Base across all three benchmarks, confirming that the gains cannot be attributed solely to the subsequent agentic SFT. Moreover, our model consistently outperforms Oprover-8B-Base, with particularly large margins on Fate-M and ProofNet, despite using only 33B mid-training tokens compared with Oprover’s 65B. This suggests that the effectiveness of mid-training depends not only on data scale, but critically on its composition: our combination of theorem-proving skill priors and agent-native trajectories provides more targeted supervision for downstream interactive proving and transfers effectively across diverse formal reasoning problems.

Data componentMiniF2FFate-MProofNet
Qwen3-8B-Base + SFT73.468.028.4
Oprover-8B-Base + SFT80.373.130.3
Qwen3-Base + Midtrain (ours) + SFT85.385.340.4

Table 5. Comparison with existing corpora.

Memory Compact

Setup. To support long-horizon difficult proofs, we use a simple, model-free trajectory compression method. Once the context length reaches 200K tokens, we trigger compression by replacing older interactions with deterministically extracted tool evidence, while preserving the original theorem, current Lean proof, and recent interactions. We allow up to 8 compactions, and other settings remain unchanged.

ModelMiniF2FFATE-MFATE-H
w/o memory compact
Qwen3.5-midtrain-9B90.193.316.7
Qwen3.5-midtrain-35B90.193.716.3
w/ memory compact
Qwen3.5-midtrain-9B97.196.728.0
Qwen3.5-midtrain-35B97.197.342.0

Table 6. Effect of memory compaction on theorem-proving performance. Performance is evaluated using pass@1, averaged over 3 runs.

Analysis. Memory compaction consistently improves performance, with the largest gains on the more challenging FATE-H benchmark. The improvement is especially pronounced for the 35B model, suggesting that effective context management is important for long-horizon proving and can better expose the capacity of larger models.

Conculusion

We introduced daVinci-Prover, combining agent-native mid-training with verifier-grounded supervised fine-tuning for interactive theorem proving based on a single harness. Experiments demonstrate consistent improvements over SFT-only baselines, while ablations support the complementary roles of mathematical skill priors and environment-native trajectories. These findings highlight agent-native mid-training as an effective foundation for developing stronger theorem-proving agents.

AI USE STATEMENT

In this work, we used generative AI tools for paper polishing and coding assistance. All AI-assisted content was carefully reviewed by the authors. For example, we manually checked AI-polished text, and AI-generated code was independently verified and tested for correctness by two authors. We take full responsibility for the final content of this work, including all text, claims, and artifacts produced with the assistance of generative AI.

REPRODUCIBILITY STATEMENT

The data we collected, the models we trained, the harness we built, and the corresponding code and training infrastructure will be fully open-sourced for reproducibility after the anonymous review period ends.

ETHICS STATEMENT

No ethics statement is required here.

LIMITATION

During the mid-training phase, we do not replay any generalization data. However, we believe that incorporating generalization replay is both necessary and important for maintaining and improving the model’s generalization capabilities.

Related Work

Mid-training. Recent work views mid-training as a useful bridge between pre-training and post-training. Instead of moving directly from pre-training to SFT or RL, mid-training uses cleaner and more task-relevant data to better prepare the model, often with learning-rate annealing [29, 19]. For example, OctoThinker [23] shows that reasoning-focused mid-training can improve both RL efficiency and final performance.

Agentic Proving Agentic provers make the interaction loop part of the solution. Prover Agent coordinates informal reasoning, formal proving, auxiliary lemmas, and Lean feedback [3]. OProver retrieves verified proofs, repairs failures, and feeds successful interactions back into training [10]; LongCat-Flash-Prover trains auto-formalization, sketching, proving, and tool use through agentic reinforcement learning [22]. Thus, interaction traces can serve as training data rather than only inference-time scaffolding.

Agent-Native Mid-Training Software-engineering research offers a complementary training perspective: Kimi-Dev treats structured workflow training as transferable skill priors, while DaVinci-Dev combines contextually and environmentally native trajectories during mid-training [26, 28]. We also focus on the stage at which agents acquire capabilities. We organize decomposition, search, revision, and environment interaction as complementary mid-training skills, then study whether these priors transfer from whole-proof generation to downstream interactive proving.

In-depth analysis of Agent Behavior

Three-panel analysis of agent trajectories: retrieval calls, strict proof success, and revision capability

Figure 9. In-depth analysis of agent trajectories.

We analyze the agent trajectories on the same 672 Putnam problems, using eight runs for DeepSeek-V4-Flash-0731 and eight runs for Qwen3.5-9B midtrain and Qwen3.5-35B-A3B midtrain models.

Retrieval utilization. DeepSeek-V4-Flash-0731, Qwen3.5-35B-A3B-Midtrain, and Qwen3.5-9B-Midtrain exhibit similar observed retrieval adoption rates of 28.45%, 27.01%, and 27.53%, respectively. The fraction of retrieval calls whose returned declarations appear in strictly verified target proofs is higher for DeepSeek (5.47%) than for Qwen3.5-35B (3.48%) and Qwen3.5-9B (4.01%). Both measures use all successful, nonempty retrieval calls as their denominator.

Proof completion across candidate checks. DeepSeek’s advantage is already visible within the first few candidate checks. By the fifth distinct target candidate checked, cumulative observed success reaches 12.15%, compared with 6.85% and 6.60% for Qwen 35B and 9B. Over the full recorded trajectories, the corresponding rates reach 23.38%, 14.66%, and 13.86%. This pattern shows higher proof completion across the observed candidate generation and revision process. The horizontal axis counts distinct checked target candidates, including drafts, rather than interaction turns.

Outcomes of iterative edits. Among content-changing edits following a failed check of the same target file with a primary diagnostic, 6.60% of DeepSeek’s edits immediately produce a strict target pass, compared with 3.43% and 3.75% for Qwen 35B and 9B. DeepSeek therefore completes the target more frequently immediately after eligible edits in these logs.

Tool call analysis of MiniF2F

Four-panel tool call analysis for 9B Base, 9B Midtrain, 35B-A3B Base, and 35B-A3B Midtrain

Figure 10. Tool call analysis. Tool colours: file_edit, lean_retrieve, lean_verify, file_write, content_search, file_read, and file_search. The dashed line indicates the number of active trajectories.

Compared with Putnam, miniF2F shows shorter tool-use trajectories and earlier proof writing and editing. Only about 8–11% of trajectories remain active at turn 30 on miniF2F, compared with 64–73% on Putnam. Putnam also places greater emphasis on early retrieval, followed by sustained editing and verification.

Across the four miniF2F models, the overall tool-use pattern is similar. Midtrained models make fewer total tool calls while recording slightly more successful tasks, with a larger reduction at 9B (8.75%) than at 35B (2.65%). Their late-stage active trajectories also decline more quickly. On Putnam, however, Midtrained models sustain longer interactions and make more calls. This suggests that Midtraining changes interaction duration differently across benchmarks.

Decomposition

Four-panel proof decomposition analysis showing local steps in proof and local steps per 1,000 source tokens for Qwen3.5-35B-A3B and Qwen3.5-9B across low, medium, and high difficulty

Figure 11. Proof decomposition analysis.

Effect of midtraining on proof decomposition. Midtrained models produce more explicit intermediate steps in their first checked target proof, particularly on harder problems. In the high-difficulty group, defined as problems unsolved by the DeepSeek reference across eight attempts, the average number of local have/suffices steps increases from 5.13 to 9.57 at 9B and from 5.73 to 9.63 at 35B. The corresponding step densities also increase, from 6.22 to 9.33 and from 6.28 to 8.15 per 1,000 lexical source tokens, respectively. Thus, these differences are not explained solely by longer code. By contrast, Base and Midtrain models show similar step counts in the low difficulty group. This pattern suggests that midtraining is associated with more explicit decomposition when constructing initial proofs for challenging problems.

Comparison with DeepSeek. DeepSeek produces more local steps in absolute terms, especially in the medium-difficulty group, but its step density is lower than that of the Midtrained Qwen models. In the high-difficulty group, its average step count is 10.17, close to the approximately ten steps produced by both Midtrained Qwen models. Therefore, DeepSeek’s higher overall success rate cannot be attributed simply to producing more intermediate steps.

Decontamination

To ensure a fair evaluation, we perform benchmark-level decontamination against all evaluation sets listed in Table 7. For Lean data, we extract the theorem signature by removing nested comments and proof bodies, normalize Unicode characters, tokenize the remaining statement, and anonymize the declaration name. We then perform exact matching against all available benchmark variants. To avoid false positives caused by short or generic statements, a token-level formal match is automatically confirmed only if the anonymized signature contains at least 20 tokens and 80 normalized characters. Exact provenance links and byte-identical raw statements are also treated as confirmed matches.

BenchmarkOverlapsTotalRate
miniF2F162446.56%
MathOlympiadBench93602.50%
ProofNet21861.08%
DeepSeek-ProverBench13250.31%
Putnam16720.15%
FATE-H/M/X03500.00%
Total292,1371.36%

Table 7. Confirmed benchmark overlaps detected in the mid-training corpus before decontamination. Approximate and review-only matches are excluded.

For informal data, we apply Unicode and math-aware normalization and require an exact match of the complete problem statement with at least 12 content tokens. Matches based only on theorem names or approximate textual similarity are not automatically classified as contamination; instead, they are reserved for manual review. For each confirmed match, we remove the entire logical parent example, including all of its semantic chunks and any replayed copies derived from it.

Before filtering, we identify 29 overlapping problems among the 2,137 evaluation problems (1.36%). When conservatively counting every token in each matched parent trajectory, these examples account for at most 11.55M tokens, corresponding to 0.03427% of the 33.7B-token mid-training corpus.

For the SFT training data, we also remove any examples that overlap with the evaluation benchmarks.

Evaluation metric

In Table 2, to account for inference efficiency and budget, we evaluate each problem using Pass@8:

Pass⁡@8(i)=1[∑j=18success⁡ij>0].(1)\operatorname{Pass}@8(i) = 1\left[\sum_{j=1}^{8}\operatorname{success}_{ij} > 0\right]. \tag*{(1)}

Tool Schema

We use the following tools to expose the agent. There are only 7 tools. For the convenience of future RL expansion, we do not use bash and Python commands to expose the agent.

Data Synthesis Costs

Teacher deployment. We deployed DeepSeek-V4-Flash-0731 on 48 NVIDIA H200 GPUs for approximately 20 days, with the teacher service shared across mid-training and SFT data synthesis. This corresponds to approximately 48 × 20 × 24 = 23,040 allocated H200 GPU-hours. This deployment-level budget includes unsuccessful attempts, discarded generations, and idle periods, in addition to generations retained in the final datasets.

ToolCategoryDescription
file_readFileRead workspace files or line ranges.
file_writeEditWrite complete workspace files.
file_editEditReplace text spans or line ranges.
file_searchSearchFind files by glob patterns.
content_searchSearchSearch file contents by text or regex.
lean_retrieveRetrievalRetrieve relevant Mathlib declarations.
lean_verifyVerificationCheck Lean code or scratch files.

Table 8. Internal tools exposed to the davici-Prover agent harness.

CPU resources for Lean verification. The CPU service supporting Lean verification used approximately eight worker instances, each provisioned with 20 CPU cores and 80 GB of memory, for roughly the same 20-day period. The estimated aggregate allocation was therefore 160 CPU cores and 640 GB of memory, corresponding to approximately 8×20×20×24=76,8008 \times20 \times20 \times24 = 76{,}800 allocated CPU core-hours.

Public-cloud rental equivalent. For a public-cloud comparison, Runpod lists H200 Secure Cloud instance and H200 SXM cluster rates of US$5.29 and US$6.35 per GPU-hour, respectively, as of November 9, 2026.2 Applying these rates to the teacher deployment yields a GPU rental-equivalent cost of approximately US$122,000–146,000. This is a reference-price estimate rather than an actual expenditure figure, and excludes the additional CPU service for Lean verification and separate storage charges.

Hyperparameters

Mid-Training Details

We perform full-parameter mid-training using the autoregressive next-token prediction objective, while retaining the one-layer multi-token prediction (MTP) objective with a loss weight of 0.1. For the MoE model, we retain the pretrained routing configuration and apply a global-batch auxiliary load-balancing loss with a coefficient of 10−310^{-3}. We use AdamW with a weight decay of 0.1. The learning rate is linearly warmed up for the first 3% of training steps and then cosine-decayed to zero. For offline-packed 32K-token sequences, we enforce sample-level isolation using per-sample cumulative sequence boundaries. These boundaries induce block-diagonal causal attention, reset positional indices and Gated DeltaNet states, and mask next-token targets that cross sample boundaries. More details are in Table 9.

HyperparameterQwen3.5-9BQwen3.5-35B-A3B
Model
InitializationQwen3.5-9B-BaseQwen3.5-35B-A3B-Base
Finetuning typefull parameterfull parameter
Training backendMegatron-BridgeMegatron-Bridge
Precisionbf16 mixedbf16 mixed
Attention backendauto, cuDNN fused preferredauto, cuDNN fused preferred
Vision modulesfrozenfrozen
MTP layers11
MTP loss coefficient0.10.1
Data
Training objectivecausal LM on all valid tokenscausal LM on all valid tokens
Valid training tokens33,699,997,41333,699,997,413
Physical training tokens34,133,245,95234,133,245,952
Fixed packed rows1,041,6641,041,664
Sequence length32,76832,768
Packing efficiency98.73%98.73%
Pre-packed THD sequencesenabledenabled
Shuffleglobal random cyclic samplerglobal random cyclic sampler
Optimization
Epochs1 (8,138 steps)1 (8,138 steps)
Micro batch size1 packed row1 packed row
Gradient accumulation1632
Global batch size128 packed rows128 packed rows
Physical token batch4,194,304 tokens/step4,194,304 tokens/step
Valid token batch∼4.14M tokens/step∼4.14M tokens/step
Optimizerdistributed Adamdistributed Adam
Peak learning rate5×10−55 \times10^{-5}5×10−55 \times10^{-5}
Minimum learning rate00
LR schedulercosinecosine
Warmup244 steps (3%)244 steps (3%)
Weight decay0.10.1
Adam betas(0.9, 0.95)(0.9, 0.95)
Adam epsilon1×10−81 \times10^{-8}1×10−81 \times10^{-8}
Gradient clipping1.01.0
Per-token lossenabledenabled
Parallelism
Number of GPUs4×84 \times8 H2004×84 \times8 H200
TP / PP / CP / DP4 / 1 / 1 / 84 / 2 / 1 / 4
EP / ETP1 / 44 / 1
Sequence parallelismenabledenabled
Parameter gather overlapenabledenabled
Gradient reduce overlapenabledenabled
MoE
Number of expertsN/A256
Experts per tokenN/A8
Token dispatcherN/Aall-to-all
Grouped GEMMN/Aenabled
Router precisionN/Afp32
Load balancingN/Aglobal auxiliary loss
Auxiliary loss coefficientN/A1×10−31 \times10^{-3}
Kernels
Bias-activation fusionenabledenabled
Masked-softmax fusionenabledenabled
Cross-entropy fusionenabledenabled
MoE permutation fusionN/Aenabled

Table 9. Qwen3.5 mid-training configurations on 32 NVIDIA H200 GPUs.

SFT Training Details

The SFT training details are provided in Table 10. The Qwen3 model is trained using the same configuration as Qwen3.5-9B.

HyperparameterQwen3.5-9BQwen3.5-35B-A3B
Model
InitializationQwen3.5-9B-BaseQwen3.5-35B-A3B-Base
Training backendSlime + Native MCoreSlime + Native MCore
Precisionbf16bf16
Attention implementationMCore flash backendMCore flash backend
Vision modulesfrozenfrozen
MTP layers11
MTP loss coefficient0.10.1
Data
Training samples35,86135,861
Maximum sequence length147,456 (144K)147,456 (144K)
Maximum position embeddings262,144262,144
Mean sequence length36,445 tokens36,445 tokens
P95 sequence length84,446 tokens84,446 tokens
Shuffleglobal shuffleglobal shuffle
Optimization
Epochs55
Per-device batch sizedynamicdynamic
Maximum tokens per GPU131,072131,072
Gradient accumulationdynamic to reach GBSdynamic to reach GBS
Global batch size128 samples128 samples
Effective token batch∼\sim4.665M tokens/step∼\sim4.665M tokens/step
Optimizerdistributed Adamdistributed Adam
Peak learning rate3×10−53 \times10^{-5}4×10−54 \times10^{-5}
Minimum learning rate4×10−64 \times10^{-6}4×10−64 \times10^{-6}
LR schedulercosinecosine
Warmup50 steps (∼\sim3.6%)50 steps (∼\sim3.6%)
Weight decay0.010.01
Adam betas(0.9, 0.98)(0.9, 0.98)
Gradient clipping1.01.0
Attention dropout0.00.0
Hidden dropout0.00.0
Cross-entropylinear fused CElinear fused CE
Parallelism
Number of GPUs4×84 \times8 H2004×84 \times8 H200
TP / PP / CP / DP4/1/1/84 / 1 / 1 / 84/1/1/84 / 1 / 1 / 8
EP / ETP1/11 / 18/18 / 1
Sequence parallelismenabledenabled
MoE dispatcherN/Aflex + DeepEP
MoE routingN/Asoftmax, top-8 of 256
MoE load balancingN/Aglobal auxiliary loss
MoE auxiliary coefficientN/A1×10−31 \times10^{-3}

Table 10. Native MCore full-parameter SFT configurations for Qwen3.5 models.

Resource Usage of Memory Compaction

Generation usage. We report generation and verification resource usage across 26,708 recorded turns. Table 11 summarizes the compaction events and cumulative input and output tokens for these records. Input tokens are summed over each call’s actual context, accounting for history replacement after compaction and including retained checkpoints in subsequent calls. Output tokens include reasoning, response text, and tool calls.

DatasetCompaction eventsInput tokensOutput tokens
MiniF2F104414.2922.44
FATE-M33140.677.89
FATE-H3651,646.5163.48
Total5022,201.4793.81

Table 11. Recorded resource usage for memory-compaction evaluation. Input and output token counts are in millions.

Compaction overhead and accounting checks. All 502 compactions use rule-based extraction without additional LLM calls for summarization. Token counts for all 1,004 pre- and post-compaction context snapshots match the historical server-side records. Thus, the input accounting includes the context actually consumed after each history replacement.

Lean verification. Recorded verification-call durations sum to approximately 11.02 hours, with a median of 0.734 seconds per call; 91.4% of calls take at most 10 seconds. These cumulative durations include overlapping concurrent calls and should not be interpreted as end-to-end elapsed time or CPU-hours.

Scope. Memory compaction is an auxiliary analysis; our main contribution is agent-native mid-training and its benefits for downstream agentic proving.

References

  1. [1]Art of Problem Solving. Art of problem solving community. https://artofproblemsolving.com/community, 2026. Accessed August 31, 2026.
  2. [2]Zhangir Azerbayev, Bartosz Piotrowski, Hailey Schoelkopf, Edward W Ayers, Dragomir Radev, and Jeremy Avigad. Proofnet: Autoformalizing and formally proving undergraduate-level mathematics. arXiv preprint arXiv:2302.12433, 2023.arxiv.org/abs/2302.12433
  3. [3]Kaito Baba, Chaoran Liu, Shuhei Kurita, and Akiyoshi Sannai. Prover agent: An agent-based framework for formal mathematical proofs. arXiv preprint arXiv:2506.19923, 2025.arxiv.org/abs/2506.19923
  4. [4]Jiangjie Chen, Wenxiang Chen, Jiacheng Du, Jinyi Hu, Zhicheng Jiang, Allan Jie, Xiaoran Jin, Xing Jin, Chenggang Li, Wenlei Shi, et al. Seed-prover 1.5: Mastering undergraduate-level theorem proving via learning from experience. arXiv preprint arXiv:2512.17260, 2025.arxiv.org/abs/2512.17260
  5. [5]Jui-Hui Chung, Ziyang Cai, Zihao Li, Qishuo Yin, Rohit Agarwal, Simon Park, Rodrigo Porto, Narutatsu Ri, Ziran Yang, Shange Tang, et al. Goedel-architect: Streamlining formal theorem proving with blueprint generation and refinement. arXiv preprint arXiv:2606.06468, 2026.arxiv.org/abs/2606.06468
  6. [6]Leonardo De Moura, Soonho Kong, Jeremy Avigad, Floris Van Doorn, and Jakob von Raumer. The lean theorem prover (system description). In International conference on automated deduction, pp. 378–388. Springer, 2015.DOI
  7. [7]Kanishk Gandhi, Ayush Chakravarthy, Anikait Singh, Nathan Lile, and Noah D Goodman. Cognitive behaviors that enable self-improving reasoners, or, four habits of highly effective stars. arXiv preprint arXiv:2503.01307, 2025.arxiv.org/abs/2503.01307
  8. [8]Yong Lin, Shange Tang, Bohan Lyu, Ziran Yang, Jui-Hui Chung, Haoyu Zhao, Lai Jiang, Yihan Geng, Jiawei Ge, Jingruo Sun, et al. Goedel-prover-v2: Scaling formal theorem proving with scaffolded data synthesis and self-correction. In International Conference on Learning Representations, volume 2026, pp. 11793–11818, 2026.arxiv.org/abs/2508.03613
  9. [9]Junqi Liu, Zihao Zhou, Zekai Zhu, Marco Dos Santos, Weikun He, Jiawei Liu, Ran Wang, Yun-zhou Xie, Junqiao Zhao, Qiufeng Wang, et al. Numina-lean-agent: An open and general agentic reasoning system for formal mathematics. arXiv preprint arXiv:2601.14027, 2026.arxiv.org/abs/2601.14027
  10. [10]David Ma, Kaijing Ma, Shawn Guo, Yunfeng Shi, Enduo Zhao, Jiajun Shi, Zhaoxiang Zhang, Gavin Cheung, Jiaheng Liu, and Zili Wang. Oprover: A unified framework for agentic formal theorem proving. arXiv preprint arXiv:2605.17283, 2026.arxiv.org/abs/2605.17283
  11. [11]Leonardo de Moura and Sebastian Ulrich. The lean 4 theorem prover and programming language. In International Conference on Automated Deduction, pp. 625–635. Springer, 2021.DOI
  12. [12]Bowen Peng, Jeffrey Quesnelle, Honglu Fan, and Enrico Shippole. Yarn: Efficient context window extension of large language models. In International Conference on Learning Representations, volume 2024, pp. 31932–31951, 2024.arxiv.org/abs/2309.00071
  13. [13]Negin Raoof, Richard Zhuang, Marianna Nezhurina, Etash Guha, Atula Tejaswi, Ryan Marten, Charlie F Ruan, Tyler Griggs, Alexander Glenn Shaw, Hritik Bansal, et al. Openthoughts-agent: Data recipes for agentic models. arXiv preprint arXiv:2606.24855, 2026.
  14. [14]ZZ Ren, Zhihong Shao, Junxiao Song, Huajian Xin, Haocheng Wang, Wanjia Zhao, Liyue Zhang, Zhe Fu, Qihao Zhu, Dejian Yang, et al. Deepseek-prover-v2: Advancing formal mathematical reasoning via reinforcement learning for subgoal decomposition. arXiv preprint arXiv:2504.21801, 2025.arxiv.org/abs/2504.21801
  15. [15]Borja Requena, Austin Letson, Krystian Nowakowski, Izan Beltran-Ferreiro, and Leopoldo Sarra. A minimal agent for automated theorem proving. arXiv preprint arXiv:2602.24273, 2026.arxiv.org/abs/2602.24273
  16. [16]Zhihong Shao, Yuxiang Luo, Chengda Lu, ZZ Ren, Jiewen Hu, Tian Ye, Zhibin Gou, Shirong Ma, and Xiaokang Zhang. Deepseekmath-v2: Towards self-verifiable mathematical reasoning. arXiv preprint arXiv:2511.22570, 2025.arxiv.org/abs/2511.22570
  17. [17]Ziju Shen, Naohao Huang, Fanyi Yang, Yutong Wang, Guoxiong Gao, Tianyi Xu, Jiedong Jiang, Wanyi He, Pu Yang, Mengzhou Sun, et al. Real-prover: Retrieval augmented lean prover for mathematical reasoning. arXiv preprint arXiv:2505.20613, 2025.arxiv.org/abs/2505.20613
  18. [18]George Tsoukalas, Jasper Lee, John Jennings, Jimmy Xin, Michelle Ding, Michael Jennings, Amitayush Thakur, and Swarat Chaudhuri. Putnambench: Evaluating neural theorem-provers on the putnam mathematical competition. Advances in Neural Information Processing Systems, 37: 11545–11569, 2024.arxiv.org/abs/2407.11214
  19. [19]Chengying Tu, Xuemiao Zhang, Rongxiang Weng, Rumei Li, Chen Zhang, Yang Bai, Hongfei Yan, Jingang Wang, and Xunliang Cai. A survey on llm mid-training. arXiv preprint arXiv:2510.23081, 2025.arxiv.org/abs/2510.23081
  20. [20]Sumanth Varambally, Thomas Voice, Yanchao Sun, Zhifeng Chen, Rose Yu, and Ke Ye. Hilbert: Recursively building formal proofs with informal reasoning. In International Conference on Learning Representations, volume 2026, pp. 38998–39031, 2026.arxiv.org/abs/2509.22819
  21. [21]Haiming Wang, Mert Unsal, Xiaohan Lin, Mantas Baksys, Junqi Liu, Marco Dos Santos, Flood Sung, Marina Vinyes, Zhenzhe Ying, Zekai Zhu, et al. Kimina-prover preview: Towards large formal reasoning models with reinforcement learning. arXiv preprint arXiv:2504.11354, 2025a.arxiv.org/abs/2504.11354
  22. [22]Jianing Wang, Jianfei Zhang, Qi Guo, Linsen Guo, Rumei Li, Chao Zhang, Chong Peng, Cunguang Wang, Dengchang Zhao, Jiarong Shi, et al. Longcat-flash-prover: Advancing native formal reasoning via agentic tool-integrated reinforcement learning. arXiv preprint arXiv:2603.21065, 2026.arxiv.org/abs/2603.21065
  23. [23]Zengzhi Wang, Fan Zhou, Xuefeng Li, and Pengfei Liu. Octothinker: Mid-training incentivizes reinforcement learning scaling. arXiv preprint arXiv:2506.20512, 2025b.arxiv.org/abs/2506.20512
  24. [24]Ran Xin, Chenguang Xi, Jie Yang, Feng Chen, Hang Wu, Xia Xiao, Yifan Sun, Shen Zheng, and Ming Ding. Bfs-prover: Scalable best-first tree search for llm-based automatic theorem proving. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 32588–32599, 2025.
  25. [25]Anyi Xu, Bangcai Lin, Bing Xue, Bingxuan Wang, Bingzheng Xu, Bochaо Wu, Bowei Zhang, Chaofan Lin, Chen Dong, Chenchen Ling, et al. Deepseek-v4: Towards highly efficient million-token context intelligence. arXiv preprint arXiv:2606.19348, 2026.arxiv.org/abs/2606.19348
  26. [26]Zonghan Yang, Shengjie Wang, Kelin Fu, Wenyang He, Weimin Xiong, Yibo Liu, Yibo Miao, Bofei Gao, Yejie Wang, Yanһao Li, et al. Kimi-dev: Agentless training as skill prior for swe-agents. In International Conference on Learning Representations, volume 2026, pp. 148369–148436, 2026.arxiv.org/abs/2509.23045
  27. [27]Huaiyuan Ying, Zijian Wu, Yihan Geng, Jiayu Wang, Dahua Lin, and Kai Chen. Lean workbook: A large-scale lean problem set formalized from natural language math problems. Advances in Neural Information Processing Systems, 37:105848–105863, 2024.arxiv.org/abs/2406.03847
  28. [28]Ji Zeng, Dayuan Fu, Tiantian Mi, Yumin Zhuang, Yaxing Huang, Xuefeng Li, Lyumanshan Ye, Muhang Xie, Qishuo Hua, Zhen Huang, et al. davinci-dev: Agent-native mid-training for software engineering. arXiv preprint arXiv:2601.18418, 2026.arxiv.org/abs/2601.18418
  29. [29]Charlie Zhang, Graham Neubig, and Xiang Yue. On the interplay of pre-training, mid-training, and rl on reasoning language models. arXiv preprint arXiv:2512.07783, 2025.arxiv.org/abs/2512.07783
  30. [30]Xueliang Zhao, Wenda Li, and Lingpeng Kong. Decomposing the enigma: Subgoal-based demonstration learning for formal theorem proving. arXiv preprint arXiv:2305.16366, 2023.arxiv.org/abs/2305.16366
  31. [31]Kunhao Zheng, Jesse Michael Han, and Stanislas Polu. Minif2f: a cross-system benchmark for formal olympiad-level mathematics. arXiv preprint arXiv:2109.00110, 2021.arxiv.org/abs/2109.00110
  32. [32]Wenhong Zhu, Hongkun Hao, Zhiwei He, Yun-Ze Song, Jiao Yueyang, Yumeng Zhang, Hanxu Hu, Yiran Wei, Rui Wang, and Hongyuan Lu. Clean-eval: Clean evaluation on contaminated large language models. In Findings of the Association for Computational Linguistics: NAACL 2024, pp. 835–847, 2024.arxiv.org/abs/2311.09154

Paper details

vlnc:W_757xcc2y
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 5

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

generative AI tools

View statement
Looking for related papers…
Contents