betaFeedback

WorksComputer Science

OpenFrontiers: A Collection of Open Research Problems and a Benchmark for Scientific Rediscovery

Abstract

Large language models (LLMs) have shown increasing promise in research-level problem solving, yet a comprehensive collection of such problems is still missing, and evaluating them on research-level problems remains challenging. We introduce OpenFrontiers, a cross-disciplinary resource that combines a structured corpus of open research problems with a retrospective benchmark of recently resolved problems in mathematics. The corpus records problem statements, context, progress, and other information for 12,151 problems. The benchmark contains 200 problems, 110 resolved in 2026 and 90 still open, each with a Lean~4 statement checked by automated probes and two independent reviews; a subset HQ-50 further filters based on quality score, etc. Our evaluation protocol grades formal proofs with two independent checkers in an isolated sandbox under closed-book, frozen-retrieval, and live-web conditions. Evaluating seven LLM agents on the HQ-50 subset under a standardized budget, we find that closed-book rediscovery is rare, at most 7 of 40 resolved problems, whereas live-web success mostly reflects retrieval of published solutions. By linking a broad research-problem corpus to a formally graded evaluation subset with documented solutions, OpenFrontiers provides a foundation for measuring research-level rediscovery and studying progress toward solving currently open scientific problems. The dataset and benchmark could be found in https://huggingface.co/collections/FlippyDora/openfrontiers.

Introduction

Large language models (LLMs) have achieved perfect performance in extremely hard theoretical science tests, like IMO, and are now increasingly directed at problems that nobody has solved yet: systems built around them have produced formally checked proofs of olympiad problems (Hubert et al., [25]) and new constructions or improved bounds for research questions (Romera-Paredes et al., 2024; Novikov et al., [33]; Georgiev et al., [20]), and they appear in the solution materials of many problems examined below. Measuring the ability to resolve open problems is harder than measuring exam performance: expert-written suites such as FrontierMath (Glazer et al., [21]), Humanity’s Last Exam (Phan et al., [36]) and GPQA (Rein et al., 2023) grade against a reference answer their authors already knew; competition sets saturate quickly and appear in training data (Balunović et al., [5]; Sainz et al., 2023; Roberts et al., 2024); and proof-level grading shows that final-answer accuracy overstates what a model has established (Petrov et al., [35]). A research problem, by contrast, is defined by the absence of a reference answer.

Measuring research-level ability therefore needs two things: a map of what is open, and tasks that can be graded at all. Open problems are scattered across problem lists, workshop reports, survey sections, prize pages, blogs and many other platforms (Bloom, [6]; The Formal Conjectures Authors, 2025; TheoremDB, 2026; Ju et al., [26]) with differing conventions for wording, status and licensing, and assembling them without rewording statements or overstating status is itself a curation problem. A formerly open problem can be graded only against a solution that appeared after the evaluated model was frozen, so tasks must be time-anchored (White et al., 2025): the model receives the formulation that was public before resolution, and the located solution is evaluator-only.

We introduce OpenFrontiers, which has three coupled parts (Figure 1). The OpenFrontiers corpus (§ 3) is a large corpus of 12,151 problem records from 62 source collections across 10 top-level fields and 32 sub-fields, with MSC 2020 codes on 65.9% of records and Lean 4 statements attached to 1,946. Uncertain status is labeled unassessed (46.0% of records) rather than promoted to open. (50.7%).

Figure 1. Overview of OpenFrontiers: registered sources are fetched and normalized into a provenance-first corpus (Section 3); curation yields the 200 Lean-stated items of OpenFrontiers-Bench (Section 4); models are evaluated under three access conditions and graded by two Lean checkers (Section 5).

Based on the open problems collection, we build a benchmark including problems that are still open and those resolved recently. OpenFrontiers-Bench holds 200 research-level mathematics problems, each with a Lean 4 statement that compiles in a pinned environment and has passed automated verifications and two independent reviews. Rediscovery-110 has 110 documented research questions that remained open into 2026 and now have a located solution, 85 with an independently checked Lean proof or exact certificate (an explicit finite object, such as a counterexample, recomputed with exact arithmetic), and Open-90 has 90 problems still open. Constructing Rediscovery-110 yielded an observation: for 55 of the 110 the solution materials credit an AI system with the key mathematics, and most checked solutions have a public Lean proof. A model whose lineage produced a solution cannot be said to rediscover it, so post-cutoff dating alone does not control contamination, and the protocol (Section 5) excludes such items per model and makes closed-book the main condition.

We evaluate seven LLM agents on HQ-50, a high-quality 50-item subset of OpenFrontiers-Bench, under closed-book, frozen-retrieval and live-web conditions, grading only Lean proofs accepted by two independent checkers. Closed-book, the agents solve 0–7 of 40 solved items, mostly three human-solved items with short constructions; with live web they solve up to 27, nearly always after fetching the published solution, and no open problem was solved without one. Frontier agents also tend to give up easily without reaching the time budget if the problem is too hard. To summarize, our contributions are:

  • Open problems corpus. A cross-disciplinary, diverse 12,151 open-problem records with verbatim statements, resolve status, Lean 4 attachments, source, and other information etc.

  • Benchmark. OpenFrontiers-Bench: 200 items with checked Lean 4 statements, evidence tiers and per-item date and AI-disclosure audits, and the HQ-50 subset.

  • Evaluation protocol. We evaluate the models’ open problem solving ability under three access conditions in an isolated sandbox, grading by two independent Lean checkers, and per-model lineage and cutoff exclusion.

  • Experiment results. We conduct different evaluations on our benchmark with seven LLM agents, showing that closed-book rediscovery is rare and that live-web success reflects retrieval of published solutions.

Related Work

Benchmarks for mathematical and scientific reasoning

LLM reasoning has been tracked with competition datasets such as GSM8K [11], MATH [24], OlympiadBench [23] and Omni-MATH [18], and with expert-written science questions in GPQA (Rein et al., 2023), TPBench [9], SciCode (Tian et al., 2024) and LAB-Bench [29]. Against contamination of static benchmarks [16] (Sainz et al. 2023) and performance drops on post-cutoff material (Roberts et al., 2024), FrontierMath [21] and Humanity’s Last Exam [36] hold out expert questions, while MathArena [5], Real-Math (Zhang et al., 2025) and LiveBench White et al., 2025) refresh items from new competitions and papers. Most grade a final answer or a unit-tested program on items written for the benchmark; FrontierMath: Open Problems [17] and OEIS Open [2] pose actually open questions but accept only machine-checked answers. OpenFrontiers also uses questions that were actually open, dates them by located solution evidence and admits expert-reviewed proofs.

Proof-level and formal evaluation. Graded as full proofs, recent olympiad problems yield few complete and rigorous arguments even from models that lead final-answer competitions ([35]), motivating proof-graded sets such as IMO-ProofBench ([32]). Formal benchmarks such as miniF2F ((Zheng et al., 2022)), ProofNet ([4]) and PutnamBench ((Tsoukalas et al., 2024)) replace the grader with a proof assistant, most often Lean 4 ([14]) with mathlib ((The mathlib Community, 2020)), and dedicated provers followed: AlphaGeometry ((Trinh et al., 2024)), AlphaProof ([25]), DeepSeek-Prover-V2 ((Ren et al., 2025)), Kimina-Prover ((Wang et al., 2025)), Goedel-Prover ([30]), Seed-Prover ([8]) and Aristotle ([1]); agentic systems such as Danus ([31]) go further and coordinate prover agents with an independent verifier around a memory of verified facts to attack research-level problems directly. Since only 16.0% of corpus records carry a Lean 4 statement, OpenFrontiers-Bench writes and reviews a pinned statement for each of its items, and our protocol grades solutions with two independent Lean checkers (§ 5.2).

Open-problem collections. Curated problem lists (the Kourovka Notebook ([27]), Kirby’s list ([28]), Smale’s problems ((Smale, 1998)), the Clay Millennium Prize Problems ([10]), The Open Problems Project ([15]), Ben Green’s list ([22]), the AIM workshop lists ([3])), community venues (the Erdős Problems site ([6]), the Open Problem Garden ([34]), Wikipedia’s subject lists ((Wikipedia contributors, 2026)), the COLT open-problem track ([12]) and structured databases (TheoremDB ((TheoremDB, 2026)), ICM Conjectures ([26]), Open Quantum Problems ([7]), Formal Conjectures ((The Formal Conjectures Authors, 2025)), the Optimization Constants ([13]) and AlphaEvolve ([19]) repositories) are each bound to one community, one format and one notion of “open”, and their statuses drift as problems are solved. OpenFrontiers registers 71 such sources under one schema with verbatim statements, per-record provenance and license text, and a status policy that labels uncertain items unassessed rather than open (§ 3); Erdős Problems and Formal Conjectures also feed OpenFrontiers-Bench (§ 4).

The OpenFrontiers Corpus

Open problems may be gradually solved over time; therefore, we cut off the status filtering to a particular date. The corpus snapshot used here holds 12,151 problem records from 62 source collections in 10 top-level fields, plus 4,751 catalogue items (source indices and pointers without statements). We did not perform deduplication based on statements in order to preserve the original problems.

Sources and acquisition

Source registry. To increase the diversity of the open-problems collection, we search as many sources as possible. The registry lists 71 sources (problem lists, workshop lists, preprints and surveys with open-question sections, prize lists, blogs and Wikipedia subject lists), of which 62 contribute problem records; the rest, such as the Kourovka Notebook ([27]) and Ben Green’s 100 problems ([22]), are catalogue-only, with bibliographic pointers and indices collected but no statements. Seven collections have at least 100 records: TheoremDB (3,909) ((TheoremDB, 2026)), the AIM Problem Lists (3,470) ([3]), ICM Conjectures (2,898) ([26]), Erdős Problems (634) ([6]), Formal Conjectures (459) ((The Formal Conjectures Authors, 2025)), Optimization Constants (115) and Open Quantum Problems (114) ([7]); eight more, including The Open Problems Project ([15]), have between 22 and 59, and the remaining 47 (including 13 Wikipedia lists ((Wikipedia contributors, 2026))) have at most 20 records each; all 62 are listed with license notes in Table 6 (Appendix § 8). The verified-colt collection (30 records: 14 open, 16 partially solved) comes from a two-reviewer fidelity audit of the 118 COLT open-problem papers of 2003–2026 ([12]).

Source collectionRecordsLicense / reuse note
TheoremDB3,909Selected packets CC0-1.0; references keep their own terms
AIM Problem Lists3,470Editorially moderated; per-list terms
ICM Conjectures2,898Attribution retained; consult original proceedings
Erdos Problems (Thomas Bloom)634See site; statements paraphrased with attribution
Formal Conjectures459Code Apache-2.0; text CC BY 4.0; third-party terms kept
Optimization Constants in Mathematics — Davis, Ivanisvili & Tao115Apache-2.0 repository; pages and bound tables attributed
Open Quantum Problems114Attribution retained; consult source for reuse terms
AlphaEvolve Repository of Problems — Georgiev et al.59Apache-2.0 repository; pages attributed
Epoch AI — FrontierMath: Open Problems49CC BY 4.0 (Epoch AI); pages and prompts attributed
Tableaux — A notebook of open problems48CC BY 4.0 (D. R. Worley); HTML to Markdown, attributed
Queueing, Predictions, and LLMs — Mitzenmacher & Shahout42CC BY 4.0; attributed adaptation, HTML to Markdown
The Open Problems Project (computational geometry)38Academic use with attribution
Community consensus34Original wording
COLT Open Problems — supplied archive30Authors keep copyright; papers and audit notes attributed
Tianyuan Workshop — Open Problems in Mathematical Logic22CC BY 4.0; attributed adaptation, HTML to Markdown
Jacobs — Twenty Open Problems in Quantum Control20Author attribution and references kept; per-paper terms
Anton Petrunin — A Gallery of Geometry Problems15CC0 1.0 (A. Petrunin); independent summaries of v2
Randomstrasse101 — Open Problems of 202515CC BY 4.0; Bandeira et al.; HTML to Markdown, attributed
Aaronson — Quantum Query Complexity12Author attribution and references kept; per-paper terms
Oberwolfach 2025 — Analytic Number Theory Problem Session11CC BY-SA 4.0; compiled by T. F. Bloom; share-alike kept
FOCS 2024 — Quantum Learning10Questions summarized with attribution; slides linked
Simons Institute — Lattices ’22 problems10Summarized with attribution to the Lattices Program
Dagstuhl 25041 — Graph Classes and Algorithmic Complexity8CC BY 4.0; attributed editorial adaptation
João Araújo — A survey on varieties generated by small semigroups and a companion website8Brief editorial summaries; article keeps its own license
Wikipedia — Computer science7CC BY-SA 4.0; adapted selections, contributors attributed
Clay Mathematics Institute — Millennium Prize Problems6Statements paraphrased; official formulations linked
Wikipedia — Loop theory and quasigroup theory6CC BY-SA 4.0; adapted selections, contributors attributed
Bertrand Clarke — A Cheat Sheet for Bayesian Prediction5Brief editorial summaries; article keeps its own license
Wikipedia — Astronomy5CC BY-SA 4.0; adapted selections, contributors attributed
Source collectionRecordsLicense / reuse note
Wikipedia — Biology5CC BY-SA 4.0; adapted selections, contributors attributed
Wikipedia — Mathematics5CC BY-SA 4.0; adapted selections, contributors attributed
Wikipedia — Physics5CC BY-SA 4.0; adapted selections, contributors attributed
Dagstuhl 25081 — Semirings in Database Theory4CC BY 4.0; attributed editorial adaptation
JMLR — Pre-trained Gaussian Processes for Bayesian Optimization4CC BY 4.0; attributed editorial adaptation
Open Problems in Mathematical Biology — Vittadello & Stumpf4CC BY-SA 4.0; attributed adaptation, HTML to Markdown
Oğuz Şavk — A survey of the homology cobordism group4Brief editorial summaries; article keeps its own license
Wikipedia — Geoscience4CC BY-SA 4.0; adapted selections, contributors attributed
Wikipedia — Information theory4CC BY-SA 4.0; adapted selections, contributors attributed
Wikipedia — Neuroscience4CC BY-SA 4.0; adapted selections, contributors attributed
Alessandro Carlotto — The general relativistic constraint equations3Brief editorial summaries; article keeps its own license
Dagstuhl — Quantum Error Correction Meets ZX-Calculus3CC BY 4.0; section authors attributed
FDIG 2025 — Open Problems in Information Geometry3Independent short summaries and pointers; no blanket license
Liouville Quantum Gravity — Holden & Sun3CC BY 4.0; attributed adaptation, HTML to Markdown
Robert Benedetto — Current trends and open problems in arithmetic dynamics3Brief editorial summaries; article keeps its own license
Universality in Computable Dynamical Systems — Old and New3Independent short summaries and pointers; no blanket license
Wikipedia — Chemistry3CC BY-SA 4.0; adapted selections, contributors attributed
Andrii Mironchenko — Input-to-state stability of infinite-dimensional systems: recent results and open questions2Brief editorial summaries; article keeps its own license
Armin Schikorra — Invitation to H-systems in higher dimensions: known results, new facts, and related open problems2Brief editorial summaries; article keeps its own license
Axial Algebras — Questions and Conjectures2Independent short summaries and pointers; no blanket license
Ewin Tang — Quantum Learning Questions2Author attribution retained; consult source for terms
Open Problem Garden2Community wiki
SIAM Optimization — Open Problems in Optimization2Independent short summaries and pointers; no blanket license
Simons Workshop — Linear Systems and Eigenvalue Problems2Independent short summaries and pointers; no blanket license
Smale’s problems for the 21st century2Paraphrased with citation
Wikipedia — Economics2CC BY-SA 4.0; adapted selections, contributors attributed
Wikipedia — Fair division2CC BY-SA 4.0; adapted selections, contributors attributed
Wikipedia — Statistics2CC BY-SA 4.0; adapted selections, contributors attributed
arXiv preprints2Per-paper license; metadata via arXiv API
JMLR — Fast Rates in PAC-Bayes1CC BY 4.0; attributed editorial adaptation
Meta-Learning Kirby’s problem list in low-dimensional topology1Paraphrased with citation
Nathan Segerlind — The complexity of propositional proofs1Brief editorial summaries; article keeps its own license
Xavier Blanc — The crystallization conjecture: a review1Brief editorial summaries; article keeps its own license
Total (62 collections)12,151

Table 6. Table 6 continued

Acquisition and formulation. To collect the data from various websites, we developed several tools as deterministic fetchers and record checksums of retrieved files. Besides, we use a unified format to store the problems. One schema covers all problem kinds; a problem item stores the verbatim statement (Markdown with LaTeX), field, MSC 2020 codes with their mapping basis, status, source and license wording, references, provenance (import and status-as-of dates), quality flags, Lean 4 attachments and partial-progress context (94.9% of records; Appendix § 12).

Status policy. Status is one of open, unassessed, partially-solved, solved, disputed or withdrawn. Historical workshop lists (AIMPL) and backlog items default to unassessed because their sources assert no current status; the corpus does not claim that they are still open. Every label records the source’s status claim as of the stored status-as-of date, not a local check: an open record has not been verified to be open, and source-side “solved” or “verified” labels are kept as claims in source_status_json, keeping the original statement.

Field classification and Lean 4 statement. To facilitate the fine-grained evaluation of models on the open problems corpus, we classify the problems according to the MSC 2020 code. Each MSC 2020 code assignment records its basis and an evidence string: source-map (2,665 assignments, from the source’s own subject category), community-map (5,241, from the record’s field label) or editorial (297, curator-assigned). Lean 4 statements ([14] (The mathlib Community 2020)) are attached from TheoremDB, the Formal Conjectures repository (pinned commit b82b08f, Lean 4.33.1) and the Erdős-problem files. They are display-only: no Lean is executed at export time, lexical sorry/admit/axiom flags are recorded instead, and formal checking happens only in the platform sandbox (Appendix § 10) and the benchmark grader (§ 5.2).

Corpus statistics

Coverage. Table 1 and Figure 2 summarize the status of OpenFrontiers. Mathematics accounts for

QuantityValue
Problem / catalogue items12,151 / 4,751
Sources registered / contributing71 / 62
Collections with ≥100\ge100 records7
Distinct source names6,294
Records from problem / workshop lists8,349 / 3,516
Top-level fields / sub-fields10 / 32
Mathematics records11,349 (93.4%)
Status open6,162 (50.7%)
Status unassessed5,591 (46.0%)
Status partially-solved354 (2.9%)
Status solved/disputed/withdrawn41 / 2 / 1
Records with ≥1\ge1 MSC 2020 code8,011 (65.9%)
Distinct MSC codes / top-level classes218 / 44
Records with a Lean 4 statement1,946 (16.0%)
Lean attachments / flagged as placeholder1,961 / 1,562
References / records with ≥1\ge121,739 / 99.7%
Statement words: mean / median / p9065.4 / 39 / 148
Records with LaTeX math10,570 (87.0%)
Records with full source text1,507
Posed year known (range 1611–2026)3,389 (27.9%)

Table 1. Key figures of the corpus (problem partition unless stated; snapshot 2026-10-10; source records, not deduplicated conjectures).

Corpus statistics for status, MSC 2020 classes, and posed decade

Figure 2. Corpus statistics for the 12,151 problem records: (a) status; (b) top-10 MSC 2020 top-level classes by records with at least one code in the class (8,011 records carry a code); (c) posed decade for the 3,389 records with a known posed year (91 records before 1950 omitted).

11,349 records (93.4%); the 802 others are mostly physics (339), computer science (285) and machine learning (70), with at most 36 records in each remaining field, so mathematics still accounts for the most formalized open problems. Within mathematics, number theory (2,867 records) and geometry and topology (2,371) lead the 32 sub-fields, and the MSC view (Figure 2b) is led by 11 Number theory and 05 Combinatorics.

Status. Half of the records are open (50.7%) and nearly as many are unassessed (46.0%); the latter reflects the status policy rather than the problems, since AIMPL (3,465), TheoremDB (1,243, largely backlog candidates) and Formal Conjectures (459) account for 5,167 of them. The open records come mainly from TheoremDB (2,644), ICM Conjectures (2,594) and Erdős Problems (633); 304 of the 354 partially-solved records come from ICM Conjectures, and only 41 are solved, since resolved material lives mostly in the catalogue partition (1,042 of its 4,751 items are labeled solved by their source).

Formal material, text and time. Of the 1,946 records with a Lean 4 statement, almost all from TheoremDB (1,109), Formal Conjectures (459) and Erdős Problems (374), 1,562 are flagged formalization-contains-placeholder (a lexical sorry or admit), and no record carries a canonical accepted proof or a locally verified Lean proof

(Appendix § 10). Statements are short and most are summaries or excerpts: only 1,507 records carry the full source text. References are nearly universal (1.79 per record). Among the 3,389 records with a known posed year, the count per decade (Figure 2c) rises from 260 in the 1970s to 869 in the 2010s, with 878 already in the 2020s.

OpenFrontiers-Bench: A Retrospective Rediscovery Benchmark

OpenFrontiers-Bench is the evaluation subset of OpenFrontiers: 200 research-level mathematics problems, each with a checked Lean 4 statement. Rediscovery-110 holds 110 publicly posed research questions that remained open into 2026 and now have a located solution (ID prefixes R, E and N mark three curation rounds); Open-90 holds 90 problems still open at the status of check. HQ-50, a high-quality 50-item subset selected for reference reliability and diversity (§ 4.3), is evaluated in § 5. All 200 items could be found in Appendix § 7.

Construction

Candidates. Solved candidates came from status changes between successive versions of the corpus’s source lists (e.g., problems newly marked solved in a later Kourovka edition, Erdős Problems threads, Formal Conjectures ((The Formal Conjectures Authors, 2025)), from 2026 arXiv preprints and from public releases of AI-produced solutions; open candidates came from corpus records with an upstream Lean statement. Nine search sweeps gave 291 unique solved leads (257 verified, 230 eligible after a first pass), and 1,061 Formal Conjectures declarations were screened for the open track (152 verified, 136 kept). A second reviewer then tried to overturn each new item and rejected 5 (Appendix § 13). Re-reviewing an earlier 90-item version removed two solved items (an answer already in the literature; a refuted solution claim) and moved four open items resolved in 2026, among them the irrationality of ζ(5)\zeta(5), into Rediscovery-110.

Admission and evidence tiers. An item enters Rediscovery-110 only if it was demonstrably a public research question before its resolution, the located solution settles it with the same quantifiers and scope as posed, no earlier solution was located, and it is not a restatement of an old theorem, a withdrawn paper or an announcement without proof; same-paper and same-release items share a split group (Appendix § 9). Tier A (85 items): an exact finite certificate was independently recomputed, or a complete Lean proof was compiled from scratch (mostly AI-assisted formalizations). Tier B (21): a complete solution corroborated by the source list, a publication or an independent proof, but not re-verified by us. Tier C (4): a solution claim pending verification. All located solution-evidence dates run after 2026-01-05.

Lean statements. Each item states its target as one theorem main in a pinned environment (Lean 4.33.1, Mathlib 0df444a, Formal Conjectures 89294ea; [14] (The mathlib Community 2020)); a yes/no question is written answer (sorry) ↔\leftrightarrow P, so a solution must fill in the answer and prove main. 93 statements were taken verbatim from Formal Conjectures, 47 adapted (e.g., an upstream constant stronger than the posed problem) and 60 newly written; upstream problem files stay out of the build, since at least 35 of them record a solved item’s answer. A checker requires every file to compile with autoImplicit off, without axioms or native_decide, and with standard axioms only; the statement with the reference answer filled in compiles for all 110 solved items, leakage patterns (answers, upstream identifiers, live status links) are absent, and automation (decide, simp, omega, aesop) proves or refutes none of the 200 (Appendix § 10).

Lean 4 statements review. Two independent reviewers checked each statement, one for fidelity to the original source and one attacking its Lean semantics, with up to two repair rounds: 139 items ended ok+ok, 38 minor+ok and 23 minor+minor, with no must-fix issue. A further pass rewrote solved-item comments that hinted at the answer (47 files), keeping the comment-stripped code unchanged. These reviews were run by AI agents.

More curation criteria such as importance scores could be found in Appendix § 9.

Rediscovery-110 and Open-90

Rediscovery-110. Rediscovery-110 spans nine areas, led by group theory and number theory (19 each), graph theory (14), analysis and PDE (13) and geometry and topology (12), and includes longstanding conjectures such as Beck–Fiala, Sendov’s conjecture and the cycle double cover conjecture (Table 2). 76 items have located solution evidence after 2026-06-30, so a model frozen before July can meet them unseen. Tier A items mostly rest on a full Lean proof of the answered statement (75 of 85); the other 10 rest on an exact certificate.

Rediscovery-110Jan–JunJul–Septotal
Tier A / B / C26/8/059/13/485/21/4
Erdős Problems221840
Kourovka Notebook8816
Formal Conjectures / Wikipedia01313
Other sources (Green’s list: 2)43741
AI produced the key mathematics144155
AI assistance disclosed only142236
No AI use disclosed61319
Open-90total
Formal Conjectures / Wikipedia; Erdős; other50/28/12
Priority pilot / core / stretch4/25/61
One direction settled by a finite certificate38

Table 2. Composition of OpenFrontiers-Bench. Rediscovery-110 columns split by the located solution-evidence month of 2026. AI classes classify disclosures in the solution materials; “none disclosed” does not imply that no AI was used.

AI involvement. For 55 of the 110 items the solution materials credit an AI system with the key mathematics (OpenAI models 46, Anthropic 5, Google 4, Harmonic 3; several items name more than one), concentrated after June (41 of 76; Figure 3); 36 disclose assistance with a human-attributed solution, and 19 disclose none. A model whose lineage produced a solution has in effect seen it, and most Tier A items have a public Lean proof, so a post-cutoff date is necessary but not sufficient: the protocol excludes such items per model and makes closed-book the main condition (§ 5.1).

Located solution-evidence month of the 110 Rediscovery-110 items, by disclosed AI involvement

Figure 3. Located solution-evidence month of the 110 Rediscovery-110 items, by disclosed AI involvement (monthly totals above the bars).

HQ-50

HQ-50. HQ-50 (40 solved and 10 open items) is chosen by a reproducible script in three layers. Hard filters keep statements with high formalization confidence and a clean review, solved items of unconditional Tier A (excluding B, C and four conditional A items) with importance ≥4\ge4 (human-solved post-cutoff exceptions at 3), and open items with importance ≥4\ge4, clarity 5, an unrepaired statement and an answer that is not easy to guess; 55 solved and 48 open items qualify. A score then ranks them (review outcome, importance, proof type, recency), and greedy selection enforces caps: one item per paper or problem family, two per public release, at most 16 Erdős and 6 Kourouka items and 8 per area among solved items, and at most 21 items whose key mathematics came from OpenAI models. The 40 solved items have 34 Lean-proof and 6 certificate references, 32 post-cutoff dates, and 21 OpenAI-, 2 Anthropic-, 2 Google-, and 2 other AI-produced solutions (13 human); 8 of the 10 open items have a finite-certificate direction. We rechecked all 50 statements to confirm their open status during the LLM agent evaluation.

Evaluation Protocol and Results

Access conditions and environment

Each attempt runs one problem in a fresh sandbox holding only that problem’s Lean workspace (with prebuilt Mathlib, so the model can compile and test its own files) and Python with numpy, sympy and mpmath. Answers, the evaluator-only package and the grading workspace are not visible; every command runs without network under an unprivileged user with private namespaces, seccomp and cgroup limits; and the problem file carries an opaque ID with status-revealing source lines removed. The three conditions differ only in retrieval, which the harness performs outside the sandbox and logs: (A) closed-book, no retrieval, the main condition; (B) frozen retrieval, search and fetch over arXiv, Wikipedia, OpenAlex and MathOverflow limited to content public before 2026-01-06, one week before the earliest solution evidence, with each hit re-checked against the historical version it displays; (C) live web, all URLs logged, a leakage reference rather than a rediscovery measure, since 34 of the 40 solved items have public Lean proofs; this mainly reveals the information retrieval ability of models.

Grading

A submission is a Lean file; natural-language proofs and certificates may accompany it but are reported separately, and the main metric counts formal solutions only. Static rules admit only Mathlib imports and reject sorry, axioms, native_decide, metaprogramming and syntax extensions; a yes/no answer must be the literal True or False. The grader compiles the reference statement with the submitted answer and the submission in separate jails, replays the submission in the kernel, compares the elaborated definitions and the statement of main with the reference, and allows only propext, Classical.choice and Quot.sound. A second, independent checker, leanprover/comparator, compares every constant the statement uses and replays the whole exported environment in the kernel without loading files the submission may have written. An item is solved only if both checkers accept with the reference answer; a proof of the opposite answer or an accepted open item freezes the item for statement review, and a checker disagreement counts as unsolved. There is no model-as-judge. A grader self-test (accept and reject cases, including a compile-time file write that passes keyword bans) and comparator’s 16 official regression attacks pass. The clean subset removes, per model, items whose key mathematics came from its lineage and items with solution evidence on or before its cutoff; at a 2026-06-30 cutoff it holds 15 items for OpenAI models, 30 for Anthropic and 32 for unrelated models. We report counts with their denominators, never answer accuracy (a yes/no guess scores 50%).

Models and harness

We evaluate seven models through vendor agent CLIs that run inside the sandbox behind a gateway, which pins the model and strips server-side tools in A and B: Claude Code for Claude Sonnet 5.5, Kimi-K3, GLM-5.3, DeepSeek-V4.1-Flash and Qwen3.8-27B (128k-token output limit), and Codex for GPT-6.1-sol and GPT-6-luna. Each attempt has 3 hours of wall-clock time, 5 CPU cores and 24 GB of memory, with one attempt per item and condition. Under the native protocol the session ends when the agent stops. Because frontier agents often stop within minutes, the continuation protocol resumes the same session, stating the remaining budget, until a submission passes the pre-submission check or the budget is spent; an attempt ends as stalled after 20 consecutive (or 30 of 40) rounds without tool use (Appendix § 11).

Results

Closed-book rediscovery is rare. Models solve 0–3 of the 40 items in A under the native protocol and 0–7 under continuation (Table 3). Continuation helps most the agents that stop earliest: Claude Sonnet 5.5 rises from 0 to 4, GPT-6.1-sol from 2 to 7 and GPT-6-luna from 0 to 2 (clean 3/30, 4/15 and 2/15), while the open-weight models change by at most two items. Across the 14 runs (7 models × 2 running protocols), the 53 A and B solves fall on eight items, and the three importance-3 items of HQ-50, all solved by humans through short constructions (R01, R04, R29), account for 44 of them; of the 21 importance-5 items only one (N28, settled by a finite certificate) was solved, once. Frozen retrieval did not help systematically: B exceeds A in two runs (by one and two items) and trails it in three, consistent with a snapshot that predates every solution.

Modellineage, cutoffA: closed-book allA: closed-book cleanB: frozen retrieval allB: frozen retrieval cleanC: live web allC: live web clean
Native protocol (the agent ends the session)
Claude Sonnet 5.5Anthropic, 2026-0600/3011/3063/30
GPT-6.1-solOpenAI, 2026-0622/1522/15156/15
GPT-6-lunaOpenAI, 2026-0600/1500/1543/15
Kimi-K3–, 2026-0622/3322/331512/33
GLM-5.3–, 2026-0822/1622/16158/16
DeepSeek-V4.1-Flash–, 2026-0931/531/5130/5
Qwen3.8-27B–, 2026-0800/1900/1910/19†
Continuation protocol (resumed until the budget is spent)
Claude Sonnet 5.5Anthropic, 2026-0643/3033/302418/30
GPT-6.1-solOpenAI, 2026-0674/1553/152712/15
GPT-6-lunaOpenAI, 2026-0622/1500/15166/15
Kimi-K3–, 2026-0633/3322/331513/33
GLM-5.3–, 2026-0800/1622/16135/16
DeepSeek-V4.1-Flash–, 2026-0931/531/5152/5
Qwen3.8-27B–, 2026-0800/1900/1920/19

Table 3. Results on the 40 solved items of HQ-50 (pass@1, 3 h per attempt). all: items solved, i.e. a Lean proof with the reference answer accepted by both checkers, out of 40; clean: solved / eligible after removing items whose key mathematics came from the model’s lineage and items with solution evidence on or before the model’s cutoff (release date where none is published; – means no lineage exclusion). C mostly measures retrieval of known solutions (§ 5.4). Open items are reported separately.

Live web measures retrieval. In C models solve 1–27 items, but 167 of the 181 C solves came from attempts that fetched a located solution source (preprint, repository or problem page), often with a public Lean proof to adapt; the C–A gap thus estimates how retrievable the answers are. For example, Erdős Problem #152 (E03), solved by a Deep-Mind prover agent with a public Lean proof since 2026-04-03, was solved in none of the 28 A and B attempts but in 11 of the 14 C runs, each after fetching the problem page or its forum thread, which links to that proof. This becomes pure solution-seeking rather than scientific rediscovery. However, even in the live web setting, none of the models can solve all 40 already solved problems perfectly, showing that there is still room to improve models’ information search and retrieval ability.

Agents stop early. Natively, Claude Sonnet 5.5 and the GPT models end most attempts within minutes (median 0.3–4.1 of 180 minutes), whereas the open-weight models run longer, Qwen3.8-27B mostly until the limit. Under continuation most agents use nearly the full budget, but most Claude Sonnet 5.5 attempts stall, repeating that no progress is possible. By Table 3, under the continuation evaluation protocol, agents turn out to solve more problems compared to the native one. Therefore even models are equipped with the intrinsic ability to solve some open problems, it should be explicitly induced by appropriate prompting and harness design.

Conclusion

We presented OpenFrontiers, a systematic collection of 12,151 research-level open-problem records from 62 source collections across 10 fields, and OpenFrontiers-Bench, 200 mathematics problems with checked Lean 4 statements, graded by two independent Lean checkers under three access conditions. Upon our evaluation, seven LLM agents rarely rediscover recent solutions under the closed-book setting, and their live-web successes mostly retrieve published solutions. Even with access to the whole Internet, models do not correctly resolve all already solved problems, revealing room for better information search and retrieval ability. Moreover, although models can solve some open problems within a fixed budget, they tend to give up early before making substantive progress, highlighting the need to develop more advanced harnesses that keep the model thinking deeper and longer rather than failing to explore the open problem thoroughly.

Benchmark item lists

Tables 4 and 5 list all 200 items of OpenFrontiers-Bench; a bullet after the ID marks the 50 items of HQ-50. The problem column is the English item title, truncated where long; it is a curator title, not the text shown to a model, which receives the Lean file of the solver packet (Appendix § 11). Areas are a curator grouping into ten areas. For Rediscovery-110, TT is the evidence tier (Appendix § 9), DateDate the year and month of the located solution evidence, and AIAI the disclosure class of the solution materials: AIAI, an AI system is credited with the key mathematics; assist, assistance is disclosed and the solution is attributed to humans; none, no AI use was disclosed in the reviewed materials, which does not imply that none occurred. For Open-90, Prio. is the implementation priority (pilot, core or stretch; not a difficulty estimate) and Cert. marks items where one direction is settled by a single finite object. Solution URLs, split groups, Lean declaration names, scope differences, review verdicts and pending gates are kept in the package files.

IDProblemAreaTDateAI
R01•Lin–Wimmer rank-minimization conjecture (rank-minimization analogue of Roth’s similarity theorem)Alg./matrixA26-09none
R02Huang’s weak majorization problem for singular values of Hadamard products of nonnegative matricesAlg./matrixA26-09none
R03Teschner’s bondage-number conjectureGraph th.A26-09AI
R04•Gowers UkU^k uncertainty conjectureAdd. comb.A26-09none
R05Kourovka 21.137: if the pp-th powers of a finite pp-group form a subgroup, must it be powerful?Group th.A26-09AI
R06Queens each attacking at most one other queen (exact extremal value)Comb.B26-08none
R07•Green problem 3: measure of open product-free setsAdd. comb.A26-07assist
R08Planar Pompeiu / Schiffer conjectureGeom./top.A26-08assist
R10Kourovka 6.30: Is the profinite completion of a residually finite Hopfian group topologically Hopfian?Group th.B26-07none
R12Kourovka Notebook Problem 18.16 (definable endomorphisms of ordered abelian groups)Group th.B26-08assist
R13Kourovka Notebook Problem 21.87 (A. Lucchini): generating a finite group from dd-generator subgroups of…Group th.B26-08AI
R14Kourovka Notebook Problem 21.142 (embedding into groups invariably generated by a pp-element and a qq-element)Group th.B26-08AI
R15Kourovka Notebook Problem 12.15 (finite 2-groups in which equal normal closures force conjugacy)Group th.A26-08AI
R16Kourovka Notebook Problem 21.88 (D. MacHale): odd-order groups with k(G)/∣G∣=1/17k(G)/|G| = 1/17Group th.B26-08none
R17Open Quantum Problems 35 (seven-party case): local dimensions admitting an AME(7,dd) stateTCS/logicB26-08assist
R18Alfeld’s conjecture on vertex interpolation by C1C^1 cubic splinesAnal./PDEA26-09none
R19Periodicity conjecture for finite-dimensional algebras (Erdmann–Skowroński)Alg./matrixA26-09AI
R20Quantum Fourier Entropy–Influence (QFEI) conjecture of Bu–García–Jaffe–Koh–LiAnal./PDEB26-08assist
R21Polujan–Pott conjecture on bent functions whose translation designs have a 2-transitive automorphism groupGroup th.A26-09assist
R22Resistance diameter of a graph versus its line graphGraph th.C26-09none
R23•Gao’s conjecture ν(G)=d(G)−1\nu(G)=d(G)-1 for the van Emde Boas zero-sum invariantAdd. comb.A26-09none
R24BFGS with exact (first-local-minimizer) strong Wolfe line searches for arbitrary line-search constants 0<c1…0<c_1\ldotsTCS/logicC26-09assist
R25Friedgut’s conjecture on influential coalitions on the continuous cube [0,1]n[0,1]^nComb.C26-09assist
R26Chen–Li gap-entropy conjecture for best-arm identification (order-oblivious instance-wise bound)TCS/logicA26-09assist
R27Horn’s coincidence conjecture for commuting self-maps of a compact convex setGeom./top.B26-09none
R28Audaenert–Kittaneh: can the Aujla–Bourin subadditivity inequality for concave matrix functions be…Alg./matrixB26-09none
R29•Bourin–Lee: optimality of the constant k/4k/4 for an odd number of contractionsAlg./matrixA26-09assist
R30Decidability of reachability for branching vector addition systems (BVAS)TCS/logicC26-07none
E01•Erdős Problem #106: is f(2k+1)=kf(2^k+1)=k for the maximal total side length of squares packed in a unit square?Geom./top.A26-07AI
E02Erdős Problem #125: does A+BA+B have positive lower density (A,BA,B digits {0,1}\{0,1\} in base 3, 4)?Add. comb.A26-03AI
E03•Erdős Problem #152: isolated elements of the sunset of a finite Sidon setAdd. comb.A26-04AI
E04•Erdős Problem #193: must an infinite bounded-step walk in Z3\mathbb{Z}^3 contain three collinear points?Geom./top.A26-08assist
E05Erdős Problem #347: is there a sequence with ratio tending to 2 whose finite subsequences all have subset…Number th.A26-01assist
E06•Erdős Problem #358: is there an increasing integer sequence whose consecutive-block sums represent every…Add. comb.A26-02assist
E07Erdős Problem #387: is there an absolute constant c>0c>0 such that every binomial coefficient C(n,k)C(n,k), with 1≤1\leq…Number th.B26-05assist
E08Erdős Problem #477: is there a polynomial f:Z→Zf:\mathbb{Z}\to\mathbb{Z} of degree at least 2 and a set A⊆ZA\subseteq\mathbb{Z} such that every…Number th.A26-06AI
E09Erdős Problem #570: Ramsey number of a cycle CkC_k versus a graph with mm edgesGraph th.B26-01none
E10Erdős Problem #633: triangles that can only be cut into a square number of congruent trianglesNumber th.B26-04AI
IDProblemAreaTDateAI
E11Erdős Problem #650: matching integers to distinct multiples in an interval of length 2N2NNumber th.A26-03assist
E12•Erdős Problem #865: does density 5/8+C5/8+C force three elements with all pairwise sums in the set?Add. comb.A26-06AI
E13•Erdős Problem #868: bases of order 2 with many representations and minimal subbasesNumber th.A26-01assist
E14•Erdős Problem #987 (second question): can the exponential-sum limsups satisfy Ak=o(k)A_k=o(k)?Anal./PDEA26-04AI
E15Erdős Problem #990: does the Erdős–Turán angular discrepancy bound hold with the number of non-zero…Anal./PDEA26-04AI
E16Erdős Problem #997: well-distribution of the fractional parts {αpn}\{\alpha p_n\} along the primesNumber th.B26-03AI
E17Erdős Problem #1044: infimum of the longest boundary of a component of a polynomial lemniscateGeom./top.A26-01assist
E18Erdős Problem #1138: primes in intervals of length C times the maximal prime gapNumber th.A26-04assist
E19Erdős Problem #1148: is every large nn of the form x2+y2−z2x^2+y^2-z^2 with max⁡(x2,z2)≤n\max(x^2,z^2)\le n?Number th.A26-03AI
E20Erdős Problem #1153: the Lebesgue function on a fixed subintervalAnal./PDEA26-02assist
E21•Erdős Problem #1196: reciprocal-log sums of primitive sets far from the originNumber th.A26-04AI
E22Erdős Problem #1197: integer dilates of a set of positive measureNumber th.A26-04assist
E23Kourovka Notebook Problem 14.54: number of conjugacy classes versus nilpotent subgroupsGroup th.A26-06none
E24Kourovka Notebook Problem 17.99: faithfulness of a germ representation of the Baumslag groupGroup th.A26-06none
E25Kourovka Notebook Problem 21.8: groups generated by class transpositions of equal modulusGroup th.A26-04none
E26Kourovka Notebook Problem 21.10: just finite presentations of finite groupsGroup th.B26-05assist
E27Kourovka Notebook Problem 21.14: endomorphisms that extend along every homomorphismGroup th.B26-04none
E28Kourovka Notebook Problem 21.24: power graphs of finite groups — cograph implies chordal?Group th.A26-01AI
E29Simple groups of non-uniform exponential growth (Kourovka Notebook 21.117(a))Group th.B26-05assist
E30Rank of Z(⟨a,B⟩)∩⟨a,B⟩Z(\langle a,B\rangle)\cap\langle a,B\rangle extensions of elementary abelian groups (Kourovka Notebook 21.150)Group th.A26-03AI
N01•Beck–Fiala conjecture: is the discrepancy of degree-tt set systems O(t)O(\sqrt{t})?Comb.A26-09AI
N02Kourovka 20.76: abelian subgroups of finite pp-groups and the bound p(2k)p^{(2k)}Group th.B26-09assist
N03Deng conjectureTCS/logicA26-07assist
N04Is ζ(5)\zeta(5) irrational?Number th.A26-09AI
N05•Erdős’s unit distance problem (Erdős Problem #90): is u(n)<n1+εu(n)<n^{1+\varepsilon} for every ε>0\varepsilon>0 and all large nn?Geom./top.A26-05AI
N06Multicolour Ramsey numbers for triangles: determine lim⁡k→∞R(3;k)1/k\lim_{k\to\infty}R(3;k)^{1/k} (Erdős Problem #183)Graph th.A26-08AI
N07Benkoski–Erdős: is σ(n)/n\sigma(n)/n bounded on weird numbers? (Erdős Problem #825)Number th.A26-01assist
N08•Sendov’s conjecture (Ilieff’s conjecture): roots and critical points of polynomials whose zeros lie in the…Anal./PDEA26-08AI
N09•Hopf problem: does the 6-sphere admit a complex structure? (MathOverflow 1973)Geom./top.A26-08AI
N10•Erdős Problem #1 (Erdős–Moser): must a set of nn positive integers in {1,…,N}\{1,\ldots,N\} with distinct subset sums have…Add. comb.A26-09AI
N11Erdős–Sós conjecture on trees (Erdős Problem #548)Graph th.A26-09AI
N12•Erdős–Simonovits rational exponents conjecture (Erdős Problem #571)Graph th.A26-09AI
N13Fuglede’s spectral set conjecture in dimension twoAdd. comb.A26-07none
N14•Kőthe’s conjecture: is the sum of two nil left ideals of a ring always nil?Alg./matrixA26-09AI
N15•Navier–Stokes Millennium Problem with smooth forcing: Clay statements (C) and (D) on R3\mathbb{R}^3 and R3/Z3\mathbb{R}^3/\mathbb{Z}^3Anal./PDEA26-09AI
N16Erdős’s degeneracy conjecture for Turán numbers of bipartite graphs (Erdős Problem #146)Graph th.A26-08AI
N17•Crouzeix’s conjecture: is ∥p(A)∥≤2max⁡z∈W(A)∣p(z)∣\lVert p(A)\rVert\le2\max_{z\in W(A)}|p(z)| for every square matrix AA?Anal./PDEA26-07AI
N18•Rockafellar’s sum problem: maximal monotonicity of sums in general real Banach spacesAnal./PDEA26-09AI
N19•Is every group sofic? (Gromov–Weiss soficity question)Group th.A26-08AI
N20Ehrhart’s volume conjecture: convex bodies whose centroid is their only interior lattice pointGeom./top.A26-08AI
N21•The cycle double cover conjecture (Szekeres 1973; Seymour 1979)Graph th.A26-07AI
N22•Rota’s unimodality conjecture for the Whitney numbers of the second kind of a matroidComb.A26-07assist
N23Jaeger’s Petersen colouring conjectureGraph th.A26-07AI
N24Is every finite spherical set Ramsey? (Erdős–Graham–Montgomery–Rothschild–Spencer–Straus; Graham)Comb.A26-09AI
N25The Pierce–Birkhoff conjectureAlg./matrixA26-09AI
N26The Jacobian conjecture (polynomial self-maps of Cn\mathbb{C}^n, all nn)Geom./top.A26-07assist
N27•Wallace’s question on countably compact cancellative topological semigroupsGeom./top.A26-08AI
N28•Stallings’ equalizer conjecture: rank of the equalizer of two monomorphisms of free groupsGroup th.A26-04none
N29Michael’s problem: continuity of characters on commutative Fréchet locally m-convex algebrasGeom./top.A26-09AI
N30Pre-triangulated categories versus triangulated categories (is the octahedral axiom TR4 a consequence of…Alg./matrixB26-08assist
N31The Erdős–Simonovits compactness conjecture for finite families containing no forestGraph th.A26-08AI
N32Chvátal’s conjecture: must some largest intersecting subfamily of an independence system be a star?Comb.B26-09assist
N33•Albertson–Berman conjecture: induced forests in planar graphs (k/n≥1/2k/n\ge1/2)Graph th.A26-08assist
IDProblemAreaTDateAI
N34•Wandering domains of entire functions with bounded forward orbitAnal./PDEA26-09AI
N35Exact exponential rate of the Cohn–Elkies linear programming bound for sphere packingGeom./top.A26-08AI
N36•Erdős–Herzog–Piranian (Erdős Problem #1041): a path of length less than 2 inside {∣f∣<1}\{|f| < 1\} joining two zeros. . .Anal./PDEA26-09AI
N37•Is ∑an/2an\sum a_n / 2^{a_n} irrational for every increasing integer sequence with an/n→∞a_n / n \to\infty?Number th.A26-07assist
N38Dirac’s question for k=4k = 4: a 4-vertex-critical graph without critical edges (Erdős Problem #944, case k=4k = 4. . .Graph th.A26-09AI
N39Two-colour van der Waerden numbers versus 2k2^k: does W(k)/2k→∞W(k)/2^k \to\infty? (Erdős 1980, conjecture (1))Graph th.A26-08AI
N40•Erdős Problem #421: is there a density-one sequence all of whose consecutive-block products are distinct?Number th.A26-07AI
N41Erdős Problem #126: does the number of distinct primes dividing ∏a≠b∈A(a+b)\prod_{a\ne b\in A}(a+b), minimised over nn-element sets. . .Number th.A26-09AI
N42•Erdős Problem #123: is {akbℓcm}\{a^k b^\ell c^m\} dd-complete for pairwise coprime aa, bb, c>1c > 1?Number th.A26-07AI
N43Erdős Problem #119 (third question): partial sums of the maximum modulus of polynomials whose zeros are a. . .Anal./PDEA26-07AI
N44Wegner’s conjecture on piercing and packing of axis-parallel rectangles (Wegner 1965)Comb.A26-06assist
N45•Mahler’s problem on transcendental entire functions mapping Liouville numbers to Liouville numbersNumber th.A26-09assist
N46•Is the McEliece–Rodemich–Rumsey–Welch bound (1.4) on the rate of binary codes strict at every δ\delta?TCS/logicA26-08AI
N47Erdős–Gimbel: does χ(G)−ζ(G)\chi(G)-\zeta(G) tend to infinity for the random graph G(n,1/2)G(n,1/2)?Graph th.A26-07assist
N48Talagrand’s convexity problem: does the 10-fold sumset of a balanced compact set of Gaussian measure ≥0.99\ge0.99. . .Add. comb.B26-05AI
N49Grünbaum’s hyperplane mass-equipartition problem in dimension 4: do four hyperplanes always cut a mass in R4\mathbb{R}^4. . .Comb.A26-08assist
N50Shapiro’s field-independence conjecture for Hurwitz sum-of-squares formulas: does the existence of an $[r,s. . . | Number th. | A | 26-05 | assist
N51Arnold’s problem: algorithmic decidability of Lyapunov stability of the origin for polynomial vector fields. . .Anal./PDEA26-09AI
N52•Courtade–Kumar most informative Boolean function conjectureTCS/logicA26-09AI

Table 4. Table 4 continued

IDProblemAreaPrio.Cert.
O01Erdős Problem #242 (Erdős–Straus conjecture): for every n>2n > 2, is 4/n4/n a sum of three distinct unit fractions?Number th.coreyes

Table 4 (PDF p. 14). Rediscovery-110 (110 items; R, E, and N stand for three rounds of data selection with different cutoff dates). • = in HQ-50. T = tier; Date = located solution evidence (YY-MM); AI = disclosure class.

IDProblemAreaPrio.Cert.
O02•Lehmer’s totient problem: is there a composite nn with φ(n)∣n−1\varphi(n) \mid n-1?Number th.coreyes
O03Erdős–Moser equation: does 1k+2k+⋯+(m−1)k=mk1^k + 2^k + \cdots+ (m-1)^k = m^k have any positive integer solution other than (k,m)(k,m). . .Number th.coreyes
O04Kurepa’s left factorial conjecture (prime version): no odd prime pp divides !p=0!+1!+⋯+(p−1)!!p = 0! + 1! + \cdots+ (p-1)!Number th.core
O05Singmaster’s conjecture: bounded multiplicity of entries (other than 1) in Pascal’s triangleNumber th.stretch
O06Perfect cuboid existenceNumber th.coreyes
O07Conway’s 99-graph problemGraph th.pilot
O08Graceful tree conjecture (Ringel–Kotzig conjecture)Graph th.core
O09Sheehan’s conjecture: every 4-regular Hamiltonian graph has a second Hamiltonian cycleGraph th.core
O10Reed’s omega, delta, and chi conjectureGraph th.core
O11Vizing’s domination conjectureGraph th.core
O12Union-closed sets conjecture (Frankl’s conjecture)Comb.core
O13The 1/31/3–2/32/3 conjectureComb.core
O14Barker sequences have length at most 13 (the Barker conjecture)Comb.pilotyes
O16Hadamard conjecture: a Hadamard matrix of order 4k4k exists for every positive integer kkComb.core
O17The Gerstenhaber problem for three commuting matricesAlg./matrixpilot
O18Marcus–de Oliveira determinantal conjectureAlg./matrixcore
O20Kaplansky zero divisor conjectureNumber th.stretch
O21Černý conjectureTCS/logicstretch
O22•Scholz addition chain conjectureTCS/logiccoreyes
O24Rule 30 center column not eventually periodic (Rule 30 Prize, Problem 1)TCS/logicstretch
O25Rational distance problem (unit square)Geom./top.core
O26Lonely runner conjectureGeom./top.stretch
O27Inscribed square problem (Toeplitz square peg problem)Geom./top.stretch
O29Irrationality of the Euler–Mascheroni constantNumber th.stretch
O30Littlewood conjectureAnal./PDEstretch
O31Zariski Cancellation Problem for affine space in characteristic zeroGeom./top.stretchyes
O32•Erdős Matching Conjecture: maximum number of edges of an rr-uniform hypergraph with no kk pairwise disjoint. . .Comb.stretchyes
O33•Erdős–Szekeres “Happy Ending” conjecture: f(n)=2n−2+1f(n) = 2^{n-2} + 1Geom./top.stretchyes
O34Gallai’s path decomposition conjecture (Erdős Problem #583)Graph th.stretchyes
O35Erdős–Selfridge odd covering problem: is there a covering system with distinct odd moduli >1> 1?Number th.stretchyes
O36•Prime power conjecture for finite projective planes (Erdős Problem #723)Comb.stretchyes
O37Maximum number of mutually unbiased bases in C6\mathbb{C}^6Comb.stretch
O38•Borsuk’s partition problem in dimensions 4≤n≤624 \le n \le62 (prove or disprove)Geom./top.pilotyes
O39Ryser’s conjecture: every Latin square of odd order has a transversalComb.stretchyes
O40Kummer–Vandiver conjecture: pp does not divide the class number of Q(ζp)+\mathbb{Q}(\zeta_p)^+Number th.stretchyes
O41Sidorenko’s conjecture: t(H,G)≥t(K2,G)e(H)t(H,G) \ge t(K_2,G)^{e(H)} for every bipartite graph HHGraph th.stretchyes
O42Mazur rotations problem (Banach–Mazur rotation problem): is every separable infinite-dimensional Banach space. . .Anal./PDEstretch
IDProblemAreaPrio.Cert.
O43Erdős Problem #159: R(C4,Kn)≪n2−cR(C_4,K_n) \ll n^{2-c} for some c>0c>0Graph th.stretch
O44Erdős Problem #20 (Erdős–Rado sunflower conjecture): is f(n,k)<cknkf(n,k)<c_k n^k?Comb.stretch
O45Erdős Problem #39: an infinite Sidon set with counting function ≫εN1/2−ε\gg_{\varepsilon} N^{1/2-\varepsilon}Add. comb.stretch
O46Chowla’s cosine problem (Erdős Problem #510): does every set AA of NN positive integers admit θ\theta with ∑n∈A…\sum_{n\in A}\dotsAdd. comb.stretch
O47Erdős Problem #572: the even-cycle Turán lower bound ex⁡(n;C2k)≫n1+1/k\operatorname{ex}(n;C_{2k})\gg n^{1+1/k} for every k≥3k\ge3Graph th.stretch
O48The Erdős–Hajnal conjecture: graphs with no induced copy of a fixed graph HH contain a clique or an independent setGraph th.stretch
O49Erdős Problem #564 (Erdős–Hajnal–Rado): is the diagonal 3-uniform hypergraph Ramsey number R3(n)R_3(n) doubly exponential?Comb.stretch
O50Erdős Problem #714: is the Kővári–Sós–Turán bound sharp, i.e. is ex⁡(n;Kr,r)≫n2−1/r\operatorname{ex}(n;K_{r,r})\gg n^{2-1/r} for every r≥2r\ge2?Graph th.stretch
O51Green’s Problem 94 (Erdős similarity problem for {1,1/2,1/4,…}\{1,1/2,1/4,\ldots\}): does every set A⊂RA\subset\mathbb{R} of positive…Anal./PDEstretch
O52Power-saving bound for 4-AP-free sets in F5nF_5^n (Green’s Problem 9(iii))Add. comb.stretch
O53•The Andrews–Curtis conjecture for balanced presentations of the trivial groupGroup th.stretch
O54Babai’s conjecture for alternating groups: diam⁡(An)\operatorname{diam}(A_n) is polynomially bounded in nnGroup th.stretch
O55Falconer’s distance set conjecture in the planeGeom./top.stretch
O56•Lehmer’s problem: Mahler measures of integer polynomials are bounded away from 1Number th.stretch
O57Auslander–Reiten conjecture for Artin algebrasAlg./matrixstretch
O58Erdős Problem #1029: does the diagonal Ramsey number satisfy R(k)/(k⋅2k/2)→∞R(k)/(k\cdot2^{k/2})\to\infty?Graph th.stretch
O59The Alon–Tarsi 7/5 short cycle cover conjectureGraph th.stretchyes
O60Friedland’s conjecture on the generic rank of complex 3-tensors in the critical rangeAlg./matrixstretch
O61Erdős–Rousseau sparse-halves problem (Erdős Problem #128): if every induced subgraph on at least ⌊n/2⌋\lfloor n/2\rfloor…Graph th.stretchyes
O62Herzog–Schönheim conjecture: in a nontrivial coset partition of a group, two subgroups have the same index…Group th.stretchyes
O63Erdős Problem #23: can every triangle-free graph on 5n5n vertices be made bipartite by deleting at most n2n^2…Graph th.stretchyes
O64Erdős Problem #364 (Erdős; Mollin–Walsh): are there three consecutive powerful numbers?Number th.stretchyes
O65Erdős Problem #324: is there an integer polynomial ff whose pairwise sums f(a)+f(b)f(a)+f(b), a<ba<b nonnegative…Number th.stretch
O66Do odd weird numbers exist? (Erdős Problem #470, first question)Number th.stretchyes
O67Erdős–Lovász Tihany conjecture: every graph with ω(G)<χ(G)=a+b−1\omega(G)<\chi(G)=a+b-1 (a,b≥2a,b\ge2) is (a,b)(a,b)-splittableGraph th.stretchyes
O68Erdős–Gyárfás conjecture: every finite graph with minimum degree ≥3\ge3 contains a cycle of length 2k2^k (k≥2k\ge2)Graph th.coreyes
O69Green’s Problem 15: is there a Lipschitz function f:N→Zf:\mathbb{N}\to\mathbb{Z} whose graph contains no 3-term arithmetic…Graph th.coreyes
O70The unfair 0-1 polynomial conjecture: if monic real polynomials P,QP,Q with nonnegative coefficients have a…Probabilitycoreyes
O71Polynomial bijection from Q×Q\mathbb{Q}\times\mathbb{Q} to Q\mathbb{Q} (MathOverflow 210603)Number th.stretchyes
O72•Kourova 8.8(b) (Anosov): a non-cyclic finitely presented group in which every element is conjugate to a…Group th.stretchyes
O73Graham’s pebbling conjecture: π(G□H)≤π(G)⋅π(H)\pi(G\square H)\le\pi(G)\cdot\pi(H) for finite connected graphsGraph th.stretchyes
O74Existence of a 3×33\times3 magic square of nine distinct squares (LaBar–Gardner problem)Number th.stretchyes
O75Gottschalk’s surjunctivity conjecture: is every group surjunctive?Group th.stretch
O76No five collinear points implies O(n2)\mathrm{O}(n^2) four-point lines (Erdős Problem #101)Geom./top.stretch
O77Green’s Problem 53: if F2nF_2^n is partitioned into KK sets AiA_i, ..., AKA_K, does some 2Ai2A_i contain a coset of…Add. comb.core
O78Green’s Problem 21 (Rado’s boundedness conjecture): is the least number of colours c(a1,…,ak)c(a_1,\ldots,a_k) needed to…Add. comb.stretch
O79Meir–Moser rectangle packing: can the rectangles 1/n×1/(n+1)1/n\times1/(n+1) (n≥1)(n\ge1) be packed into the unit square?Geom./top.stretch
O80Erdős Problem #41: does every infinite B3B_3 set A⊆NA\subseteq\mathbb{N} satisfy lim inf⁡∣A∩{1,…,N}∣/N1/3=0\liminf|A\cap\{1,\ldots,N\}|/N^{1/3}=0?Number th.stretch
O81Quadratic sensitivity conjecture: is bs⁡(f)=O(s⁡(f)2)\operatorname{bs}(f)=O(\operatorname{s}(f)^2)?Comb.stretchyes
O82Jacobson’s conjecture: in a left and right Noetherian ring, ⋂n≥1Jn=0\bigcap_{n\ge1}J^n=0Alg./matrixstretch
O83Four exponentials conjectureNumber th.stretch
O84Rudin’s conjecture: does every NN-term arithmetic progression contain only O(N)O(\sqrt{N}) perfect squares?Number th.stretch
O85Atiyah–Sutcliffe Conjecture 1: the Atiyah configuration polynomials of nn distinct points in R3\mathbb{R}^3 are linearly…Geom./top.stretchyes
O86Etingof–Ostrik finite generation conjecture (Hopf algebra case): for a finite-dimensional Hopf algebra AA…Alg./matrixstretch
O87Brennan’s conjecture: the universal integral means spectrum satisfies B(−2)=1B(-2)=1Geom./top.stretch
O88Lohn’s 2-increasing triples problem: the Gowers–Long conjecture F(n)≤n3/2F(n)\le n^{3/2}Graph th.coreyes
O89Amdeberhan–Medina–Moll conjecture: tan⁡(arctan⁡1+arctan⁡2+⋯+arctan⁡n)\tan(\arctan1+\arctan2+\cdots+\arctan n) is never an integer for n≥5n\ge5Number th.coreyes
O90Erdős Problem #66: is there A⊆NA\subseteq\mathbb{N} such that 1A∗1A(n)/log⁡n1_A*1_A(n)/\log n tends to a nonzero limit?Number th.stretch
O91Sun’s conjecture: the truncations of ∑C(2k,k)2xk\sum C(2k,k)^2x^k and of ∑C(2k,k)2xk/(k+1)\sum C(2k,k)^2x^{k/(k+1)} are irreducible over Q\mathbb{Q}Number th.coreyes
O92Sun’s conjecture: every prime pp has a primitive root g<pg<p of the form k2+1k^2+1Number th.coreyes
O93Sun’s (p4, p5, p6) conjecture: every natural number is a square plus a pentagonal number plus a hexagonal…Number th.coreyes
O94The Goormaghtigh conjecture: 31 and 8191 are the only numbers that are repunits with at least three digits in…Number th.stretchyes

Table 5. Table 5 continued

Source collections

Table 6 lists the 62 source collections that contribute problem records in the snapshot of our OpenFrontiers, sorted by record count, with a shortened form of the reuse note recorded in the source registry (metadata/sources.json); the full wording travels with every item in source.license. The other registered sources (71 are registered in total) are catalogue-only: for the Kourovka Notebook ([27]) (1,935 catalogue entries), Ben Green’s list of 100 problems ([22]), the K3 low-dimensional topology list and similar indices, only bibliographic pointers were collected, not statements, so they appear in the catalogues configuration and not in this table. Record counts are source records, not deduplicated conjectures: several collections describe the same problem, and one record may contain several sub-questions. Seven collections have at least 100 records; 47 collections have at most 20 records each. For the AlphaEvolve repository, 8 of its 67 pages (rediscovery tests of classical results, a competition problem, a programming game and a logic puzzle) are kept as catalogue rows only; the Optimization Constants and FrontierMath: Open Problems collections are imported in full.

Curation rubric

This appendix records the rules used to admit, tier, score and select benchmark items. Scores are ordinal curator judgments, not expert consensus and not predictions of model success.

Hard filters. A candidate enters Rediscovery-110 only if (i) it was demonstrably a public research question before its resolution, with a traceable source; (ii) the located solution settles the same quantifiers and scope as the posed target, so that a special case, a “sufficiently large nn” result or a strengthened hypothesis does not count; (iii) no earlier solution was located before its round’s window (after 2026-06-30 for R items, from 2026-01-01 for E and N items); (iv) the solution is not a restatement of an old theorem, a withdrawn or retracted paper, or an announcement without proof; and (v) equivalent forms, parameter variants and direct corollaries of one construction are merged into one problem family. An open item must be open in both directions at the status check, precisely posed and not a duplicate or special case of another item. Every Lean statement records its scope differences from the original question (for instance, a universe restriction that does not affect the answer).

Split groups. Items that share a solution paper, a public release or a source are bound to one split group and should be reported as a group: the OpenAI “ten advances” release (N06, N16, N19, N20, N31, N35, N46), the Epoch AI FrontierMath Erdős release (N10, N11, N12, N41), Benkoski–Erdős 1974 (N07, O66), arXiv:2609.20094 (R28, R29) and one paper solving Erdős #987 and #990 (E14, E15). All other items form singleton groups.

Evidence tiers. Tier A (85; R 14, E 23, N 48): a finite certificate was independently recomputed with exact arithmetic, or a complete Lean proof of the answered statement was compiled locally, in most cases an AI-assisted formalization; 75 Tier A items rest on a Lean proof and 10 on a certificate. Four Tier A items (R26, N13, N49, N50) are conditional, each with the condition stated in its tier reason, and are excluded from HQ-50. Tier B (21): a complete solution is corroborated by the problem list’s acceptance, publication or an independent solution, but the full proof was not independently verified. Tier C (4: R22, R24, R25, R30): a solution claim whose verification is pending.

Scores. Each item carries ordinal 1–5 curator scores for importance (1: no discernible research motivation; 3: a traceable, specialised research question; 5: a recognized central conjecture), statement clarity, formalization cost and proof cost. Importance is 3/4/5 for 22/49/39 solved items (mean 4.15) and 6/45/39 open items (mean 4.37). Scores are not combined into a composite except by the HQ-50 ranking below.

Dates. Each solved item stores the earliest located public solution date and its basis (arXiv v1, dated public proof post, problem-list edition, repository commit or manuscript date). The audit read version histories, earlier PDFs, forum threads and pinned commits, but it was not an exhaustive priority search; a later publication, simplification or formalization does not reset a date. Located dates run starting from 2026-01-05.

Leakage controls in the statements. Lean files of solved items contain no answers, upstream declaration names or live links that display status. Citations of erdosproblems.com, OEIS and Green’s problem list are given as literature references rather than live pages, Wikipedia links point to a pre-solution revision, and 16 Kourovka items cite a pre-solution arXiv version of the notebook. A second pass over the 110 solved files removed semantic hints (declarative titles became questions; background sentences pointing to the solution route were deleted). In the solver packet, items are renamed to P001–P200 in a salted order, file headers are unified and status-revealing source lines are removed.

HQ-50 selection. For HQ-50, we have several more filters compared to OpenFrontiers-Bench. Hard filters for all items: the statement compiles and passes the checker and probes, formalization confidence is high, no must-fix issue, and if both review checks raised issues are all fixed; a newly written statement must have had its primary-source wording verified. Solved items must be unconditional Tier A with importance ≥4\ge4 (importance 3 only if human-solved, post-cutoff and reviewed both passed). Open items need importance ≥4\ge4, clarity 5, both passed reviews, an unrepaired statement, an answer not easily guessed (excluding O71) and open status at the check. The most frequent failures were open-item reviews other than both passes (22), Tier B (21), open clarity below 5 (19), unfixed issues from both checks (16) and importance below 4 (21 in total); 55 solved and 48 open items qualified. The ranking adds points for review outcome (both passes +2, one passes +1), importance (5: +2; 3: −1), a full Lean reference (+2; certificate +1), recency (after 2026-07-01 +2; April–June +1), proof cost ≥4\ge4 (+1) and, for open items, a verbatim upstream statement (+1.5), a finite-certificate direction (+1) and priority; it subtracts for unread primary sources, unfixed issues, repair rounds and many scope differences. Greedy selection enforces one item per solution paper or problem family, two per public release (three for the OpenAI release of seven unrelated problems), at most 16 Erdős and 6 Kourovka solved items, at most 4 Erdős open items, at most 8 solved and 2 open items per area, and at most 21 solved items whose key mathematics came from OpenAI models, so that 19 remain for evaluating OpenAI models. Two items were excluded by hand (N03, N24; Appendix § 13).

Lean statements and checks

Corpus and platform. Corpus Lean attachments (1,961 for 1,946 problems) are display-only: no Lean is executed at export time, the recorded lexical sorry/admit/axiom scans are not a proof checker, and source-side “verified” or “solved” labels are kept as dated claims. Community submissions on the platform are checked separately (profile lean4-core-4.19.0-v1: Lean 4.19.0 with Std only, the theorem built from the stored target, only propext, Classical.choice and Quot.sound allowed, a fresh networkless container per check); a successful check attaches a receipt to the submission and never changes a problem’s status.

Benchmark package. The 200 statements form one Lake project pinned to Lean 4.33.1, Mathlib 0df444a and Formal Conjectures 89294ea, of which only the two utility libraries are vendored; a clean lake build succeeds. Each file declares exactly one theorem FrontierBench.<ID>.main, and sorry may appear only in the answer placeholder and in the proof of main. Besides the checks of § 4.1, automated probes run on main with the correct and a wrong answer for solved yes/no items, with both answers for open yes/no items and with the correct answer for value items; no probe proves or refutes a statement. As corroboration of fidelity, 50 solved items had their answered main proved in this environment from public Lean proofs or recomputed certificates.

Reviews. Two AI reviewers per item worked independently: one compared the statement clause by clause with the original source, the other tried to prove or refute it cheaply through formalization loopholes. After at most two repair rounds (178 items needed none, 20 one, 2 two), the verdicts were 139 both passes, 38 with one minor problem, and 23 with two minor problems, with no must-fix issue; formalization confidence is high for 195 items and medium for 5.

Evaluation details

Isolation. Every command of an attempt runs in a freshly built root file system that holds only the attempt’s work directory, read-only prebuilt dependencies and the Lean toolchain, under its own unprivileged user ID with all capabilities dropped, private namespaces (network included) and a seccomp filter that forbids sockets, new namespaces, mounts and ptrace. Per attempt, cgroups enforce 5 CPU cores, 24 GB of memory and 4,096 processes, and the work directory and /tmp may hold at most 10 GB. The agent CLI runs in an identical jail with a loopback-only network whose only exits are a model API gateway (which routes every request, auxiliary calls included, to the evaluated model, keeps server-side web tools only in C and removes code execution and remote MCP tools), an MCP server offering check_submission and the retrieval tools of B or C, and, in C only, a web proxy that refuses private addresses. A 20-item jail check passes on every node.

Harness and protocols. The CLIs run with their own system prompt, tools, context management and default reasoning settings. Claude Code’s output limit was raised to 128k tokens, since GLM-5.3 and Qwen3.8-27B often think longer than its default of 32k for unknown models; Qwen3.8-27B was served with sglang (262k-token context). The task prompt contains the packet README, the problem file, the submission rules and the budget. check_submission applies the grader’s static rules, compares the definitions and the statement of main with the original, compiles and lists the axioms of main, without consulting the reference answer. Under continuation, a session that ends with at least one minute left and no passing submission is resumed (claude --resume, codex exec resume) with a message stating the remaining time, the submission status and concrete tool work to try; three consecutive rounds without a successful model call end the attempt as an agent error, and the models are not told about the stall rule.

Frozen retrieval (B). The cutoff, 2026-01-06, is the same for all items and announced to the model. arXiv returns papers first submitted before the cutoff, in their latest pre-cutoff version; Wikipedia, MathOverflow and Math.SE return the last pre-cutoff revision (for Stack Exchange, only questions, answers and comments posted before the cutoff); OpenAlex shows only journal or proceedings records, because merged preprint records can carry post-cutoff abstracts; other sites are refused. Since search engines index today’s text, every hit is re-verified against the displayed historical text.

Grading. Grading runs on the evaluator’s machine with the evaluator-only answers. The compile steps run in a networkless jail whose mounts are read-only except the step’s build directory, so that code executed at compile time cannot alter the reference, the grader, the answers or Mathlib; the comparison reads the compiled modules rather than parsing source. Comparator, which exports both projects with lean4export and replays the full export, is pinned and ported to Lean 4.33, with a fix that also counts structures referenced by projections, without which its own projection attack would pass.

In C, every fetched URL is normalized (arXiv ID, DOI, repository, problem page) and compared with the item’s located solution sources, to flag attempts that fetched one.

Models and cutoffs. We used Claude Code and Codex as the agent harness. Claude Sonnet 5.5, Kimi-K3, GLM-5.3 and DeepSeek-V4.1-Flash were reached through an Anthropic-compatible bridge, Qwen3.8-27B through sglang’s Messages endpoint, and GPT-6.1-sol and GPT-6-luna through a Responses-API bridge. Grading used the lineage of each provider for the two OpenAI models and Claude Sonnet 5.5, and no lineage exclusion for the others, whose own solution credits do not occur in HQ-50. Cutoffs were 2026-06-30 for the Anthropic and OpenAI models, and the public release dates for the open-weight models (Kimi-K3 2026-06-13, Qwen3.8-27B 2026-08-05, GLM-5.3 2026-08-25, DeepSeek-V4.1-Flash 2026-09-10); the late release of DeepSeek-V4.1-Flash leaves only 5 clean items. Table 7 gives per-run details; all attempts used evalkit-0.3.

ModelProt.Cond.Median min.leanOpen acc.End reasons
Claude Sonnet 5.5nat.A0.330agent stopped 49, agent error 1
B0.4180agent stopped 48, wall clock 2
C0.4190agent stopped 48, wall clock 2
Claude Sonnet 5.5cont.A5.1330stalled 41, wall clock 4, submitted 4, agent stopped 1
B17.7470stalled 45, submitted 3, agent stopped 2
C35.9460stalled 27, submitted 23
GPT-6.1-solnat.A3.1250agent stopped 50
B3.6110agent stopped 50
C4.0280agent stopped 50
GPT-6.1-solcont.A180.0500wall clock 40, submitted 8, agent stopped 2
B180.0470wall clock 42, submitted 6, agent stopped 2
C101.1501submitted 29, wall clock 18, agent stopped 3
GPT-6-lunanat.A1.700agent stopped 50
B3.100agent stopped 50
C4.160agent stopped 49, wall clock 1
GPT-6-lunacont.A180.0360wall clock 41, agent stopped 6, submitted 2, disk quota 1
B180.0330wall clock 45, agent stopped 4, submitted 1
C180.0410wall clock 32, submitted 13, agent stopped 4, disk quota 1
Kimi-K3nat.A30.3160agent stopped 50
B26.7100agent stopped 50
C26.6280agent stopped 48, wall clock 2
Kimi-K3cont.A179.2240agent stopped 38, wall clock 8, submitted 3, stalled 1
B179.2220agent stopped 31, wall clock 12, stalled 5, submitted 2
C179.1320agent stopped 25, submitted 13, wall clock 11, stalled 1
GLM-5.3nat.A90.0170agent stopped 44, wall clock 4, agent error 2
B104.7110agent stopped 49, wall clock 1
C102.2280agent stopped 47, wall clock 2, agent error 1
GLM-5.3cont.A180.0220wall clock 26, agent stopped 24
B179.4250agent stopped 30, wall clock 18, submitted 2
C179.6381wall clock 22, submitted 14, agent stopped 14
DeepSeek-V4.1-Flashnat.A22.280agent stopped 48, agent error 2
B22.0160agent stopped 50
C14.2220agent stopped 50
DeepSeek-V4.1-Flashcont.A179.2200agent stopped 23, wall clock 14, stalled 10, submitted 3
B179.2220agent stopped 31, wall clock 10, stalled 6, submitted 3
C179.1291agent stopped 21, submitted 15, wall clock 7, stalled 7
Qwen3.8-27Bnat.A180.1210wall clock 46, agent error 3, agent stopped 1
B180.0110wall clock 30, agent error 13, agent stopped 7
C180.0180wall clock 38, agent error 6, agent stopped 5, api error 1
Qwen3.8-27Bcont.A180.0150wall clock 50
B180.0230wall clock 50
C180.0300wall clock 50

Table 7. Per-run details on HQ-50 (50 items per condition, one attempt each; 180-minute budget). Median min = median attempt wall-clock minutes; .lean = attempts that left a formal submission; Open acc. = open items accepted by both checkers, before review.

Dataset format and licensing

Format. Problems in the corpus are unified into a universal JSON Schema that documents every field, a validation report, checksums, an input manifest and other necessary information. Unknown values are null, and missing data is never converted into invented statements or proofs. Figure 4 shows an abridged problem record. It keeps the source’s own license wording, an MSC code with its mapping basis, a dated status, and an upstream Lean statement.

An abridged problem record
{"id": "problem:erdos-7",
 "record_kind": "problem",
 "title": "Erdős #7 · number theory /
   covering systems",
 "problem_statement": "Is there a distinct covering
   system all of whose moduli are odd?",
 "background": "Asked by Erdős and Selfridge ...",
 "field": {"main": "math",
   "slug": "number-theory", ...},
 "msc2020": [{"code": "11",
   "basis": "community-map",
   "evidence": "Inferred from community field:
     number-theory"}],
 "status": "open",
 "source": {"key": "erdos-problems",
   "type": "problem-list",
   "url": "https://www.erdosproblems.com/7",
   "license": "See site; statement paraphrased
     with attribution", ...},
 "provenance": {"imported_at": "2026-09-08",
   "status_as_of": "2026-09-08",
   "has_full_text": true,
   "source_record":
     "https://www.erdosproblems.com/7",
   "sha256": "71fbda4a...698e7172", ...},
 "references": [{"kind": "website",
   "citation":
     "Thomas Bloom (ed.), Erdős Problems #7",
   "url": "https://www.erdosproblems.com/7", ...},
   {"kind": "research", ...}],
 "formalisations": [{"language": "Lean 4",
   "format": "source-file",
   "commit": "b820b8fa...",
   "toolchain": "leanprover/lean4:v4.33.1",
   "license": "Apache-2.0; original file ...",
   "has_sorry_or_admit": true,
   "verification_status": "not-checked",
   "code": "... @[category research open, AMS 11]
     theorem erdos_7 : answer (sorry) ↔
       ∃ (C : StrictCoveringSystem ℤ), ∀ i,
         ¬ C.moduli i ≤ Ideal.span {2} ∧
         C.moduli i ≠ ⊤ := by
     sorry ..."},
   "proof": null, "proof_status": "not-recorded",
   "lean4_proof": null,
   "lean4_proof_status": "not-recorded",
   "quality_flags": [
     "lean-statement-from-linked-source-file",
     "formalization-contains-placeholder",
     "formalization-not-compiled-during-export"],
   "schema_version": "1.0.0",
   "snapshot_at": "2026-09-10T06:58:35.349Z", ...}

Figure 4. An abridged problem record (Erdős Problem #7) from the OpenFrontiers collection.

Licensing and intended use. OpenFrontiers is a mixed-source dataset and grants no new uniform license over third-party works. Each item keeps its source’s reuse wording in source.license and each Lean attachment its own terms in formalizations[].license; missing or ambiguous terms mean that no license was established. Terms range from CC0 and Apache-2.0 to share-alike (CC BY-SA: AIM Problem Lists, Wikipedia) and non-commercial ones (the bundled MSC 2020 taxonomy, CC BY-NC-SA 4.0), so users who need a permissive subset must filter by source key. The corpus is intended for locating research problems and evaluating research-level reasoning, not as permissively licensed training data.

Exclusions and changes

We conduct multiple rounds of benchmark filtering during construction. For example, re-reviewing the earlier version removed R09 (Kourovka 11.9; the answer was already in the literature) and R11 (Kourovka 13.23; the audit found a non-inner automorphism of the group constructed in the solution preprint), and moved four open items resolved in 2026 to Rediscovery-110: N01 (Beck–Fiala), N02 (Kourovka 20.76), N03 (Tu–Deng) and N04 (irrationality of ζ(5)\zeta(5)); IDs are not reused. The overturn review rejected five new candidates, replaced from the reserve: Erdős #986 and the rank-two Poisson conjecture for overlap with admitted items, Vaught’s conjecture because its upstream cardinal form follows trivially from the continuum hypothesis, and two special cases or duplicates of admitted items (2x2^x, 3x∈Z⇒x∈Z3^x \in\mathbb{Z} \Rightarrow x \in\mathbb{Z}; Green’s problem 28). Two items were excluded from HQ-50 by hand: N03 (two repair rounds, and the original Tu–Deng wording is known only through later papers) and N24 (its reduction rests on a single unconfirmed preprint, and the answered statement has no Lean proof). The open status will be regularly updated over time as more previously open problems are gradually resolved.

Use of AI assistants

We mainly used generative AI agents to locate candidate problems and solution materials and audit their status, write and review the Lean statements of OpenFrontiers-Bench (§ 4.1), help build the evaluation harness, grader, and statistics scripts, and polish this paper.

References

  1. [1]Tudor Achim, Alex Best, Alberto Bietti, Kevin Der, Mathis Fédérico, Sergei Gukov, Daniel Halpern-Leistner, Kirsten Henningsgard, Yury Kudryashov, Alexander Meiburg, Martin Michelsen, Riley Patterson, Eric Rodriguez, Laura Scharff, Vikram Shanker, Vladimir Sicca, Hari Sowrirajan, Aidan Swope, Matyas Tamas, and 4 others. 2025. Aristotle: IMO-level automated theorem proving. arXiv preprint arXiv:2510.01346. Developed by Harmonic.arxiv.org/abs/2510.01346
  2. [2]Tom Adamczewski. 2026. OEIS Open: How many conjectures can language models turn into theorems? arXiv preprint arXiv:2608.11941.arxiv.org/abs/2608.11941
  3. [3]American Institute of Mathematics. 2026. AIM problem lists. https://aimath.org/problemlists/. Workshop problem lists hosted on AimPL (http://aimpl.org). Accessed September 2026.
  4. [4]Zhangir Azerbayeև, Bartosz Piotrowski, Hailey Schoelkopf, Edward W. Ayers, Dragomir Radev, and Jeremy Avigad. 2023. ProofNet: Autoformalizing and formally proving undergraduate-level mathematics. arXiv preprint arXiv:2302.12433.arxiv.org/abs/2302.12433
  5. [5]Mislav Balunović, Jasper Dekoninck, Ivo Petrov, Nikola Jovanović, and Martin Vechev. 2025. MathArena: Evaluating LLMs on uncontaminated math competitions. In Advances in Neural Information Processing Systems 38 (NeurIPS 2025), Datasets and Benchmarks Track. ArXiv:2505.23281.arxiv.org/abs/2505.23281
  6. [6]Thomas F. Bloom. 2023. Erdős problems. https://www.erdosproblems.com/. Online database of problems posed by Paul Erdős, with status tracking and discussion forum. Accessed September 2026.
  7. [7]Jean-Philippe Bruneton. 2026. Open problems in quantum mechanics for AI agents. https://openquantumproblems.com/. Open Quantum Problems catalogue; literature-reviewed entries in ten research areas; source repository https://github.com/jpbruneton/open_quantum_problems. Accessed September 2026.
  8. [8]Luoxin Chen, Jinming Gu, Liankai Huang, Wenhao Huang, Zhicheng Jiang, Allan Jie, Xiaoran Jin, Xing Jin, Chenggang Li, Kaijing Ma, Cheng Ren, Jiawei Shen, Wenlei Shi, Tong Sun, He Sun, Jiahui Wang, Siran Wang, Zhihong Wang, Chenrui Wei, and 17 others. 2025. Seed-Prover: Deep and broad reasoning for automated theorem proving. arXiv preprint arXiv:2507.23726.arxiv.org/abs/2507.23726
  9. [9]Daniel J. H. Chung, Zhiqi Gao, Yurii Kvasiuk, Tianyi Li, Moritz Münchmeyer, Maja Rudolph, Frederic Sala, and Sai Chaitanya Tadepalli. 2025. Theoretical physics benchmark (TPBench)—a dataset and study of AI reasoning capabilities in theoretical physics. Machine Learning: Science and Technology, 6(3):030505. ArXiv:2502.15815.arxiv.org/abs/2502.15815
  10. [10]Clay Mathematics Institute. 2000. The Millennium Prize Problems. https://www.claymath.org/millennium-problems/. Announced 24 May 2000, Collège de France, Paris. Accessed September 2026.
  11. [11]Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168.arxiv.org/abs/2110.14168
  12. [12]Conference on Learning Theory. 2026. Open problems track of the Conference on Learning Theory (COLT). https://proceedings.mlr.press/. Open-problem papers appear in the annual COLT proceedings volumes (Proceedings of Machine Learning Research). Accessed September 2026.
  13. [13]Damek Davis, Paata Ivanisvili, and Terence Tao. 2026. Optimization constants in mathematics. https://github.com/teorth/optimizationproblems. Repository at commit 2c1968c (26 September 2026); accessed October 2026.
  14. [14]Leonardo de Moura and Sebastian Ullrich. 2021. The Lean 4 theorem prover and programming language. In Automated Deduction – CADE 28, volume 12699 of Lecture Notes in Computer Science, pages 625–635, Cham. Springer.DOI
  15. [15]Erik D. Demaine, Joseph S. B. Mitchell, and Joseph O’Rourke. 2001. The Open Problems Project. https://topp.openproblem.net/. Online compilation of open problems in computational geometry, 2001–present; originating from Mitchell and O’Rourke, Computational Geometry Column 42, Internet. J. Comput. Geom. Appl. 11(5):573–582, 2001. Accessed September 2026.
  16. [16]Chunyuan Deng, Yilun Zhao, Xiangru Tang, Mark Gerstein, and Arman Cohan. 2024. Investigating data contamination in modern benchmarks for large language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 8706–8719, Mexico City, Mexico. Association for Computational Linguistics.arxiv.org/abs/2311.09783
  17. [17]Epoch AI. 2026. FrontierMath: Open problems. https://epoch.ai/frontiermath/open-problems. CC BY 4.0. Data archive updated 7 October 2026; accessed October 2026.
  18. [18]Bofei Gao, Feifan Song, Zhe Yang, Zefan Cai, Yibo Miao, Qingxiu Dong, Lei Li, Chenghao Ma, Liang Chen, Runxin Xu, Zhengyang Tang, Benyou Wang, Daoguang Zan, Shanghaoran Quan, Ge Zhang, Lei Sha, Yichang Zhang, Xuancheng Ren, Tianyu Liu, and Baobao Chang. 2025. Omni-MATH: A universal Olympiad level mathematic benchmark for large language models. In International Conference on Learning Representations (ICLR). ArXiv:2410.07985.arxiv.org/abs/2410.07985
  19. [19]Bogdan Georgiev, Javier Gómez-Serrano, Terence Tao, and Adam Zsolt Wagner. 2025a. AlphaEvolve repository of problems. https://github.com/google-deepmind/ alphaevolve_repository_of_problems. Ac-companies arXiv:2511.02864; repository at commit 8f44745 (11 July 2026); accessed October 2026.
  20. [20]Bogdan Georgiev, Javier Gómez-Serrano, Terence Tao, and Adam Zsolt Wagner. 2025b. Mathematical exploration and discovery at scale. arXiv preprint arXiv:2511.02864.arxiv.org/abs/2511.02864
  21. [21]Elliot Glazer, Ege Erdil, Tamay Besiroglu, Diego Chicharro, Evan Chen, Alex Gunning, Caroline Falkman Olsson, Jean-Stanislas Denain, Anson Ho, Emily de Oliveira Santos, et al. 2024. FrontierMath: A benchmark for evaluating advanced mathematical reasoning in AI. arXiv preprint arXiv:2411.04872.arxiv.org/abs/2411.04872
  22. [22]Ben Green. 2025. 100 open problems. https://people.maths.ox.ac.uk/greenbj/papers/open-problems.pdf. Manuscript, circulated since 2018; version of December 2025. Accessed September 2026.
  23. [23]Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, Jie Liu, Lei Qi, Zhiyuan Liu, and Maosong Sun. 2024. OlympiadBench: A challenging benchmark for promoting AGI with Olympiad-level bilingual multimodal scientific problems. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3828–3850, Bangkok, Thailand. Association for Computational Linguistics.
  24. [24]Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring mathematical problem solving with the MATH dataset. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks (NeurIPS 2021). ArXiv:2103.03874.
  25. [25]Thomas Hubert, Rishi Mehta, Laurent Sartran, Miklós Z. Horváth, Goran Žužić, Eric Wieser, Aja Huang, Julian Schrittwieser, Yannick Schroecker, Hussain Masoom, Ottavia Bertolli, Tom Zahavy, Amol Mandhane, Jessica Yung, Iuliya Beloshapka, Borja Ibarz, Vivek Veeriah, Lei Yu, Oliver Nash, and 20 others. 2026. Olympiad-level formal mathematical reasoning with reinforcement learning. Nature, 651(8106):607–613. AlphaProof. Published online 12 November 2025.DOI
  26. [26]Haocheng Ju, Bin Dong, and Gergely Bérczi. 2026. ICMConjectures: A collection of open conjectures from the ICM proceedings. https://www.icmconjectures.com/. Announced 28 August 2026 at https://frenzymath.com/blog/icmconjectures/. Accessed September 2026.
  27. [27]Evgeny I. Khukhro and Victor D. Mazurov. 2026. Unsolved problems in group theory. The Kourovka Notebook. https://arxiv.org/abs/1401.0300. 21st edition (arXiv:1401.0300v46). Accessed September 2026.
  28. [28]Robion Kirby. 1997. Problems in low-dimensional topology. In William H. Kazez, editor, Geometric Topology (Athens, GA, 1993), volume 2.2 of AMS/IP Studies in Advanced Mathematics, pages 35–473. American Mathematical Society, Providence, RI.
  29. [29]Jon M. Laurent, Joseph D. Janizek, Michael Ruzo, Michaela M. Hinks, Michael J. Hammerling, Siddharth Narayanan, Manvitha Ponnapati, Andrew D. White, and Samuel G. Rodriques. 2024. LAB-Bench: Measuring capabilities of language models for biology research. arXiv preprint arXiv:2407.10362.arxiv.org/abs/2407.10362
  30. [30]Yong Lin, Shange Tang, Bohan Lyu, Jiayun Wu, Hongzhou Lin, Kaiyu Yang, Jia Li, Mengzhou Xia, Danqi Chen, Sanjeev Arora, and Chi Jin. 2025. Goedel-Prover: A frontier model for open-source automated theorem proving. In Second Conference on Language Modeling (COLM). ArXiv:2502.07640.arxiv.org/abs/2502.07640
  31. [31]Jih​​ao Liu, Guoxiong Gao, Zeming Sun, Bin Wu, Shurui Liu, Jiedong Jiang, Haocheng Ju, Leheng Chen, Ronnie Cheng, Xiping Zhang, et al. 2026. Danus: Orchestrating mathematical reasoning agents with fact-graph memory. arXiv preprint arXiv:2607.06447.arxiv.org/abs/2607.06447
  32. [32]Thang Luong, Dawsen Hwang, Hoang H. Nguyen, Golnaz Ghiasi, Yuri Chernovy, Insuk Seo, Junsu Kim, Garrett Bingham, Jonathan Lee, Swaroop Mishra, Alex Zhai, Huiyi Hu, Henryk Michalewski, Jimin Kim, Jeonghyun Ahn, Junhwi Bae, Xingyou Song, Trieu H. Trinh, Quoc V. Le, and Junhyuk Jung. 2025. Towards robust mathematical reasoning. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 35418–35442, Suzhou, China. Association for Computational Linguistics. Introduces IMO-ProofBench; arXiv:2511.01846.arxiv.org/abs/2511.01846
  33. [33]Alexander Novikov, Ngân Vũ, Marvin Eisenberger, Emilien Dupont, Po-Sen Huang, Adam Zsolt Wagner, Sergey Shirobokov, Borislav Kozlovskii, Francisco J. R. Ruiz, Abbas Mehrabian, M. Pawan Kumar, Abigail See, Swarat Chaudhuri, George Holland, Alex Davies, Sebastian Nowozin, Pushmeet Kohli, and Matej Balog. 2025. AlphaEvolve: A coding agent for scientific and algorithmic discovery. arXiv preprint arXiv:2506.13131.arxiv.org/abs/2506.13131
  34. [34]Open Problem Garden contributors. 2026. Open Problem Garden: A collection of unsolved problems in mathematics. https://www.openproblemgarden.org/. Accessed September 2026.
  35. [35]Ivo Petrov, Jasper Dekoninck, Lyuben Baltadzhiev, Maria Drencheva, Kristian Minchev, Mislav Balunović, Nikola Jovanović, and Martin Vechev. 2025. Proof or bluff? Evaluating LLMs on 2025 USA Math Olympiad. arXiv preprint arXiv:2503.21934.arxiv.org/abs/2503.21934
  36. [36]Long Phan, Alice Gatti, Ziwen Han, Nathaniel Li, Josephina Hu, Hugh Zhang, Chen Bo Calvin Zhang, Mohamed Shaaban, John Ling, Sean Shi, et al. 2025. Humanity’s Last Exam. arXiv preprint arXiv:2501.14249.

Paper details

vlnc:W_dgv87g9e
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 1

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

generative AI agents

View statement
Looking for related papers…
Contents