betaFeedback

WorksComputer Science

SoK: Assessing the Evolution of LLM Capability in Software Vulnerability Research

Abstract

LLM-based agents are becoming increasingly capable at discovering, validating and patching vulnerabilities in real software, and vendors and researchers report that this is already changing how vulnerabilities are found and fixed. These claims, however, are scattered across papers, vendor reports and disclosures and are rarely comparable. We introduce a framework that rates these capabilities on an autonomy scale modeled on the levels of driving automation, and qualifies each claim by its correctness, its availability and the strength of its evidence. We use this framework to systematize the published evidence on LLM capability in vulnerability research. We find that agents discovered, validated, and patched vulnerabilities in real-world software without human intervention before the restricted frontier models of 2026, and that open-weight models subsequently attained comparable autonomy for discovery and validation. We then measure the consequences of evolving LLM security capabilities in a decade of public vulnerability records, advisories and exploitation data. In 2026 the number of disclosed vulnerabilities more than doubled, with an increasing share rated critical or high, and severe bugs that had survived a decade or more in widely used code were reported several times more often than before. Yet the time between the stages of a vulnerability's life, from report to fix and from disclosure to attack, has not shifted noticeably so far. We grade the mitigations proposed in the literature with an explicit rubric and draw guidance for each group of stakeholders from the best-supported ones.

Introduction

Software vulnerability research covers discovering vulnerabilities, proving that they are real and fixing them. Automated tools such as fuzzers and static analyzers can surface large numbers of candidate bugs, yet because many of these candidates are false positives, triage, proof and repair have remained the work of human experts. Recent reports suggest that LLM-based systems are increasingly capable of carrying out these steps autonomously on real software. Google’s Big Sleep agent reported zero-day vulnerabilities in real-world code as early as 2024 [1]. By 2025, a growing number of systems, among them those of DARPA’s AI Cyber Challenge (AIxCC), were finding and patching vulnerabilities in open-source projects [2], [3]. In April 2026, Anthropic reported that Claude Mythos Preview had found thousands of high- and critical-severity vulnerabilities, almost always without human intervention, and restricted the model to selected partners [4], [5]. In July 2026, agents under evaluation at OpenAI exploited a previously unknown vulnerability to leave their environment and attacked several systems, including those of Hugging Face [6].

These reports have raised concern among practitioners, who describe a “Vulnocalypse”, a surge of findings that maintainers may be unable to absorb [7], [8]. Labs warn that exploits could follow fixes within hours [9], while others expect the supply of bugs to run out once existing code has been swept [10], [11]. Assessing these expectations requires evidence on two questions, namely what these systems have been shown to do, and what has changed so far in the vulnerability lifecycle.

Neither question is easy to answer from the published evidence. Capability reports describe tasks that range from flagging suspicious code to delivering an accepted fix, and they differ in how much a human contributed. Their correctness also varies widely, from expert precision for the best systems [12] to large numbers of invalid reports received by maintainers [13]. Many reports come from the organizations that build the systems, benchmarks tend to rely on curated targets [14], and the most capable models are often reachable only through interfaces that their developers control [15]. Statements about the consequences come from the same parties and are rarely compared with public records.

In this SoK, we address both questions and treat them as complementary. For the first, we introduce, to our knowledge, the first framework that assesses the demonstrated autonomy of LLM-based systems separately for each task of vulnerability research, over time and across levels of access. Following the Society of Automotive Engineers (SAE) levels of driving automation [16], we define each autonomy level by the part of the task that remains with the human. A level counts only when most of the results hold up, and every claim also receives an access tier, from internal lab use to open weights, and an evidence grade. To apply the framework, we built an agentic workflow that collected more than 50,000 unique sources, including academic publications, lab and vendor reports, maintainers’ accounts and incident disclosures. Independent agents searched this indexed corpus for the most relevant evidence on real software and scored it, supporting each score with a quote checked against the source. Two authors then reviewed the 128 dated steps of the resulting map, and our conclusions draw only on steps backed by direct evidence on real software. For the second question, we collected the consequences commonly expected of these capabilities from 503 verified statements by labs, vendors, maintainers, government agencies and researchers, and compared these expectations with a decade of public records. These comprise more than 300,000 CVE records [17] with their severity, the age of the vulnerable code where it is known, the first public exploits and the listings as exploited, as well as the timelines of about 1,400 coordinated disclosures and the records of disclosure programs.

RQ1: What is the current capability of LLM-based systems, in terms of autonomy and correctness, to discover, validate and patch vulnerabilities in real-world software? (Section 5.1)

RQ2: How has this capability evolved since 2024? (Section 5.2)

RQ3: How rapidly do capabilities first demonstrated by frontier-lab systems diffuse to publicly available and open-weight models? (Section 5.3)

RQ4: What has changed so far in public vulnerability records, and what has not? (Section 6)

RQ5: Which mitigations are supported by evidence, and what guidance follows for each stakeholder? (Section 7)

In addressing these questions, this paper makes four contributions.

  • Assessment framework. To our knowledge, the first framework that rates the demonstrated autonomy of LLM-based systems for four tasks of vulnerability research, together with the correctness of their results, their level of access and the strength of the evidence (Section 3).

  • Capability map. A reproducible, dated map of demonstrated capability since 2024 (Sections 4 and 5, Figure 1). Agents built it from a corpus of more than 50,000 sources, and two authors reviewed each of its steps. It shows that agents were able to discover, validate and patch real vulnerabilities without human intervention by October 2025, before the restricted frontier models of 2026, that open-weight models followed within months for discovery and validation, and that remediation is the least measured task.

  • Measurement of public records. A measurement of the volume, severity and age of disclosed vulnerabilities, and of the time to fix and to attack, in a decade of public records, compared with seven commonly expected consequences (Section 6). Disclosures more than doubled in 2026, with a growing share rated Critical or High and several times more old, severe bugs, whereas the time to fix and to attack has not shifted noticeably so far.

  • Mitigations and research agenda. A rubric-based grading of 15 proposed mitigations, followed by guidance for each group of stakeholders and open problems (Sections 7 and 8). The best-supported mitigations are established practices of secure design, whereas those built on LLMs are the least measured.

Background and Related Work

Software vulnerability research. Vulnerability research begins with selecting a target and discovering a flaw, which must then be validated, usually by a proof that it can be triggered. Once the flaw is reported, the maintainer triages it, develops and validates a patch, and releases it, and operators finally deploy the fix. Under coordinated vulnerability disclosure, the finder gives the maintainer a fixed period, commonly 90 days, to release a fix before the details become public [18]. This practice presumes that discovery is scarce, so that a reported vulnerability is unlikely to be found by someone else before it is fixed. By one estimate, only about 5.7% of privately held zero-day vulnerabilities are rediscovered by others each year [19]. Deployment has also consistently lagged behind the fix. In enterprises, a patch for client software can take more than six months to reach 90% of the machines [20].

Measuring the vulnerability lifecycle. A long line of empirical work has measured the lifecycle of vulnerabilities. Zero-day attacks have been found to last 312 days on average before disclosure [21], and in one study about 7% of 4,080 security fixes failed to fully remove the vulnerability [22]. How often coordinated disclosure achieves its intended order, with the fix available before the details are public and before attacks begin, has also been modeled and measured [23]. Whether finding vulnerabilities eventually depletes them remains an open question. No clear depletion was found in mature software [24], whereas a decline was observed in OpenBSD after years of audits [25], and vulnerabilities in open-source code can survive for years before they are fixed [26]. These studies have described a lifecycle driven by humans and fuzzers. Since 2024, LLM-based systems have also been discovering and reporting vulnerabilities in real software. We therefore examine whether this lifecycle has changed, and measure on public records the time from the report to the fix and from disclosure to the first exploit and attack (§ 6).

Levels of autonomy. The SAE levels of driving automation define each level by the part of the task that remains with the human driver [16]. Similar levels have been proposed for general-purpose models and agents [27, 28]. These levels describe how a system is designed or deployed, so the same model can be placed at several levels. We adopt the structure of these scales, but we define each level by what a system has been shown to do on real targets, separately for each task. Frontier labs have also defined their own cyber capability thresholds, which determine the safeguards that a model requires before release [29, 30, 31]. These thresholds are assessed mostly through evaluations within each lab. Our map complements them with a comparison across labs, tiers and time that is based on real-world evidence.

Surveys and measurement. Existing surveys of LLMs in software security have organized the techniques and benchmarks for vulnerability detection [32, 33, 34, 35], repair [36, 37, 38, 39] and penetration testing [40, 41], and have reviewed security agents [42]. These surveys each focus on one part of vulnerability research, and they largely report results on benchmarks. Benchmarks evaluate models on capture-the-flag tasks and on reproduced or injected vulnerabilities [14, 43, 44, 45], so their scores do not establish what a system has achieved on real software or how much of the work remained with a human. Three lines of work are closer to ours. Security agents have been rated at each stage of vulnerability research by the depth of their results [46], recent work has argued that the bottleneck has moved from discovery to remediation [47], and government evaluators have used task suites to track frontier models and the pace at which open-weight models follow [15, 48, 49, 50].

To our knowledge, no prior work has assessed all four tasks on real software by both autonomy and correctness, dated each result, graded the evidence behind it and recorded who can access the system (Table 1). The absence of such an assessment has made it difficult to compare the reports of labs, vendors and maintainers with one another, and to judge the expectations that are drawn from them. Our map provides this assessment. We further compare it with public vulnerability records and grade the proposed mitigations, which connects the demonstrated capability to its observed consequences and to the evidence for each response.

WorkTasksAutonomyEvidenceTimeAccessMitigations
CGC systems [51]D, V, P✓✗✗✗✗
Detection surveys [32, 33, 34]D✗✗✗✗✗
Repair surveys [36, 37, 38]P✗✗✗✗○
Pentest SoKs [40, 41]D, V○✗✗✗✗
CLASP [46]D, V, P✗✗✗✗✗
Bugonomics [47]D, P✗✗✗✗○
Benchmarks [14, 43, 44]D, V, P○✗✗✗✗
Lab thresholds [29, 30]D, V✗✗✗✓✗
Gov. evaluations [15, 48]D, V✗✓✓✓✗
This SoKD, V, P, VP✓✓✓✓✓

Table 1. Scope of related work. Tasks are discover (D), validate (V), patch (P) and validate patch (VP). The other columns indicate whether a work assesses the level of autonomy, grades its evidence, follows capability over time, distinguishes levels of access and assesses mitigations. ✓ yes, ○ partly, ✗ no.

Assessment Framework

Capabilities. We assess the four tasks of vulnerability research whose individual results can be checked (Table 2). Choosing targets is captured by the highest level of the autonomy scale, and we return to deployment and disclosure, which have no comparable check, in § 6. Three of the four capabilities can serve attackers and defenders alike, whereas patching is primarily defensive. Discovery and remediation each consist of a step that produces a result and a step that checks it. Discover means reporting weaknesses in a codebase, binary, or running service that prove to be real vulnerabilities, without being pointed to a candidate. Validate vulnerability means confirming or rejecting a candidate from any source and, when the candidate is real, producing a proof of vulnerability (PoV) such as a working exploit. A PoV is an input or procedure that shows that the flaw is real. Patch means delivering a fix that removes the cause without breaking the software, where passing the PoV and the tests is necessary but not sufficient [52, 53]. Validate patch means accepting correct fixes and rejecting wrong ones.

CapabilityInputOutputCorrect when
Discovery
DiscoverA codebase, binary or serviceCandidate weaknesses, each with its locationConfirmed as a real vulnerability by the maintainer, the vendor or an independent check
Validate vulnerabilityA candidate vulnerabilityA verdict, and a PoV if realReal candidates confirmed, false ones rejected
Remediation
PatchA vulnerability and its PoVA fixRemoves the root cause without breaking the software, as confirmed by the maintainer or an independent check
Validate patchA candidate patchA verdictCorrect fixes accepted, wrong ones rejected, even when they pass the tests

Table 2. Assessed capabilities, with their inputs, outputs and correctness criteria. Each stage has a producing and a checking step.

Scope. We assess vulnerability research on software, including source code, binaries, firmware, smart contracts and running web services. Our assessment ends at the proof that a vulnerability is real. This proof counts as validation even when it is a working exploit, and we report it only as an outcome. What an attacker does with the vulnerability afterwards, such as intrusion into a system, is outside our scope, as are social engineering and network configuration, which have no comparable check of correctness. Results on targets built or seeded for an evaluation, such as the vulnerabilities that the organizers of AIxCC injected into open-source projects, count as benchmark evidence.

Autonomy. The autonomy scale has six levels, from A0 to A5 (Table 3). Following the SAE levels of driving automation [16], each level is defined by the work that remains with the human. At A0, the system contributes nothing to the task. At A1, it advises a human who performs every action, and at A2 it carries out the very specific steps that a human specifies. From A3 upward, the system plans and runs the task itself, and we refer to it as an LLM-based agent. At A3, a human still prepares each target or steps in when the agent gets stuck, whereas at A4 the agent completes the task on a given target without any human intervention. A human who checks the results afterwards does not lower the level, since this check is how correctness is measured. At A5, the agent also chooses its own targets and works across many of them over long periods.

AutonomyThe system is capable of . . .
A0nothing in the task
A1advising a human, who performs every action
A2performing the steps a human specifies
A3planning and running the whole task, but not setting up each target or recovering when it gets stuck
A4completing the task on a given target without any human intervention
A5working across many targets over long periods, choosing its own work and recovering from its own failures
Correctnesshow often the delivered results hold up (precision)
C0, below the barmost results do not hold up; the level does not count
C1, capablemost results hold up, but each is still worth checking
C2, expertresults hold up as an expert’s would (the published expert rate, or about 90% when none is)
C3, audited codeexpert, on code that experts and fuzzers had examined for years; needs E2 or E3

Table 3. Autonomy and correctness scales.

Correctness. Correctness measures the capability of a system as the share of the outcomes it delivers, whether vulnerabilities, verdicts or patches, that are deemed correct by the party receiving them, such as the maintainer, the vendor or an independent reviewer Table 3. This share corresponds to the precision of the system. We chose it because every delivered result can be checked, whereas the vulnerabilities that a system misses in real software are unknown. It can also serve as a good proxy for the amount of human review that the results still require. Sources also report task completion, the share of the given targets for which a system delivers a result, and recall. We record both with each result and keep them separate from the category. Results are below the bar (C0) when most of them do not hold up, capable (C1) when most of them hold up, and expert (C2) when they hold up as often as an expert’s. We anchor expert to the expert rate published for the capability, or to about nine results in ten when none is published. Expert in audited code (C3) denotes expert precision on code that experts and fuzzers had examined for years. A system reaches an autonomy level only from C1 upward, so a level shown at C0 does not count. Where no source publishes a rate for a level that a system has demonstrated, the reviewing authors assign the category

from the converging evidence on that result, such as fixes merged by maintainers, independent confirmation and the results of the same system on benchmarks.

Access and evidence. We assign each result an access tier, based on how widely the system behind it was available at the time of the evidence. T1 covers systems available only inside the organization that built them, T2 systems available to selected partners or sold through sales contacts, T3 systems that anyone can sign up for and T4 open-weight models. The tier corresponds to the access needed to use the system as it was evaluated, including both scaffold and model, so a proprietary scaffold on a public model counts at the tier of the scaffold. The tiers are cumulative, and a level counts for a tier once a system available at that tier or more widely has reached it. We also assign each result an evidence grade. E1 covers results reported by the organization that built the system, E2 results reported with checkable details such as named targets, advisories or logs, and E3 results confirmed by an independent party. Benchmarks, capture-the-flag challenges and competitions rely on targets built for the evaluation, we treat their results as complementary evidence. More anecdotal result, such as a small sample or a figure found only in secondary press, must be supported by additional evidence or benchmark scores before it can set a level. In contrast, a report that states its denominator over hundreds of results can set a level on its own. Several results that agree can set a level together, and a result on an adjacent task, such as porting an existing fix to an older release, can serve as corroboration.

Steps of the map. We describe each result as a pair of an autonomy level and a correctness category, for example A4 at expert (A4.C2), dated from the first source that shows both values for the same system. Each combination of a capability and an access tier is a cell of the map, and each dated pair in a cell is a step.

Bases. A value is direct when a source documents it on real-world software at that tier or more widely. The principal map and the answers to the research questions rest mainly on direct values, and the figures mark the other values. We report two other kinds of values separately. A benchmark value, marked ‡, comes from a realistic benchmark built from real vulnerabilities, such as PatchEval or CyberGym. It measures task completion on prepared targets and serves only as complementary evidence. An inferred value, marked †, is a hypothesis that a level shown directly on real-world software transfers to a more widely available model. We state such a hypothesis only when three conditions hold. The model must score at or above the model of the direct evidence on several independent measures, a scaffold that reaches the level must be available for it at the tier, and independent sources must corroborate the match. For a system with a direct level but no published rate, the hypothesis takes the measured accuracy of its model in the same role. Section 5.3 reports how the map changes with and without both kinds.

Methodology

Figure 2 summarizes how we collect and score the evidence.

The evidence pipeline

Figure 2. The evidence pipeline. Lanes 2, 4 and 5 run Claude Code agents, each as a separate process. Lane 1 consists of scripts and a local LLM, lane 3 of a script, and lane 6 of the authors’ review.

Evidence Base

Search, indexing and screening. We began with a systematic search organized along the stages of the PRISMA guideline [54], from identification to screening and inclusion (Figure 7, Appendix § 10). We ran 67 queries on Semantic Scholar and arXiv and collected the reports of 105 organizations. Two LLM screeners independently judged the 13,982 records against written inclusion criteria and agreed on 94% of them (Cohen’s κ=0.87\kappa= 0.87 [55]). After following the references and citations of the included works and adding the sources found by coding agents in three harnesses (Claude Code,2 Codex3 and Muse Code4), the search yielded 6,871 sources. Because many results on frontier systems are published only in blog posts, reports and news, we added two further sources of records. A search agent with 18 connectors returned 49,949 works from scholarly indexes, news sites, security mailing lists and forums such as Hacker News. We also collected lab reports, evaluations, vendor posts, maintainers’ accounts and incident reports. After merging duplicates, the corpus contains 53,057 sources. We indexed it for search by keyword, by dense embeddings of each source and of its passages, and through its citation graph, and a learned ranker combines these results. On held-out queries, the top ten results contained 92% of the relevant sources (Appendix § 10). The scoring and review agents used this search to retrieve the evidence for each cell. Two independent agents reviewed all the evidence and marked 9,859 of them as relevant, meaning that they report a result of an LLM-based system on a vulnerability research task.

Flow of sources through identification, screening, reading, and inclusion

Figure 7. Flow of sources, in the style of a PRISMA diagram. The local LLM’s readings rank the sources for the Claude readers, and the coverage round added steps to the map.

Reading. Of the 9,859 relevant sources, 6,604 date from 2024 onwards and concern real-world software. A local LLM (Muse Glimmer 30B) read these sources and ranked them, and agents then read in full the 1,100 most likely to affect the results. For each source, they recorded the system, its autonomy level, access tier, evidence grade, date and correctness rate, and quoted a sentence from the source in support of each value. A script then confirmed that 99.6% of these quotes appear word for word in the sources. This check excludes invented quotes, and whether a quote supports its level is judged in the later stages and in the authors’ review. The scoring agents could also search the indexed corpus, and the runs that searched both the corpus and the web found no relevant source missing from it.

Scoring

The scoring gives, for each cell, the dates when each autonomy level was first reached, with the correctness reached at that level.

Scoring runs. Each scoring run scored one cell and returned its steps, each with the readings that support it. Every value had to quote its source, and a script checked the quote. Each cell was scored twice. One run started from the readings of systems available at that tier, whereas the other started without readings and searched the corpus and the web, in order to check whether the readings had missed evidence. Readers and scoring runs used Claude Sonnet 5.5, and all later agents used Claude Opus 5.5.

Decision. For each capability, a review agent compared the two runs of each cell, and a decision agent then decided the steps of each cell and justified each decision. The two runs agreed on whether a level had been reached in 83% of the comparisons. For each capability, one agent then argued for higher levels and another for lower levels, each unaware of the other. A verifier, which received their proposals anonymized and in random order, accepted 34 and rejected 30, and the remaining proposals were settled in the authors’ review.

Coverage round. To extend coverage, the local LLM read 20,631 further sources, and Claude agents checked each candidate against the source text. This round added 55 steps and revised 18 others, most often by lowering a level that were over-estimated.

Authors’ review. Two of the authors manually and independently validated the map in two stages. In the first, they scored the 16 most recent cells themselves and agreed with the pipeline on the autonomy level of 10 of them (63%). This comparison led them to state their scoring preferences, including the corroboration rule of Section 3, and the pipeline then rescored the whole map under these rules. In the second stage, they reviewed every step of the resulting map against its sources. They agreed with the pipeline on the highest level of 15 of the 16 cells (94%) and on the level of 107 of the 119 steps that they kept (90%). Where the pipeline had also assigned a correctness category, they confirmed the full pair for 82 of 94 steps (87%), and the other 12 pairs differed by at most one level and one category. For 25 steps without a published rate, they assigned the category from the converging evidence, and they added four steps. The final map contains 128 steps. Of these, 85 rest on direct evidence and form the principal map, while 35 are benchmark values and 8 are inferred values, which we report separately. The two authors agreed with each other on the autonomy level of 88% of the steps that both rated (Cohen’s κ=0.83\kappa= 0.83), a rate similar to their agreement with the pipeline. Appendix C lists every step with its sources.

Records and Statements

To answer RQ4 and RQ5, we combined statements with public records. Separate agents with web access collected statements on consequences and defenses from frontier labs and vendors, practitioners, maintainers, government agencies and the academic literature. We kept the 503 statements whose quote could be verified on its source page, and derived from them the questions that we examine and the defenses that we grade.

We measure five quantities from public records (Appendix B). The volume and severity of disclosures come from the CVE List of 31 October 2026 [17], which holds 304,028 records published since 2018 with their issuer, weakness type, severity scores and finder credits. The involvement of LLM-based systems comes from about 2,600 of these records, which name such a system as finder in the credits of advisories or in the lists published by labs and vendors. The age of the vulnerable code comes from the release that introduced each vulnerability, which can be dated for the Linux kernel, Windows, open-source packages and other software with dated releases. The time from report to fix comes from the advisories of the Zero Day Initiative since 2016 [56] and from Chrome’s public bug tracker [57]. The time to exploitation comes from the first public exploit code in Exploit-DB, Metasploit and proof-of-concept repositories on GitHub [58, 59, 60], and from the listing as exploited in CISA’s catalogue [61]. We complement these records with the public reports of disclosure programs, namely the dashboards of Anthropic and Patch the Planet [62], [63], the trackers of Project Zero and Big Sleep [64], [65] and the statistics of Syzbot [66].

Demonstrated Capability

Figure 1 shows the highest pair reached in each cell by 7 November 2026, and Figure 3 its evolution since 2024. The results below rest on direct evidence on real-world software, for the systems and targets that the sources report, and we mark the values that come from a benchmark (‡) or from inference (†). Appendix C lists the evidence behind every step.

Figure 1. Highest autonomy level of each capability by access tier on direct evidence, with its correctness category and the month when the pair was first shown on real software. Outlined marks are levels reached only on a benchmark (‡) or by inference (†), which the third line names. An asterisk marks values supported by the developer’s own report (E1).

Figure 3. Highest autonomy level over time, for any system and for open-weight models, including values reached on a benchmark or by inference (Table 9 gives the map without them). Within each level, the line rises from capable (C1) to expert (C2) and expert in audited code (C3). A filled circle marks a step with evidence E2 or E3 and a hollow circle a step with E1.

RQ1: Autonomy Today

Discovery and validation. LLM-based agents have been shown to discover and validate vulnerabilities without human intervention at expert correctness (A4·C2) at every tier. At the partner tier, Claude Mythos Preview extended this correctness in May 2026 to code that experts and fuzzers had examined for years (A4⋅C3A4\cdot C3), and independent firms confirmed 90.6% of its 1,752 assessed findings [4, 12]. Public systems reached expert correctness (A4⋅C2A4\cdot C2) around the same time, when a security firm confirmed 96% of the findings of Aikido Attack, a commercial agent that anyone can sign up for [67]. Open-weight models followed within months. GLM 5.3, released in August 2026 and assessed by the US Center for AI Standards and Innovation as the most capable open-weight model to date [49], developed end-to-end exploits at a rate close to that of Mythos Preview on a benchmark of browser-engine vulnerabilities (‡) [68]. In addition, on real software, 94.5% of the reports that a published harness produced with DeepSeek V4 Pro on a production codebase proved true in October 2026 [69].

At the lab tier, agents have also become capable of choosing their own targets and completing attacks end-to-end (A5⋅C1A5\cdot C1). In July 2026, agents under evaluation at OpenAI turned to third-party systems without human direction and, over days to weeks, found and exploited previously unknown vulnerabilities in them [6], which Hugging Face and METR independently confirmed [70, 71]. Similar events have since been reported for models from labs including Anthropic and Meta, in the labs’ own use and in external evaluations [72, 73, 74].

Patching. Agents have also been shown to patch without human intervention, at capable correctness on public models (A4⋅C1A4\cdot C1) and at expert correctness at the partner tier (A4⋅C2A4\cdot C2). By October 2025, Google’s CodeMender had upstreamed 72 security fixes to open-source projects [75], and maintainers had merged the fixes written by a depthfirst agent for Netty and by the open-source PortGPT agent for the Linux kernel [76, 77]. Expert correctness was reached around September 2026 by initiatives like OpenAI’s Defense Factory, a production pipeline for partners in which 0.53% of the fixes were rolled back [78]. For open-weight models, we infer A4 at capable correctness (A4⋅C1A4\cdot C1‡). On a benchmark of real CVEs, DeepSeek V4 Flash fixed about as many vulnerabilities as GPT-5.5 and GPT-5.6 Sol, 80% against 81 to 84% [79], and the open-source PortGPT agent, whose kernel fixes maintainers merged, also runs on open-weight models [77].

Patch validation. Agents have been checking their own patches without human intervention (A4⋅C1A4\cdot C1) since October 2025. Google’s CodeMender and AISLE’s remediation system each described an agent that tested its patches for the root cause and for regressions, and passed on only those that held [75, 80]. Public systems followed in April 2026 [81, 82]. At the partner tier, patch validation reached expert correctness (A4⋅C2A4\cdot C2) in the Defense Factory, where the fixes that the pipeline let through were rarely rolled back [78]. On a benchmark of real CVE fixes (‡), an LLM judge agreed with a careful human reading in 85% of the cases [83]. Open-weight models have reached A3·C1 on real software [84], and a reviewer agent on DeepSeek V4.1 Flash reached A4·C1 on a benchmark (‡) in October 2026 [85].

Across tiers. Of the 16 cells, 14 have reached A4 or A5 on direct evidence, and open-weight patching reaches A4 by inference (Figure 1). Open-weight patch validation stands at A3·C1 on real software, and reaches A4 on a benchmark. The highest steps of discovery and validation at the lab, partner and public tiers are independently confirmed (E3), those of patching rest on checkable reports (E2), and those of patch validation mostly on the developers’ own reports (E1), which makes remediation the least measured part of the map.

Answer to RQ1: By November 2026, LLM-based agents have been shown to discover, validate and patch vulnerabilities in real-world software without human intervention (A4), and to check their own patches. Discovery and validation have reached expert correctness (A4·C2) at every tier, open-weight models included, and code audited for years (A4·C3) at the partner tier. Patching and patch validation have reached expert correctness (A4·C2) at the partner tier and capable correctness (A4·C1) on public models. For open-weight models, we infer A4·C1 for patching (‡), and patch validation stands at A3·C1 on real software and at A4·C1 on a benchmark (‡). Agents under evaluation have also discovered and validated vulnerabilities in targets that they chose themselves (A5·C1).

RQ2: Change Since 2024

Since 2024, the highest autonomy level of every task has risen steadily, one level at a time (Figure 3). Each level has first been reached at capable correctness and later at expert correctness. In 2024, LLMs mostly performed individual steps inside pipelines designed by humans (A2⋅C1A2 \cdot C1), such as writing fuzz targets or drafting a fix for a scanner alert [87]. The first agents able to plan the whole task (A3⋅C1A3 \cdot C1) appeared in the same year and were able to find zero-days that received CVE identifiers [1, 88]. These agents reached expert correctness (A3⋅C2A3 \cdot C2) in May 2025, when maintainers confirmed or fixed 94% of the bugs reported by RepoAudit [89].

Discovery and validation reached A4 in mid-2025. The XBOW agent reached capable correctness (A4⋅C1A4 \cdot C1) around June, when the programs on HackerOne judged 72% of its 886 closed reports to be genuine [90]. Expert correctness (A4⋅C2A4 \cdot C2) followed in August, when an agentic system for Android apps confirmed 97% of its reported zero-days [91]. In May 2026, Mythos Preview extended expert correctness to code audited for years (A4⋅C3A4 \cdot C3), where it was able to find thousands of validated vulnerabilities [4, 12].

Patching and patch validation reached A4 at capable correctness (C1) in October 2025, four months after discovery. Expert correctness (C2) followed in September 2026, in initiatives like OpenAI’s Defense Factory [78]. This later date is likely to reflect how rarely the correctness of patches is reported in the literature, since patches written at lower autonomy (A2) had already reached expert correctness (C2) by May 2025 (Appendix § 12).

Open-weight models followed in the same order. For discovery and validation, they reached A4 at capable correctness (C1) in April 2026, when Chrome confirmed the

vulnerabilities reported by an agent on Kimi K2.5 [92]. Expert correctness (A4·C2) followed for both tasks in October 2026 [69], and we infer A4·C1 for patching from August 2026 (†).

Answer to RQ2. LLM-based systems progressed from performing prescribed steps, as a sort of co-pilot, (A2) in early 2024 to completing each task without intervention (A4) by October 2025, before the closed frontier models of 2026. Discovery and validation reached expert correctness (A4·C2) in August 2025 and code audited for years (A4·C3) in May 2026. Patching and patch validation reached A4 at capable correctness (C1) four months after discovery and at expert correctness (C2) in September 2026. Open-weight models followed in the same order from April 2026.

RQ3: Spread to Public and Open Weights

Public and open-weight systems have reached each level within months of the first system to show it (Table 4). At capable correctness (A4·C1), public systems followed within 7 months for every task, and open-weight models after about 10 months for discovery and validation. At expert correctness (A4·C2), public systems followed after about 9 months for discovery and validation, and open-weight models after about a year. For patching and patch validation, the inferred and benchmark values place open-weight models 10 to 12 months behind the first system (†).

Capable (C1)Expert (C2)
PublicOpen weightsPublicOpen weights
Discover5.0 [2.8, 9.9]9.9 [7.8, 15.9]8.9 [0.2, 9.7]13.7 [≥5.0]
Validate2.3 [0.1, 9.9]9.9 [7.8, 10.6]8.9 [0.2, 9.7]13.7 [≥5.0]
Patch0.2 [≥0]9.5†––
Validate patch6.4 [≥5.9]11.8†––

Table 4. Months from the first system at A4, at any tier, to the first public and open-weight system at the same correctness, on direct evidence. Brackets give the range when the earliest source at either end is set aside. A dash marks a pair not yet shown at that tier, and † a delay on inferred or benchmark values only.

QuestionPositions (source opinions)Demonstrated (systems evaluated)Public records (observed)
Are reports written by LLMs valid?• In 2025, curl’s maintainer found most reports invalid [13]. • In 2026, the same maintainer described a high volume of high-quality reports [93].• Expert precision for the best unattended systems (A4·C2) on their reported targets since August 2025 (RQ1).• Confirmed shares range from about 5% of curl’s reports in 2025 to 97% at wolfSSL [94], [95].
Is there a surge of findings?• Practitioners describe a surge of findings, the “Vulnpocalypse” [7]. • Others argue that other causes contribute [96].• CVE records 2.3 times those of 2025, at most large issuers and in every class [17]. • Records that name an AI system as finder are 2.4 to 2.8% per quarter of 2026 [17].
Are the new findings more severe?• Anthropic reports thousands of high and critical findings [4]. • curl’s maintainer sees mostly low and medium ones [97].• Findings in code audited for years, at A4·C3 (RQ1).• Share rated Critical or High up from 46% in 2025 to 56% from July to October 2026, within the same issuers and classes [17].
Will the surge pass?• Some practitioners expect it to end once existing code is swept [10], [11]. • Others argue that fixing bug by bug does not converge [98], [99].• Public systems reached unattended expert precision (A4·C2) about nine months after the first system, and audited code (A4·C3) only at the partner tier (RQ1, RQ3).• Records still rising in October 2026 [17]. • Severe bugs ten years old or more reported 6 to 11 times as often as in 2025, where bug age is known [17], [100].
Do more findings mean more fixes?• Anthropic expects defenders to gain the advantage over time [101]. • Others report that findings outrun the capacity to fix [47], [102].• Patching reached A4·C1 within months of discovery, but few reports count how many of its patches hold up (RQ2).• Coordinated disclosures fixed in a median of 96 to 119 days every year since 2016 [56]. • 5 to 26% of bulk reports to outside maintainers fixed, 58 to 94% of curated ones (Table 6).
Is exploitation getting faster?• Mandiant reports that the time to exploit has turned negative [103]. • VulnCheck finds the share exploited by the day of disclosure flat [104].• Exploits built from public fixes within hours, by one lab and on one open-weight model [9], [105].• Listed attacks a median of 2 to 5 days after the record every year since 2022, and 24 to 30% on or before its day, with no trend [61].
Does restricting frontier models contain the capability?• Anthropic limits access to vetted partners [5]. • It also reports that anyone can download a comparable model [68].• One open-weight model was not distinguishable from a restricted one four months later, and benchmarks give 8 months (RQ3).• Records that name an AI system as finder come from a few dozen organizations a quarter [17]. • Credits to 10,326 distinct finders in 2026 against 3,987 in 2025, with 70% of the credits to first-time finders [17].

Table 5. Commonly expected consequences of LLM-based vulnerability research, stated as questions. Positions are opinions of the sources cited. Demonstrated summarizes the map and lab demonstrations, for the systems and targets evaluated. Sources cover up to November 7 2026.

These delays are upper bounds, since they date the first documented use on real software, and the models themselves appear to spread faster. The open-weight GLM 5.3 was released four months after Mythos Preview became available to partners [106]. In Anthropic’s comparison, it developed end-to-end exploits about as often as Mythos Preview, in 12% of 410 attempts against 14% [68], and the US Center for AI Standards and Innovation placed it about four months behind the US frontier [49]. On twelve security benchmarks, open-weight models tend to reach the scores of closed models about 8 months later (5.4 to 10.0, Figure 4), which is consistent with the 4 to 7 months reported by the UK AI Security Institute and Epoch AI [15], [50].

Figure 4. Scores on twelve security benchmarks by release date, for closed-weight and open-weight models that set a new best score among models of their kind since 2024. Lines are a least-squares fit with one offset per benchmark and a band of two residual standard deviations. Their shift is an open-weight delay of about 8 months. Design after Clinton [86].

Answer to RQ3 callout and ablations text

Figure 9.

Answer to RQ3. Capabilities have reached public and open-weight systems within months. Public systems followed the first system at A4 within 7 months at capable correctness (C1) and 9 months at expert correctness (C2). Open-weight models followed within 10 months and 14 months for discovery and validation, and within 10 to 12 months for patching and patch validation on inferred and benchmark values (†). These delays are upper bounds. Benchmarks place open-weight models about 8 months behind closed ones, and other evaluators report shorter delays still.

Ablations. Table 9 (Appendix C) presents the map under four evidence standards. Apart from open-weight patching, which is marked as inferred, none of the answers above depends on a benchmark or inferred value of the map. When only results with checkable details or independent confirmation (E2 or E3) are kept, discovery and validation reach A4 in August 2025 instead of June, patch validation reaches A4 only in September 2026 at the lab and partner tiers, and it falls to A3 at the public tier, so that 13 of the 16 cells reach A4 or A5.

CapabilityTierDirect, E2 or E3DirectDirect and benchmarkAll values
DiscoverT1A5·C1A5·C1A5·C1A5·C1
T2A4·C3A4·C3A4·C3A4·C3
T3A4·C2A4·C2A4·C2A4·C2
T4A4·C2A4·C2A4·C2A4·C2
Validate vulnerabilityT1A5·C1A5·C1A5·C1A5·C1
T2A4·C3A4·C3A4·C3A4·C3
T3A4·C2A4·C2A4·C2A4·C2
T4A4·C2A4·C2A4·C2A4·C2
PatchT1A4·C2A4·C2A4·C2A4·C2
T2A4·C2A4·C2A4·C2A4·C2
T3A4·C1A4·C1A4·C1A4·C1
T4A2·C2A2·C2A4·C1A4·C1
Validate patchT1A4·C2A4·C2A4·C2A4·C2
T2A4·C2A4·C2A4·C2A4·C2
T3A3·C1A4·C1A4·C1A4·C1
T4A3·C1A3·C1A4·C1A4·C1

Table 9. Highest pair of each cell on 7 November 2026 under four evidence standards. The principal map is the column Direct. Bold marks a value that differs from it.

Observed Consequences

To answer RQ4, we compare the consequences commonly expected of LLM-based vulnerability research with public records since 2024. We derived seven such consequences from the 503 verified statements, on which labs, maintainers, agencies and researchers may have diverging analyses, and Table 5 sets their positions against what has been demonstrated and what the public records show. From the records, we measure the number of disclosures, their severity, the age of the vulnerable code, the finders who report them, and the time from report to fix and from disclosure to attack (Figure 6). Since few records name the tool behind a finding, these measures date the changes, and the share of each that is due to LLM-based systems remains to be determined. Where we compare the records that name an AI system as finder with other records, we use the other records of the same issuers as the comparison group.

Figure 6. Public vulnerability records since 2024. (a) CVE records per month by class of vulnerability. (b) Share of scored records rated Critical or High by quarter, with 95% intervals, for all records, for the scores that issuers give their own records, and for all records reweighted to the issuers of 2025. (c) Records per month of bugs present for ten years or more and rated Critical or High, by quarter, where the age of the bug is known. (d) Medians by quarter of the days from report to release of fixed cases at the Zero Day Initiative (with quartiles), from the CVE record to listing in CISA’s catalogue of exploited vulnerabilities, and from the CVE record to the first public exploit code. The last quarter covers October only, and recent kernel records are not all scored yet.

Disclosures more than doubled. The number of disclosed vulnerabilities more than doubled in 2026. CVE records [17] published from January to October rose from 39,638 in 2025 to 89,749 in 2026, and the monthly count nearly quadrupled, from 4,302 in January 2026 to 16,949 in October (Figure 5 and Figure 6a). The increase extended to every class of vulnerability and to about two thirds of the largest organizations that issue CVE records. Two thirds of it came from six of these issuers, namely GitHub, VulnCheck, Oracle, the Linux kernel, Chrome and Microsoft [17]. The records that vendors and projects issue for their own software rose 2.8 times, after growing by at most 1.3 times a year from 2019 to 2025. Those of third-party issuers, such as researchers and bug bounty programs, rose 1.6 times, although they had grown faster than those of vendors since 2022 (Appendix § 11). Records that name an AI system as finder rose from 268 in 2025 to 2,290 in the first ten months of 2026, or 2.4 to 2.8% of all records in each quarter. Since most records name no tool, this count is a lower bound [96]. Since vendors had access to their code throughout, this reversal suggests that the owners of code have gained the most from LLM-based discovery so far.

Figure 5. CVE records per month (left axis) and cost of a validated finding and of an accepted patch (right axis), measured to October 2026 and projected after it (shaded). Sources: CVE records [17], their forecast [107], expected end of the surge [10], [11], [108], scenario (dashed red) built from the trend of the records that credit an LLM-based system and from how findings evolved after fuzzing and audits were first introduced [25], [109], [110], cost per finding [2], [15], [111, 112, 113] and its projection [114], cost per patch [2], [115], [116], and the original chart [7], [10]. Appendix § 11 gives the details.

A growing share of disclosed vulnerabilities is rated Critical or High. The share of these records rated Critical or High rose from 46% in 2025 to 56% from July to October 2026 [17] (Figure 6b). The rise holds when the comparison is limited to the same issuers and to the same types of vulnerability. With the set of issuers fixed to that of 2025, the share has risen every year since 2023, and fastest in 2026 (Appendix § 11). Throughout 2026, the records that name an AI system as finder had about the same share as the other records of their issuers. This trend is consistent with LLM-based systems and other new tools uncovering severe vulnerabilities that had remained hidden.

Old, severe bugs surface more often. Severe bugs that had survived a decade or more were reported several times more often in 2026. Where the age of the vulnerable code is known, bugs at least ten years old and rated Critical or High were reported 6 to 11 times as often per month from July to October 2026 as in 2025, in the Linux kernel, Windows, open-source packages and other software with dated releases [17], [100] (Figure 6c). Most of this increase followed the overall rise in records, and the share of old bugs among dated records rose by 6 to 9 points. The age is known for most kernel records but for only 2 to 3% of the others, so this measure covers a part of all software (Appendix § 11). This rise is consistent with the capability shown in RQ1, where agents found vulnerabilities in code that experts and fuzzers had examined for years (A4·C3). Forecasts that the surge will end once existing code has been swept assume that vulnerabilities are sparse [10], [11], in Schneier’s term, rather than plentiful [117], [118]. In the code that we could date, the records show no sign of exhaustion so far.

Most credits go to first-time finders. The CVE records that name a finder, 43% of those of 2026, credited 2.6 times as many distinct finders from January to October as a year earlier, for 2.1 times as many records [17]. Finders who had never been credited before received 70% of these credits, whereas this share had fallen every year from 2022 to 2025, to 51%. This suggests that vulnerability discovery has become accessible to finders without an established record, as public and open-weight systems become increasingly capable (RQ3).

The time to fix has not shifted. The time from report to fix has not shifted noticeably. The Zero Day Initiative gives vendors 120 days to fix a vulnerability [119], and publishes the dates of each report and release. In a sample of 1,417 of its advisories, the median time from report to release of fixed cases stayed between 96 and 119 days in every year from 2016 to 2026 (Figure 6d). The most recent quarter, which covers six weeks and 28 fixed cases, has the highest median of the series, at 148 days, above this deadline. Chrome fixed external reports in a median of 50 days from 2020 to 2023, 44 in 2025 and 36 to 45 in 2026, and reports that name an AI system as finder in 16 to 26 days. So far, vendors have therefore kept meeting the deadline of coordinated disclosure, and Chrome has fixed the reports that name an AI system faster than the others. Whether a report is fixed at all appears to depend on how it reaches the owner of the code (Table 6). Three vendors shipped fixes for their own code at many times their earlier rate. In contrast, only 5 to 26% of the reports sent in bulk to outside maintainers were fixed, a share that predates LLMs, whereas curated

ProgramDateReportedFixed%
Built into the owner’s workflow
OSS-Fuzz*, within 90 days [109]202123,90716,95271
Curated reports to outside maintainers
Project Zero*, 2019 to 2021 [120]02/2237635193
Big Sleep [65]10/26878294
FuzzingBrain [121]02/26623658
Bulk reports to outside maintainers
syzbot*, within 90 days [66]2019–238,3872,16826
Patch the Planet [63]11/262,11850624
Anthropic, reviewed by a firm [62]11/261,87146425
Anthropic, sent directly [62]11/263,2161655
XBOW on HackerOne [90]06/251,06013012

Table 6. SHARE OF REPORTS FIXED, BY HOW THE FINDER REACHES THE OWNER OF THE CODE. SHARES ARE CUMULATIVE AT THE DATE SHOWN, OR WITHIN 90 DAYS OF THE REPORT WHERE STATED. SYSTEMS MARKED * PREDATE LLM-BASED DISCOVERY.

programs and OSS-Fuzz reached 58 to 94%. Most reports sent in bulk thus remain unfixed, which is where a larger number of findings is most likely to exceed the capacity to fix. Since these programs differ in targets, validity, severity and age, the differences cannot be attributed to the way a report is sent alone.

The time to attack has not shortened. The time from disclosure to attack has not shortened noticeably either. CISA lists a CVE in its catalogue of known exploited vulnerabilities once it has reliable evidence of an attack. For CVEs listed within 90 days of their record, the median time from record to listing was 2 to 5 days in every year from 2022 to 2026, the last covering records published up to mid-July, and 24 to 30% of the entries of each year were listed on or before the day of the record, with no trend (Figure 6d). Mandiant’s mean time from patch to first observed attack fell from 63 days in 2018 and 2019 to 5 days in 2023, before LLM-based systems were finding vulnerabilities in real software, and to an estimated −7 days in 2025 [103]. Public exploit code did not appear earlier either, with a median of 5 days after the record in 2026 against 1 to 2 days from 2020 to 2025. Observed attacks have thus not yet followed what demonstrations show to be possible. Starting from public patches, Mythos Preview was able to write browser-engine exploits within hours [9], and the open-weight GLM 5.3 developed attacks against six Firefox releases using only the information that was public at each release [105]. Incidents confirm that attackers have used LLM-based agents [146, 147, 148], but no source yet attributes a measurable share of intrusions to them [103].

Our reading. Defenders may have absorbed a larger number of more severe findings at an unchanged pace, or attackers may not yet use these systems at scale or in ways that the catalogues capture, and the records cannot yet distinguish between the two.

Answer to RQ4: The number of CVE records more than doubled in 2026, for most large issuers and for every type of vulnerability. A growing share of these records is rated Critical or High. Where the age of the bug is known, severe bugs present for ten years or more were reported 6 to 11 times as often as in 2025. Credits went to 2.6 times as many finders, and most credits went to first-time finders. In contrast, the median time from report to fix under coordinated disclosure and from record to listed attack has not shifted noticeably, while most reports sent in bulk remain unfixed. These changes are consistent with the arrival of LLM-based systems, although few records name the tool behind a finding, so the share that is due to them remains to be determined.

Mitigations and Guidance

To answer RQ5, Table 7 lists the mitigations proposed in response to LLM-based vulnerability research, grouped by their goal, and grades the evidence that it works. The grade combines the four attributes of the best evidence defined in the caption. Evidence is strong for a security outcome in deployment that is confirmed and not reported by a vendor alone, moderate for any measured outcome on real software that is not reported by a vendor alone, and weak otherwise. The grades describe how well a mitigation works in general, since no source tests one against LLM-based attackers.

MitigationNewStatementsOutcomeSettingMeasured byConfirmationEvidenceMain result, with its scope and limit
Fewer vulnerabilities to find
Memory-safe languages◦18SDOYStrongShare of memory-safety bugs among Android platform vulnerabilities fell from 76% in 2019 to below 20% in 2025 [122], [123].
Attack-surface reduction✓16SRIRModerateKernel tailoring removed 50 to 85% of the attack surface [124]. With an LLM, one vendor removed a fifth of OpenSSH’s code reachable before authentication [125].
Scanning one’s own code✓20PDORModerateThree vendors shipped many times more fixes on their own code [126, 127, 128], counting fixes shipped, not vulnerabilities left.
Formal verification with LLMs✓28PRVWeakNo-crash proofs for 239 entry points in 123 libraries, for bounded inputs, in one vendor’s report [129].
Faster validated remediation
LLM patches with human review✓33PRIRModerateMaintainers merged all the submitted kernel backports of one agent and 69% of one program’s patches [63], [77]. A standalone LLM introduced vulnerabilities on a benchmark [130].
Dependency automation, inventory11PRIRModerateProjects accepted 65% of automated security updates of dependencies and 3% broke the build [131], while most kept outdated ones [132]. Inventory rests on advice.
Update by default, release cadence25PDIRModerateSilent updates patched half of 8.4 million vulnerable hosts in 27 days, against 41 for manual ones [133], [134]. Cadence is unmeasured [127].
Proof required with each report✓10NROWeakAdopted by two bounty programs [135], [136], with no measured effect.
Automatic patch validation✓6PBVWeakAn LLM judge agreed with a careful human reading on 85% of CVE fixes on a benchmark [83]. One vendor rate on real software (RQ1).
Less damage when exploited
Compartmentalization◦8SDOModerateSite Isolation covers 94 cross-origin bypass bugs reported in Chrome from 2014 to 2018, and such reports fell from 9 to 2 in six months [137].
Zero trust, least privilege13NWeakAdvised by practitioners [8], [138] (opinion). No measured outcome.
Isolating unpatchable systems8NWeakAdvised in government guidance [139] (opinion). No measured outcome.
Independent versions (N-version)✓9PBIWeakOn a benchmark, three LLM-written versions realized 0.43 of the gain of independent ones [140].
Intrusion detection and response
Intrusion detection and response with LLMs✓17PRVWeakA vendor reports 80% precision over 120 days in production [141]. Triage missed at least 40.4% of attack alerts in one scenario [142], and injected log text misled a model [143].
Access controls on models✓19NRVWeakIdentity checks are deployed [144], [145], with no measured effect on misuse. Open weights followed within months (RQ3).

Table 7. Mitigations proposed in response to autonomous vulnerability research. New marks mitigations that LLM-based systems make newly practical, and Statements counts the verified statements on the mitigation. The best evidence has an outcome (security S, such as vulnerabilities or attack surface removed, process P, such as patches merged, or none N), a setting (deployment at scale D, real software in a limited study R, benchmark B), who measured it (independent I, the deploying organization O, a vendor of the defense or its model V) and a confirmation (by a second organization R, over several years Y). Results are observed outcomes with their scope, unless marked as opinion.

What follows from our findings. The best-supported mitigations are established practices of secure design and maintenance, such as memory-safe languages, a smaller attack surface, compartmentalization and automatic updates. Most mitigations built on LLMs have so far been measured only in vendor reports and benchmarks. Our findings suggest three points (D). First, finding vulnerabilities may no longer be what limits defenders. Agents have been shown to find them at expert correctness at every tier, yet in the programs we measured most reports sent in bulk to outside maintainers remain unfixed, whereas three vendors report shipping many times more fixes after scanning their own code (§ 6). LLM-based discovery thus seems to help most where the fix is made. Second, the critical delay may now be the time to deploy a fix, since exploits have been built from public fixes within hours in demonstrations, while Verizon reports that organizations had fully remediated only 26% of known exploited vulnerabilities in 2025 [149]. Third, restricting frontier models delays the spread of these capabilities by months only (RQ3), and defenses should assume that attackers have them. Defenders therefore seem to need LLM-based systems that find, fix and detect at the pace of attacks, built on a secure design that limits what any missed vulnerability allows.

Guidance by stakeholder. Items marked D are our deductions, which the records have not tested.

  • Maintainers and vendors should run LLM-based discovery on their own code, write new code in memory-safe languages and remove code that need not be reachable. Those with early access to restricted models should sweep their most exposed code first (D).

  • Operators should apply updates by default, merge automatic dependency updates after tests, and measure their time to deploy (D). Until a fix is deployed, compartmentalization limits what an attacker gains. Least privilege with phishing-resistant authentication, isolating systems that get no updates and automated detection and response are also advised, with no measured effect yet.

  • Finders who use LLM-based systems should send maintainers only reports they have confirmed, each with a proposed fix, and publish how many targets and reports each result comes from (D).

  • Coordinators, bounty programs and funders should pay for review, patches and backports more than for further findings, and have records name the tool that found a vulnerability (D).

  • Evaluators and policymakers should test released models on real software, plan for open-weight parity within months and track our indicators (D).

Answer to RQ5: The mitigations best supported by evidence are established practices of secure design and maintenance, led by memory-safe languages, whereas most of those built on LLMs have been measured only in vendor reports and benchmarks. Our findings suggest that defenders gain most by running LLM-based discovery where the fix is made, by shortening the time to deploy fixes, and by limiting what an unpatched vulnerability allows, since restricting models delays these capabilities by months only (D).

Open Problems

Removing vulnerability classes from existing code. The strong evidence for memory-safe languages concerns new code, whereas decade-old severe bugs now surface several times more often. LLMs have translated programs and libraries of a few thousand lines at most from C to safe Rust [150], [151] and produced no-crash proofs for fuzzing entry points of over a hundred libraries, for bounded inputs [129]. Yet a benchmark finds them weak at writing code, specification and proof together [152], and verified systems have failed mostly at their unverified interfaces [153]. Scaling both to large code bases is open.

Secure architecture by construction. Retrofitting compartmentalization and least privilege requires engineering for each component [154], [155], and zero trust rests on advice so far. We found no evaluation of agents that derive and enforce such boundaries beyond single components.

Remediation at the pace of discovery. Most bulk reports remain unfixed, and patch validation is the least measured task. Validators with known error rates, compared on benchmarks of known vulnerabilities [45], are open, as is deploying fixes before their code is public, since exploits can be built from it [9], [105].

Disclosure when findings collide. Linux maintainers now ask that a bug found with an LLM be treated as public [156], and in October 2026 three parties reported the same PHP flaw within six weeks, with exploitation attempts before the embargo ended [157], [158]. Coordinating such collisions is open.

Defenses tested against LLM-based attackers. No source behind Table 7 tests a mitigation against an LLM-based attacker, and few records name the tool behind a finding or an intrusion. Such tests and records would show whether attacks follow the demonstrations.

Contained agents. Agents under evaluation have left their environment [6], [72], and detection agents read data that attackers control [142], [143]. Containing a system that can find vulnerabilities in its own sandbox is open.

Conclusion

We separated what LLM-based systems have been shown to do in vulnerability research from what has changed in public records. On direct evidence, agents were able to discover, validate and patch real vulnerabilities without human intervention (A4) by October 2025, and restricted capabilities reached open-weight models within months. Discovery and validation have since reached expert correctness at every tier (A4·C2). In 2026, the number of CVE records more than doubled, a growing share was rated Critical or High, and severe bugs present for a decade or more surfaced several times more often. Yet the time from report to fix and from disclosure to listed attack has not shifted noticeably so far. Whether this pace holds as findings grow, and whether attacks follow the demonstrations, remains to be measured.

Corpus and Search

Search. The corpus is indexed by keyword (BM25 [159]), by dense embeddings of each source and of its passages (Qwen3-Embedding-4B [160]), and through its citation graph. A learned ranker (LambdaMART [161]) combines these results with a cross-encoder reranker [160]. We evaluated it on 256 queries, with relevance pooled as in TREC [162] from the top 20 results of every search mode and graded by the local LLM (weighted κ≈0.70\kappa\approx0.70 with Claude on 600 pairs). We tuned on 177 queries and report the other 79 in Table 8.

RankingnDCG@10R@10MRR
Lexical (BM25)0.620.790.94
Dense0.640.830.96
Passages0.730.880.97
Reciprocal rank fusion0.720.890.97
with the reranker0.750.900.97
Learned ranker (default)0.790.920.99

Table 8. Search quality on the 79 held-out test queries, with graded judgements pooled from every search mode. nDCG@10 [163] weighs the grades of the top ten results, R@10 is the share of relevant sources in the top ten, and MRR the mean reciprocal rank of the first relevant one.

Measurement Details

Records. The counts of records in § 6 use the public CVE List as of 31 October 2026. A record names an AI system as finder when its credit, or a list published by a lab or vendor, names an LLM-based system or its operator. A record is rated Critical or High when its CVSS base score is 7 or more, and we take the issuer’s score, else CISA’s, else NVD’s. With the issuers of 2025 held fixed, the share rated Critical or High was 38% in 2023, 42% in 2024, 46% in 2025 and 57% from July to October 2026. An issuer is third-party when the CVE Program lists it with none of the types Vendor, Open Source, Hosted Service or Consortium [164], or when it covers WordPress plugins.

Age, finders and time. The age of a bug is the time from the release that introduced it to its fix. A finder is an identity named in the credit of a record, and it is first-time when no earlier record since 2018 credits it. The time to fix comes from a sample of 1,417 of the 14,738 advisories of the Zero Day Initiative, stratified by quarter of release [119], in which no credit names an AI system. The time to attack and to public exploit code covers the records with at least 104 days of follow-up.

Figures. In Figure 5, the records that credit an LLM-based system grew 8.8 times a year from July 2025 to September 2026 (95% interval 5.3 to 14.8). The scenario slows this growth to a peak in the window when three practitioners expect the surge to end, and then lets it decline with a half-life of 10 to 24 months. The cost projections continue the measured curves and ease within a few months to a decline of 4.7 times a year, the rate that Epoch AI reports for a level of performance two years after its debut [114]. In Figure 4, we fit 104 best scores with one slope, one offset per benchmark and one shift for open-weight models, which gives a delay of 7.6 months (95% bootstrap interval 5.4 to 10.0).

The Final Map

Table 10 lists the 128 steps of the final map with their sources, and Table 9 the highest pair of each cell under four evidence standards. Tiers are cumulative, so a step of a more widely available system counts at every more restricted tier. A result can set a higher pair at a tier where further sources corroborate it than at a tier where it stands alone, as for the firmware agent of November 2025 at the public and open-weight tiers. A step carries an earlier date at a more restricted tier when its system was available there first, such as a tool that its authors used before releasing it. A question mark marks a level without a correctness category, † an inferred value and ‡ a benchmark value, and the principal map uses the 85 direct steps, which carry neither mark. The reports of the June 2025 step were reviewed by the vendor’s staff before submission, and without this step A4 dates from August 2025. Direct evidence has confirmed or preceded seven of the eight inferred values, and the eighth, open-weight patching at A4·C1, remains untested on real software. The full table of readings, with the rate behind each correctness category, is released with our artifact.

TierPairDateGradeSources
Discover
T1A2·C12024-01E1[165]
A2·C1‡2024-06E2[166]
A2·C12024-08E2[167]
A3·C12024-10E2[88]
A3·C1‡2024-11E2[168]
A3·C12025-01E3[89]
A2·C22025-02E2[169]
A3·C22025-05E2[89]
A4·C12025-06E1[90]
A4·C22025-08E2[91]
A3·C32026-05E2[12], [126]
A4·C32026-05E3[4], [12], [62], [170]
A5·C12026-07E3[70], [171]
T2A2·C12024-01E1[87]
A2·C1‡2024-06E2[166]
A2·C12024-08E2[167]
A3·C12024-10E2[88]
A3·C1‡2025-02E2[168]
A2·C22025-02E2[169]
A3·C12025-04E3[89]
A3·C22025-05E2[89]
A4·C12025-06E1[90]
A4·C22025-08E2[91]
A3·C32026-05E2[12], [126]
A4·C32026-05E3[4], [12], [62]
T3A2·C12024-01E1[87]
A2·C1‡2024-06E2[166]
A2·C12024-08E2[167]
A3·C12024-10E2[88]
A3·C1‡2025-02E2[168]
A2·C22025-02E2[169]
A3·C12025-04E3[89]
A3·C22025-05E2[89]
A4·C12025-11E2[67], [172]–[174]
A4·C2†2026-05E1[12], [14], [175], [176]
A4·C22026-05E3[67], [177]
A4·C22026-06E2[177]
T4A2·C12024-07E2[178]
A3·C1‡2025-05E2[89]
A3·C12025-11E2[69], [172], [173]
A4·C12026-04E3[92]
A4·C2†2026-10E1[69], [179]
A4·C22026-10E2[69]
Validate vulnerability
T1A2·C1‡2024-01E2[165]
A2·C12024-01E1[87]
A3·C1‡2024-04E2[180]
A3·C12024-05E2[181]
A4·C12025-06E1[90]
A4·C22025-08E2[91]
A2·C22025-09E1[182]
A3·C22026-01E2[183]
A3·C32026-05E2[12], [126]
A4·C32026-05E3[4], [12]
A5·C12026-07E3[70], [171]
T2A2·C12024-01E1[87]
A2·C12024-08E2[167]
A3·C1‡2025-02E2[168]
A4·C12025-06E1[90]
A4·C22025-08E2[91]
A2·C2‡2025-10E2[184]
A3·C22026-01E2[183]
A3·C32026-05E2[12], [126]
A4·C32026-05E3[4], [12]
T3A2·C12024-01E1[87]
A2·C12024-08E2[167]
A3·C1‡2025-02E2[168]
A4·C12025-09E2[185]
A2·C2‡2025-10E2[184]
A3·C22026-01E2[183]
A4·C2†2026-05E1[12], [175], [176]
A4·C22026-05E3[67]
A4·C22026-06E2[177]
T4A2·C12024-07E2[178]
A2·C2‡2025-10E2[184]
A3·C1‡2026-04E2[186]
A4·C12026-04E3[92]
A4·C12026-05E2[49], [68], [147], [173]
A4·C2‡2026-10E1[68], [179]
A4·C22026-10E2[69]
Patch
T1A2·C12024-01E2[187]
A3·C1‡2024-11E2[188]
A3·C2‡2025-05E2[44]
A2·C22025-06E2[84]
A4·C12025-10E2[14], [75], [76]
A4·C12025-10E2[77]
T2A2·C1‡2024-03E2[189], [190]
A3·C2‡2025-05E2[44]
A2·C22025-06E2[84]
A4·C12025-10E2[75], [76], [80], [191]
A4·C12025-10E2[14], [77]
A4·C22026-09E2[78]
T3A2·C1‡2024-03E2[190], [192]
A2·C22024-08E1[193]
A3·C2‡2025-05E2[44]
A2·C22025-06E2[84]
A4·C12025-10E2[14], [77]
T4A1·C1‡2024-01E2[194]

Table 10. Steps of the final map, per capability and access tier, in date order.

TierPairDateGradeSources
A2·C1‡2025-01E2[195]
A3·C1‡2025-05E2[44]
A2·C22025-06E2[84]
A3·C1‡2025-10E2[14], [77]
A4·C1†2026-08E1[77], [79]
A4·C1‡2026-10E2[85]
Validate patch
T1A2·C12024-01E1[187]
A3·C12025-10E2[14]
A4·C12025-10E1[75], [80]
A4·C1†2025-11E1[75], [196], [197]
A3·C1‡2026-06E2[83]
T2A2·C22024-08E1[198]
A3·C12025-10E2[14]
A4·C12025-10E1[80], [199]
A2·C1‡2025-11E2[197]
A3·C1‡2026-06E2[83], [115]
A4·C2†2026-06E1[83], [115], [176], [199]
A4·C22026-09E2[78]
T3A2·C22024-08E1[193]
A2·C22024-08E1[198]
A3·C12025-10E2[14]
A2·C1‡2025-11E2[197]
A4·C12026-04E1[81], [82]
A3·C1‡2026-06E2[83]
A4·C1†2026-06E1[81]–[83], [115]
T4A1·C1‡2024-01E2[194]
A2·C1‡2025-01E2[195]
A3·C12025-06E2[84]
A3·C1‡2026-06E2[200]
A1·?2026-08E2[201]
A4·C1‡2026-10E2[85]

Table 11.

References

  1. [1]Big Sleep team, “From Naptime to Big Sleep: Using large language models to catch vulnerabilities in real-world code,” Google Project Zero blog, Nov. 2024. [Online]. Available: https://projectzero.google/2024/10/from-naptime-to-big-sleep.html
  2. [2]DARPA, “AI Cyber Challenge marks pivotal inflection point for cyber defense,” DARPA news, Aug. 2025. [Online]. Available: https://www.darpa.mil/news/2025/aicx-cc-results
  3. [3]C. Zhang, Y. Park, F. Fleischer et al., “SoK: DARPA’s AI Cyber Challenge (AIxCC): Competition design, architectures, and lessons learned,” in Proceedings of the 35th USENIX Security Symposium, 2026.arxiv.org/abs/2602.07666
  4. [4]N. Carlini, N. Cheng, K. Lucas, M. Moore, M. Nasr, V. Prabhushankar, W. Xiao et al., “Assessing Claude Mythos Preview’s cybersecurity capabilities,” Anthropic research, Apr. 2026. [Online]. Available: https://www.anthropic.com/research/mythos-preview
  5. [5]Anthropic, “Project Glasswing: Securing critical software for the AI era,” Anthropic project page, Apr. 2026. [Online]. Available: https://www.anthropic.com/glasswing
  6. [6]OpenAI, “OpenAI–Hugging Face incident technical report,” OpenAI technical report, Aug. 2026. [Online]. Available: https://cdn.openai.com/pdf/67869394-cb91-4c12-888c-5cbd85c7814c/OpenAI-Hugging-Face%20Incident-Technical-Report.pdf
  7. [7]P. Venables, “The vulnpocalypse: Good news and bad news,” Blog post, Oct. 2026. [Online]. Available: https://www.philvenables.com/post/the-vulnpocalypse-good-news-and-bad-news
  8. [8]R. Mogull, “Anthropic’s Mythos is here: Defending from the vulnpocalypse,” Cloud Security Alliance blog, Apr. 2026. [Online]. Available: https://cloudsecurityalliance.org/articles/anthropic-s-mythos-is-here-defending-from-the-vulnpocalypse
  9. [9]W. Xiao, T. Abbott, N. Carlini, N. Cheng, D. Forsythe, K. Lucas, M. Nasr, and S. Sakhuja, “Measuring LLMs’ impact on N-day exploits,” Anthropic research, Jun. 2026. [Online]. Available: https://www.anthropic.com/research/n-days
  10. [10]H. Adkins, “The vulnpocalypse big picture (conceptual graph),” Post on X, 5 August 2026, Aug. 2026. [Online]. Available: https://x.com/argevestatus/205088373497282800
  11. [11]J. Corbet, “The 2026 maintainers summit,” LWN.net, Oct. 2026. [Online]. Available: https://lwn.net/Articles/1099594/
  12. [12]Anthropic, “Project Glasswing: An initial update,” Anthropic research, May 2026. [Online]. Available: https://www.anthropic.com/research/glasswing-initial-update
  13. [13]D. Stenberg, “Death by a thousand slops,” daniel.haxx.se, Jul. 2025. [Online]. Available: https://daniel.haxx.se/blog/2025/07/14/death-by-a-thousand-slops/
  14. [14]Z. Wang, T. Shi, J. He, M. Cai, J. Zhang, and D. Song, “CyberGym: Evaluating AI agents’ real-world cybersecurity capabilities at scale,” in Proceedings of the 14th International Conference on Learning Representations (ICLR), 2026.
  15. [15]UK AI Security Institute, “How far behind the frontier are leading open weight models on cyber?” AISI blog, Jul. 2026. [Online]. Available: https://www.aisi.gov.uk/blog/how-far-behind-the-frontier-are-leading-open-weight-models-on-cyber
  16. [16]SAE On-Road Automated Driving Committee, “Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles (J3016_202104),” SAE International Recommended Practice, Apr. 2021. [Online]. Available: https://saemobilus.sae.org/content/j3016_202104
  17. [17]CVE Program, “CVE list, version 5 records,” GitHub repository, release of 31 October 2026, 2026, accessed Nov. 2026. [Online]. Available: https://github.com/CVEProject/cvelistV5
  18. [18]T. Willis, “Policy and disclosure: 2025 edition,” Google Project Zero blog, Jul. 2025. [Online]. Available: https://projectzero.google/2025/07/reporting-transparency.html
  19. [19]L. Ablon and A. Bogart, “Zero days, thousands of nights: The life and times of zero-day vulnerabilities and their exploits,” RAND Corporation, Tech. Rep. RR-1751-RC, 2017. [Online]. Available: https://www.rand.org/pubs/research_reports/RR1751.htmlDOI
  20. [20]P. Kotzias, L. Bilge, P.-A. Vervier, and J. Caballero, “Mind your own business: A longitudinal study of threats and vulnerabilities in enterprises,” in Proceedings of the 26th Network and Distributed System Security Symposium (NDSS), 2019.DOI
  21. [21]L. Bilge and T. Dumitras, “Before we knew it: An empirical study of zero-day attacks in the real world,” in Proceedings of the ACM Conference on Computer and Communications Security (CCS), 2012, pp. 833–844.
  22. [22]F. Li and V. Paxson, “A large-scale empirical study of security patches,” in Proceedings of the ACM SIGSAC Conference on Computer and Communications Security (CCS), 2017, pp. 2201–2215.
  23. [23]A. D. Householder and J. Spring, “A state-based model for multi-party coordinated vulnerability disclosure (MPCVD),” Software Engineering Institute, Carnegie Mellon University, Special Report CMU/SEI-2021-SR-021, Jul. 2021. [Online]. Available: https://www.sei.cmu.edu/libary/a-state-based-model-for-multi-party-coordinated-vulnerability-disclosure-mpcvd/DOI
  24. [24]E. Rescorla, “Is finding security holes a good idea?” IEEE Security & Privacy, vol. 3, no. 1, pp. 14–19, 2005.DOI
  25. [25]A. Ozment and S. E. Schechter, “Milk or wine: Does software security improve with age?” in Proceedings of the 15th USENIX Security Symposium, 2006, pp. 93–104. [Online]. Available: https://www.usenix.org/legacy/event/sec06/tech/full_papers/ozment/ozment.pdf
  26. [26]N. Alexopoulos, M. Brack, J. P. Wagner, T. Grube, and M. Mühlhäuser, “How long do vulnerabilities live in the code? A large-scale empirical measurement study on FOSS vulnerability lifetimes,” in Proceedings of the 31st USENIX Security Symposium, 2022, pp. 359–376. [Online]. Available: https://www.usenix.org/conference/usenixsecurity22/presentation/alexopoulos
  27. [27]M. R. Morris, J. Sohl-Dickstein, N. Fiedel et al., “Position: Levels of AGI for operationalizing progress on the path to AGI,” in Proceedings of the 41st International Conference on Machine Learning (ICML), ser. Proceedings of Machine Learning Research, vol. 235, 2024, pp. 36 308–36 321.arxiv.org/abs/2311.02462
  28. [28]K. J. K. Feng, D. W. McDonald, and A. X. Zhang, “Levels of autonomy for AI agents,” Knight First Amendment Institute, AI and Democratic Freedoms essay series, 2025, arXiv:2506.12469. [Online]. Available: https://arxiv.org/abs/2506.12469
  29. [29]OpenAI, “Preparedness framework, version 2,” Apr. 2025. [Online]. Available: https://cdn.openai.com/pdf/18a02b5d-6b67-4cec-ab64-68cdfdbdebcd/preparedness-framework-v2.pdf
  30. [30]Anthropic, “Responsible scaling policy, version 3,” Feb. 2026. [Online]. Available: https://anthropic.com/news/responsible-scaling-policy-v3
  31. [31]A. Dragan, H. King, and A. Dafoe, “Introducing the Frontier Safety Framework,” May 2024. [Online]. Available: https://deepmind.google/blog/introducing-the-frontier-safety-framework/
  32. [32]H. Xu, S. Wang, N. Li, K. Wang, Y. Zhao, K. Chen, T. Yu, Y. Liu, and H. Wang, “Large language models for cyber security: A systematic literature review,” ACM Transactions on Software Engineering and Methodology, vol. 35, no. 11, pp. 1–61, Nov. 2026.
  33. [33]Z. Sheng, Z. Chen, S. Gu, H. Huang, G. Gu, and J. Huang, “LLMs in software security: A survey of vulnerability detection techniques and insights,” ACM Computing Surveys, vol. 58, no. 5, pp. 1–35, 2026.arxiv.org/abs/2502.07049
  34. [34]S. Kaniewski, F. Schmidt, M. Enzweiler, M. Ment̄h, and T. Heer, “A systematic literature review on detecting software vulnerabilities with large language models,” ACM Transactions on Software Engineering and Methodology, 2026.
  35. [35]D. Ristea, S. MacFadden, E. Shereen, M. Dwyer, S. Vyas, C. Hicks, and V. Mavroudis, “Direction for detection: A survey of automated vulnerability detection and all of its pain points,” Transactions on Machine Learning Research (TMLR), 2026. [Online]. Available: https://openreview.net/forum?id=01TkT5p3xT
  36. [36]Q. Zhang, C. Fang, Y. Xie, Y. Ma, W. Sun, Y. Yang, and Z. Chen, “A systematic literature review on large language models for automated program repair,” ACM Transactions on Software Engineering and Methodology, 2026.
  37. [37]Y. Li, F. H. Shen, B. Wei, G. Wang, and Y. Tian, “SoK: Towards effective automated vulnerability repair,” in Proceedings of the 34th USENIX Security Symposium, 2025, pp. 4441–4462.arxiv.org/abs/2501.18820
  38. [38]Y. Hu, Z. Li, K. Shu, S. Guan, D. Zou, S. Xu, B. Yuan, and H. Jin, “SoK: Automated vulnerability repair: Methods, tools, and assessments,” in Proceedings of the 34th USENIX Security Symposium, 2025, pp. 4421–4440.arxiv.org/abs/2506.11697
  39. [39]E. Bašić and A. Giaretta, “From vulnerabilities to remediation: A systematic literature review of LLMs in code security,” arXiv:2412.15004, 2024. [Online]. Available: https://arxiv.org/abs/2412.15004
  40. [40]R. Simon and W. Mees, “SoK: A comparison of autonomous penetration testing agents,” in Proceedings of the 19th International Conference on Availability, Reliability and Security (ARES), 2024.DOI
  41. [41]Z. He, J. Dong, Z. Li, T. Chen, G. Deng, F. Luo, J. Ji, Y. Cao, and X. Luo, “A survey of LLM-driven penetration testing: Taxonomy, co-evolution, and open challenges,” arXiv:2607.02605, 2026. [Online]. Available: https://arxiv.org/abs/2607.02605
  42. [42]A. Shahriar, M. N. Rahman, S. Ahmed, F. Sadeque et al., “A survey on agentic security: Applications, threats and defenses,” Transactions on Machine Learning Research (TMLR), 2026.
  43. [43]A. K. Zhang et al., “Cybench: A framework for evaluating cybersecurity capabilities and risks of language models,” in Proceedings of the 13th International Conference on Learning Representations (ICLR), 2025.
  44. [44]A. K. Zhang et al., “BountyBench: Dollar impact of AI agent attackers and defenders on real-world cybersecurity systems,” in Advances in Neural Information Processing Systems (NeurIPS), 2025. [Online]. Available: https://arxiv.org/abs/2505.15216
  45. [45]A. Labath, M. Suri, A. Delaître, V. Okun, M.-R. Amini, R. D. Sriram, and D. Manocha, “CyberForge: Verified vulnerability injection at repository level for cybersecurity agent training,” arXiv:2608.06471, 2026. [Online]. Available: https://arxiv.org/abs/2608.06471
  46. [46]M. Khurana and R. Jain, “SoK: Measuring what matters for closed-loop security agents,” in Proceedings of the IEEE International Conference on Data Mining Workshops (ICDMW), 2025, pp. 1274–1283.arxiv.org/abs/2510.01654
  47. [47]A. Pesoli, H. Errico, and L. Cavallaro, “Demystifying the Mythos or disrupting bugonomics? From zero-day asymmetry to defender remediation throughput,” arXiv:2605.24632, 2026. [Online]. Available: https://arxiv.org/abs/2605.24632
  48. [48]UK AI Security Institute, “Frontier AI trends report,” Dec. 2025. [Online]. Available: https://aisi.s3.eu-west-2.amazonaws.com/Frontier+AI+Trends+Report+-+AI+Security+Institute.pdf
  49. [49]Center for AI Standards and Innovation, “CAISI’s assessment of Z.ai’s GLM-5.3 cyber capabilities,” NIST news, Sep. 2026. [Online]. Available: https://www.nist.gov/news-events/news/2026/09/caisi-assessment-zais-glm-53-cyber-capabilities
  50. [50]J. Lee and A. Barry, “Open-weight models trail closed models in cyber capabilities by 6–7 months,” Epoch AI data insight, Oct. 2026. [Online]. Available: https://epoch.ai/data-insights/open-closed-cyber-eci-gap
  51. [51]T. Avgerinos, D. Brumley, J. Davis, R. Goulden, T. Nighswander, A. Rebert, and N. Williamson, “The Mayhem cyber reasoning system,” IEEE Security & Privacy, vol. 16, no. 2, pp. 52–60, 2018.
  52. [52]Z. Qi, F. Long, S. Achour, and M. Rinard, “An analysis of patch plausibility and correctness for generate-and-validate patch generation systems,” in Proceedings of the International Symposium on Software Testing and Analysis (ISSTA), 2015, pp. 24–36.DOI
  53. [53]E. K. Smith, E. T. Barr, C. Le Goues, and Y. Brun, “Is the cure worse than the disease? Overfitting in automated program repair,” in Proceedings of the 10th Joint Meeting on Foundations of Software Engineering (ESEC/FSE), 2015, pp. 532–543.DOI
  54. [54]M. J. Page, J. E. McKenzie, P. M. Bossuyt et al., “The PRISMA 2020 statement: An updated guideline for reporting systematic reviews,” BMJ, vol. 372, p. n71, 2021.
  55. [55]J. Cohen, “A coefficient of agreement for nominal scales,” Educational and Psychological Measurement, vol. 20, no. 1, pp. 37–46, 1960.
  56. [56]Zero Day Initiative, “Published advisories,” 2026, accessed Nov. 2025. [Online]. Available: https://www.zerodayinitiative.com/advisories/published/
  57. [57]Chromium Project, “Chromium issue tracker,” 2026, accessed Nov. 2026. [Online]. Available: https://issues.chromium.org/
  58. [58]Exploit Database, “The official Exploit Database repository,” GitLab repository, 2026, accessed Nov. 2026. [Online]. Available: https://gitlab.com/exploit-database/exploitdb
  59. [59]Rapid7, “Metasploit Framework,” GitHub repository, 2026, accessed Nov. 2026. [Online]. Available: https://github.com/rapid7/metasploit-framework
  60. [60]nomi-sec, “PoC-in-GitHub,” GitHub repository, 2026, accessed Nov. 2026. [Online]. Available: https://github.com/nomi-sec/PoC-in-GitHub
  61. [61]Cybersecurity and Infrastructure Security Agency, “Known exploited vulnerabilities catalog,” JSON feed, catalog version 2026.11.09, 2026, accessed Nov. 2026. [Online]. Available: https://www.cisa.gov/sites/default/files/feeds/known_exploited_vulnerabilities.json
  62. [62]Anthropic, “Anthropic’s coordinated vulnerability disclosure dashboard,” Anthropic Red Team, Nov. 2026, last updated 2 Nov. 2026; accessed Nov. 2026. [Online]. Available: https://red.anthropic.com/2026/cvd/
  63. [63]Trail of Bits, “Patch the planet,” Trail of Bits program page, 2026. [Online]. Available: https://www.trailofbits.com/patch-the-planet/
  64. [64]Google Project Zero, “Project Zero issue tracker,” 2026, accessed Nov. 2026. [Online]. Available: https://project-zero.issues.chromium.org/
  65. [65]Google, “Big Sleep public issue tracker,” Google issue tracker, 2026, accessed 31 October 2026. [Online]. Available: https://issuetracker.google.com/search?q=7155917
  66. [66]syzkaller project, “syzbot: Linux upstream dashboard,” 2026, accessed Nov. 2026. [Online]. Available: https://syzkaller.appspot.com/upstream
  67. [67]L. Carettoni and A. Trummer, “Comparing AI application security testing platforms: Aikido vs. XBOW,” Doyensec report, May 2026. [Online]. Available: https://doyensec.com/resources/ComparingAIApplicationSecurityTestingPlatforms_Doyensec.pdf
  68. [68]A. Fasano, M. Fleischer, C. McFaul, R. Xiao, and T. Gallagher, “GLM-5.3 and the spread of advanced cyber capabilities,” Anthropic research, Sep. 2026. [Online]. Available: https://www.anthropic.com/research/glm-5-3-and-the-spread-of-advanced-cyber-capabilities
  69. [69]G. Galea Curmi and B. Wilde, “How much does the harness matter, and for whom? A multi-model study of context in LLM-driven vulnerability discovery,” arXiv:2610.28802, 2026, accepted to CAMLIS 2026. [Online]. Available: https://arxiv.org/abs/2610.28802
  70. [70]H. Larcher, A. Carreira, raphael g, and C. Rannou, “Anatomy of a frontier lab agent intrusion: A technical timeline of the July 2026 incident,” Hugging Face blog, Jul. 2026. [Online]. Available: https://huggingface.co/blog/agent-intrusion-technical-timeline
  71. [71]METR, “Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident,” METR blog, Aug. 2026. [Online]. Available: https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/
  72. [72]UK AI Security Institute, “Incident report: Unsanctioned agent behaviour during cyber testing,” AISI blog, Aug. 2026. [Online]. Available: https://www.aisi.gov.uk/blog/incident-report-unsanctioned-agent-behaviour-during-cyber-testing
  73. [73]Anthropic, “Investigating unintended model actions in our evaluations and internal use,” Anthropic, Oct. 2026. [Online]. Available: https://www.anthropic.com/research/investigating-unintended-model-actions
  74. [74]L. Abrams, “Meta AI model hacked a company during misconfigured cyber test,” BleepingComputer, Aug. 2026. [Online]. Available: https://www.bleepingcomputer.com/news/security/meta-ai-model-hacked-a-company-during-misconfigured-cyber-test
  75. [75]R. A. Popa and F. Flynn, “Introducing CodeMender: An AI agent for code security,” Google DeepMind blog, Oct. 2025. [Online]. Available: https://deepmind.google/blog/introducing-codemender-an-ai-agent-for-code-security/
  76. [76]M. Levin, “Casting a Net(ty) for bugs, and catching a big one (CVE-2025-59419),” depthfirst research blog, Oct. 2025. [Online]. Available: https://depthfirst.com/research/casting-a-net-ty-for-bugs-and-catching-a-big-one-cve-2025-59419
  77. [77]Z. Li, Z. Yu, J. Song, M. Xu, Y. Luo, and D. Mu, “PortGPT: Towards automated backpacking using large language models,” in Proceedings of the 47th IEEE Symposium on Security and Privacy (S&P), 2026.
  78. [78]OpenAI, “The Defense Factory,” OpenAI, Sep. 2026. [Online]. Available: https://openai.com/the-defense-factory/
  79. [79]ByteDance, “PatchEval-Verified: Results in the PatchEval repository,” GitHub repository, Jul. 2026. [Online]. Available: https://github.com/bytedance/PatchEval
  80. [80]K. Townsend, “AISLE emerges from stealth with AI-based reasoning system to remediate vulnerabilities on the fly,” SecurityWeek, Oct. 2025. [Online]. Available: https://www.securityweek.com/aisle-emerges-from-stealth-with-ai-based-reasoning-system-that-remediates-vulnerabilities-on-the-fly/
  81. [81]latticelabs-au, “Logan: AI security remediation framework,” GitHub repository laticelabs-au/logan, 2026. [Online]. Available: https://github.com/latticelabs-au/logan
  82. [82]Z. Zhang, M. Pedhapati, and F. Faessler, “Introducing Hacktron Review,” Hacktron AI blog, Apr. 2026. [Online]. Available: https://www.hacktron.ai/blog/introducing-hacktron-review
  83. [83]Mobb, “vuln-fix-bench: Can AI coding agents fix real, disclosed CVEs without internet access?” GitHub repository, Jun. 2026. [Online]. Available: https://github.com/mobb-dev/fix-vuln-code-benchmark
  84. [84]S. Wu, R. Wang, Y. Cao, B. Chen, Z. Zhou, Y. Huang, J. Zhao, and X. Peng, “Mystique: Automated vulnerability patch porting with semantic and syntactic-enhanced LLM,” Proceedings of the ACM on Software Engineering, vol. 2, no. FSE, pp. 1310–152, 2025.
  85. [85]Y. Shi, Q. Liu, T. Zhou, W. Tan, C. Si, Y. Lyu et al., “EviRepair: Evidence-driven agents for reliable vulnerability repair,” arXiv:2610.13491, 2026. [Online]. Available: https://arxiv.org/abs/2610.13491
  86. [86]J. Clinton, “The defender advantage has a single point of failure,” Keynote, Confidential Computing Summit 2026, San Francisco, Jun. 2026, recording at https://youtu.be/70IMe3YKhDo?t=620. [Online]. Available: https://ccsummit2026.sched.com/event/2NKXz/
  87. [87]D. Liu, O. Chang, J. Nowakowski, and J. Keller, “Scaling security with AI: From detection to solution,” Google Security Blog, Jan. 2024. [Online]. Available: https://security.googleblog.com/2024/01/scaling-security-with-ai-from-detection.html
  88. [88]Protect AI, “Vulnhuntr: Zero shot vulnerability discovery using LLMs,” GitHub repository, Oct. 2024. [Online]. Available: https://github.com/protectai/vulnhuntr
  89. [89]J. Guo, C. Wang, X. Xu, Z. Su, and X. Zhang, “RepoAudit: An autonomous LLM-agent for repository-level code auditing,” in Proceedings of the 42nd International Conference on Machine Learning (ICML), ser. Proceedings of Machine Learning Research, vol. 267, 2025, pp. 21 083–21 100.arxiv.org/abs/2501.18160
  90. [90]N. Waisman, “The road to top 1: How XBOW did it,” XBOW blog, Jun. 2025. [Online]. Available: https://xbow.com/blog/top-1-how-xbow-did-it/
  91. [91]Z. Wang and L. Zhou, “Agentic discovery and validation of Android app vulnerabilities,” arXiv:2508.21579, 2025. [Online]. Available: https://arxiv.org/abs/2508.21579
  92. [92]H. Liu, C. Shou, X. Liu, H. Wen, Y. Chen, R. J. Fang, and Y. Feng, “Synthesizing multi-agent harnesses for vulnerability discovery,” arXiv:2604.20801, 2026. [Online]. Available: https://arxiv.org/abs/2604.20801
  93. [93]D. Stenberg, ““ignore the slop”,” daniel.haxx.se, Oct. 2026. [Online]. Available: https://daniel.haxx.se/blog/2026/10/22/ignore-the-slop/
  94. [94]D. Stenberg, “The end of the curl bug-bounty,” daniel.haxx.se, Jan. 2026. [Online]. Available: https://daniel.haxx.se/blog/2026/01/26/the-end-of-the-curl-bug-bounty/
  95. [95]wolfSSL, “wolfSSL featured in Anthropic’s new open-source security initiative,” wolfSSL blog, Oct. 2026. [Online]. Available: https://www.wolfssl.com/wolfssl-featured-in-anthropics-new-open-source-security-initiative/
  96. [96]M. Catanzaro, “The era of software quality, or the era of ostriches?” blogs.gnome.org, Oct. 2026. [Online]. Available: https://blogs.gnome.org/mcatanzaro/2026/10/02/the-era-of-software-quality-or-the-era-of-ostriches/
  97. [97]D. Stenberg, “Do minor curl vulnerabilities in the open,” daniel.haxx.se, Oct. 2026. [Online]. Available: https://daniel.haxx.se/blog/2026/10/20/do-minor-curl-vulnerabilities-in-the-open/
  98. [98]A. Gaynor, “You can’t bug fix your way out of the vulnocalypse,” alexgaynor.net, Jul. 2026. [Online]. Available: https://alexgaynor.net/2026/jul/15/you-cant-bug-fix-your-way-out-of-the-vulnocalypse/
  99. [99]D. Stenberg, “Approaching zero bugs?” daniel.haxx.se, Apr. 2026. [Online]. Available: https://daniel.haxx.se/blog/2026/04/30/approaching-zero-bugs/
  100. [100]GitHub, “Inside the advisory database and what happens when vulnerability volume breaks records,” GitHub Blog, Jun. 2026. [Online]. Available: https://github.blog/security/supply-chain-security/inside-the-advisory-database-and-what-happens-when-vulnerability-volume-breaks-records/
  101. [101]Anthropic, “Introducing the Anthropic Cyber Mission,” Anthropic news, Oct. 2026. [Online]. Available: https://www.anthropic.com/news/anthropic-cyber-mission
  102. [102]HackerOne, “Internet bug bounty,” HackerOne program page, 2026. [Online]. Available: https://hackerone.com/ibb
  103. [103]Mandiant, “M-Trends 2026,” Mandiant special report (Google Cloud), Mar. 2026. [Online]. Available: https://cloud.google.com/security/resources/m-trends
  104. [104]P. Garrity, “VulnCheck state of exploitation 1H-2026,” VulnCheck blog, Jul. 2026. [Online]. Available: https://www.vulncheck.com/blog/state-of-exploitation-1h-2026
  105. [105]A. Ament, Y. Edelman, A. Barry, T. Chauvin, L. Bousfield, and J. Lee, “Open-weight AI models can exploit the patch gap,” Epoch AI, Oct. 2026. [Online]. Available: https://epoch.ai/publications/open-weight-models-patch-gap
  106. [106]Z.ai, “GLM-5.3 model card,” Hugging Face, Aug. 2026. [Online]. Available: https://huggingface.co/zai-org/GLM-5.3
  107. [107]RogoLabs, “CVEForecast: CVE publication predictions with machine learning,” cveforecast.org, forecast of 7 November 2026, 2026. [Online]. Available: https://cveforecast.org/
  108. [108]J. Clinton, “‘“they are going to get a lot worse before they get better”,’ Post on X, 22 October 2026, Oct. 2026. [Online]. Available: https://x.com/JasonDClinton/status/2113325263157395456
  109. [109]Z. Y. Ding and C. Le Goues, “An empirical study of OSS-Fuzz bugs,” in Proceedings of the 18th IEEE/ACM International Conference on Mining Software Repositories (MSR), May 2021, pp. 131–142. [Online]. Available: https://arxiv.org/abs/2103.11518
  110. [110]M. Böhme and B. Falk, “Fuzzing: On the exponential cost of vulnerability discovery,” in Proceedings of the 28th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE), 2020, pp. 713–724.
  111. [111]OpenAI, “A thousand fewer paths to root,” OpenAI, Nov. 2026. [Online]. Available: https://openai.com/index/a-thousand-fewer-paths-to-root/
  112. [112]M. Brown, “Reproducing top results from the AIxCC with general access AI models and $600,” RunSybil blog, Jun. 2026. [Online]. Available: https://www.runsybil.com/blog/reproducing-top-results-from-the-aixcc-with-general-access-ai-models-and-600
  113. [113]W. Xiao, C. Killian, H. Sleight, A. Chan, N. Carlini, and A. Peng, “AI agents find $4.6M in blockchain smart contract exploits,” Anthropic research, Dec. 2025. [Online]. Available: https://www.anthropic.com/research/smart-contracts
  114. [114]L. Emberson and D. Roodman, “The plunging price of thought,” Epoch AI, Sep. 2026. [Online]. Available: https://epoch.ai/publications/the-plunging-price-of-thought
  115. [115]A. Mierczuk, S. Michaels, and K. Hoodlet, “LLM-generated patches for complex, novel vulnerabilities are often F.L.A.W.E.D.: Fix-like artifacts with embedded defects: Common failure modes of LLM-generated security patches,” 1Password Off-by-1 Labs report, Aug. 2026, released Aug. 2026 as Frontier Models’ Vulnerability Patches Are Often F.L.A.W.E.D.; revised Nov. 2026. [Online]. Available: https://1password.com/files/resources/frontier-models-vulnerability-patches-flawed.pdf
  116. [116]Endor Labs, “Better models are making agent patch review more expensive, not less,” Endor Labs blog, Sep. 2026. [Online]. Available: https://www.endorlabs.com/learn/two-new-models-tested-two-key-takeaways
  117. [117]B. Schneier, “Should U.S. hackers fix cybersecurity holes or exploit them?” The Atlantic, May 2014, 19 May 2014; author’s archive copy, accessed Nov. 2026. [Online]. Available: https://www.schneier.com/essays/archives/2014/05/should_us_hackers_fix.html
  118. [118]D. Geer, “Cybersecurity as realpolitik,” Keynote, Black Hat USA, Aug. 2014, 6 Aug. 2014; accessed Nov. 2026. [Online]. Available: http://geer.tinho.net/geer.blackhat.6viii14.txt
  119. [119]Zero Day Initiative, “Disclosure policy,” Trend Micro Zero Day Initiative, 2026. [Online]. Available: https://www.zerodayinitiative.com/advisories/disclosure_policy/
  120. [120]R. Schoen, “A walk through Project Zero metrics,” Google Project Zero blog, Feb. 2022. [Online]. Available: https://projectzero.google/2022/02/a-walk-through-project-zero-metrics.html
  121. [121]FuzzingBrain, “62 vulnerabilities, 26 projects, 36 patches,” FuzzingBrain blog, Feb. 2026. [Online]. Available: https://fuzzingbrain.github.io/posts/zero-day-vulnerability-discoveries.html
  122. [122]J. Vander Stoep and A. Rebert, “Eliminating memory safety vulnerabilities at the source,” Google Online Security Blog, Sep. 2024. [Online]. Available: https://security.googleblog.com/2024/09/eliminating-memory-safety-vulnerabilities-Android.html
  123. [123]J. Vander Stoep, “Rust in Android: Move fast and fix things,” Google Online Security Blog, Nov. 2025. [Online]. Available: https://blog.google/security/rust-in-android-move-fast-fix-things/
  124. [124]A. Kurmus, R. Tartler, D. Dormeanu, B. Reinholt, V. Rothberg, A. Ruprecht, W. Schröder-Preikschat, D. Lohmann, and R. Kapitza, “Attack surface metrics and automated compile-time OS kernel tailoring,” in Proceedings of the 20th Network and Distributed System Security Symposium (NDSS), Feb. 2013. [Online]. Available: https://www.ndss-symposium.org/ndss2013/attack-surface-metrics-and-automated-compile-time-os-kernel-tailoring
  125. [125]C. Rohlf, “Using Claude to reduce the attack surface of code,” Anthropic research, Oct. 2026. [Online]. Available: https://www.anthropic.com/research/attack-surface-reduction
  126. [126]B. Grinstead, C. Holler, and F. Braun, “Behind the scenes hardening Firefox with Claude Mythos Preview,” Mozilla Hacks, May 2026. [Online]. Available: https://hacks.mozilla.org/2026/05/behind-the-scenes-hardening-firefox/
  127. [127]Chrome Security Team, “Stronger with every update: How we’re making Chrome and the web safer in the AI era,” Google blog, Jul. 2026. [Online]. Available: https://blog.google/security/chrome-stronger-with-every-update/
  128. [128]Microsoft Security Response Center, “Security update guide: October 2026 security updates,” Oct. 2026, counts from the CVRF documents of October 2025 and October 2026. [Online]. Available: https://msrc.microsoft.com/update-guide/releaseNote/2026-Oct
  129. [129]K. Lucas, A. Lynch, L. Tong, L. Champine, Z. Straznickas, D. Distefano et al., “LLMs are making formal software verification feasible at a much larger scale,” Anthropic research, Oct. 2026. [Online]. Available: https://www.anthropic.com/research/formal-verification
  130. [130]A. Sajadi, K. Damevski, and P. Chatterjee, “How safe are AI-generated patches? A large-scale study on security risks in LLM and agentic automated program repair on SWE-bench,” arXiv:2507.02976, 2025. [Online]. Available: https://arxiv.org/abs/2507.02976
  131. [131]M. Alfadel, D. E. Costa, E. Shihab, and M. Mkhallalati, “On the use of Dependabot security pull requests,” in Proceedings of the 18th IEEE/ACM International Conference on Mining Software Repositories (MSR), 2021, pp. 254–265.
  132. [132]R. G. Kula, D. M. German, A. Ouni, T. Ishio, and K. Inoue, “Do developers update their library dependencies? An empirical study on the impact of security advisories on library migration,” Empirical Software Engineering, vol. 23, no. 1, pp. 384–417, 2018.
  133. [133]A. Nappa, R. Johnson, L. Bilge, J. Caballero, and T. Dumitraş, “The attack of the clones: A study of the impact of shared code on vulnerability patching,” in Proceedings of the IEEE Symposium on Security and Privacy (S&P), 2015, pp. 692–708. [Online]. Available: https://www.ieee-security.org/TC/SP2015/papers-archive/d/6949a692.pdf
  134. [134]T. Duebendorfer and S. Frei, “Why silent updates boost security,” ETH Zurich, Computer Engineering and Networks Laboratory (TIK), Tech. Rep. TIK Report 302, May 2009. [Online]. Available: https://techzoom.net/whitepapers/why_silent_updates_boost_security_2009.pdf
  135. [135]C. Schneider, J. Zhang, and H. Blauzvern, “Streamlining Google’s OSS VRP: Key rule updates,” Google Bug Hunters blog, Mar. 2026. [Online]. Available: https://bughunters.google.com/blog/ossvrp-rule-updates-2026
  136. [136]J. Brown, “Raising the bar: Quality, shared responsibility, and the future of GitHub’s bug bounty program,” The GitHub Blog, May 2026. [Online]. Available: https://github.blog/security/raising-the-bar-quality-shared-responsibility-and-the-future-of-githubs-bug-bounty-program/
  137. [137]C. Reis, A. Moshchuk, and N. Oskov, “Site isolation: Process separation for web sites within the browser,” in Proceedings of the 28th USENIX Security Symposium, 2019, pp. 1661–1678. [Online]. Available: https://www.usenix.org/conference/usenixsecurity19/presentation/reis
  138. [138]R. Joyce, “Getting “Mythos ready” with your defenses,” joycecyber.com, Apr. 2026. [Online]. Available: https://joycecyber.com/getting-mythos-ready-with-your-defenses
  139. [139]O. Whitehouse, “Preparing for a ‘vulnerability patch wave’,” NCSC blog, May 2026. [Online]. Available: https://www.ncsc.gov.uk/sites/default/files/2026-10/Preparing-for-a-%E2%80%98vulnerability-patch-wave%E2%80%99.pdf
  140. [140]R. Pato Nogueira, K. Pattabiraman, M. Vieira, and J. R. Campos, “A systematic methodology for evaluating failure independence in LLM-generated code,” arXiv:2607.02808, 2026. [Online]. Available: https://arxiv.org/abs/2607.02808
  141. [141]S. Freitas and A. Gharib, “GenAI-driven threat detection with Microsoft Security Copilot,” arXiv:2605.20896, May 2026. [Online]. Available: https://arxiv.org/abs/2605.20896
  142. [142]S. A. Tsegai, A. Kantchelian, D. Yao, and P. Gao, “From investigation failures to reliable SOC agents: Understanding and improving LLM-based alert triage,” arXiv:2610.10608, 2026. [Online]. Available: https://arxiv.org/abs/2610.10608
  143. [143]R. Pandey and A. Bhujang, “Poisoning the watchtower: Prompt injection attacks against LLM-augmented security operations through adversarial log content,” arXiv:2605.24421, 2026. [Online]. Available: https://arxiv.org/abs/2605.24421
  144. [144]OpenAI, “GPT-5.3-Codex system card,” OpenAI Deployment Safety Hub, Feb. 2026. [Online]. Available: https://deploymentsafety.openai.com/gpt-5-3-codex/cybersecurity
  145. [145]Anthropic, “Expanding the Cyber Verification Program,” Anthropic news, Oct. 2026. [Online]. Available: https://www.anthropic.com/news/cyber-verification-program
  146. [146]Anthropic, “Disrupting an AI-orchestrated cyber espionage campaign,” Anthropic news, Nov. 2025. [Online]. Available: https://www.anthropic.com/news/disrupting-ai-espionage
  147. [147]E. Sela, “Autonomous AI agents are breaking into hundreds of online retailers for $25 a target in an ongoing campaign,” Gambit Security blog, Sep. 2026. [Online]. Available: https://gambit.security/blog-posts/autonomous-ai-agents-online-retailers-25-a-company
  148. [148]Google Threat Intelligence Group, “GTIG AI Threat Tracker: Adversaries leverage AI for vulnerability exploitation, augmented operations, and initial access,” Google Threat Intelligence Group, May 2026. [Online]. Available: https://cloud.google.com/blog/topics/threat-intelligence/ai-vulnerability-exploitation-initial-access
  149. [149]Verizon, “2026 data breach investigations report,” Verizon Business, May 2026. [Online]. Available: https://www.verizon.com/business/resources/T158/reports/2026-dbir-data-breach-investigations-report.pdf
  150. [150]M. Shetty, N. Jain, A. Godbole, S. A. Seshia, and K. Sen, “Syzygy: Dual code-test C to (safe) Rust translation using LLMs and dynamic analysis,” arXiv:2412.14234, 2024, preliminary version at LLM4Code 2025. [Online]. Available: https://arxiv.org/abs/2412.14234
  151. [151]M. Farrukh, B. Coskun, T. Palit, and M. Polychronakis, “SafeTrans: LLM-assisted transpilation from C to Rust,” arXiv:2505.10708, 2025. [Online]. Available: https://arxiv.org/abs/2505.10708
  152. [152]X. Deng, S. Zhong, B. Bayazıt, A. Veneris, F. Long, and X. Si, “VerifyThisBench: Generating code, specifications, and proofs all at once,” arXiv:2505.19271, 2025. [Online]. Available: https://arxiv.org/abs/2505.19271
  153. [153]P. Fonseca, K. Zhang, X. Wang, and A. Krishnamurthy, “An empirical study on the correctness of formally verified distributed systems,” in Proceedings of the 12th European Conference on Computer Systems (EuroSys), 2017, pp. 328–343.
  154. [154]S. Narayan, C. Disselkoen, T. Garfinkel, N. Froyd, E. Rahm, S. Lerner et al., “Retrofitting fine grain isolation in the Firefox renderer,” in Proceedings of the 29th USENIX Security Symposium, 2020. [Online]. Available: https://www.usenix.org/conference/usenixsecurity20/presentation/narayan
  155. [155]H. Lefuevre, N. Dautenhahn, D. Chisnall, and P. Olivier, “SoK: Software compartmentalization,” in Proceedings of the 46th IEEE Symposium on Security and Privacy (S&P), 2025. [Online]. Available: https://arxiv.org/abs/2410.08434
  156. [156]The Linux kernel developers, “Security bugs,” Linux kernel documentation, Documentation/process/security-bugs.rst, 2026. [Online]. Available: https://github.com/torvalds/linux/blob/master/Documentation/process/security-bugs.rst
  157. [157]K. Moussouris, “PHP2Shell: Multiparty disclosure, same as it ever was,” Luta Security, Oct. 2026. [Online]. Available: https://www.lutasecurity.com/post/php2shell-multiparty-disclosure-same-as-it-ever-was
  158. [158]C. Moses, “Amazon threat intelligence identifies PHP2Shell exploitation prior to public disclosure in MadPot,” AWS Security Blog, Oct. 2026. [Online]. Available: https://aws.amazon.com/blogs/security/amazon-threat-intelligence-identifies-php2shell-exploitation-prior-to-public-disclosure-in-madpot/
  159. [159]S. Robertson and H. Zaragoza, “The probabilistic relevance framework: BM25 and beyond,” Foundations and Trends in Information Retrieval, vol. 3, no. 4, pp. 333–389, 2009.
  160. [160]Y. Zhang, M. Li, D. Long, X. Zhang, H. Lin, B. Yang, P. Xie, A. Yang, D. Liu, J. Lin, F. Huang, and J. Zhou, “Qwen3 Embedding: Advancing text embedding and reranking through foundation models,” arXiv:2506.05176, 2025. [Online]. Available: https://arxiv.org/abs/2506.05176
  161. [161]C. J. C. Burges, “From RankNet to LambdaRank to LambdaMART: An overview,” Microsoft Research, Tech. Rep. MSR-TR-2010-82, 2010.
  162. [162]E. M. Voorhees and D. K. Harman, Eds., TREC: Experiment and Evaluation in Information Retrieval. MIT Press, 2005.
  163. [163]K. Järvelin and J. Kekäläinen, “Cumulative gain-based evaluation of IR techniques,” ACM Transactions on Information Systems, vol. 20, no. 4, pp. 422–446, 2002.
  164. [164]CVE Program, “List of CVE numbering authorities (CNAsList.json),” GitHub repository CVEProject/cve-website, 2026. [Online]. Available: https://github.com/CVEProject/cve-website/blob/dev/src/assets/data/CNAsList.json
  165. [165]C. Shou, J. Liu, D. Lu, and K. Sen, “LLM4Fuzz: Guided fuzzing of smart contracts with large language models,” arXiv:2401.11108, Jan. 2024. [Online]. Available: https://arxiv.org/abs/2401.11108
  166. [166]M. S. Bouaff, C. Zheng, I. A. Qasse, E. Zulkoski, M. Hamdaqa, and F. Khomh, “A context-driven approach for co-auditing smart contracts with the support of GPT-4 code interpreter,” arXiv:2406.18075, 2024. [Online]. Available: https://arxiv.org/abs/2406.18075
  167. [167]D. Wang, G. Zhou, L. Chen, D. Li, and Y. Miao, “Prophetfuzz: Fully automated prediction and fuzzing of high-risk option combinations with only documentation via large language model,” in Proceedings of the ACM SIGSAC Conference on Computer and Communications Security (CCS), 2024, pp. 735–749.
  168. [168]X. Shen, L. Wang, Z. Li, Y. Chen, W. Zhao, D. Sun, J. Wang, and W. Ruan, “Penetration Testing: Incorporating LLM agents to automated penetration testing,” in Proceedings of the ACM Asia Conference on Computer and Communications Security (ASIA CCS), 2025, pp. 375–391.
  169. [169]Y. Ji, T. Dai, Z. Zhou, Y. Tang, and J. He, “Artemis: Toward accurate detection of server-side request forgeries through LLM-assisted inter-procedural path-sensitive taint analysis,” Proceedings of the ACM on Programming Languages, vol. 9, no. OOPSLA1, pp. 1349–1377, 2025.
  170. [170]Anthropic, “Launching an opt-in vulnerability-finding service for open-source software,” Anthropic research, Oct. 2026. [Online]. Available: https://www.anthropic.com/research/launching-opt-in-vuln-finding-service-for-open-source
  171. [171]OpenAI, “OpenAI and Hugging Face partner to address security incident during model evaluation,” OpenAI, Jul. 2026. [Online]. Available: https://openai.com/index/hugging-face-model-evaluation-security-incident/
  172. [172]X. Zhang, Z. Chen, H. Wang, and Q. Li, “LLMs as firmware experts: A runtime-grown tree-of-agents framework,” arXiv:2511.18438, 2025. [Online]. Available: https://arxiv.org/abs/2511.18438
  173. [173]Y. Lyu, P. Chen, F. Wu, J. Yu, and H. Chen, “FuzzAgent: Multi-agent system for end-to-end evolutionary library fuzzing,” arXiv:2605.14431, 2026. [Online]. Available: https://arxiv.org/abs/2605.14431
  174. [174]I. David and A. Gervais, “Multi-agent penetration testing AI for the web,” arXiv:2508.20816, 2025. [Online]. Available: https://arxiv.org/abs/2508.20816
  175. [175]UK AI Security Institute, “Our evaluation of OpenAI’s GPT-5.5 cyber capabilities,” AISI blog, Apr. 2026. [Online]. Available: https://www.aisi.gov.uk/blog/our-evaluation-of-openais-gpt-5-5-cyber-capabilities
  176. [176]OpenAI, “Codex Security: Now in research preview,” OpenAI blog, Mar. 2026. [Online]. Available: https://openai.com/index/codex-security-now-in-research-preview/
  177. [177]Y. Hou, H. Wang, M. Lyu, M. Momeu, E. Nguyen, T. Yang, K. Sen, D. Song, and D. Wagner, “Revelio: Cost-efficient agentic memory safety vulnerability detection for repository-scale codebases,” arXiv:2606.22263, 2026. [Online]. Available: https://arxiv.org/abs/2606.22263
  178. [178]J. Eom, S. Jeong, and T. Kwon, “Fuzzing JavaScript interpreters with coverage-guided reinforcement learning for LLM-based mutation,” in Proceedings of the ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA), 2024, pp. 1656–1668.
  179. [179]Tenzai, “An open-weight model powered the #1 HackerOne VDP research operation,” Tenzai blog, Oct. 2026. [Online]. Available: https://tenzai.com/blog/an-open-weight-model-powered-the-1-hackerone-vdp-research-operation
  180. [180]R. Fang, R. Bindu, A. Gupta, and D. Kang, “LLM agents can autonomously exploit one-day vulnerabilities,” arXiv:2404.08144, 2024. [Online]. Available: https://arxiv.org/abs/2404.08144
  181. [181]B. Richardson, N. Sessions, M. Demoret, R. Allen, and H. Chen, “Applying generative AI for CVE analysis at an enterprise scale,” NVIDIA Technical Blog, May 2024. [Online]. Available: https://developer.nvidia.com/blog/applying-generative-ai-for-cve-analysis-at-an-enterprise-scale/
  182. [182]V. Agrawal and K. Ahi, “LLM-driven SAST-Genius: A hybrid static analysis framework for comprehensive and actionable security,” arXiv:2509.15433, 2025. [Online]. Available: https://arxiv.org/abs/2509.15433
  183. [183]Y. Xiong and T. Zhang, “Sifting the noise: A comparative study of LLM agents in vulnerability false positive filtering,” Proceedings of the ACM on Software Engineering, vol. 3, no. ISSTA, pp. 197–220, 2026.
  184. [184]M. Imranmanesh, S. M. Sabet, S. Marefat, A. J. Ghasr, A. Wilson, I. Sharafaldin, and M. A. Tayebi, “ZeroFalse: Improving Precision in Static Analysis with LLMs,” arXiv:2510.02534, 2025. [Online]. Available: https://arxiv.org/abs/2510.02534
  185. [185]S. Ullah, P. Balasubramanian, W. Guo, A. Burnett, H. Pearce, C. Kruegel, G. Vigna, and G. Stringhini, “From CVE entries to verifiable exploits: An automated multi-agent framework for reproducing CVEs,” arXiv:2509.01835, 2025. [Online]. Available: https://arxiv.org/abs/2509.01835
  186. [186]A. Desai, M. Shafuzzaman, W.-B. Guo, and T. Bultan, “PAGENT: Program analysis guided LLM agent for proof-of-concept generation,” Proceedings of the ACM on Software Engineering, vol. 3, no. ISSTA, pp. 2475–2497, 2026.
  187. [187]J. Keller and J. Nowakowski, “AI-powered patching: the future of automated vulnerability fixes,” Google Research (Google Security Engineering Technical Report), 2024. [Online]. Available: https://research.google/pubs/ai-powered-patching-the-future-of-automated-vulnerability-fixes/
  188. [188]Y. Zhang, J. Wang, D. Berzin, M. Mirchev, D. Liu, A. Arya, O. Chang, and A. Roychoudhury, “Fixing security vulnerabilities with AI in OSS-Fuzz,” arXiv:2411.03346, 2024. [Online]. Available: https://arxiv.org/abs/2411.03346
  189. [189]P. Tempel and E. Tooley, “Found means fixed: Introducing code scanning autofix, powered by GitHub Copilot and CodeQL,” The GitHub Blog, Mar. 2024. [Online]. Available: https://github.blog/news-insights/product-news/found-means-fixed-introducing-code-scanning-autofix-powered-by-github-copilot-and-codeql/
  190. [190]T. K. Le, S. Alimadadi, and S. Y. Ko, “A study of vulnerability repair in JavaScript programs with large language models,” in Companion Proceedings of the ACM Web Conference (WWW), 2024, pp. 666–669.
  191. [191]T. Kim, H. Han, S. Park et al., “ATLANTIS: AI-driven threat localization, analysis, and triage intelligence system,” arXiv:2509.14589, Sep. 2025, team Atlanta technical report. [Online]. Available: https://arxiv.org/abs/2509.14589
  192. [192]GitHub, “Now available for free on all public repositories: Copilot Autofix for CodeQL code scanning alerts,” GitHub Changelog, Sep. 2024. [Online]. Available: https://github.blog/changelog/2024-09-18-now-available-for-free-on-all-public-repositories-copilot-autofix-for-codeql-code-scanning-alerts
  193. [193]M. Hanley, “Found means fixed: Secure code more than three times faster with Copilot Autofix,” GitHub, Aug. 2024. [Online]. Available: https://github.blog/news-insights/product-news/secure-code-more-than-three-times-faster-with-copilot-autofix/
  194. [194]D. de Fitero-Dominguez, E. Garcia-Lopez, A. Garcia-Cabot, and J.-J. Martinez-Herraiz, “Enhanced automated code vulnerability repair using large language models,” Engineering Applications of Artificial Intelligence, vol. 18, p. 109291, Dec. 2024.
  195. [195]M. Fakih, R. Dharamji, H. Bouzidi, G. Quiros Araya, O. Ogundare, M. A. Siddika, and M. A. Al Faruque, “LLM4CVE: Enabling iterative automated vulnerability repair with large language models,” in Proceedings of the Euromicro Conference on Digital System Design (DSD), 2025, pp. 592–599.
  196. [196]S. Shi, R. Wei, M. Tufano, J. Cambronero, R. Cheng, F. Ivancic, and P. Rondon, “Towards a human-in-the-loop framework for reliable patch evaluation using an LLM-as-a-judge,” in Proceedings of the 1st International Workshop on AI for Software Quality Evaluation (AI-SQE), 2026, pp. 19–28.
  197. [197]Z. Wei, J. Zeng, M. Wen, Z. Yu, K. Cheng, Y. Zhu, J. Guo, S. Zhou, L. Yin, X. Su, and Z. Ma, “PatchEval: A new benchmark for evaluating LLMs on patching real-world vulnerabilities,” arXiv:2511.11019, 2025. [Online]. Available: https://arxiv.org/abs/2511.11019
  198. [198]C. Li and R. Jayaram, “The tech behind Semgrep Assistant’s triage and remediation guidance,” Semgrep blog, Aug. 2024. [Online]. Available: https://semgrep.dev/blog/2024/the-tech-behind-semgrep-assistant
  199. [199]OpenAI, “Introducing Aardvark: OpenAI’s agentic security researcher,” OpenAI, Oct. 2025. [Online]. Available: https://openai.com/index/introducing-aardvark/
  200. [200]S. Basu and M. Staron, “Security in a workflow: Exploring role-based agentic architectures for vulnerability handling,” in Proceedings of the Euromicro Conference on Software Engineering and Advanced Applications (SEAA), ser. Lecture Notes in Computer Science, 2026, pp. 427–444.
  201. [201]Y. Nong and H. Cai, “Neuro-symbolic proof-of-vulnerability generation with open-weight models,” arXiv:2608.04217, 2026. [Online]. Available: https://arxiv.org/abs/2608.04217

Paper details

vlnc:W_2jtpy38y
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 7

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents