betaFeedback

WorksComputer Science

Large Language Models for AI Infrastructure

Abstract

Large language models (LLMs) are increasingly used to design, optimize, and operate the infrastructure behind AI training and serving, yet the literature remains fragmented across hardware design, machine learning systems, electronic design automation, and high-performance computing. We present a comprehensive survey of LLM-assisted AI infrastructure across its lifecycle, from accelerator design and kernel engineering to compilation, deployment, operations, data management, and energy optimization. We organize the literature by the primary resource the LLM manages: artifact-centric methods for computational artifacts, operational-centric methods for deployment context and system state, and resource-centric methods for data and energy. These categories span the lifecycle and cover AI-specific concerns such as distributed-training configuration, inference serving, GPU-cluster scheduling, and data loading. Building on this taxonomy, we conduct a cross-study quantitative synthesis of reported results. Because studies adopt heterogeneous benchmarks, baselines, hardware, and metrics, we compare methods only where evaluation conditions permit and align results along common dimensions: functional correctness, execution performance, throughput, resource and energy consumption, operational outcomes, and optimization cost, keeping each result's native metric, baseline, and setting rather than collapsing them into a single score. This comparison reveals where LLM-driven methods yield consistent gains, how reported gains in workload orchestration depend on the evaluation environment, from simulators to real clusters and production, and where evidence is still lacking. Together, the taxonomy, lifecycle perspective, and quantitative evaluation provide an evidence-grounded reference for LLM-driven engineering of AI infrastructure.

Introduction

Large language models (LLMs) [112, 196] are increasingly studied as tools for designing, optimizing, and operating the infrastructure used to train and serve AI models, which spans accelerator hardware, graphics processing unit (GPU) kernels, compilers and parallel runtimes, deployment and operational systems, and the data and energy that AI workloads consume. On the artifact side, LLMs generate register-transfer-level (RTL) code from natural-language specifications [25, 92, 94, 100], synthesize and optimize GPU kernels [80, 118, 172], tune compiler optimizations [26, 33, 89], and translate programs across parallel programming models [20, 37, 64]. On the operational side, they deploy research artifacts and generate software build recipes [107, 169], detect and remediate Kubernetes misconfigurations [106], diagnose failures in production and AI clusters [144, 150, 184].

CCS Concepts: • General and reference → Surveys and overviews; Evaluation; • Computing methodologies → Natural language generation; • Computer systems organization → Parallel architectures; • Hardware → Electronic design automation; Power estimation and optimization; • Software and its engineering → Source code generation; System administration; • Information systems → Data management systems.

Additional Key Words and Phrases: large language models, AI infrastructure, RTL synthesis, GPU kernels, compiler optimization, distributed training, LLM serving, GPU cluster scheduling, deployment automation, root cause analysis, data pipelines, energy efficiency and, most recently, make the runtime decisions specific to AI workloads: parallelism and sharding plans for distributed training [42, 133], routing and cache policies for inference serving [54, 131], and GPU-cluster scheduling [117, 170]. On the resource side, they build data-preparation pipelines for model training [87], catalog data lakes and extract structured records [7, 11], generate energy-efficient code [39, 122], reduce the power of RTL designs and deep-learning accelerators [128, 168], and forecast data-center cooling load [66].

These results establish a broad research agenda rather than evidence that LLMs can autonomously replace infrastructure specialists, and the literature behind them remains fragmented across machine learning, electronic design automation (EDA), computer systems, and high-performance computing (HPC). Existing surveys examine individual layers in depth, but they organize work by model adaptation, design stage, programming task, or operational workflow, and none follows an AI model through the infrastructure that builds, runs, and sustains it. We therefore view AI infrastructure as a lifecycle: hardware and software artifacts are first built, training and serving workloads are then operated on them, and data and energy are supplied to those workloads throughout. Along this lifecycle, we characterize each LLM-driven method by the resource that the LLM’s principal action manages, the feedback available to evaluate that action, and the outcome used to assess it. This view yields a taxonomy whose three groups correspond to the three phases of the lifecycle, and it allows methods to be compared across infrastructure layers while preserving differences in their experimental evidence.

The proposed taxonomy comprises three groups and eight categories. Artifact-centric work (Group A) generates or modifies computational artifacts: Hardware Design (A1), Kernel Engineering (A2), and HPC Software (A3). Compilers, simulators, numerical tests, and execution harnesses provide feedback for these tasks. Their verdicts are conditional on the test coverage and execution environment; successful evaluation is not a general correctness guarantee. Operational-centric work (Group O) uses deployment context or system state for System Integration (O1), Production Operations (O2), and Workload Orchestration (O3). Its outputs include configurations, diagnoses, operational actions, and the parallelization, serving, and scheduling policies that determine how training and serving workloads use a cluster. Resource-centric work (Group R) addresses Data Management (R1) and Energy Optimization (R2), including the quality and resource costs of data processing and the energy consequences of generated code or policies. Figure 1 summarizes these concerns. A study may address more than one concern; its primary category follows its principal contribution, with secondary objectives retained in the discussion.

Figure 1. Scope of this survey across artifact-centric, operational-centric, and resource-centric AI infrastructure.

This infrastructure-wide perspective complements specialized surveys by connecting design, deployment, operation, and resource efficiency within one analytical framework. A lifecycle map (Section 2.1) places every category on seven stages from accelerator design to power and marks which stage hand-offs LLM methods have evaluated, and a table in Appendix A.2 places every AI-specific concern, including distributed training, inference serving, GPU-cluster scheduling, and data loading, on the same stages. Table 1 situates the resulting coverage against selected related surveys. Surveys of training and serving systems are marked as related to O3 because they review systems built for AI workloads rather than LLMs that engineer such systems.

SurveyVenueA1A2A3O1O2O3R1R2Primary scope
Pan et al. [120]arXiv ’25●○○○○○○○LLMs for electronic design automation
Yang et al. [183]CSUR ’26●○○○○○○○Verilog code generation
Yu et al. [190]arXiv ’26○●○○○○○○GPU kernel generation
Ljaljević et al. [98]FGCS ’26○◐●○○○○○LLMs in HPC programming
Jiang et al. [67]arXiv ’24◐◐◐○○○○○LLMs for code generation
Pahl et al. [119]Electron. ’26○○○●◐○○○AI for infrastructure-as-code
Zhang et al. [194]CSUR ’25○○○◐●○○○AIOps in the era of LLMs
Duan et al. [44]arXiv ’24○○◐○○◐○○Systems for distributed LLM training
Miao et al. [108]arXiv ’23○◐○○○◐○○Systems for efficient LLM serving
Hong et al. [58]TKDE ’25○○○○○○◐○LLM-based Text-to-SQL
Ali et al. [5]arXiv ’26○○○○○○○●Sustainable code generation
This survey—●●●●●●●●LLMs across the AI infrastructure lifecycle

Table 1. Principal scope of selected related surveys under the proposed taxonomy. ● primary topic; ◐ related coverage; ○ not a principal topic. Colors identify the three groups.

In summary, this survey makes three contributions:

  • A cross-lifecycle survey of LLMs for AI infrastructure. To our knowledge, it is the first survey that spans the lifecycle of training and serving AI models, organized as seven stages from accelerator design, kernel development, and compilation through deployment and serving and cluster operation to data supply and power management. It covers the method families, benchmarks, and AI-specific concerns of every stage, connecting work that has so far been studied one stage at a time into a single end-to-end view of the lifecycle.

  • A lifecycle-aligned taxonomy of LLM-driven methods. The taxonomy organizes the field into eight categories across three groups: artifact-centric methods generate hardware, kernels, and parallel software; operational-centric methods integrate, operate, and orchestrate infrastructure; and resource-centric methods manage data and energy. Grouping methods by what the LLM manages keeps those with shared feedback and evaluation together and aligns each group with a lifecycle phase.

  • A quantitative analysis of LLMs’ practical impact on AI infrastructure. It characterizes the effectiveness and efficiency gains reported across artifact-centric, operational-centric, and resource-centric settings, and it compares LLM-driven workload orchestration across studies, separating simulated, real-cluster, and production evidence.

Scope. This survey covers work in which an LLM generates an infrastructure artifact, proposes an infrastructure action, or interprets system evidence. General compiler, HPC, and data-infrastructure studies are included when their methods apply to infrastructure used by AI workloads, but their results are not presented as measurements on AI workloads unless the source evaluates that setting; systems that optimize LLM execution without an LLM acting as the engineering agent are treated as context. The review aims to cover every method family, benchmark, and AI-specific concern in each category rather than to enumerate every eligible paper, so its coverage of fast-moving subfields is broad but not exhaustive. Quantitative comparisons include only studies that report a recoverable metric, result, and evaluation setting.

Organization. Section 2 introduces the lifecycle of AI infrastructure and the common task formulation. Section 3 develops the taxonomy, places the AI-specific concerns within it, and reviews methods by family. Section 4 synthesizes reported evaluations. Section 5 discusses related surveys, and Section 6 presents conclusions and research priorities.

Background and Problem Formulation

The AI Infrastructure Lifecycle

AI infrastructure comprises the hardware, software, and operational services that train and serve AI models at scale, extending from accelerators and the software stack that programs them to the management of clusters and the supply of data and power. It differs from general-purpose computing infrastructure in the two workloads it must sustain. Distributed training partitions a model and its data across many accelerators and must remain efficient and fault tolerant over long-running jobs [44], whereas inference serving must meet latency and throughput objectives under fluctuating demand [108]. Both workloads share accelerator clusters, and their efficiency depends on tightly coupled decisions across hardware, software, operations, and resource supply rather than on any single component.

We therefore organize AI infrastructure as a lifecycle of seven stages that follows a model from the hardware that executes it to the resources it consumes (Figure 2). Accelerator design (S1) produces the circuits that execute AI operators, kernel development (S2) implements these operators for a target device, and compilation (S3) lowers model graphs into executable programs. Deployment and serving (S4) partitions a model across devices and configures its runtime policies, and operation (S5) schedules jobs on shared clusters and diagnoses and recovers from failures.

Figure 2. Lifecycle of AI infrastructure engineering. Seven stages form three phases that place the artifact-centric (A), operational-centric (O), and resource-centric (R) groups on the lifecycle; the chips below each stage list the categories whose primary contribution falls in it. Solid arrows mark hand-offs at which the output of an LLM-driven method in one stage has been evaluated in the next; dashed arrows mark hand-offs without such an end-to-end evaluation.

Data supply (S6) prepares and loads the data that workloads consume, and power management (S7) accounts for and reduces their energy demand. The stages form three phases, namely building artifacts (S1–S3), operating workloads (S4–S5), and supplying resources (S6–S7), which correspond to the artifact-centric, operational-centric, and resource-centric groups of the taxonomy in § 3; each category is placed on the stage of its primary contribution.

Consecutive stages are linked by hand-offs, in which the output of one stage becomes the input of the next. Two further links complete the lifecycle: serving kernels are frequently integrated into inference engines as libraries, which connects S2 directly to S4, and the energy measured in S7 informs the design objectives of the next accelerator generation, which closes the cycle from S7 to S1. In Figure 2, a hand-off is drawn solid when the output of an LLM-driven method in one stage has been evaluated in the next, and dashed when no such end-to-end evaluation exists. The two cross-stage edges are supported by FlashInfer-Bench [178] and AgentCompile [86], which evaluate generated kernels inside serving engines (S2 to S4), and by POET [128] and AUTOGATE [168], which optimize RTL and accelerator designs for power (S7 to S1). Current work concentrates on the build-artifacts phase. LLM-driven methods for the runtime decisions of S4 and S5 are emerging, whereas concerns such as checkpointing, straggler mitigation, quantization choice, and data loading remain largely unexplored (Appendix § 8.§ 2). Most methods also operate within a single stage, and the links from accelerator design to kernel development and from data supply to power management have yet to be examined.

A Common Task Formulation

We represent an infrastructure task by (I,A,E,F,J)(\mathcal{I},\mathcal{A},\mathcal{E},F,J), where I\mathcal{I} is the space of task inputs, such as specifications, existing code, telemetry, and resource constraints; A\mathcal{A} is the space of candidate actions, such as programs, configuration edits, and diagnoses; E\mathcal{E} is the environment in which an action is evaluated, including tools, hardware, and system state; FF is the feedback mechanism; and JJ is the task-specific cost. For an input I∈II \in\mathcal{I}, evaluating a candidate a∈Aa \in\mathcal{A} yields feedback f=F(E,a;I)f=F(\mathcal{E},a;I), such as diagnostics, test outcomes, or measured performance, and the task is the constrained optimization

min⁡a∈AJ(E,a;I)subject toFeasible⁡(E,a;I)=T,(1)\min_{a\in\mathcal{A}} J(\mathcal{E},a;I) \quad\text{subject to} \quad\operatorname{Feasible}(\mathcal{E},a;I)=\mathsf{T}, \tag*{(1)}

where the predicate Feasible indicates whether aa satisfies the stated correctness, security, and service requirements and T\mathsf{T} denotes true. Tests and measurements provide evidence for this predicate only to the extent of their coverage. More generally, the feedback FF observed by a method is a proxy for Feasible and JJ: it may rely on visible rather than held-out tests, or on a simulator, surrogate recorded trace in place of the deployment environment, so a candidate that scores well under FF may still violate the requirements or incur a higher cost when deployed. Benefits such as accuracy or throughput enter JJ through a change of sign or a corresponding loss, and multi-objective studies may retain separate costs instead of a scalar weighting [110].

A method evaluates a sequence of candidates a1,…,aka_1,\ldots,a_k within an attempt budget kk and observes fi=F(E,ai;I)f_i = F(\mathcal{E},a_i;I) for each candidate; later candidates may depend on earlier observations, and one-shot generation corresponds to k=1k = 1. The observations need not be independent or deterministic, since timing noise, concurrency, and changing system state can affect them. The mechanism by which the LLM proposes these candidates constitutes the method’s adaptation strategy, which ranges over prompting and agentic loops, retrieval augmentation, supervised fine-tuning, reinforcement learning, evolutionary or tree search, and multi-agent collaboration. The formulation thus provides a common vocabulary across tasks without assuming a common optimizer, and the comparison tables of Section 3 characterize each system by I\mathcal{I}, A\mathcal{A}, E\mathcal{E}, FF, JJ, and its adaptation strategy.

Landscape of LLM-Assisted AI Infrastructure Engineering

The formulation in Equation (1) identifies common components without equating their implementations. This section organizes LLM-driven methods by the primary managed resource, places the AI-specific concerns of Section 2.1 within that organization, and compares the inputs, action spaces, feedback mechanisms, and adaptation strategies of the systems in each category.

Taxonomy Overview

Classification rule. A study’s primary category is determined by its principal contribution and main evaluation objective. The classification is an analytical convention; individual systems may combine artifact generation, deployment, and resource optimization. Programming language, lifecycle stage, and model-adaptation strategy are retained as attributes rather than used as the top-level partition. Figure 3 shows the hierarchy with every system it covers, Appendix § 8.§ 1 provides the category definitions, and Table 2–Table 9 compare the systems of each category along the components of § 2.2.

Taxonomy of LLM-driven AI infrastructure

Figure 3. Our taxonomy organizes LLM-driven AI infrastructure by its primary managed resource into artifact-centric, operational-centric, and resource-centric groups and eight categories. Each category is expanded with all systems reviewed in it, ordered by method family as in Tables 2–9.

SystemYearI\mathcal{I}A\mathcal{A}E\mathcal{E}F\mathcal{F}J\mathcal{J}Adapt.
Benchmarks and evaluation protocols
RTLLM [100]2023SCodeRTL, EDA—Corr, PPAP
VerilogEval [92]2023SCodeRTL—CorrP, SFT
Revisiting VerilogEval [125]2024SCodeRTL—CorrP
Configuration Over Selection [140]2026SCodeRTL, EDA—CorrP
PostEDA-Bench [93]2026C, TCode, CfgEDACoSuccP, MA
RTL-OPT [101]2026CCodeRTL, EDA—PPAP
Domain-adapted and internally guided models
RTLCoder [94]2023SCodeRTL—CorrSFT
CASS-RTL [4]2026SCodeRTL—CorrInt
Data-Efficient Verilog FT [27]2026SCodeRTLSiCorrSFT, MA
Verification-guided generation and repair
RTLFixer [161]2023C, TCodeRTLCoCorrRAG, P
AutoVeriFix+ [151]2026SCodeRTLCo, SiCorrP, MA
CktFormalizer [179]2026SCodeRTL, EDASt, Co, SiCorrMA
Clover [102]2026C, TCodeRTLSt, SiCorrEvo, MA
CPPL [186]2026SCodeRTLSt, CoCorrP
EstRTL [180]2026SCodeRTLLJCorrMA
Veri-Sure [90]2026SCodeRTLSt, SiCorrMA
PPA optimization and search
LLM-VeriPPA [157]2025SCodeRTL, EDACo, SiCorr, PPAP
AutoPPA [79]2026CCodeRTL, EDACo, SiPPAEvo
CODMAS [24]2026CCodeRTL, EDACo, SiPPAMA
COEVO [127]2026SCodeRTL, EDASi, CoCorr, PPAEvo
Dr. RTL [49]2026CCodeEDACoPPAMA
EvolVE [60]2026SCodeRTL, EDASi, CoCorr, PPAEvo
Agentic design flows and accelerator design
ChipGPT [25]2023SCodeEDACoPPAP
A3D [115]2026CCodeEDACo, ExPerfMA, RAG
ACE-RTL [40]2026SCodeRTLSiCorrSFT, MA
Agentic Verilog Generation [191]2026SCodeRTLSi, CoCorrP
ChipCraftBrain [48]2026SCodeRTL, EDASi, CoCorrMA, RL, RAG
Design Conductor 2.0★ [154]2026SCodeRTL, EDASi, CoCorr, PPAMA
DRCY [45]2026CDiagProdSt, HuAccMA, RAG
EDA tools, analog, and circuit design
AnalogAgent [17]2026SCodeSPICESiCorrMA
AutoSizer [189]2026S, KCfgSPICESiSuccP
CircuitLM [55]2026SCodeOffSt, LJAccMA, RAG
RKHS [2]2026SCodeEDAExPerfRAG, P
Self-Evolved ABC [188]2026CCodeEDACo, ExPPAMA, Evo

Table 2. Hardware design (A1) systems under the task formulation of Eq. (1), grouped by method family. I\mathcal{I}: S specification, C existing code or artifact, T telemetry or traces, K constraints; A\mathcal{A}: Code program or design code, Cfg configuration, Diag diagnosis or answer; E\mathcal{E}: CPU or GPU execution on that device, RTL RTL simulator or formal tool, EDA synthesis or physical-design flow, SPICE circuit simulator, Prod production system, Off offline corpus; F\mathcal{F}: St static check, Co compilation or synthesis, Si simulation, Ex execution, LJ LLM judge, Hu human; J\mathcal{J}: Corr correctness, PPA power-performance-area, Perf runtime or throughput, Acc accuracy, Succ task success; Adapt.: P prompting, RAG retrieval, SFT fine-tuning, RL reinforcement learning, Evo evolutionary or tree search, MA multi-agent, Int internal-state intervention; — none. ★ evaluation on an AI model, workload, or cluster.

SystemYearIIA\mathcal{A}E\mathcal{E}F\mathcal{F}J\mathcal{J}Adapt.
Energy-aware code generation
Energy-Efficient Code Directions [122]2024CCodeCPUExEnergyP
Energy-Efficient Code Prompting [23]2024SCodeCPU—EnergyP
Energy-Efficient Code Study [9]2025SCodeCPU—EnergyP
LASSI-EE [39]2025CCodeGPUEx, LJEnergyP
Power-oriented hardware optimization
AUTOGATE★ [168]2026C, TCodeRTL, EDASi, CoPPAMA
Coevolutionary Circuit Approximation [158]2026SCodeEDASiPPA, AccEvo
POET [128]2026CCodeRTL, EDASi, CoPPAEvo
Facility forecasting
HyperLoad [66]2025TPredOff—AccSFT

Table 9. Energy optimization (R2) systems under the task formulation of Eq. (1), grouped by method family. Abbreviations follow Table 2 and the preceding tables; new ones: A\mathcal{A}: Pred forecast; J\mathcal{J}: Energy energy or power.

Groups and categories. Artifact-centric work (Group A) produces or evaluates computational artifacts. Hardware Design (A1) concerns RTL, high-level synthesis (HLS), circuits, and EDA decisions. Kernel Engineering (A2) concerns device-level kernels and accelerator transformations. HPC Software (A3) concerns compiler decisions, parallel programs, deep-learning frameworks, and distributed scientific software. Operational-centric work (Group O) depends on deployment context or system observations. System Integration (O1) connects artifacts to an execution environment, Production Operations (O2) interprets the state of deployed systems to diagnose and remediate it, and Workload Orchestration (O3) decides how training and serving workloads use shared accelerators, memory, and interconnect over time. O3 covers parallelization and sharding plans, request routing and batching, KV-cache and expert-placement policies, job scheduling and placement, and fault-tolerance policies; its outputs are policies or plans evaluated by workload-level performance under a trace or live load. Resource-centric work (Group R) emphasizes data processing and access in Data Management (R1), or energy, power, and thermal objectives in Energy Optimization (R2).

R1 includes data-quality tasks that support resource-aware processing, even when the study does not directly optimize bytes transferred.

Assignment of boundary cases. The main objective takes precedence over auxiliary measurements. RTL generation with power as one of several PPA metrics remains in A1, whereas code generation primarily evaluated by energy savings belongs to R2. Diagnosis of an I/O incident from traces belongs to O2, while data extraction or access planning belongs to R1. Compiler-generated accelerator code is assigned by the contribution evaluated: kernel construction in A2, or general compiler heuristics in A3. Parallel translation may target both host and accelerator code and is assigned to A3 when cross-paradigm translation is the central contribution. The three operational categories are separated by the kind of decision and its evaluation. A one-time configuration judged by whether the service deploys and runs belongs to O1; a diagnosis or explanation of observed behavior belongs to O2, including the diagnosis of training failures; and a plan or policy judged by performance under load, such as a parallelism configuration, a routing policy, or a scheduling heuristic, belongs to O3. Kernels and compiler passes written for serving engines remain in A2 and A3, while a cluster-level partitioning of a model is an O3 decision. Frameworks that report several infrastructure case studies are assigned by the case with the strongest infrastructure evidence, and their other cases are discussed as secondary contributions. These rules preserve a primary category without asserting that the underlying technical problems are disjoint.

Placement of AI-specific workloads. The lifecycle of § 2.1 is orthogonal to this resource-based partition, so each AI-specific concern is placed where its managed resource lies. Decisions about how training and serving workloads occupy a cluster, including parallelism and sharding, request routing, batching, KV-cache eviction, model placement, and GPU-cluster scheduling, are O3 policies; fused communication kernels and serving runtimes are artifacts in A2 and A3; the diagnosis of training failures interprets operational evidence in O2; and training-data preparation and loading belong to R1. Systems evaluated on an AI model, workload, or cluster are starred in Table 2–9, and Appendix A.2 maps every concern to its stage, category, and the LLM-driven systems that address it.

Reading the comparison tables. Table 2–9 instantiate the formulation of § 2.2 for every system. The input II distinguishes specifications, existing artifacts, telemetry, and constraints; the action space A\mathcal{A} records what the LLM emits; the environment E\mathcal{E} records where candidate actions are executed or checked; the feedback FF lists the signals available to the method during generation, which may differ from the signals used only to evaluate it; and the column JJ records the outcomes that the evaluation reports. In this column, Corr denotes a reported correctness rate, which measures the Feasible predicate of Equation (1) rather than a cost; systems that enforce correctness without reporting its rate list only their cost. The adaptation column records how the LLM proposes candidates: prompting, retrieval, fine-tuning, reinforcement learning, search, or multi-agent decomposition. § 4 compares reported results where task sets, baselines, budgets, and hardware permit.

Artifact-Centric Automation

Artifact-centric methods propose code or design transformations and evaluate them through compilers, simulators, numerical checks, or execution harnesses. The input II is a specification or existing artifact, and aa is a candidate or edit. Feedback FF may include acceptance diagnostics, test outcomes, and measured costs. Repeated evaluation is often feasible in a controlled environment, enabling repair and search. Its reliability depends on test coverage, toolchain versions, and measurement stability; a machine-generated verdict is not necessarily deterministic or complete. This group comprises hardware design, kernel engineering, and HPC software.

Hardware Design (A1)

Task and formulation. Hardware-design applications generate or analyze RTL, HLS specifications, formal properties, testbenches, and circuit descriptions. Their evaluation uses simulation, synthesis, timing analysis, formal verification, or circuit simulation [92, 100, 120]. Syntax acceptance and simulation success provide distinct checks. Neither alone establishes that a design satisfies its intended specification or is ready for fabrication. For validated candidates, a possible scalarization of power, performance, and area (PPA) is

min⁡a αP(a)+βfmax⁡(a)+γA(a)subject toFeasible⁡(E,a;I)=⊤,(2)\min_{a}\ \alpha P(a) + \frac{\beta}{f_{\max}(a)} + \gamma A(a) \quad\text{subject to}\quad\operatorname{Feasible}(\mathcal{E},a;I)=\top, \tag*{(2)}

where P(a)P(a) is design power, fmax⁡(a)f_{\max}(a) is the maximum clock frequency, and A(a)A(a) is the area. The nonnegative weights α\alpha, β\beta, and γ\gamma include the normalization required to combine these units. This expression illustrates a design trade-off; studies may instead use constraints or Pareto comparisons [110]. Correctness checking and PPA estimation can require separate tools and different evaluation costs.

Benchmarks and evaluation protocols. VerilogEval and RTLLM established specification-to-RTL evaluation with simulation-based functional checks and, in RTLLM, an additional design-quality goal [92, 100]. Subsequent benchmarks refine the protocol itself. Revisiting VerilogEval [125] revises the task format and adds failure classification, so that results on the original and revised suites are not interchangeable, and Configuration Over Selection [140] sweeps decoding configurations across open models to show that inference-time settings can outweigh the choice of model. Two newer suites move beyond functional generation: RTL-OPT [101] pairs suboptimal designs with human-optimized references to measure PPA improvement, and PostEDA-Bench [93] evaluates agents on post-synthesis design-rule repair and PPA convergence with machine-checkable EDA toolchains. Together, these benchmarks shift A1 evaluation from code that simulates correctly toward designs that close timing, power, and sign-off checks.

Domain-adapted and internally guided models. RTLCoder [94] fine-tunes a compact open model on synthesized instruction–code pairs. Data-efficient fine-tuning [27] generates testbenches with a multi-agent workflow so that only verified pairs are used for supervision. CASS-RTL [4] leaves the weights unchanged: it identifies attention heads whose activations separate correct from incorrect RTL and steers them at inference time. These methods change the proposal distribution rather than the evaluation loop.

Verification-guided generation and repair. A second family strengthens the feedback channel. RTLFixer [161] combines retrieval with ReAct-style [185] prompting over compiler diagnostics to repair syntax errors. AutoVeriFix+ [151] first derives a Python reference model and then uses concolic testing with cycle-accurate traces to expose corner cases and prune functionally redundant logic. Clover [102] casts RTL repair as a stochastic tree-of-thoughts search that dispatches subproblems either to LLM agents or to symbolic solvers. Veri-Sure [90] aligns agents through an explicit design contract and combines trace-driven temporal analysis with formal verification, while EstRTL [180] estimates functional correctness statically before deciding whether to regenerate or repair. Other systems change the representation that the LLM edits: CktFormalizer [179] embeds a typed hardware language in Lean so that compiler and proof-state feedback guide repair and equivalence proving, and CPPL [186] routes generation through a JSON circuit intermediate representation (IR) that is checked and lowered to CIRCT. These methods share a common direction: feedback that arrives earlier, is more structured, and is formal where possible.

PPA optimization and search. LLM-VeriPPA [157] adds a PPA-oriented optimization stage after functional repair. Search-based systems instead explore populations of candidates: EvolVE [60] compares evolution strategies and pairs Monte Carlo tree search (MCTS) [21] for functional correctness with idea-guided refinement for optimization; COEVO [127] retains partially correct but architecturally promising candidates in a Pareto search over correctness, area, delay, and power; and AutoPPA [79] induces reusable optimization rules by contrasting generated code pairs. CODMAS [24] uses two dialectic agents to plan and verify pipelining and clock-gating transformations, and Dr. RTL [49] performs closed-loop timing optimization with an industrial synthesis flow, combining critical-path analysis, parallel RTL rewriting, and a library of reusable optimization skills. The evaluation environments of this family range from manually degraded textbook circuits to industrial flows.

Agentic design flows and accelerator design. ChipGPT [25] first combined generation, output management, and candidate selection in a zero-code design flow. ACE-RTL [40] couples an RTL-specialized model with a frontier reasoning model through generator, reflector, and coordinator roles, and ChipCraftBrain [48] orchestrates six specialized agents with a learned policy and solves logic-minimization subproblems symbolically. A controlled study [191] on the CVDP [126] benchmark examines how prompting structure and tool design affect agentic Verilog generation and shows that unstructured agent loops can underperform carefully prompted single-pass generation. Agentic flows also extend beyond single modules. A3D [115] automates workload analysis, HLS refactoring, and micro-architecture exploration for scientific accelerators, and DRCY [45] reviews schematic connections against component datasheets within a production design-review workflow. Design Conductor 2.0 [154] provides the most direct AI-specific example: starting from the TurboQuant [192] paper, its multi-agent harness autonomously designed VerTQ, an LLM inference accelerator. AlphaEvolve’s Verilog simplification of a tensor processing unit (TPU) arithmetic circuit, discussed with its primary O3 contribution, is a second example of LLM-driven design for AI accelerators [117].

EDA tools, analog, and circuit design. Another line of work optimizes EDA tools rather than designs. Self-Evolved ABC [188] employs LLM agents to rewrite components of the ABC logic-synthesis codebase within a loop driven by correctness and quality of results, and RKHS [2] synthesizes HLS list-scheduling heuristics from compact templates with retrieval and iterative refinement. Analog design requires circuit-level feedback: AnalogAgent [17] distills SPICE feedback into a self-evolving memory shared by generation, optimization, and knowledge-curation agents; AutoSizer [189] embeds numerical transistor sizing in an LLM-driven meta-optimizer that reshapes the search space from simulation feedback; and CircuitLM [55] grounds schematic generation in a component knowledge base and checks the result with an electrical-rule checker and an LLM judge.

Limitations. The principal evaluation limitation is the gap between a finite test suite and the intended design contract. Underspecified prompts and weak testbenches can leave functional errors undetected, and benchmark revisions, decoding configuration, and candidate budgets change reported success rates. Evaluation on AI accelerators remains scarce in A1, and the transfer of these methods to the datapaths and memory systems of training and inference hardware is largely untested. The quantitative synthesis therefore retains benchmark identity, the meaning of syntax and functional rates, and search budgets (Section 4.2). Table 2 compares the A1 systems.

3.2.2 Kernel Engineering (A2). Task and formulation. Kernel methods generate or optimize CUDA, Triton, HIP, CuTe, SYCL, AscendC, and other device-level implementations, mostly of deep-learning operators. Evaluation combines numerical comparison against a reference with execution-time measurements. KernelBench defines fastp\mathrm{fast}_p as the fraction of tasks with a correct candidate whose speedup exceeds threshold pp [118]. Its extension to a budget of kk candidates can be written as

fastp@k=1∣D∣∑d∈D1[∃i≤k:Correct⁡d(ad,i)∧Tref,dTd(ad,i)>p].(3)\mathrm{fast}_p@k = \frac{1}{|\mathcal{D}|}\sum_{d\in\mathcal{D}} \mathbb{1}\left[\exists i\leq k:\operatorname{Correct}_d(a_{d,i})\land\frac{T_{\mathrm{ref},d}}{T_d(a_{d,i})}>p\right]. \tag*{(3)}

Here D\mathcal{D} is the evaluated task set, ∣D∣|\mathcal{D}| its size, and dd indexes a task. Candidate ad,ia_{d,i} is the iith proposal for task dd, Correct⁡d(ad,i)\operatorname{Correct}_{d}(a_{d,i}) is its numerical-test verdict, and Td(ad,i)T_{d}(a_{d,i}) and Tref,dT_{\mathrm{ref},d} are the candidate and reference runtimes. The indicator 1[⋅]\mathbb{1}[\cdot] equals one when its condition holds and zero otherwise. Thus pp denotes the speedup threshold and kk the candidate budget. For iterative refinement, candidates depend on earlier feedback rather than being independent samples. A correctness test establishes agreement only for the specified inputs and numerical tolerances; broader numerical auditing remains necessary [137]. A2 is among the most active categories, and many of its studies evaluate AI operators or serving engines.

Benchmark construction. KernelBench, TritonBench, and MultiKernelBench defined the first task suites for CUDA, Triton, and multi-platform kernels, respectively [80, 118, 172]. Later benchmarks address the failure modes that these suites revealed. KernelBenchX [165] organizes Triton tasks into categories to attribute correctness and efficiency to task structure rather than to method, CUDABench [200] evaluates text-to-CUDA generation with a roofline-based performance score, and CUDAHercules [84] compares generated code with end-to-end expert systems across three GPU generations. AgentKernelArena [187] evaluates complete agent workflows rather than single calls and tests generalization to input configurations not observed during optimization. CUDABeaver [83] isolates debugging and identifies a failure mode that correctness-only benchmarks reward, termed repair by degeneration, in which a repair method passes the tests by replacing an optimized kernel with a simpler and slower one. MobileKernelBench [201] extends evaluation to the CPU backend of a mobile inference framework with on-device verification. FlashInfer-Bench [178] connects kernel generation to deployment: it derives tasks from real serving traces and substitutes generated kernels into SGLang [197] and vLLM [76] through a single interface. Benchmark design has thus progressed along three axes: from single calls to agents, from correctness to performance against expert or production references, and from fixed inputs to generalization and deployment.

Prompting with execution feedback. The simplest feedback loop returns compilation, correctness, and timing results to the model, as in the refinement experiments of KernelBench [118]. GEAK [166] applies Reflexion-style feedback to Triton kernels on AMD GPUs. CuTeGen [135] targets the CuTe abstraction and withholds low-level profiling until the kernel’s high-level structure has stabilized. AutoKernel [62] profiles an entire model, ranks bottlenecks by their Amdahl impact, and iterates Triton or CUDA implementations behind a five-stage correctness harness. PIKE [114] provides a framework for comparing multi-agent organizations for PyTorch inference optimization and studies how exploration, exploitation, and error-fixing agents interact.

Profiling- and structure-guided refinement. A third family constrains the space of edits available to the model. PEAK [153] expresses optimizations as natural-language transformations applied along human-selected sequences. Ave [105] exposes hardware instructions through a tile-based domain-specific language (DSL) and uses data-flow invariants checked by a satisfiability-modulo-theories (SMT) solver as compile-time guardrails that return counterexamples instead of sparse test failures. A two-stage tuner [132] refactors SGLang [197] kernels into explicit parameter templates and then tunes the parameters by constrained search, which reduces the randomness of direct rewriting, and HTAM [195] organizes optimization experience in a hierarchical memory of global directions and local strategies. These systems trade generality for a more informative and safer feedback signal.

RL-trained kernel models. Reinforcement learning internalizes the feedback loop in the model. CUDA-L1 [85] trains with a contrastive RL algorithm that uses measured speedups as rewards, and CUDA Agent [34] scales agentic RL with a data-synthesis pipeline and a verified profiling environment. Dr. Kernel [95] studies the failure modes of this paradigm: its KernelGYM environment detects reward hacking, its turn-level estimator removes a bias in multi-turn group relative policy optimization (GRPO) [141], and profiling-based rewards discourage so-called lazy optimizations that remain correct but slow. DICE [14] adapts supervised fine-tuning and staged RL to diffusion language models, and the GPT-5-RL study [155] evaluates a fine-tuned frontier model and the MakoraGenerate agent built around it. Collectively, these studies indicate that the reward must account for performance and guard against reward hacking in addition to correctness.

Evolutionary and agentic search. Search-based systems allocate inference-time computation to populations of candidates. KernelEvolve [88] combines agentic search with retrieval to generate kernels for heterogeneous accelerators in production at Meta. KernelFoundry [174] uses MAP-Elites [113] quality-diversity search with co-evolved meta-prompts and template-based parameter tuning, and AdaExplore [43] converts recurring failures into a memory of validity rules before a tree search that alternates local refinement with structural regeneration. AlphaLab [57] applies a general autonomous research harness to CUDA optimization among other domains. CUCo [163] is the principal example in this family that targets distributed training and serving: it co-designs computation and communication in multi-GPU kernels, converting host-driven calls to the NVIDIA Collective Communications Library (NCCL) into device-initiated communication on a correctness-first fast path and then evolving overlap strategies on a slower search path.

Cross-platform transpilation and library-grounded generation. Accelerators with little public code require additional structure. AscendCraft [171] generates kernels in a lightweight DSL and lowers them to AscendC through constrained LLM passes, whereas AscendKernelGen [22] trains a model for Ascend kernels with chain-of-thought data, supervised fine-tuning, and execution-based RL. Library-grounded systems reuse optimizations already encoded in expert libraries. CuBridge [104] lifts expert attention kernels into an executable IR and lowers adapted variants with reference guidance, FACT [56] composes CUTLASS patterns matched to subgraphs of a traced PyTorch module, and AgentCompile [86] combines compiler-instantiated templates with LLM-generated decode kernels inside a serving runtime with paged KV cache and continuous batching. GEAK and MultiKernelBench extend evaluation beyond NVIDIA devices [166, 172].

Limitations. Functional validity and improvement over an optimized reference remain separate objectives. A high pass rate does not imply that generated kernels outperform TorchInductor [8] or vendor libraries, and several benchmarks now show that refinement can trade speed for correctness. Profiling feedback improves some model–task combinations and reduces scores in others, so the reported ablations do not support a uniformly beneficial effect. Portability requires more than compilation on another platform: numerical agreement and performance should be remeasured. Finally, most A2 results are measured on isolated operators; only FlashInfer-Bench [178], AgentCompile [86], and FACT [56] report gains inside serving engines or model blocks. Table 3 and § 4.2 distinguish these outcomes.

Table 3. Kernel engineering (A2) systems under the task formulation of Eq. (1), grouped by method family. Abbreviations follow Table 2 and the preceding tables; new ones: E\mathcal{E}: NPU NPU or TPU.

SystemYearI\mathcal{I}A\mathcal{A}E\mathcal{E}F\mathcal{F}J\mathcal{J}Adapt.
Benchmark construction
KernelBench★ [118]2025CCodeGPUCo, ExCorr, PerfP
MultiKernelBench★ [172]2025CCodeGPU, NPU—Corr, PerfP
TritonBench★ [80]2025CCodeGPU—Corr, PerfP
AgentKernelArena★ [187]2026CCodeGPUCo, ExCorr, PerfMA
CUDABeaver [83]2026C, TCodeGPUCo, ExCorr, PerfP
CUDABench [200]2026SCodeGPU—Corr, PerfP
CUDAHercules [84]2026S, CCodeGPUCo, ExCorr, PerfP
FlashInfer-Bench★ [178]2026C, TCodeGPUCo, ExCorr, PerfP, MA
KernelBenchX★ [165]2026CCodeGPUCo, ExCorr, PerfP
MobileKernelBench★ [201]2026CCodeCPUCo, ExCorr, PerfMA
Prompting with execution feedback
GEAK★ [166]2025S, CCodeGPUCo, ExCorr, PerfP
PIKE★ [114]2025CCodeGPUCo, ExPerfMA
AutoKernel★ [62]2026CCodeGPUExCorr, PerfP
CuTeGen★ [135]2026CCodeGPUCo, ExCorr, PerfP
Profiling- and structure-guided refinement
PEAK★ [153]2025CCodeGPUCo, Ex, HuPerfP
Ave★ [105]2026CCodeGPUSt, ExPerfMA
HTAM★ [195]2026CCodeGPUCo, ExCorr, PerfEvo
Semantic+Search GPU Tuner★ [132]2026CCode, CfgGPUExPerfP, Evo
RL-trained kernel models
CUDA-L1★ [85]2025CCodeGPUExPerfRL
CUDA Agent★ [34]2026CCodeGPUCo, ExCorr, PerfRL
DICE★ [14]2026CCodeGPUCo, ExCorr, PerfSFT, RL
Dr. Kernel★ [95]2026CCodeGPUCo, ExCorr, PerfRL
Evolutionary and agentic search
KernelEvolve★ [88]2025CCodeGPUCo, ExCorr, PerfEvo, MA
AdaExplore★ [43]2026CCodeGPUCo, ExCorr, PerfEvo
AlphaLab★ [57]2026SCodeGPUExPerfMA
CUCo★ [163]2026CCodeGPUCo, ExPerfEvo, MA
KernelFoundry★ [174]2026CCodeGPUExPerfEvo
Cross-platform transpilation and library-grounded generation
AgentCompile★ [86]2026CCodeGPUCo, ExPerfP
AscendCraft★ [171]2026CCodeNPUCo, ExCorr, PerfP
AscendKernelGen★ [22]2026CCodeNPUCo, ExCorrSFT, RL
CuBridge★ [104]2026CCodeGPUCo, ExCorr, PerfP
FACT★ [56]2026CCodeGPUCo, ExPerfMA, RAG

Table 3.

HPC Software (A3)

Task and formulation. HPC-software applications include compiler optimization, deep-learning framework engineering, parallel program translation, and the detection or repair of concurrency errors. GCC, LLVM, tensor compilers, parallel runtimes, and test harnesses provide complementary observations. In terms of Equation (1), Feasible typically requires that the generated program or transformation preserve the semantics of the input program, as checked by tests, and JJ is runtime, instruction count, or code size. For parallelization, Amdahl’s law [6] gives an idealized bound for a fixed workload,

S(n)=1s+1−sn,(4)S(n) = \frac{1}{s + \frac{1-s}{n}}, \tag*{(4)}

where S(n)S(n) is speedup relative to serial execution, s∈[0,1]s \in[0,1] is the serial fraction, and n≥1n \ge1 is the processor count. Communication, synchronization, and memory effects can reduce realized speedup. Improvements may arise from reducing either serial or parallel work, improving memory behavior, or changing communication; they are not restricted to reducing ss. Correctness assessment must account for concurrency and floating-point effects.

Compiler optimization and heuristic synthesis. LLM Compiler [33] is a model trained to predict compiler pass sequences that reduce instruction counts. AwareCompiler [89] conditions pass generation on program features and structured compiler knowledge, while Magellan [26] couples an LLM coding agent with evolutionary search and autotuning to synthesize LLVM inlining and register-allocation heuristics. CompilerGPT [129] returns compiler optimization reports to the

LLM to guide iterative source-code rewriting. MOSAIC [152] reduces the cost of such search by having heterogeneous LLMs share one MCTS tree, which records the model that produced each improvement and favors smaller models. These methods operate on different optimization targets: pass sequences, decision heuristics, transformation schedules, and source programs.

Deep-learning compilers, frameworks, and runtimes. The software stack of AI workloads is itself an engineering target. PassNet [97] argues that LLMs should author compiler passes rather than standalone kernels and provides a dataset, benchmark, and fine-tuning recipe for pass generation on long-tail fusion subgraphs that default TorchInductor [8] compilation handles poorly. A second line of work addresses testing. FuzzGPT [41] primes LLMs with historical bug-triggering programs to generate unusual inputs for fuzzing PyTorch and TensorFlow, and AlignGuard [82] applies LLM-based test mutation to detect silent correctness bugs in torch.compile. VibeTensor [181] represents the most autonomous setting: a PyTorch-style tensor library, autograd engine, and CUDA runtime produced by coding agents under high-level human guidance and validated mainly by builds and tests. Its authors report that locally correct subsystems can compose into globally inefficient software.

Parallel translation and directive generation. UniPar [20] evaluates LLM-based translation among serial, OpenMP, and CUDA programs on its ParaTrans benchmark; ACCeLLiuM [64] fine-tunes models to generate OpenACC directives for data-parallel loops; and ParEval-Repo [37] supplies repository-level translation tasks, for which build systems and cross-file dependencies remain the main obstacles. GEPA-based prompt evolution [65] iteratively optimizes the prompts used for OpenACC pragma generation on PolyBench [130] programs without post-training the model. Other work examines scaling rather than translation: LLM-generated task-parallel code for OpenMP tasking, C++ standard parallelism, and HPX is compared for correctness and scalability across frameworks [16], and agent-generated Julia programs are evaluated at large core counts, where deadlocks, oversubscription, and memory exhaustion emerge [15].

Performance-rewarded generation and cross-architecture evaluation. An online RL approach [111] executes generated matrix-multiplication code on a supercomputer and feeds the measured GFLOPS back as the reward, with a staged quality-diversity curriculum over optimization techniques. CodegenBench [81] evaluates BLAS routines and specialized kernels across x86, Sunway, and Kunpeng processors to measure how generation quality varies with the target architecture.

Scientific-software assistants, workflows, and defects. An MPI study [124] compares prompting and retrieval strategies for detecting and repairing errors in MPI programs. PETSc Agent Evaluation [193] combines deterministic checks and LLM-based assessment in a multi-evaluator pipeline to determine whether generated code uses a production HPC library as an expert would. AstraAI [116] grounds edits to an exascale framework in retrieved code and abstract-syntax-tree context, CelloAI [12] defines repeatable benchmarks for high-energy-physics and HPC coding assistance, and LARA [38] validates generated atomistic simulation workflows through dry runs before they consume cluster resources.

Limitations. No single aggregate of these outcomes measures HPC engineering capability. Textual agreement with a reference pragma is weaker evidence than executing a parallel program successfully, and passing available tests is weaker than a semantic-preservation proof. Performance results additionally depend on compiler flags, inputs, processor counts, and hardware. The deep-learning stack is the A3 area most directly related to AI infrastructure, but its evaluations are still dominated by bug counts and isolated subgraphs rather than end-to-end training or serving time. The synthesis retains these distinctions and reports source-specific costs when available Table 4 and Table 20.

SystemYearIIA\mathcal{A}E\mathcal{E}FFJJAdapt.
Compiler optimization and heuristic synthesis
LLM Compiler [33]2023CCfgCPU—SizeSFT
AwareCompiler [89]2025CCfgCPUCoSizeSFT, RAG
CompilerGPT [129]2025C, TCodeCPUCo, ExPerfP
Magellan [26]2026CCodeCPUCo, ExPerf, SizeEvo
MOSAIC★ [152]2026CCodeGPU, CPUExPerf, CostEvo
Deep-learning compilers, frameworks, and runtimes
FuzzGPT★ [41]2023CCodeCPU, GPUExBugSFT, P
PassNet★ [97]2026CCodeGPUCo, ExPerfSFT
AlignGuard★ [82]2026CCodeGPUExBugP
VibeTensor★ [181]2026SCodeGPUCo, Ex, HuCorrMA
Parallel translation and directive generation
ACCeLLiuM [64]2025CCodeOff—AccSFT
ParEval-Repo [37]2025CCodeGPUCo, ExCorrP, MA
UniPar [20]2025CCodeCPU, GPUCo, ExCorr, PerfSFT, P
Agentic Julia at Scale [15]2026SCodeHPCCo, ExCorr, PerfMA
GEPA OpenACC [65]2026CCodeGPUCoCorr, PerfEvo
Task-Based Parallel CodeGen [16]2026S, CCodeCPUExCorr, PerfP
Performance-rewarded generation and cross-architecture evaluation
CodegenBench [81]2026SCodeCPUCo, ExCorr, PerfP
HPC CodeGen RL [111]2026SCodeHPCExPerfRL
Scientific-software assistants, workflows, and defects
AstraAI [116]2026S, CCodeHPC—RAG
CelloAI Benchmarks [12]2026S, CCodeHPCExCorr, AccP
LARA [38]2026SPlanHPCExSuccMA, RAG
MPI Error Repair [124]2026CDiag, CodeOff—AccRAG, P
PETSc Agent Evaluation [193]2026SCodeHPCCo, Ex, LJCorrP

Table 4. HPC software (A3) systems under the task formulation of Eq. (1), grouped by method family. Abbreviations follow Table 2 and the preceding tables; new ones: A\mathcal{A}: Plan plan or schedule; E\mathcal{E}: HPC HPC system or GPU cluster; JJ: Size code size, Bug defects found, Cost resource cost.

StudyBenchmarkReported resultSource location
LLM Compiler [33]Held-out LLVM-IR functions3.01% fewer instructions than -Oztbl:III
Magellan [26]LLVM inlining heuristic5.23% size improvement; 8.79% in a production evaluationSection 3.1
AwareCompiler [89]Seven benchmark suites30.03% IR-count reduction; -Oz: 29.98%Table 1
UniPar [20]ParaTrans29.4% (CUDA) and 38.2% (OpenMP) validation rateRQ2
ACCeLLiuM [64]OpenACC pragma–loop pairs50.4% exact match; 87.3% directive-type agreementIntroduction; RQ2
MPI analysis [124]Annotated MPI programs44.39–77.84% detection accuracy; 125/164 repairsTable 2; Section 5.3
CompilerGPT [129]Five programsup to 6.5× speedupTable 3

Table 20. HPC-software outcomes and source locations. Measures retain their native units and evaluation populations; UniPar uses its fine-tuned Llama model and ACCeLLiuM its fine-tuned CodeLlama model.

Operational-Centric Control

Operational methods use target-environment descriptions or observations from running systems. Actions include configurations, diagnoses, interventions, and resource-allocation policies. Evaluation may use a live service, an isolated environment, recorded traces, a replay system, or a simulator. Reproducibility depends on the environment and task: operational evaluations can often be replayed, but actions that modify production systems can introduce downtime or other costs. Retrieval, structured observations, simulation, and constrained execution can help manage these conditions, but their effectiveness requires task-specific evidence. This group comprises system integration, production operations, and workload orchestration.

3.3.1 System Integration (O1). Task and formulation. System integration connects artifacts to execution environments through dependency installation, deployment configuration, and service interfaces. A syntactically valid manifest is insufficient if the resulting service cannot execute its intended workload. Over a task set D\mathcal{D}, deployment success is

DSR=1∣D∣∑d∈D1[Deploy⁡(d)=Success⁡],(5)\mathrm{DSR} = \frac{1}{\lvert\mathcal{D}\rvert} \sum_{d \in\mathcal{D}} \mathbb{1}[\operatorname{Deploy}(d) = \operatorname{Success}], \tag*{(5)}

with success defined by the benchmark’s verifier. This measure does not establish security, sustained availability, or performance under load.

Environment construction and artifact deployment. DeployBench [169] evaluates agents on deploying research artifacts in fresh virtual machines, verifying each deployment through a hidden pipeline that executes the paper’s designated experiment. Multi-Docker-Eval [51] measures automated environment construction for repositories across multiple programming languages, in terms of both executable states and efficiency. SWE-World [149] replaces containerized execution with a learned surrogate that predicts intermediate and final test outcomes, which lowers the cost of training and evaluating software-engineering agents but makes the surrogate’s fidelity part of the evaluation. For HPC software, SpackIt [107] generates Spack [52] build recipes by combining repository analysis, retrieved examples, and diagnostic feedback from installation attempts.

Kubernetes and network policy configuration. GenKubeSec [106] fine-tunes a locally deployable model to detect, localize, explain, and remediate Kubernetes misconfigurations. KubeGuard [31] derives least-privilege manifests from runtime logs through modular prompt chaining, and Uenо and Uchiumi [162] benchmark the synthesis of Kubernetes manifests from Compose specifications. An intent-based framework [148] translates natural-language privacy requirements into Kubernetes placement rules and software-defined-networking flow rules and validates them on a testbed.

Agent–infrastructure interfaces. A Model Context Protocol server exposes quantum and HPC resources as tools, so that an LLM agent can execute quantum workflows submitted in natural language on a hybrid platform [146]. Such interfaces define the action space available to integration agents rather than evaluating a particular integration policy.

Limitations. Integration evaluation benefits from documenting dependency state, runtime versions, permissions, time limits, and success criteria. Tests of deployment, security, and service quality address different requirements and should remain separate when they are not jointly measured. None of the O1 systems is evaluated on the deployment of a training or serving stack, although such stacks are among the most dependency-intensive software that infrastructure teams deploy. Table 5 summarizes these distinctions.

SystemYearIIA\mathcal{A}E\mathcal{E}FFJJAdapt.
Environment construction and artifact deployment
SpackIt [107]2025CCfgHPCCo, ExSuccRAG, P
Multi-Docker-Eval [51]2025CCfgVMExSucc, CostP, MA
DeployBench [169]2026S, CPlan, CfgVMEx, LiSuccP
SWE-World [149]2026CCodeSurLJSuccSFT, RL
Kubernetes and network policy configuration
GenKubeSec [106]2024CDiag, CfgOff—AccSFT
K8s Migration [162]2024CCfgOff—AccP
KubeGuard [31]2025C, TCfgOff—AccP
Privacy-Aware Cloud-Edge Orchestration [148]2026SCfgK8sEx, LiSuccP
Agent–infrastructure interfaces
Quantum-HPC MCP Server [146]2026SPlanHPCEx, LiSuccP

Table 5. System integration (O1) systems under the task formulation of Eq. (1), grouped by method family. Abbreviations follow Table 2 and the preceding tables; new ones: E\mathcal{E}: VM virtual machine or container, K8s Kubernetes or cloud, Sur learned surrogate; FF: Li live system.

Production Operations (O2)

Task and formulation. Production-operations methods interpret logs, metrics, traces, and incident records, or propose actions such as restarts and configuration changes. For a labeled incident set D\mathcal{D}, root-cause classification accuracy is

Acc⁡RCA=1∣D∣∑i∈D1[R^i=Ri∗].(6)\operatorname{Acc}_{\mathrm{RCA}} = \frac{1}{|\mathcal{D}|} \sum_{i \in\mathcal{D}} \mathbf{1}[\hat{R}_i = R_i^*]. \tag*{(6)}

Here ii indexes an incident, R^i\hat{R}_i is the predicted root-cause label, and Ri∗R_i^* is its reference label. This definition must be distinguished from language-model ranking scores, which assess relative report quality. Mean time to recovery is relevant when an intervention is executed and its operational effect is measured.

Log parsing and operational analytics. A domain-adapted, instruction-following framework [74] uses chain-of-thought reasoning to parse unstructured logs from leadership-class HPC systems at production scale. EPIC [75] answers operational analytics queries over multimodal data with a hierarchical multi-agent architecture in which a top-level model orchestrates retrieval, descriptive, and predictive agents.

Root-cause analysis and support. MetaKube [150] accumulates diagnostic experience for Kubernetes failures in an episodic pattern memory, routes problems between fast and analytical pathways, and adapts a locally deployable model through post-training. StateGraph [177] organizes system-state relationships into a graph that guides root-cause analysis in Kubernetes clusters, while retrieval-based approaches [160] build root-cause knowledge bases from support tickets through fine-tuning, retrieval, or a combination of both. For HPC I/O, IOAgent [47] diagnoses performance issues from Darshan traces with a module-based pre-processor, retrieval-augmented domain knowledge, and a tree-based merger, and compares its diagnoses with the rule-based Drishti tool [18] and the LLM-based ION system [46]. HPC-LLM [138] provides retrieval-augmented, domain-adapted assistance for common HPC workflows such as job scheduling and MPI execution.

Diagnosis for AI training and serving clusters. Failures in AI clusters are dominated by hardware and user faults and are diagnosed mainly from logs [70]; non-LLM systems such as L4 and Flare automate parts of this diagnosis for training jobs [32, 70]. ARGUS [199] provides always-on, fine-grained tracing and fail-slow detection for training workloads in production GPU clusters and describes the integration of LLM agents with its traces as ongoing work; it is included as the observability substrate for such agents rather than as evidence of LLM-driven diagnosis. TSGuard [184] provides the most direct evidence of LLM-driven diagnosis: it mines historical on-call records into knowledge bases and emulates expert diagnosis through structured reasoning and iterative verification on production AI workload incidents. A cluster diagnosis agent [144] combines retrieval, diagram-of-thought reasoning, and self-play with a benchmark for AI-cluster issues.

Autonomous management. KubeIntellect [10] exposes the full Kubernetes API to a supervisor and domain-specific agents that can synthesize new tools on demand. An LLM-orchestrated big-data-as-a-service framework [13] coordinates agents across ingestion, training, deployment, monitoring, and drift detection. These systems execute actions, which makes their safety controls part of the evaluated method.

Limitations. These approaches differ in whether they parse evidence, produce a report, or execute a remediation, and evaluation should preserve that distinction. An LLM judge can rank explanations without establishing the correctness of an intervention, and shared judge and model families introduce potential bias. Transfer from one cluster, trace collection, or support corpus to another remains an empirical question. For AI clusters in particular, diagnosis is now evaluated on production incidents, but mitigation of the diagnosed straggler or failure is not. Table 6 and § 4.3 separate these forms of evidence.

SystemYearIIA\mathcal{A}E\mathcal{E}F\mathcal{F}J\mathcal{J}Adapt.
Log parsing and operational analytics
EPIC [75]2025TDiagHPCExAcc, CostMA, SFT
HPC Log Parsing [74]2026TDiagOff—AccSFT
Root-cause analysis and support
StateGraph [177]2025TDiagProd—AccRAG
HPC-LLM [138]2026SDiagOff—AccSFT, RAG
IOAgent [47]2026TDiagOff—AccRAG, P
LLM-Augmented RCA [160]2026TDiagOff—AccSFT, RAG
MetaKube [150]2026TDiagOff—AccRAG, SFT
Diagnosis for AI training and serving clusters
Cluster Diagnosis Agent★ [144]2024TDiag, CfgHPCLiAccRAG, P
TSGuard★ [184]2025TDiagProdLiAccMA, RAG
Autonomous management
KubeIntellect [10]2025SPlan, CodeK8sLi, HuSuccMA
LLM-Orchestrated Big-Data Service [13]2026SPlanDBEx, HuSuccMA

Table 6. Production operations (O2) systems under the task formulation of Eq. (1), grouped by method family. Abbreviations follow Table 2 and the preceding tables; new ones: E\mathcal{E}: DB database or data lake.

3.3.3 Workload Orchestration (O3). Task and formulation. Workload orchestration decides how training and serving workloads occupy a cluster: how a model is partitioned across devices, which requests are batched and routed where, which cached state is evicted, where experts and models are placed, and which job runs on which machine. The action is a plan or a policy π\pi, often an executable program, and the environment E\mathcal{E} is a simulator, a projection model, a trace, or a live cluster. Two objectives recur. For serving, SLO-constrained goodput counts the requests that both a time-to-first-token (TTFT) and a time-per-output-token (TPOT) target over a horizon TT,

Goodput⁡(π)=1T∑r∈R1[TTFT⁡r(π)≤τTTFT∧TPOT⁡r(π)≤τTPOT],(7)\operatorname{Goodput}(\pi) = \frac{1}{T} \sum_{r \in\mathcal{R}} \mathbf{1}\left[\operatorname{TTFT}_{r}(\pi) \le\tau_{\mathrm{TTFT}} \land\operatorname{TPOT}_{r}(\pi) \le\tau_{\mathrm{TPOT}}\right], \tag*{(7)}

where R\mathcal{R} is the set of requests in the trace and τTTFT\tau_{\mathrm{TTFT}} and τTPOT\tau_{\mathrm{TPOT}} are the latency targets. For training and cluster scheduling, the mean job completion time (JCT) is JCT⁡(π)=∣J∣−1∑j∈J(Cj(π)−sj)\operatorname{JCT}(\pi) = |\mathcal{J}|^{-1} \sum_{j \in\mathcal{J}} (C_j(\pi) - s_j), where J\mathcal{J} is the job set and CjC_j and sjs_j are the completion and submission times of job jj. A training configuration is instead scored by throughput per device or model FLOPs utilization at a fixed global batch. In all cases the policy is evaluated on a workload distribution, so the fidelity of the simulator or projection and the representativeness of the trace limit the conclusions that a result supports.

Parallelism and sharding configuration. LLM agents have been applied to the configuration of distributed training. ASAP [42] combines coordinator, analyzer, and proposal agents with profiling data, roofline analysis, and a knowledge base of past optimizations to diagnose whether a TPU training job is compute-, memory-bandwidth-, or communication-bound and to propose a sharding mesh. The Primus Tuning Agent [133] searches tensor, pipeline, expert, context, and data parallelism together with micro-batch size, recomputation, and kernel backends, using a single-node projection engine as a fast scoring oracle and an analytic memory filter that rejects infeasible candidates before any GPU time is spent. OptiCo [28] coordinates reasoning agents through a shared message pool and a failure-analysis agent to tune Megatron-LM [147] parallelization strategies.

Serving and memory policies. A second family synthesizes the runtime policies of inference serving as programs. Glia [54] organizes LLM agents into a researcher–supervisor workflow that forms hypotheses, designs simulator experiments, and analyzes telemetry, and applies it to request routing, scheduling, and autoscaling in a GPU cluster that serves LLMs. Autopoiesis [68] performs policy synthesis online: an LLM-driven evolution loop rewrites the scheduling and rescheduling policies of a serving system as workload and cluster conditions shift, and is evaluated against DistServe [198], HexGen [69], and SpotServe [109] on homogeneous, heterogeneous, and elastic clusters. CacheCraft [131] evolves KV-cache eviction policies under a strict policy interface and a cascade evaluator, using long-context benchmarks such as RULER [59] as its fitness signal. The ADRS case studies [30] apply the open-source OpenEvolve [142] framework to systems problems with simulator-based verifiers, including expert-parallel load balancing (EPLB) for MoE inference, model-to-GPU placement, and spot-instance scheduling, and a follow-up study [29] extends the analysis to further case studies and evolution frameworks.

Cluster scheduling and placement. SchedMate [170] augments existing deep-learning schedulers with three LLM components that read source code, runtime logs, and historical jobs to estimate job durations, cancel poor packing decisions, and diagnose failures. AlphaEvolve [117] evolves the machine-priority heuristic of Google’s Borg scheduler in a fleet simulator built from historical workload snapshots and deploys the result fleet-wide; the same framework also evolved a matrix-multiplication tiling heuristic used to train Gemini and a FlashAttention [36] configuration at the compiler-IR level. An agentic scheduler [99] for heterogeneous AI tool workloads pairs an LLM with an algorithmic runtime monitor to assign tools to CPU or GPU execution under a memory budget. The HPC studies in this family are smaller in scale: Incisor [78] combines program analysis with LLM reasoning to select cloud instances for HPC jobs from submission artifacts alone, a comparative study [143] evaluates many LLMs on a heterogeneous workload-mapping problem described in natural language, and a ReAct-style [185] scheduler [63] balances multiple objectives with a scratchpad memory and a constraint-enforcement module, in comparison with first-come-first-served (FCFS) and shortest-job-first (SJF) ordering and an OR-Tools [123] solver.

Context: optimizers without an LLM agent. Several serving systems search configuration spaces that overlap with O3 without using an LLM as the optimizer. AIConfigurator searches serving configurations with performance models and a calibrated database [182]; SLO-Guard autotunes eight vLLM [76] knobs, including quantization, batching limits, and prefix caching, with annealing and tree-structured Parzen estimation [103]; and Scepsy searches GPU shares, tensor-parallel degrees, and replica counts for multi-LLM agentic workflows with a throughput–latency predictor [173]. They provide baselines and evaluators for future LLM-driven methods but are not counted as evidence of LLM-driven engineering. LLM agents have not yet been applied to checkpoint frequency and placement, straggler mitigation, or the joint choice of quantization and parallelism for a deployment.

Limitations. Evidence in O3 is recent and relies largely on simulation. Glia [54], ADRS [30], CacheCraft [131], and the AlphaEvolve [117] scheduling study score candidates in simulators or evaluation harnesses, and Primus [133] scores them with a projection engine whose approximation error must be accounted for; only some of them report real-cluster validation, and only AlphaEvolve reports production deployment. Evolved policies can overfit the evaluator, and several studies report reward hacking or plateaus that required changing the policy interface. The baselines also differ in strength, from least-loaded routing to published state-of-the-art systems and expert-tuned configurations. Search cost is reported inconsistently: in hours and dollars by ADRS, in simulations by Glia, and in minutes of single-node exploration by Primus. Table 7 and § 4.3 compare these results under their native conditions.

SystemYearIIA\mathcal{A}E\mathcal{E}FFJJAdapt.
Parallelism and sharding configuration
ASAP★ [42]2025TCfgNPU—PerfMA, RAG
OptiCo★ [28]2026K, TCfgGPUExPerfMA
Primus Tuning Agent★ [133]2026K, TCfgGPU, SimEx, SiPerfMA
Serving and memory policies
ADRS★ [30]2025CPolicySimSiPerf, CostEvo
Glia★ [54]2025S, TPolicySim, GPUSi, ExPerfMA
Let the Barbarians In★ [29]2025CPolicySimSiPerfEvo
Autopoiesis★ [68]2026TPolicySim, GPUSi, LiPerfEvo
CacheCraft★ [131]2026CPolicyGPUExAccEvo
Cluster scheduling and placement
AlphaEvolve★ [117]2025CPolicySim, ProdSi, LiPerfEvo
Multiobjective HPC Scheduling [63]2025S, K, TPlanSimSiPerfP
SchedMate★ [170]2025C, TPlanHPC, SimRe, LiPerfRAG, P
Workload Mapping and Scheduling [143]2025S, KPlanOff—PerfP
Agentic CPU-GPU Scheduling★ [99]2026TPlanGPU, CPULiPerfP
Incisor [78]2026CCfgK8sStPerf, CostP

Table 7. Workload orchestration (O3) systems under the task formulation of Eq. (1), grouped by method family. Abbreviations follow Table 2 and the preceding tables; new ones: A\mathcal{A}: Policy runtime policy; E\mathcal{E}: Sim system simulator; FF: Re trace replay.

Resource-Centric Optimization

Resource-oriented methods examine data-processing quality and cost, or energy and power requirements. Their actions may still be code or configuration edits, but the main evaluation concerns the data service or resource consumed. Measurement boundaries require care: token counts, device energy, facility energy, and carbon estimates represent different quantities. Quality and resource use should be reported jointly when the method trades one against the other. This group comprises data management and energy optimization.

Data Management (R1)

Task and formulation. Data-management applications synthesize pipelines, extraction functions, queries, and access plans. Output quality may be assessed using reference records, answer labels, or data constraints. Resource measures include bytes, tokens, tool calls, storage footprint, and execution time. The task trades quality against resource use,

max⁡a∈A Q(a)subject toC(a)≤B,(8)\max_{a \in\mathcal{A}}\ Q(a) \quad\text{subject to} \quad C(a) \le B, \tag*{(8)}

where Q(a)Q(a) is output quality against the reference, C(a)C(a) is one of these resource measures, and BB is a budget; studies that report only quality correspond to an unconstrained budget. Text-to-SQL work supplies related methods for natural-language data access [58, 145].

Pipeline, query, and contract generation. Pipeline generation translates transformation specifications into executable code [73, 87]. Among R1 systems, DataFlow is most directly related to AI infrastructure: it prepares training data for LLMs with nearly 200 reusable operators and an agent that turns natural-language specifications into pipelines, and evaluates the pipelines by downstream model quality [87]. StreamLink generates Spark and Hadoop queries with locally fine-tuned models and LLM-based syntax and security checks [50], and fine-tuned models generate data contracts that formalize schema and quality expectations between producers and consumers [19].

Discovery and catalogs. Data-discovery and catalog-oriented work supports identifying and describing relevant sources: LEDD builds hierarchical semantic catalogs and table search for data lakes [7], and LAKEGEN uses LLMs to generate domain-specific tabular corpora with annotated joinability for evaluating discovery methods [35].

Extraction and analysis agents. Evaporate compares direct extraction with synthesized extraction functions, reporting both extraction quality and LLM resource use [11]. SANA examines planning, retrieval, and analysis bottlenecks using controlled and idealized tool configurations [175]; its idealized conditions estimate diagnostic headroom rather than deployable performance. NotebookRAG retrieves and re-executes cells from existing analysis notebooks to ground automated exploratory analysis [139].

Data-pipeline benchmarks. KramaBench [77] contains curated data-lake-to-insight challenges that require the end-to-end orchestration of extraction, cleaning, integration, and analysis. DW-Bench [3] poses graph-topology questions over data-warehouse schemas that combine foreign-key and lineage edges and compares tool-augmented with static methods.

Limitations. Resource claims should specify the quantities being counted. Fewer retrieval calls need not reduce total model computation when other tool calls increase, and token reductions exclude some costs of generating, validating, and executing extraction code. Except for DataFlow [87], these studies evaluate data services that can support AI workflows rather than measuring training-data throughput directly; the data loaders and input pipelines that keep accelerators supplied during training have not yet been targeted by LLM-driven methods. Evaluation on a fixed corpus with explicit quality measures and end-to-end resource accounting would support stronger comparisons (Table 8 and § 4.4).

SystemYearIIA\mathcal{A}E\mathcal{E}F\mathcal{F}J\mathcal{J}Adapt.
Pipeline, query, and contract generation
Data-Pipeline Interface [73]2024SPlan———P
AI Data Contracts [19]2025CCfgDBStAccSFT
DataFlow★ [87]2025SCodeDBExAccMA
StreamLink [50]2025SCodeDBStCorrSFT
Discovery and catalogs
LAKEGEN [35]2025SDataDB—AccP
LEDD [7]2025SDataDB——P
Extraction and analysis agents
Evaporate [11]2023CCodeOffExAcc, CostP
NotebookRAG [139]2026SCodeDBEx, HuAccRAG, MA
SANA [175]2026SPlan, CodeDBExAccMA
Data-pipeline benchmarks
KramaBench [77]2025SPlan, CodeDBExAccP, MA
DW-Bench [3]2026SDiagDBExAccP

Table 8. Data management (R1) systems under the task formulation of Eq. (1), grouped by method family. Abbreviations follow Table 2 and the preceding tables; new ones: A\mathcal{A}: Data data or catalog.

Energy Optimization (R2)

Task and formulation. Energy-oriented applications generate code, estimate facility demand, or propose power-management actions. Relevant outcomes include joules, average power, runtime, and forecast error. An energy-delay product [53] can summarize one trade-off,

EDP=ET=PavgT2,(9)\mathrm{EDP} = ET = P_{\mathrm{avg}}T^2, \tag*{(9)}

where EE is measured energy, TT is elapsed execution time, and Pavg=E/TP_{\mathrm{avg}} = E/T is average power over the same measurement interval. This optional summary is distinct from expected energy reduction and does not replace application-specific performance constraints.

Energy-aware code generation. Prompting and measurement studies examine the energy consumption of generated code on multiple platforms [9, 23, 122], including comparisons with code written by developers and green-software experts [9]; their conclusions depend on device, workload, and measurement harness. LASSI-EE [39] refactors parallel scientific code through a multi-stage loop that combines runtime power profiling, energy-aware prompting, self-correction, and an LLM judge.

Power-oriented hardware optimization. Design-oriented systems connect energy objectives to artifact optimization. POET [128] evolves RTL with power-first Pareto ranking and uses differential testing against the original design as a functional oracle. AUTOGATE [168] distills long toggle traces into clustered representations that guide LLM-based clock-gating rewrites across hierarchical designs, including the NVDLA deep-learning accelerator. A co-evolutionary method [158] evolves 8-bit approximate multipliers together with the prompts that steer the LLM. Their category follows the main objective rather than the language of the generated artifact.

Facility forecasting. HyperLoad uses an LLM-based architecture to forecast data-center cooling load [66]; lower prediction error does not itself demonstrate reduced facility energy.

Limitations. A rigorous energy evaluation reports the device, meter, workload, repetition scheme, measurement boundary, and reference implementation. Search and model inference costs should be included when assessing net savings, and carbon estimates additionally depend on the carbon-intensity trace and the time of execution. LLMs have not yet been applied to power capping or energy-aware scheduling of training and serving jobs, even though these workloads dominate the power budget of AI clusters. The available studies motivate multi-objective evaluation while leaving substantial uncertainty about transfer across platforms and scales (Table 9).

Benchmarks and Quantitative Synthesis

Benchmarks, Metrics, and Evidence Extraction

This section synthesizes the evaluations reported by the cited studies; it does not re-execute their systems. Numerical values, metric definitions, task sets, and experimental conditions were checked against the primary publications. Values are transcribed from tables, text, or printed figure labels, and derived quantities are computed from the reported values or underlying counts and identified as such; no value is read from an unlabeled graphical coordinate. Appendix § 8.§ 6 lists the source locations, and Appendix § 8.§ 4 describes the evaluation protocols.

Hardware studies commonly use VerilogEval [92] and RTLLM [100], whereas kernel studies use KernelBench [118], TritonBench [80], and MultiKernelBench [172]. HPC-software evaluations range from compiler test collections to parallel translation and defect-analysis corpora. DeployBench [169] and TraceBench [47] evaluate deployment and I/O diagnosis, respectively. Workload-orchestration studies use serving simulators such as vidur [1], projection engines calibrated by single-node benchmarks, production cluster traces, and real training and serving clusters. Data and energy studies use task-specific corpora and measurement harnesses. A shared benchmark name does not imply identical revisions, task subsets, or execution conditions.

Each result retains its metric, baseline, model, hardware, and budget where these are reported. Within-study comparisons provide the more controlled contrast, although simultaneous changes in several components still limit causal attribution. Cross-study comparisons characterize reported outcomes under their respective conditions; they do not estimate the effect of transferring a method to another study’s environment. The interpretation rules and the category-specific protocol differences are given in Appendix § 8.§ 4.

Artifact-Centric Evaluation (A1–A3)

Hardware design (A1). The A1 results distinguish two requirements of RTL automation: generating plausible candidates and validating their intended behavior. In Figure 4, self-planning [100] and VeriPPA [157] improve functional outcomes within their respective studies, while RTLCoder’s high syntax rate coexists with substantially lower functional success [94]. Language specialization and syntactic validity therefore provide an incomplete basis for selecting a hardware-generation method, and functional feedback should remain an explicit optimization target rather than a final compilation check [94, 100, 157]. Syntax and functionality use different denominators in these sources (Appendix § 8.§ 4), so their plotted separation is not a common failure probability.

RTL-generation performance across syntax, functionality, evaluation splits, and search budgets

Figure 4. RTL-generation performance across syntax, functionality, evaluation splits, and search budgets. Results follow the protocols of RTLLM, VerilogEval, RTLCoder, VeriPPA, and EvolVE [60, 92, 94, 100, 157]. Values are success rates in percent; higher is better. Arrows denote within-study changes, and dashed lines denote fixed inter-metric gaps. Panel (c) uses a logarithmic budget axis. Protocol differences are detailed in Appendix § 8.§ 4.

Table 10 further shows that domain adaptation improves both the Machine and Human VerilogEval [92] results but does not close the gap between the two specification sources. Models should therefore be evaluated on human-authored requirements in addition to automatically generated specifications, since effectiveness on one form of task description does not establish effectiveness on the other.

SystemParamsVerilogEval-Machine k=1k=1VerilogEval-Machine k=5k=5VerilogEval-Machine k=10k=10VerilogEval-Human k=1k=1VerilogEval-Human k=5k=5VerilogEval-Human k=10k=10M−HRTLLM v1.1 SyntaxRTLLM v1.1 Func.
Closed-source general models
GPT-3.5★—46.769.174.126.745.851.720.089.737.9
GPT-4★—60.070.673.543.555.858.916.5100.065.5
Models adapted to hardware description languages
ChipNeMo★ [91]13B43.4——22.4——21.0——
VerilogEval-SFT★ [92]16B46.267.373.728.845.952.317.4——
BetterV★ [121]7B64.275.479.140.950.053.323.3——
Thakur et al. [156]16B44.052.659.230.343.949.613.786.224.1
General open code models
CodeGen216B5.09.013.90.94.17.34.172.46.9
StarCoder15B46.854.559.618.126.130.428.793.127.6
Mistral-7B-v0.17B36.948.857.44.512.618.632.472.420.7
DeepSeek-Coder-6.7b6.7B54.163.867.530.242.246.223.989.634.5
Open code models after RTL-specific fine-tuning
RTLCoder-Mistral7B62.572.276.636.745.549.225.896.648.3
RTLCoder-DeepSeek6.7B61.276.581.841.650.153.419.693.148.3
Δ fine-tuning (DeepSeek-Coder → RTLCoder)+7.1+12.7+14.3+11.4+7.9+7.2−4.3+3.5+13.8

Table 10. RTL-generation systems under the RTLCoder evaluation protocol [94]. Values are transcribed from Table II of that study; for starred systems, the VerilogEval values are quoted from their original publications. Pass@kk, syntax success, and functional success are percentages; higher is better. M−H is the Machine-minus-Human pass@1 gap, and the Δ row gives the change in percentage points after RTL-specific fine-tuning. Bold marks column maxima, excluding M−H; a dash denotes an unreported value.

Method selection also depends on the verification budget. BetterV [121] achieves the highest Machine pass@1 in Table 10, whereas RTLCoder-DeepSeek [94] achieves the highest pass@10. This reversal separates first-attempt success from success obtained through additional candidates, so a single attempt budget cannot characterize both use cases. EvolVE extends the design space with structured search and reports higher success on the v2 revisions of both benchmarks [60]; the revision change, however, prevents attributing the cross-study difference to search alone. The evidence therefore favors evaluating the model, search strategy, and validator jointly under the intended engineering budget.

Kernel engineering (A2). The kernel results indicate that the value of feedback depends on how the model interacts with the evaluation environment. In Figure 5(a), the same refinement budget yields substantially different gains across models and task levels. Moreover, the fixed-budget ablation in Table 11 shows that adding profiling information improves some configurations and degrades others [118], so additional context is not an unconditional improvement. Feedback selection and presentation should be evaluated jointly with the model; adaptive selection between execution diagnostics and profiling is a research direction suggested by these results rather than an established explanation of the differences.

Three-panel kernel-generation performance figure. Panel (a): fast₁ (%) versus attempts, comparing DeepSeek-V3 and DeepSeek-R1 before and after feedback refinement; labels include V3·L1 6→19 (+13), V3·L2 4→6 (+2), V3·L3 8→14 (+6), R1·L1 12→43 (+31), R1·L2 36→72 (+36), and R1·L3 2→18 (+16). Panel (b): functional pass (%) versus tasks faster than reference (%), comparing GPT-5-RL and MakoraGenerate. Panel (c): reported speedup (×, log), showing GPT-5-RL vs TorchInductor 0.81×, MakoraGenerate vs TorchInductor 2.12×, CUDA-L1 L1 vs PyTorch eager 2.78×, CUDA-L1 L3 vs PyTorch eager 2.96×, CUDA-L1 L2 vs PyTorch eager 3.55×, and PEAK vs unoptimised input kernel 19.52×.

Figure 5. Kernel-generation performance under feedback refinement and study-specific execution references. Panel (a) uses KernelBench on L40S [118]; panel (b) uses the TorchInductor [8] protocol [155]; panel (c) includes different baselines and aggregation rules [85, 153, 155]. fast₁ requires both correctness and faster execution. Both coordinates in (b) are task percentages; speedups in (c) are factors relative to the named baseline. Higher is better, and a speedup below 1× denotes a slowdown.

Table 11. fast1fast_1 of test-time methods on KernelBench [118]. Levels 1, 2, and 3 contain 100, 100, and 50 tasks; all runs use an NVIDIA L40S GPU and the PyTorch eager reference. G, E, and P return the previous generation, the execution result, and the timing profile to the model. fast1fast_1 is the percentage of tasks that are both correct and faster than the reference; higher is better. The Δ\Delta rows give percentage-point changes. Bold marks column maxima; a dash denotes that repeated sampling is unavailable for the fixed-temperature DeepSeek-R1 endpoint.

Test-time methodCallsLevel 1Level 2Level 3
Llama-70BV3R1Llama-70BV3R1Llama-70BV3R1
Single attempt136120436082
Repeated sampling10511—314—114—
Iterative refinement, G10991807440144
Iterative refinement, G+E1051341556282212
Iterative refinement, G+E+P1071943467221418
Δ\Delta best − single attempt+6+13+31+5+10+36+8+14+16
Δ\Delta profiler (G+E → G+E+P)+2+6+2−1+1+10−6−8+6

Table 11.

The results further indicate that reliable generation and competitive optimization require separate evaluation targets. GPT-5-RL exhibits a large gap between functional success and performance success, whereas MakoraGenerate improves both and narrows the gap [155]. Evaluation should therefore assess whether an agent uses its search budget to discover faster valid implementations, not only to repair invalid ones. KernelEvolve reports functional success on all 250 KernelBench tasks but no faster-than-reference result [88], so it cannot be placed on the same performance frontier. Table 12 preserves this distinction.

Table 12. Correctness and speed of GPU-kernel generation systems. The GPT-5-RL study evaluates 264 KernelBench tasks against TorchInductor [8, 155]. The CUDA-L1 study evaluates the 250 KernelBench tasks on an A100 GPU against PyTorch eager and counts a task as faster only above 1.01× [85]. KernelEvolve reports functional success on the 250 KernelBench tasks without a speed reference or device [88], and PEAK reports speedups over input MatMul kernels on three devices [153]. Functional and Faster are task percentages, Gap is their difference in percentage points, and Speedup is a geometric (g), arithmetic (a), or device–precision (d) mean factor. Bold marks the best value within each block; n/r means not reported and a dash means not applicable.

SystemApproachFunctionalFasterGapSpeedup
GPT-5-RL study [155]
MakoraGenerateagent97.472.924.52.12×g2.12\times^{g}
GPT-5-RLsingle attempt77.021.855.20.81×g0.81\times^{g}
Claude Opus 4.5single attempt59.813.246.60.55×g0.55\times^{g}
Claude 4.5 Sonnetsingle attempt47.212.834.40.69×g0.69\times^{g}
GPT-5single attempt43.714.828.90.73×g0.73\times^{g}
Gemini 2.5 Prosingle attempt37.78.729.00.61×g0.61\times^{g}
Grok 4single attempt24.410.513.90.51×g0.51\times^{g}
Grok Code Fast 1single attempt15.83.812.00.39×g0.39\times^{g}
CUDA-L1 study [85]
Llama-3.1-405Bprompting27.22.025.20.23×a0.23\times^{a}
DeepSeek-V3prompting39.63.636.00.34×a0.34\times^{a}
OpenAI-o1prompting56.45.650.80.73×a0.73\times^{a}
DeepSeek-R1prompting71.67.264.40.88×a0.88\times^{a}
CUDA-L1RL-trained99.690.49.23.12×a3.12\times^{a}
Other studies
KernelEvolve [88]agentic search100.0n/r—n/r
PEAK [153]guided transformationsn/rn/r—19.52×d19.52\times^{d}

Table 12.

Finally, a reported speedup reflects the optimization headroom of its reference. PEAK’s speedup over unoptimized input MatMul kernels and CUDA-L1’s speedup over the KernelBench reference implementations address different deployment decisions from comparisons against TorchInductor [8, 85, 153]. Figure 5(c) therefore characterizes optimization headroom rather than relative method quality: existing implementations may leave substantial removable overhead, whereas surpassing an optimizing compiler is a different performance target. For practical adoption, the relevant reference is the implementation that would otherwise execute on the target system, and the largest reported factor alone is insufficient to select a method.

HPC software (A3). In the HPC studies, the strength of the reference is central to interpreting progress. AwareCompiler’s average IR-count reduction is substantial relative to the input programs, but it differs from the corresponding -O2 average by only 0.05 percentage points [89]. Figure 6(a) therefore indicates the remaining optimization opportunities rather than a ranking by headline reduction, and workload-level results are needed to determine where a method adds value beyond established compiler heuristics.

Source-reported outcomes in HPC-software studies

Figure 6. Source-reported outcomes in HPC-software studies. Panel (a) shows instruction-count or binary-size reduction under each study’s reference. Panel (b) shows validation, textual-agreement, defect-detection, and repair rates, with the metric named per row. Values are percentages; higher is better within the stated metric. The rows retain native definitions and do not constitute a common correctness benchmark.

The studies also differ in how optimization cost can be amortized. LLM Compiler predicts pass sequences for individual inputs; Magellan searches for reusable compiler heuristics; and

CompilerGPT rewrites selected programs using compiler reports [26, 33, 129]. These outputs have different scopes of reuse: a costly heuristic search can benefit many future compilations, whereas a source rewrite must recover its cost through subsequent executions of a single workload. Studies should therefore report both the optimization cost and the workload population over which the resulting artifact is reused; the cost measurements available in Table 20 do not yet support a common break-even estimate.

Figure 6(b) shows a separate gap in validation depth. ACCeLLiuM’s directive agreement, UniPar’s executable validation, and MPI repair success measure different stages of establishing a usable parallel program [20, 64, 124]. High agreement with a reference directive cannot substitute for execution under concurrency, and successful repair on a defect corpus does not establish performance portability. An evaluation that connects artifact-level agreement, execution correctness, and performance at the intended parallel scale would make improvements in code generation directly interpretable as improvements in HPC engineering.

Operational-Centric Evaluation (O1–O3)

System integration (O1). DeployBench [169] exposes the gap between producing installation actions and establishing a working research artifact. With a common scaffold and hidden verifier, the strongest evaluated agent completes 51.0% of the tasks, and the source’s failure analysis attributes more than half of the failures to agents that stop before the deployment has succeeded [169]. Completion judgment is therefore an engineering target in its own right: an agent requires evidence that the intended execution path works, not only a plausible sequence of setup steps. Figure 8 accordingly motivates execution-grounded stopping criteria and explicit verification of the deployed artifact.

Figure 8. Deployment success on DeployBench [169]. Pass rate is the percentage of tasks completed successfully under the hidden verifier; higher is better.

GenKubeSec addresses a narrower, complementary validation layer, the identification of security misconfigurations [106]. Its high detection scores do not establish the successful deployment or safe remediation of an entire service. Deployment readiness, policy compliance, and service quality thus require distinct checks that should be connected within one integration workflow; Appendix § 8.§ 4 describes the corresponding endpoints.

Production operations (O2). TraceBench [47] shows that the dimensions of a useful diagnosis need not improve together. Under the same IOAgent workflow, the two backing models obtain nearly identical utility scores, whereas their accuracy and interpretability differ substantially [47]. Table 13 and Figure 9 therefore support retaining multiple diagnostic criteria, since a single average would obscure which aspect of the report changes with the model. Model selection for operations should accordingly evaluate the capability required by the intended user, such as localizing a cause or providing an actionable explanation, rather than a single aggregate diagnosis score.

SystemBackingAccuracyUtilityInterp.Mean
IOAgentGPT-4o0.6410.6090.6450.632
IOAgentLlama-3.1-70B0.5130.6070.5300.550
Drishti [18]rule-based0.4590.4360.4470.447
ION [46]GPT-4o0.3800.3850.3850.383
Δ\Delta model swapGPT-4o − Llama+0.128+0.002+0.115+0.082
Δ\Delta vs. DrishtiIOAgent-GPT-4o+0.182+0.173+0.198+0.185
Δ\Delta vs. DrishtiIOAgent-Llama+0.054+0.171+0.083+0.103

Table 13. TraceBench normalized ranking scores and paired differences. A fixed GPT-4o ranker evaluates every system under the same protocol [47]. Accuracy, utility, and interpretability lie in [0,1][0, 1], with higher values indicating a more favorable judge-assigned rank. Mean is their arithmetic average. The Δ\Delta rows subtract the named comparator; positive differences favor the first term.

Figure 9. Normalized diagnosis rankings on TraceBench [47]. The GPT-4o judge assesses accuracy, utility, and interpretability. Axes display [0, 0.8] of the underlying [0, 1] range; higher is better.

Both IOAgent [47] variants also outperform the rule-based Drishti [18] and the LLM-based ION [46] on all reported criteria. The use of an LLM alone does not account for this ordering: the compared systems also differ in how they structure and process diagnostic evidence, and the experiment motivates a closer study of these workflow components without isolating their individual effects. The scores remain judge-assigned rankings, and the next validation step is to relate report quality to correct interventions and observed recovery. HPC-LLM’s answer-similarity evaluation represents a related support capability, but does not supply that operational endpoint [138].

Workload orchestration (O3). The O3 studies are the first in this synthesis to measure LLM-derived decisions on training and serving workloads at cluster scale, and Figure 7(a) shows that the size of a reported gain depends strongly on where it is measured. The largest factors are obtained in simulators and offline harnesses, for example a rebalancing algorithm for mixture-of-experts inference whose runtime falls from 19.6 to 3.7 ms and a 2.2×2.2 \times lower mean request completion time for Glia’s router [30, 54]. Most real-cluster results lie between 1.02× and 1.79×, from OptiCo’s MFU relative to the best expert configuration to Autopai’s trace time relative to SpotServe [28, 68, 109]. The exceptions are Glia’s slowdown on a four-GPU vLLM [76] testbed, which falls by more than 4.5×, and Incisor’s runtime reduction for HPC jobs [78]. In production, AlphaEvolve’s Borg heuristic recovers 0.7% of fleet-wide compute [117]. These differences cannot be attributed to simulator optimism alone. SchedMate’s simulator deviates from its physical-cluster measurements by less than 3.7%; on the physical cluster, SchedMate reduces mean job completion time by a factor of 1.24, and in simulation its factors of 1.23–1.91 span four production traces [170]. Glia likewise reports a larger reduction in slowdown on its testbed than in completion time in its simulator, although the two metrics differ. The environments differ instead in workload, scale, and the strength of the reference that the method replaces. A small factor in a production fleet can therefore represent a larger absolute saving than a large factor on a four-GPU testbed, and simulated and deployed results should be reported separately rather than ranked on one axis.

Figure 7. Workload-orchestration (O3) outcomes across studies. Panel (a) expresses each reported result as a factor relative to the reference named in the row, computed from the source’s values (reference/method for time and cost, method/reference for throughput, MFU, and score, and 1/(1−r)1/(1-r) for a reported reduction rr); bars show reported ranges and arrows lower bounds. Marker shape, explained in the legend of (b), gives the evaluation environment and color the method family. Panel (b) plots the reported wall-clock time to the solution against the same factor; diamonds mark the human-expert effort that the Glia [54] and OptiCo [28] studies report for a comparable result. Rows retain their own metrics, workloads, and baselines (Table 19 in the appendix) and are not a ranking of methods.

The strength of the reference determines what a factor means, and the within-study contrasts in Table 19 make this explicit. Glia’s router reduces mean completion time by 2.2× relative to least-loaded routing and by 1.6× relative to the policy found by OpenEvolve [142] with the same simulator, and it matches a design that took a human expert two weeks [54]. OptiCo achieves on average 101.84% of the MFU of the best expert-tuned Megatron-LM [147] configuration, a relative improvement of 1.84% [28]. ASAP is evaluated by agreement with the expert sharding mesh, which it matches in all three cases, so the speedups it reports belong to the expert configurations [42]. Primus [133] provides the most explicit decomposition of an orchestration gain. Its 27% measured improvement over the published reference includes a change from BF16 to FP8 precision. In its ablation, the benchmark-calibrated scoring oracle raises projected throughput by 36% over an analytic oracle with the same agent, and the agent framework adds 10% over a general coding agent with the same oracle. With the less accurate analytic oracle, an exhaustive non-LLM search over 530 configurations outperforms the LLM agent, with projected gains of 8.6% and 2.3%, respectively [133]. This evidence supports LLM agents as proposers of cross-axis configurations under an accurate evaluator, not as substitutes for one.

SystemDecisionEnvironmentReferenceReported resultFactorSearch cost
Parallelism and sharding configuration
ASAP [42]TPU sharding meshthree TPU jobsexpert meshmatches the expert in 3/3—n/r
Primus [133]parallelism, recompute, kernelsprojection; 32 MI355X GPUspublished BF16 recipe3,475→4,402 tok/s/GPU1.27×30 trials, 30 min
OptiCo [28]Megatron-LM strategy8 or 16 A100 GPUsbest expert configurationMFU +1.84% (0.08–8.65%)1.02×0.05–3.57 h
Serving and memory policies
Glia [54]request routersimulator; 4×A10 testbedLLQcompletion time 2.2× lower; slowdown >4.5× lower2.2×20 runs, <2 h
Autopoiesis [68](re)scheduling policy32–64-GPU clustersDistServe, HexGen, SpotServetrace time −27%, −30%, −44%1.37–1.79×online
CacheCraft [131]KV-cache evictionRULER 4k, Llama-3.1-8BChunkKVscore 69.2→84.61.22×n/r
ADRS [30]expert rebalancingMoE simulatorinternal reference19.6→3.7 ms5.3׆5 h, <$10
ADRS [30]model placementsimulatorpublished solutioncost −18.5%1.23×40 min, ≤$5
Cluster scheduling and placement
SchedMate [170]scheduler advice8-node A800 cluster; trace simulationLucidJCT 6.85→5.53 h; 1.23–1.91× in simulation1.24×9 s per advice
AlphaEvolve [117]Borg priority heuristicproduction fleetproduction heuristic0.7% of fleet compute recovered1.007×n/r
Agentic scheduler [99]tool-to-device mappingRTX A5500, 13 scenariosHEFT, StarPU, brute forceoptimal in 13/13—n/r
Incisor [78]instance selectionAWS EC2expert constraints with SkyPilotruntime −54%, cost −44%2.17×n/r

Table 19. Workload-orchestration results under native evaluation conditions. Factors are derived from the reported values as in Figure 7; a dash means that the study reports agreement or optimality rather than a ratio, and n/r means not reported. Glia’s simulator runs Llama-3-8B on four A10 GPUs; CacheCraft’s result is at 88% KV-cache compression. †ADRS reports this speedup as 5.0× in its case-study text and as 2× in its summary table; the factor shown is computed from the reported runtimes.

AlphaEvolve’s Gemini results show how gains attenuate from the decision to the job [117]: an average kernel speedup of 23% reduced overall training time by 1%. By Amdahl’s law (Equation (4)) with n=1.23n = 1.23 and 1−s1 - s the share of training time spent in the tuned kernels, this corresponds to a share of roughly 5%. Orchestration decisions act on the whole job, which is one reason why their absolute savings can exceed those of larger kernel speedups. Figure 7(b) adds the search cost. The reported solutions are found within minutes to five hours, at LLM costs of at most $20 per ADRS case study and $0.26 for an OptiCo configuration [28, 30], whereas the human effort reported for comparable outcomes is about eight hours of expert tuning and more than two weeks of expert design. OptiCo estimates that, for GPT-3 7B on 16 A100 GPUs, its 2.0% throughput gain (4,164 to 4,246 tokens/s/GPU) saves about 1,280 A100 GPU-hours over a one-trillion-token training run, so even small orchestration gains can exceed their search cost by a wide margin. These costs exclude the construction of the simulators, projection engines, and evaluators on which every result depends, which ADRS notes are difficult to design [30]. Establishing deployable value will require reporting evaluator fidelity alongside the gain, comparing against strong non-LLM optimizers that use the same evaluator, and validating policies on production traces.

Resource-Centric Evaluation (R1–R2)

Data management (R1). The studies in Figure 10 indicate different routes to improving a data service. SANA [175] separates an implementable tool configuration from a diagnostic condition with privileged information. On LakeQA [164], moving from Naive to Standard changes quality far less than moving to Ideal [175], so additional planning and retrieval machinery does not by itself realize the headroom exposed by idealized tools. The methodological implication is that planning, source selection, and analysis should be diagnosed separately before workflow complexity is increased. Retrieval calls also decrease while analysis calls increase from Naive to Ideal, so reducing one tool counter need not reduce total work.

Quality and resource use under alternative data-management configurations

Figure 10. (a) SANA [175] configurations on LakeQA [164]. (b) Evaporate [11] extraction methods. Quality and resource use under alternative data-management configurations. SANA’s Ideal condition uses oracle-like tools to diagnose headroom [175]; Evaporate compares direct extraction with synthesized code [11]. Semantic match and OpenIE Pair F1 are percentages, with higher values indicating better quality. Retrieval calls and token consumption are separate resource measures, with lower values indicating less use. Compare configurations within each subplot, not across studies.

Evaporate illustrates a complementary approach: replacing repeated per-document inference with reusable extraction code improves the average reported quality while reducing LLM token use [11]. Resource efficiency can therefore depend on where model computation is placed in a pipeline, not only on which model is used. The quality regression on Enron nevertheless shows that reusable code requires source-specific validation, which a lower aggregate cost does not remove; Table 14 retains the per-source outcomes. Together, SANA [175] and Evaporate motivate evaluating data workflows through both bottleneck diagnosis and computation reuse, with end-to-end accounting of quality and resource consumption. Their differing evaluation coverage and aggregation conventions are described in Appendix § 8.§ 4.

Table 14. Quality and resource use in data-management studies. (a) Evaporate compares direct prompting (Direct) with synthesized extraction code (Code+) on five data-lake sources; Direct is evaluated on sampled documents and Code+ on the full datasets [11]. Quality is OpenIE pair F1 (%); tokens and cost are per 10K documents, and Reduction is the per-source token ratio, averaged over sources. (b) SANA evaluates the same corpus and model under three levels of tool support, where Ideal is a diagnostic condition with privileged tools [175]. LakeQA and KramaBench-conv. contain 135 and 83 tasks; quality is semantic match (%), and the last column gives retrieval calls per task from Naive to Ideal. Higher quality and lower resource use are better. Δ\Delta values are percentage points; those in (b) are as reported by the source.

Source (format)OpenIE Pair F1LLM tokens (M)Cost ($) |
DirectCode+Δ\DeltaDirectCode+ReductionDirectCode+
FDA (TXT)45.562.8+17.3145.61.977×2 90038
Enron Emails (TXT)93.886.9−6.921.20.635×42512
Wiki NBA (HTML)44.868.2+23.4650.13.0217×13 00060
SWDE Movie (HTML)45.256.8+11.6282.92.3123×5 66046
SWDE University (HTML)53.859.0+5.2190.11.9100×3 80038
Average56.666.7+10.1258.01.9110×5 15739
BenchmarkAgentNaiveStandardIdealΔ\Delta StandardΔ\Delta IdealRetrieval calls
LakeQAgpt-5.4-nano20.719.337.0−1.5+16.35.0 → 4.1
LakeQAgpt-5-mini56.357.876.3+1.5+20.07.2 → 5.9
KramaBench-conv.gpt-5.4-nano44.657.887.9+13.3+43.43.2 → 1.6
KramaBench-conv.gpt-5-mini62.666.390.4+3.6+27.73.8 → 1.2

Table 14.

Energy optimization (R2). The energy results in Table 15 show that portability must be assessed per application and device. LASSI-EE’s cross-application averages are similar on MI100 and A100, yet individual applications differ much more, including an 81.9-point ER@5 difference for entropy [39]. Agreement between device averages therefore does not establish that a particular transformation transfers. The negative margins over the vanilla baseline for XSBench [159] on MI100 (−2.4 and −4.9 percentage points at k=3k=3 and k=5k=5); Table III of the source further motivate validating and selecting transformations on the deployment device rather than treating an energy-aware workflow as a universally beneficial replacement.

DeviceHumanLLMLLM − humanRelative
Server (Xeon Silver 4208)5.1316.082+0.951+18.5%
PC (Core i9)11.3458.588−2.757−24.3%
Raspberry Pi (Cortex-A72)0.4420.452+0.010+2.3%
ApplicationAMD MI100 k=1k=1AMD MI100 k=3k=3AMD MI100 k=5k=5NVIDIA A100 k=1k=1NVIDIA A100 k=3k=3NVIDIA A100 k=5k=5|MI100−A100|
Average (22 applications)29.649.555.228.646.954.11.1
Δ vs. vanilla(+19.1)(+22.5)(+19.2)(+18.0)(+21.4)(+20.9)
segment-reduce28.557.468.748.166.174.05.3
miniMDock20.844.151.129.650.857.96.8
XSBench3.010.616.35.315.819.43.1
jacobi22.746.552.544.866.475.222.7
lid-driven-cavity7.816.422.916.641.351.028.1
keogh13.746.459.11.05.911.747.4
entropy63.988.090.73.26.58.881.9

Table 15. Device-resolved energy outcomes of generated programs. (a) Apsan et al. measure 363 solutions to nine EvoEval problems [9, 176], using EnergiBridge on the server and PC and a Monsoon monitor on the Raspberry Pi; values are mean energy in kJ, lower is better, and positive differences indicate higher consumption by the LLM solutions. The LLM column uses the Functional prompting condition. (b) LASSI-EE reports the expected energy reduction ER@kER@k (%) over kk sampled candidates for 22 applications [39]; higher is better. Parenthesized values are margins over the vanilla LLM baseline in percentage points, and the last column is the absolute difference between the two GPUs at k=5k=5, with the largest in bold.

Budget selection introduces a further trade-off. The LASSI-EE [39] averages show smaller additional savings from later candidates than from the initial budget increase, so a deployment decision should relate this incremental benefit to the energy and latency of further search, which the execution-energy outcome does not include. In the study of Apsan et al. on EvoEval [176] problems, the ordering of human and LLM solutions reverses across platforms, which reinforces the need for measured, device-specific decisions [9].

HyperLoad occupies a different position in the resource-management loop: it improves a forecast that could inform a downstream controller [66], and its effect on energy use depends on how that controller acts on the improved information. Table 21 therefore records progress in a supporting capability, whereas Table 15 records direct program-energy outcomes. Relating forecast quality to executed control and measured consumption is required to establish end-to-end resource savings.

Table 21. HyperLoad forecast errors relative to the strongest alternative in each setting. Results use DCData cooling-load traces with different horizons and training-set sizes [66]. MSE and MAE are scaled by 10310^{3} and 10210^{2}, respectively; lower values are better and the better value in each pair is bold. Δ\Delta is relative error change, so negative values indicate an improvement.

TrainHorizonMSE (×103\times10^{3})MAE (×102\times10^{2})
Best baseHyperLoadΔ\DeltaBest baseHyperLoadΔ\Delta
100%126.15.5−9.8%5.655.39−4.6%
248.88.4−4.5%6.616.37−3.6%
4814.412.4−13.9%7.837.48−4.5%
9620.319.2−5.4%9.779.31−4.7%
50%126.25.8−6.5%5.655.44−3.7%
9620.820.0−3.8%9.919.50−4.1%
25%1216.814.0−16.7%9.278.39−9.5%
9645.444.2−2.6%14.9914.35−4.3%

Table 21.

Cross-Category Synthesis

The category comparisons yield three principles for translating reported gains into infrastructure value. First, feedback is a design component whose utility depends on the model and the task. RTL planning, kernel refinement, and energy-oriented candidate selection improve the evaluated workflows, whereas the kernel ablations show that an additional feedback channel can be counterproductive. The proposal and validation processes should therefore be designed jointly, and the feedback that improves the target outcome should be identified under a fixed budget. Figure 11 summarizes the within-study contrasts that support this conclusion.

Reported within-study improvements grouped by intervention type

Figure 11. Reported within-study improvements grouped by intervention type. Sources and evaluation budgets are given in Table 23; SANA’s idealized tools represent a diagnostic condition [175]. Rate differences are percentage points, while compiler and forecasting improvements are relative percentages. Positive values indicate improvement in the native metric. Distances across unlike metrics do not support a ranking of studies.

Second, the most useful evaluation endpoint follows the role of the generated output. A computational artifact must satisfy its functional contract and improve the relevant execution objective; a diagnosis must support a correct operational decision; an orchestration policy must improve a workload-level objective on a representative trace or cluster; and a resource-oriented action must preserve service quality while reducing the measured resource. The O3 evidence adds a requirement that applies across categories: the fidelity of the evaluator bounds the claim. Gains measured on isolated kernels shrink in end-to-end training time, and gains measured in simulators and projection engines are not directly comparable with those measured on real clusters, so the evaluation environment belongs in the result alongside the metric and the reference. The taxonomy makes these distinctions explicit. It also exposes missing connections between stages, such as report quality without recovery measurements or forecasting accuracy without observed facility savings. These connections are priorities for benchmark design rather than reasons to discard the existing results.

Third, optimization cost should be assessed over the scope of reuse. Candidate-level search, reusable compiler heuristics, and synthesized extraction functions incur costs at different points in the lifecycle, and their practical value depends on how often the validated result is subsequently used. Table 22 characterizes the reported budget dependence, while the benchmark map in Appendix § 8.7 identifies where source studies share evaluation families. Together, these analyses support comparisons that preserve setting differences and indicate which additional measurements would establish deployment value beyond a local score.

Cat.System / configurationMetricBudget levelsMetric at each levelΔ\DeltaΔ\Delta/dec.
EvolVE [60]
A1Gemini-2.0-Flash + MCTSVerilogEval-v2 pass15/50/100/200/300 nodes77.6 / 81.4 / 84.6 / 86.5 / 87.8+10.2+7.8
A1Gemini-2.0-Flash, no searchVerilogEval-v2 pass15/50/100/200/300 nodes72.4 / 74.4 / 77.6 / 78.8 / 80.1+7.7+5.9
A1GPT-OSS-120B + IGRVerilogEval-v2 pass15/50/100/200/300 nodes89.1 / 95.5 / 96.2 / 96.8 / 96.8+7.7+5.9
A1GPT-OSS-120B, no searchVerilogEval-v2 pass15/50/100/200/300 nodes92.9 / 93.6 / 94.2 / 94.2 / 94.2+1.3+1.0
A1DeepSeek-R1-FP4 + MCTSVerilogEval-v2 pass15/50/100/200/300 nodes94.2 / 96.8 / 96.8 / 97.4 / 98.1+3.9+3.0
A1DeepSeek-R1-FP4, no searchVerilogEval-v2 pass15/50/100/200/300 nodes91.0 / 94.2 / 94.9 / 94.9 / 95.5+4.5+3.5
A1DeepSeek-R1-FP4 + MCTSRTLLM-v2 pass15/50/100/200/300 nodes82.0 / 84.0 / 86.0 / 90.0 / 92.0+10.0+7.7
KernelBench [118] and GPT-5-RL [155]
A2DeepSeek-R1, Level 2fast1\mathrm{fast}_1, 100 tasks1 / 10 inference calls36 →\rightarrow 72+36+36.0
A2DeepSeek-R1, Level 1fast1\mathrm{fast}_1, 100 tasks1 / 10 inference calls12 →\rightarrow 43+31+31.0
A2DeepSeek-V3, Level 1fast1\mathrm{fast}_1, 100 tasks1 / 10 inference calls6 →\rightarrow 19+13+13.0
A2DeepSeek-V3, Level 2fast1\mathrm{fast}_1, 100 tasks1 / 10 inference calls4 →\rightarrow 6+2+2.0
A2GPT-5-RLfunctional %, 264 tasks1 / 3 attempts77.0 →\rightarrow 83.7+6.7+14.0
LASSI-EE [39]
R2LASSI-EEER@kER@k, cross-devicek=1/3/5k = 1 / 3 / 5 candidates29.1 / 48.2 / 54.6+25.5+36.5
R2vanilla LLM baselineER@kER@k, cross-devicek=1/3/5k = 1 / 3 / 5 candidates10.5 / 26.3 / 34.6+24.1+34.5

Table 22. Performance across test-time budgets. EvolVE varies the number of search nodes on VerilogEval-v2 and RTLLM-v2 [60]; KernelBench and GPT-5-RL vary the number of inference calls or attempts [118, 155]; LASSI-EE varies the number of sampled candidates kk [39]. Metric values follow the listed budget order. Δ\Delta is the endpoint change in the native metric, and Δ\Delta/dec. is the change per tenfold budget increase. Bold marks the better value of each matched pair or the best value within a block.

Related Work

Several surveys examine parts of the literature reviewed here, each organized around a particular layer of infrastructure or class of tasks.

Artifact generation and optimization. Pan et al. [120] survey LLMs for electronic design automation, and Yang et al. [183] focus on Verilog code generation; both provide domain-specific accounts of hardware generation and validation. Yu et al. [190] review automated GPU kernel generation through model adaptation and agentic optimization. Jaljajević et al. [98] survey LLMs in HPC programming, including parallel programming and code translation. Jiang et al. [67] survey LLMs for code generation more broadly, organizing data, models, evaluation, and software-development tasks. These specialized surveys provide a technical depth that a broader synthesis cannot reproduce in every subfield.

Deployment and operations. Pahl et al. [119] organize AI applications to infrastructure-as-code across development and operational activities, and Zhang et al. [194] examine LLMs for AIOps and failure management. Together, they motivate the distinction between generating a deployment configuration and interpreting evidence from a running system, although configuration, monitoring, and remediation may form successive stages of one operational workflow.

Systems for AI training and serving. Surveys of distributed training [44] and efficient LLM serving [108] catalogue the parallelism, communication, reliability, batching, and memory-management techniques that define the workloads of § 2.1. They review systems built for AI workloads rather than LLMs that engineer those systems, and they provide the vocabulary used here for the workload-orchestration category.

Data and energy. Hong et al. [58] survey LLM-based Text-to-SQL, with particular emphasis on query correctness in natural-language interfaces to data systems; data management in the present taxonomy additionally considers extraction quality, data access, and processing cost. Ali et al. [5] review sustainable code generation, which overlaps the software component of energy optimization, whereas power-oriented hardware design and facility forecasting introduce further measurements and constraints.

Distinctive perspective. This survey complements these works through its three contributions. The cross-lifecycle survey relates work on hardware, kernels, compilers, deployment, operations, orchestration, data, and energy to the training and serving of AI models, rather than treating each layer separately. The lifecycle-aligned taxonomy analyzes what the LLM manages, the feedback available to it, and the corresponding validation requirements jointly across these areas; a primary category organizes the presentation, while secondary attributes preserve the overlap with the domain-specific taxonomies above. The quantitative analysis uses primary-source outcomes to examine where LLM-driven methods improve reported results and where differing benchmarks, references, or evaluation environments restrict stronger conclusions.

Conclusion

This survey reviewed LLMs as engineering agents across the lifecycle of AI infrastructure, from accelerator design and kernel development through deployment, operation, and workload orchestration to data supply and power management. By organizing the field into a lifecycle-aligned taxonomy, characterizing each method under a common task formulation, and synthesizing the reported evidence under its native metrics and settings, it shows that LLM-driven methods now reach every stage of the lifecycle but cover few of its training- and serving-specific concerns, that their value is established most clearly where reliable feedback is available, and that reported gains must be interpreted in relation to their reference and evaluation environment. These results offer a common frame in which work from hardware, systems, HPC, and machine learning can be positioned, compared, and built upon. As evaluators become more faithful and validation extends from individual stages to the hand-offs between them, LLM-driven methods can progress from optimizing isolated artifacts to co-designing the hardware, software, and operational policies of AI infrastructure, so that the systems used to train and serve AI models are increasingly improved with the help of AI itself.

References

Evidence, Protocols, and Supplementary Analysis

Taxonomy Reference Matrix

Table 16 summarizes the assignment rule. Primary categories organize the presentation; secondary objectives and deployment dependencies remain attributes of individual studies.

IDCategoryPrimary concernRepresentative outputs and evaluation
A1Hardware DesignHardware designs and EDA decisionsRTL/HLS, circuit descriptions, formal properties; simulation, formal checks, and PPA
A2Kernel EngineeringDevice-level computationAccelerator kernels and transformations; numerical agreement and measured runtime
A3HPC SoftwareCompiler and parallel softwareCompiler decisions, translated programs, and repairs; code size, execution, and defect analysis
O1System IntegrationArtifact deployment in a target environmentConfigurations, dependency installation, and service bindings; deployment and integration tests
O2Production OperationsSystem observations and operational decisionsParsed logs, diagnoses, recommendations, and interventions; task-specific assessment and operational outcomes
O3Workload OrchestrationResource allocation for training and serving workloadsParallelization and sharding plans, routing, batching, cache, and placement policies, and schedules; simulators, projection engines, traces, and cluster measurements of throughput, latency, goodput, or job completion time
R1Data ManagementData processing, access, and associated resourcesPipelines, queries, extraction functions, and catalogs; quality, throughput, and resource use
R2Energy OptimizationEnergy, power, and thermal objectivesRefactored code, power-oriented designs, and forecasts; instrumented energy or prediction error

Table 16. Lifecycle-aligned taxonomy of LLM applications to AI infrastructure (eight categories). The principal contribution and evaluation objective determine the primary category. Secondary objectives remain attributes of a study rather than evidence that categories are disjoint.

Coverage of AI-Specific Concerns

Table 17 lists the concerns that distinguish AI infrastructure from general-purpose computing infrastructure and, for each, records its lifecycle stage (Figure 2), its taxonomy category, and the LLM-driven systems that address it. The count nn includes only systems in which an LLM acts as the engineering agent; the coverage mark is a function of nn alone, so it summarizes how much LLM-driven work exists for a concern rather than how good that work is. Concerns with n=0n = 0 are the open problems referred to in § 2.1 and § 6.

Literature Collection Protocol

Candidate pool. The review starts from an arXiv candidate pool on LLM-driven infrastructure engineering, published between 2022 and 2026, with title, authors, year, and a provisional category for each record. Authors, titles, and years were re-fetched from the arXiv API for every record that enters the review, and the provisional categories were reassigned under the rules of Section 3.1.

Targeted supplement. Because the pool under-represented the runtime decisions of training and serving, it was supplemented in this revision by targeted web and arXiv searches combining an LLM or agent term with each AI-specific concern of Table 17: parallelism or sharding configuration and Megatron [147] or DeepSpeed [134] tuning; collective communication; failure, straggler, and hang diagnosis in training clusters; serving configuration, batching, routing, autoscaling, and

SLO tuning; KV-cache eviction; mixture-of-experts expert placement; GPU-cluster job scheduling; training-data loading; and AI-driven systems research with OpenEvolve [142] or AlphaEvolve [117]. Each search was followed by backward snowballing from the retrieved papers’ related work and by verification of the arXiv or proceedings record. Searches for checkpointing, straggler mitigation, data loading, quantization choice, and power capping returned no system in which an LLM acts as the engineering agent.

Inclusion and selection. A system is eligible when an LLM generates an infrastructure artifact, proposes an infrastructure action or policy, or interprets system evidence, and when a primary paper describes its evaluation. Systems that optimize LLM execution without an LLM agent are retained only as context and are labeled as such. From the eligible records, systems were selected for detailed review to cover every method family, benchmark, and AI-specific concern of each category, preferring primary studies with recoverable evaluation settings. The selected systems’ attributes are recorded in a machine-readable table from which Figure 3 and Table 2–Table 9 are generated.

Scope of the claims. The selection is documented and covers each category’s method families, but it is not a pre-registered systematic review, and eligible papers outside the selection are individually reviewed. The quantitative synthesis in § 4 applies the additional extraction rules below.

Extraction Rules and Cross-Setting Interpretation

Provenance and arithmetic. The evidence archive records the primary PDF, version where identifiable, source location, and file hash. Numerical observations are transcribed from explicit tables, prose, or printed figure labels. Counts, rate differences, ratios, and endpoint changes are distinguished from quoted source values. Unless underlying counts are used explicitly, differences are calculated from the displayed values; small rounding differences are not evidence of a different experiment. The synthesis verifies what the sources report, not whether an independent re-execution would reproduce their results.

Comparison levels. A within-study contrast retains the study’s common task definition and records the components that change. It does not imply randomized causal identification. Cross-setting comparisons are permitted when the native metric and differing conditions remain explicit. Different baselines, devices, task subsets, or budgets qualify interpretation rather than prohibit juxtaposition. Outcomes with different meanings may be presented together as a descriptive inventory, but are not averaged into a pooled score or used to infer relative task difficulty. The protocol descriptions below apply to the corresponding main-text figures and tables.

A1: rates, revisions, and budgets. Original RTLLM [100] evaluates 30 designs with five generations per design. Its syntax percentage is generation-based, while functionality counts designs with a passing candidate (Table III). A syntax-minus-function difference is therefore not a nested failure fraction. RTLCoder [94] uses the 29-design RTLLM-v1.1 [100] set and reports pass@5; its VerilogEval [92] table combines new evaluations and values quoted from prior publications, including the GPT-3.5/GPT-4 VerilogEval rows. VeriPPA [157] uses 29 designs and the RTLLM columns of Table II; its four-shot prompt describes examples in context, not four generated candidates. EvolVE’s Table 1 uses VerilogEval-v2 [60, 125] and RTLLM-v2 and reports computational node budgets of 15–300. Node counts and model calls need not have equal cost. The syntax and functional coordinates in Figure 4 retain these source definitions. Its contours are descriptive differences between axes, not estimates of a common failure probability.

A2: threshold, selection, and execution reference. KernelBench’s fast₁ uses a strict speedup threshold of one [118]; the candidate budget is separately denoted by kk in fast⁡p@k\operatorname{fast}_{p}@k. The refinement comparison uses NVIDIA L40S and PyTorch eager. Levels 1 and 2 contain 100 tasks each; Level 3 contains 50. The GPT-5-RL [155] study uses an expanded 264-task suite and TorchInductor [8]. Its Figure 5 speedups summarize model-specific sets of correct kernels; the source separately compares their intersection in Figure 7. The Figure 5 means do not share a fixed solved-task denominator. Its one-attempt condition, three attempts with feedback, and MakoraGenerate [155] agent are distinct protocols. CUDA-L1’s main A100 results use the source’s Default reference and a speedup threshold greater than 1.01. Its limit of 20 trials occurs in training-data collection and is not assigned here as the evaluation budget [85]. Moreover, CUDA-L1 optimizes over the benchmark tasks during training; its benchmark results should not be interpreted as held-out task generalization. PEAK [153] uses custom MatMul workloads on A6000, MI200, and Adreno hardware, with human-selected transformation sequences. Its arithmetic mean across device, precision, and matrix-size results differs from a geometric mean over tasks.

A3: units and validation strength. LLM Compiler’s 3.01% concerns instruction counts on held-out functions [33], not runtime on the LLVM test suite. Zero additional compilations refers to pass selection at inference, not zero training or compilation cost. Magellan’s 5.23% and 8.79% concern different compiler-size evaluations [26]. Aware-Compiler’s 30.03% is the mean of suite-level IR-count reductions in Table 1, not a 15–25% speedup over -O3 [89]. UniPar validates translated kernels ParaTrans [20]; ACCeLLiuM’s exact-match and directive-type rates measure textual agreement [64]. MPI detection and repair use different denominators. The repair source prints 76.21%, whereas its reported 125/164 count rounds to 76.22%; the latter is used for the derived rate in this survey. CompilerGPT’s maximum speedup is one result among repeated trials on five programs [129]. These metrics are retained individually in Figure 6.

O1–O2: verifier and judge. DeployBench [169] uses fresh virtual machines, the OpenHands [167] scaffold, hidden verification, and task-specific time limits of 10–60 minutes. Five models are included in the archived source. GenKubeSec’s detection results are not proof that all proposed remediations are safe or deploy successfully [106]. TraceBench [47] compares four diagnosis tools with a GPT-4o ranker. Scores are obtained from rank transformations and aggregation across traces; the axis named accuracy is not ordinary root-cause classification accuracy. Drishti [18] is rule-based and ION [46] is LLM-based. HPC-LLM’s BERTScore of 0.808 and 0.831 in two runs is an answer-similarity metric [138], not ticket-resolution accuracy. The Kubernetes migration study [162] and the tracing, diagnosis, and log-parsing systems ARGUS, MetaKube, StateGraph, and the HPC log parser [74, 150, 177, 199] are discussed qualitatively and contribute no TraceBench measurements. AIConfigurator [182] is retained only as contextual non-LLM optimization; CPU search time and aggregate GPU-hours are reported separately, without presenting their quotient as a matched wall-clock acceleration.

O3: evaluators, references, and derived factors. Factors in Figure 7 and Table 19 are derived from source values: reference divided by method for time and cost, method divided by reference for throughput, MFU, and accuracy scores, and 1/(1−r)1/(1-r) for a reported fractional reduction rr. ASAP’s evaluation is agreement with an expert sharding mesh [42]; the 1.43×1.43 \times and 2.58×2.58 \times throughput gains it reports belong to the matched expert configurations, and the latter includes a larger batch size. Primus [133] compares an FP8 configuration found by the agent with a published BF16 reference, so its 27% measured gain combines configuration and precision; its ablation (Table 11 of the source) uses projected throughput on a different four-node cluster. OptiCo’s 1.84% is the mean relative MFU improvement over the best expert-tuned configuration across 16 model–scale settings [28]; the reported range is 0.08–8.65%. Glia’s 2.2× is mean request completion time in the vidur [1, 54] simulator with four A10 GPUs and Llama-3-8B, and its testbed result uses slowdown, a different metric. Autopoiesis [68] reports mean and maximum reductions in total trace time, which includes scheduling, reconfiguration, and serving time. CacheCraft’s factor uses the RULER [59] 4k score at 88% compression on Llama-3.1-8B [131], where the strongest baseline is ChunkKV [96]. ADRS’s expert-placement factor is computed from the reported 19.6 ms and 3.7 ms runtimes at equal imbalance [30]; the source’s case-study text reports 5.0× and its summary table 2×. SchedMate’s physical result uses an eight-node A800 cluster and Lucid [61, 170]; its simulated range spans four production traces. AlphaEvolve’s 0.7% is fleet-wide compute recovered after deployment [117], measured by the source rather than by a controlled comparison released with the paper. Search times are wall-clock times stated by the sources and exclude the construction of their evaluators.

R1: privileged information and accounting boundaries. SANA’s Naive, Standard, and Ideal conditions jointly vary planning, search, and analysis [175]. Ideal has sanitized subquestions, gold-source retrieval, and idealized analysis that may call a stronger model. A 30-tool-call limit and 600-second time limit do not equalize information access or total computation. The Ideal condition is diagnostic rather than an attainable upper-bound proof. Evaporate [11] evaluates Direct on sampled documents (20% for FDA and Wikipedia, 2% for SWDE, and 0.0002% for Enron), while Code+ processes the full datasets. These coverage differences accompany the reported quality comparison. Its approximately 110× token reduction averages per-source ratios; dividing the rounded mean token counts, 258/1.9, instead gives approximately 136×. Monetary values retain the original model and historical prices and are not current API-cost estimates. SANA’s Δ\Delta values in Table 14 are those printed by the source, computed from unrounded rates.

R2: screened candidates, meters, and forecasting. LASSI-EE [39] reports ER@kk using its screening and energy-measurement procedure; the screening includes an LLM-as-judge verdict. A VALID label does not provide a formal proof of semantic equivalence. The tabulated averages span 22 HeCBench [72] applications on MI100 and A100. The study of Apsan et al. [9] on EvoEval [176] problems uses the Functional prompting condition as the LLM reference. It uses EnergiBridge [136] on the server and PC and a Monsoon monitor on Raspberry Pi, with repeated measurements. The small Pi mean difference does not establish statistical superiority; the source reports a nonsignificant aggregate comparison there. HyperLoad [66] reference errors are selected independently as the minimum non-HyperLoad value for each metric, horizon, and training fraction. Its MSE and MAE describe cooling-load forecasts on DCData, not measured energy saved by a controller. No common percentage-efficiency scale is imposed on these studies.

Within-Study Category Figures

Figures 8 and 9 display the DeployBench [169] and TraceBench [47] values. Figure 10 retains separate data-management metrics. Their protocol differences are described above; no additional normalization is applied to make the tasks interchangeable.

Source Locations and Native Outcomes

Table 18 identifies the principal numerical sources. Locations refer to the archived PDF versions. A table entry may quote results from an earlier paper; such secondary reporting is identified in the main text. The accompanying local evidence manifest records exact filenames and hashes. Table 19 lists the workload-orchestration results summarized in Figure 7 together with their evaluation environments, references, and search costs, and Table 20 lists the HPC-software outcomes shown in Figure 6.

SourceLocationEvidence used
VerilogEval [92]Table on p. 6Machine/Human pass@k for GPT models and Verilog-SFT
RTLLM [100]Table III, p. 5Syntax rates; functional counts out of 30
RTLCoder [94]Table II, p. 4Model comparison, quoted prior results, and fine-tuning contrasts
VeriPPA [157]Table II, p. 566.89/81.37 syntax; 44.82/62.06 functionality
EvolVE [60]Table 1, p. 8Pass rates by model, search strategy, and node budget
ChipGPT [25]Evaluation, p. 7Reported 35% area reduction; contextual A1 evidence
KernelBench [118]Tables 1–2One-call baseline and ten-call sampling/refinement
GPT-5-RL [155]Figure 5, p. 16; p. 5264-task correctness, performance success, speedup, three-attempt result
CUDA-L1 [85]Tables 4–5Correct/faster task counts and arithmetic mean speedups
KernelEvolve [88]Abstract; evaluation250/250 KernelBench functional successes
PEAK [153]Table 1, p. 419.52 mean speedup; 78.26% mean documented-peak fraction
DeployBench [169]Table 2Five models on 51 tasks
GenKubeSec [106]Detection table, p. 7Precision 0.990; recall 0.999; F1 0.994
IOAgent [47]Table IVTraceBench scores for IOAgent, Drishti, and ION
ASAP [42]Section 3Agreement with expert meshes in three TPU scenarios
Primus [133]Section 4.7; Tables 8, 9, 113,475 and 4,402 tok/s/GPU; 30 trials; ablation of oracle and agent
OptiCo [28]Tables 1–2; Section 3.5MFU against expert configurations; time to best; cost-benefit estimate
Glia [54]Sections 1, 4.4, 5.1–5.3Completion time vs. LLQ and OpenEvolve; testbed slowdown; GPU-hours
Autopoiesis [68]Section 7.1Mean and maximum trace-time reductions vs. serving systems
CacheCraft [131]Section 6; scope paragraphRULER scores at 88% compression; LongBench-v2 counterexample
ADRS [30]Table 1; Section 5.2Expert-placement runtimes; placement cost; search time and cost
SchedMate [170]Section 5.2 (Table 1); Section 5.3Physical and simulated JCT; simulator error
AlphaEvolve [117]Sections 3.3.1–3.3.4Fleet compute recovered; Gemini kernel and training time; TPU circuit
TSGuard [184]Abstract; evaluationDiagnostic accuracy and verification time on production incidents
SANA [175]Tables 3–4 and 7–8LakeQA/KramaBench quality and tool calls
Evaporate [11]Table 3, p. 11Five-source OpenIE
LASSI-EE [39]Table III, p. 9ER@k values and differences from vanilla LLM
Apsan et al. [9]Table 4, p. 7Device-specific canonical and functional-prompt means
HyperLoad [66]Table 1DCData errors across horizons and training fractions

Table 18. Primary-source locations for the quantitative synthesis. Values are transcribed from the identified tables or explicit prose; differences and ratios are survey-derived unless the source reports them directly. Page numbers refer to PDF pages.

For budgets b1b_1 and b2b_2 with corresponding metric values y1y_1 and y2y_2, the descriptive endpoint change per decade is (y2−y1)/log⁡10(b2/b1)(y_2-y_1)/\log_{10}(b_2/b_1). It does not estimate a fitted scaling law or marginal return. EvolVE’s displayed values come from Table 1 without structured testbench generation [60]. The GPT-5-RL [155] three-attempt result is 221/264; its displayed 83.7% is rounded.

Cross-Category Evidence Maps

Figure 11 shows selected source-supported contrasts. Percentage-point differences and relative percentage reductions retain their native meanings. The figure does not infer causal effects from cross-study values. Table 23 identifies the quantities used.

Cat.Contrast / sourceMetricBeforeAfterChange
A1RTLLM +self-planning, GPT-3.5 [100]functional, n/3033.346.7+13.3
A1RTLLM +self-planning, GPT-4 [100]functional, n/3050.063.3+13.3
A1LLM-VeriPPA, functionality [157]functionality %44.862.1+17.2
A1LLM-VeriPPA, syntax [157]syntax %66.981.4+14.5
A2KernelBench G+E+P, R1 · L2 [118]fast1_136.072.0+36.0
A2KernelBench G+E+P, R1 · L1 [118]fast1_112.043.0+31.0
A2KernelBench G+E+P, V3 · L1 [118]fast1_16.019.0+13.0
A2KernelBench G+E+P, V3 · L2 [118]fast1_14.06.0+2.0
A2GPT-5-RL → MakoraGenerate [155]% faster21.872.9+51.1
A2V3 → R1, one attempt, L2 [118]fast1_14.036.0+32.0
A3LLM Compiler vs −Oz [33]instr. count——+3.0
O1GPT-5.4-Mini → GPT-5.3-Codex [169]pass rate %7.851.0+43.2
O2Llama-70B → GPT-4o (accuracy) [47]ranker score ×10051.364.1+12.8
O2ION (LLM) → IOAgent [47]ranker score ×10038.064.1+26.1
R1Evaporate-Direct → Code+ [11]OpenIE F156.666.7+10.1
R1SANA Naive → Ideal, LakeQA [175]semantic match %56.376.3+20.0
R2LASSI-EE vs vanilla, ER@3 [39]ER@3 %26.348.2+21.9
R2LASSI-EE vs vanilla, ER@5 [39]ER@5 %34.654.6+20.0
R2LASSI-EE vs vanilla, ER@1 [39]ER@1 %10.529.1+18.6
R2HyperLoad vs best baseline [66]relative MSE——+5.4

Table 23. Within-study contrasts underlying Figure 11. Rate changes are percentage points; compiler and forecast-error reductions are relative percentages. RTLLM contrasts use five generations, VeriPPA a four-shot GPT-4 prompt, KernelBench contrasts ten inference calls unless a single attempt is stated, LASSI-EE the stated kk, and HyperLoad a 96-step horizon; SANA’s tool comparison is diagnostic. No pooled effect or cross-metric ordering is inferred.

Figure 12 records benchmark-family associations by source paper. The GPT-5-RL and MakoraGenerate [155] configurations are treated as one study, while distinct custom HPC collections remain separate. This author-coded map summarizes the selected evidence set, not the prevalence of practices across the entire literature.

Figure 12. Benchmark-family associations in the selected evidence. Each column represents a source paper, and each marker links a paper to an evaluated benchmark family or a study-specific task collection. Family reuse does not imply identical versions, tasks, or execution conditions.

Research Agenda

Validation beyond a single test configuration. Artifact benchmarks should document shapes, data types, tolerances, program inputs, and concurrency settings. Perturbed inputs and independent validation can reveal errors that a single reference configuration misses. Formal methods can strengthen selected guarantees, while their assumptions and coverage remain part of the reported result.

Operational outcomes and recovery. Diagnosis reports should be complemented by evaluations of executed actions where appropriate. Recovery time, service impact, failed interventions, and rollback outcomes provide evidence that rank-based report assessment cannot supply. Shared telemetry should retain enough environmental context to make incident interpretation reproducible.

Uncovered training and serving concerns. LLM agents have not yet been applied to checkpoint frequency and placement, straggler mitigation, the joint choice of quantization and parallelism for a deployment, training data loaders, or power capping of AI jobs (Table 17). Each has a natural evaluator, such as goodput under failure traces, step-time variance, SLO attainment, input-pipeline stalls, or energy per token, and each is currently addressed by non-LLM optimizers that could serve as baselines.

ConcernStageCategoryLLM-driven systems (context)nnCov.
Distributed training
Parallelism and sharding plansS4O3ASAP [42], Primus Tuning Agent [133], OptiCo [28]3◐
Compute–communication overlapS2–S4A2CUCo [163]1◐
Training-accelerator circuitsS1A1AlphaEvolve (TPU arithmetic circuit) [117]1◐
Failure and straggler diagnosisS5O2TSGuard [184], Cluster Diagnosis Agent [144] (ARGUS [199], Flare [32], L4 [70])2◐
Checkpointing and recoveryS5O3none0○
Straggler mitigationS5O3none (MegaScale [71])0○
Inference serving
Routing, batching, and autoscalingS4O3Glia [54], Autopoiesis [68]2◐
KV-cache evictionS4O3CacheCraft [131]1◐
Expert and model placementS4O3ADRS [30], Let the Barbarians In [29]2◐
Quantization and parallelism choiceS4O3none (AIConfigurator [182], SLO-Guard [103], Scepsy [173])0○
Serving kernels and inference runtimesS2–S4A2FlashInfer-Bench [178], AgentCompile [86], PIKE [114], Ave [105], CuBridge [104], Semantic+Search GPU Tuner [132]6●
Inference acceleratorsS1A1Design Conductor 2.0 [154]1◐
Shared by training and serving
Deep-learning operator kernelsS2A2other AI-evaluated A2 systems (Table 3)22●
DL compilers, frameworks, and testingS3A3PassNet [97], MOSAIC [152], AlignGuard [82], FuzzGPT [41], VibeTensor [181]5●
GPU-cluster scheduling and placementS5O3SchedMate [170], AlphaEvolve (Borg) [117], Agentic CPU-GPU Scheduling [99]3◐
Training-data preparationS6R1DataFlow [87]1◐
Data loading and input pipelinesS6R1none0○
Accelerator power and facility coolingS7R2AUTOGATE (NVDLA) [168], HyperLoad [66]2◐
Power capping for AI jobsS7R2none0○

Table 17. Coverage of AI-specific training and serving concerns by LLM-driven methods. nn counts the LLM-driven systems for each concern; ● n≥4n \ge4, ◐ 1≤n≤31 \le n \le3, ○ n=0n = 0. Systems in parentheses address the concern without an LLM acting as the engineering agent and are not counted.

Evaluator fidelity for orchestration. Workload-orchestration results should report the fidelity of the simulator or projection engine against measured runs, the trace or cluster used, and the result of a strong non-LLM optimizer given the same evaluator. Policies discovered in simulation should be validated on real clusters, and their behavior under workload shift should be measured, since an evolved policy can exploit evaluator artifacts.

Cross-stage hand-offs. The lifecycle map shows no evaluated hand-off between accelerator design and kernel engineering or between data supply and power. Benchmarks that carry an artifact across stages, for example generating a kernel for a newly designed accelerator and measuring end-to-end training time, would test whether LLM methods can coordinate decisions that are currently optimized in isolation.

Resource and lifecycle accounting. Data and energy evaluations should report output quality, runtime, resource consumption, and the cost of model inference and search. Measuring a complete generate–validate–deploy–operate workflow would establish whether local artifact gains persist after integration and amortize their development cost. Cross-platform evaluations should report native results for each device before summarizing them across platforms.

References

  1. [1]Amey Agrawal, Nitin Kedia, Jayashree Mohan, Ashish Panwar, Nipun Kuwatra, Bhargav Gulavani, Ramachandran Ramjee, and Alexey Tumanov. 2024. Vidur: A Large-Scale Simulation Framework For LLM Inference. arXiv:2405.05465 https://arxiv.org/abs/2405.05465
  2. [2]Shiva Ahir and Alex Doboli. 2026. RAG-Enhanced Kernel-Based Heuristic Synthesis (RKHS): A Structured Methodology Using Large Language Models for Hardware Design. arXiv preprint arXiv:2604.26153.arxiv.org/abs/2604.26153
  3. [3]Ahmed G. A. H. Ahmed and C. Okan Sakar. 2026. DW-Bench: Benchmarking LLMs on Data Warehouse Graph Topology Reasoning. arXiv:2604.180964 [cs.AI] doi:10.48550/arXiv.2604.180964
  4. [4]Mohammad Akyash, Nowfel Mashnoor, Kimia Azar, and Hadi Kamali. 2026. CASS-RTL: Correctness-Aware Subspace Steering for RTL Generation with LLMs. arXiv:2606.05680 [cs.PL] doi:10.48550/arXiv.2606.05680arxiv.org/abs/2606.05680
  5. [5]Sabiya Banu Masthan Ali, Ossema Kirmani, Aroosa Hameed, Syed Muhammad Danish, and Gautam Srivastava. 2026. Energy-Efficient Code Generation Using Large Language Models: A Systematic Literature Review. arXiv preprint arXiv:2603.00989.
  6. [6]Gene M. Amdahl. 1967. Validity of the single processor approach to achieving large scale computing capabilities. In Proceedings of the April 18–20, 1967, Spring Joint Computer Conference. ACM, 483–485. doi:10.1145/1465482.1465560DOI
  7. [7]Qi An, Chihua Ying, Yujing Zhu, Yihao Xu, Manwei Zhang, and Jiannan Wang. 2025. LEDD: Large Language Model- Empowered Data Discovery in Data Lakes. arXiv:2502.15182 [cs.DB] doi:10.48550/arXiv.2502.15182DOI
  8. [8]Jason Ansel et al. 2024. PyTorch 2: Faster Machine Learning Through Dynamic Python Bytecode Transformation and Graph Compilation. In Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS). ACM, New York, NY, USA. doi:10.1145/3620665.3640366DOI
  9. [9]Radu Apsan, Vincenzo Stoico, Michel Albonico, Rudra Dhar, Karthik Vaidhyanathan, and Ivano Malavolta. 2026. Generating Energy-Efficient Code via Large-Language Models – Where Are We Now? Proceedings of the 48th International Conference on Software Engineering (ICSE), arXiv:2509.10099.arxiv.org/abs/2509.10099
  10. [10]Mohsen Seydkazemi Ardebili and Andrea Bartolini. 2025. KubEntellect: A Modular LLM-Orchestrated Agent Framework for End-to-End Kubernetes Management. arXiv:2509.02449 [cs.DC] doi:10.48550/arXiv.2509.02449
  11. [11]Simran Arora, Brandon Yang, Sabri Eyuboglu, Avanika Narayan, Andrew Hojel, Immanuel Trummer, and Christopher Ré. 2023. Language Models Enable Simple Systems for Generating Structured Views of Heterogeneous Data Lakes. Proceedings of the VLDB Endowment 17, 2 (2023), 92–105. arXiv:2304.09433. doi:10.14778/3626292.3626294DOI
  12. [12]Mohammad Atif, Kriti Chopra, Fang-Ying Tsai, Ozgur O. Kilic, Tianle Wang, Zhihua Dong, Douglas Benjamin, Charles Leggett, Meifeng Lin, Paolo Calafiura, and Salman Habib. 2026. CelloAI Benchmarks: Toward Repeatable Evaluation of AI Assistants. arXiv:2603.01051 [hep-ex] doi:10.48550/arXiv.2603.01051arxiv.org/abs/2603.01051
  13. [13]Aueaphum Aueawathanaphisut and Badri Raj Lamichhane. 2026. Trustworthy Self-Composable Big-Data-as-a-Service: An LLM-Orchestrated Multi-Agent Framework for Automated Data Engineering, AutoML, MLOps Deployment, and Drift-Aware Lifecycle Optimization. arXiv:2606.17915 [cs.MA] doi:10.48550/arXiv.2606.17915DOI
  14. [14]Haolei Bai, Lingcheng Kong, Xueyi Chen, Jianmian Wang, Zhiqiang Tao, and Huan Wang. 2026. DICE: Diffusion Large Language Models Excel at Generating CUDA Kernels. arXiv:2602.17115 [cs.LG] doi:10.48550/arXiv.2602.17115
  15. [15]Linus Bantel, Anna-Lena Roth, Jonas Posner, and Dirk Pflüger. 2026. Generated, Parallel, Scalable? A Study of Agentic AI-Generated Julia Code on Supercomputers. arXiv:2606.16534 [cs.DC] doi:10.48550/arXiv.2606.16534DOI
  16. [16]Linus Bantel, Moritz Strack, Alexander Strack, and Dirk Pflüger. 2026. From Compilers to Performance: Evaluating LLMs for Task-based Parallel Code Generation. arXiv:2602.22240 [cs.PL] doi:10.48550/arXiv.2602.22240DOI
  17. [17]Zhixuan Bao, Zhuoyi Lin, Jiageng Wang, et al. 2026. AnalogAgent: Self-Improving Analog Circuit Design Automation with LLM Agents. arXiv preprint arXiv:2603.23910.arxiv.org/abs/2603.23910
  18. [18]Jean Luca Bez, Hammad Ather, and Suren Byna. 2022. Drishti: Guiding End-Users in the I/O Optimization Journey. In 2022 IEEE/ACM International Parallel Data Systems Workshop (PDSW). 1–6. doi:10.1109/PDSW56643.2022.00006DOI
  19. [19]Harshraj Bhutani. 2025. AI-Driven Generation of Data Contracts in Modern Data Engineering Systems. arXiv preprint arXiv:2507.21056.
  20. [20]Tomer Bitan, Tal Kadosh, Erel Kaplan, Shira Meiri, Le Chen, Peter Morales, Niranjan Hasabnis, and Gal Oren. 2025. UniPar: A Unified LLM-Based Framework for Parallel and Accelerated Code Translation in HPC. arXiv preprint arXiv:2509.12136.arxiv.org/abs/2509.12136
  21. [21]Cameron B. Browne, Edward Powley, Daniel Whitehouse, Simon M. Lucas, Peter I. Cowling, Philipp Rohlfshagen, Stephen Tavener, Diego Perez, Spyridon Samothrakis, and Simon Colton. 2012. A Survey of Monte Carlo Tree Search Methods. IEEE Transactions on Computational Intelligence and AI in Games 4, 1 (2012), 1–43. doi:10.1109/TCIAIG.2012.2186810DOI
  22. [22]Xinzi Cao, Jianyang Zhai, Pengfei Li, Zhiheng Hu, Cen Yan, Bingxu Mu, Guanghuan Fang, Bin She, Jiayu Li, Yihan Su, Dongyang Tao, Xiansong Huang, Fan Xu, Feidiao Yang, Yao Lu, Chang-Dong Wang, Yutong Lu, Weicheng Xue, Bin Zhou, and Yonghong Tian. 2026. AscendKernelGen: A Systematic Study of LLM-Based Kernel Generation for Neural Processing Units. arXiv:2601.07160 [cs.AI] doi:10.48550/arXiv.2601.07160DOI
  23. [23]Tom Cappendijk, Pepijn de Reus, and Ana Oprescu. 2024. Generating Energy-efficient code with LLMs. arXiv preprint arXiv:2411.10599.DOI
  24. [24]Che-Ming Chang, Prashanth Vijayaraghavan, Ashutosh Jadhav, Charles Mackin, Vandana Mukherjee, Hsinyu Tsai, and Ehsan Degan. 2026. CODMMAS: A Dialectic Multi-Agent Collaborative Framework for Structured RTL Optimization. arXiv:2603.17204 [cs.CL] doi:10.48550/arXiv.2603.17204DOI
  25. [25]Kaiyan Zhang, Ying Wang, Haimeng Ren, Mengdi Wang, Shengwen Liang, Yinhe Han, Huawei Li, and Xiaowei Li. 2023. ChipGPT: How Far Are We from Natural Language Hardware Design. arXiv preprint arXiv:2305.14019.
  26. [26]Hongzheng Chen, Alexander Novikov, Ngân Vũ, Hanna Alam, Zhiru Zhang, Aiden Grossman, Mircea Trofin, and Amir Yazdanbakhsh. 2026. Magellan: Autonomous Discovery of Novel Compiler Optimization Heuristics with AlphaEvolve. arXiv preprint arXiv:2601.21096.arxiv.org/abs/2601.21096
  27. [27]Mu-Chi Chen, Po-Hsuan Huang, Yu-Hung Kao, Yen-Fu Liu, Yu-Kai Hung, Cheng Liang, Shao-Chun Ho, Chia-Heng Tu, and Shi-Hao Hung. 2026. Exploring LLM-based Verilog Code Generation with Data-Efficient Fine-Tuning and Testbench Automation. arXiv:2604.15388 [cs.AR] doi:10.48550/arXiv.2604.15388DOI
  28. [28]Sheng Chen, Zhe Tang, Weixing Zhang, Fei Yang, Yuanyuan Wang, Tianlin Li, and Yang Liu. 2026. OptiCo: Adaptive Distributed Training Optimization via Collaborative Agent Reasoning. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL). Association for Computational Linguistics, 27847–27867.
  29. [29]Audrey Cheng, Shu Liu, Melissa Pan, Zhifei Li, Shubham Agarwal, Mert Cemri, Bowen Wang, Alexander Krentsel, Tian Xia, Jongseok Park, Shuo Yang, Jeff Chen, Lakshya Agrawal, Ashwin Naren, Shulu Li, Ruijang Ma, Aditya Desai, Jiarong Xing, Koushik Sen, Matei Zaharia, and Ion Stoica. 2025. Let the Barbarians In: How AI Can Accelerate Systems Performance Research. arXiv preprint arXiv:2512.14806.arxiv.org/abs/2512.14806
  30. [30]Audrey Cheng, Shu Liu, Melissa Pan, Zhifei Li, Bowen Wang, Alex Krentsel, Tian Xia, Mert Cemri, Jongseok Park, Shuo Yang, Jeff Chen, Lakshya Agrawal, Aditya Desai, Jiarong Xing, Koushik Sen, Matei Zaharia, and Ion Stoica. 2025. Barbarians at the Gate: How AI is Upending Systems Research. arXiv preprint arXiv:2510.06189.arxiv.org/abs/2510.06189
  31. [31]Omri Sgan Cohen, Ehud Malul, Yair Meidan, Dudu Mimran, Yuval Elovici, and Asaf Shabtai. 2025. KubeGuard: LLM-Assisted Kubernetes Hardening via Configuration Files and Runtime Logs Analysis. arXiv:2509.04191 [cs.CR] doi:10.48550/arXiv.2509.04191DOI
  32. [32]Weihuao Cui, Ji Zhang, Han Zhao, Chao Liu, Jian Sha, Bingsheng He, Minyi Guo, and Quan Chen. 2025. Flare: Anomaly Diagnostics for Divergent LLM Training in GPU Clusters of Thousand-Plus Scale. arXiv preprint arXiv:2502.05413.arxiv.org/abs/2502.05413
  33. [33]Chris Cummins, Volker Seeker, Dejan Grubisic, Mostafa Elhoushi, Youwei Liang, Baptiste Rozier, Jonas Gehring, Fabian Gloelcke, Kim Hazelwood, Gabriel Synnaeve, and Hugh Leather. 2023. Large Language Models for Compiler Optimization. arXiv preprint arXiv:2309.07062.arxiv.org/abs/2309.07062
  34. [34]Weinan Dai, Hanlin Wu, Qiying Yu, Huan Guo, Jiahao Li, Chengquan Jiang, Weiqiang Lou, Yufan Song, Hongli Yu, Jiaze Chen, Wei-Ying Ma, Ya-Qin Zhang, Jingjing Liu, Mingxuan Wang, Xin Liu, and Hao Zhou. 2026. CUDA Agent: Large-Scale Agentic RL for High-Performance CUDA Kernel Generation. arXiv:2602.24286 [cs.LG] doi:10.48550/arXiv.2602.24286DOI
  35. [35]Zhenwei Dai, Chuan Lei, Asterios Katsifodimos, et al. 2025. LAKEGEN: A LLM-based Tabular Corpus Generator for Evaluating Dataset Discovery in Data Lakes. arXiv preprint arXiv:2507.04687.arxiv.org/abs/2507.04687
  36. [36]Tri Dao, Dan Fu, Stefano Ermon, Ari Rudra, and Christopher Ré. 2022. FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness. In Advances in Neural Information Processing Systems (NeurIPS). Curran Associates, Inc., Red Hook, NY, USA, 16344–16359.arxiv.org/abs/2205.14135
  37. [37]Joshua H. Davis, Daniel Nichols, Ishan Khillan, and Abhinav Bhatele. 2025. ParEval-Repo: A Benchmark Suite for Evaluating LLMs with Repository-Level HPC Translation Tasks. arXiv preprint arXiv:2506.20938.arxiv.org/abs/2506.20938
  38. [38]William Dawson, Louis Beal, Yoann Curé, Giuseppe Fisicaro, Dorian Rolland, and Luigi Genovese. 2026. LARA: Validation-Driven Agentic Supercomputer Workflows for Atomistic Modeling. arXiv:2604.22571 [physics.comp-ph] doi:10.48550/arXiv.2604.22571arxiv.org/abs/2604.22571
  39. [39]Matthew T. Dearing, Yiheng Tao, Zhiling Lan, Xingfu Wu, and Valerie Taylor. 2025. Leveraging LLMs to Automate Energy-Aware Refactoring of Parallel Scientific Codes. arXiv preprint arXiv:2505.02184.arxiv.org/abs/2505.02184
  40. [40]Chenhui Deng, Zhongzhi Yu, Guan-Ting Liu, et al. 2026. ACE-RTL: When Agentic Context Evolution Meets RTL-Specialized LLMs. arXiv preprint arXiv:2602.10218.arxiv.org/abs/2602.10218
  41. [41]Yinlin Deng, Chunqiu Steven Xia, Chenyuan Yang, Shizhuo Dylan Zhang, Shujing Yang, and Lingming Zhang. 2023. Large Language Models are Edge-Case Fuzzers: Testing Deep Learning Libraries via FuzzGPT. arXiv:2304.02014 [cs.SE] doi:10.48550/arXiv.2304.02014
  42. [42]Yuran Ding, Xinwei Xu, Xiaofan Zhang, and Zongwei Zhou. 2025. ASAP: an Agentic Solution to Auto-optimize Performance of Large-Scale LLM Training. arXiv preprint arXiv:2511.03844.arxiv.org/abs/2511.03844
  43. [43]Weihua Du, Jingming Zhuo, Yixin Dong, Andre Wang, Heiwei Sun, Zeyu Zheng, Manupa Karunaratne, Ivan Fox, Tim Dettmers, Tianqi Chen, Yiming Yang, and Sean Welleck. 2026. AdaExplore: Failure-Driven Adaptation and Diversity-Preserving Search for Efficient Kernel Generation. arXiv:2604.16625 [cs.CL] doi:10.48550/arXiv.2604.16625DOI
  44. [44]Jiangfei Duan, Shuo Zhang, Zerui Wang, Lijuan Jiang, Wenwen Qu, Qinghao Hu, Guoteng Wang, Coihen Wen, Hong Yan, Xingcheng Zhang, Xipeng Qiu, Dahua Lin, Yonggang Wen, Xin Jin, Tianwei Zhang, and Peng Sun. 2024. Efficient Training of Large Language Models on Distributed Infrastructures: A Survey. arXiv preprint arXiv:2407.20018. Kyle Dumont, Nicholas Herbert, Hayder Tirmazi, and Shrikant Upadhyay. 2026. DRCY: Agentic Hardware Design Reviews. arXiv:2603.15672 [cs.AR] doi:10.48550/arXiv.2603.15672
  45. [46]Chris Egersdoerfer, Arnav Sareen, Jean Luca Bez, Suren Byna, and Dong Dai. 2024. ION: Navigating the HPC I/O Optimization Journey using Large Language Models. In Proceedings of the 16th ACM Workshop on Hot Topics in Storage and File Systems. 86–92. doi:10.1145/3655038.3665950DOI
  46. [47]Chris Egersdoerfer, Arnav Sareen, Jean Luca Bez, Suren Byna, Dongkuan Xu, and Dong Dai. 2026. IOAgent: Democratizing Trustworthy HPC I/O Performance Diagnosis Capability via LLMs. arXiv preprint arXiv:2602.22017.DOI
  47. [48]Cagri Eryilmaz. 2026. ChipCraftBrain: Validation-First RTL Generation via Multi-Agent Orchestration. arXiv preprint arXiv:2604.19856.arxiv.org/abs/2604.19856
  48. [49]Wenji Fang, Yao Lu, Shang Liu, Jing Wang, Ziyan Guo, Junxian He, Fengbin Tu, and Zhiyao Xie. 2026. Dr. RTL: Autonomous Agentic RTL Optimization through Tool-Grounded Self-Improvement. arXiv:2604.14989 [cs.AI] doi:10.48550/arXiv.2604.14989DOI
  49. [50]Dawei Feng, Di Mei, Huiri Tan, Lei Ren, Xianying Lou, and Zhangxi Tan. 2025. StreamLink: Large-Language-Model Driven Distributed Data Engineering System. arXiv:2505.21575 [cs.DB] doi:10.48550/arXiv.2505.21575
  50. [51]Kelin Fu, Tianyu Liu, Zeyu Shang, Yigwei Ma, Jian Yang, Jiaheng Liu, and Kaigui Bai. 2025. Multi-Docker-Eval: A ‘Shovel of the Gold Rush’ Benchmark on Automatic Environment Building for Software Engineering. arXiv:2512.06915 [cs.SE] doi:10.48550/arXiv.2512.06915DOI
  51. [52]Todd G. Gamblin, Matthew LeGendre, Michael R. Collette, Gregory L. Lee, Adam Moody, Bronis R. de Supinski, and Scott Futral. 2015. The Spack Package Manager: Bringing Order to HPC Software Chaos. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC). ACM, New York, NY, USA. doi:10.1145/2807591.2807623
  52. [53]Ricardo Gonzalez and Mark Horowitz. 1996. Energy dissipation in general purpose microprocessors. IEEE Journal of Solid-State Circuits 31, 9 (1996), 1277–1284. doi:10.1109/4.535411DOI
  53. [54]Pouya Hamadian, Partha Karimi, Arash Nasr-Esfahany, Kimia Noorbakhsh, Joseph Chandler, Ali ParandehGheibi, Mohammad Alizadeh, and Hari Balakrishnan. 2025. Glia: A Human-Inspired AI for Automated Systems Design and Optimization. arXiv preprint arXiv:2510.27176.
  54. [55]Khandakar Shakib Al Hasan, Syed Rifat Raiyan, Hasin Mahtab Alvee, and Wahid Sadik. 2026. CircuitLM: A Multi-Agent LLM-Aided Design Framework for Generating Circuit Schematics from Natural Language Prompts. arXiv:2601.04505 [cs.AI] doi:10.48550/arXiv.2601.04505
  55. [56]Sina Heidari and Dimitrios S. Nikolopoulos. 2026. FACT: Compositional Kernel Synthesis with a Three-Stage Agentic Workflow. arXiv:2604.26666 [cs.DC] doi:10.48550/arXiv.2604.26666DOI
  56. [57]Brendan R. Hogan, Xiwen Chen, James T. Wilson, Kashif Rasul, Adel Boyarsky, Thomas Kamei, Anderson Schneider, and Yuriy Nevmyvaka. 2026. AlphaLab: Autonomous Multi-Agent Research Across Optimization Domains with Frontier LLMs. arXiv:2604.08590 [cs.LG] doi:10.48550/arXiv.2604.08590DOI
  57. [58]Zijin Hong, Zheng Yuan, Qinggang Zhang, Hao Chen, Junnan Dong, Feiran Huang, and Xiao Huang. 2025. Next-Generation Database Interfaces: A Survey of LLM-Based Text-to-SQL. IEEE Transactions on Knowledge and Data Engineering (2025). arXiv:2406.08426. doi:10.1109/TKDE.2025.3609486DOI
  58. [59]Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman, Shantanu Acharyya, Dima Rekesh, Fei Jia, Yang Zhang, and Boris Ginsburg. 2024. RULER: What’s the Real Context Size of Your Long-Context Language Models? arXiv:2404.06654 https://arxiv.org/abs/2404.06654
  59. [60]Wei-Po Hsieh, Ren-Hao Deng, Yao-Ting Hsieh, En-Ming Huang, and Shih-Hao Hung. 2026. EvolVE: Evolutionary Search for LLM-Based Verilog Generation and Optimization. arXiv preprint arXiv:2601.18067.arxiv.org/abs/2601.18067
  60. [61]Qinghao Hu, Meng Zhang, Peng Sun, Yonggang Wen, and Tianwei Zhang. 2023. Lucid: A Non-intrusive, Scalable and Interpretable Scheduler for Deep Learning Training Jobs. In Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS). ACM, New York, NY, USA. doi:10.1145/3575693.3575705DOI
  61. [62]Jaber Jaber and Osama Jaber. 2026. AutoKernel: Autonomous GPU Kernel Optimization via Iterative Agent-Driven Search. arXiv:2603.21331 [cs.LG] doi:10.48550/arXiv.2603.21331DOI
  62. [63]Prachi Jadhav, Hongwei Jin, Ewa Deelman, and Prasanna Balaprakash. 2025. Evaluating the Efficacy of LLM-Based Reasoning for Multiobjective HPC Job Scheduling. arXiv:2506.02025 [cs.DC] doi:10.48550/arXiv.2506.02025arxiv.org/abs/2506.02025
  63. [64]Samyak Jhaveri, Vanessa Klotzmann, and Crista Lopes. 2025. ACCELLIuM: Supervised Fine-Tuning for Automated OpenACC Pragma Generation. arXiv preprint arXiv:2509.20380.
  64. [65]Samyak Jhaveri and Cristina V. Lopes. 2026. Bridging the Gap: Empowering Small Models in Reliable OpenACC-based Parallelization via GEPA-Optimized Prompting. arXiv:2601.08884 [cs.SE] doi:10.48550/arXiv.2601.08884DOI
  65. [66]Haoyuan Jiang, Boan Qu, Junjie Zhu, Fanjie Zeng, Xiaojie Lin, and Wei Zhong. 2025. HyperLoad: A Cross-Modality Enhanced Large Language Model-Based Framework for Green Data Center Cooling Load Prediction. arXiv preprint arXiv:2512.19114.
  66. [67]Juyong Jin, Fang Wang, Jiasi Shen, Sungju Kim, and Sunghun Kim. 2024. A Survey on Large Language Models for Code Generation. arXiv preprint arXiv:2406.00515.DOI
  67. [68]Youhe Jiang, Ran Yan, You Peng, Wenshuang Li, Taiyi Wang, Fangcheng Fu, and Binhang Yuan. 2026. Autopoiesis: A Self-Evolving System Paradigm for LLM Serving Under Runtime Dynamics. arXiv preprint arXiv:2604.07144.arxiv.org/abs/2604.07144
  68. [69]Youhe Jiang, Ran Yan, Xiaozhe Yao, Yang Zhou, Beidi Chen, and Binhang Yuan. 2023. HexGen: Generative Inference of Large Language Model over Heterogeneous Environment. arXiv:2311.11514. https://arxiv.org/abs/2311.11514
  69. [70]Zhihan Jiang, Junjie Huang, Zhangbin Chen, Yichen Li, Guangba Yu, Cong Feng, Yongqiang Yang, Zengyin Yang, and Michael R. Lyu. 2025. L4: Diagnosing Large-scale LLM Training Failures via Automated Log Analysis. arXiv preprint arXiv:2503.20263.arxiv.org/abs/2503.20263
  70. [71]Ziheng Jiang, Haibin Lin, Yinmin Zhong, Qi Huang, Yangrui Chen, Zhi Zhang, Yanghua Peng, Xiang Li, Cong Xie, Shibiao Nong, Yulu Jia, Sun He, Hongmin Chen, Zhihao Bai, Qi Hou, Shipeng Yan, Ding Zhou, Yiyao Sheng, Zhuo Jiang, Haoan Xu, Haoran Wei, Zhang Zhang, Pengfei Nie, Leqi Zou, Sida Zhao, Liang Xiang, Zherui Liu, Zhe Li, Xiaoying Jia, Jianxi Ye, Xin Jin, and Xin Liu. 2024. MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs. arXiv preprint arXiv:2402.15627.arxiv.org/abs/2402.15627
  71. [72]Zheming Jin. 2023. HeCBench: Heterogeneous Computing Benchmarks. https://github.com/zjin-lcf/HeCBench
  72. [73]Sylvio Barbon Junior, Paolo Ceravolo, Sven Groppe, et al. 2024. Are Large Language Models the New Interface for Data Pipelines? arXiv preprint arXiv:2406.06596.arxiv.org/abs/2406.06596
  73. [74]Ahmad Maroof Karimi, Jong Youl Choi, Charles Qing Cao, and Awais Khan. 2026. Instruction-Tuned LLMs for Parsing and Mining Unstructured Logs on Leadership HPC Systems. arXiv preprint arXiv:2604.05168.arxiv.org/abs/2604.05168
  74. [75]Ahmad Maroof Karimi, Woong Shin, Jesse Hines, Tirthankar Ghosal, Naw Safin Sattar, and Feiyi Wang. 2025. EPIC: Generative AI Platform for Accelerating HPC Operational Data Analytics. arXiv:2509.16212 [cs.DB] doi:10.48550/arX iv.2509.16212arxiv.org/abs/2509.16212
  75. [76]Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient Memory Management for Large Language Model Serving with PagedAttention. arXiv:2309.06180 https://arxiv.org/abs/2309.06180
  76. [77]Eugenie Lai, Gerardo Vitagliano, Ziyu Zhang, Om Chabra, Sivaprasad Sudhir, Anna Zeng, Anton A. Zabreyko, Chenning Li, Ferdi Kossmann, Jialin Ding, Jun Chen, Markos Markakis, Matthew Russo, Weiyang Wang, Ziniu Wu, Michael J. Cafarella, Lei Cao, Samuel Madden, and Tim Kraska. 2025. KramaBench: A Benchmark for AI Systems on Data-to-Insight Pipelines over Data Lakes. arXiv:2506.06541 [cs.DB] doi:10.48550/arXiv.2506.06541DOI
  77. [78]Michael A. Lauren z ano, Shihan Cheng, and David A. B. Hyde. 2026. Incisor: Ex Ante Cloud Instance Selection for HPC Jobs. arXiv preprint arXiv:2604.24464.arxiv.org/abs/2604.24464
  78. [79]Chongxiao Li, Pengwei Jin, Di Huang, et al. 2026. AutoPPA: Automated Circuit PPA Optimization via Contrastive Code-based Rule Library Learning. arXiv preprint arXiv:2604.18445.arxiv.org/abs/2604.18445
  79. [80]Jianling Li, Shangzhan Li, Zhenye Gao, Qi Shi, Yuxuan Li, Zefan Wang, Jiacheng Huang, Haojie Wang, Jianrong Wang, Xu Han, Zhiyuan Liu, and Maosong Sun. 2025. TritonBench: Benchmarking Large Language Model Capabilities for Generating Triton Operators. arXiv preprint arXiv:2502.14752.arxiv.org/abs/2502.14752
  80. [81]Jie Li, Wenzhao Wu, Junqi Hu, Qinrui Zheng, Bowen Wu, Juepeng Zheng, Yutong Lu, and Haohuan Fu. 2026. CodegenBench: Can LLMs Write Efficient Code Across Architectures? arXiv:2606.04023 [cs.SE] doi:10.48550/arXiv.2606.04023DOI
  81. [82]Meiziniu Li, Dongze Li, Jianmeng Liu, and Shing-Chi Cheung. 2026. Demystifying the Silence of Correctness Bugs in PyTorch Compiler. arXiv:2604.08720 [cs.SE] doi:10.48550/arXiv.2604.08720DOI
  82. [83]Shiyang Li, Haoyang Chen, Mattia Fazzini, and Caiwen Ding. 2026. CUDABeaver: Benchmarking LLM-Based Automated CUDA Debugging. arXiv:2605.08455 [cs.LG] doi:10.48550/arXiv.2605.08455DOI
  83. [84]Shiyang Li, Zijian Zhang, Guangyan Sun, Yuebo Luo, Winson Chen, Yanzhi Wang, Mingyi Hong, and Caiwen Ding. 2026. CUDAHercules: Benchmarking Hardware-Aware Expert-level CUDA Optimization for LLMs. arXiv:2605.08467 [cs.LG] doi:10.48550/arXiv.2605.08467DOI
  84. [85]Xiaoya Li, Albert Wang, Guoyin Wang, Jiwei Li, and Chris Shum. 2025. CUDA-L1: Improving CUDA Optimization via Contrastive Reinforcement Learning. arXiv preprint arXiv:2507.14111.arxiv.org/abs/2507.14111
  85. [86]Xuanzhe Li, Ziyan Weng, Zhiyu Zhu, and Junhui Hou. 2026. AgentCompile: An LLM-Guided Compiler for Direct CUDA Inference. arXiv:2606.07665 [cs.PL] doi:10.48550/arXiv.2606.07665DOI
  86. [87]Hao Liang, Xiaochen Ma, Zhou Liu, Zhen Hao Wong, Zhengyang Zhao, Zimo Meng, et al. 2025. DataFlow: An LLM-Driven Framework for Unified Data Preparation and Workflow Automation in the Era of Data-Centric AI. arXiv preprint arXiv:2512.16676.arxiv.org/abs/2512.16676
  87. [88]Gang Liao, Hongsen Qin, Ying Wang, Alicia Golden, Michael Kuchnik, Yavuz Yetim, Jia Jiunn Ang, Chunli Fu, Yihan He, Samuel Hsia, Zewei Jiang, Jianshi Li, Uladzimir Pashkevich, Varna Puvvada, Feng Shi, Matt Steiner, Ruichao Xiao, Liuyuan Li, Nathan Yan, Xiayu Yu, Zhou Fang, Roman Levenstein, Kunming Ho, Haishan Zhu, Alec Hammond, Richard Li, Ajit Mathews, Kaustubh Gondkar, Abdul Zainul-Abedin, Ketan Singh, Hongtao Yu, Wenyuan Chi, Barney Huang, Sean Zhang, Noah Weller, Zach Marine, Wyatt Cook, Carole-Jean Wu, and Gaoxiang Liu. 2025. KernelEvolve: Scaling Agentic Kernel Coding for Heterogeneous AI Accelerators at Meta. arXiv preprint arXiv:2512.23236.arxiv.org/abs/2512.23236
  88. [89]Hongyu Lin, Haolin Pan, Haoran Luo, Yuchen Li, Kaichun Yao, Libo Zhang, Mingjie Xing, and Yanjun Wu. 2025. AwareCompiler: Agentic Context-Aware Compiler Optimization via a Synergistic Knowledge-Data Driven Framework. arXiv preprint arXiv:2510.11759.arxiv.org/abs/2510.11759
  89. [90]Jiale Liu, Taiyu Zhou, and Tianqi Jiang. 2026. Veri-Sure: Multi-Agent RTL Code Generation with Temporal Tracing, Slicing and Formal Verification. arXiv preprint arXiv:2601.19747.arxiv.org/abs/2601.19747
  90. [91]Mingjie Liu, Teodor-Dumitru Ene, Robert Kirby, Chris Cheng, Nathaniel Pinckney, Rongjian Liang, Jonah Alben, Himanshu Anand, Sanmitra Banerjee, Ismet Bayraktaroglu, et al. 2023. ChipNeMo: Domain-Adapted LLMs for Chip Design. arXiv:2311.00176 [cs.CL] https://arxiv.org/abs/2311.00176
  91. [92]Mingjie Liu, Nathaniel Pinckney, Bruck Khailany, and Haoxing Ren. 2023. VerilogEval: Evaluating Large Language Models for Verilog Code Generation. arXiv preprint arXiv:2309.07544.arxiv.org/abs/2309.07544
  92. [93]Pengju Liu, Nuo Xu, Jinwei Tang, Yu Cao, and Caiwen Ding. 2026. Bridging the Last Mile of Circuit Design: PostEDA-Bench, a Hierarchical Benchmark for PPA Convergence and DRC Fixing. arXiv:2605.06936 [cs.AR] doi:10.48550/arXiv.2605.06936DOI
  93. [94]Shang Liu, Wenji Fang, Yao Lu, Qijun Zhang, Hongce Zhang, and Zhiyao Xie. 2023. RTLCoder: Outperforming GPT-3.5 in Design RTL Generation with Our Open-Source Dataset and Lightweight Solution. arXiv preprint arXiv:2312.08617.DOI
  94. [95]Wei Liu, Jiawei Xu, Yingru Li, Longtao Zheng, Tianjian Li, Qian Liu, and Junxian He. 2026. Dr. Kernel: Reinforcement Learning Done Right for Triton Kernel Generations. arXiv:2602.05885 [cs.LG] doi:10.48550/arXiv.2602.05885DOI
  95. [96]Xiang Liu, Zhenheng Tang, Peijie Dong, Zeyu Li, Yue Liu, Bo Li, Xuming Hu, and Xiaowen Chu. 2025. ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inference. arXiv:2502.00299 https://arxiv.org/abs/2502.00299
  96. [97]Yiqun Liu, Yingsheng Wu, Ruqi Yang, et al. 2026. PassNet: Scaling Large Language Models for Graph Compiler Pass Generation. arXiv preprint arXiv:2605.29357.arxiv.org/abs/2605.29357
  97. [98]Strahinja Ljaljević, Josep Jorba, and Sergio Iserte. 2026. Exploring the role of Large Language Models in High-Performance Computing programming: A survey. Future Generation Computer Systems 184 (2026), 108618. arXiv:2608.26110. doi:10.1016/j.future.2026.108618DOI
  98. [99]Tianxi Lu and Sherief Reda. 2026. Agentic CPU-GPU Scheduling for Heterogeneous AI Workloads. arXiv preprint arXiv:2607.2242.arxiv.org/abs/2607.22242
  99. [100]Yao Lu, Shang Liu, Qijun Zhang, and Zhiyao Xie. 2023. RTLLM: An Open-Source Benchmark for Design RTL Generation with Large Language Model. arXiv preprint arXiv:2308.05345.arxiv.org/abs/2308.05345
  100. [101]Yao Lu, Shang Liu, Hangan Zhou, Wenji Fang, Qijun Zhang, and Zhiyao Xie. 2026. A New Benchmark for the Appropriate Evaluation of RTL Code Optimization. arXiv:2601.01765 [cs.AI] doi:10.48550/arXiv.2601.01765DOI
  101. [102]Zizhang Luo, Yansong Xu, Runlin Guo, et al. 2026. Clover: A Neural-Symbolic Agentic Harness with Stochastic Tree-of-Thoughts for Verified RTL Repair. arXiv preprint arXiv:2604.17288.arxiv.org/abs/2604.17288
  102. [103]Christian Lynsenstøen. 2026. SLO-Guard: Crash-Aware, Budget-Consistent Autotuning for SLO-Constrained LLM Serving. arXiv preprint arXiv:2604.17627.
  103. [104]Xing Ma, Yangjie Zhou, Wu Sun, Zihan Liu, Jingwen Leng, Yun Lin, Shixuan Sun, Minyi Guo, and Jin Song Dong. 2026. CuBridge: An LLM-Based Framework for Understanding and Reconstructing High-Performance Attention Kernels. arXiv:2605.05023 [cs.LG] doi:10.48550/arXiv.2605.05023DOI
  104. [105]Haohui Mai, Xiaoyan Guo, Xiangyun Ding, Daifeng Li, Qichu Yu, Chenzhun Guo, Cong Wang, Jiacheng Zhao, Christos Kozyrakis, and Binzhang Yuan. 2026. Ave: Guiding Agentic GPU Optimization Using Data-Flow Invariants. arXiv:2604.18616 [cs.DC] doi:10.48550/arXiv.2604.18616DOI
  105. [106]Ehud Malul, Yair Meidan, Dudu Mimran, Yuval Elovici, and Asaf Shabtai. 2024. GenKubeSec: LLM-Based Kubernetes Misconfiguration Detection, Localization, Reasoning, and Remediation. arXiv preprint arXiv:2405.19954.
  106. [107]Caetano Melone, Daniel Nichols, Konstantinos Parasyris, Todd Gamblin, and Harshitha Menon. 2025. LLMs as Packages of HPC Software. arXiv:2511.05626 [cs.SE] doi:10.48550/arXiv.2511.05626
  107. [108]Xupeng Miao, Gabriele Oliaro, Zhihao Zhang, Xinhao Cheng, Hongyi Jin, Tianqi Chen, and Zhihao Jia. 2023. Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systems. arXiv preprint arXiv:2312.15234.DOI
  108. [109]Xupeng Miao, Chunan Shi, Jiangfei Duan, Xiaoli Xi, Dahua Lin, Bin Cui, and Zhihao Jia. 2023. SpotServe: Serving Generative Large Language Models on Preemptible Instances. arXiv:2311.15566 https://arxiv.org/abs/2311.15566
  109. [110]Kaisa Miettinen. 1998. Nonlinear Multiobjective Optimization. International Series in Operations Research & Management Science, Vol. 12. Springer. doi:10.1007/978-1-4615-5563-6DOI
  110. [111]Ryo Mikasa, Shun ichiro Hayashi, Daichi Mukunoki, Tetsuya Hoshino, and Takahiro Katagiri. 2026. Improving HPC Code Generation Capability of LLMs via Online Reinforcement Learning with Real-Machine Benchmark Rewards. arXiv:2602.12049 [cs.LG] doi:10.48550/arXiv.2602.12049
  111. [112]Shervin Minaee, Tomas Mikolov, Narjes Nikzad, Meysam Chenaglu, Richard Socher, Xavier Amatriain, and Jianfeng Gao. 2024. Large language models: A survey. arXiv preprint arXiv:2402.06196 (2024).
  112. [113]Jean-Baptiste Mouret and Jeff Clune. 2015. Illuminating search spaces by mapping elites. arXiv:1504.04909 https://arxiv.org/abs/1504.04909
  113. [114]Kirill Nagaitsev, Luka Grbcic, Samuel Williams, and Costin Iancu. 2025. Optimizing PyTorch Inference with LLM-Based Multi-Agent Systems. arXiv preprint arXiv:2511.16964.arxiv.org/abs/2511.16964
  114. [115]Abinand Nallathambi, Christopher Knight, S. Thilagan Ganguly, et al. 2026. A3D: Agentic AI flow for autonomous Accelerator Design. arXiv preprint arXiv:2605.15237.arxiv.org/abs/2605.15237
  115. [116]Mahesh Natarajan, Xiaoye Li, and Weiqun Zhang. 2026. AstraAI: LLMs, Retrieval, and AST-Guided Assistance for HPC Codebases. arXiv:2603.27423 [cs.AI] doi:10.48550/arXiv.2603.27423DOI
  116. [117]Alexander Novikov, Ngân Vũ, Marvin Eisenberger, Emilien Dupont, Po-Sen Huang, Adam Zsolt Wagner, Sergey Shirobokov, Borisov Kozlovskii, Francisco J. R. Ruiz, Abbas Mehrabian, M. Pawan Kumar, Abigail See, Swarath Chaudhuri, George Holland, Alex Davies, Sebastian Nowozin, Pushmeet Kohli, and Matej Balog. 2025. AlphaEvolve: A coding agent for scientific and algorithmic discovery. arXiv preprint arXiv:2506.13131.arxiv.org/abs/2506.13131
  117. [118]Anne Ouyang, Simon Guo, Simran Arora, Alex L. Zhang, William Hu, Christopher Ré, and Azalia Mirhoseini. 2025. KernelBench: Can LLMs Write Efficient GPU Kernels? arXiv preprint arXiv:2502.10517.
  118. [119]Claus Pahl, Övğüm Can Sezen, and Florian Hofer. 2026. Artificial Intelligence for Infrastructure-as-Code—A Systematic Literature Review. Electronics 15, 4 (2026), 755. doi:10.3390/electronics15040755DOI
  119. [120]Jingyu Pan, Guanglei Zhou, Chen-Chia Chang, Isaac Jacobson, Jiang Hu, and Yiran Chen. 2025. A Survey of Research in Large Language Models for Electronic Design Automation. arXiv preprint arXiv:2501.09655.arxiv.org/abs/2501.09655
  120. [121]Zehua Pei, Hui-Ling Zhen, Mingxuan Yuan, Yu Huang, and Bei Yu. 2024. BetterV: Controlled Verilog Generation with Discriminative Guidance. arXiv:2402.03375 [cs.AI] doi:10.48550/arXiv.2402.03375
  121. [122]Huiyun Peng, Arjun Gupte, Nicholas John Eliopoulos, Chien Chou Ho, Rishi Mantri, Leo Deng, Wenxin Jiang, Yung-Hsiang Lu, Konstantin Läufer, George K. Thiruvathukal, and James C. Davis. 2024. Large Language Model for Energy-Efficient Code: Emerging Results and Future Directions. arXiv preprint arXiv:2410.09241.arxiv.org/abs/2410.09241
  122. [123]Laurent Perron and Vincent Furnon. 2024. OR-Tools. https://developers.google.com/optimization/
  123. [124]Scott Persall, Yang Gao, Shenyang Liu, and Liqiang Wang. 2026. Improving MPI Error Detection and Repair with Large Language Models and Bug References. arXiv preprint arXiv:2604.02398.
  124. [125]Nathaniel Pinckney, Christopher Batten, Mingjie Liu, Haoxin Ren, and Bruce Khailany. 2024. Revisiting VerilogEval: A Year of Improvements in Large-Language Models for Hardware Code Generation. arXiv preprint arXiv:2408.11053.arxiv.org/abs/2408.11053
  125. [126]Nathaniel Pinckney, Chenhui Deng, Chia-Tung Ho, Yun-Da Tsai, Mingjie Liu, Wenfei Zhou, BruceK Khailany, and Haoxin Ren. 2025. Comprehensive Verilog Design Problems: A Next-Generation Benchmark Dataset for Evaluating Large Language Models and Agents on RTL Design and Verification. arXiv:2506.14074 https://arxiv.org/abs/2506.14074
  126. [127]Heng Ping, Peiyu Zhang, Shixuan Li, et al. 2026. COEVO: Co-Evolutionary Framework for Joint Functional Correctness and PPA Optimization in LLM-Based RTL Generation. arXiv preprint arXiv:2604.15001.arxiv.org/abs/2604.15001
  127. [128]Heng Ping, Peiyu Zhang, Zhenkun Wang, Shixuan Li, Anzhe Cheng, Wei Yang, Paul Bogdan, and Shahin Nazarian. 2026. POET: Power-Oriented Evolutionary Tuning for LLM-Based RTL PPA Optimization. arXiv preprint arXiv:2603.19333.arxiv.org/abs/2603.19333
  128. [129]Peter Pikelbauer and Chunhua Liao. 2025. CompilerGPT: Leveraging Large Language Models for Analyzing and Acting on Compiler Optimization Reports. arXiv preprint arXiv:2506.06227.
  129. [130]Louis-Noël Pouchet. 2012. PolyBench/C: The Polyhedral Benchmark Suite. https://sourceforge.net/projects/polybench/ch/
  130. [131]Pratik Poudel, Yanzhao Wu, Sumit Jha, and Jason Liu. 2026. Discovering KV Cache Eviction Policies via LLM-Guided Program Evolution. arXiv preprint arXiv:2608.14555.arxiv.org/abs/2608.14555
  131. [132]Qiyou Qi, Yicheng Sui, Yufei Sun, et al. 2026. A Two-Stage GPU Kernel Tuner Combining Semantic Refactoring and Search-Based Optimization. arXiv preprint arXiv:2601.12698.
  132. [133]Anshu Raina, Peyman Raghazi, Yuankai Chen, Cheng Yao, Yao Fu, Vidushi Goyal, Devang Patel, Wen Xie, Zhenyu Gu, and Emad Barsoum. 2026. Beyond the Trial-and-Error Loop: Hybrid Projection and Automated Tuning for Distributed Training. arXiv preprint arXiv:2610.19423.arxiv.org/abs/2610.19423
  133. [134]Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He. 2020. DeepSpeed: System Optimizations Enable Training Deep Learning Models with Over 100 Billion Parameters. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD). ACM, New York, NY, USA. doi:10.1145/3394486.3406703DOI
  134. [135]Tara Saba, Zhiyang Chen, Jikai Jason Li, Anne Ouyang, Xujie Si, and Fan Long. 2026. CuTeGen: An LLM-Based Agentic Framework for Generation and Optimization of High-Performance GPU Kernels using CuTe. arXiv:2604.01489 [cs.LG] doi:10.48550/arXiv.2604.01489DOI
  135. [136]June Sallou, Luís Cruz, and Thomas Durieux. 2023. EnergiBridge: Empowering Software Sustainability through Cross-Platform Energy Measurement. arXiv:2312.13897 https://arxiv.org/abs/2312.13897
  136. [137]Dipankar Sarkar. 2026. The Correctness Illusion in LLM-Generated GPU Kernels. arXiv preprint arXiv:2606.20128.arxiv.org/abs/2606.20128
  137. [138]Nourin Shahin and Izzat Alsmadi. 2026. HPC-LLM: Practical Domain Adaptation and Retrieval-Augmented Generation for HPC Support. arXiv preprint arXiv:2605.16347.arxiv.org/abs/2605.16347
  138. [139]Yi Shan, Yixuan He, Zekai Shao, Kai Xu, and Siming Chen. 2026. NotebookRAG: Retrieving Multiple Notebooks to Augment the Generation of EDA Notebooks for Crowd-Wisdom. arXiv:2602.17215 [cs.HC] doi:10.48550/arXiv.2602.17215.DOI
  139. [140]Minghao Shao, Zeng Wang, Weimin Fu, Xiaolong Guo, Johann Knechtel, Ozgur Sinanoglu, Ramesh Karri, and Muhammad Shafique. 2026. Configuration Over Selection: Hyperparameter Sensitivity Exceeds Model Differences in Open-Source LLMs for RTL Generation. arXiv:2604.17102 [cs.AR] doi:10.48550/arXiv.2604.17102arxiv.org/abs/2604.17102
  140. [141]Zhihong Shao, Peiyang Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiang Ji, Baitao Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models. arXiv:2402.03300 https://arxiv.org/abs/2402.03300
  141. [142]Asankhya Sharma. 2025. OpenEvolve: An Open-Source Evolutionary Coding Agent. https://github.com/codelion/openevolve
  142. [143]Aasish Kumar Sharma and Julian Kunkel. 2025. Evaluating Large Language Models for Workload Mapping and Scheduling in Heterogeneous HPC Systems. arXiv preprint arXiv:2511.11612.DOI
  143. [144]Honghao Shi, Longkai Cheng, Wenli Wu, Yuhang Wang, Xuan Liu, Shaokai Nie, Weixv Wang, Xuebin Min, Chunlei Men, and Yonghua Lin. 2024. Enhancing Cluster Resilience: LLM-agent Based Autonomous Intelligent Cluster Diagnosis System and Evaluation Framework. arXiv preprint arXiv:2411.05349.arxiv.org/abs/2411.05349
  144. [145]Liang Shi, Zhengju Tang, Nan Zhang, Xiaotong Zhang, and Zhi Yang. 2025. A Survey on Employing Large Language Models for Text-to-SQL Tasks. Comput. Surveys 58, 2 (2025), 1–37. arXiv:2407.15186. doi:10.1145/3737873DOI
  145. [146]Masaki Shiraishi, Ikko Hamamura, Tatsuya Ishigaki, and Tadashi Kadokawa. 2026. A Model Context Protocol Server for Quantum Execution in Hybrid Quantum-HPC Environments. arXiv:2604.08318 [quant-ph] doi:10.48550/arXiv.2604.08318DOI
  146. [147]Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro. 2019. Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism. arXiv:1909.08053 https://arxiv.org/abs/1909.08053
  147. [148]Zijie Su, Muhammed Tawfiqful Islam, Mohammad Goudarzi, and Adel N. Toosi. 2026. LLM-Driven Intent-Based Privacy-Aware Orchestration Across the Cloud-Edge Continuum. arXiv:2602.16100 [cs.DC] doi:10.48550/arXiv.2602.16100DOI
  148. [149]Jicheng Sun, Huatong Song, Lisheng Huang, Jinhao Jiang, Ran Le, Zhihai Zhou, Jiemin Yao, He Wenyang Luo, Wayne Xin Zhao, Yang Song, Hongteng Xu, Tao Zhang, and Ji-Rong Wen. 2026. SCE-World: Building Software Engineering Agents in Docker-Free Environments. arXiv:2602.03419 [cs.SE] doi:10.48550/arXiv.2602.03419arxiv.org/abs/2602.03419
  149. [150]Wei Sun, Ting Wang, Xinran Tian, Wanshun Lan, Xuhan Feng, Haoyue Li, and Fan Wang. 2026. MetaKube: An Experience-Aware LLM Framework for Kubernetes Failure Diagnosis. arXiv preprint arXiv:2603.23580.arxiv.org/abs/2603.23580
  150. [151]Yan Tan, Xiangchen Meng, Zijun Jiang, and Yangdi Lyu. 2026. AutoVeriFix+: High-Correctness RTL Generation via Trace-Aware Causal Fix and Semantic Redundancy Pruning. arXiv:2603.11489 [cs.PL] doi:10.48550/arXiv.2603.11489DOI
  151. [152]Annabelle Sujun Tang, Christopher Priebe, Lianhui Qin, and Hadi Esmaeilzadeh. 2026. MOSAIC: More LLMs, Less Cost through Collaborative Reasoning for Compiler Optimizations. arXiv preprint arXiv:2602.01935.arxiv.org/abs/2602.01935
  152. [153]Muhammad Usman Tariq, Abhinav Jangda, Angelica Moreira, Madan Musuvathi, and Tyler Sorensen. 2025. PEAK: A Performance Engineering AI-Assistant for GPU Kernels Powered by Natural Language Transformations. arXiv preprint arXiv:2512.19018.arxiv.org/abs/2512.19018
  153. [154]The Verkor Team, Ravi Krishna, Suresh Krishna, and David Chin. 2026. Design Conductor 2.0: An agent builds a TurboQuant inference accelerator in 80 hours. arXiv:2605.05170 [cs.AR] doi:10.48550/arXiv.2605.05170DOI
  154. [155]Ali Tehrani, Yahya Emara, Essam Wissam, Wojciech Paluch, Waleed Atallah, Lukasz Dudziak, and Mohamed S. Abdelfattah. 2026. Fine-Tuning GPT-5 for GPU Kernel Generation. arXiv preprint arXiv:2602.11000.arxiv.org/abs/2602.11000
  155. [156]Shailja Thakur, Balegh Ahmad, Zhenxing Fan, Hammond Pearce, Benjamin Tan, Ramesh Karri, Brendan Dolan-Gavitt, and Siddharth Garg. 2022. Benchmarking Large Language Models for Automated Verilog RTL Code Generation. arXiv:2212.11140 [cs.PL] doi:10.48550/arXiv.2212.11140arxiv.org/abs/2212.11140
  156. [157]Kiran Thorat, Jiahui Zhao, Yaotian Liu, Amit Hasan, Hongwu Peng, Xi Xie, Bin Lei, and Caiwen Ding. 2025. LLM-VeriPPA: Power, Performance, and Area Optimization Aware Verilog Code Generation with Large Language Models. arXiv preprint arXiv:2510.15899.arxiv.org/abs/2510.15899
  157. [158]Martin Tomasovic and Lukas Sekanina. 2026. Multi-Objective Coevolution of Prompts and Templates for Circuit Approximation. arXiv:2606.13089 [cs.NE] doi:10.48550/arXiv.2606.13089arxiv.org/abs/2606.13089
  158. [159]John R. Tramm, Andrew R. Siegel, Tanzima Islam, and Martin Schulz. 2014. XSBench – The Development and Verification of a Performance Abstraction for Monte Carlo Reactor Analysis. In PHYSOR 2014 – The Role of Reactor Physics toward a Sustainable Future.
  159. [160]Nguyen Phuc Tran, Brigitte Jaumard, Oscar Delgado, Tristan Glatard, Karthikeyan Premkumar, and Kun Ni. 2026. LLM-Augmented Knowledge Base Construction for Root Cause Analysis. arXiv preprint arXiv:2604.06171.DOI
  160. [161]Yun-Da Tsai, Mingjie Liu, and Haoxing Ren. 2023. RTLFixer: Automatically Fixing RTL Syntax Errors with Large Language Models. arXiv:2311.16543 [cs.AR] doi:10.48550/arXiv.2311.16543DOI
  161. [162]Masaru Ueno and Tetsuya Uchiumi. 2024. Migrating Existing Container Workload to Kubernetes: LLM-Based Approach and Evaluation. arXiv preprint arXiv:2408.11428.arxiv.org/abs/2408.11428
  162. [163]Yoga Sri Varshan Varadharajan, Bodun Hu, Saurabh Agarwal, and Aditya Akella. 2026. CUCo: An Agentic Framework for Compute and Communication Co-design. arXiv:2603.02376 [cs.DC] doi:10.48550/arXiv.2603.02376DOI
  163. [164]Haonan Wang, Jiaxiang Liu, Yurong Liu, Austin Senna Wijaya, Tianle Zhou, Eden Wu, Yijia Chen, Wanting You, Reya Vir, Daniela Pinto, Grace Fan, Yusen Zhang, Juliana Freire, and Eugene Wu. 2026. LakeQA: An Exploratory QA Benchmark over a Million-Scale Data Lake. arXiv:2606.10460 https://arxiv.org/abs/2606.10460
  164. [165]Han Wang, Jintao Zhang, Kai Jiang, Haoxu Wang, Jianfei Chen, and Jun Zhu. 2026. KernelBenchX: A Comprehensive Benchmark for Evaluating LLM-Generated GPU Kernels. arXiv preprint arXiv:2605.04956.arxiv.org/abs/2605.04956
  165. [166]Jianghui Wang, Vinay Joshi, Saptarshi Majumder, et al. 2025. Geak: Introducing Triton Kernel AI Agent & Evaluation Benchmarks. arXiv preprint arXiv:2507.23194.arxiv.org/abs/2507.23194
  166. [167]Xingyao Wang, Boxuan Li, Yufan Song, Frank F. Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, Hoang H. Tran, Fuqiang Li, Ren Ma, Mingzhang Zheng, Bill Qian, Yanjun Shao, Niklas Muennighoff, Yizhe Zhang, Binyuan Hui, Junyang Lin, Robert Brennan, Hao Peng, Heng Ji, and Graham Neubig. 2024. OpenHands: An Open Platform for AI Software Developers as Generalist Agents. arXiv:2407.16741 https://arxiv.org/abs/2407.16741
  167. [168]Yiting Wang, Chenhui Deng, Chia-Tung Ho, Yanqing Zhang, Zhuo Feng, Cunxi Yu, Ang Li, Gang Qu, and Brueck Khailany. 2026. AUTOGATE: Automated Clock Gating via Toggling-Aware RTL Rewriting. arXiv:2606.17461 [cs.AR] doi:10.48550/arXiv.2606.17461DOI
  168. [169]Yuanli Wang, Yaoyao Qian, Yue Zhang, Hanhan Zhou, Jindan Huang, Tianfu Fu, Qiuyang Mang, Huanzhi Mao, Wenhao Chai, Wendong Fan, and Liqiang Jing. 2026. DeployBench: Benchmarking LLM Agents for Research Artifact Deployment. arXiv preprint arXiv:2606.05238.arxiv.org/abs/2606.05238
  169. [170]Zerui Wang, Qinghao Hu, Ana Klimovic, Tianwei Zhang, Yonggang Wen, Peng Sun, and Dahua Lin. 2025. Semantic-Aware Scheduling for GPU Clusters with Large Language Models. arXiv preprint arXiv:2510.03334.arxiv.org/abs/2510.03334
  170. [171]Zhongzhen Wen, Shudi Shao, Zhong Li, Yu Ge, Tongtong Xu, Yuanyi Lin, and Tian Zhang. 2026. AscendCraft: Automatic Ascend NPU Kernel Generation via DSL-Guided Transcompilation. arXiv:2601.22760 [cs.DC] doi:10.48550/arXiv.2601.22760DOI
  171. [172]Zhongzhen Wen, Yinghui Zhang, Zhong Li, Zhongxin Liu, Linna Xie, and Tian Zhang. 2025. MultiKernelBench: A Multi-Platform Benchmark for Kernel Generation. arXiv preprint arXiv:2507.17773.arxiv.org/abs/2507.17773
  172. [173]Otto White, Marcel Wengenlander, Britannio Jarrett, Xijin Zhou, Yanda Tao, Pedro Silvestre, Guo Li, Huanzhou Zhu, Lluís Vilanova, and Peter Pietzuch. 2026. Scepsy: Serving Agentic Workflows Using Aggregate LLM Pipelines. arXiv preprint arXiv:2604.15186.
  173. [174]Nina Wiedemann, Quentin Leboute, Michael Paulitsch, Diana Wofk, and Benjamin Ummenhofer. 2026. KernelFoundry: Hardware-aware evolutionary GPU kernel optimization. arXiv:2603.12440 [cs.DC] doi:10.48550/arXiv.2603.12440DOI
  174. [175]Austin Senna Wijaya, Jiaxiang Liu, Haonan Wang, and Eugene Wu. 2026. SANA: What Matters for QA Agents over Massive Data Lakes? arXiv preprint arXiv:2606.13904.arxiv.org/abs/2606.13904
  175. [176]Chunqiu Steven Xia, Yinlin Deng, and Liming Zhang. 2024. Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM. arXiv:2403.19114 https://arxiv.org/abs/2403.19114
  176. [177]Yong Xiang, Charley Peter Chen, Liyi Zeng, Wei Yin, Xin Liu, Hu Li, and Wei Xu. 2025. Simplifying Root Cause Analysis in Kubernetes with StateGraph and LLM. arXiv preprint arXiv:2506.02490.arxiv.org/abs/2506.02490
  177. [178]Shaliang Xing, Yiyan Zhai, Alexander Jiang, Yixin Dong, Yong Wu, Zihai Yao, Charlie Ruan, Yingyi Huang, Yineng Zhang, Liangsheng Yin, Aksara Bayyapu, Luis Ceze, and Tianqi Chen. 2026. FlashInfer-Bench: Building the Virtuous Cycle for AI-driven LLM Systems. arXiv preprint arXiv:2601.00227.arxiv.org/abs/2601.00227
  178. [179]Jing Xiong, Qi Han, Chenchen Ding, He Xiao, Zunhai Su, Chaofan Tao, and Ngai Wong. 2026. CktFormalizer: Autoformalization of Natural Language into Circuit Representations. arXiv:2605.07782 [cs.CL] doi:10.48550/arXiv.2605.07782DOI
  179. [180]Qi Xiong, Renzhi Chen, Bowei Wang, Yuqing Xiong, Libo Huang, and Lei Wang. 2026. EstRTL: Functional Estimation Guided RTL Code Generation. arXiv:2606.09867 [cs.AR] doi:10.48550/arXiv.2606.09867DOI
  180. [181]Bing Xu, Terry Chen, Fengzhe Zhou, Tianqi Chen, Yangqing Jia, Vinod Grover, Haicheng Wu, Wei Liu, Craig Wittenbrink, Wen mei Hwu, Roger Bringmann, Ming-Yu Liu, Luis Ceze, Michael Lightstone, and Humphrey Shi. 2026. VibeTensor: System Software for Deep Learning, Fully Generated by AI Agents. arXiv:2601.16238 [cs.SE] doi:10.48550/arXiv.2601.16238DOI
  181. [182]Tianhao Xu, Yiming Liu, Xianglong Lu, Yijia Zhao, Xuting Zhou, Aichen Feng, Yiyi Chen, Yi Shen, Qin Zhou, Xumeng Chen, Ilya Sherstyuk, Haorui Li, Rishi Thakkar, Ben Hamm, Yuanzhe Li, Xue Huang, Wenpeng Wu, Anish Shanbhag, Harry Kim, Chuan Chen, and Junjie Lai. 2026. AIConfigurator: Lightning-Fast Configuration Optimization for Multi-Framework LLM Serving. arXiv preprint arXiv:2601.06288.arxiv.org/abs/2601.06288
  182. [183]Guang Yang, Wei Zheng, Dong Liang, Peng Hu, Yukui Yang, Shaohua Peng, Zhenghan Li, Jiahui Feng, Xiao Wei, Kexin Sun, Deyuan Ma, Haotian Cheng, Yiheng Shen, Xiang Chen, Xing Hu, Terry Yue Zhuo, and David Lo. 2026. Large Language Model for Verilog Code Generation: Literature Review and the Road Ahead. Comput. Surveys 58, 16 (2026), 1–35. arXiv:2512.00020. doi:10.1145/3841636DOI
  183. [184]Yitao Yang, Yangtao Deng, Yifan Xiong, Baochun Li, Hong Xu, and Peng Cheng. 2025. TSGuard: Automated User-Centric Incident Diagnosis for AI Workloads in the Cloud. arXiv preprint arXiv:2506.01481.arxiv.org/abs/2506.01481
  184. [185]Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2022. ReAct: Synergizing Reasoning and Acting in Language Models. arXiv:2210.03629 https://arxiv.org/abs/2210.03629
  185. [186]Shuo Yin, Yihe Wang, Lancheng Zou, Xufeng Yao, Tinghuan Chen, Chen Bai, Zhengrong Wang, Tsung-Yi Ho, and Bei Yu. 2026. CPPL: A Circuit Prompt Programming Language. arXiv:2605.17892 [cs.AR] doi:10.48550/arXiv.2605.17892DOI
  186. [187]Sharareh Younesian, Wenwen Ouyang, Sina Rafati, Mehdi Rezagholizadeh, Sharon Zhou, Ji Liu, Yue Liu, Yuchen Yang, Hao Li, Ziqiong Liu, Dong Li, Vikram Appia, Zhenyu Gu, and Emad Barsoum. 2026. AgentKernelArena: Generalization-Aware Benchmarking of GPU Kernel Optimization Agents. arXiv:2605.16819 [cs.CL] doi:10.48550/arXiv.2605.16819DOI
  187. [188]Cunxi Yu and Haoxing Ren. 2026. Autonomous Evolution of EDA Tools: Multi-Agent Self-Evolved ABC. arXiv:2604.15082 [cs.AR] doi:10.48550/arXiv.2604.15082arxiv.org/abs/2604.15082
  188. [189]Xi Yu, Dmitrii Torbunov, Soumyajit Mandal, and Yihui Ren. 2026. AutoSizer: Automatic Sizing of Analog and Mixed-Signal Circuits via Large Language Model (LLM) Agents. arXiv preprint arXiv:2602.02849.arxiv.org/abs/2602.02849
  189. [190]Yang Yu, Peiyu Zang, Chi Hsu Tsai, Haiming Wu, Yixin Shen, Jialing Zhang, Haoyu Wang, Zhiyu Xiao, Jingze Shi, Yuyu Luo, Wentao Zhang, Chunlei Men, Guang Liu, and Yonghua Lin. 2026. Towards Automated Kernel Generation in the Era of LLMs. arXiv preprint arXiv:2601.15727.arxiv.org/abs/2601.15727
  190. [191]Patrick Yuebaoton, Siddharth Garg, and Chinmay Hedge. 2026. Exploring the Agentic Frontier of Verilog Code Generation. arXiv:2603.19347 [cs.AR] doi:10.48550/arXiv.2603.19347
  191. [192]Amir Zandieh, Majid Daliri, Majid Hadian, and Vahab Mirrokni. 2025. TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate. arXiv:2504.19874 https://arxiv.org/abs/2504.19874
  192. [193]Hong Zhang, Barry Smith, Satish Balay, Le Chen, Murat Keceli, Lois Curfman McInnes, and Junchao Zhang. 2026. An Agentic Evaluation Framework for AI-Generated Scientific Code in PETSc. arXiv:2603.15976 [cs.AI] doi:10.48550/arXiv.2603.15976DOI
  193. [194]Lingzhe Zhang, Tong Jia, Mengxi Jia, Yifan Wu, Aiwei Liu, Yong Yang, Zhonghai Wu, Xuming Hu, Philip S. Yu, and Ying Li. 2025. A Survey of AIOps in the Era of Large Language Models. Comput. Surveys 58, 2 (2025), 1–35. arXiv:2507.12472. doi:10.1145/3746635DOI
  194. [195]Yining Zhang, Mingyang Yi, Chen Wang, et al. 2026. HTAM: Hierarchical Transition-Attended Memory for Operator Optimization. arXiv preprint arXiv:2605.29734.DOI
  195. [196]Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian-Yun Nie, and Ji-Rong Wen. 2023. A Survey of Large Language Models. arXiv:2303.18223 https://arxiv.org/abs/2303.18223
  196. [197]Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E. Gonzalez, Clark Barrett, and Ying Sheng. 2023. SGLang: Efficient Execution of Structured Language Model Programs. arXiv:2312.07104 https://arxiv.org/abs/2312.07104
  197. [198]Yimin Zhong, Shengyu Liu, Junda Chen, Jianbo Hu, Yibo Zhu, Xuanzhe Liu, Xin Jin, and Hao Zhang. 2024. DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving. arXiv:2401.09670 https://arxiv.org/abs/2401.09670
  198. [199]Jiasheng Zhou, Longbin Zeng, Clavis Chen, Ruiming Lu, Qinwei Yang, Leyi Ye, Ray Ying, and Key Zhang. 2026. ARGUS: Production-Scale Tracing and Performance Diagnosis for over 10,000-GPU Clusters. arXiv preprint arXiv:2606.20374.arxiv.org/abs/2606.20374
  199. [200]Jiace Zhu, Wentao Chen, Qi Fan, Zhixing Ren, Junying Wu, Xing Zhe Chai, Chotiwit Rungreangwuthin, Yehan Ma, and An Zou. 2026. CUDA­Bench: Benchmarking LLMs for Text-to-CUDA Generation. arXiv:2603.02236 [cs.LG] doi:10.48550/arXiv.2603.02236DOI
  200. [201]Xingze Zou, Jing Wang, Yuhua Zheng, Xueyi Chen, Haolei Bai, Lingcheng Kong, Syed A. R. Abu-Bakar, Zhaode Wang, Chengfei Lv, Haoji Hu, and Huan Wang. 2026. MobileKernelBench: Can LLMs Write Efficient Kernels for Mobile Devices? arXiv:2603.11935 [cs.LG] doi:10.48550/arXiv.2603.11935DOI

Paper details

vlnc:W_2p5c256f
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents