betaFeedback

WorksComputer Science

CLEAR: Structured Specifications for Controllable Long-Horizon ML Research Engineering

Abstract

Large language model (LLM) agents are increasingly being used to automate machine learning (ML) research workflows, ranging from code development and experiment execution to research reproduction and scientific discovery. Recent systems demonstrate that LLMs can perform substantial portions of ML research engineering with limited human intervention. These advances suggest a shift from automating individual research tasks toward automating long-horizon workflows that transform research artifacts into executable ML systems. However, ML research engineering remains substantially more difficult than isolated code generation. A research artifact typically specifies a system only implicitly through a mixture of natural-language descriptions, equations, pseudo-code, implementation details, datasets, and evaluation protocols. An agent must first infer the intended computational structure, translate it into executable components, coordinate their dependencies, and repeatedly diagnose and repair failures during execution. Consequently, the quality of the final system depends not only on code generation, but also on whether the agent can maintain a consistent representation of what should be implemented throughout this long-horizon process.

CCS Concepts • Computing methodologies → Planning and scheduling; Multi-agent systems.

This work is licensed under a Creative Commons Attribution 4.0 International License.

Introduction

Large language model (LLM) agents are increasingly being used to automate machine learning (ML) research workflows, ranging from code development and experiment execution to research reproduction and scientific discovery [7, 11, 20, 26]. Recent systems demonstrate that LLMs can perform substantial portions of ML research engineering with limited human intervention [8, 13, 17, 27]. These advances suggest a shift from automating individual research tasks toward automating long-horizon workflows that transform research artifacts into executable ML systems.

However, ML research engineering remains substantially more difficult than isolated code generation. A research artifact typically specifies a system only implicitly through a mixture of natural-language descriptions, equations, pseudo-code, implementation details, datasets, and evaluation protocols. An agent must first infer the intended computational structure, translate it into executable components, coordinate their dependencies, and repeatedly diagnose and repair failures during execution. Consequently, the quality of the final system depends not only on code generation, but also on whether the agent can maintain a consistent representation of what should be implemented throughout this long-horizon process. The Missing Layer: Structured Execution Specifications. Existing systems largely operate without such an explicit intermediate representation. They directly transform heterogeneous research artifacts into plans, code, and execution traces within a tightly coupled agent loop. This design creates a fundamental problem: research artifacts are not executable specifications. Their contents often leave implicit implementation assumptions, while different parts of a paper may specify model architecture, data processing, training procedures, and evaluation criteria at different levels of abstraction. When these ambiguities are resolved implicitly during generation, errors introduced during early interpretation can become embedded in downstream code and execution states. Once failures occur, existing systems commonly recover by regenerating large portions of the workflow, further increasing computational cost and making error recovery unstable. We identify three consequences of this missing intermediate representation. First, representation ambiguity makes it difficult to distinguish what the research artifact explicitly specifies from what the agent infers during implementation. Second, execution coupling entangles interpretation, planning, coding, and debugging, allowing early mistakes to propagate across dependent components. Third, global repair forces the system to repeatedly regenerate already-correct parts of the workflow when only a localized component has failed. These issues reveal a common underlying limitation: existing agents lack a persistent, inspectable state that separates what should be implemented from how it is implemented and repaired.

CLEAR: Specification-Centered and Controllable Automation. To address this limitation, we propose CLEAR, a Controllable Long-horizon Engineering system for Automating ML Research workflows. CLEAR introduces a structured specification layer between research artifacts and executable implementations. Instead of using papers and other artifacts as direct prompts for code generation, CLEAR first converts them into structured and human verifiable execution specifications that explicitly describe implementation units, dependencies, interfaces, execution requirements, and evaluation procedures. These specifications serve as intermediate states that can be referenced at any time throughout the entire workflow, allowing downstream generation and repair to operate on localized, well-defined objectives rather than repeatedly interpreting the original artifacts.

This intermediate representation also enables two forms of controllability that are difficult to achieve in end-to-end agent loops. First, CLEAR performs specification-conditioned model routing: large models are used for global interpretation, planning, and integration, while smaller models handle specification-bounded code generation and localized repair. Because each generation task is constrained by an explicit specification, the number of smaller models used in generation can be allocated according to the task complexity. Second, CLEAR introduces a lifecycle-oriented execution harness that treats research engineering as a sequence of observable workflow states rather than an opaque generation loop. The harness tracks the specification, generation, execution, failure, and repair states, enabling targeted intervention, runtime observability, and cost-aware control.

As illustrated in Figure 1, CLEAR therefore follows a specification-centered workflow consisting of three stages. (1) Document-to-specification parsing converts heterogeneous research artifacts into structured execution specifications that expose implementation requirements and dependencies. (2) Specification-conditioned generation and repair decomposes the workflow into specification-bounded units and assigns number of smaller coding models according to the complexity of each unit, enabling localized generation and recovery. (3) Lifecycle-oriented orchestration maintains the state of the workflow during execution, monitors failures and costs, and triggers targeted repair instead of indiscriminate end-to-end regeneration. Together, these components transform ML research engineering from a tightly coupled generation loop into a controllable sequence of specification-grounded state transitions.

Figure 1. Overview of CLEAR. Structured execution specifications bridge heterogeneous research artifacts and executable implementations, enabling specification-conditioned generation, specification-conditioned repair, and lifecycle-oriented orchestration for long-horizon ML research engineering.

The central design principle of CLEAR is therefore to introduce an explicit intermediate representation between research understanding and executable realization. This separation provides a common foundation for both reliability and efficiency: specifications reduce ambiguity and localize failures, while specification-bounded tasks enable selective model amount allocation and targeted repair. Rather than increasing model size to compensate for increasingly complex workflows, CLEAR uses structured intermediate states to control where model capacity and computation are needed.

Our contributions are summarized as follows:

  1. We identify the lack of an explicit intermediate representation as a fundamental bottleneck in long-horizon ML research engineering. We formulate structured execution specifications as a persistent state that separates research interpretation from executable realization and enables localized downstream control.

  1. We propose CLEAR, a specification-centered framework that converts heterogeneous research artifacts into human verifiable execution specifications and uses them to support specification-conditioned generation, and specification-conditioned repair.

  1. We develop a lifecycle-oriented execution harness that provides fine-grained control and observability over specification, generation, execution, failure, and repair states, enabling cost-aware intervention throughout long-horizon research workflows.

  1. Extensive experiments on PaperBench and MLE-Bench Lite show that CLEAR achieves a favorable performance–efficiency trade-off. On PaperBench, CLEAR obtains a score of 29.87 at an average cost of $7.91 per paper, reducing cost by 35.16% relative to AiScientist while sacrificing only 3.86 points in performance. On MLE-Bench Lite, CLEAR achieves 77.27% Any Medal while substantially reducing repair iterations on representative tasks.

Task Formulation

Problem Definition. We formulate ML research engineering as the problem of transforming heterogeneous research artifacts into executable and evaluable computational artifacts through a sequence of interpretation, implementation, and execution decisions. Unlike conventional code generation, the input to ML research engineering may contain natural-language descriptions, equations, pseudo-code, implementation details, datasets, and evaluation protocols, which jointly specify the intended system at different levels of abstraction.

Formally, given a task type TT, a research artifact PP, optional task data DD, and user instructions SS, an agent system A\mathcal{A} produces an executable artifact CC:

C=A(T,P,D,S).(1)C = \mathcal{A}(T, P, D, S). \tag*{(1)}

The generated artifact CC may include a runnable code repository, experiment pipeline, trained model, evaluation results, or valid competition submission, depending on the task type.

A valid solution must satisfy two requirements: (i) executability, meaning that the artifact can be successfully executed or submitted under the task environment; and (ii) task performance, measured by a task-specific evaluation function. We therefore formulate ML research engineering as the following constrained optimization problem:

C∗=arg⁡max⁡C∈CT QT(C),s.t.Exec⁡T(C)=1,(2)C^* = \underset{C \in\mathcal{C}_T}{\arg\max}\ Q_T(C), \qquad \text{s.t.}\quad\operatorname{Exec}_T(C) = 1, \tag*{(2)}

where CT\mathcal{C}_T denotes the artifact space for task type TT, QT(⋅)Q_T(\cdot) is the task-specific evaluation metric, and Exec⁡T(C)\operatorname{Exec}_T(C) indicates whether CC can be executed or submitted successfully.

Task Instantiations. In this work, we consider two representative task types:

  1. Replicate AI Research (RAR), where the input is an AI research paper PrP_r and the goal is to construct a runnable reproduction artifact:

Cr=A(RAR,Pr,∅,Sr).(3)C_r = \mathcal{A}(\mathrm{RAR}, P_r, \varnothing, S_r). \tag*{(3)}
  1. Machine Learning Engineering (MLE), where the input consists of a task description PmP_m and task data DmD_m, and the goal is to construct a runnable ML pipeline or a valid competition submission:

Cm=A(MLE,Pm,Dm,Sm).(4)C_m = \mathcal{A}(\mathrm{MLE}, P_m, D_m, S_m). \tag*{(4)}

Although RAR and MLE differ in their input artifacts and evaluation protocols, they share the same underlying structure: both require an agent to interpret heterogeneous task information, translate it into implementation decisions, and iteratively construct and execute a multi-step computational workflow. This shared structure makes them suitable settings for studying long-horizon ML research engineering.

To capture this common structure, we distinguish the research artifact provided as input from the execution specification required for implementation. A research artifact describes the intended research or task objectives, often implicitly and across heterogeneous formats. In contrast, an execution specification provides an explicit representation of what needs to be implemented and how its components are related. In CLEAR, this specification consists of two complementary views: a machine-oriented specification P′P' and a human-readable method summary S′S'. We therefore formulate the workflow as

(P,D,S)→Aspec(P′,S′)→AexecC,(5)(P, D, S) \xrightarrow{\mathcal{A}_{\mathrm{spec}}} (P', S') \xrightarrow{\mathcal{A}_{\mathrm{exec}}} C, \tag*{(5)}

where (P′,S′)(P', S') serves as the intermediate representation between research understanding and executable realization.

This formulation separates what should be implemented, represented by (P′,S′)(P', S'), from how it is realized, represented by Aexec\mathcal{A}_{\mathrm{exec}}.

Our Method

Given the formulation above, CLEAR operationalizes (P′,S′)(P', S') as an intermediate state throughout long-horizon ML research engineering. The system follows a specification-first design principle: research artifacts are first transformed into structured execution specifications, after which planning, generation, execution, and repair are performed with respect to these specifications. This decomposition replaces the repeated interpretation of raw research artifacts with specification-bounded implementation and repair decisions. As shown in Figure 1, CLEAR consists of three stages:

  1. Document-to-Specification Parsing, where the system receives the task type TT, research artifact PP, optional task data DD, and optional user instruction SS, and transforms these heterogeneous inputs into the structured execution specification (P′,S′)(P', S').

The machine-oriented specification P′P' captures implementation requirements, component dependencies, interfaces, execution constraints, and evaluation procedures, while the human-readable summary S′S' provides a concise abstraction of the intended research method for downstream planning and coordination.

(2) Specification-Conditioned Generation, where a strong planning model uses (P′P', S′S') to decompose the workflow into localized, specification-bounded subtasks and coordinates lower-cost coding models to generate the initial repository C0C_0. The explicit specification allows the number of lower-cost coding workers to be adjusted according to task complexity and enables independent subtasks to be generated in parallel while preserving their specified dependencies and interfaces.

(3) Lifecycle-Controlled Execution and Repair, where the generated repository C0C_0 is executed and evaluated under the task environment. When failures occur, CLEAR identifies affected subtasks and performs targeted, specification-conditioned iterative repair rather than regenerating the entire workflow. The process iterates through execution, diagnosis, and repair until a runnable and evaluable artifact CC is obtained.

A visual lifecycle harness serves as the control layer across all three stages. Rather than treating LLM agents as a single black-box generator, the harness explicitly controls when and how agents parse, plan, generate, execute, and repair. It exposes intermediate specifications and execution artifacts for inspection and user intervention, while tracking runtime, token usage, and estimated cost. This lifecycle-oriented control makes the workflow observable and steerable, while the specification-centered decomposition enables CLEAR to localize failures, allocate the number of lower-cost coding model selectively, and reduce unnecessary regeneration throughout long-horizon ML research engineering.

Document-to-Specification Parsing

The first stage of CLEAR constructs the execution specification that serves as the intermediate state for downstream implementation. Given a task type TT, research artifact PP, optional task data DD, and user instruction SS, CLEAR transforms these heterogeneous inputs into two complementary views: a machine-oriented structured specification P′P' and a human-readable method summary S′S'. Rather than providing another free-form description of the input, (P′P', S′S') explicitly captures what needs to be implemented and how the resulting system should be executed and evaluated.

We formalize the parsing process as

(P′,S′)=g(T,P,D,S),(6)(P', S') = g(T, P, D, S), \tag*{(6)}

where gg denotes the document-to-specification process. The structured specification P′P' records implementation-relevant requirements, including algorithms, inputs and outputs, dependencies, parameters, interfaces, evaluation metrics, and submission constraints. The summary S′S' provides a concise and human-readable view of the inferred research method and serves as an inspection interface for users. Both views are derived from the same underlying evidence and jointly represent the execution specification used throughout the subsequent workflow.

As illustrated in Figure 2, CLEAR constructs (P′P', S′S') through three processing steps, followed by an optional review step:

Figure 2. Document-to-specification parsing in CLEAR. Heterogeneous research artifacts are parsed into traceable evidence units and synthesized into complementary machine-oriented and human-readable specification views (P′P', S′S'). An optional review step provides a control point for resolving ambiguous requirements before downstream code generation.

(1) Visual document parsing, which extracts text, page structure, formulas, tables, figures, captions, and algorithmic descriptions from the input research artifact, preserving both semantic content and document structure.

(2) Traceable evidence packaging, which identifies implementation-relevant evidence and organizes it into traceable evidence units. Each unit retains its source context, allowing specification entries to remain grounded in the original research artifact rather than being generated as unsupported summaries.

(3) Structured specification construction, where the planning model MpM_p synthesizes the evidence units into the structured specification P′P' and the summary of the corresponding method S′S'. The resulting representation explicitly organizes the implementation requirements, component relationships, execution constraints, and evaluation procedures.

(4) Specification review, where users can inspect S′S' and, when necessary, correct or refine the inferred understanding before code generation. This optional intervention provides an explicit control point for resolving ambiguous or incorrectly interpreted requirements before they propagate to downstream implementation.

The resulting (P′P', S′S') therefore transforms implicit and heterogeneous research understanding into an explicit, traceable, and reviewable execution state. This separation allows downstream planning, generation, and repair to operate on specification-bounded units rather than repeatedly interpreting the original research artifacts, reducing ambiguity and limiting the propagation of early-stage interpretation errors.

Specification-Conditioned Generation

Given the execution specification (P′P', S′S'), CLEAR generates the initial repository C0C_0 through specification-bounded decomposition and heterogeneous model collaboration. The key idea is to separate global reasoning from localized implementation: the stronger planning model MpM_p handles algorithm interpretation, project-level decomposition, and cross-component integration, while lower-cost coding models McM_c implement individual subtasks whose requirements and interfaces are explicitly specified by (P′,S′)(P', S').

The generation process consists of three steps:

  1. Specification-guided planning. The planning model MpM_p jointly reads P′P' and S′S' and decomposes the implementation into nn specification-bounded subtasks I1:n={I1,…,In}I_{1:n} = \{I_1, \ldots, I_n\}. Each subtask specifies its implementation requirements, relevant files, dependencies, interfaces, and expected outputs, so that localized workers can operate without repeatedly interpreting the original research artifact.

  1. Parallel local generation. For each subtask IjI_j, a lower-cost coding worker McM_c generates the required code under the specified file paths and interfaces. Independent subtasks can be processed in parallel, allowing the number of low-cost coding workers to be dynamically allocated for local implementation without repeatedly invoking the stronger planning model.

  1. Specification-aware integration. The planning model MpM_p collects the generated components and verifies their dependencies, interfaces, file structure, and cross-component consistency against (P′,S′)(P', S'). It resolves conflicts when necessary and integrates the validated components into the initial repository C0C_0.

We formalize the generation stage as

C0=G(P′,S′;Mp,Mc,n),(7)C_0 = G(P', S'; M_p, M_c, n), \tag*{(7)}

where GG denotes the specification-conditioned generation process, MpM_p performs global planning and integration, McM_c performs localized implementation. Here, nn controls the degree of parallel decomposition: more complex workflows with more independent implementation units activate more coding workers, whereas simpler workflows require fewer workers.

This design turns long-horizon implementation into a collection of specification-bounded local decisions while preserving global coordination. The stronger model is therefore reserved for tasks that require project-level reasoning, whereas lower-cost models handle routine and parallelizable coding work, improving the performance–cost trade-off of generation.

Lifecycle-Controlled Execution and Repair

The generated repository C0C_0 is not assumed to be correct. CLEAR therefore treats execution and repair as a controlled lifecycle rather than a single post-generation verification step. The lifecycle harness repeatedly executes the repository, observes execution feedback, localizes failures, and invokes targeted repair until a runnable and evaluable artifact CC is obtained. Specifically, the process consists of three steps:

  1. Execution and diagnosis. CLEAR executes C0C_0 in the task environment and collects runtime output, error messages, test results, and task-specific evaluation feedback. The harness uses these observations to identify the affected implementation components and determine whether further repair is required.

  1. Specification-conditioned repair. When a failure is detected, the coding model McM_c receives the relevant execution feedback together with the corresponding specification-bounded subtask and modifies only the affected files or components. Unlike full repository regeneration, this localized repair preserves already validated components and avoids unnecessary model computation.

  1. Task-specific validation. The repaired repository is re-executed and evaluated under the task requirements. For MLE tasks, this includes evaluation on the provided task data DD and submission-specific checks, such as a competition submission validation. The lifecycle continues until the artifact satisfies the required execution and evaluation conditions or reaches the predefined intervention limit.

We formalize the execution and repair process as

C=E(T,D,C0;Mc),(8)C = E(T, D, C_0; M_c), \tag*{(8)}

where EE denotes the lifecycle-controlled execution and repair process. The resulting artifact CC is therefore produced through an iterative loop:

C0→executeO1→diagnoseIj→repairC1→execute⋯→C,(9)C_0 \xrightarrow{\text{execute}} O_1 \xrightarrow{\text{diagnose}} I_j \xrightarrow{\text{repair}} C_1 \xrightarrow{\text{execute}} \cdots\to C, \tag*{(9)}

where OtO_t denotes the execution and evaluation feedback at iteration tt. By localizing repair to specification-bounded components and using execution feedback to guide subsequent actions, CLEAR avoids unnecessary full-workflow regeneration and preserves successful intermediate components. The lifecycle harness further provides runtime observability and control over the generation, execution, and repair states, enabling cost-aware intervention throughout the long-horizon workflow.

Lifecycle-Oriented Orchestration

Long-horizon ML research engineering involves multiple dependent stages, from document parsing and user review to generation, execution, and repair. Failures or costly decisions can occur at any stage, making a monolithic agent difficult to monitor, interrupt, and recover. CLEAR therefore treats the workflow as a sequence of observable and controllable lifecycle states, managed by a visual execution harness.

Interruptible and recoverable control. CLEAR maintains task states, inputs, and intermediate artifacts in a persistent priority queue. This enables concurrent task scheduling as well as recovery from service interruptions without restarting completed stages. Users can pause, stop, or resume tasks through the frontend, with control actions applied at safe lifecycle boundaries. The harness also records failures, timeouts, interruptions, and recovery states for subsequent diagnosis.

Human-in-the-loop checkpoints. CLEAR introduces an explicit checkpoint after document-to-specification parsing, where the generated method summary S′S' is presented to the user before code generation begins. Users can inspect and revise the parsed understanding, correcting missing assumptions, parameters, or evaluation requirements before they propagate to downstream implementation. The workflow resumes only after the user confirms or updates S′S'.

Observable execution. CLEAR decomposes the workflow into 15 operational stages that span parsing, planning, generation, execution, validation, evaluation, and output archiving (as shown in the document of our anonymous repository https://anonymous.4openscience.org/r/CLEAR-VSS/CLEAR_Running_Stages.pdf). The harness reports the current state of the lifecycle and exposes relevant intermediate artifacts, execution logs, and repair states through the frontend. In addition, it tracks runtime, token consumption, and estimated model cost, providing both users and the system with visibility into execution progress and resource usage.

In general, the lifecycle harness transforms CLEAR from a black-box code generator into an observable and steerable execution system. The persistent state enables recovery, checkpoints allow targeted human intervention, and stage-level observability enables cost-aware monitoring and diagnosis throughout the long-horizon workflow.

Experiment Settings

Datasets. We consider two ML research engineering tasks, RAR and MLE (§ 2), and evaluate CLEAR using one dataset per task.

  • For RAR tasks, PaperBench [17] evaluates whether agents can understand 20 top-tier ML papers from scratch, implement their codebases, and successfully run experiments.

  • For MLE tasks, MLE-Bench [2] evaluates agents’ ability to iteratively improve performance across 75 ML competitions. We noticed that MLE-Bench Lite is a carefully selected subset from MLE-Bench, providing an option that only requires about 14\frac{1}{4} of the testing resources for teams with limited experimental resources while ensuring task diversity as much as possible. Meanwhile, the baseline methods referenced in this article were compared on MLE-Bench Lite. To ensure a fair comparison, we rigorously tested CLEAR on MLE-Bench Lite under the same conditions as the baseline method. Finally, We use MLE-Bench’s Lite subset (22 tasks) as a lower-cost alternative to full evaluation.

The above two complementary datasets effectively evaluate the efficiency and cost of agent execution and optimization in ML research engineering during long-duration complex tasks.

Evaluation metrics. For PaperBench, its task score is automatically evaluated based on a large model, which is the weighted mean of 8,316 leaf scoring nodes. For MLE-Bench Lite, we evaluated the dataset on 22 tasks, calculating the percentage of tasks that resulted in valid submissions out of the total number of tasks, the percentage of tasks that scored above the median, the percentage of tasks achieving bronze, silver and gold medals, and the percentage of tasks obtaining any medal.

Baselines. We compare CLEAR against the following baselines on each dataset:

  • On PaperBench, we compared our method with three different agent methods, namely AiScientist, BasicAgent, and IterAgent, under the same evaluation protocol, despite their distinct processes and structures.

  • On MLE-Bench Lite, we report comparative results against strong autonomous ML engineering systems, including methods such as AIDE [8], ML Master 2.0 [29], and LoongFlow [19].

Implementation details. Following the settings in AiScientist, we use Gemini-3-Flash and GLM-5 [25] as the two backbone LLMs to instantiate the aforementioned baseline methods. For our own method, CLEAR, to better meet the working needs of our designed model collaboration, we have chosen Claude-Opus 4.6, which possesses strong thinking capabilities, as the planning model MpM_p, while Claude-Sonnet 4.6, which has similar habits but is more cost-effective, has been selected as the coding model McM_c.

The proposed visual lifecycle harness in the CLEAR features a human-in-the-loop checkpoint that allows users to inspect and revise specifications before code generation. We declare that: in the following experiments, CLEAR did not use manual review of system generated documents in all tests and did not make any modifications to the system generated content.

In PaperBench and MLE-Bench Lite, each run is allocated an Nvidia A800 80GB GPU and a 24-hour budget for each task. For the comprehensive evaluation of PaperBench, we adopted the official evaluation protocol [17] and used GPT-5.4 as our judging model. All experimental data provided in this article are based on a fixed evaluation protocol, which ensures a fair comparison between different methods.

Experimental Results

We evaluate CLEAR from three perspectives:

  • RQ1: Effectiveness. Can CLEAR reliably perform long-horizon ML research engineering across both RAR and MLE settings?

  • RQ2: Efficiency. Does CLEAR achieve a favorable performance–cost trade-off compared with existing ML research agents?

  • RQ3: Mechanism. How do structured specifications, specification-conditioned generation and specification-conditioned repair contribute to CLEAR’s effectiveness and efficiency?

To answer the above three questions, we first evaluate overall effectiveness and efficiency on PaperBench and MLE-Bench Lite in Sections 5.1 and 5.2. We then analyze the contributions of the key design choices in Section 5.3.

Finally, as a supplement to the above experimental results, a section of token cost statistics will be shown in Section 5.4.

Main Results on PaperBench

Table 1 summarizes the overall performance on PaperBench. CLEAR achieves an average score of 29.87 across 20 tasks, ranking second on 10 tasks, while maintaining an average execution cost of only $7.91 per task. These results indicate that CLEAR achieves a competitive performance–cost trade-off compared with existing agent frameworks. We further analyze the comparison from three perspectives. (1) Structured collaboration improves over lightweight agents. Compared with the BasicAgent baseline, CLEAR improves the average score from 22.58 to 29.87 (+32.29%) with only a $3.01 increase in average cost per task. This comparison shows that the additional computation introduced by the CLEAR workflow is accompanied by a substantial performance improvement while retaining relatively low execution cost. (2) More iterations do not necessarily yield better efficiency. Compared with IterAgent, CLEAR achieves a higher average score while reducing the average cost by 85.6% ($7.91 vs. $54.90). This comparison suggests that increasing the number of interaction rounds alone does not necessarily yield a favorable performance–cost trade-off, whereas CLEAR maintains competitive performance with substantially lower execution cost. (3) Adaptive resource allocation provides a favorable alternative to scaling a single strong model. Compared with AiScientist using the same model family, CLEAR reduces the average cost by 35.2% while retaining 88.6% of the performance. The result suggests that dynamically allocating a variable number of coding workers under strong planning guidance is more cost-effective than uniformly increasing model capacity.

TaskGemini BasicGemini IterGemini AiScientistGLM BasicGLM IterGLM AiScientistOpus & Sonnet CLEAR
adaptive-pruning24.533.0527.2530.8211.9333.2621.17
all-in-one20.8645.1346.2933.7844.4349.47<u>48.20</u>
bam48.4645.04<u>56.59</u>51.4547.9161.1152.33
bbox15.438.3033.7923.5519.28<u>30.02</u>28.23
bridging-data-gaps12.5912.4423.099.8012.5026.46<u>26.16</u>
fre21.6723.8935.2121.6016.67<u>28.98</u><u>28.71</u>
ftrl5.874.1510.113.716.708.34<u>8.82</u>
lbcs17.7515.26<u>27.90</u>20.6822.7430.1026.24
lca-on-the-line12.9718.3030.2322.5526.15<u>28.53</u>24.64
mechanistic-understanding14.8621.8929.9532.4934.9640.55<u>35.69</u>
pinn26.6330.8149.9222.1825.7758.7649.97
rice10.438.8810.876.560.2710.18<u>10.58</u>
robust-clip15.4510.4318.2822.43<u>27.56</u>28.6619.82
sample-specific-masks25.3933.3436.7736.93<u>41.26</u>44.1337.35
sapg11.4512.6519.856.994.9531.69<u>24.19</u>
sequential-neural53.51<u>60.24</u>64.9427.2035.5349.3249.85
stay-on-topic8.3713.6920.133.698.8114.81<u>16.26</u>
stochastic-interpolants17.0417.3718.8132.1828.0642.10<u>39.77</u>
test-time-model-adaptation15.2718.1332.4517.8121.1927.33<u>28.46</u>
what-will-my-model-forget6.618.9917.87<u>25.14</u>10.7530.8220.98
Average Score19.2620.60<u>30.52</u>22.5822.3733.7329.87
Average Cost<u>$6.25</u> | $27.44$15.67 | $4.90$54.90 | $12.20$7.91

Table 1. Main results on the full PaperBench evaluation. Bold and underlined denote the best and second-best results, respectively. The blue shaded column highlights CLEAR (Ours).

MethodModelValidAbove MedianBronzeSilverGoldAny Medal
AIDEGemini77.2754.554.559.0931.8245.45
LoongFlowGemini77.2777.2712.1225.76<u>39.39</u><u>77.27</u>
AiScientistGemini100.00<u>86.36</u>18.1831.8231.8281.82
AIDEGLM77.2750.004.5513.6422.7340.91
ML-Master 2.0GLM100.0081.8218.1813.64<u>31.82</u>63.64
AiScientistGLM100.0090.919.09<u>31.82</u>40.9181.82
CLEAR (Ours)Opus & Sonnet100.00<u>86.36</u><u>13.64</u>36.3727.28<u>77.27</u>

Table 2. Main results (%) on MLE-Bench Lite. Bold and underlined denote the best and second-best results, respectively. The blue shaded line highlights CLEAR (Ours).

At the task level, CLEAR exhibits substantial variation across different PaperBench instances. It achieves competitive results on all-in-one, bridging-data-gaps, mechanistic-understanding, and pinn, where it ranks among the top-performing configurations. Meanwhile, larger performance gaps remain on tasks such as adaptive-pruning and robust-clip. These results indicate that the effectiveness of CLEAR is task-dependent and leave room for more adaptive task-specific generation and repair strategies.

Main Results on MLE-Bench Lite

Table 2 reports the main results of MLE-Bench Lite. Compared with the baseline method, CLEAR matches the best Valid Submission rate and remains competitive on Above Median and Any Medal. All 22 tasks formed valid submissions, with 86.36% of tasks achieving scores above the median and 77.27% of tasks winning medals. The above results support the effectiveness of CLEAR in executing ML research engineering tasks under MLE setting.

In terms of performance–cost trade-off, Figure 3 compares the validation AUC trajectories of CLEAR and AiScientist on the Detecting Insults task from MLE-Bench Lite. Without human intervention, CLEAR reaches a validation AUC of 0.982 after 35 repair iterations, with 17 best-score updates. In comparison, AiScientist requires 48 repair iterations to reach the same level. Specifically, the figure shows two main trends: (1) CLEAR reaches the same validation AUC earlier in most stages, suggesting that the structured specification layer provides more effective guidance for downstream generation and repair; (2) AiScientist tends to make larger but sparser improvements, while CLEAR makes smaller and more frequent updates. This suggests that CLEAR’s specification-conditioned repair process enables more fine-grained engineering optimization with fewer repair iterations.

Figure 3. CLEAR autonomously improving performance on a competition-style ML task over iterations. On MLE-Bench Lite’s Detecting Insults task, it completes 35 experiment cycles, reaching a validation AUC of 0.982 with 17 best-so-far updates. Compared to AiScientist, CLEAR reduces iterations by 27.08%.

Together with the PaperBench results, these findings answer RQ1 and RQ2: CLEAR is effective across both RAR and MLE settings, and it achieves a favorable performance–cost trade-off by maintaining competitive task performance while reducing generation cost or repair iterations.

Mechanism Analysis

To answer RQ3, we examine the three key design choices of CLEAR: Document-to-specification parsing, Specification-conditioned generation and Specification-conditioned repair. Table 4 and Figure 4 report the corresponding results on PaperBench and MLE-Bench Lite.

Figure 4. Ablation results on MLE-Bench Lite. All values are percentages (%). "CLEAR w/o specs" means CLEAR only uses MpM_p to parse input documents, without processing by our designed skills. "CLEAR using GLM" means CLEAR uses GLM-5 for MpM_p and McM_c at the same time.

MethodModelAvg ScoreAvg Cost
CLEAROpus & Sonnet29.87$7.91
CLEAR w/o specsOpus & Sonnet24.14$10.08
CLEARGLM26.41$11.95

Table 4. Ablation results on PaperBench full evaluation. "CLEAR w/o specs" means CLEAR only uses MpM_p to parse input documents, without processing by our designed skills. Bold and underlined denote the best and second-best results on each task.

Document-to-specification parsing. We first remove the document-to-specification parsing skills, denoted as CLEAR w/o specs, and let the planning model MpM_p directly interpret the input documents.

Removing structured specifications consistently degrades both effectiveness and efficiency. On PaperBench, the average score drops by 19.18% while the average cost increases by 27.43%. On MLE-Bench Lite, performance decreases by 9.09%, 10.53%, and 17.64% on Valid Submission, Above Median, and Any Medal, respectively. These results support the role of (P′,S′)(P', S') as an explicit intermediate representation: turning the understanding of the task into more structured representation reduces ambiguity before implementation and avoids repeated interpretation during downstream generation and repair.

Specification-conditioned generation. We next examine heterogeneous model assignment using CLEAR w/ GLM, where both the planning model MpM_p and coding model McM_c are replaced by GLM-5 while keeping the rest of the CLEAR architecture unchanged. This variant removes the heterogeneous model assignment while retaining the specification-centered workflow. This homogeneous GLM-5 configuration retains the specification-centered workflow but differs from the full CLEAR configuration in both model assignment and backbone. Compared with full CLEAR, its PaperBench score decreases by 11.59% and its average cost increases by 51.07%. On MLE-Bench Lite, it also underperforms CLEAR on Above Median and Any Medal, while achieving the same Valid Submission rate. These results show that the full Opus–Sonnet configuration achieves a better performance–cost trade-off than the homogeneous GLM-5 configuration.

Overall, the ablation results support both design principles of CLEAR: structured specifications improve the reliability and efficiency of task understanding, while heterogeneous model assignment improves the performance–cost trade-off by matching model capacity to stage-specific requirements.

Specification-conditioned repair. To better understand how CLEAR behaves during iterative repair, we analyze repair behavior on four representative MLE-Bench Lite tasks covering from image classification, text classification, tabular prediction to sequence-to-sequence learning.

Task NameCLEAR
originalw/o specswith GLM-5
adaptive-pruning517,006622,0683,022,296
all-in-one647,319963,7262,727,358
bam407,987884,0383,653,356
bbox519,198717,9012,676,488
bridging-data-gaps596,981852,0552,262,324
fre690,2221,065,6294,050,093
ftrl608,695755,0942,678,289
lbcs497,164850,4782,726,949
lca-on-the-line705,993887,3561,693,312
mechanistic-understanding463,635974,4243,172,406
pinn501,899684,9873,100,193
rice706,8671,021,4893,116,092
robust-clip551,805699,0401,684,801
sample-specific-masks399,290556,9143,039,807
sapg559,333918,1523,187,321
sequential-neural389,7311,062,3453,678,553
stay-on-topic596,446768,5872,913,926
stochastic-interpolants768,990988,8504,610,972
test-time-model-adaptation549,805708,1092,956,024
what-will-my-model-forget608,380851,1342,805,173
Average Token Cost564,337841,6192,987,787

Table 5. Token consumption of CLEAR and its two variants on PaperBench full evaluation.

As summarized in Table 3, CLEAR performs 35–49 repair iterations across these tasks, with an average of 41.75. The number of required iterations varies substantially across tasks, indicating that repair behavior will be different between tasks. We define a useful repair as an iteration that improves the best validation score achieved so far. The useful-repair ratio ranges from 35.14% to 71.43%, averaging 52.37%. Thus, more than half of the repair iterations on average improve the best-so-far validation score, although the frequency of such improvements varies across tasks. These results characterize CLEAR’s behavior during iterative repair: across the analyzed tasks, a substantial fraction of bounded, localized repair iterations lead to measurable improvements within the predefined repair budget.

TaskTask TypeRepairsBest-score UpdatesUseful Repair Rate
aerial-cactus-identificationImage Classification371335.14%
detecting-insults-in-social-commentaryText Classification351748.57%
nomad2018-predict-transparent-conductorsTabular493571.43%
text-normalization-challenge-english-languageSeq2Seq462554.35%
Average–41.7522.5052.37%

Table 3. Repair dynamics of CLEAR on representative MLE-Bench Lite tasks. We report the number of repair iterations, best-score updates, and the proportion of useful repairs.

Token Consumption Analysis

Dollar cost provides a direct measure of practical deployment efficiency, but it depends on the pricing of the underlying models. To complement dollar cost with a pricing-independent usage indicator, we additionally report token consumption on the full PaperBench evaluation.

Table 5 compares CLEAR with its two variants. CLEAR consumes an average of 564K tokens per task, compared with 842K for CLEAR w/o specs and 2.99M for CLEAR w/ GLM-5. CLEAR w/o specs therefore consumes 49.1% more tokens than the full CLEAR configuration. This observation is consistent with the hypothesis that maintaining an explicit intermediate specification reduces repeated interpretation during downstream generation and repair. The homogeneous GLM-5 configuration consumes more than five times as many tokens as the full Opus–Sonnet configuration. This result is consistent with the overall efficiency advantage observed for the full CLEAR system. However, because the configurations use different model families and tokenizers, the cross-model token counts should be interpreted as a system-level usage comparison rather than a model-independent measure of computational cost.

Related Work

We review two lines of related work that underpin our study: LLM-based autonomous scientific research systems and multi-agent cooperation workflows for complex task execution.

Automated scientific research. Recent work has explored the use of LLM agents for automating scientific research, including idea generation[11], experiment execution[7], code development[10, 16, 26], and scientific writing[12, 20]. These developments position autonomous scientific research as a realistic goal for general-purpose agent systems[15, 18, 22].

Among them, automating ML research engineering is particularly representative and challenging, as it requires transforming heterogeneous research artifacts into executable systems under limited cost and runtime constraints[2, 4, 29]. Recent systems such as BasicAgent[17], Leeroo[13], AIDE[8], AIBuildAI[27], and Ai-Scientist[4] demonstrate the feasibility of this paradigm. However, these systems often operate directly on heterogeneous research artifacts without an explicit intermediate representation, which can lead to cascading errors and expensive repair loops.

Multi-agent cooperation workflows. Multi-agent cooperation has become a common paradigm for improving LLM agents on complex tasks. Early frameworks explore role-playing agents, dynamic team formation, and programmable multi-agent conversations[5, 9, 21]. These works show that task decomposition and agent specialization can improve problem solving, but coordination is often mediated through flexible natural-language interaction and relies on human-designed prompts heavily. In software engineering, multi-agent workflows have been adapted to code generation, repository editing, and issue resolution. MetaGPT and ChatDev organize agents through standardized software-development pipelines[6, 14]. SWE-agent emphasizes agent-computer interfaces for repository-level editing and testing[24], while AutoCodeRover and CodeR further incorporate code search, program analysis, and task graphs for issue resolution[3, 28].

Despite recent advances, multi-agent systems still suffer from coordination failures, inter-agent misalignment, communication overhead, and weak verification[1, 23]. These issues become more severe in long-horizon ML research engineering, where task understanding, execution feedback, and repair decisions must remain consistent across stages. This motivates introducing a more explicit intermediate representation between research understanding and executable implementation.

Conclusion

We presented CLEAR, a specification-centered framework for controllable long-horizon ML research engineering. CLEAR introduces an intermediate execution specification between heterogeneous research artifacts and executable implementations, enabling specification-conditioned planning, localized code generation, and targeted failure repair. Built upon this representation, CLEAR further integrates heterogeneous model assignment with lifecycle-oriented execution control, allowing different models to be dynamically allocated according to their capabilities and roles. Experiments on PaperBench and MLE-Bench Lite demonstrate that CLEAR achieves competitive performance while maintaining favorable performance–cost trade-offs across both research automation and machine learning engineering scenarios. Ablation studies further validate the importance of structured specifications and adaptive model collaboration. Overall, our results suggest that future ML research engineering agents can benefit from explicitly separating what should be implemented from how it should be implemented, executed, and refined.

Future Work. Several directions remain open. First, the reliability of execution specifications could be further improved through automatic consistency checking, cross-modal verification, and execution-based validation. Second, the number of low-cost coding workers could be allocated more adaptively based on task difficulty, uncertainty, failure type, and expected repair value. Finally, the specification layer could be extended into a persistent research engineering memory that accumulates reusable implementation patterns, failure modes, and repair strategies across tasks.

Acknowledgments

This work was funded by the Strategic Priority Research Program of the CAS under Grants No. XDB0680102, the National Natural Science Foundation of China (NSFC) under Grants No. U25B2076, 62472408 and 62441229, and the National Key Research and Development Program of China under Grants No. 2023YFA1011602. All content represents the opinion of the authors, which is not necessarily shared or endorsed by their respective employers and/or sponsors.

Ethical Considerations

This article is aimed at the problem that in automatic ML research engineering, directly operating on heterogeneous research artifacts without an explicit intermediate representation causes cascading errors and expensive repair loops. To mitigate this issue, we propose CLEAR, a controllable system for long-horizon ML research workflow in which a structured specification layer transforms heterogeneous research artifacts into human verifiable execution specifications. With respect to data usage, all experiments are conducted exclusively on publicly available academic benchmarks, including PaperBench and MLE-Bench Lite. No private, proprietary, or sensitive data is used at any stage of data collection, training, or evaluation. Furthermore, the proposed method does not introduce or expose any new personal information. As a result, this study does not raise concerns related to privacy, data misuse, or unauthorized access, and complies with common ethical standards for academic research in information retrieval and natural language processing.

References

  1. [1]Mert Cemri, Melissa Z. Pan, Shuiyi Yang, Lakshya A. Agrawal, Bhavya Chopra, Rishabh Tiwari, Kurt Keutzer, Aditya Parameswaran, Dan Klein, Kannan Ramchandani, Matei Zaharia, Joseph E. Gonzalez, and Ion Stoica. 2025. Why Do Multi-Agent LLM Systems Fail? arXiv:2503.13657 [cs.AI] https://arxiv.org/abs/2503.13657
  2. [2]Jun Shern Chan, Neil Chowdhury, Oliver Jaffe, James Aung, Dane Sherburn, Evan Mays, Giulio Starace, Kevin Liu, Leon Maksin, Tejal Patwardhan, et al. 2025. Mle-bench: Evaluating machine learning agents on machine learningarxiv.org/abs/2410.07095
  3. [3]Dong Chen, Shaoxin Lin, Muhan Zeng, Daoguang Zan, Jian-Gang Wang, Anton Cheshkov, Jun Sun, Hao Yu, Guoliang Dong, Artem Aliev, Jie Wang, Xiao Cheng, Guangtai Liang, Yuchi Ma, Pan Bian, Tao Xie, and Qianxiang Wang. 2024. CodeI: Issue Resolution with Multi-Agent and Task Graphs. arXiv:2406.01304 [cs.CL] https://arxiv.org/abs/2406.01304
  4. [4]Guoxin Chen, Jie Chen, Lei Chen, Jiale Zhao, Fanzhe Meng, Wayne Xin Zhao, Ruihua Song, Cheng Chen, Ji-Rong Wen, and Kai Jia. 2026. Toward Autonomous Long-Horizon Engineering for ML Research. arXiv preprint arXiv:2604.13018 (2026).arxiv.org/abs/2604.13018
  5. [5]Weize Chen, Yusheng Su, Jingwei Zuo, Cheng Yang, Chenfei Yuan, Chi-Min Chan, Heyang Yu, Yaxi Lu, Yi-Hsin Hung, Chen Qian, Yujia Qin, Xin Cong, Ruobing Xie, Zhiyuan Liu, Maosong Sun, and Jie Zhou. 2024. AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors. In International Conference on Learning Representations (2024), B. Kim, Y. Yue, S. Chaudhuri, K. Fragkiadakis, M. Khan, and Y. Sun (Eds.), Vol. 2024. 2004–20136. https://proceedingslcr.cc/paper_files/paper/2024/file/578e65dec35d00c708d4c64bce32971-Paper-Conference.pdfarxiv.org/abs/2308.10848
  6. [6]Sirui Hong, Mingchen Zhuge, Jiaqi Chen, Xiawu Zheng, Yuheng Cheng, Ceyao Zhang, Jinlin Wang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, Lingfeng Xiao, Chenglin Wu, and Jürgen Schmidhuber. 2024. MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework. In International Conference on Learning Representations.arxiv.org/abs/2308.00352
  7. [7]Qian Huang, Jian Wang, Jian Wang, Percy Liang, and Jure Leskovec. 2024. MLAgentBench: Evaluating Language Agents on Machine Learning Experimentation. In International Conference on Machine Learning. PMLR, 20271–20309.arxiv.org/abs/2310.03302
  8. [8]Zhengyao Jiang, Dominik Schmidt, Dhruv Srikanth, Dixing Xu, Ian Kaplan, Denis Jacenko, and Yuxiang Wu. 2025. Aide: Ai-driven exploration in the space of code. arXiv preprint arXiv:2502.13138 (2025).arxiv.org/abs/2502.13138
  9. [9]Guohao Li, Hasan Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. 2023. CAMEL: Communicative Agents for “Mind” Exploration of Large Language Model Society. In Advances in Neural Information Processing Systems, A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (Eds.), Vol. 36. Curran Associates, Inc., 51991–52008. https://proceedings.neurips.cc/paper_files/paper/2023/file/a3621ee907def47c1b952ade25c67698-Paper-Conference.pdf
  10. [10]Zongwei Li, Zhonghang Li, Zirui Guo, Xubin Ren, and Chao Huang. 2025. Deep-Code: Open Agentic Coding. arXiv:2512.07921 [cs] doi:10.48550/arXiv.2512.07921
  11. [11]Cong Lu, Shengran Hu, and Jeff Clune. 2025. Automated Capability Discovery via Model Self-Exploration. In ICLR 2025 Workshop on Foundation Models in the Wild.arxiv.org/abs/2502.07577
  12. [12]Chris Lu, Cong Lu, Robert Tjarko Lange, Yutaro Yamada, Shengran Hu, Jakob Foerster, David Ha, and Jeff Clune. 2026. Towards end-to-end automation of AI research. Nature 651, 8107 (2026), 914–919.arxiv.org/abs/2606.15497
  13. [13]Alireza Mohammadshahi, Ali Shaikh, and Majid Yazdani. 2024. Leeroo orchestrator: Elevating llms performance through model integration. arXiv preprint arXiv:2401.13979 3 (2024).
  14. [14]Chen Qian, Wei Liu, Hongzhang Liu, Nuo Chen, Yufan Dang, Jiahao Li, Cheng Yang, Weize Chen, Yusheng Su, Xin Cong, Juyuan Xu, Dahai Li, Zhiyuan Liu, and Maosong Sun. 2024. ChatDev: Communicative Agents for Software Development. arXiv:2307.07924 [cs.SE] https://arxiv.org/abs/2307.07924
  15. [15]Samuel Schmidgall, Yusheng Su, Ze Wang, Ximeng Sun, Jialian Wu, Xiaodong Yu, Jiang Liu, Michael Moor, Zicheng Liu, and Emad Barsoum. 2025. Agent laboratory: Using llm agents as research assistants. Findings of the Association for Computational Linguistics: EMNLP 2025 (2025), 5977–6043.
  16. [16]Minju Seo, Jinheon Baek, Seongyun Lee, and Sung Ju Hwang. 2026. Paper2Code: Automating Code Generation from Scientific Papers in Machine Learning. arXiv:2504.17192 [cs.CL] https://arxiv.org/abs/2504.17192
  17. [17]Giulio Starace, Oliver Jaffe, Dane Sherburn, James Aung, Jun Shern Chan, Leon Maksin, Rachel Dias, Evan Mays, Benjamin Kinsella, Wyatt Thompson, et al. 2025. PaperBench: Evaluating AI’s Ability to Replicate AI Research. In International Conference on Machine Learning. PMLR, 56843–56873.
  18. [18]Jiabin Tang, Lianghao Xia, Zhonghang Li, and Chao Huang. 2026. Ai-researcher: Autonomous scientific innovation. Advances in Neural Information Processing Systems 38 (2026), 9481–9520.arxiv.org/abs/2505.18705
  19. [19]Chunhui Wan, Xuanan Dai, Zhuo Wang, Minglei Li, Yanpeng Wang, Yinan Mao, Yu Lan, and Zhiyen Xiao. 2025. Longflow: Directed evolutionary search via a cognitive plan-execute-summarize paradigm. arXiv preprint arXiv:2512.24077 (2025).arxiv.org/abs/2512.24077
  20. [20]Yidong Wang, Qi Guo, Wenjin Yao, Hongbo Zhang, Xin Zhang, Zhen Wu, Meishan Zhang, Xinyu Dai, Min Zhang, Qingsong Wen, et al. 2024. Autosurvey: Large language models can automatically write surveys. Advances in neural information processing systems 37 (2024), 115119–115145.arxiv.org/abs/2406.10252
  21. [21]Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, Ahmed Hassan Awadallah, Ryan W White, Doug Burger, and Chi Wang. 2023. AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation. arXiv:2308.08155 [cs.AI] https://arxiv.org/abs/2308.08155
  22. [22]Tengyue Xu, Zhuoyang Qian, Gaoge Liu, Li Ling, Zhentao Zhang, Biao Wu, Shuo Zhang, Ke Lu, Wei Shi, Ziqi Wang, et al. 2026. IdeaStory: An Automated Pipeline for Transforming Research Concepts into Complete Scientific Narratives. arXiv preprint arXiv:2601.20833 (2026).
  23. [23]Bingyu Yan, Zhibo Zhou, Litian Zhang, Lian Zhang, Ziyi Zhou, Dezhuang Miao, Zhoujun Li, Chaozhuo Li, and Xiaoming Zhang. 2025. Beyond Self-Talk: A Communication-Centric Survey of LLM-Based Multi-Agent Systems. arXiv:2502.14321 [cs.MA] https://arxiv.org/abs/2502.14321
  24. [24]John Yang, Carlos Jimenez, Alexei Zhelnin, Kilian Liere, Shunyu Yao, Karthik Narasimhan, and Ofir Press. 2024. SWE-agent: Agent-computer Interfaces Enable Automated Software Engineering. In Advances in Neural Information Processing Systems (2024), A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (Eds.), Vol. 37. Curran Associates, Inc., 50528–50562. doi:10.52202/079017-1601
  25. [25]Aohan Zeng, Xin Lv, Zhenyu Hou, Zhengxiao Du, Oinkai Zheng, Bin Chen, Da Yin, Chendi Ge, Chenghua Huang, Chengxing Xie, et al. 2026. Glm-5: from vibe coding to agentic engineering. arXiv preprint arXiv:2602.15763 (2026).
  26. [26]Jenny Zhang, Shengran Hu, Cong Lu, Robert Lange, and Jeff Clune. 2025. Darwin godel machine: Open-ended evolution of self-improving agents. arXiv preprint arXiv:2505.22954 (2025).arxiv.org/abs/2505.22954
  27. [27]Ruiyi Zhang, Peijia Qin, Qi Cao, Li Zhang, and Pengtao Xie. 2026. AIBuildAI: An AI Agent for Automatically Building AI Models. arXiv preprint arXiv:2604.14455 (2026).arxiv.org/abs/2604.14455
  28. [28]Yuntong Zhang, Haifeng Ruan, Zhiyu Fan, and Abhik Roychoudhury. 2024. AutoCodeRover: Autonomous Program Improvement. In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis (Vienna, Austria) (ISSTA 2024). Association for Computing Machinery, New York, NY, USA, 1592–1604. doi:10.1145/3650212.3680384DOI
  29. [29]Xinyu Zhu, Yuzhu Cai, Zexi Liu, Bingyang Zheng, Cheng Wang, Rui Ye, Jiaao Chen, Hanrui Wang, Wei-Chen Wang, Yuzhi Zhang, et al. 2026. Toward ultra-long-horizon agentic science: Cognitive accumulation for machine learning engineering. arXiv preprint arXiv:2601.10402 (2026).arxiv.org/abs/2601.10402

Paper details

vlnc:W_m2y3vu4n
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents