betaFeedback

WorksComputer Science

Towards Agentic Proof Harness Engineering

Abstract

Formal verification of real-world projects requires substantial manual effort, from specifying correctness properties to constructing the infrastructure needed to check them. We envision verification agents that assist developers throughout this process. Among the many tasks involved, one stands out as both foundational and amenable to automation: proof-harness engineering. A proof harness defines an isolated verification environment for a program unit, such as a function or module, by providing symbolic inputs, initialization code, and models of its dependencies. Today, proof-harness engineering remains largely a manual task performed by verification engineers. To explore the potential of coding agents for this task, we propose an agentic proof-harness engineer that automatically constructs verifier-ready harnesses from a software project and a designated verification target. We combine the agent with formal soundness and precision guards to validate generated abstractions and guide their refinement. We evaluate the feasibility of our approach in an exploratory case study on curl and libuv, using three historical assertion violations already embedded in the source code as verification properties. Our results show that the agent successfully constructs proof harnesses that enable CBMC to detect all three violations. We further demonstrate the potential for reusing these harnesses with other verification tools.

Introduction

Automatic software verification has become a mature field, with numerous tools (verifiers) supporting program verification. The annual Competition on Software Verification (SV-COMP) [4] provides a large and diverse benchmark suite on which the effectiveness and efficiency of verifiers are evaluated. Despite significant progress in recent years, applying software verification tools to real-world software projects remains challenging. Verifiers typically operate on well-defined verification tasks, whereas real-world projects consist of multiple source files, depend on project-specific build configurations and external libraries, and require suitable initialization and environment models. Developers must therefore bridge the gap between a software project and a verifier-ready verification task.

In this paper, we envision coding agents assisting developers in this process and focus on the task of proof-harness engineering. A proof harness defines an isolated verification environment for a program unit, such as a function, by providing symbolic inputs, initialization code, and abstractions of its dependencies. Our goal is an autonomous verification-harness agent that constructs complete proof harnesses for C projects given the project source code, its build instructions, and a unit to be verified. The agent constructs the harness by recovering and abstracting information already present in the project, such as how the target is initialized, used, and interacts with its dependencies. The resulting harness should over-approximate the target’s possible executions without excluding relevant program behaviors. Importantly, the agent does not invent the specification; the properties to be verified remain under the control of the developer. Such properties are frequently expressed in program code through assertions and sanity checks [8].

Constructing a proof harness requires substantial project-specific knowledge. Recent work on HarnessForge [2] automates the extraction of self-contained verification tasks from C projects, but requires manually provided harness code, including symbolic inputs, initialization, and models of external dependencies. We investigate whether a coding agent can autonomously construct the complete proof harness from the project itself, without requiring developers to provide any harness code. To address the risk of unsound or imprecise abstractions, we introduce independent formal guardrails that check the soundness of dependency abstractions and guide their refinement using verifier feedback.

To evaluate the feasibility of agentic proof-harness engineering, we implement a prototype and conduct an exploratory case study on two real-world C projects, curl and libuv. We use three historical assertion violations as verification targets. Our agent successfully constructs proof harnesses that enable CBMC to detect all three violations. We further investigate the reuse of these harnesses with other verification tools, demonstrating their potential beyond the verifier used during construction while also exposing limitations in cross-verifier compatibility.

Agentic Proof Harness Engineering

Our goal is to provide initial insights on how (coding) agents can be employed to automate the task of proof harness engineering. We demonstrate the task with a simple example taken and adapted from Beyer et al. [2].1

Proof Harness Engineering. Figure 1 (left) shows an example function add implemented as part of a larger software system. Our goal is to automatically check whether the assertion encoded in the function can be violated. Verifying this property directly in the context of the complete system is challenging: add may be reachable from thousands of call sites, while the states in which it

Target function and a corresponding proof harness

Figure 1. Target function (left) and a corresponding proof harness (right).

// Target function in "./src/counter.c"
counter_status_t add(short some_value) {
    short value = toAdd(some_value);

    if (...){
        // Over-flow handling
        return COUNTER_OVERFLOW;
    }

    short noi_old = gc.noi;
    short cv_old = gc.cv;

    gc.noi++;
    gc.cv += value;

    short diff = gc.cv - cv_old;
    assert(gc.noi >= noi_old &&
           0 <= diff && diff <= RANGE);

    return COUNTER_OK;
}
// Proof Harness
#include "./src/counter.c"

short toAdd(short val) {
    short r = nondet_short();
    assume(r >= 0 && r <= RANGE
           || (val == SHRT_MIN
               && r == -RANGE));
    return r;
}

int main(void) {
    gc.noi = nondet_short();
    gc.cv = nondet_short();
    add(nondet_short());
    return 0;
}

is invoked depend on the composition, configuration, and environment of the project.

A proof harness provides a compact abstraction of the execution environment. Figure 1 (right) shows a corresponding harness for add. Instead of enumerating its callers, the harness initializes the relevant global state symbolically and invokes add with a symbolic input. The dependency toAdd is replaced by a model that summarizes its relevant behavior. The verifier can then analyze the resulting self-contained task independently of the remainder of the project.

Proof-harness engineering requires substantial project-specific knowledge. The engineer must determine which program state influences the target and how it should be initialized, replace omitted dependencies by abstractions that preserve their relevant behavior, and account for the project’s build configuration, including types, macros, and other compile-time choices. In our example, the abstraction of toAdd encodes a formal summary relating its inputs to possible return values. In particular, for some_value == SHRT_MIN, the dependency may return -RANGE, which can lead to an assertion violation in add. Omitting this behavior from the abstraction could therefore hide a real violation. Constructing such a harness therefore requires exploring and reasoning about information distributed throughout the project.

Agentic Engineering. Given a target system SS, a function ff, and a verifier V\mathcal{V}, we seek to construct a harness HfH_f that satisfies two requirements:

  • Soundness. The harness over-approximates the inputs, program states, and dependency behaviours that may occur when executing ff in SS.

  • V\mathcal{V}-precision. The over-approximation is sufficiently precise for V\mathcal{V} to establish the property encoded in ff or expose a feasible counterexample.

Constructing such a harness requires exploring the system SS, identifying dependencies of ff, recovering relevant project information, and constructing formal abstractions. We envision an autonomous agent performing this engineering work by inspecting the project and its configurations, executing tools, and iteratively constructing and revising candidate harnesses. Figure 2 illustrates this workflow. Since the agent itself provides no formal guarantees, we propose two formal guardrails. A soundness guard (Algorithm 1) checks whether each abstraction AdA_d of a dependency dd over-approximates the behavior of the original dependency. Violations are returned to the agent for repair. A precision guard (Algorithm 2) uses feedback from VV to identify insufficiently precise abstractions and guide their refinement. Failed checks and spurious counterexamples are returned to the agent for revision. A more detailed description is provided in Section A.

Workflow of Agentic Proof Harness Engineering

Figure 2. Workflow of Agentic Proof Harness Engineering

Related Work. Agentic approaches to software engineering are rapidly emerging, including agents for general software-engineering tasks [15], program repair [6], and automated program analysis [5]. Their application to formal verification is even more recent, with work on agentic interaction with proof assistants [14, 12] and specification and proof generation [10]. Closest to our proposal, agentic model checking [13] employs agents to synthesize specifications, stubs, and function-level verification code for bounded model checking. Our approach focuses on the construction of verification environments from existing project information. The verification properties remain developer-specified, and the generated harnesses can be processed by existing verifiers through an SV-COMP-compatible representation. Prior work on proof-harness construction [9, 1, 2] automates parts of this process but still requires manually supplied harness code.

Evaluation

To evaluate the feasibility of agentic proof-harness engineering, we implement a prototype of the proposed harness agent. We conduct an exploratory case study on curl2, a widely used C project comprising more than 4,500 files and 630,000 lines of code3. We investigate whether our agent can construct proof harnesses to expose two historical bugs in CURL and additionally explore its applicability to a historical bug in LIBUV4 and our example in Figure 1. Our evaluation is guided by the following research question:

Can an agent construct proof harnesses that expose and verify historical bugs?

Experimental Setup

We select three historical assertion violations documented in pull requests from CURL and LIBUV: CURL #22785, CURL #23292, and LIBUV #5021. Table 1 provides an overview of the selected cases. We selected these pull requests because (1) they cover different functionalities within the respective projects, (2) the violated assertions were already present in the source code, allowing us to focus on the agent’s harness construction capabilities, and (3) the corresponding pull requests provide the original and fixed revisions for evaluating the generated harnesses. We also discuss our example with an assertion violation from Figure 1. The agent is employed to build a harness for the original and fixed revisions independently. We provide the agent with the name of target function in its initial prompt. The prompt is shown in Section D. We use openai/gpt-6-luna with a temperature of 0.0 and limit of 60 actions per run.

ProjectPRComponentBug Description
CURL#22785DNS-over-HTTPSAn assertion in doh_resp_decode_httpsrr() fails on malformed DNS responses containing trailing bytes.
CURL#23292HTTP headersMalformed headers with empty names violate an assertion in Curl_dynhds_add().
LIBUV#5021UnicodeAn off-by-one assertion in uv_wtf8_to_utf16() rejects the valid Unicode code point 0x10FFFF.
Figure 1—CountingtoAdd() overflows for SHRT_MIN

Table 1. Historical assertion violations selected for our case study.

To avoid interference with the original project, we isolate agent runs: The agent can read every file in the original project, but can only modify and create files in a dedicated verification/ directory. Actions are executed in an isolated Docker container. At the end of the agent run, the verification/ directory should include a proof harness that compiles against the original project. Our main verifier is CBMC v5.95.15 with a timeout of 5 minutes6. The harness Table 2. Verification results for historical assertion violations. ✓: safe; ✖: unsafe; ?: unknown; E: error. Desired outcomes are ✖ for violated assertions and ✓ or ? for resolved assertions (the latter remains inconclusive).

follows the SV-COMP conventions, thus allowing us to employ different verifiers to the same harness. We also evaluated UAutomizer [11] (the winner of the SV-COMP 2026), CPAchecker [3], and KLEE [7] on the harnesses generated for CBMC. All experiments were carried out on a single machine equipped with an Intel Xeon W-2265 CPU (12 physical cores, 24 hardware threads, 4.8 GHz) and 251 GiB of RAM, running Ubuntu 22.04.

Results

In Table 2, we report the results of our harness construction using CBMC as the guiding verifier. We additionally evaluate the generated harnesses with three other verifiers. For all three pull requests (cases), our harness agent successfully constructs a proof harness that enables CBMC to detect the historical assertion violation in the original revision. When applied to the revised versions, CBMC no longer reports a violation, returning ✓ (safe) or ? (unknown). Notably, CBMC establishes safety for the revised version of libuv #5021. In this case, the fix corrects an assertion on the maximum Unicode code point, making it consistent with the range accepted by the decoder.

CaseAssertionCBMCKLEEUAutomizerCPAchecker
CURL #22785Violated✖✖EE
Resolved??EE
CURL #23292Violated✖✖E?
Resolved??✓?
LIBUV #5021Violated✖✖✖✖
Resolved✓??✓
Figure 1Violated✖✖✖✖

Table 2. Verification results for historical assertion violations.

Harness reuse. Proof harnesses are not necessarily one-time artifacts; they offer two opportunities for reuse. First, developers can reuse proof harnesses across revisions of a project, for example, to reverify an implementation after fixing an assertion violation. With CBMC, we found that two of the three generated harnesses could be reused without modification, while one required further refinement of the dependency abstraction. Second, developers can reuse proof harnesses with different verifiers. We investigate this scenario by analyzing the generated harnesses with KLEE, Ultimate Automizer, and CPAchecker. For libuv, all four verifiers detect the historical assertion violation. On the resolved revision, CBMC and CPAchecker establish safety, while KLEE and Ultimate Automizer return unknown. For curl, CPAchecker and Ultimate Automizer struggle with unsupported C constructs and library functions, potentially due to differences between the project’s C dialect and the GNU C subset supported by SV-COMP verifiers. KLEE requires additional definitions from CURL source files, which we provide manually. With these additions, KLEE detects both historical assertion violations in CURL and reports unknown on the resolved revisions.

These results provide initial evidence that agent-generated proof harnesses can be reused across project revisions and verification tools, while highlighting the challenges of achieving portability without manual adaptations.

Impact of Guards. We investigate the impact of our soundness and precision guards on harness construction for the motivating example in Figure 1 (Table 2, last row). We find that while openai/gpt-6-luna is capable of constructing sound harnesses, it struggles to achieve sufficient precision to distinguish real assertion violations from spurious counterexamples. Our precision guard identifies spurious counterexamples and guides the agent toward increasingly precise harnesses, while the soundness guard ensures that refinements preserve the behavior of the original dependencies. A qualitative analysis of the refinement process is provided in Section B. The precision guard also proves valuable in our real-world case studies, preventing 11 false-positive reports with CBMC.

Discussion and Future Research

We discuss two research directions arising from our approach and the remaining challenges toward trustworthy agentic system verification.

Towards Trustworthy Agentic System Verification. We envision coding agents assisting developers throughout the verification process, from constructing proof harnesses for components of large software systems to configuring verification tools, interpreting results, and repairing violations. Today’s coding agents are already capable of assisting with many of these tasks, but they provide no correctness guarantees. An agent may fail unpredictably, incorrectly claim that a program has been verified, or modify the specification to achieve a desired verification result. Such behavior undermines trust in agent-generated results, particularly in safety-critical software development.

We believe that our approach provides a first step toward trustworthy agentic system verification. By combining coding agents with independent formal guardrails, we aim to establish trust in agent-generated artifacts without relying on the correctness of the agents’ reasoning. A natural next step is to extend proof-harness engineering from individual functions to entire software systems. We envision agents systematically constructing verification environments for program units throughout a project, exploiting information about their interactions to improve precision and enable system-level reasoning. Establishing formal guarantees when combining such local analyses remains a key research challenge.

Towards Automated Benchmark Construction. Constructing software verification benchmarks is challenging and time-consuming. Existing benchmark suites contain artificially generated programs and tasks extracted from real-world projects, for which dependencies and execution environments often require manual abstraction. Agentic proof-harness engineering could automate this process, enabling larger and more diverse benchmark suites from real-world software. A central challenge is ensuring that generated tasks preserve the properties of the original software. While sound over-approximations support safety proofs, they may introduce spurious violations. This raises the question of how to construct sufficiently precise tasks with expected verdicts that are independent of the verifier used during construction. In the longer term, if proof-harness engineering becomes an integral part of system-level verification, the resulting benchmarks could directly reflect the workloads encountered by verifiers in practice.

Challenges of Proof-Harness Engineering. Several challenges for agentic proof-harness engineering remain. First, validating dependency abstractions may be infeasible when implementations are unavailable or verification is inconclusive. Dynamic analysis could help construct candidate abstractions in such cases, although establishing soundness remains a challenge. Second, isolated verification environments may admit initial states unreachable in the original system. Recovering relevant execution contexts while preserving developer-defined specifications remains an open challenge. Finally, our current refinement strategy provides no guarantee of progress or termination, as the agent may repeatedly fail to construct sufficiently precise abstractions.

Limitations of Coding Agents. Our experiments also exposed practical limitations of coding agents. Agents entered unproductive loops, spent considerable effort searching for suitable build configurations, and attempted prohibited operations, including modifying developer-provided assertions. Restricting project access mitigates some of these problems. Agent runs are also nondeterministic. In two runs on a CURL case, we observed different strategies: in the first, the agent identified the violation through code reasoning and confirmed it through concrete execution; in the second, CBMC discovered the violation directly. Such variability complicates reproducibility and motivates further research into controlling and evaluating agentic verification workflows.

Conclusion

In this paper, we propose the problem of agentic proof-harness engineering and present our first steps toward addressing it. Our prototype combines a coding agent with independent soundness and precision guards to construct verification tasks from real-world C projects. An exploratory case study on CURL and LIBUV demonstrates the feasibility of this approach, enabling existing verifiers to expose historical assertion violations. Our results also provide initial evidence that generated proof harnesses can be reused across different verifiers. We see agentic proof-harness engineering as a promising first step toward trustworthy agentic system verification, with opportunities for system-level verification and automated benchmark construction.

Agentic Proof Harness Engineering: Details

We implement a prototype of the agentic proof-harness engineer for C projects introduced in § 2. Our design consists of a harness agent guided by two components: a soundness guard that validates dependency abstractions and a precision guard that uses verifier feedback to guide their refinement. The soundness guard is described in Algorithm 1 and the precision guard in Algorithm 2. Importantly, the agent does not need to construct a sufficiently precise harness in a single step. It starts with a permissive abstraction and progressively improves its precision, while each restriction on dependency behavior is checked for soundness.

Harness Agent

We implemented a custom LLM-based agent to construct a proof harness HfH_f from a given C project SS and target function ff. Our design goal was to construct over-approximating proof harnesses without modifying the original project SS.

Proof Harness Initialization. The agent starts by locating the target function in the C project, inspecting its signature, identifying its dependencies, and searching for a suitable build configuration. It then constructs a proof harness similar to Figure 1. First, the agent creates a driver function main that initializes the inputs and global state used by the target function and invokes it. To over-approximate the possible initial states, the driver assigns nondeterministic values to the inputs and relevant global variables. We deliberately impose no additional restrictions on these values, treating the target function as an independently invoked unit. Second, the agent constructs symbolic mocks (abstractions) for dependencies that are excluded from the verification task. Initially, these abstractions admit unconstrained behavior. Their precision is subsequently improved through verifier-guided refinement, with each proposed restriction checked by the soundness guard.

Restricted Project Access. During proof-harness engineering, the agent has read-only access to the original C project SS and may modify files only within a dedicated verification/ directory. We enforce these restrictions at the operation level by blocking unauthorized file operations and commands and redirecting the agent to permitted operations. We additionally prohibit access to the project’s Git history. We found these restrictions necessary, as the agent attempted to inspect previous commits and modify developer-provided assertions to achieve its verification goal. By restricting these operations, we ensure that the agent constructs the harness without altering the verification problem.

Tools. To support proof-harness engineering, we provide the agent with the following tools:

  • describe_schema: Describes the expected proof-harness format.

  • verify: Executes CBMC on the current harness and checks reported counterexamples against the original implementation. It reports safe, spurious counterexample, or reproduced violation.

  • validate: Checks whether the current dependency abstractions over-approximate the behavior of their original implementations. It reports sound or provides a counterexample demonstrating an unsound abstraction.

  • test: Executes the original implementation with concrete inputs chosen by the agent.

These tools allow the agent to perform the soundness and precision checks described in our approach. However, the availability of these tools does not guarantee that the agent invokes them correctly or sufficiently, nor that the resulting harness is sound and V\mathcal{V}-precise. We therefore employ external soundness and precision guards that enforce validation independently of the agent’s decisions and invoke further refinement when necessary.

Configuration. The full configuration of our harness agent used in our experiments is shown in Table 3. The prompt and othe relevant artifacts are available on Zenodo 7.

ParameterValue
Modelgpt-6-luna (via LiteLLM)
Reasoning effortoff
Temperature0; 0.7 for the step after a refused command
Max. output tokens per step8,192
Max. steps per run60
VerifierCBMC 5.95.1, --unwinding-assertions
Time limit per verifier call300 s
Repeated commandsrefused after 5 repetitions

Table 3. Agent configuration.

Soundness Guard

The soundness guard ensures that the agent does not introduce unsound restrictions when abstracting dependencies. For each dependency dd replaced by an abstraction AdA_d, we require that AdA_d over-approximates the behavior of dd (written as d⇒Add \Rightarrow A_d). This includes return values, but also the observable effect on the program state.

As shown in Algorithm 1, the guard validates each proposed dependency abstraction against its original implementation. If validation reveals a behavior of dd excluded by AdA_d, the resulting counterexample CEXd\mathrm{CEX}_d is returned to the agent for repair. The revised abstraction is checked again before being accepted. Only successfully validated abstractions are incorporated into the harness.

Algorithm 1 SoundnessGuard
Require: Software system S, target function f, proof harness H_f
Ensure: Proof harness H_f for f
1: D ← Dependencies(H_f)
2: for all d ∈ D do
3:     A_d ← GetAbstraction(H_f, d)
4:     CEX_d ← Validate(d, A_d) ▷ Check d ⇒ A_d
5:     while CEX_d ≠ ∅ do
6:         H_f ← AgentRepair(S, H_f, d, CEX_d)
7:         A_d ← GetAbstraction(H_f, d)
8:         CEX_d ← Validate(d, A_d)
9:     end while
10: end for
11: return H_f
Algorithm 2 PrecisionGuard
Require: Verifier V, software system S, target function f, proof harness H_f
Ensure: Verdict v, counterexample CEX
1: (verdict, CEX) ← V(H_f)
2: if verdict = Safe then
3:     return (Safe, ∅)
4: else if CEX ≠ ∅ then
5:     if Feasible(S, f, CEX) then
6:         return (Unsafe, CEX) ▷ Feasible counterexample
7:     else
8:         return (RefNeeded, CEX) ▷ Refinement required
9:     end if
10: else
11:     return (Unknown, ∅) ▷ Verifier returned unkown
12: end if

In practice, validation is performed by a verification task: the agent generates a small program that calls the original dd with arbitrary arguments and asserts AdA_d on its result, and runs the verifier on it. If the verifier finds arguments for which the assertion fails, dd can behave in a way that AdA_d excludes and these arguments are returned to the agent as CEXdCEX_d. If no such arguments are found, AdA_d is accepted and recorded. The agent only uses the accepted abstractions in its harness.

Our validation approach relies on an initially sound over-approximation and preserves this property through checked refinements. Under these conditions, the soundness of the resulting harness does not depend on the correctness of the agent’s reasoning.

Precision Guard

Soundness alone does not guarantee that a verifier can establish the desired property. An over-approximating harness may admit behaviors that cannot occur in the original system, potentially leading to spurious counterexamples. The precision guard therefore aims to refine the harness until it is sufficiently precise for the chosen verifier V\mathcal{V}.

As shown in Algorithm 2, the precision guard invokes V\mathcal{V} on the current harness HfH_f. If the verifier establishes the property, the process terminates successfully. Otherwise, a reported counterexample is checked by attempting to reproduce the violation through concrete execution of the target function. If the violation is reproduced, the counterexample is passed to the agent which reports it to the developer. Otherwise, the counterexample is returned to the harness agent to guide further refinement. Each proposed refinement is again subject to the soundness guard.

Importantly, this check does not establish whether the initial state of the counterexample is reachable in the original software system. We assume that developers encode the intended preconditions and correctness properties directly in the program. The harness agent therefore does not need to infer restrictions beyond those that real callers of the target visibly establish.

In our prototype, counterexamples are checked by replay. The tool extracts from the verifier’s trace the arbitrary values used by the verifier. The agent then compiles the driver against the real program and runs it with these arbitrary values from the verifier. If the same assertion fails, the violation is confirmed and reported. If the program runs without failure, the counterexample is spurious i.e. it relied on behaviour that only the over-approximated abstraction allows.

Conversely, when the verifier reports that no violation exists, the tool checks that at least one execution of the harness reaches the end of the created driver. Otherwise, the assumptions exclude all executions and the result would be vacuous, such results are rejected.

The process terminates when the verifier establishes the property, a counterexample is reproduced, or verification remains inconclusive. Thus, the harness need not represent the execution environment exactly; it only needs to be sufficiently precise for the verifier to establish a conclusive result.

Example of Abstraction Refinements

In Figure 3, we give the abstractions of the dependencies of target function f=addf = \texttt{add} of Figure 1 while constructing HfH_f. Function add calls one function toAdd given in Figure 3 (left). Function toAdd is supposed to bring the input value val into the range [−RANGE,+RANGE] plus make it positive when the build flag VAL_POS is set to 1. It, however, does not correctly achieve this because of an overflow when val equals SHRT_MIN. As a consequence, the assertion in target function add of Figure 1 (left) is violated.

Function toAdd and its abstractions in different refinements

Figure 3. Function toAdd and its abstractions in different refinements

For constructing the harness HfH_f for a makefile with VAL_POS=1 and RANGE=10, the harness agent starts with an overapproximation of toAdd just specifying

// REAL FUNCTION
short toAdd(short val) {
#if VAL_POS
    if (val < 0) {
        val = -val;
    }
#endif
    if (val < -RANGE) {
        return -RANGE;
    } else if (val > RANGE) {
        return RANGE;
    }
    return val;
}
// 1st abstraction
short toAdd(short val) {
    return nondet_short();
}
// 2nd abstraction
short toAdd(short val) {
    short r = nondet_short();
    assume(-RANGE <= r && r <= RANGE);
    return r;
}
// 3rd abstraction
short toAdd(short val) {
    short r = nondet_short();
    assume(r >= -RANGE && r <= RANGE &&
           (r >= 0 || r == -RANGE));
    return r;
}
// 4th abstraction
short toAdd(short val) {
    short r = nondet_short();
    assume((r >= 0 && r <= RANGE)
           || (val == SHRT_MIN
               && r == -RANGE));
    return r;
}

toAdd to return a value of type short (see Figure 3 (right), 1st abstraction). Step VALIDATE in Algorithm 1 sees this to be a valid overapproximation and does not enter its refinement loop. When invoking the verifier V\mathcal{V} on the current HfH_f in the precision guard (Algorithm 2), it however finds a counterexample (invocation with val=0, return value -67) leading to a violated assertion. This counterexample is, however, not feasible (in an execution of add with this input value using the real function toAdd, the assertion in the target function is not violated). The agent thus refines the abstraction of toAdd (see Figure 3 (right), 2nd abstraction). This abstraction again passes the soundness guard, but not the precision guard which produces another spurious counterexample (val=0, return -4). A further refinement produces the 3rd abstraction, again being sound, but not passing the precision guard (counterexample val=0, return -10). The fourth abstraction now passes the precision guard with a feasible counterexample (val=SHRT_MIN, return -10) and this completes the construction of harness HfH_f. The agent thus reports Unsafe and also gives harness and feasible counterexample to a developer for further inspection.

Note that in our example here, the soundness guard never needs to refine the abstraction; all abstractions are valid overapproximations of toAdd. In our initial experiments, we have however seen that some LLMs do provide wrong abstractions, making the usage of a soundness guard mandatory for sound harnesses.

Minimal Verifier-Specific Harness Adaption

Our agent supports multiple harness formats per verification task. Importantly, the agent does not produce verifier-specific harnesses, but provides us with sufficient context to construct them deterministically (e.g. target file, additional source files, dependency abstractions, and build flags). Therefore, the same harness can be reused with multiple verifiers by using minimal deterministic adaptions. For CBMC, used inside the agent’s loop, the harness additionally records every value of every symbolic input, so that counterexamples can be replayed. For CPAchecker and Ultimate Automizer, the tool generates an SV-COMP-style harness. It uses the standard __VERIFIER_* functions and a failing assertion calls reach_error. The program is preprocessed with GCC into a single file. For KLEE, the same harness is mapped to KLEE’s functions and preprocessed with clang. All tasks therefore share the same driver, target code and dependency abstractions. They only differ in how they express symbolic inputs, assumptions and failures.

Prompt

The prompt in Listing 1.1 summarises the harness format and the guidance the agent receives to generate a harness. The agent’s overall behaviour, including the task description and the rules it must follow, is set by a system prompt. The full prompt is part of the artifact.

Prompt given to the agent to generate the harness

Figure 1.1. Prompt given to the agent to generate the harness (excerpt).

Write two files in <project>/verification/<target>/:

spec.json    target and its source file; extra source files to include with their
             real code; headers; build flags; how to link the real program for
             replay; loop bound; mocks.
driver.c    a small program that calls the target the way a real caller would.

Driver
- Inputs that real callers may vary: unknown values, __VERIFIER_nondet_<type>().
  A fixed input checks only that one input and proves nothing.
- Guarantees that real callers provide: __VERIFIER_assume(<condition>).
- Keep inputs small, so that every loop fits within the bound.
- Derive the calling context from existing callers and tests.
- Do not add assertions; only the project’s own assertions are checked.
[...]

Dependencies
- Functions without code in the harness return ANY value. If such a function
  matters, include its source file or mock it.
- Mocks are allowed only for functions defined outside the target’s file, and only
  for number-valued parameters and return values. A mock returns an unknown value,
  optionally restricted by a rule: a C condition over the return value r and the
  parameter names.

Refinement (given through the hints after each tool result)
- Spurious counterexample: the tool argues a mock argument is received, the value
  it returned, and the value the real function returns. Propose a rule that
  excludes the mock’s value.
- Before use, every rule must pass "check" against the real function.
  Refuted: the hint shows an input where the real function breaks the rule;
  propose a weaker rule. Not refuted: add the rule to the spec and run again.
[...]

Finish with a verdict reported by a tool.

References

  1. [1]Dirk Beyer, Po-Chun Chien, Bo-Yuan Huang, Nian-Ze Lee, and Thomas Lemberger. A case study in firmware verification: Applying formal methods to Intel TDX module. In Sebastian Junges and Guy Katz, editors, TACAS , Part II, volume 16506 of Lecture Notes in Computer Science, pages 42–64. Springer, 2026. doi:10.1007/978-3-032-22749-2\_3.DOI
  2. [2]Dirk Beyer, Po-Chun Chien, Bo-Yuan Huang, Nian-Ze Lee, and Thomas Lemberger. HarnessForge: Automated Extraction of Verification Tasks from Industry-Scale Software Projects. In FSE Companion, pages 202–206. ACM, 2026. doi: 10.1145/3803437.3806420.DOI
  3. [3]Dirk Beyer and M. Erkan Keremoglu. CPACHECKER: A Tool for Configurable Software Verification. In Proc. CAV, LNCS 6806, pages 184–190. Springer, 2011. doi:10.1007/978-3-642-22110-1\_16.arxiv.org/abs/0902.0019
  4. [4]Dirk Beyer and Jan Strejcek. Evaluating Software Verifiers for C, Java, and SV-LIB - (Report on SV-COMP 2026). In Sebastian Junges and Guy Katz, editors, TACAS, volume 16506 of Lecture Notes in Computer Science, pages 461–502. Springer, 2026. doi:10.1007/978-3-032-22749-2\_23.
  5. [5]Islem Bouzenia, Cristian Cadar, and Michael Pradel. Evaluating LLM agents on automated software analysis tasks. CoRR, abs/2604.11270, 2026. URL: https://doi.org/10.48550/arXiv.2604.11270, arXiv:2604.11270, doi:10.48550/ARXIV.2604.11270.
  6. [6]Islem Bouzenia, Premkumar Devanbu, and Michael Pradel. RepairAgent: An Autonomous, LLM-Based Agent for Program Repair. In ICSE, pages 2188–2200, 2025. doi:10.1109/ICSE555347.2025.00157.arxiv.org/abs/2403.17134
  7. [7]Cristian Cadar and Martin Nowack. KLEE symbolic execution engine in 2019. Int. J. Softw. Tools Technol. Transf., 23(6):867–870, 2021. URL: https://doi.org/10.1007/s10009-020-00570-3, doi:10.1007/S10009-020-00570-3.
  8. [8]Casey Casalnuovo, Premkumar T. Devanbu, Abílio Oliveira, Vladimir Filkov, and Baishakhi Ray. Assert use in github projects. In Antonia Bertolino, Gerardo Canfora, and Sebastian G. Elbaum, editors, 37th IEEE/ACM International Conference on Software Engineering, ICSE 2015, Florence, Italy, May 16-24, 2015, Volume 1, pages 755–766. IEEE Computer Society, 2015. doi:10.1109/ICSE.2015.88.DOI
  9. [9]Nathan Chong, Byron Cook, Jonathan Feldman, Konstantinos Kallas, Kareem Khazem, Felipe R. Monteiro, Daniel Schwartz-Narbonne, Serdar Tasiran, Michael Tautsch­nig, and Mark R. Tuttle. Code-level model checking in the software development workflow at Amazon Web Services. Softw. Pract. Exp., 51(4):772–797, 2021. URL: https://doi.org/10.1002/spe.2949, doi:10.1002/SPE.2949.
  10. [10]Haoran Ding, Zhaoguo Wang, and Haibo Chen. FM-Agent: Scaling Formal Methods to Large Systems via LLM-Based Hoare-Style Reasoning. CoRR, abs/2604.11556, 2026. URL: https://arxiv.org/abs/2604.11556, arXiv:2604.11556.
  11. [11]Frank Schüssele, Manuel Bentele, Daniel Dietsch, Matthias Heizmann, Xinyu Jiang, Dominik Klumpp, and Andreas Podelski. Ultimate Automizer and the Abstraction of Bitwise Operations - (Competition Contribution). In TACAS (3), volume 14572 of Lecture Notes in Computer Science, pages 418–423. Springer, 2024. doi:10.1007/978-3-031-57256-2\_31.DOI
  12. [12]Alessandro Sosso, Akhil Arora, and Bas Spitters. Agentic proving for program verification. CoRR, abs/2605.23772, 2026. URL: https://arxiv.org/abs/2605.23772, arXiv:2605.23772. R.K. Shanmuganathan, C. Richter, H. Wehrheim
  13. [13]Youcheng Sun, Jiawen Liu, Daniel Kroening, and Jason Xue. Agentic Model Checking. CoRR, abs/2605.21434, 2026. URL: https://arxiv.org/abs/2605.21434, arXiv:2605.21434.
  14. [14]Haoxin Tu, Huan Zhao, Yahui Song, Mehtab Zafar, Ruijie Meng, and Abhik Roychoudhury. Agentic verification of software systems. Proceedings of the ACM on Software Engineering, 3(FSE):3558–3581, 2026. URL: http://dx.doi.org/10.1145/3808164, doi:10.1145/3808164.
  15. [15]John Yang, Carlos E Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik R Narasimhan, and Ofir Press. SWE-agent: Agent-computer interfaces enable automated software engineering. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL: https://arxiv.org/abs/2405.15793.

Paper details

vlnc:W_t3mgakq3
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY-SA 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents