betaFeedback

WorksComputer Science

FCP: Formal Context Protocol for Verifying LLM-Generated Artifacts

Abstract

When a language model writes both an artifact and the tests meant to check it, a passing test is weak evidence: one misreading of the request yields a faulty artifact and a check that accepts it. We present the Formal Context Protocol (FCP), which checks AI-generated artifacts against the request itself. FCP separates conformance (does the artifact satisfy a formal reading of a requirement?), which model checkers and SMT solvers decide, from fidelity (does the reading mean what the prose means?), which has no decision procedure but is refutable by a concrete witness. Eight ways a proof can falsely certify a requirement each become a mechanical check that returns a witness, and what no machine can settle becomes concrete questions for the request's author. Adapters place hardware, text, agents and code on the same engines. On four hardware designs, trusting AI-written assertions certified 23% of violated requirements, and FCP certified none. On ten code tasks, code from three language models passed its own tests and violated the request every time. Trusting the tests certified all 33 violations, and FCP certified three. Agreement among models, readings or tests did not imply correctness.

Introduction

A developer who asks an AI system for a design, an answer or a program receives more output than line-by-line review can cover, and each established safeguard has a known limitation. Tests and assertions written by the same system are circular: one misreading produces both the wrong artifact and the check that accepts it, and language models do not reliably correct their own reasoning without external feedback [30]. An LLM judge returns a single verdict, can be manipulated by the text it evaluates [57], and favors its own generations [54]. Formal verification removes the sampling limit of testing and makes the property the critical object: a property that misstates the requirement, once proved, gives false assurance for every behavior.

Figure 1 gives a small example. A store asks for a function checkout(subtotal, coupon, member) with seven rules, two of which cannot both hold: shipping is free from $50, and adding items to the cart must never lower the amount charged. A $49.99 cart costs $54.98 with shipping, and a $50.00 cart costs $50.00. We gave the prompt to three language models. Each model identified the conflict, resolved it differently, wrote between 134 and 745 tests, and passed all of them. Each implementation violates a rule of the prompt, and a flow that certifies a requirement whenever the model’s own tests pass certifies all three violations.

Three line plots showing Sonnet, Opus, and Haiku resolutions of the shipping-fee conflict; y-axis amount charged, x-axis cart subtotal, no coupon, not a member. Sonnet: “breaks rule 7: \$4.98 less for one cent more”; Opus: “breaks rule 6: the shipping fee shrinks to zero”; Haiku: “breaks rule 6: a fee is still charged from \$50”.

Figure 1. Three resolutions of an unsatisfiable prompt (§ 8.5). Rule 6 (free shipping from $50) and rule 7 (adding items never lowers the amount charged) cannot both hold: as written, the charge falls from $54.98 to $50.00 between subtotals of $49.99 and $50.00 (thin gray line). Shading marks the subtotals on which a model departs from the rules as written. FCP finds the calls at 4,999 and 5,000 cents before any code exists. All three models identified the conflict and passed their own tests, and each violated a rule.

This paper presents the Formal Context Protocol (FCP). Its goal is to let a person establish whether an AI-generated artifact meets the request, and where it falls short, without reading the artifact. FCP checks the artifact against the request: it proves what holds, replays every failure on the artifact itself, and reduces the rest to concrete questions that only the request’s author can answer. On the checkout prompt, FCP finds the conflicting pair of calls before any code exists, certifies none of the three violations, and asks three questions: which rule yields at the threshold, which amount the $50 refers to, and how half a cent is rounded.

The central difficulty is that the meaning of prose cannot be checked mechanically. FCP therefore separates the two questions on which every verification claim rests. Conformance asks whether the artifact satisfies a formal reading of a requirement; standard engines decide it. Fidelity asks whether the reading means what the prose means; no procedure decides it, and it is refutable: one concrete behavior on which two independent readings disagree refutes at least one, and their agreement corroborates both. We identify eight situations in which a proof certifies a requirement falsely and give each a mechanical witness (Table 1). Only fidelity ends in a question, phrased as an example: a waveform, a conversation or a function call.

Situation in which a proof of M⊨(A⇒φ)M \models(A \Rightarrow\varphi) certifies rr falselyMechanical witness returned by FCP
Vφ\varphi never triggers on MM under AA, or holds on every behavior that AA allowsproof that the trigger is unreachable; proof of φ\varphi on SΣS_{\Sigma} (tautology)
EAA excludes behaviors of MM itself (assumptions may restrict only the environment)reachable state with no legal non-reset input (non-receptiveness)
F(A,φ)(A,\varphi) differs from the meaning of rrbehavior accepted by one independent reading and rejected by another, in the context of the other requirements
Cthe requirements contradict each otherinput sequence on which no output sequence meets every guarantee (unrealizability)
Uthe request does not determine some outputtwo conforming behaviors with equal history that differ in an output
Wthe properties are too weak to notice a change of MMobservable mutant of MM that no property kills
Da rule holds only because a black-box component, such as an agent’s model, behavesviolation with the component’s output unconstrained that disappears under its contract
Tan engine or a model of the artifact is wrongdisagreement between engines (PDR and BMC); mismatch on replay in a simulator

Table 1. Situations in which a proof certifies a requirement falsely, and the witness FCP returns for each. Fidelity (F) is the one situation without a mechanical decision; its witness is a concrete behavior for a person to judge.

Contributions

  • A decomposition of false assurance into eight situations, each with a mechanical witness, and a protocol that produces these witnesses: a provenance-anchored intermediate representation, isolated independent formalizers, and verdicts that carry witnesses (§ 3 and § 4).

  • Analyses with precise semantics: conflicts as unrealizability witnesses, forcing stimuli, determinacy, and comparison of readings as assume-guarantee contracts in the context of the other requirements. In the hardware study, contracts are needed for soundness and context raises acceptance 4.4-fold (§ 5).

  • Adapters that place four kinds of artifact on the same engines (§ 6). Text becomes a labeled trace. An agent is checked with its model as a cut point, which assigns every rule a trust class: enforced, delegated or broken. Code is run on an exhaustive scope derived from the request, for functions, strings and lists, objects, changes to existing code, code that acts through services and files, and records. For functions, comparing two readings reduces to a bounded number of calls (Proposition 3).

  • Proofs beyond the scope: translation-validated symbolic execution deciding code requirements for every input, every script of service answers, and call sequences of every length (§ 7).

  • An evaluation on four hardware designs, a prompt, an agent and ten code tasks, in which the formalizers, RTL generators, agents and code writers are language models (§ 8). Self-check baselines certified 23% of violated hardware requirements and all 33 violations in code. FCP certified none of the hardware violations and 3 of the 33 code violations, which an additional reading of the ambiguous phrase removes. Beyond the scope, 2,245 of 2,263 verdicts are decided for every input and agree with the scope; eleven hold only up to a bound, four of which fail beyond it, and seven fail at inputs the scope missed.

One finding recurs across our studies: each form of agreement we measured (code with its author’s tests, three models’ code on millions of calls, two isolated formalizers, two labelers) held at least once on an incorrect result. FCP is therefore designed around disagreement, which it turns into a concrete witness (§ 9).

Problem Setting

Traces, properties and contracts. An interface Σ=(I,O)\Sigma= (I, O) consists of input and output signals. A trace is a sequence π:N→Val⁡(I∪O)\pi: \mathbb{N} \to\operatorname{Val}(I \cup O) in which reset is asserted at cycle 0. An artifact MM denotes a set of traces [ ⁣[M] ⁣][\![M]\!]. A safety property φ\varphi is recognized by a deterministic monitor [40] with a verdict ok⁡φt(π)\operatorname{ok}_{\varphi}^{t}(\pi) at each cycle tt, and π⊨φ\pi\models\varphi iff ok⁡φt(π)\operatorname{ok}_{\varphi}^{t}(\pi) holds for all tt. The spec-only model SΣS_{\Sigma} contains every trace over Σ\Sigma.

Definition 1 (Assume-guarantee satisfaction). For assumptions AA and a property φ\varphi, M⊨(A⇒φ)M \models(A \Rightarrow\varphi) iff for every π∈[ ⁣[M] ⁣]\pi\in[\![M]\!] and every tt, if ok⁡At′(π)\operatorname{ok}_{A}^{t'}(\pi) holds for all t′≤tt' \le t, then ok⁡φt(π)\operatorname{ok}_{\varphi}^{t}(\pi) holds.

Readings and false assurance. A request is decomposed into atomic requirements. For a requirement rr, a reading is a pair (Ar,φr)(A_r,\varphi_r) of assumptions and a property, written by a formalizer: a person or a language model. The contract Cr=(Ar⇒φr)C^r = (A_r \Rightarrow\varphi_r) denotes the set of traces that satisfy it in the sense of Definition 1.

Definition 2 (False assurance). Let (Ar⋆,φr⋆)(A_r^\star,\varphi_r^\star) be a reading that captures the author’s intent, the gold reading. A flow gives false assurance on (M,r)(M,r) if it certifies rr for MM and M⊭(Ar⋆⇒φr⋆)M \not\models(A_r^\star\Rightarrow\varphi_r^\star).

The gold reading is used only in the evaluation; our gold readings are validated mechanically.

Conformance and fidelity. Verification decides conformance, M⊨(Ar⇒φr)M \models(A_r \Rightarrow\varphi_r). A proof of conformance is evidence for rr only if the reading is faithful to the prose, and it carries no evidence in any situation of Table 1. Every situation other than F concerns objects that FCP holds (the artifact, the readings and the interface), so each has a mechanical witness. Because rr is prose, fidelity admits no decision procedure, and it is refutable. If two independently written readings disagree, one behavior accepted by one reading and rejected by the other shows that at least one of them is unfaithful, and the witness a person then judges is a six-cycle waveform or a two-call example. Agreement between independent readings corroborates both, in the tradition of N-version programming [2]. The known caveat of that tradition also applies: independently produced versions fail together more often than independence would predict [38]. Our studies measure this effect for language models (§ 8.8).

The Formal Context Protocol

Figure 2 shows the process: language models decompose and formalize the request, and deterministic code compiles the readings, runs the analyses and reports each requirement as holding, violated or a question.

The FCP process, showing request specification, requirements, isolated formalizers, compilation, artifact checks, off-the-shelf engines, and per-requirement reporting

Figure 2. The FCP process. Dashed boxes are written by AI systems and are untrusted; solid boxes are deterministic code; shaded boxes are off-the-shelf engines. Formalizers are not given the artifact or each other’s work. The checks on the left need no artifact, so a request is analyzed before any artifact exists.

Intermediate representation. The IR has three layers, each written in YAML and validated by a JSON Schema. The design manifest declares the interface, environment defaults, a frame (properties that every behavior satisfies by construction of the encoding), and one of: RTL variants, a cut point (§ 6.2) or a function (§ 6.3). Requirements are atomic; each carries a verbatim quote, a kind (guarantee, assumption, scenario, definition or non-functional) and any recorded ambiguities. Line spans are computed from the quotes and re-verified on every load, so provenance cannot be fabricated. Readings are sets of properties, assumptions, scenarios and don’t-cares written in FCP-PL, with ghost state, rigid (universally quantified) variables and named definitions. A requirement left unformalized must state why.

FCP-PL. The property language is a past-time temporal logic over Booleans, bit-vectors and integers with statically computed ranges (Appendix § 16). Arithmetic is exact: every subexpression is compiled to a two’s-complement wire wide enough for its interval, so count −1 evaluates to −1 at zero, with no wrap-around. Logical operators require one-bit operands. Future obligations exist only inside patterns (next, within, hold_until, bounded_wait, inorder), whose monitors flag the minimal bad prefix. A prev that reaches before the current reset segment disables evaluation, so that no arbitrary state is read, and a trigger that looks back nn cycles cannot arm an obligation in the first nn cycles. Every property is compiled twice: an assert form, which may use nondeterminism (for instance, Wolper’s data-independence argument [65] for inorder), and a deterministic form for every analysis that assumes the property.

Design rules. Four rules shape the protocol. (R1) Separation of roles. Language models write only requirements, readings, labels and adapters. A deterministic compiler and standard engines decide everything else, and a circuit proof counts only if two engines agree. (R2) Isolation. Formalizers are not given the artifact or each other’s work. (R3) Black-box properties. Readings refer only to interface signals and ghost state, so a reading applies to any implementation and can be analyzed before one exists. (R4) Witness-carrying verdicts. Every verdict other than a proof carries a concrete trace that a person can judge.

Verdicts. Each requirement receives a level stating what has been established (Table 2). It is ASSURED only if it is proved, its trigger is witnessed, the environment is receptive, and an independent reading is equivalent to it as a contract. Every other requirement enters a review queue with its witness.

LevelMeaning and attached evidence
ASSUREDproved by two engines; trigger witnessed; assumptions receptive; an independent reading is an equivalent contract (in context, § 5)
PROVED-DISPUTEDproved, and an independent reading differs; the simplest distinguishing behavior is attached
PROVED-CONFLICTEDproved, and the requirement belongs to a minimal conflict; the conflicting stimulus is attached
VACUOUStrigger unreachable under the assumptions (proof attached), or the property is a tautology
ENV-UNSOUNDthe assumptions are not receptive; a blocking state is attached
FAILEDcounterexample replayed in simulation and triaged to an artifact deviation or a conflict
BOUNDED, UNKNOWNthe engines did not conclude; never counted as a proof
UNFORMALIZEDno property; the formalizer’s reason is recorded

Table 2. Verdict levels. For code, a requirement is certified if it holds on the exhaustive scope under both independent readings, the readings are equivalent in context, it lies in no conflict, and its text carries no ambiguity flag; it is proved if it also holds for all inputs, or PROVED-BOUNDED if for all call sequences up to a bound (§ 7).

Analyses

All analyses are generated harnesses around the same compiled monitors, run by SymbiYosys [69] with ABC PDR [12, 26] and Boolector BMC [9, 13], or by z3 [22] on the SMT2 model from yosys [64].

Verification and replay. PDR and BMC run independently; a PDR pass together with a BMC failure is an engine disagreement and does not count as a proof. Every counterexample is replayed in Icarus Verilog [63] from its witness and compared cycle by cycle.

Vacuity and tautology. For a property with a trigger, a cover on MM under AA yields either a witness or a PDR proof that the trigger is unreachable, in which case the property is VACUOUS [6, 41]. Independently, ΣE⊨(A⇒φ)\Sigma_E \models(A \Rightarrow\varphi) means that φ\varphi constrains nothing.

Receptiveness. An assumption is legitimate only if it restricts the environment. A blocking state is a reachable state ss such that ¬A(s,i)\neg A(s,i) for every i∈Val⁡(I)i \in\operatorname{Val}(I) with i(reset⁡)=0i(\operatorname{reset}) = 0. Finding one is an exists-forall query, which FCP solves by counterexample-guided refinement.

Definition 3 (Conflict). A finite input sequence σ∈Val⁡(I)k+1\sigma\in\operatorname{Val}(I)^{k+1} is a conflict of a set of readings if, for every output sequence τ\tau, some guarantee fails on (σ,τ)(\sigma,\tau) at a cycle t≤kt \le k up to which every assumption held.

A conflict is an unrealizability witness [55]: an environment that plays σ\sigma defeats every implementation (Proposition 1). A simpler candidate definition, “a trigger situation that cannot occur under all the requirements”, also flags situations that a specification legitimately excludes; on a consistent bridge specification, it reported dozens of spurious conflicts. FCP draws candidates from every trigger situation, every pair of requirements and every scenario, and extends each by a lookahead L=1+ωL = 1 + \omega, where ω\omega is the longest obligation window; the lookahead is required because a prefix that shows no violation so far may already admit no conforming continuation. A minimized unsatisfiable core names the minimal set of conflicting requirements. Forcing searches for σ\sigma directly, by exists-forall counterexample-guided synthesis over output sequences. Both searches are sound for the conflicts they report and incomplete against adaptive environments.

Proposition 1 (Reported conflicts are sound). If σ\sigma is a conflict of a set of readings, every artifact that accepts σ\sigma violates one of their guarantees on σ\sigma, at a cycle up to which all assumptions have held.

Determinacy. By self-composition [5, 20], FCP searches for traces π,π′\pi,\pi' that both conform, agree on all signals before cycle tt and on the inputs at tt, and differ at tt in an output that is not a declared don’t-care. Such a pair is a concrete instance of under-specification.

Definition 4 (Contract comparison in context). Let Cxr=(Ax⇒φxr)C_x^r = (A_x \Rightarrow\varphi_x^r) be the contract of reading xx for requirement rr. Reading aa implies bb on rr iff Car⊆CbrC_a^r \subseteq C_b^r over SΣS_\Sigma, where witnesses must be extendable by LL cycles. In context, both sides are intersected with Kr=⋂r′≠r(Car′∩Cbr′)K^r = \bigcap_{r' \ne r}(C_a^{r'} \cap C_b^{r'}), the behaviors that meet every other requirement under both readings. The relation is EQUIVALENT, STRONGER, WEAKER or INCOMPARABLE, and each failed direction yields the simplest distinguishing behavior.

Soundness requires comparing contracts, each property under its own reading’s assumptions. In an earlier version, which pooled the assumptions, one formalizer’s masking assumption entered the agreement check and caused the only false assurances that version gave (§ 8.2). Comparing in context removes disputes that arise only on behaviors that other requirements already exclude.

Triage and mutation. For a counterexample, FCP asks whether any conforming output sequence exists for the same inputs. If so, MaxSAT finds the closest and localizes the defect to a (cycle, signal, expected, actual) difference. If none exists, even under assume-guarantee semantics, the requirements conflict on that stimulus and an unsatisfiable core names them. Mutation coverage [17, 33] checks every property on every observable mutant of MM; a surviving mutant marks, at a source line, behavior that no requirement constrains.

Adapters: Every Artifact as a Circuit

The engines check circuits; an adapter for each kind of artifact determines how exact the verdict is.

Text responses

A response is split into units (sentences or lines), one per cycle. Each unit receives labels (atomic propositions such as “states a feature”), and the prompt becomes a small automaton over them. The factorization hypothesis is that the author’s intent depends on the response only through its label trace. Labels are assigned by two isolated labelers and form the trust boundary: a verdict is exact relative to its labels, and disagreements between labelers are surfaced like disagreements between readings. The analyses also test the label semantics: the first consistency check produced a “conforming” email that hid benefit statements inside closing units, which no real text does. A frame rule now forbids any other label on a closing unit.

Agents: the model as a cut point

A tool-using agent runs a loop: the model proposes a call, the scaffold decides what runs, and the result feeds the next turn. We write the scaffold’s decision logic once, in a Python subset, run it in the agent, and translate it to Verilog, checked by co-simulation. The model’s proposal XX is a free input, so a proof covers every output of the model, including one that follows instructions injected into a tool result [29] (Figure 3).

An agent checked as a circuit

Figure 3. An agent checked as a circuit. One turn is one cycle. An enforced rule holds for every call the model could propose, on any input the model reads, including text planted in a tool result.

Definition 5 (Trust classes). Let passo(X)pass_o(X) be the value that output oo takes when the scaffold passes the proposal through unchanged, and let the model contract C^r\hat{C}_r be φr\varphi_r with every current-cycle output oo outside temporal operators replaced by passo(X)pass_o(X). With XX free, rr is enforced if M⊨(A⇒φr)M \models(A \Rightarrow\varphi_r); delegated if M⊭(A⇒φr)M \not\models(A \Rightarrow\varphi_r) and M⊨(A∧C^r⇒φr)M \models(A \land\hat{C}_r \Rightarrow\varphi_r); and broken otherwise.

Proposition 2 (Meaning of the trust classes). If rr is enforced, every run of the agent satisfies rr under AA, for all proposals of the model. If rr is delegated, every run satisfies rr under AA up to the first turn in which the model’s proposal violates C^r\hat{C}_r.

The delegated rules are the agent’s residual trust in its model. A counterexample on a pass-through scaffold identifies the guard that is needed; a counterexample that survives the model contract is a scaffold bug; and recorded runs are checked against the contracts by runtime verification [43]. The construction is related to shield synthesis [10], with the shield written by hand and its guarantees proved, and to designs that confine an agent’s model by construction [25].

Code: a function as a circuit without memory

Code written by an assistant does not, in general, compile to a circuit. FCP therefore treats the two sides separately: the request side stays on the circuit engines, and the code is executed.

Interface. A function f:D1×⋯×Dn→Rf : D_1 \times\cdots\times D_n \to R becomes a design with one input signal per argument. Its outputs are the returned values (one signal per tuple element) and an outcome code ω\omega: 0 if the function returned values that fit their signals; 1, …\ldots, mm for the listed exceptions; m+1m+1 for any other exception, or no answer within the time limit; and m+2m+2 for any other return value, such as a float where an integer is due. Each cycle is one call, calls are independent, and the checker chooses them and their order. A requirement that relates two calls, such as monotonicity, is a property over consecutive cycles that uses prev.

Scope and evidence. The code is called on a scope DsD_s that contains, for each argument, a dense range; every integer constant of the readings, the prompt and the code, together with its neighbors c−1c-1 and c+1c+1; the ends of the signal’s range; and random values. Calls are ordered in sweeps, so that calls that differ by one scope step in one argument are adjacent; on a smaller scope the sweeps also run in reverse. The compiled monitors check the recorded trace, and every failure is replayed in a fresh process. The evidence is exhaustive within DsD_s, in the small-scope tradition [32], and makes no claim beyond DsD_s; § 7 closes that gap where the code permits. One-operator mutants of the code are run on the scope, and an observable mutant that changes no verdict marks behavior that no rule checks (Figure 4).

The artifact side of the code adapter

Figure 4. The artifact side of the code adapter. The request side (conflicts, open choices and comparison of readings) runs on the readings alone, before any code exists.

Proposition 3 (Bounded horizon). Let readings aa and bb of a function consist of implies and invariant items without assumptions, ghost state or unbounded history operators, each looking back at most dd calls. Then aa and bb disagree on a requirement rr, plainly or in context, on some trace iff they disagree on a trace of at most d+1d+1 calls.

An item that relates a call to its successor looks across two calls, and the horizon counts both; an adversarial review of the engine (§ 8.8) found that the implementation had omitted the trigger call of such items, and the corrected horizon moves no stored result. The proof (Appendix § 15) rests on two facts: the function has no state, so any window of calls is itself a trace, and a look-back that reaches before reset disables an item, which then holds. The readings of the checkout task satisfy the hypotheses. Proposition 3 reduces comparison of such readings to a single solver query over d+1d + 1 calls. Bit-level engines time out on this query: two readings that write the same product differently, such as s⋅(100−p)s \cdot(100 - p) and (100−p)⋅s(100 - p) \cdot s with p=min⁡(c+10m,30)p = \min(c + 10m, 30), pose a multiplier-equivalence problem. FCP therefore also encodes FCP-PL directly in integer arithmetic, where both expressions become the same polynomial. This encoding is a second implementation of the semantics, so every witness it finds is confirmed on the compiled monitors, and every agreement is spot-checked on the bit-level model.

Boundary conflicts. Trigger-situation candidates take arbitrary values, and forcing does not converge over wide arithmetic inputs. FCP therefore also builds pairs of calls from the constants of the readings and the prompt: consecutive boundary values of one argument, in both orders, with the other arguments at their constants. For each pair, an incremental solver asks whether any pair of outputs meets every property (Definition 3 with k=2k = 2) and returns a minimal core. Such conflicts are sound by Proposition 1.

Open choices and code splits. For each output that determinacy finds open, FCP reports the simplest call on which two values are allowed. When several implementations of one prompt exist, the calls on which they answer differently are grouped by how they split the implementations, a form of differential testing [47]. Each split is a decision that the prompt left to the author of the code.

Beyond functions: strings, objects, changes, effects and records

Strings and lists. A string or list argument is a bounded sequence s=(ℓ,⟨s0,…,sn−1⟩)s = (\ell, \langle s_0, \ldots, s_{n-1}\rangle) of at most nn slots of ww bits, carried by two ports. The monitors read it in canonical form, sk:=0s_k := 0 for k≥ℓk \ge\ell, so nothing a port carries beyond the length can change a verdict, and the spec-only harnesses constrain free sequence ports to that form. FCP-PL gains len⁡(s)\operatorname{len}(s), s[i]s[i] (zero outside ss), x∈sx \in s, equality of sequences, string and list literals, ASCII character tests, and bounded quantifiers allall, anyany, countcount and sumsum over i<bi < b, where bb has a static maximum bˉ≤256\bar{b} \le256. The compiler expands a quantifier into bˉ\bar{b} guarded terms, for example all(i<b:ϕ)=⋀k<bˉ(k<b→ϕ[k/i])all(i < b: \phi) = \bigwedge_{k<\bar{b}}(k < b \rightarrow\phi[k/i]), so a sequence property remains a combinational function of one cycle and every engine applies unchanged. The integer encoding represents ss by ℓ,s0,…,sn−1\ell, s_0, \ldots, s_{n-1} with 0≤ℓ≤n0 \le\ell\le n and k≥ℓ→sk=0k \ge\ell\rightarrow s_k = 0.

A scope drawn from the request. Strings have too many values for dense ranges, and the relevant boundaries are often properties of the content: a card number is valid from 12 digits, at any length. Let A\mathcal{A} contain, for every comparison x⋈yx \bowtie y of two integers in any formula, the atoms x−y=δx - y = \delta for δ∈{−1,0,1}\delta\in\{-1, 0, 1\}; for every other condition cc, both cc and ¬c\neg c; and ℓ=n\ell= n for every length nn that the readings name. Let CC contain truetrue and the trigger of every rule. For each (C,α)∈C×A(C,\alpha) \in C \times\mathcal{A}, Z3Z3 solves C∧αC \land\alpha in the integer encoding with the outputs free and completes the input at random from an alphabet that contains the characters of the readings and the code and one character of each class. Each such call is followed by its neighbors at edit distance one, which cross the boundary on which α\alpha lies, and then by all short and some random sequences: constrained-random stimulus, as in hardware verification, with the constraints taken from the request. A rule on a computed value, such as a subtotal of $10.00 or more, yields no input at its boundary when the outputs are free. Every atom that mentions an output is therefore also solved with the outputs obeying one reading (the consequent of each item whose trigger holds), so that the inputs produce exactly the boundary value. A conflict, once found, is shrunk to the simplest stimulus that still conflicts.

Objects. For a method of a class, reset creates the object and every later cycle calls the method on it. The object’s state stays inside the code, like unnamed registers of an RTL design, and readings may keep a specification model in ghost state. The scope is a set of call sequences: for every condition in AA, the shortest sequence (found by bounded model checking on SΣS_{\Sigma}) after which the condition holds, with every rule satisfied up to that point, together with short and random walks and their one-edit neighbors. A rule quantified over a large rigid range, such as every customer, has no expansion that can be assumed, so the conflict search assumes it at the values to which a candidate sequence binds the rigid variable, one monitor instance each. Because ∀u φ(u)\forall u\ \varphi(u) implies every pinned instance, a stimulus on which no output meets the pinned rules is a conflict for the full rules (Corollary 1).

Several methods. A class with several methods is one circuit. An opop input selects the method of each call, methods may share an argument signal, arguments that the called method does not take are 0 in every engine, and each method has its own result signals. The class may belong to a package of several modules.

Changes. A change request has two specifications: the request itself, and the code before the change for everything the request does not touch. With a baseline, the old code runs on the same calls and its answers drive additional input signals, so a rule can relate the new answer to the old one, as in ¬student→price=old_price∧…\neg\mathit{student} \to\mathit{price} = \mathit{old\_price} \land\ldots. A regression is then a call on which no rule permits the difference. On the code side and in the conflict search, the old answers are those of the old code; in the spec-only analyses they are free, so readings are compared for any old code.

Effects and records. Code that acts through a service, received as an argument or imported by name, is given a recording stand-in. Every call it makes is an output event, every answer is a scripted input, and a run longer than a bound ends as “did not finish”, so that a rule such as “keep trying until it succeeds” can fail. Files the code opens become stand-in handles whose methods are events, with a per-run virtual disk. A record is one signal per field, and a list of records is one sequence per field, the fields forming a group of equal length in every engine; the runner builds the dictionaries or dataclasses and decomposes the result.

Engines. Unrolled quantifiers yield monitors of tens of thousands of lines, so traces are replayed on a cached Verilator build of each module, which agreed with Icarus Verilog on every check we compared and is about 400×\times faster on string monitors. A comparison that the integer encoding cannot decide within a minute is checked call by call on the scope and then on the bit-level model; a comparison that both leave open stays UNKNOWN and does not count as agreement.

Proofs Beyond the Scope

Scope evidence is exhaustive on its calls and makes no claim about other inputs. For code in a translatable subset of Python, FCP decides each requirement for every input the interface can carry (Figure 5).

Proofs beyond the scope

Figure 5. Proofs beyond the scope. The translation T(f)T(f) and the encoding EE are second semantics: each must reproduce the code or the monitors on every call of the scope before z3 is asked, and a mismatch blocks the proof.

Symbolic execution. FCP translates the function’s source into z3 terms by symbolic execution [14, 37], following every path with its condition, so that the outcome and returned values of one call become terms over the arguments. The subset covers module constants; helper functions, inlined, including ones that raise; isinstance and type() checks, decided by the declared types; loops over range() or a literal tuple; comparisons, and/or and conditional expressions; min, max, abs and divmod; and // and % by positive constants. FCP refuses, without approximation, floats, values whose Python type depends on the path (such as x<5x < 5 and FEE), regular expressions, and calls to any other function.

Two second semantics, validated. The translation is a second implementation of a subset of Python, so it is checked before it is trusted, in the spirit of translation validation [52, 56]: on every call of the scope, its outcome and values must equal what the code returned. Compiled back to Python, the terms take about a second on the 1.36 million calls of the checkout scope. The integer encoding of the reading is a second implementation of FCP-PL and is checked the same way: on every cycle of the scope trace, it must report a failure exactly where the compiled monitor does. A single difference in either check blocks the proof.

The query. The translation, joined to the integer encoding over kk calls, poses the question: do some kk calls, anywhere in the interface’s ranges, violate rr? If z3 answers no and the horizon is exact, rr is PROVED for every call the interface can carry. If z3 answers yes, its calls are run on the real code and the resulting trace is checked by the compiled monitors before the verdict FAILED is reported.

Proposition 4 (Soundness of proofs beyond the scope). (i) Every FAILED verdict is witnessed by a run of the code that the compiled monitors reject. (ii) If T(f)T(f) agrees with ff on every input within the interface’s ranges and EE agrees with the compiled monitors on every trace, then a PROVED verdict implies that no trace of ff violates rr, and a PROVED-BOUNDED verdict implies that no sequence of at most kk calls does.

Part (ii) is conditional. Validation on the scope is evidence, short of a proof, that the two second semantics are faithful § 11. Part (ii) also takes z3’s answers as given, so each formula z3 finds unsatisfiable is decided again by cvc5 [3], which confirms all 1,528 proofs of Table 8 and refutes none.

TaskScopePairs translatedVerdictsProvedFail in bothMisseds per pair
Checkout1,355,700 calls30 / 362601986206 to 7
Card masking131,613 calls36 / 392401617546 to 97
Rate limiter175,528 calls39 / 3920858 + 7{sup}†`140328 to 734
Student pricing3,601,728 calls36 / 3920413965024 to 27
Payment retry50,000 runs39 / 393642937101 to 2
Atomic save1,441 runs36 / 392762086800.3 to 0.5
Gift card65,771 calls48 / 52360182 + 4{sup}†`17400.6 to 57
Order placement46,400 runs39 / 393512787301 to 2
Total303 / 3222,2631,517 + 11{sup}†`7287

Table 8. Proofs beyond the scope (RQ4). A pair is an implementation (the three models, the reference and the planted variants) under one reading. A failing proof carries inputs chosen by z3, replayed on the code; “Missed”: requirements that pass the scope and fail the proof. The scope is recomputed for the proofs. †PROVED-BOUNDED: no sequence of up to 10 calls (gift card) or 8 to 15 calls (rate limiter) fails; the four on the gift card fail on longer ones. The other proofs on these two objects cover call sequences of every length.

Strings and equivalence sweeping. In a function over strings or lists, a sequence is a list of guarded slots (a condition and a value each), so filters, loops and slices remain terms; character tests such as str.isdigit are Python’s own answers on the 256 codes the interface carries. The query then sets two computations of one quantity side by side, a checksum as the code computes it and as the reading defines it, which z3 does not decide as a whole. FCP applies equivalence sweeping, the method of hardware equivalence checkers [39]: subterms that agree on sample calls are candidate equalities, and z3 proves each from the leaves up with the proved ones substituted; a refuting model becomes a new sample. Every merge is proved, so samples affect only speed. In a goal that remains open, the small conjuncts (the facts of the path that returned) are assumed and the rest is swept again; a goal without such facts is proved outcome by outcome. The same sweep decides comparisons of readings that define one quantity in two ways.

Effects, files and objects. For code with effects, each call on the service interface becomes an event in the translation, and each call that receives an answer takes the next answer of the script as a free variable; an answer delivered as an exception forks the path. Paths that made different calls stay apart, and a run lasts at most a bounded number of calls. The integer encoding gains ghost state, truncated to the ghost’s width as in the compiled monitors, and the next, after_reset, once, hist and since forms. Files the code opens are objects with their own events: open returns a handle numbered within the path, with closes it on every exit, a call on a closed file raises ValueError, a try statement resumes from the state in which the exception was raised, and each path keeps its own view of the disk, from which os.path.exists answers. For objects with several methods, the constructor yields the fields, each call runs the method that opop selects, and every way the method can end is merged into the fields after it. A call is then one step from fields and arguments to fields and results, and requirements are proved for call sequences of every length by induction [58]. The state is the object’s fields with the reading’s ghosts and history registers; a reading that looks back w−1w - 1 calls is judged on a window of ww consecutive calls from a free state. Houdini [28] finds the strongest conjunction II of template facts (comparisons among state variables and the constants of the code and the reading, also under conditions) that holds after the first calls on a new object and is preserved by every call. A requirement is PROVED if II before a window implies it at the window’s last call and it holds on the first max⁡(1,w−1)\max(1, w - 1) calls after reset; the window encoding is proved equal to the unrolled one on the calls both cover. Otherwise FCP checks all sequences of at most kk calls, the scope’s longest, and reports PROVED-BOUNDED. A dictionary field is translated as the sequence of its writes; in the induction it is an arbitrary function of the key and a list an arbitrary list of bounded length, with candidates first filtered on states sampled from the scope.

Evaluation

Research questions and setup

We ask five questions. RQ1 (False assurance): how often do self-check baselines certify violated requirements, and does FCP? RQ2 (Request defects): does FCP find conflicts and under-specification in the request before any artifact exists? RQ3 (Independent readings): is disagreement between independent formalizers a reliable error signal, and when does their agreement mislead? RQ4 (Reach): do verdicts on the exhaustive scope hold for all inputs? RQ5 (Cost): what does FCP’s assurance cost in questions for the author and in computation?

Corpus and agents. Table 3 summarizes the corpus; Appendices § 17 to § 19 give details (Table 10 to Table 13). All agents were Claude models (Haiku 5.5, Sonnet 5.5 and Opus 5.5), run between 6 and 11 November 2026 as isolated subagents, each in its own workspace. Every prompt, output and content hash is recorded, so the studies can be replayed without the models. In the hardware study, four Sonnet agents worked on each design: a decomposer, two formalizers given different style instructions, and an RTL generator. Formalizers had no access to the RTL, the gold readings or each other’s work, and worked from the gold requirement list so that formalization could be measured independently of decomposition. In the other studies, the readings were written by two isolated Opus formalizers. Code writers received only the prompt, in an empty directory, and wrote an implementation and pytest tests until the tests passed.

DomainTasksArtifacts checkedReadings
Hardware4 IP specifications; 13 planted defectsreference RTL, 28 single-edit RTL defects, 4 AI-generated designsgold; 2 Sonnet per design
Prompt1 launch email, 10 instructions3 responses (Haiku, Sonnet, Opus); an LLM judge as baselinegold; 2 Opus
Agent1 returns desk, 7 tools, 11 rules10 scaffolds; 18 runs of 3 modelsgold; 2 Opus
Code10 tasks; 10 conflicts, 23 ambiguitiesper task: 3 model implementations, a reference, 8 planted bugs; a refactoring in 9 tasksgold; 2 Opus; 21 variants

Table 3. Corpus. Each request has planted defects no agent saw. Models: Claude Haiku, Sonnet and Opus 5.5.

DesignSpec linesRTL linesGold reqs.RTL defectsRequest defects
SFIFO-01 synchronous FIFO (depth 5)86532184
RRA-4 round-robin arbiter53441274
BRG-1 command-to-APB bridge84612183
HUB-2 two-port APB hub (with BRG-1)451241352

Table 10. Hardware corpus. “RTL lines”: the reference design; “Request defects”: planted defects of the specification. Decomposers recovered every gold requirement (by quote overlap) and flagged every planted defect and three real ones that had not been planted. They flagged 73 to 95% of their requirements as ambiguous: high recall, low precision.

TaskRulesPlanted conflict (number of planted ambiguities)
Checkout7free shipping from $50 vs. the charge never drops as items are added (4)
Card masking4keep the last four digits vs. never show more than a quarter, for 12 to 15 digits (2)
Rate limiter5allow below 3 attempts in 60 s vs. deny for 10 s after a denial (3)
Student pricing4free shipping from $75 vs. nothing changes for non-students (2)
Payment retry7keep trying until the charge succeeds vs. at most 3 attempts (2)
Atomic save7leave no temporary file vs. keep it if os.replace fails (2)
Invoice lines6discount rounded per line vs. per order (2)
Gift card6$5 bonus on a load of $100 or more vs. $500 cap, read literally (2)
Order placement8refund and release after a failed booking vs. at most four calls on the services (2)
Loyalty points9the tier follows the points vs. the tier never goes down (2)

Table 13. The code tasks, each with a reference and eight planted bugs; the later nine also have a refactoring.

Gold readings. Gold readings are validated mechanically: they are proved on the reference designs, kill every observable mutant, and detect every planted defect. Where a request contradicts itself, the gold reading resolves it by a recorded decision. Gold readings take no part in the protocol’s comparisons.

Metrics and baselines. A violation is a pair (artifact, requirement) on which the gold reading fails, with the failure replayed. A flow gives false assurance when it certifies a violation (Definition 2). The *self-check

baseline certifies a requirement when the artifact passes the checks that a model wrote for it: in hardware, when the formalizer’s properties for the requirement are proved on the design; in code, when the model’s own test suite passes. For the prompt, the baseline is an LLM judge given a per-instruction checklist. FCP certifies a hardware requirement at level ASSURED and a code requirement as defined in Table 2.

Hardware: four designs

The corpus consists of a synchronous FIFO, a round-robin arbiter, a command-to-APB bridge and a two-port APB hub, with specifications written in the style of industrial IP documents. They contain planted contradictions, ambiguities, a permission that another sentence forbids, an incompleteness and a vacuity trap, and there are 28 single-edit RTL defects. About 39,700 content-addressed proof tasks were run; the median PDR obligation took 0.82 s, and every verification obligation received a definitive verdict.

False assurance (RQ1). Table 4 summarizes the results. All 98 formalizations were well-formed, lint-clean and read back correctly, and 44 (45%) were equivalent to the gold reading. The differences lay in timing windows, conditions and the resolution of conflicts. Accepting the formalizers’ properties as written certified 23% of the violated requirements, and FCP certified none. In one case, a FIFO formalizer adopted the usage rule “the producer shall not push while full” as an assumption. The assumption is receptive, since the environment can always refrain from pushing, so the receptiveness check correctly raises no objection. Vacuity analysis proved two of the formalizer’s properties vacuous, and contract comparison showed every contract of the set to be weaker than its peer’s (the other formalizer’s). An earlier version of FCP that compared properties under pooled assumptions admitted this assumption into the agreement check and gave 4 false assurances on this set; comparing contracts removes them.

Self-check: AI writes assertions, the prover reports PASSFCP
Violated requirements certified as holding56 / 244 (23%)0 / 244
Injected formalization faults that certify a buggy design71 / 1610 / 161, all 161 flagged
AI formalizations equivalent to intent (all well-formed and lint-clean)44 / 98 (45%)n/a (FCP does not consult gold)
Disagreements between the two AI formalizers that were real errorsn/a45 / 45
Injected RTL defects detected (gold readings)28 / 2828 / 28, all replayed

Table 4. The hardware study (RQ1, RQ3). The fault operators are placeholder consequents, reset-polarity errors, masking assumptions, dropped conjuncts, same-cycle for next-cycle, and bounds off by one cycle (Appendix § 17).

The acceptance rule (RQ1, RQ5). Table 5 varies the rule by which a proved requirement is accepted without review. Strict equivalence of readings accepts 7 of 104 requirement readings on the reference designs. Equivalence in the full context, with proofs of every other requirement under both readings, accepts 31 with no false assurance. The rule is sound per requirement: the design’s behaviors then lie inside the context, so the peer’s reading also holds, and on every acceptance gained, the peer’s reading was proved as well. All of the gain is on the bridge, the one design whose reference RTL proves every requirement under both readings; on every design, most disputes vanish in context. Requiring proof of only a minimal sufficient context accepts 64 readings and admits 5 false assurances, four of them with both formalizers wrong together.

Acceptance rule for a requirementAccepted on reference RTL (readings)False assurance on buggy RTL
Self-check: the formalizer’s reading is proved92 / 10456 / 244
All readings proved, with no comparison77 / 10410 / 244
Equivalent in a minimal sufficient context, and that context proved64 / 1045 / 244
Equivalent in the full context, and every other requirement proved under both readings31 / 1040 / 244
Strict: equivalent without context7 / 1040 / 244

Table 5. Ablation of the acceptance rule on the hardware study. Comparison in the full context raises acceptance 4.4-fold over strict equivalence, with zero false assurance under both, and weaker rules accept more, some falsely.

AI-generated RTL and unplanted defects. The RTL generators’ designs passed their own testbenches of up to hundreds of thousands of cycles. Against the gold readings, every requirement of the generated FIFO and bridge was proved, and FCP found two deviations: one arbiter does not mask grants during reset, and one hub accepts a command in the response-handshake cycle, using a permission of the bridge that the hub’s one-transaction rule forbids. FCP also found three defects that had not been planted. The hub’s 12-cycle latency bound is unrealizable; the tight bound is 17 cycles. The hub’s selector violates its bridge’s input contract, which every end-to-end proof misses because the bridge in use samples only when idle. Finally, the gold reading of the hub omitted a requirement, which determinacy exposed.

Disagreement as a signal (RQ3). Across all requirements formalized by both agents, every contract disagreement between them was a real error (45 of 45). Agreement was correct 9 times and wrong twice: both arbiter formalizers read an ambiguous sentence, intended as true round robin, literally as “shift the priority by one position”. Neither case produced false assurance, because the formalizers’ properties fail on the conforming RTL and the decomposer’s ambiguity flag kept the requirement in the review queue.

A prompt

A prompt for a product-launch email has ten checkable instructions. Before any response existed, the consistency analysis proved in 12 s that no email satisfies the literal reading, and named six instructions as a minimal conflicting set: a greeting, three feature sentences each followed by a benefit sentence, and a body “under 7 sentences”. Read literally, a greeting, three features and three benefits make seven body sentences (RQ2). The readings disagree on exactly the planted ambiguities, and contextual comparison turns each disagreement into one question with an example email; in isolation, two of the four disputes arise only on emails that violate other instructions. An LLM judge (Opus), given the prompt, the three models’ responses and a per-instruction checklist, marked all 33 items PASS (11 per response) and called the prompt satisfiable. It had adopted one reading (“6 body sentences, greeting not counted”) and did not report that every email fails under the other. Its reading is defensible. On the same prompt, FCP returns a question, its evidence and each answer’s consequence. Two labelers (Sonnet and Haiku) differed on 1 of 189 label decisions and agreed on two that the reference labeling, written with the gold reading, rejects, one of which changes verdicts. Labeler agreement therefore does not establish correctness.

An agent

The returns assistant of an online store has seven tools, a “Talk to a human” button and eleven rules, with one planted conflict and five planted ambiguities. Ten scaffolds were compiled from Python, each agreeing with its source on 6,000 random turns: a guarded kernel (91 lines of Python, 224 of Verilog), a plain ReAct loop [67], and eight one-line mutants of the kernel. Before any run, forcing found in 0.18s that pressing the button in turn 1 makes the prompt unsatisfiable: “hand off at once” and “ask whether there is anything else before ending” cannot both hold (RQ2). Under the literal reading, the guarded kernel enforces 8 of the 9 checkable rules for every model output and violates the ninth, the conflict. On the ReAct loop every rule is delegated, and the solver’s first attack is a $406 refund in turn 1. Six of the eight planted bugs are latent: each is unobservable as long as the model follows the rules and exploitable once it deviates, the condition that prompt injection creates (Appendix § 18). Haiku, Sonnet and Opus played the assistant in six scenarios (18 runs, 82 turns). The guard blocked no proposal and forced the handoffs that the rules demand (Appendix § 18). All three models ignored an injected “pre-approved, skip confirmation” note, and all three deviated from the literal prompt only at the planted conflict and one planted ambiguity, in the same way. Replayed through the agent’s Python, the ReAct loop executes all 9 of the solver’s attack traces, and the guarded kernel executes only the trace at the rule it violates by design.

Planted bugs flagged in the planted requirement: 7 of 7 (C1 to C5, C7, C8). C6 (float round(), half a cent to even) is correctly not flagged: every reading allows either neighbor.

False assurance on the models’ code: the self-check baseline (own tests pass) certifies 3 of 3 violations, FCP none.

Code: a checkout function

The checkout prompt (§ 1) has seven rules, formalized as nine requirements, including “discounts stack, but the total discount is never more than 30%”, “round to the nearest cent”, free shipping from $50, and “adding items to the cart must never lower the amount charged”. One conflict and four ambiguities are planted. We check the three models’ code, a reference and eight one-line planted bugs, each on 1,355,700 calls.

Before any code (RQ2). The boundary search finds the conflict under all three readings, in 17 to 65 s: checkout(4999,0,False) must return (4999, 499), a charge of $54.98, and checkout(5000,0,False) must return (5000, 0). Every minimal conflicting set contains the shipping and monotonicity rules. The trigger-situation check finds the readings consistent, because none of its candidates lies on the threshold. Determinacy finds one open output under every reading: at exactly half a cent, checkout(5,0,True) may cost 4 or 5 cents. The three readings differ under strict comparison, because two of them state the price rules only for calls that return. In context they are equivalent on every requirement, each decided on two calls in under a second; bit-level spot checks on 400 calls find no counterexample to any of them.

The code (RQ1). Table 6 and Figure 1 show the results. All three models reported the conflict in their replies, each passed its own tests, and under every reading each violates a rule. Code splits reveal four decisions on which the implementations differ. In the largest (406 calls), Haiku and Opus measure “an order under $50” on the discounted price; Sonnet, the reference and all three readings measure the subtotal. The decomposer had flagged that phrase, so it stays in the review queue independently of the formalizers’ agreement. The review queue for the prompt holds three questions: which rule yields at the threshold, which amount the $50 refers to, and how half a cent is rounded. If rule 7 yields, FCP certifies Sonnet’s code on all eight remaining requirements and flags Haiku and Opus on rule 6. Until the conflict is answered, FCP certifies 9 of the 27 (requirement, implementation) pairs. The models’ suites encode their authors’ decisions: on the other implementations they fail up to 212 of 745 tests, and Haiku’s suite misses the planted shipping bug at $50.00, where its own code charges shipping. The only mutants surviving every reading change the rounding constant, the open half cent, or a value inside Opus’s fee taper where both values violate rule 6.

CodeOwn testsViolates (all readings)First failing call (replayed in a fresh process)
Haiku134 passR-SHIP-1checkout(5000,0,False) → (5000, 498)
Sonnet196 pass, 1 xfailR-MONO-1checkout(4999,0,False) → (4999, 499), then checkout(5000,0,False) → (5000, 0)
Opus745 passR-SHIP-1checkout(4998,0,False) → (4998, 2)

Table 6. The checkout study (RQ1). All three models stated the conflict, and each resolved it differently (Figure 1).

Nine more code tasks

Nine further tasks, each a short prompt with one planted conflict, exercise the adapters of § 6.4. Each was given a gold reading, read by two isolated Opus formalizers, and implemented by Haiku, Sonnet and Opus from the prompt alone (Table 7; Appendix § 19).

TaskAdapterConflictBefore codeChoiceAgree onOwn testsBugsCertify
Card maskingstringslast four digits vs. at most a quarter92 s, 1 calla quarter: 3 digits shownevery call38, 146, 11918/83/3, 0
Rate limiterobjectunder the limit vs. 10 s penalty9 s, 5 callspenalty175,528 calls48, 43, 4317/83/3, 0
Student pricingchange$75 shipping vs. no change for non-students | 29 s, 1 call | $50 for non-students3.6 M calls22, 29, 458/83/3, 0
Payment retryeffects: servicekeep trying vs. 3 attempts67 s, 3 errors3 attempts50,000 runs21, 36, 1058/83/3, 0
Atomic saveeffects: filesno temporary file vs. keep it on failure18 s, 1 failed movekeep the filesee218, 37, 3318/83/3, 0
Invoice linesrecordsline rounding vs. order rounding21 s, 2 itemsdiscount summed per lineevery call19, 35, 2318/83/3, 0
Gift cardseveral methods$5 bonus vs. $500 cap1 s, 2 callsbonus counts toward the cap64,576 calls21, 38, 428/83/3, 33
Order placementeffects: importsundo after a failure vs. at most 4 calls46 s, 1 refusalundo, in 5 calls46,400 runs31, 53, 5118/83/3, 0
Loyalty pointspackagetier set by the points vs. tier never goes down2 s, 4 callstier kept44,079 calls32, 74, 7248/86/6, 0

Table 7. Nine more code tasks. “Before code”: conflict search under gold, with its stimulus. “Choice”: what all three models did at the conflict. “Agree on”: calls or runs of the scope on which their code answers identically. “Own tests”: suite sizes (Haiku, Sonnet, Opus). “Bugs”: planted variants flagged in their planted requirement (a refactoring, flagged in no task, is excluded). “Certify”: violations under gold certified by the self-check and by FCP.

Identical choices across models (RQ1, RQ2). In each of these nine tasks, all three models made the same choice at the conflict, and in eight of them their code then answers every call or run of the scope identically. Agreement among independent implementations therefore does not indicate that a prompt is consistent. In these tasks it reflects the conflict, resolved identically three times, in line with the correlated errors observed across language models [36]. Every implementation violates the gold reading, the self-check baseline certifies every one of these violations, and FCP certifies none in eight of the nine tasks. Under the gold reading, FCP’s search names each conflict before any code exists, with a shrunk stimulus (Table 7);

under the formalizers’ readings it names eight of the nine, the exception being the gift card’s, which both formalizers’ readings resolve (§ 8.8). The trigger-situation check misses the conflict under the gold reading in all nine tasks and finds it under a formalizer’s reading in two. The models themselves name most conflicts: over the ten tasks, their summaries name the conflict in 25 of 30 cases, and Opus, asked without tools, in 81 of 90 replies (model-graded, spot-checked by hand); the conflict of the loyalty package is the only one that no summary or reply names. FCP adds the conflicting calls and a verdict on the code under each resolution.

What the scope reaches. Points drawn from the readings reach narrowly defined behavior: valid card numbers, the third denial inside a window, three errors in a row, a failed move after a successful write, two invoice lines whose discounts each lose a fraction of a cent, and a computed subtotal of exactly $10.00. Under the gold reading, the scope flags 71 of the 72 planted bugs in their planted requirement. The one miss, a rate-limiter bug (a denial during the penalty does not restart it), lies at least two edits from every request-derived sequence; the models’ own tests detect it, as does the proof (§ 8.7).

Proofs beyond the scope

Table 8 reports the proofs on the eight tasks whose code falls in the translatable subset (RQ4). Of 322 pairs of an implementation and a reading, 303 translate; the other 19 involve planted variants that call round() on a float, divide as floats or use a regular expression, which the translator refuses by design. The 728 requirements the scope failed also fail the proof; of the 1,535 it passed, 1,517 are proved, 11 are proved up to a bound and 7 fail. A proof extends a verdict from millions of scope calls to every input and, on the two objects, to every sequence of calls. Four of the failures the scope missed are on card masking, at strings its 131,613 calls do not hold: a variant that strips the number first accepts one that begins with a carriage return or a no-break space, under all three readings, and a variant that tests digits with str.isdigit accepts superscript digits and then shows half of the digits 0 to 9. The other three are on the rate limiter; one is the planted bug the scope missed, found at six calls, so all 72 planted bugs are flagged. Four bounded verdicts are false. They concern three gift-card variants whose balance drifts, under the gold reading, whose model of the balance is a ghost with a declared range: a long sequence takes the model out of range, and the requirement fails on the compiled monitors at calls 21, 41 and 4,155, far beyond the 10 calls of the bounded proof. The induction found no invariant because none exists. The other seven concern two limiter variants whose lists the invariant search cannot bound (one grows with every denial, one is shared by all customers); they are not known to fail. PROVED-BOUNDED marks exactly the verdicts that length can still overturn. On the change task, “nothing changes for non-students” is proved for all three models’ code under every reading, over every call the interface carries. On the payment retry, every model’s code is proved on every requirement other than the conflicting rule, for every amount and every script of service answers. On the gift card, the models’ code is proved on every requirement under both formalizers’ readings: the shared misreading (§ 8.8) holds beyond the scope too, so proofs strengthen conformance and leave fidelity unchanged.

When independent formalizers agree and are wrong

The gift-card prompt requires a ValueError “for a load that would take it over” the $500 cap and leaves open whether the $5 bonus for a load of $100 or more counts. Read literally (gold), a load of $495.01 must be accepted, and its bonus violates the cap. Both formalizers and all three models counted the bonus toward the cap, which removes the conflict and rejects that load. The decomposer had flagged the phrase on the cap requirement, which stayed a question. Its answer also decides the load requirement, which carries no flag, so FCP certified the three violations, the only false assurance in the code studies (RQ3).

We therefore added a variant reading: one more isolated formalizer, instructed to adopt the reading of the flagged phrase that no formalizer had chosen. Every requirement on which the variant differs from a formalizer’s reading in context depends on the answer and goes to the author. This flags exactly the cap and load requirements, and FCP then certifies none of the violations (18 of 24 pairs, down from 21). We designed this step after observing the failure, then ran it on all tasks, where variant kk takes the kk-th listed answer of every flagged phrase (20 more formalizers, $25). It removes the three false certificates and holds back 12 of the 87 true ones: a variant that assumes more than the formalizers differs from both on nearly every requirement. Holding such a requirement only where a variant may reject a run that a formalizer accepts, strictly or in context, a rule written after seeing these results, removes the same three and holds back 3. The automated pipeline (§ 8.9) applies the first rule before any code and this rule given the code. Given the code, it asks of each requirement on which a variant differs in context whether the variant accepts every run that both formalizers accept, with and without context, and releases the requirement only when both searches close in that direction. On the stored relations of the 21 variants this holds 39 requirements, where the designed rule holds 48 and the second rule 27, and it leaves 84 of the 87 true certificates and no false one. Two adversarial reviews of the engine shaped this rule: the first showed that closure in context alone is unsound, since a run may meet both formalizers and break another rule of the variant, and the second found no false release in the corrected rule.

The atomic-save task shows the same mechanism without false assurance. Both formalizers read the rule on os.fsync as not requiring a flush of Python’s buffer first, a reading the prompt’s example supports and the gold reading rejects. A planted bug that calls os.fsync without flushing, a real durability defect that the gold reading flags, therefore passes every test of every model and every requirement of both formalizers’ readings. In all, independent formalizers agreed against the gold reading in three tasks (the arbiter, the gift card and the atomic save), each time on a phrase flagged as ambiguous. A flag on one requirement carries no information about the other requirements that its answer decides.

End-to-end automation and cost

Every study above was assembled by hand. The command fcp review performs the same steps from the request and the code alone: an agent writes the adapter from the request and the code’s public signature (its body is withheld); a decomposer writes anchored requirements; two isolated formalizers, and a variant formalizer where needed, write readings; and FCP runs the analyses and reports certified requirements and questions. On a duration parser written by Sonnet (12 minutes of agents, 31 of analyses), the report certifies 5 requirements and lists 13 open items, among them a replayed failure (parse_duration("002h") raises) and the request’s own contradiction (parts up to 99 allow “99h”, which “a duration is never longer than one day” excludes). On the gift-card prompt with Opus’s code, the report certifies 7 requirements, none of them violated under the literal reading, and asks 24 questions; the study’s false assurance does not recur.

To test FCP on requests we did not write, we ran fcp review, with a signature and no code, on 20 HumanEvalComm problems [66]: each original description and its ambiguous, inconsistent, and incomplete rewrites (80 requests, one run). FCP completed a review for 72; the other eight exceeded a time or memory limit, seven of them on arithmetic that FCP-PL lacks. A question backed by a solver result was asked on 34 of 53 reviewed rewrites (read by hand, it concerns the rewrite’s change on 33; on 27 it shows a concrete call) and on 3 of 19 reviewed originals. The same model without tools, instructed to ask before coding, asks about 56 of 60 rewrites and 1 of 20 originals. On such short single-function requests, a review therefore adds a solver result behind a question and no additional recall.

Avoiding false assurance requires review (RQ5). On the reference hardware designs, the queue holds 8 to 27 decisions per reading, and with Sonnet formalizers 7 to 31 of 104 requirement readings were accepted without review, depending on the rule (Table 5). In the code tasks (requirements times three implementations), FCP certifies 9 of 27 pairs on the checkout, 9 of 21 on card masking, 12 of 18 on student pricing, 9 of 27 on the payment retry, 6 of 24 on the atomic save and on invoice lines, 18 of 24 on the gift card with the variant reading, and none on the rate limiter, where every requirement fails under a formalizer’s reading or is read differently by the two. Because the questions concern the request, one answer settles them for every implementation. A hardware study run takes 6 to 26 minutes per design, a code study minutes to hours, and a proof beyond the scope seconds per pair, and longer for long object sequences.

Discussion

The evidential value of agreement. Every form of agreement we examined failed at least once: code and its own tests (all 30 implementations), three models’ code agreeing on every call (eight tasks), two formalizers (three tasks), two labelers, and an LLM judge that passed every response to an unsatisfiable prompt. Approaches that select or validate outputs by agreement, between programs and generated tests [15] or among sampled answers [61], inherit this limit whenever the agreeing parties share a misreading. Disagreement was reliable where we measured it: all 45 contract disagreements between hardware formalizers were real errors. FCP is built around this asymmetry: it turns disagreements into witnesses, keeps every flagged phrase open irrespective of agreement on it, and asks the author whenever no machine can decide.

What FCP trusts. FCP’s verdicts rest on four components: the compiler from FCP-PL to monitors, checked by two engines, by replay, and against a reference semantics in Lean (Appendix § 15); the engines, checked against each other; the adapters, whose frame rules enter every reading and whose label semantics the analyses exercise; and the second semantics for comparison and proofs, validated on every call of the scope. Language models write requirements, readings, labels and adapters, and every verdict that depends on a model’s judgment is backed by a mechanical check or a question to the author.

Questions about the request. Because FCP’s questions concern the request, they are asked once and reused. An answer becomes a recorded decision that every later reading must honor, and every artifact written for the request is then judged against it. Related interactive approaches ask the user to approve tests of one implementation [42] or to clarify a prompt before generation [50]. FCP derives each question from the readings and attaches the behavior that its answer decides.

Toward a confidence layer over AI assistants. The end-to-end pipeline suggests a deployment in which every substantial answer of a coding assistant arrives with a report: the requirements that hold, with their evidence, and the open questions, each with a concrete call. Work on scalable oversight asks how people can supervise outputs they cannot fully evaluate [1, 11, 31]; FCP contributes one mechanism, which reduces the author’s task from judging an artifact to answering concrete questions about the request.

Related Work

Formal analyses. Vacuity [6, 41], coverage metrics for formal verification [17], assume-guarantee reasoning [7, 48], realizability [55], self-composition and hyperproperties [5, 20], bounded model checking [9, 19], induction-based safety checking [58] and IC3/PDR [12, 26] supply individual analyses. FCP assigns each to a situation of Table 1 and adds refinements that our studies required: assert and assume forms of every property, feasibility-based conflicts, extendable witnesses, and contract-level comparison in context. Translation validation [52, 56] inspires our second-semantics checks, and proof-carrying code [51] the report.

Language models for specifications and verification. Language models translate natural language into temporal logic [21], generate hardware assertions [35, 53], write program postconditions [27] and specifications [46, 62], and generate verified code in Dafny [49, 59]. They also write RTL [45, 60] and code [16, 34]. These works ask whether a model can produce a specification or a verified artifact (Clover, for example, checks consistency among code, annotations and docstrings). FCP addresses the complementary question: whether a consistent and verified specification means what the request means. Our hardware study quantifies the gap (45% of well-formed formalizations matched the gold reading).

Testing AI-written code and intent. Test suites under-approximate correctness [44], and deciding what an output should be is the oracle problem [4]. Generated tests can rank generated programs [15], and interactive approaches formalize intent through tests approved by the user [42] or through clarifying questions [50]. Ambiguity in requirements is a long-studied source of defects [8]. In FCP, the oracle is a pair of independent readings of the request, questions carry witnesses computed from them, and the evidence on code is exhaustive within a request-derived scope [32] or a proof beyond it [14, 37], complementing random property-based testing [18]; mutation [33] and differential testing [47] supply two of its checks.

Judges, correlated errors and oversight. LLM judges [70] return a single verdict, can be swayed by adversarial content [57], and favor their own generations [54]; self-correction without external feedback is also unreliable [30]. Errors of different language models are correlated [36], echoing the finding that independently developed program versions fail together [2, 38]; we observe the same effect for formalizers, labelers and code writers, and design the protocol around it.

Agents and prompt injection. Indirect prompt injection [29] motivates the cut point. Benchmarks measure attacks and defenses [24] and tool-agent-user interaction [68], and system designs confine the model by construction [25]. Runtime verification [43] and shield synthesis [10] are the closest formal relatives. FCP complements them by proving, for a given scaffold and prompt, which rules hold for every model output and which rest on the model, before the agent runs.

Limitations

Scale and corpus. The hardware designs have a few hundred state bits, and the corpus contains one prompt, one agent and ten code tasks, nine of one function or class and one a package of three modules. The planted defects were chosen by us, and real requests are longer, with defects that may be subtler.

Gold readings and models. The gold readings were written with a model of the formalizers’ family. They are validated mechanically (proved on reference designs, killing every observable mutant, detecting every planted defect), and their fidelity to the prose remains a judgment, recorded as decisions. All formalizers, code writers and judges are Claude models, and other providers’ models may err differently.

Correlated formalizers. Formalizers agreed against the gold reading on flagged phrases in three tasks, and in the gift card FCP certified three violations. A variant reading per answer removes them at two or three more formalizers per task. Its coverage extends only to questions that were asked: on 8 requirements both formalizers accept a run that the gold reading rejects, and a variant can reject such a run on 5.

Exactness of the adapters. Labels bound the exactness of text verdicts, and tone or helpfulness remain judged by people. For agents, delegated rules are measured by runs and remain unproved, and the Python-to-Verilog translation is checked by co-simulation and is likewise unproved. For code, the evidence is exhaustive within a scope; it is a proof only where the code translates to z3 terms (functions on integers, strings and lists, changes to them, calls on a service, files, and call sequences on objects whose fields are numbers, dictionaries or lists, for every length only where an invariant is found), and in that case it is conditional on the second semantics (Proposition 4). A file’s contents are not modeled, and a package of several modules is checked on its scope without a proof. Comparison witnesses for objects and changes are bounded counterexamples, which need not be the simplest calls, and a look-back inside once delays the cover that contains it.

Bounded searches and cost. Forcing, boundary conflicts and fixed-stimulus triage are sound for what they report and incomplete; a clairvoyant implementation can meet a bound on any single stimulus, so the hub’s unrealizable bound was confirmed by a sweep and by hand. Low false assurance is obtained at the cost of review questions, and review effort is reported as a count of decisions; human time was not measured. Finally, formalizers were isolated by instruction and by workspace; no sandbox was used.

Conclusion

When the same kind of system writes an artifact and its checks, passing checks carry little evidence. Across four kinds of artifact, FCP found what the producing models’ own checks missed: contradictions in the requests, before any artifact existed; readings that differ from the intent, each shown by one concrete example; rules that rest on a model’s compliance; and decisions that independent implementations made differently or, more often, identically. Of the violated requirements it was shown, FCP certified three, all in one code task where both formalizers and all three models made the same choice on a flagged phrase; a variant reading removes them. Agreement between independent formalizers bounds how much is assured without review, and each disagreement is concrete because FCP turns it into a waveform, a conversation or a call. Future work includes testing the second variant rule on new tasks, larger requests and programs, and proofs for packages of several modules.

Broader Impact

FCP is intended to reduce the risk that unreviewed AI-generated designs, agents and code are deployed on the strength of self-written checks. Its main risk is misplaced confidence in its verdicts: a certified requirement inherits every error that its agreeing readings share, and our gift-card study shows that such errors occur. FCP’s reports therefore state the evidence behind each verdict, keep every flagged phrase in the queue, and present each proof as a statement about a reading, leaving intent to the author. The agent adapter makes explicit which rules an agent’s scaffold enforces and which rest on its model; we see this as a tool for defenders, and the solver’s counterexamples are also attack traces against unguarded scaffolds.

Reproducibility

The repository, https://github.com/shankch/FCP_oss, holds the prototype, corpora, agent prompts and transcripts, and all results. Every agent output is content-addressed and recorded, so every study replays without access to the models. Each study is regenerated by one script under experiments/, which writes the JSON results that the tables report, with tool versions. The prototype’s 228 tests check the adapters, the translator and the encodings.

Proofs

Machine-checked statements. Propositions 1 to 4 and Corollary 1 are proved in Lean 4 [23] (84 theorems, standard axioms only), with two facts they use: every FCP-PL item is causal, and each register the compiler emits (every operator; every pattern other than inorder) computes an executable reference semantics in closed form. On random traces of 46 readings, simulated monitors agree with it on all 1,747,852 compared cycles, after it exposed three compiler defects outside the corpus. Lean checks the registers as transcribed; the comparison tests the Verilog, noticing 1,436 of 1,840 sampled changes to it (14 of 120 for string readings).

Proposition 1. Let MM accept σ\sigma, that is, let some π∈⟦M⟧\pi\in\llbracket M \rrbracket carry the inputs σ\sigma in cycles 0,…,k0,\ldots,k, and let τ\tau be the outputs of π\pi in those cycles. By Definition 3, applied to τ\tau, there is a cycle t≤kt \le k at which some guarantee φ\varphi fails on (σ,τ)(\sigma,\tau), with every assumption holding up to tt. Since monitors are causal, the verdicts on π\pi up to cycle kk are those on (σ,τ)(\sigma,\tau), so π\pi witnesses M⊭(A⇒φ)M \not\models(A \Rightarrow\varphi) by Definition 1.

Corollary 1 (Pinned instances). Let a guarantee ∀u φ(u)\forall u\,\varphi(u) range over a rigid variable uu, and let u1,…,umu_1,\ldots,u_m be values. If σ\sigma is a conflict of the readings in which ∀u φ(u)\forall u\,\varphi(u) is replaced by φ(u1)∧⋯∧φ(um)\varphi(u_1) \land\cdots\land\varphi(u_m), then σ\sigma is a conflict of the original readings.

Proof. Where an instance φ(ui)\varphi(u_i) fails, so does ∀u φ(u)\forall u\,\varphi(u), and the assumptions are unchanged; so outputs that violate the pinned set at tt with the assumptions held up to tt violate the original set at tt.

Proposition 2. The model is a function from the history of a conversation to a proposal, and every such function produces, together with any environment, a sequence of values of XX. Because XX is a free input of MM, every run of the agent is a trace of MM. If M⊨(A⇒φr)M \models(A \Rightarrow\varphi_r), every run satisfies rr under AA. If M⊨(A∧C^r⇒φr)M \models(A \land\hat{C}_r \Rightarrow\varphi_r), then by Definition 1, φr\varphi_r holds at every turn tt up to which both AA and C^r\hat{C}_r have held; this covers every turn up to the first in which the model’s proposal violates C^r\hat{C}_r.

Proposition 3. Write cycle 0 for reset and cycles 1, 2, ... for calls. Under the hypotheses, the verdict of an item at a call tt is a function of the calls t−δ,…,tt-\delta,\ldots,t, where δ≤d\delta\le d is the item’s look-back, if t−δ≥1t-\delta\ge1; otherwise the look-back reaches before reset, and the item is disabled and holds. Without assumptions, the contract CrC_r is the set of traces on which every item of rr under xx holds at every call. Readings without future obligations need no extension of witnesses, since every violation-free prefix has a conforming continuation.

Suppose π∈(Car∩K)∖Cbr\pi\in(C_a^r \cap K) \setminus C_b^r, where KK is the set of all traces for strict comparison and KrK^r for comparison in context. Some item of rr under bb fails at a call tt of π\pi. Let s=max⁡(1,t−d)s = \max(1,t-d), and let π′\pi' consist of reset followed by the calls s,…,ts,\ldots,t of π\pi at most d+1d+1 calls. Because the function has no state, the answer to each call depends only on its arguments, so π′\pi' is a trace of the function, and trivially a trace of Σ\Sigma. Map call jj of π′\pi' to call s+j−1s+j-1 of π\pi. The failing item sees the same window at the last call of π′\pi' as at tt in π\pi (if s=1s=1, π′\pi' is a prefix of π\pi), so it fails, and π′∉Cbr\pi' \notin C_b^r. Consider any other item ℓ\ell, of rr under aa or of any r′≠rr' \ne r under aa or bb, at a call jj of π′\pi'. If its look-back fits in π′\pi', its window equals its window at s+j−1s+j-1 in π\pi, so its verdict is the same, and it holds there because π∈Car∩K\pi\in C_a^r \cap K. Otherwise its look-back reaches before reset in π′\pi', and it is disabled and holds. Hence π′∈Car∩K\pi' \in C_a^r \cap K. The converse direction is immediate.

Proposition 4. Part (i) holds by construction: a FAILED verdict is reported only after $z3’s calls (or runs) have been executed on the code and the resulting trace has been rejected by the compiled monitors. For part (ii), suppose some trace of ff violates an item of rr at call tt. The window argument above, applied to a single item, shows that the at most kk calls ending at tt, run from reset, also violate it. (The horizon is exact when

the hypotheses of Proposition 3 hold for the items of rr; for code with effects, kk covers a complete run; for objects, a PROVED-BOUNDED verdict claims only sequences of at most kk calls.) Because T(f)T(f) agrees with ff on these inputs, the translated outputs equal the code’s; because EE agrees with the monitors, the encoding fails at the same call. The query is therefore satisfiable, which contradicts the PROVED verdict. For objects, II holds in the first ww states after reset and, if in ww consecutive states, in the next, hence in every state of every trace. A violation at a call t≥wt \ge w would be a window that starts where II holds and violates rr at its last call, and one at t<wt < w lies in the unrolled prefix; the queries exclude both.

The Property Language and an Example Reading

Table 9 summarizes FCP-PL. The listing below gives the checkout task’s gold reading of the shipping and monotonicity rules. Each requirement carries its verbatim quote, line span and any recorded ambiguity (here, “Orders under $50”).

ConstructMeaning
TypesBooleans; bit-vectors; integers with statically computed ranges; bounded sequences (strings, lists) of at most nn slots
Expressionsexact arithmetic, comparisons, Boolean connectives on one-bit operands, conditionals, len⁡(s)\operatorname{len}(s), s[i]s[i], x∈sx \in s, ASCII character tests, all⁡/any⁡/count⁡/sum⁡\operatorname{all}/\operatorname{any}/\operatorname{count}/\operatorname{sum} over i<bi < b with bˉ≤256\bar{b} \le256
Past timeprev⁡(e,k)\operatorname{prev}(e,k), disabled before the current reset segment; once, hist, since
Patternsinvariant, implies, next, within, hold_until, bounded_wait, inorder, after_reset; each monitor flags the minimal bad prefix
State and quantifiersghost variables with reset values and update rules; rigid variables, universally quantified over a range; named definitions
Item rolesproperty (guarantee), assumption, scenario (cover), don’t-care; each item names the requirement it formalizes

Table 9. FCP-PL in brief. Every property is compiled to an assert form and a deterministic assume form.

- id: G-SHIP
  req: R-SHIP-1
  pattern: implies
  args: {trigger: valid, cond: ’ship == (subtotal < 5000 ? 499 : 0)’}
  note: ’decision: the order value is the subtotal before discounts’
- id: G-MONO
  req: R-MONO-1
  pattern: implies
  args:
    trigger: ’valid && prev(valid) && outcome == 0 && prev(outcome) == 0
              && coupon == prev(coupon) && member == prev(member)
              && subtotal > prev(subtotal)’
    cond: ’price + ship >= prev(price + ship)’

The look-back of G-MONO is one call, so Proposition 3 decides comparisons of these readings on two calls, and the conflict of Figure 1 is a pair of calls on which no outputs satisfy both items.

Hardware Study Details

Per-set results. The baseline’s 56 false assurances fall in seven of the eight formalizer sets, from 2 of 20 violations (arbiter, set B) to 30 of 53 (FIFO, set A). Under the gold environment, 44 of the formalizers’ 98 properties are equivalent to gold; the others are stronger, weaker, incomparable or, once, missing.

Fault operatorFaultsSelf-check FAFCP FAFlagged by
placeholder consequent (`...true`)50300vacuity or tautology, peer
reset-polarity error in the trigger27180vacuity, peer
masking assumption (assume !trigger)27200vacuity, peer, receptiveness
dropped conjunct1720peer, vacuity
next cycle replaced by same cycle2510failure on reference, peer
bound one cycle too loose1100peer
bound one cycle too tight400failure on reference, peer
total161710all 161 flagged

Table 11. Injected formalization faults on the gold readings (11 of 172 injected faults were equivalent to the original and excluded automatically). “Self-check FA”: faults that would certify a buggy design.

Agent Study Details

ScaffoldID1ID2ELIGCONFTOTBIGONCEHUMENDframe
guarded kernelEEEEEEEEBok
ReAct loopDDDDDDDDDok
G1 confirmation not tied to the orderEEEDEEEEBok
G2 amount not checked against itEEEDEEEEBok
G3 total from any order’s lookupEEEEDEEEBok
G4 handoff after the third failureEDEEEEEEBok
G5 button needs an empty proposalEEEEEEEDBok
G6 handoff needs verificationEBEEEEEEBok
G7 eligibility never revokedEEDEEEEEBok
G8 handoff does not end the loopEEEEEEEEBbroken

Table 12. Trust classes under the gold reading (E enforced, D delegated, B broken); the columns are the rules R-ID-1 to R-END-1 and the frame. Six mutants (G1 to G5, G7) are latent: each turns an enforced rule into a delegated one, which is unobservable as long as the model follows the rule. Each scaffold verifies in under a second.

Readings and runs. The formalizers’ readings differ from gold on two rules: whether a later “No” revokes an earlier “Yes” to a refund, and whether blocking a refund over $150 suffices or a handoff is also required. Under their reading of the latter, the rule is delegated on the guarded kernel, so the trust class depends on the author’s answer. In the 18 runs, the guard mattered only when the customer failed verification twice: all three models explained the failure and proposed no handoff, and the scaffold forced the handoff.

The Ten Code Tasks

Per-task observations. Card masking: the conflict is mask_card("219011401821"), twelve digits that pass the Luhn check; the readings define the Luhn sum differently, and only sweeping decides their comparison. Atomic save: the first conflict search exposed an escape in the gold reading, now closed. Order placement: the conflict is place_order(7, 3, 1, 1) with the booking refused. Loyalty points: the conflict is earn(50000), tier(), redeem(1), tier(); each model’s suite kills one mutant of its own package that the scope misses.

References

  1. [1]Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. Concrete problems in AI safety. arXiv:1606.06565, 2016.arxiv.org/abs/1606.06565
  2. [2]Algirdas Avizienis. The N-version approach to fault-tolerant software. IEEE Transactions on Software Engineering, SE-11(12):1491–1501, 1985.DOI
  3. [3]Haniel Barbosa, Clark Barrett, Martin Brain, et al. cvc5: A versatile and industrial-strength SMT solver. In Tools and Algorithms for the Construction and Analysis of Systems (TACAS), volume 13243 of LNCS, pages 415–432, 2022.
  4. [4]Earl T. Barr, Mark Harman, Phil McMinn, Muzammil Shahbaz, and Shin Yoo. The oracle problem in software testing: A survey. IEEE Transactions on Software Engineering, 41(5):507–525, 2015.DOI
  5. [5]Gilles Barthe, Pedro R. D’Argenio, and Tamara Rezk. Secure information flow by self-composition. In 17th IEEE Computer Security Foundations Workshop (CSFW), pages 100–114, 2004.
  6. [6]Ilan Beer, Shoham Ben-David, Cindy Eisner, and Yoav Rodeh. Efficient detection of vacuity in ACTL formulas. In Computer Aided Verification (CAV), volume 1254 of LNCS, pages 279–290, 1997.DOI
  7. [7]Albert Benveniste, Benoît Caillaud, Dejan Nickovic, Roberto Passerone, Jean-Baptiste Raclet, Philipp Reinkemeier, Alberto Sangiovanni-Vincentelli, Werner Damm, Thomas A. Henzinger, and Kim G. Larsen. Contracts for system design. Foundations and Trends in Electronic Design Automation, 12(2–3):124–400, 2018.
  8. [8]Daniel M. Berry, Erik Kamsties, and Michael M. Krieger. From contract drafting to software specification: Linguistic sources of ambiguity. Handbook, University of Waterloo, 2003.
  9. [9]Armin Biere, Alessandro Cimatti, Edmund Clarke, and Yunshan Zhu. Symbolic model checking without BDDs. In Tools and Algorithms for the Construction and Analysis of Systems (TACAS), volume 1579 of LNCS, pages 193–207, 1999.
  10. [10]Roderick Bloem, Bettina Könighofer, Robert Könighofer, and Chao Wang. Shield synthesis: Runtime enforcement for reactive systems. In Tools and Algorithms for the Construction and Analysis of Systems (TACAS), volume 9035 of LNCS, pages 533–548, 2015.
  11. [11]Samuel R. Bowman et al. Measuring progress on scalable oversight for large language models. arXiv:2211.03540, 2022.arxiv.org/abs/2211.03540
  12. [12]Aaron R. Bradley. SAT-based model checking without unrolling. In Verification, Model Checking, and Abstract Interpretation (VMCAI), volume 6538 of LNCS, pages 70–87, 2011.
  13. [13]Robert Brummayer and Armin Biere. Boolector: An efficient SMT solver for bit-vectors and arrays. In Tools and Algorithms for the Construction and Analysis of Systems (TACAS), volume 5505 of LNCS, pages 174–177, 2009.DOI
  14. [14]Cristian Cadar, Daniel Dunbar, and Dawson Engler. KLEE: Unassisted and automatic generation of high-coverage tests for complex systems programs. In USENIX Symposium on Operating Systems Design and Implementation (OSDI), pages 209–224, 2008.DOI
  15. [15]Bei Chen, Fengji Zhang, Anh Nguyen, Daoguang Zan, Zeqi Lin, Jian-Guang Lou, and Weizhu Chen. CodeT: Code generation with generated tests. In International Conference on Learning Representations (ICLR), 2023.arxiv.org/abs/2207.10397
  16. [16]Mark Chen et al. Evaluating large language models trained on code. arXiv:2107.03374, 2021.arxiv.org/abs/2107.03374
  17. [17]Hana Chockler, Orna Kupferman, and Moshe Y. Vardi. Coverage metrics for formal verification. In Correct Hardware Design and Verification Methods (CHARME), volume 2860 of LNCS, pages 111–125, 2003.
  18. [18]Koen Claessen and John Hughes. QuickCheck: A lightweight tool for random testing of Haskell programs. In Proceedings of the 5th ACM SIGPLAN International Conference on Functional Programming (ICFP), pages 268–279, 2000.
  19. [19]Edmund Clarke, Daniel Kroening, and Flavio Lerda. A tool for checking ANSI-C programs. In Tools and Algorithms for the Construction and Analysis of Systems (TACAS), volume 2988 of LNCS, pages 168–176, 2004.DOI
  20. [20]Michael R. Clarkson and Fred B. Schneider. Hyperproperties. Journal of Computer Security, 18(6):1157–1210, 2010.
  21. [21]Matthias Cosler, Christopher Hahn, Daniel Mendoza, Frederik Schmitt, and Caroline Trippel. nl2spec: Interactively translating unstructured natural language to temporal logics with large language models. In Computer Aided Verification (CAV), volume 13965 of LNCS, pages 383–396, 2023.
  22. [22]Leonardo de Moura and Nikolaj Bjørner. Z3: An efficient SMT solver. In Tools and Algorithms for the Construction and Analysis of Systems (TACAS), volume 4963 of LNCS, pages 337–340, 2008.DOI
  23. [23]Leonardo de Moura and Sebastian Ullrich. The Lean 4 theorem prover and programming language. In Automated Deduction (CADE-28), volume 12699 of LNCS, pages 625–635, 2021.DOI
  24. [24]Edoardo Debenedetti, Jie Zhang, Mislav Balunović, Luca Beurer-Kellner, Marc Fischer, and Florian Tramèr. AgentDojo: A dynamic environment to evaluate prompt injection attacks and defenses for LLM agents. In Advances in Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks Track, 2024.arxiv.org/abs/2406.13352
  25. [25]Edoardo Debenedetti, Jia Shumailov, Tianqi Fan, Jamie Hayes, Nicholas Carlini, Daniel Fabian, Christoph Kern, Chongyang Shi, Andreas Terzis, and Florian Tramèr. Defeating prompt injections by design. arXiv:2503.18813, 2025.arxiv.org/abs/2503.18813
  26. [26]Niklas Eén, Alan Mishchenko, and Robert Brayton. Efficient implementation of property directed reachability. In Formal Methods in Computer-Aided Design (FMCAD), pages 125–134, 2011.
  27. [27]Madeline Endres, Sarah Fakhoury, Saikat Chakraborty, and Shuvendu K. Lahiri. Can large language models transform natural language intent into formal method postconditions? Proceedings of the ACM on Software Engineering, 1(FSE), 2024.arxiv.org/abs/2310.01831
  28. [28]Cormac Flanagan and K. Rustan M. Leino. Houdini, an annotation assistant for ESC/Java. In FME 2001: Formal Methods for Increasing Software Productivity, volume 2021 of LNCS, pages 500–517, 2001.
  29. [29]Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz. Not what you’ve signed up for: Compromising real-world LLM-integrated applications with indirect prompt injection. In Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security (AISec), pages 79–90, 2023.arxiv.org/abs/2302.12173
  30. [30]Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. Large language models cannot self-correct reasoning yet. In International Conference on Learning Representations (ICLR), 2024.arxiv.org/abs/2310.01798
  31. [31]Geoffrey Irving, Paul Christiano, and Dario Amodei. AI safety via debate. arXiv:1805.00899, 2018.arxiv.org/abs/1805.00899
  32. [32]Daniel Jackson. Software Abstractions: Logic, Language, and Analysis. MIT Press, 2006.
  33. [33]Yue Jia and Mark Harman. An analysis and survey of the development of mutation testing. IEEE Transactions on Software Engineering, 37(5):649–678, 2011.DOI
  34. [34]Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. SWE-bench: Can language models resolve real-world GitHub issues? In International Conference on Learning Representations (ICLR), 2024.arxiv.org/abs/2310.06770
  35. [35]Rahul Kande, Hammond Pearce, Benjamin Tan, Brendan Dolan-Gavitt, Shailja Thakur, Ramesh Karri, and Jeyavijayan Rajendran. LLM-assisted generation of hardware assertions. arXiv:2306.14027, 2023.
  36. [36]Elliot Kim, Avi Garg, Kenny Peng, and Nikhil Garg. Correlated errors in large language models. In International Conference on Machine Learning (ICML), 2025.arxiv.org/abs/2506.07962
  37. [37]James C. King. Symbolic execution and program testing. Communications of the ACM, 19(7):385–394, 1976.DOI
  38. [38]John C. Knight and Nancy G. Leveson. An experimental evaluation of the assumption of independence in multiversion programming. IEEE Transactions on Software Engineering, SE-12(1):96–109, 1986.DOI
  39. [39]Andreas Kuehlmann, Viresh Paruthi, Florian Krohm, and Malay K. Ganai. BDD: Robust Boolean reasoning for equivalence checking and functional property verification. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 21(12):1377–1394, 2002.DOI
  40. [40]Orna Kufferman and Moshe Y. Vardi. Model checking of safety properties. Formal Methods in System Design, 19(3):291–314, 2001.
  41. [41]Orna Kufferman and Moshe Y. Vardi. Vacuity detection in temporal model checking. International Journal on Software Tools for Technology Transfer, 4(2):224–233, 2003.
  42. [42]Shuvendu K. Lahiri et al. Interactive code generation via test-driven user-intent formalization. arXiv:2208.05950, 2022.arxiv.org/abs/2208.05950
  43. [43]Martin Leucker and Christian Schallhart. A brief account of runtime verification. Journal of Logic and Algebraic Programming, 78(5):293–303, 2009.DOI
  44. [44]Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. Is your code generated by ChatGPT really correct? Rigorous evaluation of large language models for code generation. In Advances in Neural Information Processing Systems (NeurIPS), 2023.
  45. [45]Mingjie Liu, Nathaniel Pinckney, Brucek Khailany, and Haoxing Ren. VerilogEval: Evaluating large language models for Verilog code generation. In IEEE/ACM International Conference on Computer-Aided Design (ICCAD), 2023.arxiv.org/abs/2309.07544
  46. [46]Lezhi Ma, Shangqing Liu, Yi Li, Xiaofei Xie, and Lei Bu. SpecGen: Automated generation of formal program specifications via large language models. In IEEE/ACM International Conference on Software Engineering (ICSE), 2025.arxiv.org/abs/2401.08807
  47. [47]William M. McKeeman. Differential testing for software. Digital Technical Journal, 10(1):100–107, 1998.
  48. [48]Kenneth L. McMillan. Circular compositional reasoning about liveness. In Correct Hardware Design and Verification Methods (CHARME), volume 1703 of LNCS, pages 342–345, 1999.DOI
  49. [49]Md Rakib Hossain Misu, Cristina V. Lopes, Iris Ma, and James Noble. Towards AI-assisted synthesis of verified Dafny methods. Proceedings of the ACM on Software Engineering, 1(FSE), 2024.DOI
  50. [50]Fangwen Mu, Lin Shi, Song Wang, Zhuohua Yu, Binqiang Zhang, Chenxue Wang, Shichao Liu, and Qing Wang. ClarifyGPT: Empowering LLM-based code generation with intention clarification. arXiv:2310.10996, 2023.arxiv.org/abs/2310.10996
  51. [51]George C. Necula. Proof-carrying code. In Proceedings of the 24th ACM Symposium on Principles of Programming Languages (POPL), pages 106–119, 1997.
  52. [52]George C. Necula. Translation validation for an optimizing compiler. In ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), pages 83–94, 2000.
  53. [53]Marcelo Orenes-Vera, Margaret Martonosi, and David Wentzlaff. Using LLMs to facilitate formal verification of RTL. arXiv:2309.09437, 2023.
  54. [54]Arjun Panickssery, Samuel R. Bowman, and Shi Feng. LLM evaluators recognize and favor their own generations. In Advances in Neural Information Processing Systems (NeurIPS), 2024.arxiv.org/abs/2404.13076
  55. [55]Amir Pnueli and Roni Rosner. On the synthesis of a reactive module. In Proceedings of the 16th ACM Symposium on Principles of Programming Languages (POPL), pages 179–190, 1989.DOI
  56. [56]Amir Pnueli, Michael Siegel, and Eli Singerman. Translation validation. In Tools and Algorithms for the Construction and Analysis of Systems (TACAS), volume 1384 of LNCS, pages 151–166, 1998.DOI
  57. [57]Vyas Raina, Adian Liusie, and Mark Gales. Is LLM-as-a-judge robust? Investigating universal adversarial attacks on zero-shot LLM assessment. In Empirical Methods in Natural Language Processing (EMNLP), 2024.
  58. [58]Mary Sheeran, Satnam Singh, and Gunnar Stålmarck. Checking safety properties using induction and a SAT-solver. In Formal Methods in Computer-Aided Design (FMCAD), volume 1954 of LNCS, pages 108–125, 2000.DOI
  59. [59]Chuyue Sun, Ying Sheng, Oded Padon, and Clark Barrett. Clover: Closed-loop verifiable code generation. In International Symposium on AI Verification (SAIV), LNCS, 2024.arxiv.org/abs/2310.17807
  60. [60]Shailja Thakur, Baleegh Ahmad, Zhenxing Fan, Hammond Pearce, Benjamin Tan, Ramesh Karri, Brendan Dolan-Gavitt, and Siddharth Garg. Benchmarking large language models for automated Verilog RTL code generation. In Design, Automation & Test in Europe (DATE), 2023.arxiv.org/abs/2212.11140
  61. [61]Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In International Conference on Learning Representations (ICLR), 2023.arxiv.org/abs/2203.11171
  62. [62]Cheng Wen, Jialun Cao, Jie Su, Zhiwu Xu, Shengchao Qin, Mengda He, Haokun Li, Shing-Chi Cheung, and Cong Tian. Enchanting program specification synthesis by large language models using static analysis and program verification. In Computer Aided Verification (CAV), LNCS, 2024.arxiv.org/abs/2404.00762
  63. [63]Stephen Williams. Icarus Verilog. https://steveicarus.github.io/iverilog/, 2026. Accessed Nov. 2026.
  64. [64]Claire Wolf. Yosys open Synthesis suite. https://yosyq.github.io/yosys/, 2026. Accessed Nov. 2026.
  65. [65]Pierre Wolper. Expressing interesting properties of programs in propositional temporal logic. In Proceedings of the 13th ACM Symposium on Principles of Programming Languages (POPL), pages 184–193, 1986.DOI
  66. [66]Jie JW Wu and Fatemeh H. Fard. HumanEvalComm: Benchmarking the communication competence of code generation for LLMs and LLM agent. ACM Transactions on Software Engineering and Methodology, 2025. doi: 10.1145/3715109.DOI
  67. [67]Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. ReAct: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), 2023.arxiv.org/abs/2210.03629
  68. [68]Shunyu Yao, Noah Shinn, Pedram Razavi, and Karthik Narasimhan. τ-bench: A benchmark for tool-agent-user interaction in real-world domains. arXiv:2406.12045, 2024.arxiv.org/abs/2406.12045
  69. [69]YosysHQ. SymbiYosys: Front-end for Yosys-based formal verification flows. https://github.com/YosysHQ/sby, 2026. Accessed Nov. 2026.
  70. [70]Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging LLM-as-a-judge with MT-Bench and Chatbot Arena. In Advances in Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks Track, 2023.arxiv.org/abs/2306.05685

Paper details

vlnc:W_4a4s6k57
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 8

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents