WorksComputer Science
Verification-Guided Safety-Repair of Learned Controllers
Abstract
Formal verification can certify that a learned controller is safe. But what should be done when verification fails? Simply suppressing actions that lead to unsafe next states is insufficient because the dangerous action that ultimately caused the safety violation might have occurred many steps earlier. Retraining modifies the policy globally, thereby potentially introducing new failures. Instead, we introduce ReachECO, a verification-guided repair procedure that exploits the information of failed multi-step reachability analysis for synthesizing explicit constraints on controller behavior. For Differentiable Weightless Controllers, the repairs compile into exact circuit patches that leave the controller unchanged everywhere outside the targeted policy cells. We prove soundness and termination of the repair procedure for ReachECO, as well as, assuming exhaustive continuation search, completeness relative to the reachability abstraction. Empirically, we demonstrate \method's ability to successfully repair otherwise unsafe controllers in three continuous-control benchmarks: ACC, Docking2D and Car2D.
Introduction
Deep Reinforcement learning (RL) has been extensively used to train agents that interact with digital environments, such as computer games [74]. Recently, RL has also become an important part of post-training for large-language-models [52]. The original inspiration for RL, however, was to synthesize controllers for agents that interact with the physical world, such as cyber-physical, robotic, and autonomous systems [58, 68]. Training these with RL is especially appealing when their desired behavior is difficult to hardcode, but can be expressed through a reward function [66].
While powerful, using RL for training systems that are meant to interact in real-world environments also has some downsides. In particular, empirically optimizing the reward function does not provide any formal guarantees on the resulting controller [2, 14, 39]. This is particularly consequential in safety-critical applications, such as cyber-physical systems, where failures can have severe physical consequences. Stating safety requirements is often straightforward. For example, an autonomous vehicle should not collide with another road user, a robotic system should not enter a forbidden region, or a physical process should remain within prescribed operating limits [82]. However, enforcing them is far from straightforward, as the consequences of an action may only become apparent several steps later.
Arguing about safety therefore requires closed-loop reasoning about the evolution of the system (controller as well as environment) over time.
A common approach to closed-loop verification of neural-network control systems is to compute the set of reachable states, or an over-approximation of it, by repeatedly propagating sets through the neural-network controller and the system dynamics [77, 35, 36, 22, 73, 33, 60, 44, 63, 72]. If the set of all reachable states does not intersect with the unsafe set, the controller can indeed be certified to be safe. What, however, can be done if the sets do intersect? This is the central question we address in this work: given a trained controller that cannot be certified to be safe, how can it be repaired such that it subsequently passes closed-loop safety verification?
Existing approaches to closed-loop controller repair can be broadly categorized into two categories: local and trajectory-level. Local methods derive safety repairs from single-step constraints or counterexamples [79, 59]. However, a state might be safe itself, but have no available action that will keep it safe in the next state. For instance, a car approaching an obstacle may already be traveling too fast for braking to avoid a later collision. Consequently, single-step repair shifts the difficulty of reasoning about the system’s temporal dynamics to the availability of additional safety information that summarizes future behavior, such as a larger losing set, i.e., a set of states from which an unsafe state is eventually unavoidable. Computing such sets exactly, however, can be computationally intractable and is even formally undecidable for sufficiently general hybrid systems [31].
Trajectory-level methods influence the multi-step behavior of the controller, such that it avoids not only unsafe regions of the state space but also regions that inevitably lead to unsafe states at a later step [83, 30, 46, 48, 47]. Such methods typically consider individual trajectories and do not consider the verifier results during the repair. Hence, they can only establish post-hoc formal guarantees for all possible trajectories. Furthermore, they generally rely on fine-tuning to repair the original neural network, thereby forfeiting any guarantees of termination or progress [13]. One exception is Lu et al. (2023), which searches directly over counterfactual controller behaviors. Its guarantee, however, is tied to the analyzed run-time violation rather than to all trajectories from an initial-state region.
In this work, we take inspiration from the setting of static feed-forward networks. There, neural-network repair can be formulated directly as a problem of enforcing explicit constraints on the output: verification may identify inputs or regions whose outputs must change, and the repaired network is required to satisfy these constraints. This approach enables strong formal guarantees on the repair procedure [29, 25, 69] (Stoutoudis & Thakur 2021). However, extending this idea to closed-loop systems is nontrivial because the required modification may lie several time steps before the observed safety violation, and the repair constraint must account for the resulting future behavior.
Our main contribution, ReachECO, solves this problem in an effective yet efficient way for abstraction-based controllers. ReachECO uses closed-loop reachability analysis to derive repair constraints over future behavior, bringing constraint-based repair and its formal guarantees to the multi-step closed-loop setting.
Specifically, we formulate ReachECO for abstraction-based neural-network controllers [37, 72, 71] and instantiate it with Differentiable Weightless Controllers (DWCs) [41]: these are controllers that have an exact boolean circuit representation, such that repairs can be formulated as exact local circuit patches. We then prove termination and, for an exhaustive variant, completeness relative to the environment over-approximation. Finally we evaluate ReachECO on several closed-loop control benchmarks.
Verification-Guided Closed-Loop Repair
We now present Reachability-derived Engineering Change Order (ReachECO), a verification-guided procedure that derives controller-repair constraints from failed multi-step closed-loop reachability analysis. The name is inspired by engineering change orders (ECOs), which modify an existing circuit implementation to satisfy revised design requirements while preserving as much of the original implementation as possible [43, 42]. Similarly, ReachECO seeks to restore certifiability through localized patches to the controller rather than replacing it globally. Before we do so, we introduce the main concepts necessary to reason about the properties of discrete systems that operate in a continuous environment, in particular their safety. Formal definitions and additional details are given in Appendix § 9.
Abstraction-based Control. We study a control task with continuous state space and system dynamics governed by a differential equation, , where is a control action. We consider controllers that are evaluated every time units, with the selected action held constant between controller updates (called zero-order hold). While cyber-physical systems necessarily have to process real-valued inputs, because they operate in continuous (real-world) environments, it has been observed that forming abstractions of the continuous state-space into finite sets can often preserve near-optimal behavior [1]. Similarly, it has been observed that strong quantization of a neural controller’s input space often incurs only limited performance degradation for standard continuous-control tasks [40 b].
Such abstractions can substantially simplify formal verification: controllers defined over a finite input partition avoid much of the complexity associated with having to bound neural network activations over continuous input regions, enabling significantly more scalable closed-loop reachability analysis [37, 71, 72, 78]. Consequently, in this work, we focus on piecewise-constant abstraction-based controllers (Def. 2, Appendix § 9), which partition the state space into finitely many axis-aligned policy cells and assign a single action to each cell .
Differentiable Weightless Controllers Importantly to maintain efficiency, the use of abstraction-based controllers is not contradictory to gradient-based learning. In particular, Differentiable Weightless Controllers (DWCs) [41] can be trained with gradient descent but nevertheless admit an exact representation as a Boolean circuit. Their quantized input encoding creates the finite policy partition , while their quantized outputs lead to a finite action alphabet . Hence, DWCs provide a natural instance of the abstraction-based controller class considered here. DWC circuits consist primarily of Boolean lookup tables (LUTs). A 6-input LUT (LUT6), as commonly provided by modern FPGAs, stores the truth table of an arbitrary Boolean function of six inputs, requiring only bits [4]. Consequently, DWC inference can be implemented directly as Boolean circuit evaluation without floating-point multiplication. Despite this strongly quantized representation, DWCs have been shown to learn policies of comparable quality as more conventional neural-network controllers on challenging continuous-control tasks, such as Humanoid [41]. Hence, we instantiate our framework for DWCs. In § 2.3, we exploit the exact circuit representation to realize repairs as bit-exact local patches of the original controller.
Abstraction-Based Closed-Loop Reachability Analysis. Closed-loop reachability analysis propagates sets of possible states through the controller and system dynamics. Given a set and action , we assume sound over-approximations of the one-step successor set and of all states visited before the next controller step, where soundness means that no trajectory is omitted that the true system starting in under could take. Every over-approximation is further enclosed by an axis-aligned hyperrectangle in , which we call a box, to enable efficient propagation by the verifier that performs the reachability analysis.
For an abstraction-based controller, let every box be contained in exactly one policy cell, which we denote by . Hence, the controller assigns a single exact action to all states in . For such a box, we denote by a set of successor boxes by computing and, because the post-image may intersect multiple of the policy cells, subsequently splitting along policy-cell boundaries. For scalability, if multiple boxes fall into the same policy cell, they may also be merged again [37, 71, 72, 78].
Now, starting from an initial set of boxes, , repeated application of yields a sound over-approximation of the closed-loop reachable set. Let denote the collection of reachable boxes at step , then we define, for any time horizon :
Consequently, over-approximates every state visited by the closed-loop system from during the first control steps. We now have all components to formally define our central object of interest: certified safety.
Definition 1 (Certified safety). A controller is called certified safe to horizon , if
where is the set of unsafe states. We call certified safe to horizon if for some step the reachable-set computation establishes an inductive fixed point,
In addition, for a reach-avoid task, any trajectory that has entered the absorbing goal region is subsequently assumed to be safe.
A controller that has been proven certified safe is also safe under the actual dynamics . Conversely, however, the inability to certify a controller does not imply that it is necessarily unsafe. This, however, is not a shortcoming of this specific definition. It is known that exact reachability, and therefore safety, is not only computationally intractable but formally undecidable for sufficiently general hybrid systems [31].
Controller Repair. When a controller fails safety verification, a natural next step is to repair it. Formally, we define a repair instance as an abstraction-based controller together with sound propagation operators and for the system dynamics, an initial set , an unsafe set , an optional absorbing goal set , and a verification budget . The goal is to obtain a controller that is certified safe in the sense of Definition 1, while retaining as much of the behavior of the original controller as possible. Repairs may assign each policy cell a new action from a finite repair alphabet or keep the original action .
ReachECO
ReachECO enables effective yet efficient controller repair. Algorithm 1 and 2 provide pseudo-code, based on a number of arguments and subroutines that we will introduce in this section. Appendix C gives additional details and formal definitions. We provide a high-level sketch of ReachECO’s working principles and how it achieves local repair in a closed-loop setup.
At its core, ReachECO alternates between forward closed-loop reachability analysis and backward repair search. It maintains the sets of reachable boxes computed under the current controller together with a certified losing set . The latter contains regions for which the algorithm has established that no admissible continuation can avoid a future safety violation. Initially, may contain only the unsafe set and can be expanded as additional losing regions are certified. Throughout is a subset of the true losing set . We call action one-step safe from a box if its inter-sample tube avoids and its post-image avoids the current losing set underapproximation:
Suppose forward verification encounters one or more boxes for which is not fulfilled. ReachECO collects these failing boxes as the initial repair obligations . We then search for alternative controller actions that admit a safe continuation. The continuation backend determines which candidate actions are explored. If no continuation exists from a failing box, then intervening at that time step is already too late, and ReachECO backtracks to . The required safe continuation horizon, however, increases by one as we want to establish a safe continuation up to , precisely where the initial safety violation was encountered. This
Algorithm 1 REACHECO
1: Input: $\pi$, $\mathcal{X}_0$, $\mathcal{L}$, budget $T$, backend $\mathcal{W}$
2: $\pi' \leftarrow \pi$; $\Gamma \leftarrow \emptyset$; $\mathcal{X}_0 \leftarrow \mathcal{X}_0$; $k \leftarrow 0$
3: while $k < T$ do
4: $(\mathcal{X}_{k+1}, \mathcal{F}) \leftarrow \operatorname{REACHSTEP}(\mathcal{X}_k,\pi',\mathcal{L})$
5: if $\mathcal{F} \ne \emptyset$ then
6: $(s,\pi',\Delta\mathcal{L}) \leftarrow \operatorname{BREPAIR}(\mathcal{F},\mathcal{L},\mathcal{W},k)$
7: $\mathcal{L} \leftarrow \mathcal{L} \cup \Delta\mathcal{L}$ {certified losing}
8: if $s = \operatorname{UNKNOWN}$ then
9: return $\operatorname{UNKNOWN}$
10: end if
11: $k \leftarrow \operatorname{EARLIESTAFFECTED}(\pi',\Delta\mathcal{L})$
12: else if $\bigcup \mathcal{X}_{k+1} \subseteq \bigcup \mathcal{X}_k$ then
13: return $(\pi',\infty)$ {inductive invariant}
14: else
15: $k \leftarrow k+1$
16: end if
17: end while
18: return $(\pi',T)$
Algorithm 2 BREPAIR. $\Gamma$ is the set of committed constraints and is stored across calls.
1: Input: next step failing boxes $\mathcal{F}$, $\mathcal{L}$, $\mathcal{W}$, $t$
2: $\Gamma \leftarrow \emptyset$; $\mathcal{L}_0 \leftarrow \mathcal{L}$
3: while true do
4: $(\Gamma_w,\mathcal{F}_o,\Delta\mathcal{L}) \leftarrow \operatorname{FINDCONT}(\mathcal{F},\mathcal{W})$
5: $\mathcal{L} \leftarrow \mathcal{L} \cup \Delta\mathcal{L}$; $\Gamma \leftarrow \Gamma \cup \Gamma_w$
6: if $\mathcal{F}_o = \emptyset$ then
7: $(s,\pi',\mathcal{F}_o,\Gamma,\overline{\Gamma}) \leftarrow \operatorname{JOINT}(\Gamma,\overline{\Gamma})$
8: if $s = \operatorname{SAT}$ then
9: $\Gamma \leftarrow \Gamma \cup \overline{\Gamma}$
10: return $(\operatorname{REPAIR},\pi',\mathcal{L}\setminus\mathcal{L}_0)$
11: else if $s = \operatorname{UNKNOWN}$ then
12: return $(\operatorname{UNKNOWN},\pi',\mathcal{L}\setminus\mathcal{L}_0)$
13: end if
14: end if
15: if $\exists(B,h)\in\mathcal{F}_o : h=t+1$ then
16: return $(\operatorname{UNKNOWN},\pi',\mathcal{L}\setminus\mathcal{L}_0)$
17: end if
18: $\mathcal{F} \leftarrow \operatorname{BACKTRACK}(\mathcal{F}_o)$
19: end while
backtracking and forward search process continues until a safe continuation is found for every failure or the initial set is reached. During this search, ReachECO may also certify additional regions as losing and add them to .
Each successful continuation search yields a repair constraint. Each constraint ensures a safe continuation from the obligation that generated it. Constraints discovered across different failures and repair rounds are accumulated and must ultimately be satisfied by a new controller . This joint realization is nontrivial because independently derived constraints may require incompatible actions in the same policy cell. Section 2.2 describes how ReachECO resolves such conflicts while preserving progress of the repair procedure. Once a jointly realizable repair is found, the controller is updated. Because this change may alter previously computed reachable sets, ReachECO resumes forward verification from the earliest time step affected by either the controller modification or an expansion of .
In summary, at the core of ReachECO lies an intuitive procedure of alternating forward propagation with lazy backtracking. Our main contribution lies in how safety failures are handled when they are encountered: resolving conflicts while ensuring finite progress and, under exhaustive-search assumptions, completeness relative to the verifier abstraction.
Obligations, Constraints and Conflicts
We next describe how ReachECO represents and combines the repairs discovered during backward search. We first introduce obligations, which encode the need to establish a safe continuation from a reachable region. We then show how successful continuation searches are stored as repair constraints. Finally, we describe how constraints from different failures are jointly resolved while ensuring progress of the repair procedure.
Obligations. An obligation is a pair , where is a box and is the number of future control steps for which a safe continuation must be established. The initial obligation set is obtained from the boxes that violated the SAFESTEP predicate during forward reachability propagation. Each one is initialized with horizon . Starting from , FINDCONT uses the same sound set propagation as forward verification, but explores alternative actions proposed by .
Thus, for an obligation , the search has the following alternating structure:
The backend determines which candidate actions are explored. An exhaustive backend considers every action in . This leads to a finite-horizon safety game-like search (Tabuada, 2009).
If FINDCONT cannot establish an -step safe continuation from , the obligation is returned as unresolved (). BACKTRACK then replaces it by obligations on its predecessor box at the preceding time step and increases the required continuation horizon to . Thus, every backtracking step searches for an earlier intervention while accounting for one additional future step. If no safe continuation can be established from time step 0, REACHECO returns UNKNOWN: under the employed reachability abstraction and continuation backend, no feasible continuation could be established from that obligation.
Constraints. A successful continuation search is represented by a finite game DAG containing the safe continuations discovered by the backend . Each node corresponds to a box and stores the candidate actions that admit a safe continuation for the remaining horizon . At the root node , these are exactly the box and horizon of the corresponding obligation. The successor boxes generated by each action, that were established to satisfy Equation (5), appear as child nodes with horizon .
Together with its root time step , the game forms a repair constraint . By retaining all feasible continuations found by , rather than committing immediately to a single one, the subsequent joint realization step has flexibility when combining repairs from different failures. We call anchored if , since its root has reached the initial time step and cannot be backtracked further. REACHECO maintains two sets of constraints. The committed constraints were established in previous calls to BREPAIR, whereas the current constraints were generated during the present call. This distinction matters during conflict resolution because committed constraints where generated under an earlier controller.
Conflict Resolution. Once every current obligation has produced a repair constraint, JOINT searches for a single stationary controller that satisfies all accumulated constraints. Satisfying a constraint means selecting one of its continuations: whenever a game node is reached, the controller must choose one of the actions stored at that node. Since every box belongs to a policy cell , realizing a continuation ultimately imposes action requirements on these cells.
Constraints generated independently need not be jointly realizable. In particular, different continuations may require incompatible actions in the same policy cell, while the repaired controller must assign exactly one action to each cell. We encode this finite joint-realization problem using CPSAT [53], requiring one action per policy cell and satisfaction of every repair constraint. Among feasible assignments, we prefer controllers that remain close to the original policy.
Accordingly, JOINT has three possible outcomes: SAT, returning a jointly satisfying controller; BACKTRACK, returning a set of conflicting non-anchored constraints as new obligations ; or UNKNOWN, if the conflict cannot be resolved because it involves only anchored constraints. If the constraints are jointly infeasible, REACHECO first backtracks conflicting non-anchored constraints generated in the current repair round. If the conflict instead involves anchored constraints, non-anchored committed constraints may be discarded in favor of newly anchored constraints. Anchored constraints are never discarded. Consequently, REACHECO returns UNKNOWN only when repair fails at the initial time step: either no safe continuation can be established for an anchored obligation, or the anchored constraints are jointly infeasible under the employed abstraction and backend.
ECO Patch Compilation
The repair from JOINT returns a set of policy cells , where actions differ from the original controller. The repair is realized as:
For DWCs, an initial thermometer encoding [16] provides the threshold predicates needed to identify each policy cell. We therefore implement each as an AND reduction over these bits. The resulting one-hot guards are encoded into a memory address selecting the corresponding correction , which is added to the DWC’s integer output. Address zero stores the zero correction, so the controller is unchanged outside the repaired cells. The final BRAM-based output decoding stage of the DWC remains unchanged. Figure 1 illustrates the constructed patch. It is similar to support-network repair in REASSURE [25] and lookup-based patch architectures [69], but realizes the repair exactly as a digital circuit. Note that the patch can be executed largely in parallel with the original controller. Circuit synthesis and implementation details are given in Appendix § 12.

Figure 1. ECO patch architecture. Existing thermometer predicates are AND-reduced to implement the policy-cell identification , from these an address selecting the stored correction is obtained. The correction is applied exactly and only to repaired policy cells.
Theoretical Guarantees
The purpose of ReachECO is to provide controllers with formal guarantees; hence, the repair procedure itself should also admit formal guarantees. We state these below as theorems. The corresponding proofs and implementation assumptions are given in Appendix § 10. Soundness applies to the full procedure. Completeness is stated with respect to the final learned losing set of ReachECO.
Theorem 1 (Soundness). Suppose ReachECO returns with rather than UNKNOWN. Then is certified safe to horizon (Definition 1). In particular, no trajectory of the closed-loop system under starting in enters during the first control steps, and for all time if .
Theorem 2 (Termination). Assume that the initial set has a finite abstract representation, and that the policy partition , the continuation alphabet , and the verification horizon are finite. Furthermore, assume JOINT always returns a repair if there exists one under the constraints. Consider monotone-losing set expansion and persistent merge-mask refinement as described in Appendix § 9. Then ReachECO terminates and returns either , with , or UNKNOWN.
Here denotes that every transition in the unmerged -step unfolding from under satisfies SafeStep with respect to a losing set . This losing set is the last losing set constructed during execution of ReachECO and contains . The recursive definition is given in Appendix § 9. We denote with the set of all controllers with the action alphabet over the grid .
Theorem 3 (Completeness under exhaustive search). Under the assumptions of Theorem 2, let be the fixed initial box representation, containing every anchored root. Let denote the final losing set constructed during execution. Use the exhaustive backend over the original action alphabet . If some satisfies
then ReachECO returns a controller that is certified safe (Definition 1).
While exhaustive continuation search is largely aspirational, due to potentially large action spaces, Corollary 2.4 follows immediately from it. Importantly, Corollary 2.4 implies that ReachECO always returns a repair if there exists one in the searched action alphabet and under the utilized Tube and Post image overapproximations and JOINT does not time out.
[Completeness relative to the continuation backend] Under the assumptions of Theorem 2, suppose there exists a stationary controller satisfying
Furthermore, suppose that for every box encountered in its unmerged continuation with remaining horizon , the backend explores . Then ReachECO returns a certified safe controller.
Theorems 1, Theorem 2 and Corollary 2.4 are directly applicable to our ACC and Car2D experiments. Docking2D employs an additional greedy restart optimization; however, with minor modifications, all theorems extend to this setting. In particular, one needs to account for an additional outer loop that only returns once a repair-free pass has been performed. Hence, the theoretical guarantees also hold for its experiments.
ReachECO attempts to synthesize a controller that remains close to the original policy in terms of the -distance in the popcount integer space. However, finding an optimal repair is generally computationally hard: even minimal strategy repair in finite reachability games is NP-complete [28]. Hence, ReachECO employs heuristics and accepts non-optimal assignments after a search timeout.
Experiments
We investigate three tasks, showing different properties of ReachECO across them: Adaptive Cruise Control (ACC) [27, 70], Docking2D from the AINNCS competition [45], and Car2D, which uses the NAV dynamics from [45]. We briefly summarize Docking2D and Car2D here and defer their full analysis to Appendix A. All experiments were performed on an AMD EPYC 9534 (Zen 4) CPU, with an NVIDIA L40S GPU used where applicable. In particular, ReachECO’s reachability analysis runs on the GPU. Our implementation is available on GitHub.
Adaptive Cruise Control
Setup. A controlled vehicle must avoid a constant-velocity lead vehicle while maintaining comfortable acceleration. Relative distance and velocity are denoted by and , respectively, with indicating that the vehicles approach each other. We train five DWCs following Kresse & Lampert [41]. The system has a known losing set (Figure 2, everything below dotted black line): states above its boundary can avoid collision by braking, whereas in states below the line, an eventual collision is unavoidable. We initialize ReachECO with boxes entirely outside satisfying and , covering of the corresponding safe region. The unsafe set is defined as , corresponding to a rear-end collision between the lead and controlled vehicle. All ReachECO runs use , and all runs reach a closed-loop fixed point within this budget, certifying infinite-horizon safety. Details of the dynamics and training are given in Appendix E.
Figure 2. ACC (, with ): Showing , the repaired cells with the associated magnitude change, and the final fixed-point . The learned losing set is shown in orange. Changes in the controller are localized close to the losing set boundary, which ReachECO was not provided with in this run.
Losing set knowledge reduces runtime. We cumulatively evaluate four levels of losing set knowledge: (i) no expansion, (ii) expansion, (iii) initialization with an offline-computed fixed point, and (iv) the exact braking invariant. uses 11 discrete actions including zero and both extrema. We additionally evaluate a single-action full-braking backend , under (ii).
| Method | [s] |
| No losing-set | |
| Online losing-set | |
| Initial losing-set | |
Table 1. ACC: verification + repair runtime [s] at over 5 seeds.
Table 1 shows that stronger safety information reduces repair runtime. For , runtime drops from 163s without losing-set learning to 29s with online learning, 11s with an underapproximated initialized losing set, and 3s with . The full-braking and invariant additionally scales to (Table 2). At this initial set size, games of depth-11+ are necessary, making infeasible. As shown in Figure 5 in the Appendix: losing-set expansion eliminates a large number of deep continuation games. At , this reduces depth-3–7 games by a large amount, while the knowledge of requires only depth-1 games.
Table 2. ACC at . denotes the runtime for verification plus repair. If verification fails, reports repair time only. Runtime and repair size are reported over five trained model seeds. † denotes a ReLU network. For ReachECO, Param. denotes the number of patch LUT6s; for NCubeV+REASSURE, results are reported only for the network from [70], and Param. denotes the number of additional floating-point parameters. The first column indicates the losing set information provided: denotes the known losing set, denotes only the unsafe set.
| Method | Verified | [s] | Param. | Change [%] | mean | |
| ReachECO | ✓ | |||||
| BBReach + Supervised | ✓ | 0 | ||||
| NCubeV + Supervised† | ✓ | 0 | ||||
| NCubeV + REASSURE† | ? | 88 | 2350 | 1.2 | 0.003 | |
| ReachECO () | ✓ | |||||
| STLGym† | × | 0.0 |

Figure 5. ACC game depths for different losing set configurations.
Repairs are localized. ReachECO successfully repairs all five DWC seeds and increases their mean reward from 2434 to 3861; per-seed results are reported in Table 5 in the Appendix. Qualitatively, Figure 2 shows that the resulting modifications concentrate near the boundary of , even when ReachECO is provided only with the unsafe set . Table 2 further shows that access to and the more general backend reduces both the number of modified policy cells and the mean action change.
Table 5: ACC evaluation over 1,000 rollouts per controller. Reward is mean standard deviation. Unsafe denotes sampled collisions. All controllers, including the reference and invariant-repaired DWCs, use the same initial-state sampling protocol.
| Seed | Reward | Unsafe (%) | Architecture |
| Teuber et al. [70] | 3254.16 2001.49 | 12.70 | ReLU |
| Before repair | |||
| 1 | 2439.83 2589.29 | 25.60 | DWC |
| 2 | 1957.28 2778.07 | 33.40 | DWC |
| 3 | 3183.04 1929.58 | 12.20 | DWC |
| 4 | 2090.53 2646.71 | 29.80 | DWC |
| 5 | 2499.29 2387.39 | 22.10 | DWC |
| After invariant repair | |||
| 1 | 3943.31 40.14 | 0.00 | DWC |
| 2 | 3916.49 38.87 | 0.00 | DWC |
| 3 | 3895.33 39.61 | 0.00 | DWC |
| 4 | 3797.75 37.71 | 0.00 | DWC |
| 5 | 3754.34 48.64 | 0.00 | DWC |
Table 5. ACC evaluation over 1,000 rollouts per controller. Reward is mean standard deviation. Unsafe denotes sampled collisions. All controllers, including the reference and invariant-repaired DWCs, use the same initial-state sampling protocol.
Other benchmarked methods require or do not certify. We compare against methods with and without access to . When is available, we construct counterexample-guided repair loops using either BBReach [72] for DWCs or NCubeV [70] for ReLU controllers. Verification identifies failing regions, based on which the controller is repaired and reverified until the first successful certificate is obtained. For supervised repair, we fine-tune on the identified failures together with safe transitions from the original controller to reduce unintended changes; full details are given in Appendix E. For the ReLU controller, we additionally combine NCubeV with REASSURE [25], which constructs a local patch network from the identified unsafe regions. We cannot certify the resulting patched network with NCubeV because its architecture is outside the verifier’s supported network class, and therefore report the resulting repair size without a closed-loop certificate. Among the methods that obtain a certificate of safety, ReachECO is faster and produces smaller behavioral changes than the supervised fine-tuning baselines, requiring only a few hundred additional LUT6s to repair the DWC. REASSURE style patches yield a smaller amount of changes. This is at least in part due to the fact that the underlying ReLU network is already safer than the initial DWCs. Nevertheless, it requires a patch size of 2350 additional floating-point parameters.
In a second setup, we assume that only the unsafe set is known; for this setting, we use STLGym [30] to fine-tune the original controller for an additional 200,000 time steps. Figure 6 in the Appendix shows the resulting reward and empirical safety across training. Training is unstable across seeds: in some runs, reward degrades substantially, and none of the resulting controllers achieves 100% empirical safety.

Figure 6. ACC reward and empirical safety during STL-Gym fine-tuning across five seeds (Hamilton et al., [30]). None of the five seeds achieves empirical safety when evaluated from a small region slightly above the invariant boundary. Both reward and empirical safety vary across seeds and training checkpoints.
Docking2d and Car2D: Takeaways
The main analysis of these two case studies is deferred to Appendix A, we highlight main results.
Docking2D: Reachability enables highly localized patches. Docking2D has a large continuous state space and long closed-loop horizons. Nevertheless, ReachECO certifies all our three DWC controllers, till docked, while modifying only a small number of cells. For the seed exhibiting the most empirically unsafe transitions, only 3,309 policy cells are changed, corresponding to less than of the overall policy cells.
Car2D: enabling certification through small changes. The original controller exhibits no unsafe behavior in 1,000 sampled rollouts, yet closed-loop reachability fails to certify it, identifying a potential safety violation after three timesteps. ReachECO repairs the controller in 42 s, requiring continuation games of up to depth 3 while modifying only 39 policy cells. Several repaired regions are almost never visited by the sampled trajectories. These regions may correspond either to rare behaviors missed by sampling or to conservatism introduced by the reachability overapproximation; in either case, modifying them is sufficient to make the controller certifiable.
Summary and Discussion
We introduced ReachECO, which enables targeted repair of abstraction-based neural network controllers to fulfill formally certified safety conditions. On a technical level, ReachECO brings the constraint-based view established in static neural-network repair to the closed-loop setting by explicitly accounting for the future behavior induced by controller modifications. Repair constraints are derived lazily from failed forward reachability analysis and subsequent finite-horizon continuation reasoning. This formulation enables strong formal guarantees on the repair procedure, including termination and, for an exhaustive variant, completeness relative to the reachability abstraction. For the special class of Differentiable Weightless Controllers (DWCs), we further showed that the resulting repairs can be compiled into exact circuit patches. Because ReachECO only modifies regions encountered during closed-loop reachability analysis, the resulting repairs can remain highly localized in the controller’s input space. At the circuit level, the additional logic consists primarily of LUT6s, allowing the repair to be realized as a lightweight overlay on the original controller.
Some limitations remain: ReachECO relies on deep continuation games, which can become computationally expensive for long-horizon problems. This can be mitigated by additional prior information, such as knowledge of the losing region. Our experiments show that even partial losing-set information can substantially reduce the required search. Even though repair patches are highly localized in the controller’s input space, many of them might be needed for originally unsafe controllers. A promising direction for future work is to exploit additional structure during patch compilation.
Our current implementation additionally relies on relatively coarse, benchmark-specific post-image and inter-sample tube computations chosen for efficient GPU execution. Tighter GPU-based reachability procedures could reduce conservatism, while more general tube constructions would make the approach easier to transfer across dynamical systems. Finally, our experiments consider relatively low-dimensional systems, as is common in closed-loop reachability benchmarks. Scaling both verification and the additional repair search to higher-dimensional dynamics remains the most important future challenge.
AI USE STATEMENT
In this work, we used generative AI tools for developing conceptual frameworks, formulating mathematical claims, assisting in the writing of the presented proofs, for providing feedback on research methodology and experiments, implement methods, and interpreting results. We have not used generative AI tools for proposing or refining hypotheses, supporting qualitative and thematic data analysis, or providing critical ingredients for proving mathematical claims. Synthetic data sets generation, assisting with translation, cleaning and reformatting datasets are not applicable to this work.
Additionally, we used generative AI tools to assist with figure generation, code prototyping, drafting and polishing text, brainstorming, searching for information, and proposing keywords for the paper. The tube overapproximations used in this work were initially derived with assistance from generative AI tools. We reviewed all AI-assisted work. The original prototype implementation was extensively audited, rewritten, and restructured, and AI-provided technical information was carefully reviewed and independently verified by at least one of the authors. We also conducted conventional literature searches to verify and complement AI-assisted information retrieval. We take responsibility for the final content of this work, including text, claims, code, and other artifacts produced with the aid of generative AI.
REPRODUCIBILITY STATEMENT
Our implementation is available on GitHub. Implementation details are provided in Appendix § 9. We have provided proofs of our Theorems in Appendix § 10. Furthermore, we have provided detailed descriptions of all case study environments we investigate in Appendix § 11.
Additional Experiments
Docking2D
Setup. Docking2D is a challenging benchmark for neural-network safety controller verification. For example, none of the entries to the ARCH-COMP AINNCS competition in 2025 was able to certify the full provided initial region Lopez et al., [45]. We consider a larger initial region, covering the competition’s region, and, moreover, verify the reach-avoid property proposed in Mandal et al., [51], rather than only the original 40-step ARCH-COMP specification. The resulting set propagation requires more than 850 time steps before all sets have converged to the goal region around the origin.
We train three DWC policies with PPO; training details are given in Appendix § 11. For continuation search, uses a backup controller that targets the docking point at a desired speed of and proposes the corresponding discrete action. The trained DWCs achieve reward close to the competition tanh policy, with some seeds remaining empirically unsafe before repair. We split the initial set into four smaller initial boxes. To reduce repeated recomputation, we use a greedy restart strategy: after a repair, verification resumes from the depth at which the corresponding continuation game was constructed rather than from the earliest potentially affected time step. Since certification is reported only after a subsequent repair-free pass over all initial boxes, this optimization does not weaken the final guarantee. Our main insights are:
Depth-1 games are insufficient. When performing repair, we observe a total of 21 depth-2 games. These occur when the intersample tube does not allow a safe continuation, requiring backtracking.
Repairs are highly localized. Table 3 reports reward, repair, and verification time as well as additional LUT6s for patch circuits. Seed 1 is the only controller that is empirically safe over the entire evaluation region, yet still requires modifications to 5 policy cells before it can be certified. Seed 2 exhibits empirical failures outside the certified initial region but none from ; ReachECO can certify it without having to modify the controller at all. Seed 3 is empirically unsafe and requires the largest circuit patch, adding roughly 6.4k LUT6s. Nevertheless, only 3,309 policy cells are modified, corresponding to less than of the overall cells. Thus, even when the compiled circuit patch is comparatively large, the behavioral modification remains extremely local. Figure 3 visualizes these changes and shows that they are concentrated primarily around the docking region.
Table 3: Docking2D: reports empirical safety for 1000 rollouts from the 130 meter starting circle. reports empirical safety from the initial region (before repair). reports the overall time for both repair and verification. tanh denotes the controller utilized in Lopez et al. [45]. Param reports the increase in size in terms of LUT6s of the repaired controller. Note that naive repair is an estimated lower bound and not synthesized.
| original | after repair | Verified | [h] | ours | naive | |||
| tanh | 95.2 | 100.0 | 3.096 0.069 | · | · | · | · | |
| 1 | 100.0 | 100.0 | 3.066 0.067 | 3.066 0.067 | ✓ | 0.25 | +50 | +92k |
| 2 | 95.6 | 100.0 | 2.994 0.061 | 2.994 0.061 | ✓ | 0.38 | +0 | +87k |
| 3 | 96.0 | 75.9 | 3.003 0.148 | 3.060 0.066 | ✓ | 1.09 | +6.4k | +120k |
Table 3.
Figure 3. Docking2D: repaired regions of the DWC seed 3 projected onto the plane.
Reachability information substantially reduces patch size. We compare against a naive patching scheme that does not use reachability-derived information, but instead patches every fully safe policy cell whose transition can enter the unsafe set from anywhere in the safe region. As shown in Table 3, the resulting patches are substantially larger than those produced by ReachECO, requiring up to several orders of magnitude more LUT6s. Moreover, these numbers are conservative: the naive construction considers only cells fully contained in the safe set and therefore underapproximates the patch that would be required over the entire boundary.
Car2D
Setup. We investigate a modified version of the nonlinear NAV benchmark from the ARCH-COMP AINNCS competition (Lopez et al., [45]). We train a DWC controller with TD3 (Fujimoto et al., [26]), with specifics given in Appendix § 11. The controller must reach the goal region while avoiding the two obstacles. Figure 4 shows the initial set and the obstacles. During repair, we modify only the steering output and consider a repair alphabet consisting of three increasingly large steering corrections in each direction, together with the two extreme steering actions.

Figure 4. Car2D rollouts before and after repair, with modified policy cells highlighted.
Repair restores certifiability. The original controller is empirically safe over 1000 rollouts sampled from the initial set, exhibiting no observed unsafe trajectories. Nevertheless, formal verification with our BBReach implementation finds a reachable-set intersection with the unsafe set at time step . We subsequently apply ReachECO. During repair, ReachECO constructs 1358 depth-1, 88 depth-2, and 7 depth-3 continuation games. After 42 s, ReachECO returns a repaired controller modifying 39 policy cells that passes verification. Figure 4 compares rollouts of the original and repaired controllers and visualizes the modified policy cells. Notably, the upper repaired regions are virtually absent from the sampled rollouts. Their modification is nevertheless required for the reachability analysis to pass.
Appendix: Related Work
Shielding. Shielding enforces safety at runtime by monitoring the nominal controller and overriding actions when necessary ([12, 2]). Shields may be synthesized offline or perform additional safety reasoning online ([7] (Königshofer et al. 2022)). Corsi et al. [20] use offline verification to identify regions in which an online shield needs to be active. More recently, Maderbacher et al. [49] propose an architecture in which a performance-optimized controller is permitted to act only inside a certified recoverable region, while a baseline controller intervenes otherwise.
Our repaired controller is closely related to a sparse, statically compiled shield: it overrides the nominal policy only on a finite set of state regions. The main distinction is that ReachECO uses failed closed-loop reachability verification to discover the required interventions offline, incorporates them directly into the nominal controller, and reverifies the resulting controller. Consequently, deployment requires neither runtime safety reasoning nor invocation of a separate backup controller.
Closed-Loop Verification of Neural Network Controllers Safety verification of neural-network controlled systems is often performed with reachability analysis: an over-approximation of the set of reachable closed-loop states is propagated forward in time and checked for intersection with an unsafe set. A large ecosystem of methods and tools implements this basic recipe with different set representations and abstractions of the network ([77, 35, 36, 22, 73, 44, 33, 32, 60, 63, 71, 72]). The ARCH-COMP AINNCS category provides a yearly comparison between some of these tools [45]. A second line of research focuses on certificate-based verification, reducing the need to argue about entire trajectories to single-step queries on the certificate [17, 81, 51, 23]. However, this adds the need to both synthesize an appropriate certificate and subsequently check it.
Another reachability-related line of work trains controllers that are constant on a finite partition of the state space, so that closed-loop verification never has to perform over-approximation of the neural network. Jin et al. [37] introduce Trainify, where a neural network controller is trained in a CEGAR loop that iteratively refines an abstraction of the state space until the control objectives can be fulfilled; since actions are constant per abstract state, neural network verification is skipped entirely and model checking is performed. Tian et al. [72] instead fix an initial abstraction, train on it, and perform reachability analysis (BBReach), again exploiting that constant actions make over-approximating the network unnecessary. Tian et al. [71] extend the controller class to piecewise-affine decision networks, enabling coarser abstractions, and Yang et al. [78] extend abstraction-based controller verification to probabilistic safety.
Constraint based Repair. Counterexample-guided finetuning is not guaranteed to terminate [13]. Hence, there has been work to provide stronger repair guarantees by modifying the internal parameters of a network based on input-output constraints [29, 64, 8, 69, 65] or adding patches to it [25]. These approaches reason about the network specification itself rather than the closed-loop system dynamics.
Local Controller Repair. Encompasses methods that rely on single step reasoning, either performing repair based on counterexamples [59] or one-step reachability sets [79]. Teuber et al. [70] utilize a control envelope to transform the global controller safety problem into a local one. However, this requires the envelope to be available. Santa Cruz et al. [59], perform constraint optimization at a known counterexample while safeguarding certain global properties. Mandal et al. [51] reduce the multistep repair problem to a single-step one by fine-tuning a Lyapunov-barrier certificate together with the controller; this shifts the computational complexity to checking the certificate.
Trajectory-Level Controller Repair. Many global controller repair methods rely on parameter fine-tuning, often based on STL robustness [50, 24]. Examples are STLGym [30] and ISAR [47]. Other methods rely on discovering failing trajectories and supervised fine-tuning [48] or direct changes to the policy discovered through exhaustive reasoning [46]. Lastly, Zhou et al. (2020) and Yu et al. [80] utilize a runtime monitor and perform fine-tuning of the controller based on runtime feedback.
Set-based training A set of methods attempts to improve the likelihood that a set-based closed-loop proof will succeed [76, 75, 18, 62]. In these methods set-based information is included in the training loss, aiming to reduce over-approximation during verification.
Circuit Rectification Repairing a deployed circuit with a small patch instead of re-synthesizing it is long-standing practice in logic design, where engineering change orders (ECOs) rectify an implementation towards a revised specification with minimal modification [43, 42]. Our patch compilation adopts this philosophy for learned controllers, with one key difference: the revised specification is not written by a designer but generated automatically, in the form of action-correction constraints discovered by a closed-loop reachability verifier.
Strategy Repair Synthesis based on discrete-reachability games has a long history [21]. However, even for discrete state-spaces, prior work on repair has addressed the problem of strategy repair only relatively recently [28, 15]. ReachECO addresses an analogous problem for learned controllers that operate on continuous dynamics, however, in our setting the underlying game is not available a priori: relevant transitions are constructed lazily from failed closed-loop reachability verification.
Formal Definitions and Algorithm Details
Reachability
As in the main body of text: we study a task with a continuous state space and system dynamics governed by a differential equation, , where is a control action.
Now, let denote the state reached after time from under constant input . For a set of states, , and an action , define
Because of the continuous nature of the underlying dynamics, exact reachability cannot be computed in general. We therefore rely on sound over-approximations.
Problem Setting
We now formally define piecewise-constant abstraction-based controller, as they are the controller type studied throughout.
Definition 2 (Piecewise-constant abstraction-based controller). A controller, , is called piecewise-constant abstraction-based, if is finite and exposes a finite axis-aligned policy cell partition of such that for any : . Hence, equivalently induces a finite map , and we write for the set of all controllers over the fixed policy partition and action alphabet .
Let us now formally define the repair problem.
Definition 3 (Repair instance). A repair instance consists of an abstraction-based controller , a finite set of continuation actions , sound propagation operators and , an initial set , given as a collection of boxes, an unsafe set , an optional absorbing goal set , and a verification budget .
Definition 4 (Repair class). Given a repair instance, set for every . Then
is the repair class of . It contains all controllers that either retain the original action or select a continuation action independently on each policy cell.
Definition 5 (Certified repair). A certified repair of a repair instance is a controller that is certified safe according to Definition 1.
Controllability and Losing Regions
Given a failing reachable region, repair must determine how many time steps before the reported violation an intervention is needed. Whether an intervention at a given state can still prevent the violation depends on whether safety can be maintained from the resulting successor states. For the infinite time case this is captured by the notion of controlled invariance [10, 11]; its maximal set corresponds to the viability kernel in viability theory [5] and to the winning region of a safety game (Tabuada, 2009).
We define controllability relative to the finite continuation alphabet used by repair.
Definition 6 (Controlled invariant set). A set , with , is controlled invariant if
The maximal controlled invariant subset of is denoted .
Dually, let denote the losing set, i.e., the states from which safety cannot be maintained indefinitely using only actions from . If a controlled invariant set is known, it yields a one-step condition for repair: the repaired action must keep the current transition safe and map all successors into the invariant.
When is unavailable, the one-step repair condition above cannot be applied directly. ReachECO instead starts from a sound under-approximation , which may initially contain only the unsafe set .
Hence, we define the SAFESTEP predicate in eq. 4 as the verifiers analogue to the ideal one-step condition above. Since may be a strict under-approximation of , however, satisfying SafeStep may still lead to a safety violation in later time steps.
For later use, let us now additionally define the predicate . For a box contained in policy cell and a fixed controller , and, for , define
Thus checks the continuation under itself. To make the dependence on the losing set in SafeStep explicit, we write evaluated with in SafeStep.
Algorithm Details
Forward Verification. ReachECO uses our implementation of BBReach for forward reachability analysis [72], to propagate the current repaired controller in REACHSTEP. At time step , the initial failing frontier, the set of obligations, is
A nonempty frontier triggers BREPAIR. After a repair or an expansion of , verification resumes from the earliest affected time step. If a reachable-set fixed point is reached before the horizon budget , we obtained an unbounded safety certificate. Furthermore, intersections with the absorbing goal are removed from further reachability steps.
Notes on Merging. Merging boxes that belong to the same policy cell is essential for the scalability of BBReach-based reachability analysis [72]. Without merging, every propagated box has a unique parent in the preceding timestep. Merging destroys this property: a merged box may contain successors originating from several distinct parent boxes.
To retain the efficiency of merging while recovering an exact lineage when a safety violation is encountered, ReachECO records the candidate parents contributing to each merged box. Upon detecting a failing merged box, it replays only these candidate parents from the preceding timestep, with merging disabled. Thus, this refinement is performed lazily and only along the portion of the reachability computation relevant to the safety violation. If no safety violation remains after replay, the original violation was induced by merging. The corresponding obligation is removed from the failing frontier, and a merge mask is updated with the timestep and policy-cell identifier at which merging must be disabled in subsequent forward propagations.
Otherwise, replay yields one or more genuinely failing unmerged boxes, each with a unique parent. Consequently, BACKTRACK is only applied to unmerged boxes: for an obligation , if is the unique parent of in the preceding timestep, BACKTRACK replaces by . Repeating this operation therefore recovers a unique predecessor lineage despite the use of merging during the original forward propagation.
Repair Constraints. Recall a repair constraint , with root timestep and game DAG . Let denote the set of actions stored at node , which allow a safe continuation. For a controller , the constraint is satisfied recursively as follows: at a node , the action , and, if , the same condition must hold for every child node corresponding to a box . We write
if this condition holds from the root of .
For a set of constraints , we write
Growing does not reinterpret existing games. Additional exclusions are performed by forward reverification and newly generated constraints.
Losing-Set Expansion A box may be certified losing under the continuation alphabet if:
Then and it may be added to . If we find a losing box, our implementation greedily attempts to certify the whole policy cell, that contains the certified losing box, as losing.
Joint Realization. JOINT searches for a controller such that . denotes the committed constraints from prior repair rounds, while represents the current constraints that were generated under the current . As all constraints are derived independently they need not be jointly realizable, since each policy cell must be assigned a single action. We encode their joint realization as a CP-SAT problem using OR-Tools [53]. Binary variables select one action per cell, and indicates activation of a game node :
where denotes the root node of the game DAG of .
Every active node must select a stored safe action, and selecting an action activates all corresponding child nodes: for every node of a game in ,
y_v \land w_{q(B_v),a} \Longrightarrow y_{v^{\prime}}\quad \text{for every }a \in \mathcal{A}_v\text{ and every child }v^{\prime}\text{ of }v\text{ induced by }a.
Joint-Realization Heuristics. Optimal joint realization can be expensive, so JOINT employs several heuristics. For a set of repair constraints , define its support as the set of policy cells occurring in any of its game DAGs,
We call the current constraint cells.
JOINT first restricts the realization search to the current constraint cells, such that policy cells outside are not modified. If no feasible joint realization is found, the search is repeated over
allowing cells occurring in both committed and current constraints to be modified.
Among feasible assignments, JOINT searches for one minimizing the distance to the original controller. We limit this optimization to 60 s and, if optimality has not been established by then, accept the best feasible assignment found.
Additional implementation details. The original BBReach implementation uses Ariadne [9] for post-image propagation. We instead implement propagation directly on the GPU using interval arithmetic for and sound continuous-time bounds for . For efficiency, boxes in the same policy cell are merged by their bounding box. This is a more aggressive merging scheme than employed in [72]. Benchmark-specific propagators are given in Appendix E. We perform each tube check for 10 segments, escalating to a 100 for any boxes where violations were found. Only after this refinement do we accept a tube as failing. Furthermore, after a certified repair is found, the repair is localized to only cells visited by the repaired controller. This information can be simply extracted from the stored forward propagation sets. Last, floating-point operations are soundly overapproximated or checked for exactness throughout the implementation.
Appendix: Theorems
Verifier assumptions. We build on the sound reachability procedure of Tian et al. [72]. Our proofs rely on the following assumptions:
ReachStep is covering: for every accepted step and every , , where Post and Tube are sound.
ReachStep reports every source box whose abstract tube intersects or whose abstract post-image intersects . Hence,
EarliestAffected returns a rewind time step no later than any previously verified step affected by a controller modification, an expansion of , or a refinement of the reachability computation with respect to merging; the subsequent sets are recomputed.
Every obligation in the backward cone (the initial obligations of Eq. (9) and every obligation produced from them by Backtrack) satisfies , where is the controller that generated the failing frontier.
FindCont evaluates candidate actions with the same SafeStep predicate and Succ relation as the forward verifier: every action stored at a game node satisfies , and, if , all boxes in appear as children of with horizon .
All calls to ReachStep, Joint, FindCont, EarliestAffected, Backtrack terminate. In particular Joint either returns UNKNOWN, SAT with a valid repair, or Backtrack, with .
Assume further that each abstract successor and predecessor computation returns finitely many boxes and that the continuation backend terminates.
For reach–avoid tasks we treat the goal set as terminal: a trajectory that enters (satisfying the safety condition at entry) has completed the task and is not modeled further. Certified safety then means: every closed-loop trajectory from avoids until it enters . Non-entering trajectories avoid till .
For simplicity, we state the guarantees with respect to the unmerged reachability computation. Merging is used only as a scalability optimization in the implementation. Whenever a merged box contributes to a failing or unresolved frontier, Backtrack replays the corresponding reachability step without merging and future forward steps retain this refinement. Thus all repair constraints are derived from the unmerged predecessor relation.
Theorem 1 (Soundness). Suppose ReachECO returns with rather than UNKNOWN. Then is certified safe to horizon (Definition 1). In particular, no trajectory of the closed-loop system under starting in enters during the first control steps, and for all time if .
Proof. By assumption 1, every accepted reachability step soundly covers the sampled nonterminal successors, and its inter-sample tubes are disjoint from by assumption 2. Whenever , , or the reachability computation changes, EarliestAffected causes every potentially invalidated step to be recomputed (assumption 3). Thus, at termination, all steps in the accepted prefix are valid for the returned controller .
If the algorithm returns , every step has been accepted. Hence,
so is certified safe to horizon by Definition 1.
If the algorithm instead returns at step , every step has been accepted, and the fixed-point test establishes
Together with
this establishes certified safety to horizon by Definition 1.
By assumption 1, the computed sets and tubes cover the concrete trajectories throughout the accepted prefix. In the fixed-point case, assumption 1 and the inclusion above allow the accepted transition from to be applied inductively. Its tube checks therefore establish safety for all subsequent steps (assumption 2).
For reach-avoid tasks, this argument applies to trajectories that have not terminated: every successor outside remains covered by the propagated sets. Trajectories removed upon safe goal entry have completed the task under assumption 8. Thus concrete trajectories avoid throughout the certified horizon, subject to this terminal-goal convention.
Lemma 1 (Failure constraints eliminate the current controller). Let be the controller whose forward propagation generated a failure, and let be a repair constraint generated from a backward obligation , where has horizon . Then
Consequently, for every such that ,
where
Proof. Throughout this proof, is evaluated with the losing set used when the obligation was established. Since the losing set only grows, every subsequent continuation-game check also enforces this predicate. lies in the backward cone of a failure of (assumption 4),
By soundness of the continuation-game constraint (assumption 5),
Hence
If additionally , then
and therefore
Lemma 2 (Termination of backward repair). Assume a finite verification horizon , finitely branching predecessor expansion, a terminating continuation backend , and finite committed constraint sets. Then every call to BREPAIR terminates.
Proof. Persistent merge refinement and its associated replay introduce only finitely many additional steps; it therefore suffices to consider a suffix without such steps. Consider an arbitrary call to BREPAIR caused by a failure at time step . Suppose, for contradiction, that the call has an infinite execution trace.
Let be the backward unfolding of the failing frontier. Its nodes are occurrences of box-horizon pairs and
The initial frontier is finite, every node has finite out-degree (assumption 7), and
Hence
Consider an arbitrary iteration of the assumed infinite trace. Since the trace does not terminate, neither BREPAIR nor JOINT returns REPAIR or UNKNOWN (assumption 6).
Thus the iteration can continue only by
returning one or more current constraints for further backtracking, or
removing one or more previously committed non-anchored constraints in favor of an anchored current constraint.
In Case 1, each returned pair is replaced by predecessors . Hence the corresponding occurrence advances strictly in the finite tree . A current constraint that is returned for backtracking is removed from and therefore cannot be backtracked again at the same occurrence. Thus Case 1 can occur only finitely many times.
In Case 2, at least one constraint from the committed set present at entry to the call is removed. Since this set is finite, Case 2 can also occur only finitely many times.
Hence the assumed infinite trace cannot exist. Therefore every call to BREPAIR terminates.
Theorem 2 (Termination). Assume that the initial set has a finite abstract representation, and that the policy partition , the continuation alphabet , and the verification horizon are finite. Furthermore, assume JOINT always returns a repair if there exists one under the constraints. Consider monotone-losing set expansion and persistent merge-mask refinement as described in Appendix C. Then ReachECO terminates and returns either , with , or UNKNOWN.
Proof. By Lemma 2, every call to BREPAIR terminates. It remains to show that only finitely many such calls can occur.
Since and are finite,
We apply the following argument between losing-set updates, merge refinements, and their associated replay steps, starting after the first successful realization in such an interval.
Let be the committed constraint set immediately before the -th call to BREPAIR, let
and let be the current controller. By construction of JOINT (assumption 6, trivial for ),
Consider an arbitrary successful call in this sequence.
There are two cases.
Case 1: no committed constraint is removed. Then
At least one constraint generated from the current failure is newly committed. By Lemma 1,
Together with 15,
and therefore
Case 2: a committed constraint is removed. By construction, this is permitted only in favor of a newly generated anchored constraint ; anchored constraints are never discarded. Hence
Again, by Lemma 1,
Since , equation 15 gives , so
and therefore
By equation (16) in Case 2, and by equation (15) together with in Case 1, every successful call strictly decreases
in lexicographic order. Since
and , only finitely many calls to BREPAIR can occur while is fixed. Thus only finitely many successful repair calls can occur in such an interval.
The merge mask has finitely many timestep–cell entries and grows monotonically. Moreover, fixed, finitely branching propagation from the finite initial representation, over the finite action alphabet and finite horizon, generates only finitely many candidate losing boxes. Hence the losing set can also grow strictly only finitely many times.
Each refinement entails only finitely many replay steps. Consequently, there are only finitely many such intervals and repair calls. Between repairs and updates, forward verification advances over the finite horizon. Hence ReachECO terminates.
Theorem 3 (Completeness under exhaustive search). Under the assumptions of Theorem 2, let be the fixed initial box representation, containing every anchored root. Let denote the final losing set constructed during execution. Use the exhaustive backend over the original action alphabet . If some satisfies
then ReachECO returns a controller that is certified safe (Definition 1).
Proof. Since , we have . By monotone losing-set expansion, throughout execution. Therefore
Thus the witness passes every safety test used during construction of the exhaustive unmerged games.
Let be the exhaustive continuation game. For , equation (8) and induction on give
the action passes SafeStep, every successor admits its continuation under , and, since the exhaustive backend explores every action of , the game retains this action and all its successor branches at every node (assumption 5).
Suppose, for contradiction, that the algorithm returns UNKNOWN.
Every anchored constraint has and remaining horizon (obligations never exceed horizon , cf. line 15 of Algorithm 2). Due to the replay induced by merging the anchored constraint is entirely unmerged. Since implies for , the witness satisfies and hence . This holds for both committed and current constraints, so
Thus no obligation at time step zero lacks a winning continuation, and no anchored subset is jointly infeasible. Hence the algorithm cannot return UNKNOWN.
By Theorem 2 it terminates and returns some with ; by Theorem 1, is certified safe to horizon , in particular safe for the first control steps.
[Completeness relative to the continuation backend] Under the assumptions of Theorem 2, suppose there exists a stationary controller satisfying
Furthermore, suppose that for every box encountered in its unmerged continuation with remaining horizon , the backend explores . Then ReachECO returns a certified safe controller.
Proof. The Proof of Theorem 3 applies unchanged, since the backend explores the witness action at every relevant node.
Appendix: Environments
Linear ACC
System Dynamics. Our implementation follows {[70]}[70]. Let denote the distance from the controlled vehicle to the lead vehicle and their relative velocity, with indicating that the vehicles are approaching each other. Under a relative acceleration held constant over one sampling interval, the dynamics are
We use a sampling period of s and relative acceleration bounds m/s. The normalized controller action determines m/s; positive actions correspond to ego-vehicle braking.
We consider collision avoidance, requiring at all times, including between samples. For this specification, the analytic braking controlled-invariant region is
The condition ensures that the available distance exceeds the stopping distance under maximal braking when the vehicles are approaching each other.
Tube over-approximation. Under a constant relative acceleration , the continuous-time trajectory from an initial state is
For an initial box , position is monotone in both initial coordinates for . Consequently, its lower and upper bounds over a time segment are
Each extremum is obtained by evaluating the corresponding quadratic at the segment endpoints and, when , at its stationary point if that point lies within the segment. Velocity bounds follow from the endpoint extrema of and . Together, these bounds form an axis-aligned enclosure of every trajectory starting in throughout the segment.
Reinforcement Learning. Again following {[70]}[70], all controllers are trained with PPO {[61]}[61] for a total of 200,000 time steps. In contrast to {[70]}[70], we train with cleanrl {[34]}[34] instead of stable-baselines-3 {[56]}[56]. Our five seeds results are reported in Table 5. We utilize the hyperparameters given in Table 4.
| Hyperparameter | Value | |
| PPO | Total time steps | 200000 |
| PPO | Learning rate | (annealed) |
| PPO | Parallel environments | 1 |
| PPO | Rollout steps | 2048 |
| PPO | Batch | 2048 |
| PPO | Update epochs | 10 |
| PPO | Discount | 0.99 |
| PPO | GAE | 0.95 |
| PPO | Clipping coefficient | 0.2 |
| PPO | Clipped value loss | true |
| PPO | Advantage normalization | true |
| PPO | Entropy / value-loss coefficient | 0.0 / 0.5 |
| PPO | Max gradient norm, target KL | 0.5, none |
| DWC | Thermometer bits per dimension | 81 |
| DWC | Thermometer type | uniform |
| DWC | Grid domain | m, m/s |
| DWC | LUT layers width | |
| DWC | LUT fan-in | 6 |
| DWC | Initial log | |
| DWC | Final squashing | off |
| DWC | Critic layers width, act. | , ReLU |
Table 4. ACC: training hyperparameters.
For the ReLU baseline, we use the 4-layer MLP from {[70]}[70] with 64 units per layer, and for the DWC baseline we use a 2-layer DWC with 128 LUT-6 per layer. The per-step reward function is defined as , encouraging small action changes. In addition, a large penalty of is applied for collisions. When the distance between the cars grows larger than , a penalty of is given ending the episode.
Baselines
In Table 2, Change [%] denotes the fraction of states for which the repaired controller output differs from the original controller output by more than . For ReLU networks, this quantity is estimated using 100,000 Monte Carlo samples. Mean denotes the average magnitude of the controller-output change over all sampled states.
BBReach + Supervised This baseline adapts the repair procedure of Yang et al. [79]. We first perform multi-step closed-loop reachability analysis (Yang et al. [79] perform only one-step analysis) using our BBReach implementation. Whenever verification reaches the unsafe set, we collect the failing one-step transition and add the predecessor states to a supervised repair dataset, assigning a target action of 1.05, corresponding to full braking. To mitigate degradation of the original policy, we additionally collect 100,000 safe transitions generated by the unrepaired DWC and retain their original actions as supervised targets.
We then retrain the controller for 1000 gradient steps with batch size 256, sampling each minibatch equally from the repair and preservation datasets. After retraining, we rerun closed-loop reachability analysis and repeat this procedure until the controller verifies successfully for a horizon of 50 time steps. Finally, we run the ReachECO verifier requiring that no further repair constraints are generated. We report the total runtime of this complete repair-and-reverification loop.
The invariant is required as only one-step repair can be performed with this technique.
STLGym fine-tuning. We adapt the dense robustness reward of Hamilton et al. [30] to the ACC specification
evaluated at the sampled states of each episode. Starting from the same pretrained NCubeV ReLU PPO controller, we perform five independent fine-tuning runs with a budget of 200,000 environment steps each. At step , PPO receives
where . To expose the controller to difficult initial states, we draw 10% of training resets slightly inside the analytic braking boundary. We evaluate each run at ten checkpoints during training. Every checkpoint has sampled trajectories from near-boundary starts that reach . We therefore do not attempt a formal safety certificate.
The 5 seed runs are shown in Figure 6.
NCubeV + REASSURE This baseline uses the NCubeV framework introduced by Teuber et al. [70] together with their four-layer ReLU MLP. NCubeV utilizes a provably safe control envelope derived using differential dynamic logic (Platzer, [55]) to reduce infinite-horizon closed-loop safety verification to neural-network verification against the envelope. REASSURE (Fu & Li, [25]) repairs ReLU networks by constructing localized patch networks around identified buggy regions.
We combine the two approaches as follows. NCubeV identifies regions in which the controller violates the safe control envelope, and we provide these regions to REASSURE together with the desired controller outputs. REASSURE then constructs local patches enforcing the desired behavior. For ACC, full braking provides such a safe target action for states inside the controllable region.
NCubeV + Supervised This baseline is similar to BBReach + Supervised; NCubeV identifies unsafe regions from which the supervised repair dataset is generated (each sample having its correct output set to 1.05). Additionally, an equal number of safe transitions (100,000) is produced by the unrepaired neural network. With this data, the neural network is retrained until a loss threshold is reached. This procedure is repeated until NCubeV can not provide any unsafe regions and verifies the safety of the controller.
Extended Experimental Evaluation
Figure 5 shows the depth of the games played with . Learning the losing set decreases the number of deeper games. Table 5 shows performance of the 5 DWC seeds before and after invariant repair, as well as the ReLU baseline from Teuber et al. [70].
After repair, all seeds are certified and hence, there are no remaining unsafe empirical trajectories.
Docking2D
System Dynamics. Planar spacecraft docking is modeled using the linear Clohessy–Wiltshire-Hill equations [19]. Let denote the relative position and the relative velocity of the spacecraft. Under a thrust held constant over one sampling interval, the dynamics can be written as
with state and
As in prior work, we use a sampling period of s, spacecraft mass kg, and orbital mean motion s. The normalized controller action determines the thrust components in newtons. These dynamics and parameters follow [45, 57, 51].
Specification We follow the AINNCS competition benchmark [45], which requires
at all times, including between controller samples. We take as initial set
which is larger than in [45].
The terminal goal is specified following [51]:
We remove any partial-box entering the goal set that satisfies the velocity bound from further reachability propagation, considering it successfully docked.
Tube over-approximation. To account for the continuous-time evolution between controller samples, we augment the state with the constant control input,
Since , the augmented dynamics are linear,
Hence, for a constant action over one sampling interval, the trajectory is given exactly by
Thus, continuous-time propagation only requires evaluating the matrix exponential of the constant matrix .
For an initial state box , the augmented initial set is
At any fixed time , let
The exact axis-aligned enclosure of the propagated box is
where and are taken elementwise. Evaluating these bounds at and retaining the first four coordinates gives the next-state box. To conservatively enclose the continuous-time reachable tube between controller samples, we partition into time segments, following standard continuous-time over-approximation constructions systems [3]. For a segment , define
Let denote the reachable box at the segment midpoint. For any , the deviation from the midpoint state can be conservatively bounded componentwise by
Hence, the reachable tube over the entire segment is contained in
The first four coordinates therefore give an axis-aligned over-approximation of all states reached from during the segment.
Reinforcement Learning. We utilize the environment and reward functions of Ravaioli et al. [57], and train three seeds for two million environment steps each. Our initial state lies on the circle of radius 130 m around the dock, and evaluation is run from this same set. Table 6 summarizes the hyperparameters used for training the DWC controller with PPO.
Car2D
System dynamics. The cars state is comprised of , planar position, heading, and speed. The controller outputs an acceleration and turn command . Each command is held constant for s and scaled to physical acceleration and turn rate . The continuous dynamics are the same as the NAV benchmark in AINNCS [45].
Hence, and , while
| Hyperparameter | Value |
| Total time steps | 2000000 |
| Learning rate | (constant) |
| Parallel environments | 4 |
| Rollout steps per environment | 1000 |
| Batch / minibatch | 4000 |
| Update epochs | 30 |
| Discount | 0.968559 |
| GAE | 0.928544 |
| Clipping coefficient | 0.3 |
| Clipped value loss | true |
| Advantage normalization | true |
| Entropy / value-loss coefficient | 0.0 / 1.0 |
| Max gradient norm, target KL | 0.5, 0.01 |
| Thermometer bits per dimension | 161 |
| Thermometer type | Gaussian , uniform |
| Grid domain | m, m/s |
| LUT layers width | |
| LUT fan-in | 6 |
| Initial log | |
| Final tanh squashing | on |
| Critic layers width, act. | , tanh |
Table 6. Docking2D: training hyperparameters.
Specification. The unsafe set consists of contact with any obstacle or violation of . The two rectangular obstacles, expressed as Cartesian products of and intervals, are
For the 1 m 1 m initial-box experiment, we verify
The goal set is . The verification budget was set to controller steps. After a safe transition, any portion of a post-image box contained in is removed from further propagation.
Tube over-approximation. For a fixed controller action, the closed-form flow described above is used to compute an interval enclosure of the state at any fixed time . Evaluating this enclosure at gives the post-image box. To conservatively enclose the continuous-time trajectory between controller samples, we partition into time segments . For each segment, we first evaluate the closed-form flow at , obtaining a box
Since
we have
Let
Because is affine in both and , this maximum is attained at one of the four endpoint combinations. Writing , every trajectory over the segment therefore satisfies
Heading and speed are affine in time,
so their bounds over are obtained directly from their endpoint extrema. The Cartesian product of these four intervals forms a sound axis-aligned over-approximation of the reachable tube over the segment.
Controller training. The DWC controller was trained with TD3 [26] for 120,000 environment steps. The training reward is progress towards the environment’s goal box, minus a per-step and action penalty, with an additional reward for reaching the goal and a penalty for unsafe termination. The trained policy uses a policy-grid.
Circuit Size
We use the DWC hardware synthesis pipeline of Kresse & Lampert (2026b) and extend it with our ECO patch. We first briefly review the circuit of a DWC, then describe the patch circuit and report hardware numbers.
Differentiable Weightless Controller circuit
A DWC consists of four stages [41], with the first three mirroring Differentiable Weightless Networks [6] and Logic Gate Neural Networks [54]:
Thresholding. Each observation, given as a 12-bit fixed-point word, is compared against the learned thresholds of its dimension. The comparator outputs form the thermometer encoding: bit is set if and only if the observation exceeds threshold .
LUT layers. A stack of layers (two throughout this work), each a set of 6-input lookup tables. The inputs of every LUT are wired to fixed outputs of the previous layer, so each LUT maps directly onto one LUT6 primitive of the FPGA.
Popcount. Per action dimension, the outputs of the final layer assigned to it are summed by an adder tree.
Output lookup. A final table maps each popcount to an action value.
We omit stage (4): for typical DWCs it requires one BRAM per action dimension, and the patch affects it only marginally (it may require a larger memory, which is very unlikely to exceed the capacity of a single BRAM).
Throughout, we set the number of pipeline stages of the DWC controller to eight cycles (nine including the final BRAM). The patch contains a matching number of pipeline stages, so it adds no latency. Fewer pipeline stages are possible; we did not sweep this parameter. All synthesis results were obtained with Vivado 2022.2, with implementation on a Xilinx Artix-7 xc7a100t (speed grade ) targeting a clock frequency of 100 MHz. All reported designs meet this target.
Table 7 reports the synthesized sizes of the investigated DWCs.
Patch Circuit
As described in § 2.3, a repair modifies a set of policy cells , with each modified cell receiving an integer action correction . The patch adds this correction to the controller’s popcount whenever the input lies in a repaired cell, and leaves the output unchanged otherwise. We write and number the repaired actions . Each is a product of one interval per input dimension, , where is its interval index along dimension .
Guards. Let input dimension have thresholds , inducing intervals, and thermometer bits
| Controller | Patch | Cells | Controller LUT | Controller FF | Patch Logic LUT | Patch Logic FF | Patch Memory LUT |
| ACC | |||||||
| ACC | |||||||
| Car2D | 39 | 1,969 | 1,708 | 64 | 62 | 12 | |
| Docking2D | seed 1 | 5 | 6,860 | 3,873 | 36 | 43 | 14 |
| Docking2D | seed 2 | 0 | 6,894 | 3,889 | no patch | ||
| Docking2D | seed 3 | 3,309 | 6,878 | 3,866 | 5,249 | 3,362 | 1,166 |
Table 7. Synthesized hardware at for ACC (mean and standard deviation over five seeds) and for the Car2D and Docking2D controllers. Cells is the number of repaired policy cells. The controller is the DWC without its patch (thresholding, LUT layers and popcounts with their pipeline registers). Vivado 2022.2, Artix-7 xc7a100t (); all designs meet 100 MHz with and without their patch.
so that interval of dimension is . Let denote the index of the interval containing . Then
where the first condition is omitted for and the second for . Hence, membership in one interval is a function of at most two thermometer bits that the controller already computes. Each distinct interval is implemented once and shared between repaired cells, where they overlap. The guard of is the conjunction of its interval predicates , and it fires exactly for inputs in . Since policy cells are disjoint, at most one guard fires.
Address encoder. The correction is stored at address . Address zero holds the zero correction and is selected when no guard fires. Since the guards are one-hot, bit of the -bit address is the disjunction of the guards whose address has bit set.
Memory. The correction memory has words. Per action dimension , it stores in two’s complement with the smallest width that represents 0 and every . We synthesize the memory as a separate block and report its size separately.
Adders. For each action dimension , the stored correction is sign-extended and added to the controller’s popcount. Let denote the maximum popcount of action dimension ; the adder has the smallest two’s-complement width that holds every sum, i.e.
Pipelining. The patch is a separate block next to the controller. Its guards, address and memory word are registered and the word is delayed, so the correction reaches the adder in the same cycle as the popcount of the same input. The patched controller thus keeps the latency of the original one.
References
- [1]David Abel. A theory of state abstraction for reinforcement learning. In Conference on Artificial Intelligence (AAAI), 2019.DOI
- [2]Mohammed Alshiekh, Roderick Bloem, Rüdiger Ehlers, Bettina Könighofer, Scott Niekum, and Ufuk Topcu. Safe reinforcement learning via shielding. In Conference on Artificial Intelligence (AAAI), 2018.arxiv.org/abs/1708.08611
- [3]Matthias Althoff, Olaf Stursberg, and Martin Buss. Reachability analysis of linear systems with uncertain parameters and inputs. In IEEE Conference on Decision and Control (CDC), 2007.DOI
- [4]AMD. UltraScale Architecture Libraries Guide. Advanced Micro Devices, Inc., 2026.
- [5]Jean-Pierre Aubin. A survey of viability theory. SIAM Journal on Control and Optimization, 1990.DOI
- [6]Alan T. L. Bacellar, Zachary Susskind, Mauricio Breternitz Jr., Eugene John, Lizy K. John, Priscila M. V. Lima, and Felipe M. G. França. Differentiable weightless neural networks. In International Conference on Machine Learning (ICML), 2024.arxiv.org/abs/2410.11112
- [7]Osbert Bastani. Safe reinforcement learning with nonlinear dynamics via model predictive shielding. In American control conference (ACC), 2021.arxiv.org/abs/1905.10691
- [8]Fabian Bauer-Marquart, David Boetius, Stefan Leue, and Christian Schilling. Specrepair: Counter-example guided safety repair of deep neural networks. In International Symposium on Model Checking Software (SPIN), 2022.
- [9]Luca Benvenuti, Davide Bresolin, Alberto Casagrande, Pieter Collins, Alberto Ferrari, Emanuele Mazzi, Alberto Sangiovanni-Vincentelli, and Tiziano Villa. Reachability computation for hybrid systems with Ariadne. IFAC Proceedings Volumes, 2008.
- [10]Dimitri Bertsekas. Infinite time reachability of state-space regions by using feedback control. IEEE Transactions on Automatic Control (TAC), 1972.DOI
- [11]Franco Blanchini. Set invariance in control. Automatica, 1999.DOI
- [12]Roderick Bloem, Bettina Könighofer, Robert Könighofer, and Chao Wang. Shield synthesis: Run-time enforcement for reactive systems. In International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS), pp. 533–548. Springer, 2015.
- [13]David Boetius, Stefan Leue, and Tobias Sutter. A robust optimisation perspective on counterexample-guided repair of neural networks. In International Conference on Machine Learning (ICML), 2023.arxiv.org/abs/2301.11342
- [14]Lukas Brunke, Melissa Greeff, Adam W. Hall, Zhaocong Yuan, Siqi Zhou, Jacopo Panerati, and Angela P. Schoellig. Safe learning in robotics: From learning-based control to safe reinforcement learning. Annual Review of Control, Robotics, and Autonomous Systems, 2022.arxiv.org/abs/2108.06266
- [15]Tiziana Calamoneri, Pierre Gaillard, Giacomo Paesani, and Giuseppe Perelli. Strategy repair in reachability games via a graph quotientation. In Scandinavian Symposium on Algorithm Theory (SWAT), 2026.DOI
- [16]Hugo CC Carneiro, Felipe MG França, and Priscila MV Lima. Multilingual part-of-speech tagging with weightless neural networks. Neural Networks, 2015.DOI
- [17]Ya-Chien Chang, Nima Roohi, and Sicun Gao. Neural Lyapunov control. Conference on Neural Information Processing Systems (NeurIPS), 2019.arxiv.org/abs/2005.00611
- [18]Puja Chaudhury, Alexander Estornell, and Michael Everett. Learning verifiable control policies using relaxed verification. In IEEE Conference on Decision and Control (CDC), 2025.arxiv.org/abs/2504.16879
- [19]William Hilllick Clohessy and R. S. Wiltshire. Terminal guidance system for satellite rendezvous. Journal of the Aerospace Sciences, 1960.DOI
- [20]Davide Corsi, Guy Amir, Andoni Rodríguez, César Sánchez, Guy Katz, and Roy Fox. Verification-guided shielding for deep reinforcement learning. In Reinforcement Learning Conference (RLC), 2024.arxiv.org/abs/2406.06507
- [21]Luca De Alfaro, Thomas A. Henzinger, and Orna Kupferman. Concurrent reachability games. Theoretical Computer Science (TCS), 2007.
- [22]Souradeep Dutta, Xin Chen, and Sriram Sankaranarayanan. Reachability analysis for neural feedback systems using regressive polynomial rule inference. In International Conference on Hybrid Systems: Computation and Control (HSCC), 2019.DOI
- [23]Alec Edwards, Andrea Peruffo, and Alessandro Abate. Fossil 2.0: Formal certificate synthesis for the verification and control of dynamical models. In International Conference on Hybrid Systems: Computation and Control (HSCC), 2024.arxiv.org/abs/2311.09793
- [24]Georgios E Fainekos and George J. Pappas. Robustness of temporal logic specifications for continuous-time signals. Theoretical Computer Science (TCS), 2009.DOI
- [25]Feisi Fu and Wenchao Li. Sound and complete neural network repair with minimality and locality guarantees. In International Conference on Learning Representations (ICLR), 2022.arxiv.org/abs/2110.07682
- [26]Scott Fujimoto, Herke van Hoof, and David Meger. Addressing function approximation error in actor-critic methods. In International Conference on Machine Learning (ICML), 2018.
- [27]Nathan Fulton and André Platzer. Safe reinforcement learning via formal methods: Toward safe control through proof and learning. In Conference on Artificial Intelligence (AAAI), 2018.DOI
- [28]Pierre Gaillard, Fabio Patrizi, and Giuseppe Perelli. Strategy repair in reachability games. In European Conference on Artificial Intelligence (ECAI), 2023.
- [29]Ben Goldberger, Guy Katz, Yossi Adi, and Joseph Keshet. Minimal modifications of deep neural networks using verification. In Conference on Logic for Programming, Artificial Intelligence and Reasoning (LPAR), 2020.DOI
- [30]Nathaniel Hamilton, Preston K. Robinette, and Taylor T. Johnson. Training agents to satisfy timed and untimed signal temporal logic specifications with reinforcement learning. In International Conference on Software Engineering and Formal Methods (SEFM), 2022.DOI
- [31]Thomas A. Henzinger, Peter W. Kopke, Anuj Puri, and Pravin Varaiya. What’s decidable about hybrid automata? In ACM Symposium on Theory of Computing (STOC), 1995.
- [32]Haimin Hu, Mahyar Fazlyab, Manfred Morari, and George J. Pappas. Reach-SDP: Reachability analysis of closed-loop systems with neural network controllers via semidefinite programming. In IEEE Conference on Decision and Control (CDC), 2020.arxiv.org/abs/2004.07876
- [33]Chao Huang, Jiameng Fan, Xin Chen, Wenchao Li, and Qi Zhu. POLAR: A polynomial arithmetic framework for verifying neural-network controlled systems. In International Symposium on Automated Technology for Verification and Analysis (ATVA), 2022a.arxiv.org/abs/2106.13867
- [34]Shengyi Huang, Rousslan Fernand Julien Dossa, Chang Ye, Jeff Braga, Dipam Chakraborty, Kinal Mehta, and João G. M. Araújo. CleanRL: High-quality single-file implementations of deep reinforcement learning algorithms. Journal of Machine Learning Research (JMLR), 2022b.arxiv.org/abs/2111.08819
- [35]Radoslav Ivanov, James Weimer, Rajeev Alur, George J. Pappas, and Insup Lee. Verisig: verifying safety properties of hybrid systems with neural network controllers. In International Conference on Hybrid Systems: Computation and Control (HSCC), 2019.
- [36]Radoslav Ivanov, Taylor Carpenter, James Weimer, Rajeev Alur, George Pappas, and Insup Lee. Verisig 2.0: Verification of neural network controllers using taylor model preconditioning. In International Conference on Computer Aided Verification (CAV), 2021.DOI
- [37]Peng Jin, Jiaxu Tian, Dapeng Zhi, Xuejun Wen, and Min Zhang. Trainify: A CEGAR-driven training and verification framework for safe deep reinforcement learning. In International Conference on Computer Aided Verification (CAV), 2022.
- [39]Hanna Krasowski, Jakob Thumm, Marlon Müller, Lukas Schäfer, Xiao Wang, and Matthias Althoff. Provably safe reinforcement learning: Conceptual analysis, survey, and benchmarking. Transactions on Machine Learning Research (TMLR), 2023.arxiv.org/abs/2205.06750
- [40]Fabian Kresse and Christoph H. Lampert. Learning quantized continuous controllers for integer hardware. In Learning for Dynamics and Control Conference (L4DC), 2026a.arxiv.org/abs/2511.07046
- [41]Fabian Kresse and Christoph H. Lampert. Differentiable weightless controllers: Learning logic circuits for continuous control. International Conference on Machine Learning (ICML), 2026b.arxiv.org/abs/2512.01467
- [42]Smita Krishnaswamy, Haoxing Ren, Nilesh Modi, and Ruchir Puri. DeltaSyn: An efficient logic difference optimizer for ECO synthesis. In Proceedings of the 2009 International Conference on Computer-Aided Design (ICCAD), 2009.DOI
- [43]Chih-chang Lin, Kuang-Chien Chen, Shih-Chieh Chang, Malgorzata Marek-Sadowska, and Kwang-Ting Cheng. Logic synthesis for engineering change. In ACM/IEEE Design Automation Conference (DAC), 1995.
- [44]Diego Manzanas Lopez, Sung Woo Choi, Hoang-Dung Tran, and Taylor T. Johnson. NNV 2.0: The neural network verification tool. In International Conference on Computer Aided Verification (CAV), 2023.DOI
- [45]Diego Manzanas Lopez, Matthias Althoff, Luis Benet, Samuel Coogan, Marcelo Forets, Akash Harapanahalli, Taylor T. Johnson, Tobias Ladner, Christian Schilling, Huan Zhang, and Xiangru Zhong. ARCH-COMP25 category report: artificial intelligence and neural network control systems (AINNCS) for continuous and hybrid systems plants. In International Workshop on Applied Verification of Continuous and Hybrid Systems (ARCH), 2025.DOI
- [46]Pengyuan Lu, Ivan Ruchkin, Matthew Cleaveland, Oleg Sokolsky, and Insup Lee. Causal repair of learning-enabled cyber-physical systems. In IEEE International Conference on Assured Autonomy (ICAA), 2023.arxiv.org/abs/2304.02813
- [47]Pengyuan Lu, Matthew Cleaveland, Oleg Sokolskiy, Insup Lee, and Ivan Ruchkin. Repairing neural network-based control policies with safety preservation. Research Directions: Cyber-Physical Systems, 2025.DOI
- [48]Deyun Lyu, Jiayang Song, Zhenya Zhang, Zhijie Wang, Tianyi Zhang, Lei Ma, and Jianjun Zhao. Autorepair: Automated repair for AI-enabled cyber-physical systems under safety-critical conditions. arXiv preprint arXiv:2304.05617, 2023.arxiv.org/abs/2304.05617
- [49]Benedikt Maderbacher, Stefan Schupp, Ezio Bartocci, Roderick Bloem, Dejan Ničković, and Bettina Könighofer. An adaptive, provable correct simplex architecture. International Journal on Software Tools for Technology Transfer, 2025.DOI
- [50]Oded Maler and Dejan Nickovic. Monitoring temporal properties of continuous signals. In International symposium on formal techniques in real-time and fault-tolerant systems (FTRTFT), 2004.DOI
- [51]Udayan Mandal, Guy Amir, Haoze Wu, Ieva Daukantas, Fletcher Lee Newell, Umberto J. Ravaioli, Baoluo Meng, Michael Durling, Milan Ganai, Tobey Shim, Guy Katz, and Clark Barrett. Formally verifying deep reinforcement learning controllers with lyapunov barrier certificates. In Formal Methods in Computer-Aided Design (FMCAD), 2024.arxiv.org/abs/2405.14058
- [52]Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Conference on Neural Information Processing Systems (NeurIPS), 2022.arxiv.org/abs/2203.02155
- [53]Laurent Perron and Frédéric Didier. CP-SAT v9.12, 2025. URL https://developers.google.com/optimization/cp/cp_solver/.
- [54]Felix Petersen, Christian Borgelt, Hilde Kuehne, and Oliver Deussen. Deep differentiable logic gate networks. In Conference on Neural Information Processing Systems (NeurIPS), 2022.arxiv.org/abs/2210.08277
- [55]André Platzer. Differential dynamic logic for hybrid systems. Journal of Automated Reasoning, 2008.DOI
- [56]Antonin Raffin, Ashley Hill, Adam Gleave, Anssi Kanervisto, Maximilian Ernestus, and Noah Dormann. Stable-baselines3: Reliable reinforcement learning implementations. Journal of Machine Learning Research (JMLR), 2021.
- [57]Umberto J Ravaioli, James Cunningham, John McCarroll, Vardaan Gangal, Kyle Dunlap, and Kerianne L. Hobbs. Safe reinforcement learning benchmark environments for aerospace control systems. In IEEE Aerospace Conference (AERO), 2022.
- [58]Timothy Rupprecht and Yanzhi Wang. A survey for deep reinforcement learning in markovian cyber–physical systems: Common problems and solutions. Neural Networks, 2022.
- [59]Ulises Santa Cruz, James Ferlez, and Yasser Shoukry. Safe-by-repair: A convex optimization approach for repairing unsafe two-level lattice neural network controllers. In IEEE Conference on Decision and Control (CDC), 2022.
- [60]Christian Schilling, Marcelo Forets, and Sebastián Guadalupe. Verification of neural-network control systems by integrating Taylor models and zonotopes. In Conference on Artificial Intelligence (AAAI), 2022.
- [61]John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017.arxiv.org/abs/1707.06347
- [62]Keyi Shen and Glen Chou. Parallel differentiable reachability for learning and planning with certified neural dynamics and controllers. In Robotics: Science and Systems (RSS), 2026.
- [63]Chelsea Sidrane, Amir Maleki, Ahmed Irfan, and Mykel J. Kochenderfer. Overt: An algorithm for safety verification of neural network control policies for nonlinear systems. Journal of Machine Learning Research (JMLR), 2022.
- [64]Matthew Sotoudeh and Aditya V. Thakur. Provable repair of deep neural networks. In ACM SIG-PLAN International Conference on Programming Language Design and Implementation (PLDI), 2021.arxiv.org/abs/2104.04413
- [65]Xinyu Sun, Wanwei Liu, Shangwen Wang, Tingyu Chen, Ye Tao, and Xiaoguang Mao. AutoRIC: Automated neural network repairing based on constrained optimization. ACM Transactions on Software Engineering and Methodology (TOSEM), 2025.DOI
- [66]Richard S. Sutton and Andrew G. Barto. Reinforcement learning: An introduction. MIT Press Cambridge, 1998.DOI
- [68]Chen Tang, Ben Abbate Matteo, Jiaheng Hu, Rohan Chandra, Roberto Martín-Martín, and Peter Stone. Deep reinforcement learning for robotics: A survey of real-world successes. Annual Review of Control, Robotics, and Autonomous Systems, 2025.
- [69]Zhe Tao, Stephanie Nawas, Jacqueline Mitchell, and Aditya V. Thakur. Architecture-preserving provable repair of deep neural networks. Proceedings of the ACM on Programming Languages (PACMPL), 2023.DOI
- [70]Samuel Teuber, Stefan Mitsch, and André Platzer. Provably safe neural network controllers via differential dynamic logic. Conference on Neural Information Processing Systems (NeurIPS), 2024.DOI
- [71]Jiaxu Tian, Dapeng Zhi, Si Liu, Peixin Wang, Cheng Chen, and Min Zhang. Boosting verification of deep reinforcement learning via piece-wise linear decision neural networks. Conference on Neural Information Processing Systems (NeurIPS), 2023.
- [72]Jiaxu Tian, Dapeng Zhi, Si Liu, Peixin Wang, Guy Katz, and Min Zhang. Taming reachability analysis of dnn-controlled systems via abstraction-based training. In International Conference on Verification, Model Checking, and Abstract Interpretation (VMCAI), 2024.arxiv.org/abs/2211.11127
- [73]Hoang-Dung Tran, Xiaodong Yang, Diego Manzanas Lopez, Patrick Musau, Luan Viet Nguyen, Weiming Xiang, Stanley Bak, and Taylor T. Johnson. NNV: the neural network verification tool for deep neural networks and learning-enabled cyber-physical systems. In International Conference on Computer Aided Verification (CAV), 2020.DOI
- [74]Oriol Vinyals, Igor Babuschkin, Wojciech Czarnecki, Michaël Mathieu, Andrew Dudzik, Junyoung Chung, David Choi, Richard Powell, Timo Ewalds, Petko Georgiev, Junhyuk Oh, Dan Horgan, Manuel Kroiss, Ivo Danihelka, Aja Huang, Laurent Sifre, Trevor Cai, John Agapiou, Max Jaderberg, and David Silver. Grandmaster level in StarCraft II using multi-agent reinforcement learning. Nature, 2019.
- [75]Panagiotis Vlantis, Leila Bridgeman, and Michael Zavlanos. Failing with grace: Learning neural network controllers that are boundedly unsafe. In Learning for Dynamics and Control Conference (L4DC), 2023.
- [76]Yixuan Wang, Chao Huang, Zhaoran Wang, Zhilu Wang, and Qi Zhu. Verification in the loop: Correct-by-construction control learning with reach-avoid guarantees. arXiv preprint arXiv:2106.03245, 2021.
- [77]Weiming Xiang and Taylor T. Johnson. Reachability analysis and safety verification for neural network control systems. arXiv preprint arXiv:1805.09944, 2018.
- [78]Junfeng Yang, Min Zhang, Xin Chen, and Qin Li. Formal verification of probabilistic deep reinforcement learning policies with abstract training. In International Conference on Verification, Model Checking, and Abstract Interpretation (VMCAI), 2025.
- [79]Xiaodong Yang, Tom Yamaguchi, Hoang-Dung Tran, Bardh Hoxha, Taylor T. Johnson, and Danil Prokhorov. Neural network repair with reachability analysis. In International Conference on Formal Modeling and Analysis of Timed Systems (FORMATS), 2022.arxiv.org/abs/2108.04214
- [80]Emily Yu, Djordje Žikelić, and Thomas A. Henzinger. Neural control and certificate repair via runtime monitoring. In Conference on Artificial Intelligence (AAAI), 2025.arxiv.org/abs/2412.12996
- [81]Hengjun Zhao, Xia Zeng, Taolue Chen, and Zhiming Liu. Synthesizing barrier certificates using neural networks. In International Conference on Hybrid Systems: Computation and Control (HSCC), 2020.DOI
- [82]Weiyе Zhao, Tairan He, Rui Chen, Tianhao Wei, and Changliu Liu. State-wise safe reinforcement learning: A survey. International Joint Conference on Artificial Intelligence (IJCAI), 2023.arxiv.org/abs/2302.03122
- [83]Weichao Zhou, Ruihan Gao, BaekGyu Kim, Eunsuk Kang, and Wenchao Li. Runtime-safety-guided policy repair. In International Conference on Runtime Verification (RV), 2020.arxiv.org/abs/2008.07667

