WorksComputer Science
Learning Regions of Attraction for Transverse Feedback Linearization under Controller Singularities
Abstract
Transverse feedback linearization (TFL) achieves path following by exactly linearizing the error dynamics, but its controller is undefined on a singular set, such as zero speed for a unicycle or zero thrust for a quadrotor. The constrained region of attraction, the set of initial states from which the closed loop converges to the path without violating state or regularity constraints, determines where the controller can be safely activated. Exploiting the exact linear error dynamics, we show that this region is the largest forward invariant subset of the admissible set. We then decompose its state-constrained Zubov value function into an analytically known quadratic cost-to-go and a safety value function. We learn only the latter, by fitted value iteration without infinite-horizon trajectory labels. The resulting estimate combines an inner layer verified by interval branch and bound with a learned outer layer tested on samples. On a unicycle following a circle and a sinusoid and on a planar quadrotor, it covers on average 72.0%, 83.4% and 19.1% of the simulated region of attraction. In comparison, the quadratic cost-to-go and two physics-informed neural network (PINN) solvers of Zubov's equation cover at most 11.3%, 4.0% and 0.0% on average. Code and animations are available at https://gradslab.github.io/roa_tfl
Introduction
Set stabilization and path following are central problems in control and robotics [42, 7, 10]. Transverse feedback linearization (TFL) addresses both with a coordinate transformation and a feedback that inverts a decoupling matrix [43, 44, 4]. The dynamics transverse to the path then become linear and exponentially stable, while the tangential and internal dynamics govern the motion along the path. TFL controllers have been designed for unicycles [42], car-like robots and cars with trailers [4, 7, 4], quadrotors with and without a payload or a failed rotor [47, 5, 6, 8, 9, 11], and robotic arms, also when carried by a quadrotor [50] (Hladik et al. 2013).
In many of these works, the decoupling matrix and the Jacobian of the coordinate transformation are shown to be nonsingular on the path following manifold. By continuity and the inverse function theorem, they remain nonsingular on a neighbourhood of it, so the convergence guarantees are local [44, 7]. Beyond this neighbourhood, the domain of validity of TFL is limited by topological obstructions to globally regular path-following coordinates for closed paths. It is also limited by system-dependent singularities of the decoupling matrix, such as zero speed for the dynamically extended unicycle and zero thrust for the quadrotor ([43, 7, 10]). Consequently, a trajectory that starts far from the path may reach these singularities even though its transverse error decays exponentially while the controller is defined. We therefore seek the constrained region of attraction, that is, the set of initial states from which the closed-loop trajectories converge to the path while the controller remains well defined and all state constraints hold for all future time.
The Zubov value function characterizes regions of attraction ([18]), and with an obstacle term for the state constraints it is finite precisely on the constrained region of attraction ([24]). The difficulty is computing it. Grid-based solvers of the associated Hamilton-Jacobi equation suffer from the curse of dimensionality ([40]), and sum-of-squares methods ([53]) require polynomial dynamics, whereas TFL closed loops are rational, trigonometric and singular. Neural solvers of Zubov equations treat equilibria or, in discrete time, compact invariant sets ([34, 28, 37, 48, 49, 39]). Most are trained on labels from trajectories simulated until convergence, and none handles a path-following target set together with controller singularities. On the TFL closed loops of this paper, we measure coverage against the region of attraction found by simulating a set of reference states, which we call the simulated region of attraction (§ 5). Two such solvers ([34, 37]), both physics-informed neural networks (PINNs), cover on average at most 11.3 %, 4.0 % and 0.0 % of it. To the best of our knowledge, no existing method learns the region of attraction of a TFL closed loop. The closest works on feedback linearization and path following ([45, 22]) do not estimate it for a given controller (Appendix § 7).
Our key observation is that existing Zubov solvers do not exploit the exact linear error dynamics of TFL. On the admissible set, where the controller is well defined, the coordinate transformation is regular and the state constraints hold, these dynamics make the constrained region of attraction the largest forward invariant subset. They also split the state-constrained Zubov value function exactly into a known quadratic cost-to-go and a safety value function that accounts for the singularities and the constraints. As a result, only the safety value function has to be learned, from short trajectory segments and without infinite-horizon trajectory labels. These results hold for arbitrary relative degrees, an optionally regulated rate of progress and internal dynamics that need not be stable. Beyond TFL, the same algorithm applies to closed loops with an exponentially stable equilibrium, with the quadratic form of the linearization in place of the exact cost-to-go. The decomposition is then no longer exact, yet the estimates cover between 86.5 % and 98.9 % of the simulated region of attraction on one seed of five benchmarks (Appendix D). Our main contributions are as follows.
We characterize the constrained region of attraction of a TFL closed loop as the largest forward invariant subset of the admissible set (Theorem 2). We also decompose its Zubov value function into a known quadratic cost-to-go and a safety value function (Lemma 1).
We propose an algorithm that learns only the safety value function, by fitted value iteration without infinite-horizon trajectory labels (Algorithm 1).
We construct a two-layer estimate with a verified inner layer and an outer layer tested on samples (Theorem 3), with statistical bounds on what the tests can miss (§ 4). On a unicycle and a planar quadrotor, it covers more of the simulated region of attraction than the quadratic cost-to-go and two PINN solvers of Zubov’s equation (Table 1).
Section 2 formulates the problem, Section 3 presents the method, Section 4 states its guarantees, and Section 5 reports the simulations. The appendices give the proofs and further details.
Notation. For a function and a vector field , is the Lie derivative, written in coordinates. We write for the solution of the closed loop from in physical time, and for the distance of a fixed Riemannian metric on the state space, with . Moreover, , is the interior of a box , and is a sublevel set of .
Preliminaries and Problem Formulation
Transverse feedback linearization
We follow the geometric formulation of TFL of Nielsen and Maggiore [44], and its use for path following with a regulated rate of progress of Akhtar et al. [7], and we refer to these works for details. Consider a smooth control-affine system
on an -dimensional smooth manifold , where and are smooth vector fields on . Let be a smooth position map, where is , , or its quotient by periods in some coordinates, with the metric induced by the Euclidean metric. Let be open and let be smooth, defining a regular path with rank on . Let , with or , be a smooth progress map with on . On , we write and for and .
The controller regulates the transverse outputs and, optionally, the rate of progress along . Let and, when the rate of progress is regulated, be integers such that, on and for , for and for . The transverse coordinates are with and , and the tangential coordinates are . If the rate of progress is regulated, let and let the coordinate transformation be . Otherwise, let and . In both cases we assume . The decoupling matrix has the rows , in the first case only, and for . Wherever has full row rank, the regulated outputs have vector relative degree , or , and the TFL controller is
where the first row of each vector is present only when the rate of progress is regulated, is the right inverse of (equal to when ), and is the reference rate of progress. The error coordinates are when the rate of progress is regulated and otherwise, with dimension or . The gains and are chosen such that with Hurwitz and block diagonal. The remaining dimensions of the state carry the internal dynamics of the closed loop, which contain the motion along ([43]). They may be unstable, and our results do not assume their stability.
Path following manifold. As in Nielsen and Maggiore [43] and Akhtar et al. [7], the path following manifold is the maximal controlled invariant subset of the lift of the path. In other words, it is the set of states from which a control signal can keep the position on for all future time. Wherever the rows of that belong to are linearly independent, is the embedded submanifold of dimension (Appendix § 8).
Examples. Our experiments use two systems that regulate the rate of progress, both with . The dynamically extended unicycle has , and , and it follows a unit circle and a sinusoid. The planar quadrotor with the thrust as a controller state has , and , and it follows a vertical unit circle. Appendix § 8 gives their dynamics, gains, thresholds and state constraints.
The admissible set
Let be a compact box in a fixed chart, with periodic coordinates identified, whose bounds are the state constraints. In this chart, let be the Jacobian of , let , and let , which equals when . The controller (2) is undefined where loses rank, the coordinate transformation loses rank where , and the progress map may lose smoothness where its gradient is unbounded, as at the centre of a circular path. We call the union of these sets the singular set. Given thresholds , the admissible set is
where the last inequality is imposed only when the rate of progress is regulated. The closure is compact because is. We assume and call the inequalities in (3) the regularity constraints. Their positive thresholds keep off the singular set, so the closed-loop vector field of (1) and (2) is smooth on a neighbourhood of . Since scales with , grows large near the singular set, so we integrate the normalized field , where and is the diagonal matrix of inverse half-widths of . This change of time scale preserves orbits, forward invariance and convergence within , and all step lengths in training and in the tests are measured in the time of , which we call normalized time.
Target set and constrained region of attraction
The target set is . Since on , has full row rank there, so . Without regulation of the rate of progress, . With it, is the part of on which the path is traversed at the reference rate. We make the following assumption.
Assumption 1 (a) The closed-loop vector field and the error map are smooth on a neighbourhood of . (b) for all , with Hurwitz. (c) , so on .
Parts (a) and (b) follow from § 2.1 and § 2.2. Part (c) keeps the motion along the path inside the state constraints and off the singular set, and Appendix § 8 checks it for our examples. The set is compact and forward invariant (Appendix § 9), and it may be disconnected, as for the unicycle, which can follow the path driving forward or in reverse (Appendix § 8). The constrained region of attraction of is
By Theorem 2, a trajectory from a state of leaves , by violating a regularity constraint or a state constraint. Deciding membership in directly thus requires simulation.
Problem 1 Given , and satisfying Assumption 1, compute a set that contains a neighbourhood of , is described by sublevel sets of explicit functions, and lies in up to a set of bounded measure, with a stated confidence. The set must be obtained without simulating trajectories until they converge, and it should cover as large a share of as possible.
Methodology
We address Problem 1 with a two-layer estimate. The inner layer is a sublevel set of a known quadratic function of the error coordinates, and its inclusion in is verified by interval branch and bound. The outer layer is a sublevel set of a learned approximation of the state-constrained Zubov value function, and its inclusion is tested on samples. The estimate is the union of the two layers. This section describes its steps, which Algorithm 1 collects at the end, and § 4 states the guarantees of each layer.
Decomposition of the value function
Let , where is a positive diagonal scaling matrix. Since is Hurwitz, for every symmetric the Lyapunov equation has a unique symmetric solution . The quadratic cost-to-go is , and the running cost is with a cost scale . Appendix B gives , and for our examples.
Let , , be the distances to the regularity constraints of (3) and to the nonperiodic faces of . Each is locally Lipschitz on a neighbourhood of and positive on , and at every point of at least one vanishes. The obstacle function is
where the constants satisfy . Hence is locally Lipschitz on , vanishes on a neighbourhood of , and tends to infinity at . For a state whose trajectory exists and stays in for all , the state-constrained Zubov value function of Grüne and Zidani [24] and its bounded transform are
with integration in physical time. For every other state, and . Theorem 2 in § 4 shows that .
Lemma 1 Let Assumption 1 hold. Then for all . Consequently, for every and such that for all , Define the margin function by . For every $x \in \Omega_{\mathrm{ad}},
where if the trajectory of leaves . In particular, and on . Moreover, for every and every such that for all ,
The proof is in Appendix § 9. Identity (8) has the form of the self-consistency equation of an undiscounted safety value function [55], so we call the safety value function. Since is known in closed form, only has to be learned, and the quadratic term supplies the strict decrease of along trajectories where is constant. Moreover, the bound holds for every approximation of the form plus a nonnegative function, which gives the inclusion of Theorem 3(ii). When the inner layer is not negligible, we add a shaping cost to , to steepen just outside the inner layer. Its integral is not known in closed form, so absorbs it, and the decomposition, the training and the guarantees extend to it (Appendix § 12). For brevity, this section presents the method without it.
Neural candidate
We approximate on in the form
with and outside , and we call the candidate. Here is a multilayer perceptron with weights and . Its features are built from the state, the error coordinates and the distances to the singular set (Appendix § 12). The constant is the first percentile of over the collocation points, the admissible samples on which the network is trained (Appendix § 12), and the factor gives on . Like the exact solution, is plus a function that is at least . Hence on , and every sublevel set with lies in . These sublevel sets therefore satisfy the regularity constraints and the state constraints by construction, although their forward invariance is not automatic.
Training and level selection
Fitted value iteration. Training solves (8) by regression. The update follows the semi-Lagrangian scheme (5.1) of Grüne and Zidani [24], with a target network as in reach-avoid reinforcement learning [26]. Let denote the solution of . Since (8) concerns the orbit only, it holds in normalized time and for every step . For each collocation point , we integrate by the fourth-order Runge-Kutta method (RK4) over a step with substeps of length . We then store the end point and the running maximum over the substep end points, including , with if a substep leaves . These quantities do not depend on the network, so they are computed once. Training keeps an exponential moving average of the weights. With a copy of these averaged weights, refreshed after every epoch, the target is
which is bounded even where the exact value is infinite and equals zero when the step leaves . Since , the loss is where , and elsewhere the one-sided loss only keeps below , so that zero loss implies . Finally, for a point whose step leaves , this loss and the obstacle penalty below carry the factor with an exit weight .
Step setting. Unrolling (10) over successive steps shows that its only continuous fixed point that vanishes on is for every step , which is because the decomposition is exact (Appendix § 12). A long step is therefore admissible on a TFL closed loop, and it carries the value of its end point over a long arc of the orbit in each update. Since the best step within a training budget still depends on the system, the method does not fix it in advance. It trains seed 0 with a long-step setting (, , ) and with a short-step setting (, , ), where is the 90th percentile of over admissible samples. It then keeps, for all seeds, the setting whose estimate holds the larger share of box samples after the level selection and the final test described below, before fine-tuning. This share is the score by which level selection ranks snapshots, and it uses no reference labels.
Penalties. In addition to the regression loss, three penalties encode properties of the exact solution. The finite-step penalty asks to decrease over steps of unit length in normalized time that stay in , the decrease penalty asks for where , and the obstacle penalty asks for where . Except on the quadrotor, every term carries an exponential loss weight, which keeps the gradient at the levels near one where the outer level is decided. In addition, the states whose unit step violates the decrease most join the data during training (Appendix § 12).
Level selection. The inner layer is , with one percent below the largest tested level that passes the interval verification of § 4, and . A state satisfies the finite-step decrease condition for a function if, for some horizon , for all and . In level selection, . A state with that fails this condition is a counterexample. The condition is tested numerically, with the exemptions of Appendix § 12. The outer level is the lowest value of among the sampled counterexamples, or one if none is found, and we call the annulus. Training saves a snapshot of the network every five epochs. Screening sets for every snapshot from counterexamples sought on samples of the box, of its faces and next to the singular set, and ranks the snapshots by the share of box samples in . The three best are then retested on fresh samples, and the best one after the retest is kept (Appendix § 12).
Counterexample-guided fine-tuning. Since the outer level is the lowest value among the counterexamples, a handful of states determines it. Each of four rounds of fine-tuning therefore searches fresh samples for counterexamples with and runs 300 Adam steps on them, with a decrease margin and an anchor term that holds elsewhere. The networks of these rounds and the trained network are screened like training snapshots. With the kept network and its level fixed, the final test draws i.i.d. uniform samples of the box, with a seed disjoint from those of all preceding stages. The final level is then the minimum of and the values of at the counterexamples that this test finds. If the kept network is the trained one, the counterexamples of its earlier final test count as well. Algorithm 1 summarizes the method.
Guarantees
This section characterizes , gives sufficient conditions for the inclusion of both layers of the estimate in , and states what the tests of the outer layer guarantee. The proofs are in Appendix § 9.
Theorem 2 Let Assumption 1 hold. A state belongs to if and only if its trajectory exists and remains in for all . Consequently, is the largest forward invariant subset of , and .
The set equality is the conclusion of Grüne and Zidani [24], here for a target set that may be disconnected and without stability assumptions on the internal dynamics. The next result gives sufficient conditions for the inclusion of both layers in .
| Algorithm 1: Two-layer estimate of the constrained region of attraction of a TFL closed loop |
| Input: closed loop , admissible set , quadratic cost-to-go , obstacle function , set of step settings |
| Output: estimate |
| Inner layer: set one percent below the largest tested level that passes the interval verification of § 4, and set , |
| Step setting: on seed 0, run the lines below up to the level selection and a final test for every setting in , and keep, for all seeds, the setting whose estimate holds the largest share of box samples |
| Step data: from every collocation point , integrate over the step and store the end point and the running maximum |
| for each epoch do |
| freeze a copy of the averaged weights and form the target of (10) |
| update on the regression loss of to , with the factor on points whose step leaves , plus the three penalties |
| add the states that violate the finite-step decrease most to the data |
| end |
| Level selection: for every snapshot, set to the lowest value of among the sampled counterexamples, and keep the snapshot whose estimate holds the largest share of box samples |
| Fine-tuning: for four rounds, search fresh samples for counterexamples with , train on them and repeat the level selection |
| Final test: draw i.i.d. samples and lower to the lowest value of among the counterexamples found |
Table 1.
Theorem 3 Let Assumption 1 hold, let , and . (i) Suppose that every with satisfies the regularity constraints of (3), and that at every such point on the field points strictly into across every active face. Then is forward invariant and . (ii) Let satisfy , as does for every . Then . (iii) Let (i) hold, let be as in (ii) and let . Suppose that every with satisfies the finite-step decrease condition for . Then .
The inclusion of part (ii) holds by construction for every candidate, and we check the hypotheses of part (i) by interval branch and bound (Moore et al., [41]) in floating-point arithmetic with outward rounding (Appendix § 11).
Statistical bounds. Two bounds quantify what the sample tests can miss (Appendix § 9). First, the final test fixes the network and the level before it draws its samples, so a scenario bound applies to it (Proposition 4). With probability at least , the counterexamples of the implemented test that remain in the annulus have measure at most . However, such a bound does not bound the measure of the states of the outer layer outside (Kenanian et al., [29]). Second, the direct test of unsafe states therefore simulates fresh i.i.d. samples until they reach the inner layer and counts the samples of an estimate that fail to do so inside . With confidence , the states of the estimate outside then have measure at most the one-sided Clopper–Pearson bound (Proposition 5), which is about for .
| Case | Share of box | Inner | Quadratic | Zubov PINN | Dirichlet-Zubov PINN | No decomp. | Ours | Exact | FP |
| Circle | 84.8 | 6.4 | 8.3 | 85.4 | 0.00 | ||||
| Sinusoid | 95.0 | 1.1 | 2.1 | 88.1 | 0.00 | ||||
| Quadrotor | 56.2 | 0.0 | 0.0 | 46.8 | 0.00 |
Table 1 (PDF p. 9). Coverage (%) of the simulated region of attraction on the TFL cases, mean std over four seeds. Exact : exact value function cut at the level of the candidate. FP: largest share of reference states inside an estimate and outside that region.
What is not proved. The interval verification of Theorem 3(i) relies on the accuracy of the mathematical library and on the one percent reduction of the level, so we call the inner layer verified, not proved. The hypothesis of part (iii) is tested on samples only, so the outer layer is not proved either. The direct test classifies each sample by a finite simulation, so its bound holds up to the integration error (Appendix G). It changes no estimate, so the estimate itself is still obtained without simulating trajectories until they converge.
Simulations
Setup. Tables 5 and 6 in Appendix § 12 list the configuration. It is the same for all systems, except for two loss weights and the shaping cost on the quadrotor (Appendix § 12). Four seeds are trained per case. The reference simulation classifies quasi-random states of the box by integrating the closed loop until the state converges to or leaves (Appendix § 12). The simulated regions of attraction hold 84.8 %, 95.0 % and 56.2 % of the three boxes (Figure 1). The coverage of an estimate is the share of the reference states in the simulated region of attraction that it contains. Training, the step setting and level selection never use the reference simulation. However, we developed the configuration, including the two step settings, while watching coverage on seed 0 (Appendix § 12), and all reported runs, seed 0 included, were trained after these choices. The step setting keeps the long step on all three cases.
Figure 1. Slices of the three cases, the unicycle at heading north and speed 1 and the quadrotor at the orbit velocity, level tilt and hover thrust. The estimate is the union over four seeds.
Baselines. All baselines pass through level selection and the final test. The quadratic baseline is alone. The Zubov PINN of Liu et al. [34] and the Dirichlet-Zubov PINN of Meng and Liu [37] learn from simulated labels over four seeds. The scale of their tanh transform is searched on seed 0, and neither receives fine-tuning (Appendix D). The no-decomposition baseline uses the same network, data, losses, level selection and fine-tuning without the term in the value function. The safety-value baseline sets , so it has no running cost and learns along trajectories alone, on seed 0 without fine-tuning.
How large are the estimates? The estimates cover most of the simulated region of attraction on the unicycle and a smaller part of it on the quadrotor (Table 1). Fine-tuning raises the mean single-seed coverage from 70.1 % to 72.0 % on the circle and from 82.1 % to 83.4 % on the sinusoid. No reference state lies inside an estimate and outside the simulated region of attraction. The direct test of unsafe states, with i.i.d. box samples per case, finds one unsafe sample in the union of four estimates of each case. With confidence and up to the integration error, the states of each union outside therefore occupy at most of the box (Appendix G). The quadrotor is the hardest of the three cases, with six states and a negligible inner layer (Appendix G). Counterexamples hold its outer levels between 0.12 and 0.25. At these levels the exact value function covers 46.8 %, of which the outer layers reach 41 %, and the bound alone excludes 37.5 % of its simulated region of attraction. Both the fit and the level therefore limit this estimate.
Does the decomposition matter? The candidate covers 60.2, 52.0 and 19.1 points more than the no-decomposition baseline, which must learn the integral of the running cost and covers 11.8 %, 31.4 % and 0.0 % under the same tests. The safety-value baseline has no running cost. On seed 0, before fine-tuning, it covers 11.1 %, 33.1 % and 1.3 %, against 72.0 %, 82.5 % and 24.9 % for the candidate (Table 8 in Appendix G).
| Case | Variant | Coverage | FP | |
| Circle | full method | 0.869 | 72.0 | 0.000 |
| Circle | PDE residual loss alone | 0.195 | 10.5 | 0.000 |
| Circle | safety-value baseline | 0.743 | 11.1 | 0.002 |
| Circle | no finite-step penalty, no counterexamples | 0.768 | 63.4 | 0.000 |
| Circle | no exponential loss weight | 0.777 | 66.7 | 0.000 |
| Circle | sum form | 0.995 | 9.9 | 0.000 |
| Sinusoid | full method | 0.907 | 82.5 | 0.000 |
| Sinusoid | safety-value baseline | 0.928 | 33.1 | 0.002 |
| Quadrotor | full method | 0.314 | 24.9 | 0.000 |
| Quadrotor | safety-value baseline | 0.483 | 1.3 | 0.000 |
Table 8. Ablations on seed 0 before fine-tuning, with the same level selection and final test. FP: share (%) of reference states inside the estimate and outside the simulated region of attraction.
How do the PINN baselines compare? They cover much less than the candidate on all three cases (Table 1), although both use simulated labels and the candidate uses none. The lowest sampled value outside the inner layer belongs to a counterexample in every seed for the Zubov PINN on the circle and the quadrotor, and for the Dirichlet-Zubov PINN on the sinusoid and the quadrotor. In these cases the estimate reduces to the inner layer. However, level selection is not what limits them. Even at the level that the reference simulation itself would allow, no PINN network of any scale, seed, network size or training snapshot covers more than 20.2 %, 33.7 % and 2.4 % (Appendix D).
Closed loops with an equilibrium. On seed 0 of five benchmarks of the literature, the step setting of Algorithm 1 keeps the long step on two and the short step on the others. After fine-tuning, the estimates cover between 86.5 % and 98.9 % of the simulated regions of attraction (Table 3 in Appendix D).
| System | Share, long | Share, short | Kept | Coverage, long | Coverage, short | Coverage, kept, fine-tuned |
| Circle | 61.1 | 59.3 | long | 72.0 | 69.8 | 74.0 |
| Sinusoid | 78.4 | 54.3 | long | 82.5 | 57.2 | 83.7 |
| Quadrotor | 14.0 | 5.3 | long | 24.9 | 9.4 | 22.9 |
| Van der Pol, | 30.3 | 38.8 | short | 77.3 | 98.9 | 98.9 |
| Van der Pol, | 15.4 | 21.3 | short | 54.1 | 74.9 | 89.2 |
| Pendulum | 30.2 | 27.4 | long | 93.5 | 84.6 | 94.2 |
| 3-D system | 50.0 | 40.0 | long | 84.7 | 67.8 | 86.5 |
| Power system | 15.7 | 29.9 | short | 47.5 | 90.3 | 92.4 |
Table 3. Step setting of Algorithm 1 on seed 0. Share: share of box samples in the estimate (%) before fine-tuning, which decides the setting without reference labels. Coverage (%) of the simulated region of attraction is shown for comparison, before fine-tuning for both settings and after fine-tuning for the kept one.
Conclusions
For a TFL controller with state constraints, the exact linear error dynamics make the constrained region of attraction the largest forward invariant subset of the admissible set. They also split its Zubov value function into a known quadratic cost-to-go and a safety value function. Learning only the latter by fitted value iteration gives a two-layer estimate, with an inner layer verified by interval branch and bound and an outer layer tested on samples. On average, this estimate covers 72.0 %, 83.4 % and 19.1 % of the simulated regions of attraction of a unicycle on two paths and of a planar quadrotor. The method needs the model, and we did not attempt the spatial quadrotor. Two further directions remain. First, a rigorous flow enclosure on the annulus would turn the test of the outer layer into a verification. Second, on the quadrotor the estimate reaches 41 % of what the exact value function covers at the same levels, so both a better fit and higher outer levels remain to be gained.
Generative AI Use: Generative AI tools were used to improve the syntax and grammar of some paragraphs in the manuscript. The authors reviewed and verified all AI-assisted content.
Related Work
Zubov functions with constraints. Camilli et al. [18] extended Zubov’s equation to perturbed systems, and Grüne and Zidani [24] added state constraints through an obstacle term. Our value function is their equation (3.4) without the perturbation. Xue et al. [53] use the same kind of value function with a prescribed running cost built from a polynomial, whose integral along trajectories is unknown, and compute convex sum-of-squares inner approximations for polynomial systems. Lemma 1 gives this integral in closed form for TFL closed loops. Converse Lyapunov-barrier theorems show that a smooth Lyapunov function exists on sets such as when the target set is compact and uniformly asymptotically stable, for perturbed continuous-time systems (Meng et al. [38]) and for hybrid systems (Meng and Liu [36]). Such a function is not known in closed form. Neural solvers of Zubov equations (Liu et al. [34, 33]; Kang et al. [28]) and their state-constrained versions in continuous time (Meng and Liu [37]) and discrete time (Serry et al. [48, 49]) learn the value function of an equilibrium or of a compact invariant set. Those that certify the result verify Lie-derivative conditions in continuous time or one-step conditions in discrete time, with satisfiability-modulo-theories or bound-propagation tools. Xue et al. [52] characterize the interior of the maximal robust region of attraction of an equilibrium of a state-constrained, perturbed discrete-time system by a Bellman equation, a discrete-time analogue of the equation of Grüne and Zidani [24]. Serry et al. [49] note that initial estimates of the region of attraction for non-singleton sets remain open, and our inner layer gives one for TFL closed loops.
Safety value functions and reachability. Equation (8) is the self-consistency equation of an undiscounted safety value function (Yu et al. [55]), and reach-avoid reinforcement learning uses a discounted form of this recursion with a target network (Hsu et al. [26]). Our update follows the semi-Lagrangian scheme (5.1) of Grüne and Zidani [24] with such a network. In the same way, Li et al. [31] bootstrap the data term of a Zubov PINN through the network at the end of a short integration. DeepReach solves reachability equations by a residual loss (Bansal and Tomlin, [12]), and Bokanowski et al. [17] use neural backward recursions for finite-horizon problems with a maximum running cost and obstacles. The closest precedent for our problem is Oishi et al. [45], who synthesize, on a grid and in low dimension, parameterized feedback-linearizing controllers together with the set of states on which they satisfy state and input constraints. Our setting differs in the target set, the singular decoupling matrix, the learned function, and the known decrease that the running cost provides. In the language of viability theory, is the viability kernel of under the closed loop.
Local estimates and sampled certificates. Manchester [35] computes inner estimates of the regions of attraction of hybrid limit cycles, starting from a quadratic form on the linearized transverse dynamics and enlarging it by sum-of-squares. Faulwasser and Findeisen [22] use a sublevel set of a quadratic in transverse coordinates as the terminal region of a path-following predictive controller, for a transverse feedback linearization without a singular decoupling matrix. Finite-step Lyapunov conditions go back to Aeyels and Peuteman [2] and Geiselhart et al. [23]. Bobiti and Lazar [15] verify a decrease condition on samples with Lipschitz margins on an annulus around a local set obtained from the linearization. Our annulus has the same shape and is tested on samples only, and our inner layer is verified by interval arithmetic. Liu et al. [34] combine a local quadratic region with a neural sublevel set, with the inclusion checked by a formal verifier. Li et al. [31] certify an annulus above an inner level, and Zanotta et al. [56] certify an annulus statistically above an independently certified inner set. The anchored networks of Liu [32] give an inclusion of sublevel sets with a constant that depends on the parameters. Inclusion (ii) of Theorem 3 holds with no such constant. Boffi et al. [16] bound the generalization error of certificates learned from trajectories, and Proposition 4 is the one-dimensional case of Campi and Garatti [19]. Kenanian et al. [29] show by example that a bound on the measure of counterexamples to the decrease does not bound the measure of non-converging states. Berkenkamp et al. [14] and Richards et al. [46] close the gap between samples and the continuum with Lipschitz constants on a grid. In our setting the Lipschitz constant grows as the distances to the singular set shrink, which makes such a grid expensive in four to six dimensions.
Neural Lyapunov functions and TFL. Neural Lyapunov functions with counterexample-guided training and exact verification ([20, 1, 21, 51, 54]) reach systems of six to twelve states. Barreau and Bastianello [13] learn a maximal Lyapunov function as a quadratic plus a signed residual, optimized jointly on a prescribed compact domain. Our candidate shares this additive form. Its quadratic is fixed and exact on the admissible set, and its network is nonnegative. We found no prior network composed with an obstacle function by a maximum. The TFL literature ([43, 44, 4, 7, 5, 10]) proves local exponential stability of the target set and treats regularity as an assumption. For the quadrotor, Assumption A1 of Al Lawati and Akhtar [10] assumes nonzero thrust on a neighbourhood of the path, and their Remark 2 notes that positive thrust is guaranteed only near the path following manifold.
The Two Systems
Both systems are instances of § 2.1 with , a regulated rate of progress and , so , and in both the internal dynamics are the progress along the path. Derivatives of circle-valued maps are computed with local angular lifts. The formulation also covers controllers that do not regulate the rate of progress. For example, the kinematic unicycle with constant speed of Nielsen and Maggiore [43] has , and one transverse output with , so , , and the motion along the path at the fixed speed is internal dynamics.
Path following manifold. Let be the submatrix of formed by its rows for , and let , an open set. On the differentials of the components of are linearly independent, by the argument of Isidori [27] with of full row rank, so is an embedded submanifold of of dimension . It is the maximal connected invariant subset of . Indeed, a state of with has a nonzero derivative , , that no input affects, so its position leaves immediately. Conversely, from a state with , the feedback keeps while the state remains in . Hence , and a trajectory that stays in a compact subset of with converges to . Since implies that , and hence , has full row rank, the target set of § 2.3 satisfies .
Unicycle. The dynamically extended unicycle, with , and , a simpler model than the car-like robot of Akhtar et al. [7] with the same singularity at zero speed, has state and
so is the speed, the acceleration and the turn rate. Both outputs have relative degree two, , , , and
The controller is (2) with (double pole at ) and , so the rate of progress is driven to . The error coordinates are with and . The target set is the path traversed at the rate of progress . For the unicycle it has two components, because depends on the velocity and not on the heading. In the first the heading is along the path and the speed is positive, and in the second the heading is against the path and the speed is negative. On both components, and are bounded away from zero and the state is inside the box, as the expressions below show, so is compact and lies in . The reference simulation counts convergence to either component.
Circle. The path is with , so , , , , . Box: , (speed in ), .
Sinusoid. The path is with , so , , , . The closed loop is -periodic in , taken in . The box is , . On the speed has magnitude . The thresholds are and , so excludes (circle) or (sinusoid) and, for the circle, the disc of radius around the centre.
Planar quadrotor. The planar quadrotor, with , and , has state and
in units with unit mass, unit gravity and unit path radius. The thrust is a controller state (the dynamic extension of Akhtar et al. [5]), and the tilt rate is the second input, as provided by a multirotor autopilot with an inner rate loop. For the loop , both outputs have relative degree three: , the angular rate, , and
so and . The controller is (2) with (triple pole at ) and (double pole at ). The error coordinates are with , and is the loop traversed counter-clockwise at angular rate . On the thrust is . The box is , , (thrust-to-weight ratio up to four), . The thresholds are those of the unicycle, so excludes and the disc of radius 0.05 around the centre. The reference simulation treats a sign change of between two steps as a departure through the singular set.
Constants. Table 2 lists the scaling and the cost scale . The scale is for the long-step setting and for the short-step setting of § 3.3, where is the 90th percentile of over admissible samples. Both constants are computed from the model, without simulating trajectories.
| Case | ||
| Circle | 0.0444 | 1.550, 1.396, 1.011 |
| Sinusoid | 0.0608 | 1.001, 0.625, 1.000 |
| Quadrotor | 0.0288 | 1.546, 2.439, 5.953, 0.703, 1.034 |
Table 2. Constants computed before training: the cost scale of the long-step setting and the diagonal of .
Proofs
The target set. Since is compact and is continuous, is compact. A trajectory from keeps while it stays in . It cannot reach , because the first such point would lie in by continuity of and Assumption 1(c). Hence is forward invariant.
Proof of Lemma 1. We prove the lemma with the shaping cost of Appendix § 12, for which , and gives the statement. By Assumption 1(b), on , hence and . Integrating along a trajectory that stays in on gives . Let the trajectory of stay in for all . Substituting in (6) with the shaping cost, with . By Theorem 2, and . Since vanishes on a neighbourhood of the compact set and vanishes on , and for all large , so converges to a finite and . Replacing by changes only where , and there it gives . Hence , which is (7) for . If the trajectory leaves , both sides are infinite by definition. Since and , and . For (8) and (15), split the supremum over and and use and for . In normalized time , , so the same identities hold along the orbit of with in place of . The convergence part of the proof of Theorem 2 uses only Assumption 1, and its finiteness part uses only the first identity of this lemma, so the argument is not circular.
Substituting (7) in the obstacle equation of Theorem 4.4 of Grüne and Zidani [24] gives, formally, the variational inequality on . Identity (8) is the dynamic programming principle (4.11) of Grüne and Zidani [24] written for . While is only nonincreasing along trajectories in , keeps the strict decrease of Remark 4.3 of Grüne and Zidani (2015): for every and , , hence when and .
Proof of Theorem 2. Let the trajectory of stay in while it exists. By Assumption 1(a), is bounded on the compact set , so the solution exists for all . By Assumption 1(b), . If stays bounded away from zero along a sequence , a subsequence of converges to a point of with , that is, a point of . This point would lie at positive distance from , which is impossible. Hence and . This argument uses no property of the internal dynamics. The converse holds by the definition of . A forward invariant subset of consists of states whose trajectories never leave , which gives the maximality. Let . The closure of its trajectory consists of the trajectory and its limit points, which lie in by Assumption 1(c). It is therefore a compact subset of , and the continuous function is bounded on it. By the first identity of Lemma 1, for all . The shaping cost vanishes on , which the trajectory enters and does not leave because is nonincreasing and tends to zero, so its integral is finite. Hence . Conversely, implies by definition that the trajectory of stays in .
Proof of Theorem 3. (i) Let . By hypothesis, satisfies the regularity constraints, so . Let . By Lemma 1, is nonincreasing along the trajectory on , so there. Suppose . Since is compact and is smooth on a neighbourhood of it, exists, lies in , and has by continuity. By hypothesis it satisfies the regularity constraints, so it lies on a nonperiodic face of , and points strictly into across every active face there. Then the trajectory lies outside just before , which contradicts for . Hence , the trajectory stays in , so is forward invariant, and by Theorem 2. The points of have and lie in , so . (ii) The inequality means , and . (iii) Let with , and define states as follows. If or , which implies by (ii), stop. Otherwise and , so for some the orbit of from stays in over and has . If the construction stops at some , the orbit stays in afterwards because is forward invariant by (i). Otherwise the steps, each of length at least one, cover all normalized time. In both cases the orbit of from stays in for all normalized time. By Assumption 1(a), is bounded on by some , so the physical time diverges, the trajectory of stays in for all , and by Theorem 2.
Proposition 4 Fix a measurable function , levels and a measurable counterexample test before drawing i.i.d. samples from the uniform distribution on . Let be the minimum of and the values of at the sampled counterexamples with , and let be the set of counterexamples of the test, as implemented, in . Then for every .
Proof of Proposition 4. Let , nondecreasing and left-continuous. If there is nothing to prove. Otherwise let . The set of counterexamples in has measure at least , the event $G(c_2) > \varepsilon requires and hence no sample in , and that has probability at most . This is the one-dimensional case of the scenario theorem of Campi and Garatti [19]. □
Proposition 5 Let the hypotheses of Theorem 3(i) hold, and fix a measurable set and a horizon in normalized time before drawing i.i.d. samples from . Let
Then . Let be the number of samples in , and for let solve , with . Then .
Proof of Proposition 5. A state of lies in and reaches along a trajectory in . By Theorem 3(i), is forward invariant and lies in , so the state lies in , and . The count of samples in is binomial with parameters and . Since increases with , the event is the event , with the largest such that , and it is empty if there is no such . The distribution function of the binomial at decreases in its success probability, so . This is the one-sided Clopper–Pearson bound. □
Baselines and Closed Loops with an Equilibrium
Zubov PINN. The Zubov PINN follows Liu et al. [34]: two hidden layers of 30 tanh units, , residual , a boundary term, and a data term on 3000 simulated states. It is trained for Adam steps on batches of 2048 from collocation points. On the TFL closed loops the residual is divided by of § 2, and the running cost is . The inputs are the normalized state with angles as sine and cosine, and four snapshots are screened. Both PINNs are set to one outside .
Dirichlet-Zubov PINN. We follow Meng and Liu [37]. With and their scaling constant 0.1, the same residual is solved with the field . The function is zero at the target and one on the 2000 admissible states closest to the boundary of . The data term integrates along the simulated trajectories, collocation and the boundary term act on admissible states, and the residual is divided by , a positive weight.
Scale search. Both PINNs use , with as in the definition of . The divisor is searched on seed 0 by the coverage after level selection and the final test, with the scoring of the reported numbers. For the Zubov PINN the grid is . For the Dirichlet-Zubov PINN it is , extended to 300 when 100 is best and to 1000 when 300 is best. Ties go to the smallest . On the circle and the quadrotor the Zubov PINN ties at the inner layer at every scale, as does the Dirichlet-Zubov PINN on the sinusoid and the quadrotor. On the sinusoid, the smaller divisor gives no gain for the Zubov PINN.
Larger PINNs and the oracle level. On the TFL closed loops we also trained both PINNs on seed 0 with the features and the network of the candidate, at and . The best of these four runs covers 18.2%, 2.5% and 0.0% on the three cases. To bound what any choice of level could give the PINNs there, we also compute an oracle level for every training snapshot of every PINN run. These runs are those of the scale searches, the four seeds and the larger networks. It is the largest level at which no reference state outside the simulated region of attraction lies in the estimate, so it uses the reference labels that the method never sees. The best of these estimates covers 20.2%, 33.7% and 2.4% on the three cases.
Other baselines on the TFL closed loops. The no-decomposition baseline keeps the features of the candidate and its factor . Its function is , and the running cost is integrated along the steps and included in the target. Like sum-of-squares (Xue et al. [53]), moment methods (Korda et al. [30]) require polynomial dynamics and are not applied to the TFL closed loops (§ 1).
Scoring. We score our reimplementations of both PINNs with the level selection and the final test of this paper. For the PINNs, the no-decomposition baseline and the safety-value baseline, is set on samples, because their functions have no lower bound . The PINNs are not clipped at zero, because clipping creates plateaus on which no step can decrease, and the oracle level above bounds what any choice of level could give them.
Closed loops with an equilibrium. Algorithm 1 also applies to a closed loop with an exponentially stable equilibrium at the origin. Its admissible set consists of the states of that satisfy the state constraints and avoid the obstacles, and is built as in (5). We take with , where solves for , and the running cost , where is that of the step setting, with the 90th percentile of over admissible samples. Since away from the origin, the integral of is no longer known in closed form. The candidate (9) and the target with an accumulated cost (Appendix § 12) therefore use in place of and , and replaces in the features. For the same reason, can decrease along a trajectory, so the fixed point of the target can overestimate , for every step (Appendix § 12). The network, the training and the fine-tuning are otherwise unchanged. On these systems the inner layer is a sublevel set of at a level set on samples, so it is not verified. Moreover, Theorem 2 is not available, because a trajectory that stays in need not converge. The estimates are therefore tested on samples only.
Systems. Each system has its equilibrium at the origin. Leaving the box counts as leaving . The reversed Van der Pol oscillator is , , on for and for . The pendulum is , , with , the gains of Example 5 of Liu et al. [34] and a torque bound that we add, on . The 3-D system is , , , with and , on . The power system is , , on . The reference simulation integrates states with RK4, step and horizon (60 for the power system), and a state has converged when the scaled state has norm below for one time unit.
Results. We ran Algorithm 1 on seed 0 of the five systems (Table 3). Its step setting keeps the long step on two of them and the short step on the others. On every system, as on the TFL cases, the share of box samples ranks the two settings as coverage does. After fine-tuning, the estimates of the kept setting cover between and of the simulated regions of attraction, and no reference state lies inside one of them and outside the simulated region of attraction. We ran one seed per system and did not compare with baselines on these systems.
Interval Verification of the Inner Layer
The hypotheses of Theorem 3(i) are statements about explicit functions on a box, and we check them by interval branch and bound (Moore et al. [41]). The box is split into cells. A cell is discarded when interval arithmetic shows that on it or that it contains no violating point. A cell contains no violating point when all distances to the regularity constraints are positive on it and, if it touches a face, the normal component of the field has the inward sign. The remaining cells are bisected along their widest normalized side until none is left. The checks run on the closed box with the periodic angle over . The lower bound of the quadratic form is the larger of two bounds: the one from the eigendecomposition of , with each squared projection of bounded below by zero, and . The distances and the field components are the symbolic expressions of Appendix § 8, evaluated in float64 interval arithmetic with outward rounding by two units in the last place per operation and with enclosed constants. For each case the level is found by bisection over twelve rounds between zero and the sampled minimum of over samples outside and samples on box faces where the field points outward. The last verified level is multiplied by 0.99. This one percent reduction absorbs the float32 evaluation of in the training pipeline and the residual effects of the eigendecomposition of and the float32 storage of and . Table 4 lists the verified levels, the share of the box in , the number of cells and the time. The verified claim concerns the set and does not otherwise involve . The verification was run with the cost scale of the short-step setting, and for the long-step setting the level is one third of the verified one, which describes the same set. Table 4 gives the levels for .
| Case | (% box) | Cells | Time (s) | ||
| Circle | 0.0350 | 0.0344 | 5.40 | 944395 | 588 |
| Sinusoid | 0.0170 | 0.0168 | 1.01 | 1047557 | 110 |
| Quadrotor | 0.0002 | 0.0002 | 0.00 | 21270981 | 3626 |
Table 4. Verified inner layers of the TFL cases, with the cells and time of the final verification.
Configuration
Training setup. The candidate uses the network, optimizer and collocation points of Table 5, with the level selection and fine-tuning of § 3. We set the configuration on seed 0, with level selection and a final test, while watching the coverage of the reference simulation. In a first stage we chose the weight of the finite-step penalty and , which gave the short-step setting. In a second stage, with these weights fixed, we chose the step , the cost scale and the exit weight one at a time on the quadrotor, which gave the long-step setting. The other constants of Table 5 and Table 6 come from an earlier version of the method on the unicycle. All reported runs, seed 0 included, were trained after these choices. Since the reference simulation guided these choices on seed 0, the coverage of seed 0 is not an independent test of the configuration, and Table 7 lists every seed.
| Value function and candidate | |
| Value function | , , of the step setting, with the 90th percentile of over admissible samples |
| Shaping cost | if the inner layer holds at least 1% of box samples, and otherwise. The function is a smooth step from to . The share on the sinusoid is 1.01%, close to this threshold |
| Network | : 64-64-64 tanh. Features: normalized state (angles as sine and cosine), , for the constraints on and . Softplus sharpness . The constant is the first percentile of on collocation points |
| Initialization | The network starts close to , which lies below the exact solution |
| Training | |
| Collocation | Sobol box samples, near the faces, next to the singular set (admissible ones kept) |
| Step setting | Long: (80 RK4 substeps), , . Short: (8 RK4 substeps), , . Each system keeps the setting whose seed-0 estimate, after level selection and the final test and before fine-tuning, holds the larger share of box samples. The regression loss and the obstacle penalty of a point whose step leaves are multiplied by |
| Target network | Copy of the averaged weights (decay 0.999), refreshed every epoch |
| Optimization | Adam, 100 epochs, batch 4096, cosine learning rate . Exponential loss weight with (no such weight on the quadrotor) |
| Decrease penalty | Weight 10, on , where |
| Obstacle penalty | Weight 10, on , where and |
| Finite-step penalty | Weight 100 (10 on the quadrotor), unit step with 40 RK4 substeps, on with and the end of the unit step from , on steps in with |
| Violating states | Every 5 epochs up to epoch 90: the states whose unit step violates the decrease most (no margin), in a pool of box samples and next to the singular set, 4 Gaussian neighbours (standard deviation 0.01 of the half-widths), rolling buffer |
Table 5. Configuration of every case: value function, candidate and training. Step lengths are in normalized time.
| Level selection and fine-tuning | |
| Level selection | Horizons , RK4 substep 0.025, states with , which lie in , exempt from the decrease. Snapshots every 5 epochs and at epoch 0. A test on a number of box samples adds half as many near the faces, a fifth as many next to the singular set and half as many on the faces, where states with outward field count as counterexamples. Screening: box samples, 3 rounds. Retest of the top 3: box samples, 5 rounds. Ranking by share of fixed box samples. Final test: i.i.d. uniform box samples |
| Fine-tuning | 4 rounds, each on the counterexamples with for a level estimated again in that round, pool of box samples, near the faces and next to the singular set, 8 Gaussian neighbours per counterexample, 300 Adam steps (learning rate , batch 4096), weight 100 on counterexamples whose step stays in , weight 10 on those that leave it, decrease margin 0.2, counterexample sets capped at , anchor on box samples, exponential loss weight with on all systems |
| Reference simulation | |
| Reference | Sobol states, RK4 step 0.01, horizon 30 (unicycle) and 40 (quadrotor). Converged: held for one time unit, and (unicycle), (quadrotor). States that have neither converged nor failed at the horizon count as outside |
Table 6. Configuration of every case: level selection, fine-tuning and reference simulation. Step lengths are in normalized time, except in the reference simulation.
| Case | Seed | before | Cov. before | Round | sel. | final | Annulus | Cov. | FP |
| Circle | 0 | 0.8695 | 71.99 | 4 | 0.9700 | 0.9700 | 574520 | 74.04 | 0.000 |
| Circle | 1 | 0.8570 | 69.16 | 4 | 0.9768 | 0.9768 | 544826 | 70.46 | 0.000 |
| Circle | 2 | 0.8539 | 69.82 | 3 | 0.9786 | 0.9739 | 570552 | 72.52 | 0.000 |
| Circle | 3 | 0.8358 | 69.23 | 4 | 0.9687 | 0.9687 | 548199 | 70.85 | 0.000 |
| Sinusoid | 0 | 0.9071 | 82.53 | 1 | 0.9770 | 0.9770 | 784823 | 83.75 | 0.000 |
| Sinusoid | 1 | 0.8668 | 81.64 | 1 | 0.9722 | 0.9700 | 790751 | 84.01 | 0.000 |
| Sinusoid | 2 | 0.8777 | 80.96 | 2 | 0.9609 | 0.9609 | 772925 | 82.31 | 0.000 |
| Sinusoid | 3 | 0.9099 | 83.16 | 4 | 0.9697 | 0.9677 | 787672 | 83.65 | 0.000 |
| Quadrotor | 0 | 0.3140 | 24.86 | 0 | 0.3922 | 0.2506 | 151731 | 22.86 | 0.000 |
| Quadrotor | 1 | 0.1226 | 13.53 | 0 | 0.2991 | 0.1226 | 135868 | 13.53 | 0.000 |
| Quadrotor | 2 | 0.3431 | 23.73 | 0 | 0.3689 | 0.2472 | 136567 | 21.03 | 0.000 |
| Quadrotor | 3 | 0.2541 | 23.15 | 0 | 0.3472 | 0.1823 | 145811 | 19.07 | 0.000 |
Table 7. Per-seed results. Before: the trained network. Round: selected fine-tuning round (0 is the trained network). Sel. and Final: level after the retest and after the final test. Annulus: final-test samples between and the selected level. FP as in Table 1.
Step setting. Unrolling the target over successive steps, with the shaping cost below, shows that its only continuous fixed point that vanishes on is for every step . The candidate satisfies both conditions, because and vanish on . On a TFL closed loop this function is the safety value function . Indeed, is the exact cost-to-go of the error dynamics, so the cost accumulated beyond is the nonnegative shaping cost alone. Hence is nondecreasing in , and the margin function can replace in the supremum (proof of Lemma 1). A long step then reaches the fixed point in fewer epochs, because each update carries the value of the end point over a long arc of the orbit. When the quadratic form is only a local Lyapunov function, as for the closed loops with an equilibrium of Appendix D, can decrease along a trajectory. The fixed point then overestimates for every step, so no step is exact there. In every case the method compares the two settings of § 3.3 on seed 0 by the share of box samples in the estimate, after level selection and the final test and before fine-tuning. It then keeps the setting with the larger share for all seeds. The comparison uses no reference labels, and both estimates have passed the same tests. The two settings differ in the cost scale and the exit weight as well as in the step, because each was developed as a whole (see the training setup above). Table 3 in Appendix D lists the comparison for every system.
Shaping cost. When the inner layer holds at least one percent of the box, we add a shaping cost with to , to steepen just outside the inner layer. Otherwise , as on the quadrotor. The quadrotor also uses the weight 10 for the finite-step penalty, against 100 on the unicycle, and no exponential loss weight during training. Here is a smooth nondecreasing function of , equal to zero below and to one above , where is the level of the inner layer. Its integral is not known in closed form, so absorbs it. With in physical time, the decomposition holds with , under the conventions of Lemma 1, and (8) becomes
In normalized time the accumulated cost is . Training then stores with the end point, the running maximum is , and the target is . The decrease penalty asks for and the obstacle penalty for where . For all of these reduce to the expressions of § 3. Theorem 3 is unaffected, because still holds.
Penalties. Lie derivatives in the decrease penalty and the obstacle penalty are taken at points of differentiability. The finite-step penalty acts on steps of unit length in from to . It asks for with and the accumulated shaping cost over the unit step, and it is applied to the values . An error in at a level moves by only the factor of it, so without the exponential loss weight the loss would carry almost no gradient at the levels near one. Every five epochs, the states of a fresh pool of whose unit step violates the decrease most, without the margin , join the data. Each comes with four Gaussian neighbours, and the data are kept in a rolling buffer of . The network starts close to , which lies below the exact solution. Where , any network with represents exactly, so the network has to approximate only where , on states whose trajectories later come closer to a constraint. Every term carries the exponential loss weight with , and none during training on the quadrotor. Table 5 gives every constant.
Screening and retest. Screening uses box samples and the additional samples of Table 6. The three best snapshots are retested on fresh box samples, and the best one is kept with the lowest level it received.
Additional Results
Unsafe samples and the final test. The direct test of unsafe states draws i.i.d. box samples per case and simulates each sample in the union of the four estimates until it reaches the inner layer. Of the 641 790, 815 296 and 135 702 samples in the unions, 1, 1 and 1 leave the box first, and finer integration confirms all 3. These counts give the bounds of § 5, and each bound holds for every seed. By Theorem 3(iii), the annulus of every seed whose estimate contains an unsafe sample holds counterexamples that the final test missed. The final test found a counterexample and lowered the level in seven of the twelve runs (Table 7).
How reliable is the direct test of unsafe states? It is at least as accurate as the reference simulation. The test uses seeds disjoint from all other tests, the RK4 integration of the normalized field with substeps of length 0.025 as in the final test, and . Every sample that reached did so within 8 units of normalized time. On 9000 reference states, the test and the reference simulation disagree on 41, and the reference simulation with a four times finer step agrees with the test on 38 of them. In the other 3, the test counts as unsafe a state that the finer reference finds converging, which is the conservative direction. None of the disagreeing states lies in an estimate, so this comparison does not test the safe labels inside the estimates. The 3 samples counted as unsafe still leave with four and sixteen times finer substeps, and the reference simulation with step 0.001 finds that they leave the box.
What do the two tests leave open? With and , Proposition 4 says with confidence that the uniform measure of counterexamples in the annulus is at most . As noted in § 4, this alone does not bound the measure of states of the outer layer outside . The reason is that a function can decrease everywhere except on a set of small measure while most trajectories diverge [29]. Both tests integrate numerically and check membership in at substeps of length 0.025, so a crossing of the singular set between substeps could go unnoticed. The test of Proposition 5 also checks the sign of at every substep, which detects a sign change of between substeps but not a crossing of the other constraints. The union of the outer layers over several seeds is therefore also tested on samples only. A deterministic check of the hypothesis of Theorem 3(iii) needs an enclosure of the flow over a unit step on every cell of the annulus, and we did not compute one.
What limits the outer layer? Both the level and the fit limit it, and the fit matters most on the quadrotor. The exact value function cut at the level of the candidate covers 85.4%, 88.1% and 46.8% of the simulated region of attraction, and the outer layers cover 84%, 95% and 41% of these shares. On the unicycle the levels lie above 0.96, and the states that the exact value function leaves out at these levels have trajectories that pass near a constraint before converging. They therefore have a large , and only a level closer to one would include them. On the quadrotor, counterexamples hold the level between 0.12 and 0.25, while the exact value function cut at 0.9 covers 80.5%. At these levels the bound excludes 37.5% of the simulated region of attraction, against 0.1% on the circle and 0.0% on the sinusoid.
What do the components contribute? Fine-tuning helps on the unicycle and is never selected on the quadrotor, where screening kept the trained network in four of four runs. Each of these networks therefore has two final tests, one before and one after fine-tuning, and its reported level is the lower of the two (Table 7). The draw of the final test alone moves the coverage of a seed by up to 7.0 points. Table 8 removes components on the circle, where the PDE residual loss alone, without fitted value iteration, penalties, counterexamples, exponential loss weight or weight averaging, reaches 10.5 %. The sum form replaces the maximum in (9) by , and it has neither the obstacle penalty nor the one-sided loss of § 3.3, because both refer to the maximum in (9).
Does the integration step change the reference labels? It changes few of them, at most 0.74 % on a case. We simulated 5000 of the reference states of each case again with an RK4 step of 0.0025 in place of 0.01. The label changes on 0.24 %, 0.24 % and 0.74 % of these states on the circle, the sinusoid and the quadrotor.
What limits the inner layer? Contact with the constraints bounds it from above, and the interval verification stops below that bound. On samples, the sublevel sets of first meet a regularity constraint, or a face of the box where the field points outward, at levels of at most 0.0417, 0.0188 and 0.0019 on the three cases. The verified levels are 0.0350, 0.0170 and 0.0002 (Table 4), and on the quadrotor the verification at the next level exceeds its budget of cells. The first contact lies near the centre of the circle, on the speed constraint for the sinusoid, and on the zero-thrust face for the quadrotor. Thrust enters only through the acceleration coordinates, so a state with no thrust can have a small , and the inner layer of the quadrotor is negligible.
Computation time. All computations run on the CPU of a laptop (Apple M1). The trainings ran eight at a time on one thread each, next to the level selection of other runs, and took 44 to 73 minutes per seed under this load. Level selection took 1 to 13 minutes, fine-tuning 2 to 59 minutes, and the interval verification 110 to 3626 seconds per case.
References
- [1]Alessandro Abate, Daniele Ahmed, Alec Edwards, Mirco Giacobbe, and Andrea Peruffo. FOSSIL: A software tool for the formal synthesis of Lyapunov functions and barrier certificates using neural networks. In ACM International Conference on Hybrid Systems: Computation and Control (HSCC), pages 1–11, 2021.
- [2]Dirk Aeyels and Joan Peuteman. A new asymptotic stability criterion for nonlinear time-variant differential equations. IEEE Transactions on Automatic Control, 43(7):968–971, 1998.DOI
- [4]Adeel Akhtar and Christopher Nielsen. Path following for a car-like robot using transverse feedback linearization and tangential dynamic extension. In IEEE Conference on Decision and Control and European Control Conference (CDC-ECC), pages 7974–7979, 2011.DOI
- [5]Adeel Akhtar, Steven L. Waslander, and Christopher Nielsen. Path following for a quadrotor using dynamic extension and transverse feedback linearization. In IEEE Conference on Decision and Control (CDC), pages 3551–3556, 2012.DOI
- [6]Adeel Akhtar, Steven L. Waslander, and Christopher Nielsen. Fault tolerant path following for a quadrotor. In IEEE Conference on Decision and Control (CDC), pages 847–852, 2013.DOI
- [7]Adeel Akhtar, Christopher Nielsen, and Steven L. Waslander. Path following using dynamic transverse feedback linearization for car-like robots. IEEE Transactions on Robotics, 31(2):269–279, 2015.DOI
- [8]Adeel Akhtar, Sajid Saleem, and Jinjun Shan. Path invariant controllers for a quadrotor with a cable-suspended payload using a global parameterization. IEEE Transactions on Control Systems Technology, 30(5):2002–2017, 2022.DOI
- [9]Adeel Akhtar, Sajid Saleem, and Jinjun Shan. Path following of a quadrotor with a cable-suspended payload. IEEE Transactions on Industrial Electronics, 70(2):1646–1654, 2023.DOI
- [10]Mohamed Al Lawati and Adeel Akhtar. Path-following control of a quadrotor using quasi-static transverse feedback linearization. In European Control Conference (ECC), pages 3454–3459, 2026a.
- [11]Mohamed Al Lawati and Adeel Akhtar. Quasi-static fault-tolerant feedback control of a quadrotor under rotor failure with provable safety guarantees. In IEEE Conference on Control Technology and Applications (CCTA), pages 681–686, 2026b.
- [12]Somil Bansal and Claire J. Tomlin. DeepReach: A deep learning approach to high-dimensional reachability. In IEEE International Conference on Robotics and Automation (ICRA), pages 1817–1824, 2021.arxiv.org/abs/2011.02082
- [13]Matthieu Barreau and Nicola Bastianello. Learning maximal regions of attraction on compact sets using Taylor Lyapunov networks. arXiv preprint arXiv:2408.17246, 2024.arxiv.org/abs/2408.17246
- [14]Felix Berkenkamp, Riccardo Moriconi, Angela P. Schoellig, and Andreas Krause. Safe learning of regions of attraction for uncertain, nonlinear systems with Gaussian processes. In IEEE Conference on Decision and Control (CDC), pages 4661–4666, 2016.DOI
- [15]Ruxandra Bobiti and Mircea Lazar. Automated-sampling-based stability verification and DOA estimation for nonlinear systems. IEEE Transactions on Automatic Control, 63(11):3659–3674, 2018.DOI
- [16]Nicholas M. Boffi, Stephen Tu, Nikolai Matni, Jean-Jacques E. Slotine, and Vikas Sindhwani. Learning stability certificates from data. In Conference on Robot Learning (CoRL), volume 155 of Proceedings of Machine Learning Research, pages 1341–1350, 2021.arxiv.org/abs/2008.05952
- [17]Olivier Bokanowski, Averil Prost, and Xavier Warin. Neural networks for first order HJB equations and application to front propagation with obstacle terms. Partial Differential Equations and Applications, 4(5):45, 2023.arxiv.org/abs/2210.04300
- [18]Fabio Camilli, Lars Grüne, and Fabian Wirth. A generalization of Zubov’s method to perturbed systems. SIAM Journal on Control and Optimization, 40(2):496–515, 2001.
- [19]Marco C. Campi and Simone Garatti. The exact feasibility of randomized solutions of uncertain convex programs. SIAM Journal on Optimization, 19(3):1211–1230, 2008.DOI
- [20]Ya-Chien Chang, Nima Roohi, and Sicun Gao. Neural Lyapunov control. In Advances in Neural Information Processing Systems, volume 32, pages 3245–3254, 2019.arxiv.org/abs/2005.00611
- [21]Hongkai Dai, Benoit Landry, Lujie Yang, Marco Pavone, and Russ Tedrake. Lyapunov-stable neural-network control. In Robotics: Science and Systems (RSS), 2021.arxiv.org/abs/2109.14152
- [22]Timm Faulwasser and Rolf Findeisen. Nonlinear model predictive control for constrained output path following. IEEE Transactions on Automatic Control, 61(4):1026–1039, 2016.DOI
- [23]Roman Geiselhart, Rob H. Gielen, Mircea Lazar, and Fabian R. Wirth. An alternative converse Lyapunov theorem for discrete-time systems. Systems & Control Letters, 70:49–59, 2014.DOI
- [24]Lars Grüne and Hasnaa Zidani. Zubov’s equation for state-constrained perturbed nonlinear systems. Mathematical Control and Related Fields, 5(1):55–71, 2015.DOI
- [26]Kai-Chieh Hsu, Vicenç Rubies-Royo, Claire J. Tomlin, and Jaime F. Fisac. Safety and liveness guarantees through reach-avoid reinforcement learning. In Robotics: Science and Systems (RSS), 2021.DOI
- [27]Alberto Isidori. Nonlinear Control Systems. Springer, 3rd edition, 1995.
- [28]Wei Kang, Kai Sun, and Liang Xu. Data-driven computational methods for the domain of attraction and Zubov’s equation. IEEE Transactions on Automatic Control, 69(3):1600–1611, 2024.arxiv.org/abs/2112.14415
- [29]Joris Kenanian, Ayca Balkan, Raphaël M. Jungers, and Paulo Tabuada. Data driven stability analysis of black-box switched linear systems. Automatica, 109:108533, 2019.arxiv.org/abs/1803.03687
- [30]Milan Korda, Didier Henrion, and Colin N. Jones. Inner approximations of the region of attraction for polynomial dynamical systems. In IFAC Symposium on Nonlinear Control Systems (NOLCOS), volume 46 of IFAC Proceedings Volumes, pages 534–539, 2013.arxiv.org/abs/1210.3184
- [31]Haoyu Li, Xiangru Zhong, Bin Hu, and Huan Zhang. Two-stage learning of stabilizing neural controllers via Zubov sampling and iterative domain expansion. In Advances in Neural Information Processing Systems, volume 38, pages 82230–82259, 2025.arxiv.org/abs/2506.01356
- [32]Jun Liu. Universal approximation of maximal Lyapunov functions with anchored neural networks. arXiv preprint arXiv:2608.17290, 2026.arxiv.org/abs/2608.17290
- [33]Jun Liu, Yiming Meng, Maxwell Fitzsimmons, and Ruikun Zhou. LyZNet: A lightweight Python tool for learning and verifying neural Lyapunov functions and regions of attraction. In ACM International Conference on Hybrid Systems: Computation and Control (HSCC), pages 1–8, 2024.
- [34]Jun Liu, Yiming Meng, Maxwell Fitzsimmons, and Ruikun Zhou. Physics-informed neural network Lyapunov functions: PDE characterization, learning, and verification. Automatica, 175:112193, 2025.DOI
- [35]Ian R. Manchester. Transverse dynamics and regions of stability for nonlinear hybrid limit cycles. In IFAC World Congress, volume 44 of IFAC Proceedings Volumes, pages 6285–6290, 2011.arxiv.org/abs/1010.2241
- [36]Yiming Meng and Jun Liu. Lyapunov-barrier characterization of robust reach–avoid–stay specifications for hybrid systems. Nonlinear Analysis: Hybrid Systems, 49:101340, 2023.arxiv.org/abs/2211.00814
- [37]Yiming Meng and Jun Liu. Towards learning and verifying maximal Lyapunov-barrier functions with a Zubov PDE formulation. In European Control Conference (ECC), 2026.arxiv.org/abs/2511.09523
- [38]Yiming Meng, Yinan Li, Maxwell Fitzsimmons, and Jun Liu. Smooth converse Lyapunov-barrier theorems for asymptotic stability with safety constraints and reach-avoid-stay specifications. Automatica, 144:110478, 2022.arxiv.org/abs/2009.04432
- [39]Yiming Meng, Ruikun Zhou, and Jun Liu. Learning regions of attraction in unknown dynamical systems via Zubov–Koopman lifting: Regularities and convergence. IEEE Transactions on Automatic Control, 70(10):6672–6687, 2025.arxiv.org/abs/2311.15119
- [40]Ian M. Mitchell, Alexandre M. Bayen, and Claire J. Tomlin. A time-dependent Hamilton–Jacobi formulation of reachable sets for continuous dynamic games. IEEE Transactions on Automatic Control, 50(7):947–957, 2005.DOI
- [41]Ramon E. Moore, R. Baker Kearfott, and Michael J. Cloud. Introduction to Interval Analysis. SIAM, 2009.DOI
- [42]Christopher Nielsen and Manfredi Maggiore. Maneuver regulation via transverse feedback linearization: Theory and examples. In IFAC Symposium on Nonlinear Control Systems (NOLCOS), volume 37 of IFAC Proceedings Volumes, pages 57–64, 2004.
- [43]Christopher Nielsen and Manfredi Maggiore. Output stabilization and maneuver regulation: A geometric approach. Systems & Control Letters, 55(5):418–427, 2006.DOI
- [44]Christopher Nielsen and Manfredi Maggiore. On local transverse feedback linearization. SIAM Journal on Control and Optimization, 47(5):2227–2250, 2008.DOI
- [45]Meeko Oishi, Ian Mitchell, Claire Tomlin, and Patrick Saint-Pierre. Computing viable sets and reachable sets to design feedback linearizing control laws under saturation. In IEEE Conference on Decision and Control (CDC), pages 3801–3807, 2006.
- [46]Spencer M. Richards, Felix Berkenkamp, and Andreas Krause. The Lyapunov neural network: Adaptive stability certification for safe learning of dynamical systems. In Conference on Robot Learning (CoRL), volume 87 of Proceedings of Machine Learning Research, pages 466–476, 2018.arxiv.org/abs/1808.00924
- [47]Ashton Roza and Manfredi Maggiore. Path following controller for a quadrotor helicopter. In American Control Conference (ACC), pages 4655–4660, 2012.
- [48]Mohamed Serry, Haoyu Li, Ruikun Zhou, Huan Zhang, and Jun Liu. Safe domains of attraction for discrete-time nonlinear systems: Characterization and verifiable neural network estimation. In IEEE Conference on Decision and Control (CDC), pages 5774–5781, 2025.
- [49]Mohamed Serry, Maxwell Fitzsimmons, and Jun Liu. Safe and robust domains of attraction for discrete-time systems: A set-based characterization and certifiable neural network estimation. arXiv preprint arXiv:2603.03082, 2026.
- [50]Hamza Tariq and Adeel Akhtar. Dual path following for an aerial manipulator under state and input constraints via transverse feedback linearization. arXiv preprint arXiv:2611.01442, 2026.
- [51]Junlin Wu, Andrew Clark, Yiannis Kantaros, and Yevgeniy Vorobeychik. Neural Lyapunov control for discrete-time systems. In Advances in Neural Information Processing Systems, volume 36, pages 2939–2955, 2023.
- [52]Bai Xue, Naijun Zhan, and Yangjia Li. A characterization of robust regions of attraction for discrete-time systems based on Bellman equations. IFAC-PapersOnLine, 53(2):6390–6397, 2020.
- [53]Bai Xue, Qiuye Wang, Naijun Zhan, Shijie Wang, and Zhikun She. Synthesizing robust domains of attraction for state-constrained perturbed polynomial systems. SIAM Journal on Control and Optimization, 59(2):1083–1108, 2021.
- [54]Lujie Yang, Hongkai Dai, Zhouxing Shi, Cho-Jui Hsieh, Russ Tedrake, and Huan Zhang. Lyapunov-stable neural control for state and output feedback: A novel formulation. In International Conference on Machine Learning (ICML), volume 235 of Proceedings of Machine Learning Research, pages 56033–56046, 2024.
- [55]Dongjie Yu, Haitong Ma, Shengbo Eben Li, and Jianyu Chen. Reachability constrained reinforcement learning. In International Conference on Machine Learning (ICML), volume 162 of Proceedings of Machine Learning Research, pages 25636–25655, 2022.arxiv.org/abs/2205.07536
- [56]Pietro Zanotta, Panos Stinis, and Ján Drgoňa. SCORE: Statistical certification of regions of attraction via extreme value theory. arXiv preprint arXiv:2603.29658, 2026.
