betaFeedback

WorksEngineering

Implicit DeepReach Without Implicit Differentiation

Abstract

We consider the problem of approximating backward reachable and reach-avoid tubes for high-dimensional nonlinear systems whose Hamiltonians require numerical optimization over constrained controls and disturbances. Existing neural Hamilton-Jacobi (HJ) reachability methods alleviate the dimensionality bottleneck of grid-based solvers, but evaluating Hamiltonians without closed-form expressions introduces an additional computational challenge. We show that, under suitable assumptions, the gradients required for value-function training can be computed without differentiating through the inner optimization problem. Specifically, we establish an envelope-type gradient identity for the saddle point Hamiltonian, under which the optimal control and disturbance enter backpropagation as constants without gradient tracking. This enables end-to-end neural HJ reachability with numerically evaluated Hamiltonians while avoiding both solver unrolling and implicit differentiation. We demonstrate the accuracy and computational efficiency of the proposed approach on nonlinear reachability and reach-avoid problems of different scales.

Introduction

Hamilton–Jacobi (HJ) reachability provides a principled framework for reasoning about safety and robustness in dynamical systems. Given a target or unsafe set, reachability analysis characterizes the states from which an adversarial disturbance can force the system into the target despite the action of a controller [33, 17]. The resulting backward reachable tube (BRT) provides both a safety certificate and an associated feedback controller, making HJ reachability a powerful tool for safety-critical control.

A primary challenge in HJ reachability is the curse of dimensionality. Classical methods solve the associated Hamilton-Jacobi-Isaacs (HJI) equation or variational inequality on a state-time grid [34, 21], these methods are accurate but restricted to relatively low-dimensional systems. Neural approaches such as DeepReach [2] alleviate this bottleneck by representing the value function with a neural network and enforcing the HJI equation at sampled state-time points. Subsequent work has further improved the scalability and accuracy of this paradigm through additional supervision and guidance [40, 42].

In this work, we focus on a separate bottleneck that arises when the Hamiltonian of the system does not admit a closed form solution and is instead defined implicitly by a saddle point problem. While optimal control and disturbance inputs can admit closed-form expressions in some standard reachability problems, more general nonlinear dynamics, coupled control–disturbance interactions, and complex admissible input sets generally require solving a numerical optimization problem to evaluate the Hamiltonian [5, 11 b]. Recent approaches address such Hamiltonians through learned surrogates, data-driven approximations, or conservative bounds [7, 27] (Chilakamarri et al. 2025). These methods make the Hamiltonian more tractable before or outside the value-function computation, but they usually require additional computation or data collection and can introduce approximation error.

An alternative is to retain the numerical Hamiltonian optimization directly within neural HJI training, as explored in related implicit deep learning approaches for optimal control [12]. However, embedding the optimization into the training loop introduces an additional differentiation bottleneck. Each value-function update requires repeatedly solving the pointwise control-disturbance game, and conventional differentiation approaches propagate gradients through its solution. Unrolling the inner solver can incur substantial memory and computational overhead, while exact implicit differentiation typically requires solving an additional linear system for each Hamiltonian evaluation. In this work, we show that neither approach is necessary. By establishing an envelope-type gradient identity for the saddle point Hamiltonian, we obtain the gradients required for neural HJI training without differentiating through the inner optimization. In particular, our main contributions are: (i) End-to-end HJ reachability with implicit Hamiltonians. We develop a neural HJ reachability framework that directly incorporates numerically solved control-disturbance games into value-function training, enabling the treatment of nonlinear systems without tractable closed-form Hamiltonians. (ii) Theoretically justified gradients without optimizer sensitivities. Using a saddle point envelope argument, we establish exact gradient formulas under light assumptions, eliminating the need for solver unrolling or implicit differentiation through the inner optimization. (iii) Numerical validation. We demonstrate the accuracy and scalability of our approach on nonlinear reachability and reach-avoid problems in dimensions ranging from 2 to 50, showing close agreement with reference solutions and reliable closed-loop performance.

Background

Hamilton-Jacobi Reachability

Consider the controlled dynamical system over a finite time horizon

x˙(s)=f(x(s),u(s),d(s)),s∈[t,T],(1)\dot{x}(s) = f\left(x(s), u(s), d(s)\right), \qquad s \in[t,T], \tag*{(1)}

where x(s)∈Rnx(s) \in\mathbb{R}^{n} is the state, u(s)∈Uu(s) \in\mathcal{U} is the control, and d(s)∈Dd(s) \in\mathcal{D} is an adversarial disturbance with tt denoting an arbitrary initial time and TT is a fixed terminal time. Let the target set be represented by

L={x∈Rn:ℓ(x)≤0},(2)\mathcal{L} = \left\{x \in\mathbb{R}^{n} : \ell(x) \le0\right\}, \tag*{(2)}

for a sufficiently regular target function ℓ\ell. The target set can represent either a set of goal states or unsafe states, and ℓ(x)\ell(x) is typically chosen as a signed distance function.

The Backward Reachable Tube (BRT) ([33, 2]) of a given problem consists of states from which the disturbance can force the system into L\mathcal{L} at some time over [t,T][t,T] despite the action of the controller. This is characterized via the value function

V(x,t)=inf⁡d(⋅)sup⁡u(⋅)min⁡s∈[t,T]ℓ(x(s)),(3)V(x,t) = \inf_{d(\cdot)} \sup_{u(\cdot)} \min_{s\in[t,T]} \ell(x(s)), \tag*{(3)}

and the BRT is given by the zero sublevel set of the value function for any given tt, that is,

V(t)={x∈Rn:V(x,t)≤0}.(4)\mathcal{V}(t) = \{x \in\mathbb{R}^{n} : V(x,t) \le0\}. \tag*{(4)}

Define the HJI Hamiltonian

H(x,p):=max⁡u∈Umin⁡d∈Dp⊤f(x,u,d).(5)H(x,p) := \max_{u\in\mathcal{U}} \min_{d\in\mathcal{D}} p^{\top}f(x,u,d). \tag*{(5)}

where pp is the adjoint variable to the dynamics. Under standard regularity assumptions and the Isaacs condition, the value function satisfies the HJI variational inequality (VI), which reads

min⁡{∂tV(x,t)+H(x,∇xV(x,t)), ℓ(x)−V(x,t)}=0,(6)\min\{\partial_t V(x,t) + H(x,\nabla_x V(x,t)),\ \ell(x) - V(x,t)\} = 0, \tag*{(6)}

with terminal condition V(x,T)=ℓ(x)V(x,T) = \ell(x) for all x∈Rnx \in\mathbb{R}^{n}. Crucially, the VI formulation accounts for the possibility of reaching the target set at any time within the horizon, rather than only at the terminal time, distinguishing it from the standard HJI PDE.

Similarly, we can define the Backward Reach-Avoid Tube (BRAT) ([9]), which characterizes states from which the controller can reach the target set L\mathcal{L} despite adversarial disturbances, while avoiding an unsafe set G\mathcal{G}, defined as

G={x∈Rn:g(x)>0}.(7)\mathcal{G} = \{x \in\mathbb{R}^{n} : g(x) > 0\}. \tag*{(7)}

Here gg is also typically chosen as a signed distance function describing obstacles or unsafe states of the system, such as joint positions that violate a robot’s safety requirements. We note for goal-reaching problems, the roles of the control and disturbance in the Hamiltonian are reversed. Nonetheless, the BRAT can be similarly defined as the zero sublevel set of the corresponding value function V(x,t)V(x,t), which satisfies a slightly different variational inequality

max⁡{min⁡{∂tV(x,t)+H(x,∇xV(x,t)), ℓ(x)−V(x,t)}, g(x)−V(x,t)}=0,(8)\max\{\min\{\partial_t V(x,t) + H(x,\nabla_x V(x,t)),\ \ell(x) - V(x,t)\},\ g(x) - V(x,t)\} = 0, \tag*{(8)}

with terminal condition V(x,T)=max⁡{ℓ(x),g(x)}V(x,T) = \max\{\ell(x),g(x)\}. Here the outer maximum enforces the state constraint by excluding trajectories that enter the unsafe set during the horizon. The BRAT V(t)\mathcal{V}(t), can be recovered similarly following (4).

Neural HJ Reachability

To alleviate the curse of dimensionality of grid-based HJ solvers, neural approaches such as DeepReach ([2]) approximate the value function using a neural network Vθ(x,t)V_{\theta}(x,t) parameterized by θ\theta. Following the physics-informed neural network (PINN) paradigm ([37, 45]), the network is trained by minimizing the HJI variational inequality residual and enforcing the terminal condition at sampled state-time points. Specifically, the training objective takes the form

L(θ)=LVI(θ)+λLT(θ),(9)L(\theta) = L_{\mathrm{VI}}(\theta) + \lambda L_T(\theta), \tag*{(9)}

where

LVI(θ)=E(x,t)∼ρ[(min⁡{∂tVθ(x,t)+H(x,∇xVθ(x,t)),ℓ(x)−Vθ(x,t)})2](10)L_{\mathrm{VI}}(\theta)=\mathbb{E}_{(x,t)\sim\rho}\left[\left(\min\left\{\partial_t V_\theta(x,t)+H(x,\nabla_xV_\theta(x,t)),\ell(x)-V_\theta(x,t)\right\}\right)^2\right] \tag*{(10)}
LT(θ)=Ex∼ρT[(Vθ(x,T)−V(x,T))2].(11)L_T(\theta)=\mathbb{E}_{x\sim\rho_T}\left[\left(V_\theta(x,T)-V(x,T)\right)^2\right]. \tag*{(11)}

Here, ρ\rho and ρT\rho_T denote the sampling distributions over interior state-time points and terminal states, respectively, and λ>0\lambda>0 balances the two loss terms. The formulation above corresponds to the BRT problem; the BRAT loss follows analogously using the reach-avoid variational inequality.

A key requirement of these approaches is the repeated evaluation of the Hamiltonian H(x,∇xVθ)H(x,\nabla_xV_\theta) during training. Existing neural HJ methods typically consider settings in which the saddle point problem admits a closed-form solution. However, for more complex dynamics, determining the optimal control and disturbance may require numerical optimization. Addressing the resulting differentiability and computational cost bottlenecks remains an open challenge for neural HJ methods.

Neural HJ Reachability with Implicit Hamiltonians

As discussed in § 2, existing neural HJ reachability methods typically rely on Hamiltonians that admit closed-form evaluations. For general nonlinear dynamics with constrained controls and disturbances, the Hamiltonian is defined implicitly through a saddle point optimization problem. Under a standard projected gradient ascent-descent (PGDA) formulation, the inner optimization can be recast as the fixed-point problem over (u⋆,d⋆)(u^\star,d^\star) with each iteration defined as

uk+1=ΠU(uk+ηu∇u[pθTf(x,uk,dk)]),u^{k+1}=\Pi_U\left(u^k+\eta_u\nabla_u\left[p_\theta^{\mathsf T}f(x,u^k,d^k)\right]\right),
dk+1=ΠD(dk−ηd∇d[pθTf(x,uk,dk)]).d^{k+1}=\Pi_D\left(d^k-\eta_d\nabla_d\left[p_\theta^{\mathsf T}f(x,u^k,d^k)\right]\right).

Here, ΠU\Pi_U and ΠD\Pi_D denote projections onto the admissible sets, and ηu,ηd>0\eta_u,\eta_d>0 are the step sizes. Under suitable convex–concave assumptions PGDA can provably converge, more general saddle point problems may require alternative algorithms [19, 23, 30]. Regardless of the solver, differentiating through the inner optimization introduces an additional computational bottleneck in neural HJI training. Two conventional approaches exist: (i) Automatic Differentiation (AD): Unrolling the inner solver and backpropagating through its iterations, which can incur substantial computational and memory overhead especially as problem size and number of inner iterations increase. (ii) Implicit Differentiation: Using the implicit function theorem (IFT) [28] it is possible to compute the sensitivities of the saddle point directly under

Algorithm 1:Gradient-free neural HJ reachability with implicit Hamiltonians.
Input:Dynamics ff, target ℓ\ell, value network VθV_\theta, learning rate α\alpha
Output:Trained value function VθV_\theta
while not converged do
Sample {(xi,ti)}i=1N∼ρ\{(x_i,t_i)\}_{i=1}^{N}\sim\rho
pi←∇xVθ(xi,ti)p_i\leftarrow\nabla_xV_\theta(x_i,t_i)
(Vi,Vt,i)←(Vθ(xi,ti),∂tVθ(xi,ti))(V_i,V_{t,i})\leftarrow(V_\theta(x_i,t_i),\partial_tV_\theta(x_i,t_i))
for i=1,…,Ni=1,\ldots,N do
(ui⋆,di⋆)←SaddleSolve⁡(xi,stopgrad⁡(pi))(u_i^\star,d_i^\star)\leftarrow\operatorname{SaddleSolve}(x_i,\operatorname{stopgrad}(p_i))
Hi←piTf(xi,stopgrad⁡(ui⋆,di⋆))H_i\leftarrow p_i^{\mathsf T}f(x_i,\operatorname{stopgrad}(u_i^\star,d_i^\star))
Ri←min⁡{Vt,i+Hi,ℓ(xi)−Vi}R_i\leftarrow\min\{V_{t,i}+H_i,\ell(x_i)-V_i\}
end
L←1N∑i=1NRi2L\leftarrow\frac{1}{N}\sum_{i=1}^{N}R_i^2
θ←θ−α∇θL\theta\leftarrow\theta-\alpha\nabla_\theta L
end

Table 1.

\begin{bmatrix} \partial_\theta u^\star\\ \partial_\theta d^\star \end{bmatrix} = -\left[ \begin{array}{cc} \nabla^2_{uu}(p_\theta^{\mathsf T}f) & \nabla^2_{ud}(p_\theta^{\mathsf T}f)\\ \nabla^2_{du}(p_\theta^{\mathsf T}f) & \nabla^2_{dd}(p_\theta^{\mathsf T}f) \end{array} \right]^{-1} \left[ \begin{array}{c} \partial_\theta\nabla_u(p_\theta^{\mathsf T}f)\\ \partial_\theta\nabla_d(p_\theta^{\mathsf T}f) \end{array} \right]\bigg|_{(u^\star,d^\star)}. \tag*{(12)}\end{bmatrix}

Here we assume an interior saddle point for simplicity. While efficient, boundary projections can lead to singular or ill-conditioned Jacobians and subsequent numerical instabilities, especially in large-scale problems, as demonstrated in [25]. Both approaches introduce additional differentiation costs on top of the repeated saddle point solves required during training, rendering them impractical at scale.

In this work, we show that differentiating through the inner optimization is unnecessary. Leveraging a saddle point envelope type argument, we establish exact gradients for neural HJI training without differentiating through the inner solver, which has also been observed in the optimal transport setting Amos [1]; Park et al. [35]. We state the formal theory in § 3.1 and present the resulting end-to-end training algorithm in § 3.2.

Theoretical Analysis

We now establish an envelope type gradient identity that enables neural HJ reachability training without differentiating through the inner saddle point optimization.

Notation. We write z=(u,d)∈U×Dz=(u,d)\in\mathcal{U}\times\mathcal{D}, and use zz and (u,d)(u,d) interchangeably. Let pθ:=∇xVθ(x,t)p_\theta:=\nabla_xV_\theta(x,t) and φ(p,z):=p⊤f(x,z)\varphi(p,z):=p^\top f(x,z), so that H(x,p)=max⁡u∈Umin⁡d∈Dφ(p,u,d)H(x,p)=\max_{u\in\mathcal{U}}\min_{d\in\mathcal{D}}\varphi(p,u,d) by (5).

Assumptions. For an arbitrary parameter θ0\theta_0, we assume: (A1) U\mathcal{U} and D\mathcal{D} are nonempty, compact and convex; (A2) f(x,⋅)f(x,\cdot) is C1C^1; (A3) the mapping θ↦pθ\theta\mapsto p_\theta is differentiable at θ0\theta_0; (A4) there exists δ0=δ(pθ0)>0\delta_0=\delta(p_{\theta_0})>0 such that, for every p∈B0:={p∈Rn:∥p−pθ0∥2<δ0}p\in\mathcal{B}_0:=\{p\in\mathbb{R}^n:\lVert p-p_{\theta_0}\rVert_2<\delta_0\}, the Isaacs condition max⁡u∈Umin⁡d∈Dφ(p,u,d)=min⁡d∈Dmax⁡u∈Uφ(p,u,d)\max_{u\in\mathcal{U}}\min_{d\in\mathcal{D}}\varphi(p,u,d)=\min_{d\in\mathcal{D}}\max_{u\in\mathcal{U}}\varphi(p,u,d) holds; (A5) the function φ(pθ0,⋅,⋅)\varphi(p_{\theta_0},\cdot,\cdot) has a unique saddle point on U×D\mathcal{U}\times\mathcal{D}, which we denote by z⋆(pθ0)z^\star(p_{\theta_0}).

Theorem 1 Under (A1)–(A5), the mapping θ↦H(x,pθ)\theta\mapsto H(x,p_\theta) is differentiable at θ0\theta_0, with

∂θH(x,pθ)∣θ0=f(x,z⋆(pθ0))⊤(∂θpθ)∣θ0+[∇z(pθ⊤f(x,z))]∣z=z⋆(pθ0)⊤(∂θz⋆(pθ))∣θ0\left.\partial_\theta H(x,p_\theta)\right|_{\theta_0} = f(x,z^\star(p_{\theta_0}))^\top\left.(\partial_\theta p_\theta)\right|_{\theta_0} + \left.[\nabla_z(p_\theta^\top f(x,z))]\right|_{z=z^\star(p_{\theta_0})}^\top\left.(\partial_\theta z^\star(p_\theta))\right|_{\theta_0}
=f(x,z⋆(pθ0))⊤(∂θpθ)∣θ0.(13)= f(x,z^\star(p_{\theta_0}))^\top\left.(\partial_\theta p_\theta)\right|_{\theta_0}. \tag*{(13)}

In other words, the extra term that would result from chain rule does not appear. Thus, the saddle point can be treated as a constant during backpropagation, with no gradients propagated through z⋆(pθ)z^\star(p_\theta) or the solver that computes it.

We defer the proof of Theorem 1 to Appendix Section E, numerical validation of Theorem 1 is also provided in Appendix Section C.

Remark. We highlight several important properties of Theorem 1. (i) Assumptions (A1)–(A5) are compatible with standard HJ reachability settings involving compact convex input constraints, sufficiently regular dynamics, and locally well-defined saddle points. In particular, the result does not require strong convexity-concavity or second-order nonsingularity conditions crucial for IFT. (ii) Theorem 1 applies pointwise at any θ0\theta_0 satisfying these assumptions and therefore extends throughout training wherever they continue to hold. We validate this numerically and measure the effect of inexact inner solves in Appendix Section C, enabling our proposed Algorithm 1. (iii) The result applies to arbitrary compact and convex control and disturbance sets, beyond simple box constraints.

Moreover, the gradient identity is independent of the numerical saddle point solver. (iv) Finally, the result does not require differentiability of the optimal control and disturbance mappings with respect to the network parameters. Consequently, the gradient identity remains valid for saddle points in the interior or on the boundary of the admissible sets.

Gradient-Free Training for Neural HJ Reachability with Implicit Hamiltonians

Following the envelope identity established in § 3.1, we develop an end-to-end neural HJ training framework that incorporates numerically evaluated Hamiltonians without differentiating through the inner saddle-point solver. We describe its main implementation components below.

Exact terminal condition enforcement through network parameterization. Following Singh et al., [41], we enforce the terminal condition Vθ(x,T)=ℓ(x)V_\theta(x,T)=\ell(x) by parameterizing the value function as

Vθ(x,t)=e−(T−t)ℓ(x)+(1−e−(T−t))Nθ(x,t),(14)V_\theta(x,t)=e^{-(T-t)}\ell(x)+\left(1-e^{-(T-t)}\right)N_\theta(x,t), \tag*{(14)}

where NθN_\theta denotes the neural network output. Unlike Singh et al., [41], we opt for an exponential decay to reduce the influence of the nonsmooth terminal function ℓ(x)\ell(x) away from TT, which empirically we observe improves training. This parameterization satisfies the terminal condition exactly, eliminating the need for the terminal loss (11).

Saddle point solver. We primarily employ PGDA to solve the inner saddle point problem, using adaptive step sizes based on the Barzilai–Borwein (BB) method Barzilai and Borwein, [4] to improve practical convergence. The step sizes are updated as

ηuk+1=clip⁡(∥uk+1−uk∥22(uk−uk+1)⊤(∇uϕk+1−∇uϕk),ηmin⁡,ηmax⁡),\eta_u^{k+1}=\operatorname{clip}\left(\frac{\left\lVert u^{k+1}-u^k\right\rVert_2^2}{\left(u^k-u^{k+1}\right)^\top\left(\nabla_u\phi^{k+1}-\nabla_u\phi^k\right)},\eta_{\min},\eta_{\max}\right),
ηdk+1=clip⁡(∥dk+1−dk∥22(dk+1−dk)⊤(∇dϕk+1−∇dϕk),ηmin⁡,ηmax⁡).(15)\eta_d^{k+1}=\operatorname{clip}\left(\frac{\left\lVert d^{k+1}-d^k\right\rVert_2^2}{\left(d^{k+1}-d^k\right)^\top\left(\nabla_d\phi^{k+1}-\nabla_d\phi^k\right)},\eta_{\min},\eta_{\max}\right). \tag*{(15)}

Here, ηmin⁡\eta_{\min} and ηmax⁡\eta_{\max} are predefined bounds for numerical stability, and step sizes are updated independently for each sample. This provides a computationally inexpensive adaptive step-size selection, particularly useful when the saddle point objective is relatively flat. We evaluate its effectiveness against a fixed step size solver in Section C. We note that while PGDA convergence depends on the structure of the underlying game, the proposed training framework is independent of the saddle point solver and can accommodate any alternative numerical methods.

Combining the exact terminal-condition parameterization with gradient-free Hamiltonian evaluation yields the end-to-end training procedure summarized in Algorithm 1. After training, the BRT is recovered from the zero sublevel set of VθV_\theta, and the corresponding feedback policy can be obtained using the same saddle point solver. For conciseness, Algorithm 1 is written in BRT formulation; the BRAT extension follows analogously.

Experiments

We consider three numerical experiments, covering both BRT and BRAT problems, with state dimensions from 2 to 50. In all cases, the Hamiltonians are implicitly defined and require iterative solvers to compute the saddle points, and optimal saddle points occur both in the interior and on the

Learned and reference value functions for the energy-constrained pursuit-evasion problem

caption: Learned and reference value functions for the energy-constrained pursuit-evasion problem. Left: Learned VθV_\theta; Middle: grid-based reference; Right: BRT volume differences.

boundaries of the admissible sets, providing sufficient numerical evidence supporting our approach. All experiments were conducted on a single NVIDIA RTX 5090 GPU. Code will be made available upon publication of the work. We refer to Appendix Section B and Section D for additional details.

Energy Constrained Pursuit–Evasion Example

We first consider a two-player zero-sum differential game with state x=(r,q)∈X=[0,2]×[−0.5,1.5]x=(r,q)\in\mathcal{X}=[0,2]\times[-0.5,1.5], where rr denotes the relative distance between the players and qq their relative energy resource, with q>0q>0 indicating an advantage for the evader. The evader and pursuer select controls u,d∈[0,2]u,d\in[0,2], respectively, under the dynamics

r˙=(1+0.2q)tanh⁡(u)−1.2(1−0.1q)tanh⁡(d),q˙=−0.2u2+0.2d2.(16)\begin{aligned} \dot{r} &= (1+0.2q)\tanh(u)-1.2(1-0.1q)\tanh(d),\\ \dot{q} &= -0.2u^2+0.2d^2. \tag*{(16)} \end{aligned}

The target set is defined by L={(r,q):r≤0.25}∪{(r,q):q≤0}\mathcal{L}=\{(r,q):r\leq0.25\}\cup\{(r,q):q\leq0\}, with terminal function ℓ(r,q)=min⁡{r−0.25,q}\ell(r,q)=\min\{r-0.25,q\}, the target set is reached when either the relative distance falls below the given threshold or the evader exhausts its resource advantage.

Given the finite time horizon T=1.5T=1.5, the BRT can be defined as in § 2, the Hamiltonian is separable in uu and dd, allowing their optimal values to be computed independently from their respective stationarity conditions

pr(1+0.2q)sech⁡2(u)−0.4pqu=0,−1.2pr(1−0.1q)sech⁡2(d)+0.4pqd=0,(17)\begin{aligned} p_r(1+0.2q)\operatorname{sech}^2(u)-0.4p_qu &= 0,\\ -1.2p_r(1-0.1q)\operatorname{sech}^2(d)+0.4p_qd &= 0, \tag*{(17)} \end{aligned}

subject to the input constraints, with p=[pr,pq]Tp=[p_r,p_q]^\mathsf{T} and p=∇xV(x,t)p=\nabla_xV(x,t). The Hamiltonian is provably convex-concave over the input domain, and the separable structure enables comparison against a grid-based level-set solver while retaining nontrivial numerical optimization of the optimal inputs.

We refer to Appendix Section D for details on model parameterization and training details. As shown in Figure 1, the learned value function closely matches the grid-based reference solution, achieving a training loss on the order of 10−610^{-6} and a BRT volume error of only 0.11%.

Extending to High Dimensions

We extend the 2D energy-constrained pursuit-evasion example to NN independent evader-pursuer pairs, with joint state x=(r1,q1,…,rN,qN)∈R2Nx = (r_1, q_1, \ldots, r_N, q_N) \in\mathbb{R}^{2N} and dynamics

\begin{cases} \dot{r}_i = (1 + k_i^1 q_i)\tanh(u_i) - 1.2(1 - k_i^2 q_i)\tanh(d_i), \\ \dot{q}_i = -\alpha_i u_i^2 + \beta_i d_i^2, \end{cases} \qquad i = 1,\ldots,N. \tag*{(18)}\end{cases}

The target set is defined by L={x:ℓ(x)≤0}\mathcal{L} = \{x : \ell(x) \le0\}, where ℓ(x)=min⁡i=1,…,N{ri−rireach,qi−qireach}\ell(x) = \min_{i=1,\ldots,N}\{r_i-r_i^{\mathrm{reach}}, q_i-q_i^{\mathrm{reach}}\} so that the target is reached whenever any pair violates its separation or energy threshold.. We refer to Appendix Section B for the complete problem formulation.

We consider 10D and 50D extensions to evaluate scalability of our proposed method. As shown in Figure 5, training converges in both cases, with increasing residual losses at higher dimensions given the added complexity. To assess the learned BRTs without grid-based reference solutions, we perform a closed-loop consistency test on 2048 randomly sampled initial states for each problem, comparing BRT predictions with rollouts under the learned saddle-point policy, similar to that of [2]. The column-normalized confusion matrices in Figure 2 show that agreement remains high, particularly for states predicted to be unreachable (99.1% in 10D and 96.4% in 50D). Agreement for predicted-reachable states decreases to 94.7% and 85.4%, respectively, indicating greater classification disagreement as dimensionality increases.

Closed-loop consistency of learned BRT predictions across state dimensions

Figure 2. Closed-loop consistency of learned BRT predictions across state dimensions, quantifying mismatches between BRT predictions and closed-loop rollouts. Confusion matrices are column-normalized by predicted class.

Training loss curves for the energy-constrained pursuit-evasion problems in 2D, 10D, and 50D

Figure 5. Training loss curves for the energy-constrained pursuit-evasion problems in 2D, 10D, and 50D. The losses decrease and stabilize across all three settings, demonstrating convergence of our proposed neural HJ training procedure. Higher-dimensional problems exhibit larger final residual losses.

Two-Vehicle Collision Avoidance in 6D

We consider a two-vehicle collision avoidance problem based on Bansal and Tomlin [2], with joint state x=(x1,…,x6)∈R4×S1×S1x = (x_1,\ldots,x_6) \in\mathbb{R}^{4} \times\mathbb{S}^{1} \times\mathbb{S}^{1}. Unlike the standard constant speed setting, we introduce steering-dependent speed for the evader, resulting in the modified dynamics

{x˙1=0.75cos⁡x3,x˙4=0.75e−0.4(u/3)2cos⁡x6,x˙2=0.75sin⁡x3,x˙5=0.75e−0.4(u/3)2sin⁡x6,x˙3=d,x˙6=u,\begin{cases} \dot{x}_1 = 0.75\cos x_3, & \dot{x}_4 = 0.75e^{-0.4(u/3)^2}\cos x_6, \\ \dot{x}_2 = 0.75\sin x_3, & \dot{x}_5 = 0.75e^{-0.4(u/3)^2}\sin x_6, \\ \dot{x}_3 = d, & \dot{x}_6 = u, \end{cases}

where u,d∈[−3,3]u,d \in[-3,3] denote the evader’s and pursuer’s turn-rate controls, respectively. This modification introduces nonlinear control dependence into the Hamiltonian, requiring numerical optimization of the evader’s optimal control. We follow the capture target formulation of Bansal and Tomlin [2], with the complete problem definition given in Appendix Section B.

Training yielded a MSE loss of approximately 2×10−32 \times10^{-3}. We note although the model is trained directly in the full 6D state space, a reference solution can be computed in 3D using relative coordinates, exploiting the translation and rotation symmetries of the dynamics. As shown in Figure 3, the learned BRT closely matches a grid-based reference solution across multiple two-dimensional slices, with slightly conservative predictions. Importantly, we obtain the desired results without enforcing said symmetries during neural HJ training.

Learned and reference BRTs across relative-heading slices

Figure 3. Learned and reference BRTs across relative-heading slices, showing close agreement with a grid-based reference solution.

We further evaluate the learned value function through closed-loop trajectory rollouts in Figure 6 in the Appendix, using the same numerical Hamiltonian optimization as during training. The trajectories demonstrate both successful evasion and capture, with the inner solver achieving residual errors below 10−410^{-4} across the tested scenarios. The steering-dependent speed also changes the reachable set compared with the constant speed setting: by reducing its speed during rapid turning, the evader can escape from certain configurations that would otherwise lead to capture. The trajectories also illustrate how initially reachable states may leave the BRT over time.

Closed-loop trajectories for the two-vehicle collision avoidance problem

Figure 6. Closed-loop trajectories for the two-vehicle collision avoidance problem, using feedback controls obtained from the learned value function and online Hamiltonian optimization. Left & Right: successful evasion; Middle: capture. The learned value function is overlaid along each trajectory to illustrate its temporal evolution.

Quadrotor Reach-Avoid Control under Adversarial Disturbances

We consider a 6D planar quadrotor reach-avoid problem under adversarial wind disturbances. The quadrotor is controlled through two bounded rotor thrusts and must reach a prescribed target region while avoiding ground collision and excessive pitch angles. The wind disturbance enters through nonlinear velocity-dependent drag, requiring numerical optimization to determine the worst case disturbance. We refer to Appendix Section B for the complete dynamics and problem formulation.

To address training difficulties arising from the nested BRAT variational inequality, we adopt a curriculum learning strategy common in PINNs literature [2, 20, 44], where we gradually extend the sampled time horizon from TT to 00 during training. We evaluate the learned value function through a closed-loop consistency test on 2048 randomly sampled initial states at t=0t = 0. As shown in Table 1, the rollout outcomes agree with the predicted reach-avoid classifications in over 90% of cases for both predicted classes, with most discrepancies occurring near the zero level set. We further visualize representative trajectories in Figure 4, where feedback controls and worst-case disturbances derived from the learned value function produce outcomes consistent with the predicted BRAT.

Rollouts for quadrotor control under disturbances

Figure 4. Rollouts for quadrotor control under disturbances using the learned VθV_\theta. Successful and failed trajectories are shown, with outcomes consistent with the BRAT predictions.

ReachedCrashedTimeoutTotal
Vθ(x,0)≤0V_\theta(x,0) \le0All837 (90.8%)71 (7.7%)14 (1.5%)922
Vθ(x,0)≤0V_\theta(x,0) \le0Boundary4644797
Vθ(x,0)≤0V_\theta(x,0) \le0Confident791277825
Vθ(x,0)>0V_\theta(x,0) > 0All62 (5.5%)911 (80.9%)153 (13.6%)1126
Vθ(x,0)>0V_\theta(x,0) > 0Boundary2453481
Vθ(x,0)>0V_\theta(x,0) > 0Confident388581491045

Table 1. Closed-loop consistency of BRAT predictions using learned value function. “Boundary” denotes ∣Vθ(x,0)∣<0.05|V_\theta(x,0)| < 0.05. Both crashed and timeout cases have Vθ(x,0)>0V_\theta(x,0) > 0.

Conclusion

We presented an end-to-end neural HJ reachability framework for nonlinear systems with implicitly defined Hamiltonians. Using a perturbation argument based on saddle-point inequalities, we prove that gradients can be computed without differentiating through the inner optimization, avoiding the overhead and numerical instability under solver unrolling or implicit differentiation. Experiments on both reachability and reach-avoid problems demonstrate the effectiveness and scalability of our approach, extending neural HJ methods to a much broader and more practical class of problems.

Related Work

We provide a review of the literature on neural HJ reachability and subsequent works that extend these methods to more complex and challenging problems. Additionally, we provide an overview of recent advances in methods for handling implicit Hamiltonians.

High-dimensional neural HJ reachability. DeepReach [2] introduced a mesh-free neural approach for solving HJ reachability problems in dimensions beyond the range of conventional grid-based solvers. Rather than discretizing the state space, DeepReach directly parameterizes the value function and minimizes the HJI variational inequality residual. Subsequent approaches have improved the scalability and training of neural HJ solvers. Sharpless et al. [40] augment the HJI loss with supervision generated from tractable linearized differential games and demonstrate neural value-function learning for systems with tens of state dimensions. Similarly, Feng et al. [8] and MADR [42] leverage MPC-based supervision to improve neural HJ reachability learning. These methods train the HJ value function end-to-end, but assume that the Hamiltonian required by the HJI residual can be evaluated efficiently. On the theoretical side, Hofgard [16] establishes conditional convergence guarantees for DeepReach, while Lin and Bansal [26] provide formal safety assurances through error bounds and reachable-tube corrections. Related theoretical work also studies neural approximation of HJ-type PDEs [38].

HJ reachability without closed-form Hamiltonians. Several recent works address HJ reachability problems in which the Hamiltonian is difficult to evaluate or unavailable in closed form. (Chilakamarri et al., 2025) learn a neural Hamiltonian surrogate from black-box dynamics samples for subsequent use in HJ solvers. [7] construct a conservative data-driven Hamiltonian directly from trajectory data, while [27] extend HJ-based verification to complex perception-based controllers under state uncertainty. These approaches make a difficult Hamiltonian tractable through approximation or bounding before it is used by the value function solver. In contrast, we retain the numerical optimization defining the Hamiltonian as an explicit component of neural HJI training and differentiate the value-function objective through its solution.

Implicit Hamiltonians in Control. Implicit optimization layers have been studied broadly in machine learning and differentiable optimization, with Jacobian-Free Backpropagation (JFB), which provides an inexpensive alternative to exact implicit differentiation [10, 15, 14, 31 b]. In control, [25] combine operator splitting with JFB to enable scalable end-to-end training through embedded CBF-QP safety layers, which demonstrates that Jacobian-free differentiation can substantially reduce the computational cost of repeatedly differentiating through optimization problems along control trajectories. More closely related to our setting, [11] develop an end-to-end method for high-dimensional optimal control in which the Hamiltonian minimization is solved implicitly and differentiated using JFB. Subsequent work establishes convergence guarantees for the resulting approximate-gradient training procedure [12]. Imitation-based function-encoder policies instead sidestep the Hamiltonian, transferring zero-shot across tasks [24]. As shown in § 3.1, the particular structure of the HJ reachability problem allows for exact gradient evaluation of the training objective without requiring approximate differentiation techniques, providing a more direct and natural alternative to existing approaches for handling implicit Hamiltonians.

Additional Experimental Details

This appendix provides supplementary problem formulations, Hamiltonian derivations, and implementation details for the numerical experiments presented in § 4.

High-Dimensional Energy-Constrained Pursuit–Evasion

We extend the two-player pursuit–evasion problem in § 4 to NN independent evader-pursuer pairs, with joint state x=(r1,q1,…,rN,qN)∈X⊂R2Nx=(r_1,q_1,\ldots,r_N,q_N)\in\mathcal{X}\subset\mathbb{R}^{2N}. For each pair ii, rir_i and qiq_i denote the relative distance and energy resource, respectively, with controls ui∈[0,uˉi]u_i\in[0,\bar{u}_i] and di∈[0,dˉi]d_i\in[0,\bar{d}_i]. The dynamics are

\begin{cases} \dot{r}_i=(1+k_i^1q_i)\tanh(u_i)-1.2(1-k_i^2q_i)\tanh(d_i),\\ \dot{q}_i=-\alpha_i u_i^2+\beta_i d_i^2, \end{cases} \qquad i=1,\ldots,N, \tag*{(19)}\end{cases}

where ki1,ki2,αi,βi>0k_i^1,k_i^2,\alpha_i,\beta_i>0 may vary across pairs to model heterogeneous maneuverability and energy consumption rate. The target set consists of states in which at least one pair violates its separation or energy threshold, and is represented by the terminal function

ℓ(x)=min⁡i=1,…,N{ri−rireach,qi−qireach}.(20)\ell(x)=\min_{i=1,\ldots,N}\left\{r_i-r_i^{\mathrm{reach}},q_i-q_i^{\mathrm{reach}}\right\}. \tag*{(20)}

The corresponding target set is

L=⋃i=1N({ri≤rireach}∪{qi≤qireach}).\mathcal{L}=\bigcup_{i=1}^{N}\left(\{r_i\le r_i^{\mathrm{reach}}\}\cup\{q_i\le q_i^{\mathrm{reach}}\}\right).

Since the individual evader–pursuer pairs are dynamically decoupled, the Hamiltonian separates as H(x,p)=∑i=1NHi(xi,pi)H(x,p)=\sum_{i=1}^{N}H_i(x_i,p_i), where xi=(ri,qi)x_i=(r_i,q_i) and pi=[pri,pqi]Tp_i=[p_{r_i},p_{q_i}]^{\mathsf{T}}. Thus, the optimal control and disturbance can be computed independently for each pair using the same numerical optimization procedure as in the 2D case. We also note that under suitable ki1k_i^1 and ki2k_i^2 we can preserve the convex-concave structure of the Hamiltonian, satisfying the necessary assumptions for the theory.

As the problem dimension increases, the reachable set occupies a larger portion of the state space, leading to an imbalance between reachable and unreachable states under uniform sampling. To address this imbalance and stabilize training, we employ importance sampling to ensure that decreasing training loss leads to improved closed-loop performance.

Two-Vehicle Collision Avoidance

We consider a finite-horizon collision avoidance problem between two planar vehicles. The joint state is x=(x1,…,x6)∈R4×S1×S1x=(x_1,\ldots,x_6)\in\mathbb{R}^{4}\times\mathbb{S}^{1}\times\mathbb{S}^{1}, where (x1,x2,x3)(x_1,x_2,x_3) and (x4,x5,x6)(x_4,x_5,x_6) denote the positions and headings of the pursuer and evader, respectively. We modify the standard constant-speed dynamics [2] by introducing steering dependent speed for the evader:

{x˙1=0.75cos⁡x3,x˙4=0.75e−0.4(u/3)2cos⁡x6,x˙2=0.75sin⁡x3,x˙5=0.75e−0.4(u/3)2sin⁡x6,x˙3=d,x˙6=u,(21)\begin{cases} \dot{x}_1=0.75\cos x_3,\qquad\dot{x}_4=0.75e^{-0.4(u/3)^2}\cos x_6,\\ \dot{x}_2=0.75\sin x_3,\qquad\dot{x}_5=0.75e^{-0.4(u/3)^2}\sin x_6,\\ \dot{x}_3=d,\qquad\dot{x}_6=u, \tag*{(21)} \end{cases}

where u,d∈[−3,3]u,d \in[-3,3] denote the evader’s and pursuer’s turn-rate controls, respectively. The evader’s speed decreases with the magnitude of its turn rate, introducing nonlinear control dependence into the Hamiltonian. For the time horizon T=1T=1, the capture target is defined by

ℓ(x)=(x1−x4)2+(x2−x5)2−0.25,(22)\ell(x) = \sqrt{(x_1-x_4)^2+(x_2-x_5)^2}-0.25, \tag*{(22)}

with L={x:ℓ(x)≤0}\mathcal{L} = \{x:\ell(x)\leq0\}. The corresponding Hamiltonian is

H(x,p)=0.75(p1cos⁡x3+p2sin⁡x3)−3∣p3∣+max⁡u∈[−3,3][0.75e−0.4(u/3)2(p4cos⁡x6+p5sin⁡x6)+p6u],(23)H(x,p)=0.75(p_1\cos x_3+p_2\sin x_3)-3|p_3| +\max_{u\in[-3,3]}\left[0.75e^{-0.4(u/3)^2}(p_4\cos x_6+p_5\sin x_6)+p_6u\right], \tag*{(23)}

where p=∇xV(x,t)p=\nabla_xV(x,t). While the pursuer’s optimal turn rate admits a closed-form solution, the evader’s optimal control is determined through an inner optimization problem due to the nonlinear dependence of the Hamiltonian on its turn rate.

We further observe that the curvature of the evader’s maximization objective is determined entirely by the sign of the term (p4cos⁡x6+p5sin⁡x6)(p_4\cos x_6+p_5\sin x_6). Specifically, the objective is strongly concave for p4cos⁡x6+p5sin⁡x6>0p_4\cos x_6+p_5\sin x_6>0 and strongly convex for p4cos⁡x6+p5sin⁡x6<0p_4\cos x_6+p_5\sin x_6<0 over the admissible control set. Consequently, the global maximizer either lies on the domain boundary or can be provably recovered through methods such as projected gradient ascent. Finally, we use the computational domain

X=[−1.5,1.5]2×[−π,π)×[−1.5,1.5]2×[−π,π),\mathcal{X}=[-1.5,1.5]^2\times[-\pi,\pi)\times[-1.5,1.5]^2\times[-\pi,\pi),

with state-time samples drawn uniformly during training.

Quadrotor Reach-Avoid Control under Adversarial Disturbances

We consider a six-dimensional planar quadrotor subject to adversarial wind disturbances, with dynamics and constraints specified below. Let x=(x1,x2,x3,x4,x5,x6)∈R6x=(x_1,x_2,x_3,x_4,x_5,x_6)\in\mathbb{R}^6 denote the quadrotor’s position, velocity, pitch angle and pitch rate, respectively. The two thrusters of the rotor can be controlled directly via u∈U=[0,uˉ]2u\in U=[0,\bar{u}]^2, the adversary chooses wind velocity d∈D={d∈R2:d\in D=\{d\in\mathbb{R}^2: ∥d∥2≤dˉ\lVert d\rVert_2 \le\bar{d}. The quadrotor operates under standard dynamics as in [29, 39], which are given by

{x˙1=x3,x˙2=x4,x˙3=−(u1+u2)sin⁡x5−k(x,d)(x3−d1)m,x˙4=(u1+u2)cos⁡x5−k(x,d)(x4−d2)m−g0,x˙5=x6,x˙6=LI(u2−u1).\left\{ \begin{aligned} \dot{x}_1 &= x_3,\\ \dot{x}_2 &= x_4,\\ \dot{x}_3 &= \frac{-(u_1+u_2)\sin x_5-k(x,d)(x_3-d_1)}{m},\\ \dot{x}_4 &= \frac{(u_1+u_2)\cos x_5-k(x,d)(x_4-d_2)}{m}-g_0,\\ \dot{x}_5 &= x_6,\\ \dot{x}_6 &= \frac{L}{I}(u_2-u_1). \end{aligned} \right.

Here m=1m=1 kg is the mass, I=0.05I=0.05 kg m2^2 is pitch inertia, L=0.2L=0.2 m is the rotor’s arm length, we use g0=9.81g_0=9.81 m/s2^2 to denote gravity. We also set the maximum thrust to uˉ=7\bar{u}=7 N and the maximum wind speed to dˉ=3\bar{d}=3 m/s. While the rotor dynamics are affine in control, the wind disturbance enters via a nonlinear drag force which we define here as

k(x,d)=c1+c2(x3−d1)2+(x4−d2)2.k(x,d)=c_1+c_2\sqrt{(x_3-d_1)^2+(x_4-d_2)^2}.

Here the force model combines linear and quadratic drag based on relative air velocity, while not the highest fidelity aerodynamics model, it is still commonly used for simulation and analysis ([13, 22]). For our example we consider coefficients c1=0.5c_1=0.5 and c2=0.05c_2=0.05. The terminal time is set to T=2T=2s. For simplicity, we consider a box region over which we sample and solve the resulting BRAT problem, which we define as Ω=[−2,2]×[−0.5,4]×[−3,3]×[−3,3]×[−1.2,1.2]×[−4,4]\Omega=[-2,2]\times[-0.5,4]\times[-3,3]\times[-3,3]\times[-1.2,1.2]\times[-4,4].

The reach target of the problem is a boxed region over x1x_1 and x2x_2 at [0,1]×[2,3][0,1] \times[2,3], we define the reach and avoid set functions as

ℓ(x)=max⁡{−x1,x1−1,2−x2,x2−3},g(x)=max⁡{−x2,∣x5∣−π3},\begin{aligned} \ell(x) &= \max\{-x_1, x_1-1, 2-x_2, x_2-3\},\\ g(x) &= \max\{-x_2, |x_5|-\frac{\pi}{3}\}, \end{aligned}

and subsequently the target and unsafe sets as

L={x:ℓ(x)≤0},G={x:g(x)>0}\mathcal{L} = \{x : \ell(x) \le0\}, \qquad\mathcal{G} = \{x : g(x) > 0\}

following the conventions established in § 2. We note that the unsafe set encodes two failure scenarios: the first occurs when the quadrotor reaches the ground, and the second occurs when the pitch angle exceeds the prescribed safety limit, potentially leading to a loss of stability and a drone flip-over. We place no direct target restrictions on x3,x4,x5,x6x_3, x_4, x_5, x_6, although a successful arrival at the target must nevertheless satisfy g(x)≤0g(x) \le0. The optimal wind disturbance is determined by the constrained optimization problem

d⋆∈arg max⁡∥d∥2≤dˉ−k(x,d)m[p3(x3−d1)+p4(x4−d2)].d^{\star} \in\underset{\lVert d\rVert_2 \le\bar{d}}{\operatorname{arg\,max}} -\frac{k(x,d)}{m}\left[p_3(x_3-d_1)+p_4(x_4-d_2)\right].

Under our choice of parameters, the maximization admits a unique solution for every state and adjoint, though obtaining d⋆d^{\star} always requires an iterative solver, making this a challenging yet suitable testbed for our proposed approach.

Additional Numerical Results

Numerical Verification of The Theory

We provide numerical verification of our main theoretical results § 3.1 here in this section. We mainly consider the the 2D energy constrained pursuit–evasion problem as the example to focus on. Our focus is to validate the accuracy of the training gradient ∇θL\nabla_{\theta}\mathcal{L} under our proposed approach in saddle point optimization. For simplicity we apply kk steps of projected gradient ascent/descent with fixed step size of 0.1. Early stopping is disabled to ensure a fair comparison. We compare two different scenarios:

  • Gradient Free (ours): the inner optimization is untracked and only the obtained saddle point is supplied to the loss function.

  • Automatic Differentiation (AD): each inner optimization step is fully tracked and unrolled during backpropagation.

For value function construction we consider two different parameterizations: (i) a polynomial parameterization for the value function with sensitivity under control, and (ii) a neural network with the same size used in our numerical testing. For both cases we initialize the value functions so that the adjoints remain strictly positive,thus ensuring the min-max structure and a strictly interior saddle point. for each testing case, we sample with batch size 1024 uniformly over the domain. We report gradient consistency results for both test cases under different solver iterations k∈{1,2,5,10,…,500}k \in\{1,2,5,10,\ldots,500\} and display the results in Table 2. The gradient accuracy over the trainable weights is governed by the accuracy of the inner solve, and vanishes with it. As kk increases, the Hamiltonian residuals ∥∂uφ∥∞\lVert\partial_u\varphi\rVert_\infty and ∥∂dφ∥∞\lVert\partial_d\varphi\rVert_\infty decrease, and ∥Δ∇θL∥\lVert\Delta\nabla_\theta\mathcal{L}\rVert decreases with them. For the polynomial VθV_\theta the solve converges to machine precision by k≈200k \approx200, and the two gradients coincide exactly at k=500k = 500. The neural network converges more slowly because under random initialization its adjoint pθp_\theta is small and the Hamiltonian is only weakly curved in the controls. Nevertheless the gradient discrepancies are negligible at k=500k = 500 for both cases especially regarding the O(1)O(1) gradient norms. For cases where the saddle points lie on the boundaries, the results follow trivially. Specifically, when evaluated at the exact boundary, the gradient difference is exactly 0, since no fixed-point error needs to be accounted for. Overall the numerical results fully verifies our theoretical claims.

kkPolynomial VθV_\thetaNeural Network VθV_\theta
∥Δ∇θL∥\lVert\Delta\nabla_\theta\mathcal{L}\rVert∥∂uφ∥∞\lVert\partial_u\varphi\rVert_\infty∥∂dφ∥∞\lVert\partial_d\varphi\rVert_\infty∥Δ∇θL∥\lVert\Delta\nabla_\theta\mathcal{L}\rVert∥∂uφ∥∞\lVert\partial_u\varphi\rVert_\infty∥∂dφ∥∞\lVert\partial_d\varphi\rVert_\infty
12.4×10−32.4 \times10^{-3}2.7×10−12.7 \times10^{-1}1.2×10−11.2 \times10^{-1}2.0×10−42.0 \times10^{-4}4.7×10−24.7 \times10^{-2}4.2×10−24.2 \times10^{-2}
23.9×10−33.9 \times10^{-3}2.1×10−12.1 \times10^{-1}1.1×10−11.1 \times10^{-1}3.9×10−43.9 \times10^{-4}4.7×10−24.7 \times10^{-2}4.2×10−24.2 \times10^{-2}
55.2×10−35.2 \times10^{-3}1.5×10−11.5 \times10^{-1}6.8×10−26.8 \times10^{-2}9.1×10−49.1 \times10^{-4}4.5×10−24.5 \times10^{-2}4.0×10−24.0 \times10^{-2}
103.7×10−33.7 \times10^{-3}7.8×10−27.8 \times10^{-2}3.2×10−23.2 \times10^{-2}1.6×10−31.6 \times10^{-3}4.2×10−24.2 \times10^{-2}3.8×10−23.8 \times10^{-2}
201.0×10−31.0 \times10^{-3}2.2×10−22.2 \times10^{-2}7.3×10−37.3 \times10^{-3}2.6×10−32.6 \times10^{-3}3.6×10−23.6 \times10^{-2}3.3×10−23.3 \times10^{-2}
501.2×10−51.2 \times10^{-5}4.4×10−44.4 \times10^{-4}8.2×10−58.2 \times10^{-5}3.3×10−33.3 \times10^{-3}2.4×10−22.4 \times10^{-2}2.3×10−22.3 \times10^{-2}
1008.7×10−98.7 \times10^{-9}6.7×10−76.7 \times10^{-7}4.6×10−84.6 \times10^{-8}2.2×10−32.2 \times10^{-3}1.3×10−21.3 \times10^{-2}1.4×10−21.4 \times10^{-2}
2007.9×10−157.9 \times10^{-15}1.6×10−121.6 \times10^{-12}1.5×10−141.5 \times10^{-14}5.5×10−45.5 \times10^{-4}4.5×10−34.5 \times10^{-3}5.4×10−35.4 \times10^{-3}
50001.1×10−151.1 \times10^{-15}1.1×10−151.1 \times10^{-15}7.4×10−67.4 \times10^{-6}2.1×10−42.1 \times10^{-4}3.1×10−43.1 \times10^{-4}

Table 2. Gradient consistency test under different value function parameterizations and solver accuracies. Here we define Δ∇θL:=∇θLad−∇θLours\Delta\nabla_\theta\mathcal{L} := \nabla_\theta\mathcal{L}_{\mathrm{ad}} - \nabla_\theta\mathcal{L}_{\mathrm{ours}}. All terms are in 2-norm unless otherwise specified. Gradient norm for the polynomial case is ∥∇θL∥=14.0\lVert\nabla_\theta\mathcal{L}\rVert= 14.0 and for the neural network case ∥∇θL∥=6.7\lVert\nabla_\theta\mathcal{L}\rVert= 6.7.

Inner Solver Comparison

The inner saddle point problem (5) is solved approximately at every training step. While our theory in § 3.1 ensures gradient consistency at the fixed point, the quality of the truncated solve can still affect training. We hereby conduct an ablation study and compare between fixed and adaptive step size selection in the saddle point optimization under the same gradient descent rule. We test on the 10D energy constrained pursuit–evasion example, where both optimizers are given the same budget of 200 iterations. Two copies of the value network are trained under exactly the same sampling and with identical weight initialization; hence, any difference in training is due to the inner solver alone. We also keep a reference solution for the fixed point solver, which uses PGDA with a fixed step size of 0.1 and 5000 iterations. This achieves a median fixed point error of 1.6×10−61.6 \times10^{-6} at each sampling mini-batch and serves as a ground truth reference when comparing gradient accuracies. We run a total of 10000 training steps for the test.

We present the results in Table 3. Under the same iteration budget, adaptive step sizing yields a substantially more accurate inner solve than a carefully selected fixed step size. Importantly, the computational overhead is marginal, allowing us to maintain the fast training required for neural HJ reachability at large. However, we note that despite the lower solver accuracy, using plain PGDA

% of stationary samples*Hamiltonian gap ↓\downarrowGradient alignment ↑\uparrowFinal Training loss ↓\downarrowPer training step solver time ↓\downarrow
Fixed η=0.1\eta= 0.122%4.8%0.9658.55×10−48.55 \times10^{-4}0.214s
Adaptive η\eta98.9%0.4%0.9998.30×10−48.30 \times10^{-4}0.247s

Table 3. Inner solver accuracy comparison between fixed and adaptive step size selection. Hamiltonian gap and gradient alignment measure relative value and gradient errors against the reference solution. Numbers averaged across 10K training steps except for the final loss.

with a fixed step size does not lead to significant errors in either the gradient updates or the overall training loss behavior. We attribute this to the stochasticity of Adam and mini-batch training in general. The key takeaway we want to highlight is that convergence of the value function can still be expected even when the inner solve admits a small error.

Training Setup and Details on Numerical Experiments

We provide details on the training setup used in this paper. We consider a total of five different experiments. In all examples the value function VθV_\theta is defined per (14), with NθN_\theta being a residual MLP with tanh activations and terminal condition enforced exactly by construction. Training only minimizes the VI residual at sampled locations. For all examples except for the quadrotor, we employ a uniform sampling scheme over the state-time space. We train all networks with Adam optimizer [18] with an initial learning rate of 10−310^{-3}. The learning rate is halved whenever training hits a plateau, the patience for which we set differently for each example. The default solver for the inner optimization is PGDA with Barzilai–Borwein adaptive step sizing, as introduced in § 3 with initial step size 0.10.1. We terminate early whenever the fixed-point residual reaches 1×10−41 \times10^{-4}. The maximum number of iterations for the inner optimization is set separately for each individual example. We try to balance between solver accuracy and speed during training. All code is implemented in PyTorch [36], and all runs use random seed 0; problem-specific settings are listed in Table 4.

ExampleTTTraining StepsBatch SizeNetwork Size (Width, # of Layers)LR patienceInner Iters.
Energy Constrained Pursuit–Evasion (2D)1.530k256(128, 3)2000100
Energy Constrained Pursuit–Evasion (10D)1.550k256(128, 4)3000200
Energy Constrained Pursuit–Evasion (50D)1.5100k256(256, 4)5000500
Two-Vehicle Collision Avoidance1.050k1024(128, 4)3000100
Planar Quadrotor (BRAT)2.070k1024(256, 4)300010∗10^{*}

Table 4. Problem-specific training settings. Inner iterations are the maximum number of iterations of the inner solver.

Proof of Theorem

For completeness, we restate the assumptions and main theorem from § 3.1 and provide its detailed proof, together with the supporting lemmas.

Notations. Write z=(u,d)∈U×Dz = (u,d) \in U \times D. We denote

pθ:=∇xVθ(x,t),(24)p_\theta:= \nabla_x V_\theta(x,t), \tag*{(24)}
φ(p,u,d)=φ(p,z):=p⊤f(x,u,d)=p⊤f(x,z),(25)\varphi(p,u,d) = \varphi(p,z) := p^\top f(x,u,d) = p^\top f(x,z), \tag*{(25)}

where we slightly abuse notations, and used zz and (u,d)(u,d) interchangeably, and thus

H(x,p)=max⁡u∈Umin⁡d∈Dp⊤f(x,u,d)=max⁡u∈Umin⁡d∈Dφ(p,u,d).H(x,p) = \max_{u \in U} \min_{d \in D} p^\top f(x,u,d) = \max_{u \in U} \min_{d \in D} \varphi(p,u,d).

Assumptions. For an arbitrary parameter θ0\theta_0, we assume the following:

(A1) U\mathcal{U} and D\mathcal{D} are nonempty, compact and convex;

(A2) f(x,⋅)f(x,\cdot) is C1C^1;

(A3) the mapping θ↦pθ\theta\mapsto p_\theta is differentiable at θ0\theta_0;

(A4) there exists δ0=δ(pθ0)>0\delta_0=\delta(p_{\theta_0})>0 such that, for every p∈B0:={p∈Rn:∥p−pθ0∥2<δ0}p\in\mathcal{B}_0:=\{p\in\mathbb{R}^n:\|p-p_{\theta_0}\|_2<\delta_0\}, the Isaacs condition

max⁡u∈Umin⁡d∈Dφ(p,u,d)=min⁡d∈Dmax⁡u∈Uφ(p,u,d)\max_{u\in\mathcal{U}}\min_{d\in\mathcal{D}}\varphi(p,u,d)=\min_{d\in\mathcal{D}}\max_{u\in\mathcal{U}}\varphi(p,u,d)

holds;

(A5) the function φ(pθ0,⋅,⋅)\varphi(p_{\theta_0},\cdot,\cdot) has a unique saddle point on U×D\mathcal{U}\times\mathcal{D}, which we denote by z⋆(pθ0)z^\star(p_{\theta_0}).

Theorem 1 Under (A1)–(A5), the mapping θ↦H(x,pθ)\theta\mapsto H(x,p_\theta) is differentiable at θ0\theta_0, with

∂θH(x,pθ)∣θ0=f(x,z⋆(pθ0))⊤(∂θpθ)∣θ0+[∇z(pθ⊤f(x,z))∣z=z⋆(pθ0)]⊤(∂θz⋆(pθ))∣θ0⏟0 contribution(26)\left.\partial_\theta H(x,p_\theta)\right|_{\theta_0} = f\left(x,z^\star(p_{\theta_0})\right)^\top\left.\left(\partial_\theta p_\theta\right)\right|_{\theta_0} + \underbrace{\left[\left.\nabla_z\left(p_\theta^\top f(x,z)\right)\right|_{z=z^\star(p_{\theta_0})}\right]^\top\left.\left(\partial_\theta z^\star(p_\theta)\right)\right|_{\theta_0}}_{0\ \text{contribution}} \tag*{(26)}
=f(x,z⋆(pθ0))⊤(∂θpθ)∣θ0.= f\left(x,z^\star(p_{\theta_0})\right)^\top\left.\left(\partial_\theta p_\theta\right)\right|_{\theta_0}.

In other words, the extra term that would result from chain rule does not appear. Thus, the saddle point can be treated as a constant during backpropagation, with no gradients propagated through z⋆(pθ)z^\star(p_\theta) or the solver that computes it.

Proof sketch: The key idea of our proof is to avoid differentiating through the saddle point solution map, which may be nondifferentiable. Instead, we directly analyze perturbations of the Hamiltonian using saddle point inequalities. Together with compactness and uniqueness, these bounds establish differentiability of the Hamiltonian and yield the desired gradient identity through the chain rule.

Proof. Let S(p)S(p) denote the set of saddle points at pp, that is,

S(p):={z=(u,d)∈U×D:φ(p,u~,d)≤φ(p,u,d)≤φ(p,u,d~)∀u~∈U, d~∈D}.(27)S(p) := \left\{z=(u,d)\in\mathcal{U}\times\mathcal{D}:\varphi(p,\tilde{u},d)\leq\varphi(p,u,d)\leq\varphi(p,u,\tilde{d})\quad\forall\tilde{u}\in\mathcal{U},\ \tilde{d}\in\mathcal{D}\right\}. \tag*{(27)}

Step 1: Existence of Saddle Point. By (A4), for every p∈B0:={p∈Rn:∥p−pθ0∥2<δ0}p\in B_0:=\{p\in\mathbb{R}^n:\|p-p_{\theta_0}\|_2<\delta_0\}, the Isaacs condition

max⁡u∈Umin⁡d∈Dφ(p,u,d)=min⁡d∈Dmax⁡u∈Uφ(p,u,d)=:v(28)\max_{u\in\mathcal{U}}\min_{d\in\mathcal{D}}\varphi(p,u,d)=\min_{d\in\mathcal{D}}\max_{u\in\mathcal{U}}\varphi(p,u,d)=:v \tag*{(28)}

holds. Here, the extrema are attained by compactness and continuity. Let u⋆u^\star attain the outer maximum on the left and d⋆d^\star the outer minimum on the right. On one hand,

φ(p,u⋆,d⋆)≥min⁡d∈Dφ(p,u⋆,d)=v,\varphi(p,u^\star,d^\star)\geq\min_{d\in\mathcal{D}}\varphi(p,u^\star,d)=v,

where in the first step, we used the property of minimum, and in the second step, we used the definition of vv in (28). On the other hand,

φ(p,u⋆,d⋆)≤max⁡u∈Uφ(p,u,d⋆)=v,\varphi(p,u^\star,d^\star)\leq\max_{u\in\mathcal{U}}\varphi(p,u,d^\star)=v,

using the same argument. Combining these two inequalities, we have

φ(p,u⋆,d⋆)=v.(29)\varphi(p,u^\star,d^\star)=v. \tag*{(29)}

Next, we argue that (u⋆,d⋆)(u^\star,d^\star) is a saddle point. On one hand, we have, for every u~∈U\tilde{u}\in\mathcal{U},

φ(p,u~,d⋆)≤max⁡u∈Uφ(p,u,d⋆)=v=φ(p,u⋆,d⋆),(30)\varphi(p,\tilde{u},d^\star)\leq\max_{u\in\mathcal{U}}\varphi(p,u,d^\star)=v=\varphi(p,u^\star,d^\star), \tag*{(30)}

by the definition of d⋆d^\star. On the other hand, similarly, for every d~∈D\tilde{d}\in\mathcal{D},

φ(p,u⋆,d~)≥min⁡d∈Dφ(p,u⋆,d)=v=φ(p,u⋆,d⋆),(31)\varphi(p,u^\star,\tilde{d})\geq\min_{d\in\mathcal{D}}\varphi(p,u^\star,d)=v=\varphi(p,u^\star,d^\star), \tag*{(31)}

by the definition of u⋆u^\star. Together, (30) and (31) establish that (u⋆,d⋆)(u^\star,d^\star) is a saddle point, that is (u⋆,d⋆)∈S(p)(u^\star,d^\star)\in S(p). Thus, S(p)≠∅S(p)\neq\varnothing for every p∈B0p\in B_0, which is the existence of saddle point. Moreover, by the definition of the Hamiltonian HH, (29) becomes

H(x,p)=φ(p,u⋆,d⋆).(32)H(x,p)=\varphi(p,u^\star,d^\star). \tag*{(32)}

Step 2: Linearization Error Bound. We bound the error of linearizing HH around pθ0p_{\theta_0}. Given any fixed p′∈B0p'\in B_0, write (u0,d0):=z⋆(pθ0)(u_0,d_0):=z^\star(p_{\theta_0}) and let (u′,d′)∈S(p′)(u',d')\in S(p') be any saddle point at p′p' (which exists by Step 1).

Upper Bound. Using (SV) of Lemma 2, we have

H(x,p′)≤φ(p′,u′,d0),H(x,p')\leq\varphi(p',u',d_0),
H(x,pθ0)≥φ(pθ0,u′,d0).H(x,p_{\theta_0})\geq\varphi(p_{\theta_0},u',d_0).

Subtracting these two inequalities, we have

H(x,p′)−H(x,pθ0)≤(p′−pθ0)⊤f(x,u′,d0).(33)H(x,p')-H(x,p_{\theta_0})\leq(p'-p_{\theta_0})^\top f(x,u',d_0). \tag*{(33)}

Lower Bound. Similarly, using (SV) of Lemma 2, we have

H(x,p′)≥φ(p′,u0,d′),H(x,pθ0)≤φ(pθ0,u0,d′).H(x,p') \ge\varphi(p',u_0,d'), \qquad H(x,p_{\theta_0}) \le\varphi(p_{\theta_0},u_0,d').

Subtracting these two inequalities, we have

H(x,p′)−H(x,pθ0)≥(p′−pθ0)Tf(x,u0,d′).(34)H(x,p')-H(x,p_{\theta_0}) \ge(p'-p_{\theta_0})^{\mathsf T}f(x,u_0,d'). \tag*{(34)}

Error Bound. Let the linearization error be

R(p′):=H(x,p′)−H(x,pθ0)−(p′−pθ0)Tf(x,u0,d0).(35)R(p') := H(x,p')-H(x,p_{\theta_0})-(p'-p_{\theta_0})^{\mathsf T}f(x,u_0,d_0). \tag*{(35)}

Plugging (33) and (34) into (35), we have

(p′−pθ0)T(f(x,u0,d′)−f(x,u0,d0))≤R(p′)≤(p′−pθ0)T(f(x,u′,d0)−f(x,u0,d0)).(36)(p'-p_{\theta_0})^{\mathsf T}\bigl(f(x,u_0,d')-f(x,u_0,d_0)\bigr) \le R(p') \le(p'-p_{\theta_0})^{\mathsf T}\bigl(f(x,u',d_0)-f(x,u_0,d_0)\bigr). \tag*{(36)}

Next, we establish the linearization error bound as

∣H(x,p′)−H(x,pθ0)−(p′−pθ0)Tf(x,z⋆(pθ0))∣=∣R(p′)∣(37)\left|H(x,p')-H(x,p_{\theta_0})-(p'-p_{\theta_0})^{\mathsf T}f\bigl(x,z^\star(p_{\theta_0})\bigr)\right|=|R(p')| \tag*{(37)}

by the definition (35), and z⋆(pθ0)=(u0,d0)z^\star(p_{\theta_0})=(u_0,d_0), and

∣R(p′)∣≤∥p′−pθ0∥2ω(p′)(38)|R(p')|\le\lVert p'-p_{\theta_0}\rVert_2\omega(p') \tag*{(38)}

by applying the Cauchy–Schwarz inequality to (36), where

ω(p′)=max⁡{∥f(x,u0,d′)−f(x,u0,d0)∥2,∥f(x,u′,d0)−f(x,u0,d0)∥2}.(39)\omega(p')=\max\left\{\left\lVert f(x,u_0,d')-f(x,u_0,d_0)\right\rVert_2,\left\lVert f(x,u',d_0)-f(x,u_0,d_0)\right\rVert_2\right\}. \tag*{(39)}

Step 3: Differentiability of HH. We prove that H(x,⋅)H(x,\cdot) is differentiable at pθ0p_{\theta_0}. By Lemma 3, as B0∋p′→pθ0B_0\ni p'\to p_{\theta_0}, then for any choice of saddle point (u′,d′)∈S(p′)(u',d')\in S(p') (which exists by Step 1), (u′,d′)→(u0,d0)(u',d')\to(u_0,d_0). By (A2), f(x,⋅)f(x,\cdot) is continuous, thus ω(p′)→0\omega(p')\to0 as p′→pθ0p'\to p_{\theta_0}. Therefore,

∣R(p′)∣∥p′−pθ0∥2≤ω(p′)→0\frac{|R(p')|}{\lVert p'-p_{\theta_0}\rVert_2}\le\omega(p')\to0

as p′→pθ0p'\to p_{\theta_0}. Since R(p′)R(p') does not depend on the choice of the saddle point (u′,d′)(u',d'), this implies that H(x,⋅)H(x,\cdot) is differentiable at pθ0p_{\theta_0}, and ∇pH(x,p)∣pθ0=f(x,z⋆(pθ0))\left.\nabla_pH(x,p)\right|_{p_{\theta_0}}=f\bigl(x,z^\star(p_{\theta_0})\bigr).

Step 4: Gradient-free. By (A3), pθp_\theta is differentiable at θ0\theta_0, thus by Step 3 and the chain rule,

∂θH(x,pθ)∣θ0=(∇pH(x,p)∣pθ0)T∂θpθ∣θ0=f(x,z⋆(pθ0))T∂θpθ∣θ0.\left.\partial_\theta H(x,p_\theta)\right|_{\theta_0} =\left(\left.\nabla_pH(x,p)\right|_{p_{\theta_0}}\right)^{\mathsf T}\left.\partial_\theta p_\theta\right|_{\theta_0} =f\bigl(x,z^\star(p_{\theta_0})\bigr)^{\mathsf T}\left.\partial_\theta p_\theta\right|_{\theta_0}.

Supporting Lemmas.

Lemma 2 (Saddle Value). [43 Section 5.4.2] If (u⋆,d⋆)(u^\star, d^\star) is a saddle point at pp, then H(x,p)=φ(p,u⋆,d⋆)H(x,p)=\varphi(p,u^\star,d^\star) and

φ(p,u,d⋆)≤H(x,p)≤φ(p,u⋆,d),∀u∈U, d∈D.\varphi(p,u,d^\star) \le H(x,p) \le\varphi(p,u^\star,d), \qquad\forall u \in\mathcal{U},\ d \in\mathcal{D}.

Lemma 3 (Continuity of saddle points at pθ0p_{\theta_0}). Let S(p)S(p) denote the set of saddle points at pp, that is,

S(p):={z=(u,d)∈U×D:φ(p,u~,d)≤φ(p,u,d)≤φ(p,u,d~)∀u~∈U, d~∈D}.(40)S(p) := \left\{z=(u,d)\in\mathcal{U}\times\mathcal{D}:\varphi(p,\widetilde{u},d)\le\varphi(p,u,d)\le\varphi(p,u,\widetilde{d})\quad\forall\widetilde{u}\in\mathcal{U},\ \widetilde{d}\in\mathcal{D}\right\}. \tag*{(40)}

Under (A1), (A2), and (A5), if the sequence pk→pθ0p_k\to p_{\theta_0} with pk∈B0p_k\in\mathcal{B}_0 (defined in (A4)) and zk∈S(pk)z_k\in S(p_k), then zk→z⋆(pθ0)z_k\to z^\star(p_{\theta_0}).

Proof. Write z0:=z⋆(pθ0)z_0:=z^\star(p_{\theta_0}) and zk=(uk,dk)z_k=(u_k,d_k) for all kk. Suppose the contrary, that is, there exists ϵ>0\epsilon>0 such that there is a subsequence {zk′}\{z_{k'}\} with ∥zk′−z0∥2≥ϵ\lVert z_{k'}-z_0\rVert_2\ge\epsilon for all k′k'.

By (A1), since U×D\mathcal{U}\times\mathcal{D} is compact, by the Bolzano-Weierstrass theorem [3 Theorem 3.4.8], there exists a convergent subsequence {zk′′}\{z_{k''}\} of {zk′}\{z_{k'}\} such that zk′′→zˉ=(uˉ,dˉ)∈U×Dz_{k''}\to\bar{z}=(\bar{u},\bar{d})\in\mathcal{U}\times\mathcal{D} and

∥zˉ−z0∥2≥ϵ,(41)\lVert\bar{z}-z_0\rVert_2\ge\epsilon, \tag*{(41)}

by continuity of ∥⋅∥2\lVert\cdot\rVert_2. Also, remark that pk′′→pθ0p_{k''}\to p_{\theta_0} by assumption. Since zk′′=(uk′′,dk′′)∈S(pk′′)z_{k''}=(u_{k''},d_{k''})\in S(p_{k''}), we have [43 Section 5.4.2]

φ(pk′′,u~,dk′′)≤φ(pk′′,uk′′,dk′′)≤φ(pk′′,uk′′,d~),∀u~∈U, d~∈D.(42)\varphi(p_{k''},\widetilde{u},d_{k''})\le\varphi(p_{k''},u_{k''},d_{k''})\le\varphi(p_{k''},u_{k''},\widetilde{d}),\qquad\forall\widetilde{u}\in\mathcal{U},\ \widetilde{d}\in\mathcal{D}. \tag*{(42)}

Consider φ(p,u,d)=pTf(x,u,d)\varphi(p,u,d)=p^{\mathsf{T}}f(x,u,d). By (A2), f(x,⋅)f(x,\cdot) is continuous on U×D\mathcal{U}\times\mathcal{D}. Together with continuity of inner product, they imply φ\varphi is jointly continuous in (p,u,d)(p,u,d). Thus, taking k′′→∞k''\to\infty gives

φ(pθ0,u~,dˉ)≤φ(pθ0,uˉ,dˉ)≤φ(pθ0,uˉ,d~),∀u~∈U, d~∈D.\varphi(p_{\theta_0},\widetilde{u},\bar{d})\le\varphi(p_{\theta_0},\bar{u},\bar{d})\le\varphi(p_{\theta_0},\bar{u},\widetilde{d}),\qquad\forall\widetilde{u}\in\mathcal{U},\ \widetilde{d}\in\mathcal{D}.

This implies that zˉ=(uˉ,dˉ)\bar{z}=(\bar{u},\bar{d}) is a saddle point at pθ0p_{\theta_0} [43 Section 5.4.2]. On the other hand, by (A5), S(pθ0)={z0}S(p_{\theta_0})=\{z_0\} is a singleton. Thus, we have zˉ∈S(pθ0)={z0}\bar{z}\in S(p_{\theta_0})=\{z_0\}, but this contradicts (41) and thus contradicts our original assumption. ▪

References

  1. [1]Brandon Amos. On amortizing convex conjugates for optimal transport. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=TQ5WUuS_4ai.arxiv.org/abs/2210.12153
  2. [2]Somil Bansal and Claire J. Tomlin. DeepReach: A deep learning approach to high-dimensional reachability. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pages 1817–1824. IEEE, 2021. doi: 10.1109/ICRA48506.2021.9561949.DOI
  3. [3]Robert G Bartle and Donald R Sherbert. Introduction to real analysis, volume 2. Wiley New York, 2000.
  4. [4]Jonathan Barzilai and Jonathan M Borwein. Two-point step size gradient methods. IMA journal of numerical analysis, 8(1):141–148, 1988.DOI
  5. [5]John T. Betts. Practical Methods for Optimal Control and Estimation Using Nonlinear Programming. SIAM, 2010.DOI
  6. [7]Jason J. Choi, Christopher A. Strong, Koushil Sreenath, Namhoon Cho, and Claire J. Tomlin. Data-driven hamiltonian for direct construction of safe set from trajectory data. In 2025 IEEE 64th Conference on Decision and Control (CDC), pages 2951–2958. IEEE, 2025. doi: 10.1109/CDC57313.2025.11312763.DOI
  7. [8]Zeyuan Feng, Le Qiu, and Somil Bansal. Bridging Model Predictive Control and Deep Learning for Scalable Reachability Analysis. In Proceedings of Robotics: Science and Systems, Los Angeles, CA, USA, June 2025. doi: 10.15607/RSS.2025.XXI.114.DOI
  8. [9]Jaime F Fisac, Mo Chen, Claire J Tomlin, and S Shankar Sastry. Reach-avoid problems with time-varying dynamics, targets and constraints. In Proceedings of the 18th international conference on hybrid systems: computation and control, pages 11–20, 2015.arxiv.org/abs/1410.6445
  9. [10]Samy Wu Fung, Howard Heaton, Qiwei Li, Daniel McKenzie, Stanley Osher, and Wotao Yin. JFB: Jacobian-free backpropagation for implicit networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 6648–6656, 2022.
  10. [11]Eric Gelphman, Deepanshu Verma, Nicole Tianjiao Yang, Stanley Osher, and Samy Wu Fung. End-to-end training of high-dimensional optimal control with implicit hamiltonians via jacobian-free backpropagation. In 2026 American Control Conference (ACC). IEEE, 2026a.arxiv.org/abs/2510.00359
  11. [12]Eric Gelphman, Deepanshu Verma, Nicole Tianjiao Yang, Stanley Osher, and Samy Wu Fung. On the convergence of jacobian-free backpropagation for optimal control problems with implicit hamiltonians. arXiv preprint arXiv:2602.00921, 2026b.arxiv.org/abs/2602.00921
  12. [13]Gautier Hattenberger, Murat Bronz, and Jean-Philippe Condomines. Evaluation of drag coefficient for a quadrotor model. International Journal of Micro Air Vehicles, 15:17568293221148378, 2023.
  13. [14]Howard Heaton and Samy Wu Fung. Explainable ai via learning to optimize. Scientific Reports, 13 (1):10103, 2023.DOI
  14. [15]Howard Heaton, Samy Wu Fung, Aviv Gibali, and Wotao Yin. Feasibility-based fixed point networks. Fixed Point Theory and Algorithms for Sciences and Engineering, 2021(1):21, 2021.arxiv.org/abs/2104.14090
  15. [16]William Hofgard. Convergence guarantees for neural network-based hamilton–jacobi reachability. In NeurIPS 2024 Workshop on Data-driven and Differentiable Simulations, Surrogates, and Solvers, 2024. URL https://openreview.net/forum?id=SH1isYEjz9.arxiv.org/abs/2410.02904
  16. [17]Rufus Isaacs. Differential Games: A Mathematical Theory with Applications to Warfare and Pursuit, Control and Optimization. John Wiley & Sons, New York, 1965.
  17. [18]Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.arxiv.org/abs/1412.6980
  18. [19]Galina M Korpelevich. The extragradient method for finding saddle points and other problems. Matecon, 12:747–756, 1976.
  19. [20]Aditi Krishnapriyan, Amir Gholami, Shandian Zhe, Robert Kirby, and Michael Mahoney. Characterizing possible failure modes in physics-informed neural networks. Advances in neural information processing systems, 34:26548–26560, 2021.arxiv.org/abs/2109.01050
  20. [21]Sudeep Kundu and Karl Kunisch. Policy iteration for hamilton-jacobi-isaacs equations with control constraints and comparison with hamilton-jacobi-bellman equations. arXiv preprint arXiv:2609.29368, 2026.arxiv.org/abs/2609.29368
  21. [22]Minhyeok Kwon and Yongsoon Eun. Quadrotor dynamics in a wind field: Equilibria analysis and energy dissipation. International Journal of Control, Automation and Systems, 22(11):3275–3284, 2024.DOI
  22. [23]Alistair Letcher, David Balduzzi, Sébastien Racan iere, James Martens, Jakob Foerster, Karl Tuyls, and Thore Graepel. Differentiable game mechanics. Journal of Machine Learning Research, 20 (84):1–40, 2019.arxiv.org/abs/1905.04926
  23. [24]Xingjian Li, Kelvin Kan, Deepanshu Verma, Krishna Kumar, Stanley Osher, and Ján Drgoňa. Zero-shot transferable solution method for parametric optimal control problems. In 2026 American Control Conference (ACC), pages 2003–2010. IEEE, 2026a.arxiv.org/abs/2509.18404
  24. [25]Xingjian Li, Kelvin Kan, Deepanshu Verma, Krishna Kumar, Stanley Osher, and Samy Wu Fung. End-to-end learning of safe optimal feedback control in high dimensions with control barrier function layers. arXiv preprint arXiv:2607.20674, 2026b.arxiv.org/abs/2607.20674
  25. [26]Albert Lin and Somil Bansal. Generating formal safety assurances for high-dimensional reachability. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 10525–10531. IEEE, 2023.arxiv.org/abs/2209.12336
  26. [27]Albert Lin, Alessandro Pinto, and Somil Bansal. Robust verification of controllers under state uncertainty via hamilton–jacobi reachability analysis. In Proceedings of the 8th Annual Learning for Dynamics and Control Conference, volume 331 of Proceedings of Machine Learning Research, pages 1432–1446. PMLR, 2026.arxiv.org/abs/2511.14755
  27. [28]Jonathan Lorraine, Paul Vicol, and David Duvenaud. Optimizing millions of hyperparameters by implicit differentiation. In International conference on artificial intelligence and statistics, pages 1540–1552. PMLR, 2020.arxiv.org/abs/1911.02590
  28. [29]Robert Mahony, Vijay Kumar, and Peter Corke. Multirotor aerial vehicles: Modeling, estimation, and control of quadrotor. IEEE robotics & automation magazine, 19(3):20–32, 2012.DOI
  29. [30]Eric Mazumdar, S Shankar Sastry, and Michael I Jordan. On finding local nash equilibria (and only local nash equilibria) in zero-sum games. ACM/IMS Journal of Data Science, 2(2):1–26, 2025.arxiv.org/abs/1901.00838
  30. [31]Daniel McKenzie, Howard Heaton, and Samy Wu Fung. Differentiating through integer linear programs with quadratic regularization and davis-yin splitting. Transactions on Machine Learning Research, 2024a. ISSN 2835-8856. URL https://openreview.net/forum?id=H8IaxrANWl.arxiv.org/abs/2301.13395
  31. [33]Ian M. Mitchell, Alexandre M. Bayen, and Claire J. Tomlin. A time-dependent hamilton–jacobi formulation of reachable sets for continuous dynamic games. IEEE Transactions on Automatic Control, 50(7):947–957, 2005a. doi: 10.1109/TAC.2005.851439.
  32. [34]Ian M Mitchell, Alexandre M Bayen, and Claire J Tomlin. A time-dependent hamilton-jacobi formulation of reachable sets for continuous dynamic games. IEEE Transactions on automatic control, 50(7):947–957, 2005b.
  33. [35]Yesom Park, Eric Gelphman, Stanley Osher, and Samy Wu Fung. Implicit neural optimal transport via fixed-point optimization. arXiv preprint arXiv:2605.10792, 2026.arxiv.org/abs/2605.10792
  34. [36]Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.arxiv.org/abs/1912.01703
  35. [37]Maziar Raissi, Paris Perdikaris, and George E Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378:686–707, 2019.
  36. [38]Hayden Schaeffer. Neural network approximation and structural complexity of semilinear pdes with lower bounds. arXiv preprint arXiv:2610.40384, 2026.
  37. [39]David SeWell, Xingjian Li, Stepan Tretjakov, Krishna Kumar, and David Fridovich-Keil. Neural operators for multi-task control and adaptation. Transactions on Machine Learning Research, 2026. ISSN 2835-8856. URL https://openreview.net/forum?id=jciObOZ5Wm.arxiv.org/abs/2604.03449
  38. [40]William Sharpless, Zeyuan Feng, Somil Bansal, and Sylvia Herbert. Linear supervision for nonlinear, high-dimensional neural control and differential games. In Proceedings of the 7th Annual Learning for Dynamics and Control Conference, volume 283 of Proceedings of Machine Learning Research, pages 365–377. PMLR, 2025.arxiv.org/abs/2412.02033
  39. [41]Aditya Singh, Zeyuan Feng, and Somil Bansal. Exact imposition of safety boundary conditions in neural reachable tubes. In 2025 IEEE International Conference on Robotics and Automation (ICRA), pages 5489–5495. IEEE, 2025.arxiv.org/abs/2404.00814
  40. [42]Ryan Teoh, Sander Tonkens, William Sharpless, Aijia Yang, Zeyuan Feng, Somil Bansal, and Sylvia Herbert. MADR: MPC-guided adversarial DeepReach. arXiv preprint arXiv:2510.18845, 2025.arxiv.org/abs/2510.18845
  41. [43]Lieven Vandenberghe and Stephen Boyd. Convex optimization, volume 1. Cambridge university press Cambridge, 2004.
  42. [44]Sifan Wang, Shyam Sankaran, and Paris Perdikaris. Respecting causality is all you need for training physics-informed neural networks. arXiv preprint arXiv:2203.07404, 2022.arxiv.org/abs/2203.07404
  43. [45]Sifan Wang, Shyam Sankaran, Hanwen Wang, and Paris Perdikaris. An expert’s guide to training physics-informed neural networks. arXiv preprint arXiv:2308.08468, 2023.arxiv.org/abs/2308.08468

Paper details

vlnc:W_pyk2va26
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 3

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Engineering

Not recorded for this paper

View statement
Looking for related papers…
Contents