Certified Neural Approximation of Robust Nonlinear Model Predictive Control

Introduction

Model predictive control (MPC) has been well-established as a reliable feedback control approach that is based on solving an underlying optimal control problem repeatedly and online in a receding horizon fashion [11, 21, 28]. As such, it can handle a variety of systems, can include input, state, and output constraints, and ensure near-optimal closed-loop behavior with respect to a given control objective. These properties have led to its widespread adoption in industry and research. However, the need to solve the optimal control problem online can impose a computational burden, posing implementation challenges in resource-constrained environments, such as embedded hardware, due to the demanding nature of the optimization process.

To address this challenge, various approaches have been proposed. For example, in the linear–quadratic case with linear dynamics, quadratic costs, and polytopic state and input constraints, explicit MPC yields a feedback law that is piecewise affine in the state. This structural property enables explicit MPC to compute the feedback law offline in closed form, thereby avoiding the need to solve an optimization problem online [1, 35]. However, the number of regions of the piecewise affine law grows, in the worst case, exponentially with the total number of inequality constraints, which in turn increases with the horizon length and the number of constrained states and inputs. Consequently, both the offline computation and memory complexity become rapidly intractable as the problem size grows.

This motivates the development of approximation- and learning-based approaches that mimic the implicit MPC feedback law [7, 8, 12, 15, 17, 20, 22, 23, 24, 27, 29, 33]. Among the available function approximators, neural networks (NNs) are particularly attractive and employed in this work due to their universal approximation capabilities [13, 14] and their ability to maintain a fixed evaluation effort with growing state dimension, input dimension, and dataset sizes. These properties have motivated numerous neural imitation-learning approaches for approximating MPC feedback laws. Nevertheless, the choice of function approximator alone does not determine how closed-loop guarantees are obtained. Existing approaches can be distinguished more coherently by how the approximation error is incorporated into the control design and how compliance with a prescribed error bound is certified.

One prominent guarantee-transfer paradigm consists of two steps. First, the MPC is designed to be robust against a prescribed approximation error, which is interpreted as a bounded disturbance acting on the system state. Second, an explicit approximation of the resulting MPC feedback law is constructed such that its error satisfies the prescribed bound. An early NN-based formulation of this approach was presented in [12], where Hoeffding’s inequality is used to validate statistically whether the learned controller satisfies the admissible input-error bound. If validation fails, additional samples are generated, and the learning procedure is repeated. Building on this framework, [24] extends the approach to nonlinear setpoint tracking and introduces a less conservative validation criterion that jointly accounts for approximation error and model mismatch. In [2], statistical post-validation is replaced by a deterministic condition that combines Lipschitz properties of the MPC feedback law and the NN with the coverage of the training data.

This disturbance-based guarantee-transfer principle is independent of the chosen function approximator. In [29], a Gaussian process is used to learn the MPC feedback law, and compliance with the required error bound is established through tailored training-data selection. Recently, [33] employs automatic kernel interpolation to construct an explicit surrogate whose approximation error is uniformly bounded by design, under the assumption that the feedback law belongs to the associated reproducing kernel Hilbert space (RKHS) with a known norm bound. Similarly, [30] uses kernel interpolation on the closed-loop reachable set and Monte Carlo reachability analysis to certify safety and closed-loop performance. These approaches share the disturbance-based guarantee-transfer principle of the NN methods above, but differ in the choice of approximator, the associated regularity assumptions, and the mechanism used to certify the approximation error.

Other NN-based approaches obtain guarantees through direct error certification, structural restrictions, or online correction mechanisms. In [8], semidefinite programming is used to bound the worst-case difference between the NN and MPC feedback laws, while [27] uses an optimization-based projection to preserve constraint satisfaction and stability. Building on constrained NN approximations of explicit MPC laws [7, 23], deep piecewise-affine networks provide approximation guarantees by design [17]. For nonlinear MPC, [15] validates NN proposals online and uses an MPC-intrinsic fallback when needed, while [20, 22] learn MPC laws from trajectory data, with [22] improving data efficiency through gradient-informed Sobolev training.

Against this background, we develop a constructive framework for the certified neural approximation of robust nonlinear MPC. The framework adopts the disturbance-based viewpoint of [12, 24, 29, 33], but derives a deterministic certificate from dataset coverage and Lipschitz bounds, thereby preserving closed-loop guarantees without relying on statistical validation or requiring membership of the MPC feedback law in a prescribed RKHS with a corresponding norm bound. We generalize the framework of [2] to MPC feedback laws that need not be globally Lipschitz continuous, establishing guarantee transfer under local regularity conditions along the closed-loop trajectory. To this end, we first design a robust MPC for a prescribed disturbance bound and derive conditions ensuring that a neural network trained on MPC state-input data satisfies this bound, thereby preserving safety and stability without post-verification or auxiliary safety mechanisms. We further validate these guarantees in closed-loop operation. Our main contributions are:

1. A priori conditions are derived that link the Lipschitz properties of the MPC feedback law and the coverage of the training data to certified bounds on the approximation error.

2. A constructive offline procedure is developed to obtain a neural network approximator that satisfies these bounds, enabling systematic training of the network.

3. It is shown that, under these conditions, the learned neural network controller inherits the recursive feasibility and stability properties of the robust MPC, ensuring safe and reliable closed-loop operation.

The remainder of this article is structured as follows. In Section 2, we formulate the problem and introduce the robust MPC scheme. Section 3 introduces the basics of the neural networks used, details our theoretical results on the neural network approximation of the underlying MPC, and further includes an overview of the constructive procedure enabling the generation of the neural networks in line with the previously established theory. We illustrate the effectiveness of our proposed approach in Section 4 in simulation using a 4-dimensional continuously stirred tank reactor example before concluding in Section 5.

Robust MPC as a Basis for Certified Controller Approximation

We first show that the neural network approximation error enters the system as a bounded input disturbance. Using Lipschitz continuity of the system dynamics, this input disturbance can be equivalently reformulated as a bounded additive state disturbance. This reformulation allows us to cast the closed-loop system in a standard robust control setting. Based on this representation, we introduce the robust nonlinear MPC scheme that forms the basis for transferring safety and stability guarantees to the neural network controller. The presentation focuses on a robust MPC formulation for additive state disturbances, which is sufficient for our subsequent analysis. We conclude the section by recalling the key closed-loop properties of the robust MPC, in particular, recursive feasibility and input-to-state stability.

System Description and Control Task

We consider nonlinear discrete-time systems given by

where $x_k \in\mathbb{R}^{n_x}$ and $u_k \in\mathbb{R}^{n_u}$ describe the system state and input, respectively, and $k \in\mathbb{N}_0$ denotes the discrete time index. The transition map is given by $f : \mathbb{R}^{n_x} \times\mathbb{R}^{n_u} \to\mathbb{R}^{n_x}$ and captures the dynamics of the system. The objective is to stabilize the system (1) at the origin (w.l.o.g.) while satisfying input and state constraints given by

While model predictive control is well-suited for this task, it requires the online solution of a possibly large-scale optimization problem, typically through iterative methods. Thus, we aim to control the system (1) by a low-complexity neural network approximation $\kappa_{\mathrm{NN}}$ of the model predictive feedback law $u=\kappa(x)$ to reduce computational demands or enable formal verification of the explicit feedback law. However, the use of the neural network introduces an approximation error

which manifests as an additive disturbance on the control input. If this disturbance is appropriately accounted for in the MPC design, the safety and stability guarantees can be transferred to the neural network controller.

To do so, we rely on a pre-specified bound on the neural network’s approximation error, which we ensure through suitable data selection and training. That is, we assume that we know $\epsilon>0$ such that the approximation error in (3) satisfies $\forall x\in\mathcal{X}:\ \lVert e\rVert\leq\epsilon$.² We then reformulate the resulting bounded input disturbance as a bounded additive disturbance acting on the state of the system (1) under the feedback law $\kappa$. This state disturbance is then addressed within a robust MPC formulation, allowing guarantees established for the closed-loop system under the robust MPC to carry over to any approximate controller that satisfies the prescribed error bound. The crucial step is to design a neural network that provably adheres to the predefined approximation error. To this end, we exploit Lipschitz continuity properties of both the MPC feedback law and the neural network, together with a data coverage criterion for the training data with respect to the domain $\mathcal{X}$.

We make the following assumptions on the properties of the system and constraint sets.

Note that the differentiability property in Assumption 1 is needed to apply a known result from parametric optimization regarding the structural properties of the MPC feedback law, which constitutes the first of two key components enabling the transfer of guarantees from the robust MPC to the neural network controller. In practice, many commonly considered systems satisfy this assumption.

As before, Assumption 2 is necessary to ensure the desired structural properties of the optimal control problem underlying the MPC feedback law. However, most practical MPC formulations employ polytopic or ellipsoidal sets for $\mathcal{X}$ and $\mathcal{U}$, for which Assumption 2 holds.

From Input Approximation Error to State Disturbance

Most robust MPC formulations account for state disturbances. Therefore, we reformulate the approximation error as an equivalent state disturbance, enabling the application of a broad class of robust MPC formulations that provide a priori guarantees of closed-loop robustness to the neural network controller’s approximation error.

Assumptions 1 and 2 imply Lipschitz continuity of the transition map $f$ with respect to both $x$ and $u$ on the compact domain $\mathcal{X}\times\mathcal{U}$. That is, there exist finite constants $L_{f,x}>0$ and $L_{f,u}>0$ such that

hold for all $x,x'\in\mathcal{X}$ and $u,u'\in\mathcal{U}$. Hence, (4a) implies that the bounded input approximation error induces a bounded additive disturbance $w$ on the system states,

exploiting knowledge of a bound $\epsilon>0$ on the approximation error. We emphasize that this approximation error bound is a design parameter. We will later show that it is indeed possible to construct a neural network approximator that satisfies any prescribed bound.

Having rewritten the approximation error as a bounded additive disturbance in (5), the closed-loop dynamics under the neural network controller fit the standard setting of robust MPC.

Robust MPC Design and Closed-Loop Guarantees

The general idea of robust MPC schemes is to analyze a nominal system model subject to appropriately tightened constraints, such that satisfaction of the tightened constraints by the undisturbed system implies satisfaction of the original constraints by the disturbed system. This enables the establishment of robust recursive feasibility guarantees in closed-loop operation. Furthermore, the optimal value function of a robust MPC scheme is typically used as a Lyapunov function candidate to derive robust stability guarantees for the closed-loop system.

Without loss of generality, we adopt the robust MPC scheme proposed in [19]³, whose underlying optimal control problem is given by

Here, $\bar{x}_{n|k}$ denotes the nominal state prediction $n$ steps ahead of the current time $k$, $u_{n|k}$ denotes the planned future control inputs $n$ steps ahead of the current time $k$, and $N\in\mathbb{N}$, denotes the prediction horizon. The nominal state predictions are computed using the nominal system model (6b) initialized with the most recent state information $x_k$. The control objective is encoded in the cost function in (6a), which is composed of the stage cost $l:\mathbb{R}^{n_x}\times\mathbb{R}^{n_u}\to[0,\infty)$ and the terminal cost $V_{\mathrm{T}}:\mathbb{R}^{n_x}\to[0,\infty)$. We assume the following for the stage and the terminal cost functions.

Assumption 3 The stage and the terminal cost functions are twice continuously differentiable, i.e., $l\in C^2(\mathbb{R}^{n_x}\times\mathbb{R}^{n_u},[0,\infty))$ and $V_{\mathrm{T}}\in C^2(\mathbb{R}^{n_x},[0,\infty))$. Moreover, $l$ and $V_{\mathrm{T}}$ are positive definite w.r.t. to the origin.

Note that Assumption 3 is satisfied by many commonly used stage and terminal cost functions, including quadratic costs typically employed in regulation (and, by extension, tracking) problems. Moreover, Assumption 3 implies that the stage and terminal cost functions are Lipschitz continuous on the compact domain $\mathcal{X}\times\mathcal{U}$ and $\mathcal{X}$, respectively, with Lipschitz constants $L_l,L_v>0$.

We impose the input constraint in (6c), where $\ominus$ denotes the Pontryagin difference. This tightening of the original input constraints in (2) accounts for the approximation error introduced by the neural network controller.

Similarly, the state constraints in (2) appear in a tightened form in (6d), where they are imposed on the nominal system state. This tightening captures the effect of the (propagated) approximation error of the neural network controller on the system’s behavior. It is performed based on the sets

Here the sets $\mathcal{W}_n$ overapproximate the possible — but a priori unknown — realizations of the mismatch between the nominal system and the disturbed system after $n$ steps. Hence, the constraint tightening in (6d) ensures that the state constraints will be satisfied by the system (1) under the approximate neural network controller.

The terminal region constraint in (6e), with a suitably chosen parameter $\alpha_v>0$, is used to guarantee recursive feasibility and stability of the closed-loop system under the MPC.

The optimal control problem (6) is repeatedly solved at any time $k$ with updated initial condition $x_k$, and the first element $u^*_{0|k}$ of the optimal input sequence $\mathbf{u}^*_k=(u^*_{0|k},\ldots,u^*_{N-1|k})$ is applied to the system (1). This implicitly defines the MPC feedback law

We assume that the following conditions hold [19].

Under these assumptions, the MPC defined by the optimal control problem (6) is recursive feasible and robustly stabilizes the system (1), as stated in the following theorem.

These results establish the foundation for extending the safety and stability guarantees of the MPC scheme to the neural network controller, as discussed in the next section.⁴

Certified Neural Approximation of the MPC Feedback Law

We begin by reviewing the fundamentals of feed-forward neural networks, which we employ as function approximators to learn the MPC feedback law. Thereafter, we present our main contribution. First, we derive conditions under which the neural network is guaranteed to satisfy a prescribed approximation error bound. Second, we discuss how safety and stability guarantees of the original MPC transfer to the neural network controller. The section concludes with an algorithm for constructing the neural network controller.

Feed-Forward Neural Networks

We employ neural networks to approximate the feedback law $\kappa_{\mathrm{NN}}$, motivated by their ability to represent nonlinear functions accurately [13, 14], while retaining a fixed online evaluation structure that is independent of the size of the training dataset. This is particularly attractive for our approach, where large offline datasets are required, as discussed in § 3.1.1. Specifically, we restrict ourselves to feed-forward neural networks trained offline via supervised learning techniques and ensure differentiability of the neural networks by choosing differentiable activation functions. Particularly, the $j^{\text{th}}$ layer of a NN can be written as

Here, $\zeta_j$ denotes the affine transformation of the $j^{\text{th}}$ layer’s inputs, defined through the weight matrix $A_j \in\mathbb{R}^{n_{h_j} \times n_{h_{j-1}}}$ and the bias vector $b_j \in\mathbb{R}^{n_{h_j}}$. The nonlinear function $\sigma_j : \mathbb{R}^{n_{h_j}} \to\mathbb{R}^{n_{h_j}}$ encodes the element-wise application of an activation function to $\zeta_j(h_{j-1})$ to produce the output $h_j$. A feed-forward NN is now the composition of all layers and their activations, denoted as

where $m$ is the total number of layers in the network. Since the activation functions $\sigma_j$ are differentiable, and $\zeta_j$ is affine

and therefore differentiable, their composition (9) is also differentiable. This implies that the NN (9) is Lipschitz continuous on the compact domain $\mathcal{X}$.

By exploiting Lipschitz continuity of the neural network, we derive conditions under which the NN satisfies a prescribed error bound in the next section. However, it is important to note that the results presented are not limited to feed-forward neural networks. Rather, any neural network that is Lipschitz continuous on $\mathcal{X}$ can be used. We formalize this requirement in the more general assumption below.

Approximation Error Bounds

The results presented here develop the Lipschitz-based approximation approach introduced in [2 Theorem 1] into a certified approximation setting for robust nonlinear MPC. As in that work, our approach relies primarily on two key aspects: (i) the careful design of the training data and the accuracy of the NN approximator on the training data, and (ii) regularity properties of the MPC feedback law (8). We begin by discussing the requirements for the training data and the neural network’s accuracy on it.

The neural network is trained on a dataset $\mathcal{D} = \{(x_i,\kappa(x_i)) \mid i \in\{1,\ldots,n_{\mathcal{D}}\}\}$ consisting of $n_{\mathcal{D}}$ observations of the MPC feedback law (8) on the domain $\mathcal{X}$. Specifically, we will explore coverage properties of the set of training inputs, $\{x_i \mid i = 1,\ldots,n_{\mathcal{D}}\}$, on $\mathcal{X}$, as defined below.

We further assume that an arbitrary prescribed training accuracy on a dataset $\mathcal{D}$ can be attained through a suitable choice of network architecture and training procedure, as formalized in Assumption 10. This assumption is motivated by Lemma 9, which establishes the existence of a neural network achieving any prescribed accuracy on a finite dataset.

Lemma 9 establishes the existence of a feedforward neural network with any prescribed in-sample accuracy, whereas Assumption 10 additionally requires that such accuracy be attainable through a suitable training procedure and architecture. Both statements concern only the samples in $\mathcal{D}$; they imply neither convergence of a particular training procedure nor generalization beyond the training data. The training-set bound is extended to a uniform approximation bound below by combining it with the regularity properties of the MPC and NN policies and the coverage of the training inputs. We therefore begin by considering the regularity of the MPC feedback law (8). To this end, we note that the optimal control problem (6) is a parametric optimization problem, where the parameter is the initial state $x_k$ in (6b). Specifically, we can express the nominal state $\bar{x}_{n\mid k}$, $n \in\{0,\ldots,N\}$, in (6b) as a function of the initial state and the input (sub-)sequence, as $\bar{x}_{n\mid k}=\psi(n;x_k,u_k)$. Since $\psi$ is defined via the $n$-fold application of the dynamics $f$, and $f$ is twice continuously differentiable in $x$ and $u$, $\psi$ is twice continuously differentiable in $x_k$ and $u_k$. Substituting $\psi(n;x_k,u_k)$ for the states $\bar{x}_{n\mid k}$ in the cost function (6a) and the constraints (6d) and (6e) enables us to write the optimal control problem (6) generically as

Again, since self-compositions and sums of twice continuously differentiable functions remain twice continuously differentiable, Assumptions 1, 2, and 3 imply that the functions $J$ and $G_i$, $i=1,\ldots,N(n_c^u+n_c^x)+1$ in (10), are twice continuously differentiable on the interior of $\mathcal{U}^N\times\mathcal{X}$. We will explore the following result from the field of parametric optimization.

We now derive an approximation bound linking dataset coverage and Lipschitz constants.

PROOF. We begin by writing the left-hand side of (11) as

Applying the triangle inequality, we obtain the upper bound

By Assumption 7, and as $x^*\in\mathcal{N}(x_k)$, we obtain further that

Now, it follows straightforwardly that

which, together with the requirement that $\|x_k-x^*\|\leq\left(L_{\mathrm{MPC}}(x_k)+L_{\mathrm{NN}}\right)^{-1}(\epsilon-\epsilon_{\mathcal{D}})$, implies the result in (11).

We now turn to a detailed discussion of the conditions in Proposition 11.

Remark 14 Assuming that LICQ, SOSC, and SCS hold can be restrictive. However, these assumptions are often employed in explicit MPC and parametric optimization to ensure that the optimal solution is well-behaved, see, e.g., [5] or [34]. LICQ is typically ensured by design of the constraints (6c), (6d), and (6e), or via regularization or constraint relaxation techniques, see, e.g., [28, 31]. SCS is usually only lost when arbitrarily small perturbations of the state $x_k$ lead to active set changes. However, the sets on which this is the case are typically lower-dimensional subsets of $\mathcal{X}$, and hence, Lebesgue null sets. As a result, it is unlikely in practice that a state sample lies in these sets. Satisfaction of the SOSC is more challenging; however, our approach allows for violations of the SOSC on Lebesgue-null sets, analogous to the SCS condition. For a more comprehensive overview of the regularity properties of parametric optimization problems, see, e.g., [9].

To better understand the implications of Theorem 13, we proceed by discussing some of its practical aspects.

Remark 15 If an upper bound $L_{\mathrm{MPC}}^{\max}$ on the local Lipschitz constants $L_{\mathrm{MPC}}(x_k)$ can be estimated over all regions of the state space where the MPC feedback law is locally Lipschitz continuous, Theorem 13 generalizes accordingly. Specifically, a state-independent distance bound $\delta:= (L_{\mathrm{MPC}}^{\max} + L_{\mathrm{NN}})^{-1}(\epsilon- \epsilon_D)$ can then be used to establish the result, at the cost of potentially increased conservatism. Hence, Theorem 13 provides a condition on the minimum coverage of $\mathcal{X}$ that must be achieved by the training inputs. To enforce the distance condition in Theorem 13, one can choose a parameter $0 < \eta\le\delta$ and construct the training dataset to provide an $\eta$-coverage of $\mathcal{X}$. However, $\eta$-coverage alone does not ensure that the corresponding training point lies in the same neighborhood of local Lipschitz continuity as $x_k$. These regions are determined by the underlying MPC problem and may have arbitrary geometry; in particular, they need not be norm balls or have a uniform radius. For each state $x_k$, the relevant neighborhood is the one associated with the corresponding region of the MPC problem. The result therefore remains conditional on the existence, for each considered state $x_k$, of a training point that satisfies both the distance and neighborhood conditions of Theorem 13. The parameter $\eta$ thus acts as a tuning parameter, which must be selected heuristically or via appropriate tuning strategies tailored to the specific application.

Recursive feasibility and Stability Guarantees

Building on the results of Theorem 13, we can now guarantee safety, i.e., constraint satisfaction, and stability of the system (1) under the neural network controller. Specifically, Theorem 13 provides conditions under which a neural network satisfies a prescribed bound on the approximation error, which can be used to design the underlying MPC to be robust against such errors, as described in § 2. This robustness enables safe substitution of the MPC with the neural network controller. In particular, if the conditions of Theorem 13 are satisfied (point-wise) along the closed-loop state trajectory of system (1) under the neural network controller, the safety and stability guarantees established in Theorem 5 are preserved. Moreover, under these conditions, the neural network controller maintains the invariance of the feasible set of initial conditions for the closed-loop system, as established in Corollary 6.

The following theorem provides a formal statement of the result under practical and realistically attainable conditions, explicitly accounting for stochastic disturbances that arise in closed-loop operation, which results in a probabilistic view on the results in Proposition 11 and Theorem 13.

PROOF. The proof proceeds in three main steps. First, we revisit the robust MPC design under the presence of an additional disturbance, as specified in condition (iia), and demonstrate the feasibility of the robust MPC formulation required by condition (iii). Second, we show that the feedback law of this robust MPC is almost surely sufficiently regular at the sampled closed-loop states $x_k$ where it is evaluated, leveraging conditions (iia) and (iv). This guarantees the local Lipschitz continuity of the feedback law in neighborhoods around the observed closed-loop states. Third, leveraging conditions (i) and (v), we establish that the approximation error at any observed closed-loop state satisfies the bound stated in Theorem 13. Together, these results imply that the guarantees provided by the robust MPC design in Theorem 5 transfer to the neural network controller.

Step 1: We begin by considering the system (1) subject to the additive disturbance $\nu_k$, given by $x_{k+1}=f(x_k,u_k)+\nu_k$.

Analogously to the formulation in (5), we define a single additive disturbance term $\bar{w}_k$ that captures both the effect of the approximation error induced by the neural network controller and the state disturbance $\nu_k$. Specifically,

This yields the bound $\lVert\bar{w}_k\rVert\leq\lVert f(x_k,u_k+e_k)-f(x_k,u_k)\rVert+\lVert\nu_k\rVert\leq L_{f,u}\epsilon+\epsilon_\nu$, where $\epsilon_\nu>0$ is a constant such that $\mathcal{W}_\nu\subseteq\{\nu\mid\lVert\nu\rVert\leq\epsilon_\nu\}$. Such a constant $\epsilon_\nu>0$ exists due to the compactness of $\mathcal{W}_\nu$. Including $\epsilon_\nu$ in (7) yields

This establishes the feasibility of the robust MPC design required in condition (iii), which is henceforth assumed.

Step 2: Given condition (iia), for a fixed state $x_k\in\mathcal{X}_N\setminus\mathcal{Z}$, the successor state $x_{k+1}$ is a random variable determined by the disturbance $\nu_k$ and follows the probability measure $\mu_x$. Specifically, $\mu_x$ is the pushforward of $\mu_\nu$ under the translation map $T(z)=z+f(x_k,u_k+e_k)$. Since the Lebesgue measure $\lambda$ is translation invariant and $\mu_\nu\ll\lambda$ by assumption, it follows that $\mu_x\ll\lambda$, i.e., $\mu_x$ is absolutely continuous with respect to the Lebesgue measure. Together with condition (iv), this yields $\mu_x(\mathcal{Z})=0$, or equivalently, $\Pr[x_{k+1}\in\mathcal{X}_N\setminus\mathcal{Z}]=1$ almost surely. By induction, and using condition (v), it follows that $x_k\in\mathcal{X}_N\setminus\mathcal{Z}$ for all $k$, almost surely. Consequently, the regularity conditions of Proposition 11 are almost surely satisfied at all visited closed-loop states, implying local Lipschitz continuity of the MPC feedback law in neighborhoods of these states.

Step 3: The established (local) Lipschitz continuity of the MPC feedback law in neighborhoods of all visited closed-loop states, together with conditions (i) and (vi), directly implies that the error at these states almost surely satisfies the bound stated in Theorem 13. Since, by condition (iii), the underlying MPC is robust with respect to both the approximation error induced by the neural network controller and the additive disturbance $\nu_k$, the closed-loop guarantees established in Theorem 5 almost surely carry over to the neural network controller.

We conclude the theoretical analysis by briefly discussing the conditions of Theorem 16. Condition (i) requires Assumptions 1, 2, 3, 4, 7, and 10, whose implications and restrictiveness were discussed upon introduction.

Condition (ii) assumes bounded additive disturbances with a continuous distribution of compact support, encompassing, e.g., model-plant mismatch, external disturbances, and measurement noise. Condition (iia) is standard in robust and stochastic MPC, while (iib) covers, among others, uniform and truncated Gaussian distributions; the distribution itself need not be known explicitly or analytically tractable.

Condition (iii) requires robustness of the MPC against the combined disturbance, achievable through standard constraint tightening as described in § 2.3.

Condition (iv) requires the irregularity set $\mathcal{Z}$ to have Lebesgue measure zero, supported by Remark 14 and empirical evidence reported in [34].

Finally, condition (v) requires initial MPC feasibility and excludes states where the feedback law lacks regularity.

Procedure for Certified NN Approximation

We now present a procedure to generate a NN controller that provides the guarantees discussed in § 3.2 and discuss the limitations of implementing this method in a practical setting.

A possible implementation is summarized in Algorithm 1. The resulting NN is fixed at run time and does not require any online adaptation or re-training. In particular, once provisional bounds on the approximation error and the Lipschitz constants of the MPC law and the NN have been selected, sampling and training may be carried out in a single offline procedure, provided that the resulting network satisfies the prescribed bounds. Thus, enlarging the training dataset does not affect online execution times, as the evaluation time at deployment depends only on the NN architecture.

We first specify a bound $\epsilon_{\mathcal{D}}$ on the expected error at the training points of the NN with respect to the underlying MPC law $\kappa(x)$. This bound may be chosen based on domain knowledge of the problem, an engineering guess, or a single preliminary NN training on an arbitrary initial dataset $\mathcal{D}_0$ from a similar domain. Given such a bound, we select the design parameter $\epsilon$ in Theorem 13 so that $\epsilon\geq\epsilon_{\mathcal{D}}$, which can be chosen larger to include additional robustness margins.

Once $\epsilon$ has been fixed, we design the robust MPC as in § 2.3. At this stage, state disturbances $\epsilon_\nu$ can also be incorporated. This yields a robust MPC feedback law $\kappa(x)$.

Given the MPC design, we next provide bounds on the local Lipschitz constants of both the NN and the MPC on $\mathcal{X}$. An upper bound $L_{\mathrm{MPC}}$ for the feedback law (8) may be obtained via power iteration, finite-difference schemes, or specialized methods [26, 32]. Similarly, we fix $L_{\mathrm{NN}}$ for the NN architecture and estimate the robustly forward-invariant set $\mathcal{X}_{N}$, on which the MPC law is defined and from which the training data are drawn.

With the bounds $\epsilon_D$, $L_{\mathrm{NN}}$, and $L_{\mathrm{MPC}}$ specified and Theorem 13, we compute the coverage radius $\delta$. We then select a radius $\eta\leq\delta$ for the over-coverage. Having fixed $\eta$, we employ sub-sampling or space-filling methods [18] to compute an $\eta$-coverage of $\mathcal{X}_{N}$, which is also a $\delta$-coverage of $\mathcal{X}_{N}$. Evaluating the MPC feedback law (9) on these points in open-loop yields the final training dataset $\mathcal{D}$.

We then initialize the neural network with the chosen architecture and train it once on the fixed dataset $\mathcal{D}$. The trained network must satisfy $\hat{\epsilon}_{D} := \max_{(x_i,\kappa(x_i))\in\mathcal{D}} \lVert\kappa(x_i)-\kappa_{\mathrm{NN}}(x_i)\rVert\leq\epsilon_D$ and $\hat{L}_{\mathrm{NN}} \leq L_{\mathrm{NN}}$. Because these quantities are known only after training, hyperparameter tuning or retraining may be required. Any network satisfying both inequalities is admissible, and no further refinement is needed.

Algorithm 1 Generation of NN Controller

Require: Constraint set $\mathcal{X}$.

1. Specify or estimate a nominal approximation error bound $\epsilon_D$

2. Choose a design approximation error bound $\epsilon\geq\epsilon_D$.

3. Design a robust MPC using $\epsilon$ and, optionally, a disturbance bound $\epsilon_\nu$, obtaining the feedback law $\kappa(x)$.

4. Specify Lipschitz bounds $L_{\mathrm{MPC}}$ and $L_{\mathrm{NN}}$ on $\mathcal{X}$.

5. Estimate the robustly forward invariant set $\mathcal{X}_{N}$ of the robust MPC.

6. Compute the required coverage radius $\delta= \frac{\epsilon-\epsilon_D}{L_{\mathrm{MPC}}+L_{\mathrm{NN}}}$ according to Theorem 13.

7. Select a radius $\eta\leq\delta$ for over-coverage.

8. Compute an $\eta$-coverage of $\mathcal{X}_{N}$ and form the training dataset $\mathcal{D}$ by evaluating $\kappa(x)$ on the coverage points.

9. Train the neural network $\kappa_{\mathrm{NN}}$ on $\mathcal{D}$, such that it satisfies $\hat{\epsilon}_{D} \leq\epsilon_D \leq\epsilon$ and $\hat{L}_{\mathrm{NN}} \leq L_{\mathrm{NN}}$.

10. Output the final NN controller $\kappa_{\mathrm{NN}}(x)$, which approximates $\kappa(x)$ on $\mathcal{X}_{N}$ with error bound $\epsilon$.

Example

We evaluate the approach on an isothermal continuously stirred tank reactor in which the reaction $2A+B\to C$, $C\to2D$, and the selective removal of $D$ are realized. The continuous-time dynamics are

Here, $x=[c_A,c_B,c_C,c_D]$ denotes the species concentrations in $\mathrm{mol\,L^{-1}}$, and $u=c_{A,f}$ is the controlled feed concentration of species A in $\mathrm{mol\,L^{-1}}$. The dilution rate is $d=0.1\,\mathrm{min}^{-1}$, and the fixed feed concentration is $c_{B,f}=0.75\,\mathrm{mol\,L^{-1}}$. The reaction parameters are $k_1=3.2\,\mathrm{L^2\,min^{-2}}$, $k_2=0.1\,\mathrm{min}^{-1}$ and $k_{\mathrm{sep}}=0.1\,\mathrm{min}^{-1}$. The model is discretized using fourth-order Runge–Kutta with $T_s=0.5\,\mathrm{min}$.

Over $\mathcal{X}\times\mathcal{U}$, the numerically estimated Lipschitz constants of the undisturbed discrete dynamics w.r.t. the 2-norm are $L_{f,x}=1.11$ and $L_{f,u}=0.05$. We add $\tilde{v}_k=[\tilde{v}_{1,k},\tilde{v}_{2,k},\tilde{v}_{3,k},\tilde{v}_{4,k}]^\mathsf{T}$, as an additive disturbance satisfying $\lVert\tilde{v}_k\rVert_2\leq\epsilon_v$ with $\epsilon_v=0.001$ to the discretized system.

Controller Design

The robust MPC uses the constraint sets $\mathcal{X}:=\{x\mid0.05\,\mathrm{mol\,L^{-1}}\leq c_\gamma\leq0.45\,\mathrm{mol\,L^{-1}},\ \gamma\in\{A,B,C,D\}\}$ and $\mathcal{U}:=\{u\mid1.00\,\mathrm{mol\,L^{-1}}\leq u\leq1.50\,\mathrm{mol\,L^{-1}}\}$, together with the quadratic stage and terminal costs $l(x,u)=(x-x_{\mathrm{ref}})^\mathsf{T}Q(x-x_{\mathrm{ref}})+(u-u_{\mathrm{ref}})^\mathsf{T}R(u-u_{\mathrm{ref}})$ and $V_T(x)=(x-x_{\mathrm{ref}})^\mathsf{T}P(x-x_{\mathrm{ref}})$, respectively, where $Q=\operatorname{diag}(2,1,1,1)$, $R=1$, and

The matrix $P$ is obtained from fixed-point iterations of the discrete-time algebraic Riccati equation for the model linearized at the reference. The terminal level is set to $\alpha_V=0.10$. We use the constant reference $x_{\mathrm{ref}}=[0.25,0.25,0.25,0.25]^\mathsf{T}\,\mathrm{mol\,L^{-1}}$ and $u_{\mathrm{ref}}=1.25\,\mathrm{mol\,L^{-1}}$. We choose a prediction horizon of $N=10$ steps, corresponding to a prediction horizon of $5.00\,\mathrm{min}$. We conservatively set the designable approximation-error bound to $\epsilon=0.05$, corresponding to 10% of the input range.

Neural Network Architecture

We approximate the underlying robust MPC law by a fully connected feed-forward neural network (cf. § 3.1) with 3 hidden layers, each containing 32 neurons, and Gaussian error linear unit activation functions. The network takes the 4-dimensional concentration state as input and outputs the scalar feed concentration $c_{A,f}$.

Data Generation and Training

We seed $194\,481 = 214$ points equally spaced within the state constraint set. Using these points, we estimate a grid approximation $\hat{\mathcal{X}}_N \subseteq\mathcal{X}_N$ of the robustly forward-invariant set of the underlying MPC, which leaves 60 626 strictly feasible and converged grid points.

Before dataset generation, we estimate the Lipschitz constant of the MPC policy at 10 000 points in $\hat{\mathcal{X}}_N$ using central finite differences with a step size of $1 \times10^{-4}$. This yields $L_{\mathrm{MPC}} = 3.1531$. For the NN architecture described in In Section 4.2, we prescribe the analytical design bound $L_{\mathrm{NN}} = 3.5$, which is subsequently enforced via weight projection during training [10]. The projection approach is further explained in supplementary Section S2 and in [10]. We choose the dataset error $\epsilon_D = 0.01$ and the overall approximation-error bound $\epsilon= 0.05$. Therefore, we can calculate $\delta= 0.0060122$, and we choose $\eta= \delta$. For the bounds and all subsequent analysis, the 2-norm was used.

We construct a candidate grid over the bounding box of $\hat{\mathcal{X}}_N$ and retain points that pass a membership check with the invariant set. The final $\eta$-cover contains 6 417 883 centers. The MPC policy is evaluated at every center to obtain the final training dataset $\mathcal{D}$. We then train a newly initialized NN on $\mathcal{D}$ for 50 epochs using a learning rate of $5 \times10^{-3}$ and a batch size of 128. The resulting NN achieves an $R^2$-score of 0.9999 on a validation set containing 10 000 points that are distinct from all training points. The maximum approximation error over the complete training dataset is $\hat{\epsilon}_D = 1.3759 \times10^{-3} < 0.01$. For the trained NNs Lipschitz bound obtained from the layer operator norms and activation Lipschitz constants is $\hat{L}_{\mathrm{NN}} = 3.4999$. Thus, both quantities remain below the used design values.

All training, sampling, and simulations were performed in Python on a Lenovo ThinkPad T14 Gen 2a running Windows 11, equipped with an AMD Ryzen 7 PRO 5850U CPU and 32 GB RAM. MPC was implemented using IPOPT via CasADi [3], and the neural network using PyTorch [4, 25].

An additional 2-dimensional result is shown in supplementary Section S1

Results

We evaluate the trained NN on the simulated isothermal reactor in closed-loop using the robust MPC used for data generation as a comparison. The closed-loops for both controllers were simulated starting from 100 independently selected initial states for 160 steps with a sampling time of 0.5 min. To assess the controller near the boundary of the estimated invariant set, points near the eight coordinate-wise extrema of

the retained grid are included explicitly. The remaining initial states are sampled uniformly without replacement from $\hat{\mathcal{X}}_N$. None of the selected initial states were previously used for training or validation. For each initial state, the robust MPC and NN controller experience the same additional disturbance realization.

Figure 1 shows the resulting trajectories in the $(c_A,c_B)$- and $(c_C,c_D)$-projections of $\hat{\mathcal{X}}_N$. The enlarged regions show trajectories initialized close to different boundaries of the estimated admissible set. The trajectories generated by the robust MPC and its NN approximation remain close over the complete simulation horizon and approach the reference. No state-constraint or input-constraint violations are observed for either controller. In particular, all 16 000 evaluated states remain inside $\hat{\mathcal{X}}_N$. Set membership is evaluated in the full 4-dimensional state space and not solely from the 2-dimensional projections shown in Figure 1.

Table 1 reports the computation times obtained from 16 000 sequential single-state controller evaluations. Both controllers are evaluated on the same hardware using a single CPU thread, and no batching is employed for the NN controller. The mean computation time decreases from 20.73 ms for the robust MPC to 0.59 ms for the NN controller, corresponding to a mean speed-up of approximately 35 times.

	Mean (ms)	Max. (ms)	Min. (ms)
Robust MPC	20.73	85.79	16.40
NN controller	0.59	1.85	0.48

The pointwise approximation errors along the same independently generated trajectories are summarized in Table 2. The maximum observed error is $8.62 \times10^{-4}$, which is below the prescribed approximation bound. No violation of the approximation bound occurs among the 16000 evaluations. The minimum error is $5.73 \times10^{-9}$ and is therefore close to the numerical precision relevant to this evaluation.

	Mean	Max.	Min.
Approximation error	$4.76 \times10^{-5}$	$8.62 \times10^{-4}$	$5.73 \times10^{-9}$

These independent closed-loop results are consistent with the theoretical guarantees established through the underlying robust MPC, the coverage, and the certified approximation bound. In particular, the absence of approximation error bound, state, and input constraint violations supports the conditions of Theorem 13 along all evaluated trajectories and demonstrates the applicability of Algorithm 1.

Conclusion

We developed a constructive framework for certified neural approximation of robust nonlinear model predictive controllers. By interpreting the approximation error as a bounded input disturbance and embedding Lipschitz-based error bounds into a robust MPC design, we established conditions under which recursive feasibility and input-to-state stability of the original controller are preserved under neural replacement. In contrast to post-verification approaches, the proposed method provides an offline design procedure that links dataset coverage and Lipschitz constants to explicit uniform approximation guarantees.

The framework combines results from parametric optimization, robust MPC theory, and Lipschitz continuity of neural networks to derive verifiable error bounds. The simulation study on a nonlinear isothermal CSRT model confirms that the neural controller reproduces the closed-loop behavior of the robust MPC within the certified invariant region while significantly reducing online computation time.

Future work will investigate domain partitioning to reduce conservatism induced by global Lipschitz bounds and data requirements associated with uniform coverage, as well as adaptive or trajectory-based sampling and mixture-of-experts architectures to improve scalability to higher-dimensional systems. We will also consider relaxing the regularity assumptions on the MPC feedback law to broaden applicability of the proposed framework.

Declaration of Generative AI and AI-assisted technologies in the writing process

During the preparation of this work, the authors used Chat-GPT 5.5 and 5.6-Sol from OpenAI in order to improve language flow and wording. After using this tool/service, the authors reviewed and edited the content as needed and take full responsibility for the content of the publication.

2-Dimensional Example

In addition to the continuous stirred tank reactor example presented in the main part of the paper, we further evaluate the approach on a nonlinear drone model describing vertical motion. The state $x = [x_1, x_2] = [z, v_z]$ consists of altitude and vertical velocity, and the dynamics are given by

Here, $u$ is the input thrust force, $k_z = 0.0001\ \mathrm{s}\,\mathrm{m}^{-2}$ is the cubic drag coefficient in $z$-direction, $m = 1\ \mathrm{kg}$ is the mass of the drone, $g = 9.81\ \mathrm{m}\,\mathrm{s}^{-2}$ is the acceleration due to gravity, and $T_s = 0.05$ is the discretization time. Additionally, $\tilde{\nu}_k = [\tilde{\nu}_{1,k}, \tilde{\nu}_{2,k}]^{\mathsf{T}}$ denotes an additive disturbance satisfying $\lVert\tilde{\nu}_k\rVert_2 \le\epsilon_\nu$ with $\epsilon_\nu= 0.001$. The undisturbed system therefore has Lipschitz constants $L_{f,x} = 1.05$ and $L_{f,u} = 0.05$ w.r.t. the 2-norm, which can be calculated directly due to its control-affine structure.

Controller Design

For the robust MPC, we define the state constraint set $\mathcal{X}$ by $-20\ \mathrm{m} \le x_1 \le40\ \mathrm{m}$ and $-25\ \mathrm{m}\,\mathrm{s}^{-1} \le x_2 \le25\ \mathrm{m}\,\mathrm{s}^{-1}$, with positive velocity pointing upward, and the input constraint set $\mathcal{U}$ by $-30\ \mathrm{N} \le u \le30\ \mathrm{N}$.

We use a quadratic stage cost $l(x,u) = (x-x_{\mathrm{ref}})^{\mathsf{T}}Q(x-x_{\mathrm{ref}})+(u-u_{\mathrm{ref}})^{\mathsf{T}}R(u-u_{\mathrm{ref}})$, where $Q = \operatorname{diag}(5,1)$ and $R = 0.1$. For the terminal cost, we choose $V_T(x) = (x-x_{\mathrm{ref}})^{\mathsf{T}}P(x-x_{\mathrm{ref}})$ with

obtained via fixed-point iterations of the discrete-time algebraic Riccati equation. The terminal set is defined as the $\alpha_V$-level set of $V_T$ with $\alpha_V = 100$. We set a constant reference of $x_{\mathrm{ref}} = [25.0\ \mathrm{m}, 0\ \mathrm{m}\,\mathrm{s}^{-1}]^{\mathsf{T}}$ and $u_{\mathrm{ref}} = g \cdot m = 9.81\ \mathrm{N}$.

The designable approximation error bound is set to $\epsilon= 3$, which is small relative to the admissible range of $u$.

Horizon length and conservatism

For the robust MPC formulation in (6), the disturbance sets $W_n$ may grow with the prediction step. Their radius $r_n$ is

Hence, $W_n$ remains bounded for $L_{f,x}<1$, grows linearly for $L_{f,x}=1$, and may grow geometrically for $L_{f,x}>1$.

For the drone model, $L_{f,x}=1.05>1$, and the disturbance sets therefore grow along the prediction horizon. With $L_{f,u}=0.05$, $\epsilon=3$, and a chosen $N=30$, the radius of $W_{N-1}=W_{29}$ is approximately

For comparison, the state domain has widths 60 and 50 in the altitude and velocity directions, respectively. Under the 2-norm used in the example, the tightening by $W_{29}$ reduces these widths by $2r_{29}\approx18.7$, corresponding to reductions of approximately 31% and 37%, respectively. This illustrates that the horizon-dependent tightening is already substantial for $N=30$. We therefore selected this horizon as a practical trade-off between improved look-ahead and increased conservatism.

A tube-based MPC formulation with an ancillary feedback controller could modify this propagation by stabilizing the error dynamics. Under suitable conditions ensuring contraction of the resulting closed-loop error dynamics, the corresponding uncertainty sets could remain bounded or grow substantially more slowly. However, the use of an ancillary feedback controller alone does not guarantee such behavior; establishing boundedness or contraction would require additional assumptions and analysis beyond the scope of the present framework.

Neural Network Architecture

We approximate the underlying robust MPC law by a fully connected feed-forward neural network (cf. § 3.1) with 3 hidden layers, each containing 16 neurons and using a hyperbolic tangent activation function. The network takes the state $x$ as input and outputs the control input $u$. This architecture is used for the initial and final training phases described in § 6.4.

Data Generation and Training

We seed 20 814 initial points equally spaced within the state bounds. Using these points, we estimate a robustly forward-invariant set $\hat{\mathcal{X}}_N \subseteq\mathcal{X}_N$ of the underlying MPC using a backwards-pruning approach.

With the same seed points, we perform an initial training of the NN architecture (cf. § 6.3) for 200 epochs with a learning rate of $5 \times10^{-3}$, which yields an approximation error of $\epsilon_D = 1.9876$, rounded to $\epsilon_D = 2$ in the 2-norm.

Before dataset generation begins, we estimate the Lipschitz constants of both the MPC and the neural network on $\hat{\mathcal{X}}_N$ using a power-iteration method based on repeated Jacobian–vector and vector–Jacobian products. For the MPC, we obtain $L_{\mathrm{MPC}} = 19.887$, which we round to $L_{\mathrm{MPC}} = 20$, and for the initially trained neural network, we obtain 21.436, from which we chose $L_{\mathrm{NN}} = 25$ for caution. This results in $\delta= 0.0222$, based on which we set $\eta= \delta$. We then compute an $\eta$-coverage of $\mathcal{X}_N$ by calculating a bounding box of $\hat{\mathcal{X}}_N$ on which a space-filling sampling algorithm is employed.

This results in 1 231 052 coverage points $x_i$, which are used to construct the final training dataset $\mathcal{D}$. These points provide an $\eta$-coverage of $\hat{\mathcal{X}}_N$, i.e.,

Equivalently, the balls of radius $\eta$ centered at the coverage points cover $\hat{\mathcal{X}}_N$. Consequently, two neighboring coverage points whose corresponding balls overlap are separated by at most $2\eta$. This bound applies to neighboring points; it does not impose a distance bound on arbitrary pairs of coverage points. A cutout of the cover is shown in Figure S1

We then train a newly initialized NN on the dataset $\mathcal{D}$. The training concludes after 100 epochs with a learning rate of $5 \times10^{-3}$ and a batch size of 1080, achieving an $R^2$-score of 0.9999 calculated on a validation set of 10 000 data points, which are distinct from any training data points. Moreover, the trained NN achieves $\hat{\epsilon}_D = 1.9222$ and a Lipschitz estimate $L_{\mathrm{NN}}^{\mathrm{an,final}} = 23.22$, both of which satisfy the initial bounds.

All training, sampling, and simulations were performed in Python on a Lenovo ThinkPad T14 Gen 2a running Windows 11, equipped with an AMD Ryzen 7 PRO 5850U CPU and 32 GB RAM. MPC was implemented using IPOPT via CasADi, and the neural network using PyTorch.

Results

We simulate the system in closed-loop, controlled by the robust MPC and its NN approximation for 100 random initial states. Specifically, the initial states are drawn from a uniform distribution over $\hat{\mathcal{X}}_N$. The closed-loop system is simulated for 15 s. We observe that the state trajectories under the robust MPC and the NN controller remain close over the whole simulation time and all converge to a neighborhood of the reference point (Figure S2). Furthermore, we can observe that no constraint violations occur, and that all trajectories stay inside $\hat{\mathcal{X}}_N$. Table S1 shows the mean, maximum, and minimum computation times for both the MPC and NN controller along the 100 trajectories, indicating a significant speed-up using the NN controller. For the same trajectories, Table S2 presents the occurring approximation error in $l_2$-norm, all of which are below our allowed bound of $\epsilon= 3$. The minimal value is given as 0 since the first observed digit is at the seventh decimal point and therefore not clearly distinguishable from numerical noise.

	Mean (ms)	Max (ms)	Min (ms)
MPC	41.79	213.98	24.65
NN approx.	0.52	1.98	0.44

	Mean	Max	Min
Approx. errors	0.1943	1.3497	$7.5449 \times10^{-6}$

The results show how safety and stability guarantees established via the underlying MPC transfer to the neural network controller. Furthermore, the results suggest that Theorem 16 is satisfied along all 100 closed-loop trajectories, indicating that the feedback law of the designed MPC is well-behaved

and that the proposed Algorithm 1 is suited to compute the NN approximator.

Weight-projected neural network training

The Lipschitz constant of the feed-forward neural network introduced in Section 3.1 of the main article can be bounded using the Lipschitz constants of its individual layers. Let $L_{\sigma_j}$ denote a Lipschitz constant of the activation function $\sigma_j$, and let $A_j$ denote the weight matrix of the $j$-th affine layer. Using the submultiplicativity of induced matrix norms and the composition of the network layers, we obtain the analytical upper bound

where $\theta$ denotes the collection of all network parameters and $m$ is the number of weight matrices. Fixed linear input or output transformations contribute additional induced-norm factors to (S1). Equivalently, they can be incorporated into the first or last affine map when evaluating the bound. Since the bias vectors $b_j$ are constant additional factors and therefore do not affect differences between network outputs, they do not contribute to $\widehat{L}_{\mathrm{NN}}$ [10].

To enforce a prescribed bound $L_{\mathrm{NN}} > 0$, the network is trained using projected gradient-based optimization. Starting from the parameters $\theta^{(k)}$, an unconstrained optimizer step with respect to the supervised loss $\mathcal{L}$ first produces tentative parameters $\widetilde{\theta}^{(k+1)}$. These parameters are subsequently mapped back to the feasible parameter set

We employ a uniform radial rescaling of all $m$ weight matrices. The scaling factor is defined as

The updated parameters are then given by

Because induced matrix norms are positively homogeneous, the resulting bound satisfies

Thus, every completed optimization step yields parameters that satisfy the prescribed analytical Lipschitz bound [10].

A soft penalty on violations of $L_{\mathrm{NN}}$ may additionally be included in the training objective to reduce the magnitude and frequency of the rescaling steps. Related projection-based approaches are discussed in greater detail in [10].

Existence of a neural network with prescribed finite-dataset accuracy

This section provides the complete proof of Lemma 9 in the main article. Specifically, we show that, for any finite dataset with pairwise distinct inputs and any prescribed tolerance $\epsilon_{\mathcal{D}} > 0$, there exists a feed-forward neural network whose approximation error at every data point is at most $\epsilon_{\mathcal{D}}$. The result concerns existence and finite-dataset accuracy only. This statement does not imply that a particular training algorithm finds such a network or that the resulting network satisfies an approximation bound outside the training dataset.

PROOF. The proof consists of two steps. First, we show that there exists a continuous function that interpolates the training data. Second, we apply the universal approximation theorem to conclude the existence of a neural network that approximates this interpolant with $\epsilon_{\mathcal{D}}$ accuracy. As a result, this neural network satisfies $\lVert\widetilde{f}(x_i) - y_i \rVert\le\epsilon_{\mathcal{D}}$ point-wise on the training data, which proves the claim.

Step 1: Consider the set $S = \{x_i \mid i \in\{1,\ldots,n_{\mathcal{D}}\}\} \subset\mathbb{R}^d$ consisting of the training inputs. Since $S$ is finite, it is closed. Now, define the function $g : S \to\mathbb{R}^p$ by $g(x_i) = y_i$ for all $i = 1,\ldots,n_{\mathcal{D}}$. We now aim to construct a continuous extension of the function $g$ from the set $S$ to the entire space $\mathbb{R}^d$, yielding the desired interpolant. To this end, we consider the component functions $g_j : S \to\mathbb{R}$, such that $g(x) = (g_1(x),\ldots,g_p(x))$, and each component satisfies $g_j(x_i) = y_i^{(j)}$, where $y_i = (y_i^{(1)},\ldots,y_i^{(p)}).$ As $\mathbb{R}^d$, equipped with any $p$-norm, is a normal topological space and $S$ is closed, the Tietze extension theorem [16 Theorem 12.4] guarantees that each real-valued component function $g_j$ can be continuously extended from $S$ to $\mathbb{R}^d$. We denote the continuous extension of $g_j$ by $q_j$, and define the continuous interpolant $q:\mathbb{R}^d \to\mathbb{R}^p$ of the training data by $q(x)=(q_1(x),\ldots,q_p(x))$.

Step 2: We consider a compact set $\mathcal{K} \subset\mathbb{R}^d$ of interest such that $S \subset\mathcal{K}$. By the universal approximation theorem [13], for any $\delta' > 0$, there exists a neural network $\tilde{f}:\mathbb{R}^d \to\mathbb{R}^p$ such that $\sup_{x \in\mathcal{K}}\|\tilde{f}(x)-q(x)\|_\infty< \delta'$. The result can be extended to arbitrary $p$-norms with $1 \le p < \infty$ by using norm equivalences in finite-dimensional real spaces. Specifically, for any $1 \le p < \infty$, there exists a constant $c_p \ge1$ such that $\|v\|_p \le c_p\|v\|_\infty$ for all $v \in\mathbb{R}^p$, and trivially, $c_p=1$ if $p=\infty$. Thus, and taking $\delta'=\epsilon_D/c_p$, we obtain $\sup_{x \in\mathcal{K}}\|\tilde{f}(x)-q(x)\| < \epsilon_D$, and in particular, $\|\tilde{f}(x_i)-y_i\| \le\epsilon_D$ for all $i$, which concludes the proof.