WorksComputer Science
Inner Product Regularization for Imitation Learning
Abstract
Robots that learn from demonstrations should extrapolate their policy despite state perturbations such as sensor noise, scene variation, and compounding errors. One approach to this robustness is regularization. Prior regularizers consider each state on its own and limit how sensitive the policy is to any perturbation at that state. By contrast, we explore how the policy's output shifts when given the same perturbation at different states. Our insight is that, locally, the robot should respond to state perturbations in a consistent manner: e.g., if changing an irrelevant object position does not impact the policy at one state, that same change should not shift the policy output at a nearby state. Applying this insight, we develop inner product regularization (InneR). We show that the inner product on action space induces an inner product on state space through the policy Jacobian (the Fisher information for stochastic policies), and that the null space of this inner product governs which state perturbations have a first-order effect on the policy. By clustering states whose inner products are aligned, averaging the inner products within each cluster, and minimizing the smallest eigenvalues of that average, InneR shapes the learned policy so that nearby states share a common null space. InneR enforces consistent responses to perturbations without limiting policy expressiveness, and can be appended to existing imitation learning pipelines for deterministic or stochastic policies. In simulated experiments, InneR makes learned policies more robust to irrelevant state dimensions and shifts between training and test distributions, whereas norm penalties often perform worse than behavior cloning. On a real robot arm performing three manipulation tasks, InneR raises the average success rate of a diffusion policy from 18% to 57% without additional data.
Introduction
The goal of imitation learning is to extrapolate a generalized policy from a limited set of expert examples. Consider Figure 1: during training, the robot is shown how to close the coffee machine, and at test time it must generalize that policy across new scenarios, which expose it to states that are not in its training dataset (Spencer et al. 2021). For example, sensor noise shifts the measured pose of objects, the scene itself varies between trials, and small errors in the robot’s actions can compound over time. Regularization techniques are often employed to enhance robustness in the face of these perturbations. Instead of relying on additional data or expert oversight, regularization enforces an inductive bias on the learned policy during training.
Figure 1. (Left) Robot learning to close a coffee machine. At test time the robot encounters a new state. As compared to the states in its training dataset, this new state has some perturbations that are task-relevant (shifting the pose of the coffee machine, ) and some that are irrelevant (shifting the banana, ). (Right) Local state perturbations have a first-order effect on the policy outputs. At each state the policy is sensitive to some perturbation directions and ignores others; averaging over nearby states reveals the directions to which the policy is most and least sensitive. Under standard behavior cloning these responses are inconsistent across nearby states, and thus the policy reacts to both and . We propose InneR, an inner product regularization approach that enforces similar responses to the same perturbation at nearby states: the policy still responds to but now ignores .
Existing regularizers try to limit how sensitive the policy is to changes in state. For instance, one standard approach is to minimize the norm of the model’s parameter vector [24, 28]. Other recent techniques penalize the norm of the policy Jacobian with respect to state [20, 43], or, for stochastic policies, the Fisher information of the action distribution with respect to input [17] (Shen et al. 2019). These penalties share a common structure: each state is treated on its own, and the policy is made less sensitive to every perturbation at that state. But not every perturbation should be ignored, e.g., a policy that is robust to moving clutter in the background should still respond when its target object shifts to the right. Uniformly shrinking the policy’s sensitivity dampens both responses, fundamentally limiting the robot’s ability to control each of its action dimensions.
We propose an alternate perspective on regularization in behavior cloning. Instead of asking how much the policy changes at a given state, we focus on how the policy responds to the same perturbation at different states. Our intuition is that — at least within a local neighborhood — the robot
Corresponding author: Benjamin A. Christie, Department of Mechanical Engineering, Virginia Tech, Blacksburg, VA, 24060, USA. Email: [email protected]. should respond to perturbations in a consistent manner. For example, if a shift in lighting does not affect the robot’s policy at one state, the same shift should not affect the robot’s policy at a nearby state. Conversely, if a perturbation does substantially impact the task (e.g., the target object moves), then the robot’s policy should change at both states. This intuition implies structure, and therefore regularization, for the learned policy:
The first-order response to state perturbations should be locally consistent across the policy.
Applying this insight, we develop inner product regularization for imitation learning. The designer starts by choosing an inner product on action space to measure how different two actions are. We show that — pulled back through the policy — the inner product on action space induces an inner product matrix on state space, and this state-space inner product governs which local perturbations have a first-order effect on the policy outputs. The same construction extends to stochastic policies, where the Fisher information of the action distribution with respect to state plays the role of the inner product matrix. Perturbations in the null space of this matrix are ignored by the policy, while perturbations in its row space remain influential. Our insight then becomes a condition on this null space: nearby states should share a common null space, making their responses to perturbations locally consistent.
We propose inner product regularization (InneR) as a tool for enforcing this local null space consistency. InneR works alongside standard imitation learning algorithms by clustering the states in each training batch according to their inner product alignment, averaging the inner product matrices within each cluster, and then penalizing the shared null space. When we average the inner product matrix across a set of states, the smallest eigenvalues of that average measure how far the states are from sharing a common null space. Regularizing the policy to minimize these eigenvalues therefore encourages consistency, and this consistency does not come at the cost of expressiveness. Since the regularizer is zero for any family of full-rank Jacobians with a common row space, InneR can align the directions that the policy ignores without reducing the policy’s sensitivity to the directions that matter.
In summary, we make the following contributions:
Connecting local perturbations to the inner product on state space. We show that the inner product on action space induces an inner product matrix on state space through the policy Jacobian, and that the null space of this matrix characterizes which local perturbations have no first-order effect on the policy. For stochastic policies, we show that this matrix is the Fisher information with respect to state.
Aligning the null space using active subspace learning. We show the smallest eigenvalues of the expected inner product are zero exactly when the individual inner products share a null space. This yields an eigenvalue regularizer that enforces consistent responses to perturbations without reducing the rank of the policy. This method contrasts with commonly-employed Frobenius and nuclear norm penalties on the Jacobian.
Localizing the regularization by inner product alignment. We implement our theory with the InneR algorithm for deterministic or stochastic policies. To encourage local consistency, InneR first measures the geometric alignment between inner product matrices. InneR then clusters states which are already aligned, and regularizes each cluster separately so that the relevant state directions can change over the course of a task.
Comparing to alternative regularizers. We compare InneR to unregularized and regularized imitation learning frameworks across multiple model types in controlled simulations and in the real world.
Related Work
We develop a metric-based regularizer for imitation learning that shapes the state space inner product so that nearby states reject the same perturbations.
Robustness in Imitation Learning. A central challenge in imitation learning is covariate shift ((Spencer et al. 2021)): at test time the robot encounters states outside of its training dataset, and small errors compound over the task [42]. Many approaches address this shift with more data, either by querying a human for corrections online [42, 21] or by injecting noise into the demonstrations [25, 36]. Because policies readily latch onto task-irrelevant features [13], a second line of work learns task-relevant state representations by backpropagating loss gradients [33] or applying image perturbations [34]. A third line treats the learner as a dynamical system and bounds the growth of errors with control theoretic tools [29, 35, 6]. Our work requires neither additional data nor a latent representation: we regularize the policy directly in the state space it acts upon.
Distance Metrics. A core idea behind our approach is that a distance in action space induces a distance in state space: states that the policy maps to similar actions are functionally equivalent and therefore “nearby.” In reinforcement learning this idea underlies state abstractions that aggregate states with the same optimal action [27] and bisimulation metrics that compare states through their rewards, transitions, or action distributions, and then shape a learned representation accordingly [15, 1] (Zhang et al. 2021). Recent research has also begun to leverage this idea in imitation learning, clustering states that lead to similar action sequences for robustness to changes in viewpoint and appearance [26]. Rather than learning a latent space, we regularize the pullback of the action space metric through the policy itself. The same pullback appears in Riemannian motion policies [38] and in the geometry of deep generative models, which pull the output metric or its Fisher-Rao counterpart back through the decoder [4, 5, 3]. In these prior works the pullback metric is computed through a given map, whereas we shape that metric during training.
Jacobian Regularization. In practice, our inner product shaping is a form of regularization on the policy Jacobian with respect to state, whose singular values measure the policy’s sensitivity to local perturbations. Penalizing its magnitude dates back to double backpropagation [14] and contractive autoencoders [40], and remains standard in supervised learning [20], reinforcement learning [8], and imitation learning (Wu et al. 2025; Xie et al. 2026). The magnitude is often measured by the Frobenius norm [20] (Xie et al. 2026) or by the nuclear norm [43]. For stochastic models, the analogous quantity is the Fisher information with respect to the input, whose trace or largest eigenvalue can be penalized directly (Shen et al. 2019; Glocker et al. 2023). Kullback-Leibler (KL)-based smoothness penalties regularize the same matrix implicitly [30] (Zhang et al. 2020). These penalties can be enforced locally, at individual states [23, 31], or globally, through constraints on the network weights [18]. Closer to our insight, tangent propagation makes a model invariant along state directions that are specified by the designer [41] (Simard et al. 1991).
Our approach fundamentally differs by targeting the expected inner product across neighboring states: norm penalties indiscriminately reduce the policy’s sensitivity and rank, limiting the robot’s ability to control independent action dimensions, whereas our method preserves action rank and learns which directions to ignore. This matrix is the vector-valued analogue of the expected gradient outer product, whose leading eigenvectors span the input directions a learned function depends on [37] and which active subspace methods use for post-hoc dimensionality reduction [11] (Zahm et al. 2020). We instead adapt this geometric framework into a training regularizer that penalizes the trailing eigenvalues, enforcing a perturbation-rejecting null space in the policy Jacobian at neighboring states.
Problem Statement
We consider a standard imitation learning problem where a robot tries to learn a control policy from a dataset of offline expert demonstrations.
States and Actions. Let and denote the state and action spaces, with states and actions . We focus on continuous environments where and are subsets of finite-dimensional inner product spaces. Furthermore, we assume that the state dimension exceeds the action dimension (). In our experiments the robot has direct access to a state vector (e.g., we do not consider image state observations); the state could be measured using object detection, segmentation, or pose estimation methods [39, 7] (Wen et al. 2024). Our actions consist of robot position or velocity commands.
Dataset. The robot learns from an offline set of expert task demonstrations. Specifically, the robot is given a dataset of states and the corresponding actions taken at those states. We let denote the distribution of states visited by the expert, so that the states in are samples from distribution .
Policy and Loss Function. Based on the dataset, the robot extrapolates a control policy that maps from states to actions. For now we will assume that this policy is deterministic (§ 4), but later we will also explore stochastic policy variants (§ 5). The control policy is instantiated by a differentiable model (e.g., a neural network) parameterized by weights . We assume that is defined on an open set containing , and that the policy is differentiable with respect to both its input and weights . We learn these weights to minimize a loss function across the dataset. For example, in behavior cloning the deterministic policy is trained to match the actions in the dataset:
Our approach is agnostic to the specific choice of this loss function. We simply assume that the robot trains to minimize loss across the given dataset .
Perturbations. For any state in the dataset , the robot knows exactly what action it should take. But at test time the robot will inevitably experience new states , where perturbation represents a shift due to to sensor noise, environmental variations, compounding execution errors, or unseen environment configurations. We specifically consider local perturbations where the state is slightly modified; here is the Euclidean norm and is small enough that a first-order Taylor expansion of around is accurate. Our goal is to ensure that the policy is robust to these local perturbations. Put another way, the robot should not change its actions if a state perturbation is irrelevant to the current task, but it should remain sensitive to task-relevant changes to the state.
Inner Product Regularization: Deterministic Policies
Our insight is that — if a given perturbation has no impact on the policy at state — that same perturbation should not impact the policy at a nearby state . We can therefore improve robustness to local perturbations by regularizing the learned policy to enforce this spatial consistency. Here we formalize policy responses to local perturbations from a geometric perspective. We use the inner product on action space to induce an inner product matrix on state space: this inner product matrix determines what local perturbations have a first-order effect on the policy output (§ 4.1). Next, we apply principles of active subspace learning to shape the null space of the expected inner product across the state space (§ 4.2). We reach an eigenvalue condition that, when satisfied, causes the policy to reject similar perturbations at nearby states to the first order.
Inner Products in Action and State Spaces
Let be the policy’s predicted action for the current task at state , and let be the policy’s predicted action for the current task at a new state . We want to know whether perturbation has a meaningful impact on the policy’s behavior—i.e., does adding cause the model’s predicted action to change?
Inner Product on Action Space. To answer this question, we first need to know the difference between actions and . We therefore define a metric on the action space. We choose this metric to be induced by an inner product on , so that it takes the form:
where is the symmetric and positive definite action inner product matrix. Often designers set as the identity matrix; but one can also tune to alter the relative sensitivity of different action dimensions, so that small changes cause a large increase in distance .
Induced Distance in State Space. Metric quantifies the distance in action space between and . Next, we apply this metric to understand the distance in state space between and . We define as the pullback pseudometric induced on the state space by via the policy :
Under pseudometric states are close together if they map to similar actions, and states are far apart if they map to dissimilar actions. Note that this distance in state space is inherently task-specific: as the policy changes for different tasks, different states become closer together or farther apart. Put another way, the distance pseudometric is a function of both the chosen inner product on action space as well as the task-specific policy the robot is learning.
Because is a metric, by definition the induced satisfies non-negativity, symmetry, and the triangle inequality. However, fails the identity property because the policy maps a continuous higher-dimensional state space into a lower-dimensional action space (), and therefore is inherently non-injective. We conclude that is a pseudometric with for some , and therefore there exist some perturbations that do not change the resulting action, i.e., .
Inner Product on State Space. To determine which perturbations do not impact the policy’s behavior, we focus on the pseudometric . Starting with the expression and applying a first order Taylor Series approximation to the robot policy, we obtain:
where perturbation and is the Jacobian of the policy with respect to state evaluated at a given state :
We note that the first order approximation in Equation (4) is increasingly accurate when and the perturbations become increasingly small.
Plugging Equation (4) back into Equations (2) and (3), we obtain a local approximation of the distance metric:
Here is the symmetric and positive semi-definite inner product matrix on the state space:
Strictly speaking, because is only positive semi-definite, it defines a degenerate inner product (and is a seminorm). Recalling that is the constant inner product matrix on the action space, is now the induced inner product matrix on the state space, and the mapping between these two inner products is determined by the policy Jacobian at a given state . The inner product also reflects how distances in state space depend upon the task: when the robot learns a new policy , it changes the Jacobian , and this yields a different inner product matrix .
Interpreting the Inner Product. The inner product matrix in state space determines how the policy reacts to perturbations. By construction, has a null space of at least dimension (i.e., state dimension minus action dimension): because is positive definite, the null space of equals the null space of , and . The null space has dimension exactly when has full row rank , and local perturbations that lie in this null space have no first-order effect on the policy’s actions. Conversely, local perturbations with a non-zero component in the row space of have a non-zero first-order effect on the policy outputs. Because depends on Jacobian , these null and row spaces may vary across both the system state and the intended task.
Active Subspace Learning
In Equation (7) we presented the inner product matrix that determines which perturbations have a first-order impact on the policy’s behavior. In what follows, we explore how to regularize so that the policy is robust to irrelevant perturbations while still generalizing task-relevant actions to previously unseen states. Our approach draws from active subspace learning Constantine et al. [11], a technique that seeks to identify and reject the input directions which have a negligible impact on the model’s output.
Connecting the Inner Product to Policy Robustness. To understand how to regularize , we apply our core insight: if a perturbation does not impact the policy output at state , that same perturbation should not impact the policy output at states near . Written in terms of our metric on state space, for every perturbation :
where and are nearby states, e.g., states within some region . We will formalize these local regions in Section 6. For now — to better introduce our regularization concept — we simply treat and as arbitrary states in .
One naive method for satisfying Equation (8) is to constrain to be constant across . If , then the policy has a consistent response to perturbations. But by forcing to be invariant to state, we also constrain the policy to have an identical Jacobian across (up to an isometry in the action space), and this prevents the policy from learning nuanced behaviors. Rather than regularizing the entire inner product , we instead focus on its null space, since local perturbations which lie in this null space have no first-order effects on the policy. More specifically, Equation (8) holds for every perturbation exactly when the null spaces of both and coincide. We can therefore enforce our core insight and make nearby states robust to similar perturbations by regularizing to maintain a consistent null space across subsets of the state space.
Expected Inner Product. In what follows we outline how to perform this null space regularization on the state space inner product matrix. For clarity of exposition, here we will assume that we want to enforce a consistent null space across all the states in dataset . To do so, we start averaging the inner product matrix across these individual states:
By construction, is symmetric positive semi-definite. Because each is also positive semi-definite, if and only if for every state in the average. Hence the null space of is exactly the intersection of the individual null spaces, and the rank of grows as these null spaces become less aligned; if they share no common direction, is strictly positive definite.
Eigenvalues of the Expected Inner Product. We next extract the eigenvalues of sorted from largest to smallest:
Because there are action dimensions, the eigenvectors associated with the first eigenvalues span the active subspace, i.e., the state directions to which the policy is most sensitive on average; these eigenvalues are not part of our regularization. By contrast, the smallest eigenvalues measure how far the individual null spaces are from sharing a common -dimensional subspace. When , then all the inner products must share an identical null space and Equation (8) is satisfied.
This result provides our main theoretical takeaway. To shape the imitation learning policy so that it is consistently robust to the same state perturbations, we can compute the expected inner product matrix across (sections of) the state space, and then regularize the policy during training to minimize the smallest eigenvalues of that expected inner product matrix. Because the null space and row space of are orthogonal complements, sharing an -dimensional null space is equivalent to sharing an -dimensional row space. Accordingly, the policy can still change how it maps the relevant state directions to actions as the state varies, but the set of irrelevant state directions remains fixed.
Connection to Jacobian Regularization. This construction has clear connections to existing Jacobian regularization techniques. Consider Equation (7): because is a symmetric positive definite matrix, the null space of matrix is identical to the null space of Jacobian . Methods that regularize or the null space of may therefore also regularize the null space of the inner product matrix. To better connect this Jacobian regularization with our inner product approach, below we will assume that . Existing works regularize either the Frobenius or nuclear norm of the policy Jacobian:
Frobenius Norm . Regularizing the squared Frobenius norm of is equivalent to minimizing the sum of the eigenvalues of . In other words, is equivalent to the nuclear norm .
Nuclear Norm . Regularizing the nuclear norm of is equivalent to minimizing the sum of the square roots of the eigenvalues of . This increasingly pushes the smaller eigenvalues of towards zero as compared to the squared Frobenius norm.
Note that each individual policy Jacobian has at most rank and a null space of at least dimension . This null space already exists, and neither regularizer aligns it across states. The squared Frobenius norm shrinks every singular value of in proportion to its size, uniformly reducing the policy’s sensitivity in all directions, including task-relevant ones. The nuclear norm regularizer drives smaller singular values of to zero more strongly. However, neither outcome is necessarily desirable. We typically want the rank of to be , since this means that changing the state can separately affect each individual action dimension (e.g., the different joint or gripper positions). When we increase the dimension of the null space of individual matrices (e.g., through the nuclear norm), we are actually decreasing the rank of , and limiting how meaningful perturbations can guide the policy output. By contrast, our regularizer only penalizes the components of that act on a shared subspace: it is zero for any family of full-rank Jacobians with a common row space, and therefore does not require reducing rank.
At a higher level, applying regularization to individual matrices does not encourage consistent responses to state perturbations. The policy may have completely different responses to the same perturbation at nearby states and , and separately regularizing and does not penalize this. The eigenvalues in Equation (10) can only emerge with the expected inner product matrix (the equivalent eigenvalues are all zero for ), and this fundamentally separates our methodology from approaches that regularize individual policy Jacobians.
Inner Product Regularization: Stochastic Policies
Thus far our proposed inner product regularization has explored deterministic policies of the form . However, state of the art imitation learning policies are often stochastic (Urain et al. 2026). We will specifically focus on Gaussian policies, normalizing flows, and diffusion policies. Instead of outputting a single action for a given state, these stochastic policies define a state-dependent probability distribution over actions:
Just like their deterministic counterparts, stochastic policies should be robust to local perturbations. Consider two nearby states and as well as a perturbation . If perturbation has no impact on the policy’s action distribution at state , then that same perturbation should not change the policy’s action distribution at state .
Our strategy for regularizing stochastic policies parallels the geometric analysis from § 4. We will use divergence in the action distribution to induce an inner product matrix on state space, average that inner product across the dataset, and then regularize its smallest eigenvalues to enforce a shared null space. The key difference is how the inner product is computed. In § 5.1 we show that the stochastic inner product in state space is actually the Fisher information matrix . To better connect this result to , and to move towards practical implementation, we work through examples with Gaussian and diffusion policies in § 5.2.
The Fisher Information Matrix as an Inner Product
We begin by formalizing the distance between two action distributions. Let be the current state, and let be a nearby state resulting from perturbation . To quantify the difference between the action distributions produced by the policy at and , we use Kullback-Leibler (KL) divergence:
Like Equation (3), KL divergence asserts that two states are close together if they map to similar action distributions, and far apart if their action distributions diverge. Unlike Equation (3), here is not a pseudometric: it is not symmetric, but both directions agree to second order and behaves locally like a squared distance as we show in Equation (16).
Although KL divergence itself is not a metric, we can obtain a squared distance on state space by applying a Taylor series approximation to Equation (12). Define as the gradient of the log probability of an action with respect to state :
and let be the corresponding Hessian. Expanding in Equation (12) to second order around , the approximation becomes:
We can further simplify Equation (14) by recognizing that the first order term is zero due to the fact that is a global minimizer of . The second order term can be rewritten with the Fisher information identity, . This leaves us with:
Interestingly, the Hessian of the KL divergence with respect to the perturbation is the Fisher information matrix of the action distribution with respect to the state . This version of the Fisher information matrix treats as the parameter of the distribution, as opposed to the more common Fisher information with respect to the model weights . Matrix quantifies how sensitive the policy’s probability distribution is to changes in state . Rewriting Equation (15), defines an inner product matrix on the state space for stochastic policies:
where is a symmetric positive semi-definite matrix. Comparing Equation (16) to Equation (6), we see that is analogous to . Local state perturbations which lie in the null space of do not change the policy’s action to the first order; similarly, state perturbations which lie in the null space of do not change the policy’s action distribution to the first order.
Expected Inner Product and Eigenvalues. Given the analogy between and , the next steps for active subspace regularization follow from § 4.2. To ensure that nearby states are robust to the same perturbations, we first calculate an average inner product . We then extract and minimize the smallest eigenvalues of , where is the estimated rank of . The rank of is often greater than action dimension , since a state perturbation can change the shape of the action distribution (e.g., its covariance) and not only its mean. Overestimating penalizes fewer eigenvalues than the shared null space should contain, which weakens the regularizer. Underestimating demands more zero eigenvalues than desirable, causing the regularizer to penalize relevant state dimensions and compete with the imitation loss . In our experiments we therefore choose conservatively (); alternatively, can be inferred from the spectrum of for a briefly trained policy. We leave this adaptive estimate of for future work.
Connection to Fisher Information Regularization. Prior work has regularized the Fisher information with respect to the model input to make classifiers and conditional density estimators robust to adversarial perturbations, e.g., by suppressing its largest eigenvalue (Shen et al. 2019) or penalizing its trace [17]. Like the Jacobian penalties in § 4.2, these methods regularize each on its own and lower the penalty by shrinking the policy’s sensitivity. InneR differs in two ways. First, we are regularizing an averaged instead of individual matrices. Second, we target direction rather than magnitude: the loss in Equation (26) keeps the policy highly sensitive to task-relevant perturbations.
Application to Gaussian and Diffusion Policies
Here we apply the analysis from § 5.1 to Gaussian and diffusion policies. We derive an exact or estimated Fisher information matrix, and show how our approach applies to these common imitation learning frameworks.
Relating Stochastic and Deterministic. We first study a Gaussian setting to better connect the state-space inner products for stochastic policies, , and deterministic policies, . Consider a policy of the form: . Here is a Gaussian distribution over actions with state-dependent mean and constant covariance . Let be the Jacobian. This Jacobian is equivalent to Equation (5) where the deterministic policy output is replaced by the mean action. Applying Equation (13), we have that:
Plugging this value for into Equation (17) and taking the expectation, the Fisher information matrix is:
Gaussian policies therefore provide a setting where the inner product can be exactly computed. Comparing Equation (19) with Equation (7), we find that — for Gaussian policies with constant covariance — the inner products are equivalent when . Recall that matrix specifies the relative sensitivity of different action dimensions. The inverse covariance is functionally equivalent: relatively smaller or larger values in indicate which action dimensions should have a tighter distribution, and inverting this matrix makes Equation (19) increasingly sensitive to changes in action dimensions with the least variability. Hence, Gaussian policies provide an example of the direct relationship between the inner products and .
Diffusion Policies. We next outline how we estimate the inner product matrix for diffusion policies [9]. Here the policy predicts a chunk of future actions based on the last states. Accordingly, the policy input is the stacked state sequence with , where is the dimension of a single state. Unlike Gaussian policies, a diffusion policy does not provide its likelihood in closed form, and thus we cannot directly compute the gradient from Equation (13).
In our experiments we approximate using the evidence lower bound (ELBO) of the policy’s reverse diffusion process. Let be the denoising loss, where is a diffusion step, is the cumulative noise retention parameter, and is the added noise [19]. The negative ELBO is a weighted version of this loss:
where the weights are fixed by the noise schedule and is independent of state . Since the ELBO is a tractable lower bound on , we use its gradient as our estimate:
In practice we evaluate the expectation over in Equation (21) by importance sampling: is drawn from a proposal distribution that favors low-noise steps (where the action distribution is more accurate), and each draw is weighted by . This importance sampling approach leaves Equation (21) unbiased while lowering its variance.
Once is estimated as shown above, we use it to compute the Fisher information matrix following Equation (17), where the expectation over actions is replaced by an average over actions sampled from the current policy, . We require so that the estimated can reach rank . Because Equation (21) is evaluated with a finite number of samples, the estimate of is noisy, and the outer product in Equation (17) adds the covariance of this noise to . This bias is positive semidefinite and shrinks as more samples are used. It vanishes along any direction in which the denoising loss is insensitive to the state for every , so minimizing the estimated eigenvalues enforces a stronger version of our desired null space condition. The error of our ELBO approximation is the gradient of the bound’s gap, which vanishes when the learned reverse process exactly inverts the forward noise process.
Each of the sampled actions uses draws of , and each draw requires one forward and one backward pass through the denoiser, so estimating costs passes through the diffusion model per state. The actions themselves are sampled via several steps of the reverse process without tracking gradients; we never need to differentiate through the sampling process when computing Equation (21).
InneR Algorithm
Sections 4 and 5 theoretically outline how regularizing the state space inner product can make a learned policy robust to local perturbations . Equations (9) and (10) provide the essential steps: averaging the inner product over individual states, and then minimizing the eigenvalues which correspond to the shared null space. However, thus far we have only discussed applying these steps across the entire dataset . This naive implementation would encourage the policy to learn a consistent null space across all states — i.e., encouraging every state to be robust to the same perturbations. Global consistency makes sense for tasks where the robot must always attend to the same state elements (for example, picking up a block). But, global regularization is undesirable when the relevant state information changes over the course of the task (for example, picking up a block and then stacking it on a target).
Our inner product regularization algorithm (InneR) addresses this challenge by first clustering the dataset into local regions before shaping each region’s average inner product. In Section § 6.1 we obtain local regions using inner product alignment. For simplicity of explanation we focus on the inner product for deterministic policies, but the same steps apply to with replaced by (§ 5.1). Next, in § 6.2 we present the resulting InneR algorithm block. We highlight that InneR can be appended to existing imitation learning architectures, and we provide guidance for parameters which the designer may need to tune.
Clustering Based on Inner Product Alignment
To motivate localized regularization we start with an example. Consider a table-top setting with two blocks and one cup; the system state could be broken into components for the robot and each object, i.e. . Imagine that the robot is learning to “stack block1 on top of block2.” Throughout the entire motion the robot’s actions should remain invariant to the cup’s state. At the start of the task the robot must focus on , and modifications to can be ignored. But once the robot grasps and lifts block1, then perturbations to ought to alter the robot’s actions . Hence, whether a change in is relevant depends on the current context.
We refine our key insight in order to address this challenge. Within § 4.2 we argued that if a perturbation does not impact the policy output at state , that same perturbation should not impact the policy output at states near . Now we reverse that logic and define two states as nearby if they are both robust to similar perturbations. Under this definition, nearby states need not be close in ; rather, they are states at which the policy attends to similar state directions. We propose to cluster states based on the alignment between their inner product matrices . Groups of states that have the most aligned inner products are treated as regions, and we locally apply our regularization to these regions to further increase their shared null space alignment.
Inner Product Alignment. Alignment refers to the relative spaces spanned by matrices, and not the magnitude of those matrices. To determine whether two arbitrary inner product matrices are aligned, we first compute the thin singular value decomposition of their corresponding Jacobians:
Here the columns of are the right singular vectors of associated with its nonzero singular values. By construction the columns of and span the row spaces of and , and we can use the Jacobian to obtain without forming the inner product matrix. We next calculate the chordal Grassmann distance between the two row spaces ([12]). This distance depends only on the principal angles between two subspaces, independent of the specific basis vectors used to represent them. Up to a constant factor, the chordal Grassmann distance is:
where is the Frobenius norm and defines an orthogonal projection matrix for the row space. Combining Equations Equations (22) and (23) quantifies the alignment between the row space of the inner product at different states.
Clustering and Regularization. Equipped with our ability to measure inner product alignment, we can now cluster states that have the most similar null spaces. For each state in dataset we compute . We then apply Equations Equations (22) and (23) to get the pairwise distances with every other inner product matrix (resulting in a total of pairwise distances). Multiple clustering strategies are viable at this point, and our InneR algorithm is not tied to any specific approach. However, because our experiments leverage -nearest neighbors, we will focus on that method in what follows. From the pairwise distances we identify the states with the lowest distance from (including itself, at distance zero), i.e., the inner products that are best aligned with . This defines a local region . When performed in parallel across the dataset, we obtain regions , where the -th region consists of state and the other states with the best aligned inner products. In practice we form these regions within each training batch rather than across the full dataset: the pairwise distances are computed between the states of the batch, and the regions are recomputed at every gradient step as the policy changes (§ 6.2).
Each region contains the inner products that are most closely aligned with . Our local regularization approach seeks to augment their alignment by applying the active subspace learning from § 4.2. Building on Equation Equation (9), we compute the average across every local region :
We then apply Equation Equation (10) and extract the sorted eigenvalues for each of these averaged inner products:
The shared null space for the -th region corresponds to the last elements . Accordingly, to shape a policy so that it is locally robust to similar perturbations, our regularization loss penalizes the fraction of each region’s inner product that lies in the shared null space:
where is the stop-gradient operator which treats its argument as a constant when differentiating. We note that eigenvalues are a function of the policy weights . Each term in Equation Equation (26) lies between 0 and 1 regardless of the scale of the policy Jacobian. This normalization matters: the raw eigenvalues $\lambda_{m+1}^i,\ldots,\lambda_n^i
Algorithm 1 InneR for Deterministic Policies
Require: Dataset , policy , action space inner product , neighbors , regularization weight
while not converged do
Sample batch
Compute base loss across batch
for each state in batch do
▷ state Jacobian
▷ get inner products
with no-grad do ▷ cluster by alignment
right singular vectors of
▷ Grassmann distance
indices of the smallest (including )
end with
for each state in batch do
▷ average inner product
eigenvalues of
▷
▷ penalize the shared null space
▷ augment base loss
Update policy parameters to minimize loss
can be driven to zero simply by shrinking everywhere, which is the behavior of existing norm penalties in § 4.2 that we seek to avoid. Dividing by the total removes this incentive, and detaching the total prevents the loss from being lowered by inflating the leading eigenvalues instead of reducing the smallest ones. Because the denominator is the trace of , the normalization adds no computational cost, and it also makes the weight (described below) more comparable across tasks and policy classes. To combine our regularizer with existing imitation learning algorithms we simply augment the given loss . The updated loss adds our shared null space penalty:
where is a hyperparameter that determines the weight of the regularizer. We discuss this (and the effects of other parameter selection) in § 6.2.
Extension to Stochastic Policies. Within the steps above we focused on deterministic policies with inner product . However, the same alignment and clustering approaches can be applied to stochastic policies by replacing with and with (§ 5.1). Matrix then contains the leading eigenvectors of , which we obtain as the leading right singular vectors of the matrix of sampled scores without forming , and Equation (26) penalizes the smallest eigenvalues of each relative to its detached trace.
Algorithm and Implementation
An outline of InneR is provided in Algorithm 1. After computing the base loss for the given imitation learning method, InneR adds the following steps: calculating the Jacobians and inner products for each state in the batch (lines 4-6), clustering the states based on inner product alignment (lines 7-10), and then computing the fraction of each region’s inner product that lies in a shared null space (lines 12-16).
Hyperparameter Tuning. InneR introduces three parameters the designer must set: , , and (and, for stochastic policies, the rank from § 5.1). Matrix should default to identity. If the designer wants to make the policy more sensitive to mistakes in a specific action dimension (e.g., opening or closing a gripper), then the designer should increase the relative value of that diagonal element of . Scalar is the number of nearest neighbors for clustering. As increases to the batch size , InneR encourages a globally consistent null space; conversely, as the regions become increasingly local. In our experiments we set . Finally, is the weight of the added InneR loss. Because the loss is normalized in Equation (26), designers should not need to significantly change when working with different state dimensions. We found to be effective in single-integrator simulations and hardware experiments, and to be effective in drone-landing simulations. Larger values let the regularizer dominate early in training — before the policy has learned which state directions matter — and can collapse every state onto a single shared null space.
Computational Cost. InneR increases the time required for training an imitation learning policy. In our testing with a 16-dimensional state in § 7, InneR increased the training time by 6.5x on a multilayer perceptron (MLP) and 5x on a diffusion policy. The added cost has three sources. First, the inner product at every state of the batch requires additional passes through the policy with the computation graph retained: backward passes to form for the MLP, and forward and backward passes of the denoiser (plus sampling the actions) to estimate the scores for the diffusion policy. Second, because depends on these derivatives, its gradient must be propagated back through them, i.e., a second-order derivative of the policy; this backward pass is the single most expensive step for both policy classes (roughly a third of each gradient step’s complexity). Finally, clustering and regularizing the batch require singular value decompositions, subspace distances, and eigendecompositions of matrices. For a deterministic MLP policy, the additional derivatives and linear algebra computations each account for roughly half of the added time; with a diffusion policy the score estimate accounts for roughly three quarters.
Single-Integrator Simulations
We begin by conducting simulation experiments in a point-mass environment to investigate InneR’s behavior in a controlled setting. All code for reproducing these simulations is available on our project site: https://collab.me.vt.edu/inner.
Setup. We study a goal reaching task where the robot is modeled as a single-integrator with linear dynamics. The state dimension is ( is varied in some simulations) and the action dimension is . The state concatenates the position of the robot ( entries), the position of the goal ( entries), and irrelevant noise entries. The action is the robot’s displacement: at each timestep the robot’s position is incremented by while the goal and noise remain fixed. During our experiments the initial state is uniformly randomly sampled from for both data collection and testing. In each demonstration the expert shows a trajectory of 40 state-action pairs that move directly from the start to the goal at a speed of at most one unit per timestep (the total number of state-action pairs in is equal to 40 times the number of demonstrations). At test time the robot is similarly given 40 timesteps to reach the goal using its learned policy. We report final state error, the average distance between the robot’s final state and the goal across 1000 random test starts.
Static and Dynamic Goals
There are two versions of the point-mass environment. In static the goal position always occupies the same entries of the state. In dynamic there are two goals in the environment, and the correct goal for the current trial is signaled by the sign of the final state element: if this switch is positive the robot should reach the first goal, and if negative the robot reaches the second. The second goal occupies the entries after the first, so that entries remain noise. Static is an example of a task where the policy’s response to state perturbations can be globally consistent (i.e., the same state elements are always relevant or irrelevant). By contrast, in dynamic the relevant elements of the state change in different regions, motivating a local regularization approach.
MLP and Diffusion Policies. For each experiment we test a deterministic policy (MLP) and a stochastic policy (Diffusion). MLP has two hidden layers of width 256. Diffusion is a denoising diffusion model with steps (Ho et al. [19]; Chi et al. [9]); its noise prediction network has three hidden layers of width 256 and outputs a single action per state (i.e., and in § 5.2). We sample actions with 10 DDIM steps at test time, and estimate the Fisher information in Equation (17) with sampled actions and draws of per action. Both MLP and Diffusion policies are trained with Adam (Kingma and Ba [22]) and a batch size of : the MLP for 200 epochs at a learning rate of , and the diffusion policy for 1000 epochs at . These settings are held constant across methods and experiments.
Baselines. We test InneR with , , and for MLP; for Diffusion we increase and conservatively set . InneR is compared to four baselines: standard behavior cloning (BC), Frobenius norm regularization (Hoffman et al. [20]), Nuclear norm regularization (Scarvelis and Solomon [43]), and InneR-C. All algorithms are given the exact same training data, model architecture, training hyperparameters, and start states at test time. For Frobenius and Nuclear the loss is the BC loss plus 0.01 times the average of the squared Frobenius norm or nuclear norm of each individual Jacobian. For the diffusion policy we use analogous quantities, the trace of (weighted by ) and the sum of the square roots of its eigenvalues (weighted by ). We selected these weights by sweeping over several orders of magnitude; larger weights monotonically increased the final state error. InneR-C is an ablation of our method without clustering. In practice, InneR-C is equivalent to Algorithm 1 with so that each local region is the entire batch. We repeat every experiment over 10 random seeds; the figures report the mean and standard error across these seeds, with the colors of Figure 2 used throughout.
Figure 2. Single-integrator simulation from Section 7.1. Plots show the error between the robot and the goal as state dimension increases (lower is better). The far left of each plot corresponds to the minimum state dimension for that setting, and each added state element is irrelevant noise that is sampled uniformly at random in both the demonstrations and the test states. Ideally the robot is robust to these variations in the irrelevant state dimensions, and so the final state error should be constant in . InneR is the only method whose error stays nearly flat in every setting besides Diffusion–Dynamic, where it is consistently lower than the alternatives while maintaining a similar positive slope. InneR-C matches it in static, where one global null space is correct, but fails in dynamic. Shaded regions show standard error across 10 seeds.
How Does InneR Scale with Irrelevant State Dimension?
In Figure 2 we plot final state error as a function of the number of state dimensions, where every added state coordinate provides information irrelevant to the task. For static there are irrelevant state dimensions, and for dynamic there are irrelevant dimensions (since the goal not selected is irrelevant). We fix the number of demonstrations at a value where BC does not solve the task (100 and 400 demonstrations for the MLP on static and dynamic, and 50 and 200 for the diffusion policy). The error of BC grows steadily with every irrelevant dimension (for the MLP on static, from 0.5 at to 7.4 at ), and the norm baselines are worse than BC at every . InneR stays nearly flat (growing only from 0.2 to 0.9) and is the only method below BC in both dynamic settings. As expected, we find that InneR-C is effective in static where the same state dimensions are always relevant, but falls short in dynamic where the relevant perturbations change across the state space.
How Does InneR Scale with Dataset Size?
In Figure 3 we focus on the static task, where a single shared null space is correct for every state, and plot final state error as a function of the number of demonstrations (on dynamic the methods order as in Figure 2). InneR and InneR-C bring the error below one unit with 100 demonstrations for the MLP and 50 for the diffusion policy, whereas BC needs 400 and 100. To better understand these results, we also plot what each method is actually regularizing. We see that the norm baselines shrink the squared Frobenius norm of the Jacobian far below its value under standard BC, whereas InneR leaves this norm at or above the BC value (the trace of for the diffusion policy behaves the same way, though it is not shown for brevity). On the other hand, InneR minimizes the fraction of the averaged inner product that lies in the shared null space, which the norm baselines actually increase. These results indicate that InneR effectively retains the expressiveness of the given policy while regulating its shared null space.
Figure 3. Single integrator simulation from Section 7.2 on the static task with state dimension . (Left) Final state error as the number of demonstrations increases for the MLP and Diffusion policies. Here InneR and InneR-C are both effective since the same state entries always contain relevant or irrelevant information. (Right) What each method regularizes, measured for the final MLP policy at 200 training states: the squared Frobenius norm of the Jacobian (log scale) and the shared null space ratio of Equation (26) (InneR-C matches InneR, and is not shown for brevity). Standard regularizers like Frobenius and Nuclear are designed to decrease the magnitude of each Jacobian, and by flattening these individual Jacobians they actually make the perturbation responses inconsistent across the dataset: their null space ratio rises above the BC value. By contrast, InneR leaves for each individual state Jacobian at or above its BC value, and locally aligns the Jacobians so that they have a consistent null space. Colors follow Figure 2.
Is InneR Robust to Shifted Training and Test Distributions?
Next, we collect training data where the irrelevant state dimensions are correlated with the intended goal. Each irrelevant entry of a demonstration start state is a convex combination of its own uniform draw and one coordinate of the active goal, mixed so that their Pearson correlation is ; at test time the initial state is uniformly sampled without this spurious correlation. The correlation allows the policy to estimate the goal from the irrelevant entries, but a policy that relies on this shortcut fails once it disappears at test time. Our results for dynamic with 400 demonstrations for the MLP and 300 for the diffusion policy are shown in Figure 4. InneR roughly halves the error of BC for both policies (3.2 against 7.0 for the MLP, 2.7 against 5.2 for Diffusion), while every other regularizer is worse than BC. InneR-C fails most severely for the MLP, since the one global null space it enforces cannot cope with the goal switch.

Figure 4. Single-integrator simulation from Section 7.3 highlighting the dynamic task. At training time the irrelevant state dimensions are correlated with the active goal (), but at test time this correlation is removed, so a policy that reads the goal from the irrelevant entries fails (i.e., causal confusion ([13])). InneR is the only method that improves on BC.
What if Information is Dispersed Throughout the State?
So far our experiments separate the state dimensions into relevant and irrelevant entries, meaning that the shared null space is axis-aligned. Here we apply a nonlinear mapping to the underlying state so that the relevant information is scrambled throughout the states that the policy inputs. We apply a per-entry tanh bend between two affine layers, , where is a random matrix with the condition number at most 3, is a random rotation, and are random offsets, and the -th coordinate of the nonlinear transformation is given by with , , and argument is the -th coordinate of the vector-valued input of . This mapping is invertible (so that no data is lost) and is drawn once per random seed; every entry of now mixes relevant and irrelevant entries of . In addition, the system dynamics are now nonlinear in the coordinates the policy observes. Our results for dynamic with 800 demonstrations for the MLP and 600 for the diffusion policy are shown in Figure 5. InneR lowers the error of BC from 4.4 to 2.4 for the MLP and from 2.2 to 1.3 for Diffusion, whereas the norm baselines again increase it.

Figure 5. Single-integrator simulation from Section 7.4 highlighting the dynamic task. An invertible nonlinear transformation is applied to the state before it is input to the policy, so that the task-relevant information is dispersed across every entry the policy observes and the dynamics are nonlinear in those coordinates. InneR lowers the error of BC despite this nonlinear dispersion.
Drone Landing Simulations
Having investigated the performance of InneR in the controlled single-integrator environment, we next consider more realistic simulations involving a drone-landing task with nonlinear dynamics.
Setup. In the drone-landing task, a Crazyflie 2.0 (Giernacki et al. [16]) quadrotor simulated in the gym-pybullet-drones (Panerati et al. [32]) environment must fly toward a specified landing point, descend, and come to rest on the ground. The action specifies the desired rotation rate for the drone’s four motors. The state dimension is ( is varied in some simulations), in which the drone’s state is
consisting of the drone’s position (), orientation specified in both quaternion () and Euler angle () formats, linear velocity (), angular velocity () and goal position (). The remaining dimensions are task-irrelevant and will be introduced in greater detail below.
For both training and testing, the initial horizontal position is sampled uniformly from , with altitude fixed at , zero initial velocity, and level orientation. The static target is sampled uniformly with and . Its height is fixed at , corresponding to the drone’s nominal center height at rest.
An expert controller first guides the drone toward the target, then descends and switches off its motors after touchdown. It commands a desired velocity with magnitude at most and produces motor commands at 48 Hz, while the simulator advances at 240 Hz. Each demonstration contains at most 576 state-action pairs, corresponding to 12 seconds of demonstration, including up to one second of recording after touchdown.
At test time, each policy receives at most 960 control steps (20 s), to land. Success requires ground contact together with horizontal error at most , vertical error at most , speed at most , and roll-pitch magnitude at most rad, maintained for ten consecutive control steps.
MLP and Diffusion Policies. For each experiment, we test a deterministic MLP policy and a stochastic Diffusion policy. MLP has two hidden layers of width 128 with ReLU activations. Diffusion is a denoising diffusion model with steps ([19, 9]); its noise prediction network has three hidden layers of width 128 with ReLU activations. Both policies map the current observation to a single four-dimensional motor command, with and for the diffusion policy in § 5.2. We sample diffusion actions using 10 DDIM (Song et al. 2021) steps at test time, and estimate the Fisher information in (17) with sampled actions using 3 DDIM steps and draws of per action. Both policy architectures are trained with Adam (Kingma and Ba [22]) using a batch size of , and a learning rate of for 100 epochs. Within each policy family and experimental condition, all compared methods use the same architecture, training demonstrations, training settings and test cases and are evaluated at the last epoch.
Baselines. We test InneR using and retained rank for MLP, and for Diffusion, and and for both policies. InneR is compared to four baselines: unregularized behavior cloning (BC), Frobenius norm regularization (Hoffman et al. [20]), Nuclear norm regularization (Scarvelis and Solomon [43]), and InneR-C. Frobenius uses for MLP and for Diffusion. Nuclear uses for MLP and for Diffusion. As above, InneR-C is an ablation of our method without clustering (equivalent to setting ), keeping unchanged. We report the mean and standard error across 10 random seeds for each experiment.
How does InneR Scale with Irrelevant State Dimension?
In Figure 6, we plot landing success rate as a function of the observation dimension while fixing the training set at 500 demonstrations for MLP and 100 for Diffusion. Starting from the 19-dimensional drone state and goal, we append the recorded poses of zero to three distractor drones. Each pose contributes seven entries: three position coordinates and a four-dimensional quaternion. These observations vary over time but do not affect the controlled drone’s dynamics or landing objective. We evaluate all methods on the same 250 held-out test cases and report success rate (SR) using the aforementioned success criteria.
Figure 6. Drone landing simulation from Section 8.1. Plots show the landing success rate as state dimension increases (higher is better). The state dimension sweep starts at , and each additional 7 state coordinates come from a distractor drone in both the demonstrations and the test states. InneR largely outperforms BC at all tested dimensions for MLP, and is close to BC for Diffusion. InneR-C performs poorly for MLP but is competitive for Diffusion.
For MLP, InneR has a higher average success rate than BC at every tested dimension, achieving 97.3%, 91.0%, 88.0%, and 77.2% at , 26, 33, and 40, respectively. For Diffusion, InneR improves on BC at , 26, and 33. At , their mean success rates are similar: 80.9% for InneR and 82.2% for BC. InneR-C has a low success rate for MLP but remains competitive for Diffusion, suggesting that the nonlinear drone dynamics require smaller clusters than were required in the linear single-integrator dynamics of
§ 7 when using deterministic policies. For both MLP and Diffusion policies, the Frobenius and Nuclear baselines attain a low average success rate.
How Does InneR Scale with Dataset Size?
In Figure 7, we plot landing success rate as a function of the number of training demonstrations: for MLP, and for Diffusion. The policy input contains the 19-dimensional drone state and goal together with three irrelevant entries sampled uniformly from and held fixed within each episode, giving . Within each policy family, all methods receive the same training demonstrations and are evaluated on the same test cases.
Figure 7. Drone landing simulation from § 8.2. Plots show the landing success rate as number of demonstrations increases (higher is better) with state dimension . (Left) Success rate as the number of demonstrations increases for the MLP and Diffusion policies. InneR is effective for both MLP and Diffusion policies, and InneR-C is only effective for the Diffusion policy. (Right) What each method regularizes (measured for the final MLP policy at 200 training states), i.e. the squared Frobenius norm of the Jacobian (log scale) and the shared null space ratio of Equation (26). Both Frobenius and nuclear norm baselines reduce the Jacobian norm. Frobenius has a higher null space ratio than BC at 300, 500, and 700 demonstrations, whereas Nuclear is below BC except at 700 demonstrations. InneR consistently reduces the null-space ratio without uniformly reducing the Jacobian norm. Colors follow Figure 2.
For MLP policies, InneR improves on BC at larger numbers of demonstrations (500 and 700), reaching 98.8% success at 700 demonstrations compared with 88.4% for BC. For Diffusion policies, InneR improves on BC at all tested demonstration dataset sizes. InneR-C performs poorly for the MLP policy but remains competitive for the Diffusion policy. The Frobenius and Nuclear baselines have lower mean success rates than BC at every tested dataset size in both policy families. To examine what the regularizers change, we also plot the squared Frobenius norm of the MLP policy Jacobian and the InneR null-space ratio. The Frobenius and Nuclear norm baselines reduce the Jacobian norm below BC at every dataset size. In contrast, InneR reduces the null space ratio relative to BC at all four dataset sizes. As in Section 7.2, these results distinguish suppressing overall policy sensitivity from regularizing sensitivity in shared null space directions.
Is InneR Robust to Shifted Training and Test Distributions?
Next, we collect training data in which irrelevant observations are correlated with the landing target. The 22-dimensional input contains the 19-dimensional drone state and goal together with a three-dimensional distractor goal that is held fixed within each episode. During training, the horizontal distractor coordinates are generated to have correlation with the corresponding true-goal coordinates. At test time, this correlation is removed while the true goal remains available in the observation. We use 500 demonstrations for MLP and 100 for Diffusion, and evaluate each policy on the same test cases within its policy family.
Figure 8 plots the success rate under the shifted test distribution. InneR achieves the highest mean success rate in both policy families under the shifted test distribution. It outperforms BC (88.4% vs. 77.0% for MLP and 76.6% vs. 59.4% for Diffusion). InneR-C improves Diffusion success to 68.0% but reduces MLP success to 34.3%; both Frobenius and Nuclear baselines perform worse than BC in both policy families. These results support improved landing reliability for InneR under the tested shift.

Figure 8. Drone landing simulation from § 8.3. In training, irrelevant state dimensions are correlated with the active goal (), but at test time this correlation is removed, so a policy that reads the goal from the irrelevant entries fails (i.e., causal confusion [13]). InneR improves upon BC for both MLP and Diffusion, while InneR-C only improves upon BC for Diffusion.
Figure 10. Our real-world tasks. Each task is shown at the start, middle, and end of a successful InneR rollout with the three distractors (apple, orange and banana). (Left) in close, the robot pulls the handle down to latch the lid of the coffee maker. (Middle) In stack, the robot picks up the black cube and places it on the tan cube (teal and orange segmentations, respectively). (Right) In push, the robot pushes the pot into the target region. Tracked objects are tinted by the segmentation masks that our perception pipeline uses to estimate their poses, and the background is faded for clarity.
What if Information is Dispersed Throughout the State?
Next, following Section 7.4 we mix relevant and irrelevant information through an invertible nonlinear transformation of the 22-dimensional observation from the dataset-size experiment. We use 500 training demonstrations for MLP policies and 100 for Diffusion policies. After training set normalization, the policy receives input , where we construct the nonlinearity elementwise: with having condition number at most 3. The matrix is orthogonal and chosen independently in each seed; entries of and are sampled uniformly from . We use and . The mapping preserves the observation’s information while dispersing it across coordinates. One transformation is drawn per seed and held fixed across methods, training, and testing. Regularization is computed with respect to the transformed observation , and the drone dynamics, expert actions, and landing objective remain unchanged.
Figure 9 reports the success rate over the dispersion, with 250 test cases per run. For MLP, InneR outperforms BC (86.4% vs. 76.8%). InneR-C achieves 35.0%, while Frobenius and Nuclear baselines achieve 7.5% and 5.2%, respectively. For Diffusion, InneR and BC have similar mean success rates (69.1% and 68.4%), whereas InneR-C achieves the highest rate at 77.6%. Both Frobenius and Nuclear norm baselines have near-zero success. Thus, InneR retains high MLP success after information is dispersed, while InneR-C gives the largest mean Diffusion improvement in this setting.

Figure 9. Drone landing simulation from Section 8.4. An invertible nonlinear transformation is applied to the state before it is input to the policy, so that the task-relevant information is dispersed across every entry the policy observes and the dynamics are nonlinear in those coordinates. InneR improves upon BC for MLP and InneR-C improves upon BC for Diffusion despite this nonlinear dispersion.
Hardware Experiments
The controlled simulations of § 7 and § 8 suggest that InneR outperforms standard behavior cloning and regularization alternatives that penalize the entire spectrum of the inner product matrix. To evaluate how well InneR performs with physical robots, we present real-world studies across three tasks: “closing a coffee maker”, “stacking two cubes”, and “pushing a pot”.
Experimental Setup. We use a 7-DoF Franka Emika Panda robot arm with a UMI gripper ([10]) and a stationary Orbbec Femto Mega RGB-D camera. Objects are segmented with SAM3 ([7]) on the first frame and tracked across frames with EdgeTAM ((Zhou et al. 2025)). We project the depth under each mask into the robot’s frame of reference to recover each object’s pose. When tracking five objects, this process runs near our control frequency of 10 Hz on an NVIDIA RTX 3080 GPU.
Training Data. We collected 25 teleoperated demonstrations for close, 50 for stack, and 25 for push. To cover more of the workspace, we augmented each set with copies of the demonstrations rotated about the robot’s base, so that the arm’s motion and every object rotate together. We also mirrored the motions of the robot and the objects about the -axis, for a total of 100 demonstrations in push and 300 in stack and close. The state is the width and pose of the gripper, as well as the pose of each object. The action is the gripper’s target pose and width 0.5 s ahead. For each task, the distractor conditions add the position and yaw of an apple, an orange, and a banana to the state. These distractors are irrelevant to the robot’s task.
Independent Variables. We compare three methods from § 7 and § 8: BC, Nuclear, and InneR. Within each task, every method is given the exact same training data, model architecture, and hyperparameters. The policy is a diffusion model ([9]) whose denoiser is an MLP with three hidden layers (256 units, ReLU activation), trained for gradient steps using the Adam optimizer ( with cosine decay). We deploy an exponential moving average of the weights (decay 0.999). Actions are sampled with DDIM ((Song et al. 2021)) using 50 denoising steps. For InneR we use , , , and ( with no distractors, as the state has two few dimensions to leave a null space at ). We estimate with sampled actions and draws of per action.
Dependent Variables. We measure the success rate over 20 trials per method, task, and distractor condition. In push, success is when the pot ends in the target region, in stack, the black cube rests on top of the tan cube, and in close, the coffee maker’s lid should be latched closed. To examine how each regularizer affects the robot’s policy, we also report the trace of the Fisher information and the shared null space ratio of Equation (26), as in Sections 7.2 and § 8.2, and each policy’s selectivity. We define a policy’s sensitivity to an object as the average distance that the commanded gripper position moves when the object is displaced by 1 cm in the table plane (with the diffusion noise held fixed), and its selectivity as its mean sensitivity to the task-relevant objects divided by its mean sensitivity to task-irrelevant distractors.
Hypotheses. We test three hypotheses:
H1: InneR will outperform behavior cloning without regularization and baselines that regularize the policy globally.
H2: InneR will be more robust to observation noise than baselines.
H3: InneR will scale more effectively with the number of distractors than baselines.
Results. Our results are summarized in Figure 11. To address H1, we average each method’s success rate across the three tasks with zero and three distractors. InneR has an average success rate of 65.8%, compared to 45.8% for BC and 21.7% for Nuclear. With no distractors InneR and BC perform similarly (75.0% and 73.3%), so the regularizer does not reduce performance when there is nothing to ignore. The gap widens with three distractors: InneR scores 56.7% versus BC’s 18.3%. Nuclear scores worse than BC even without distractors (25.0%) and never succeeds in stack.
Figure 11. Success rates in our real-world experiments for BC, Nuclear, and InneR; the number above each bar is the success rate in percent. Colors follow Figure 2. (Left) Close with zero to three distractors. (Middle, Right) Stack and push with zero and three distractors. Error bars show standard error over 20 trials. Averaged across the three tasks, InneR matches BC with no distractors (75.0% vs. 73.3%) and outperforms both baselines with three distractors.
To address H2, we add zero-mean Gaussian noise to every object position that the policy observes in close with three distractors, drawn independently at each control step (10 trials per method and noise level). For reference, the position estimates from our perception pipeline vary by roughly 1 mm (standard deviation) between frames. Figure 12 shows that InneR has the highest success rate with 1 mm of added noise (50% vs. 20% for BC and 40% for Nuclear) and with 1.5 mm (40% vs. 10% and 30%). At 2 mm Nuclear and InneR fail. InneR therefore remains the most successful method under moderate observation noise, but its success rate falls about as quickly as the baselines’. These results partially support H2.
Figure 12. Success rate in close with three distractors as Gaussian noise is added to the object positions that the policy observes. The horizontal axis is the standard deviation of the added noise; 0 mm repeats the three distractor results of Figure 11. InneR has the highest success rate up to 1.5 mm
Turning to H3, we consider close with zero to three distractors (Figure 11, left). For each method we fit a logistic regression of trial success on the number of distractors (80 trials per method) and apply a Wald test ([2]) of the null hypothesis that the regression slope is zero, i.e., that the method’s success probability does not change with the number of distractors. BC’s success rate falls from 75% to 30% (a linear trend of percentage points per distractor), and the test rejects the null hypothesis (, ). InneR stays between 60% and 80% ( points per distractor), and its rate of change with respect to the number of distractors is not statistically significant (, ). Nuclear’s rate of change is also not significant (, ), but it is the least successful method without distractors. We next compare the methods when distractors are present by pooling the conditions with one to three distractors (60 trials per method). InneR succeeds in 68% of these trials, whereas BC and Nuclear each succeed in 37%. For each baseline, a two-proportion -test ([2]) rejects the null hypothesis that it has the same success probability as InneR ( in both cases). Altogether, these results support H3.
Discussion. Figure 13 repeats the measurements of Section 7.2 and § 8.2 for the three-distractor policies. Nuclear reduces the trace of by more than two orders of magnitude on every task, whereas InneR retains 66% to 83% of BC’s. InneR also has the lowest shared null space ratio on all three tasks (e.g., 0.5% vs. 1.8% for BC in close). As in simulation, these results distinguish suppressing the policy’s overall sensitivity from regularizing its sensitivity in shared null space directions.
Figure 13. The regularization results from our hardware experiments, measured for the three-distractor diffusion policies at training states. (Left) Trace of the Fisher information (log scale). (Right) Shared null space ratio of Equation (26), with in place of the inner product, , . Nuclear shrinks the Fisher information by more than two orders of magnitude, whereas InneR keeps it close to BC’s level and attains the lowest null space ratio on every task. Error bars show standard error over 8 batches of 256 states.
Finally, we investigate how these differences appear in the robot’s actions at the states it has encountered during real-world trials (cf. Figure 14). InneR is the most selective method on every task: displacing a task object changes its action 6.5 to 8.4 times more than displacing a distractor, compared to 5.1 to 7.6 times for BC and 4.8 to 6.8 times for Nuclear. Table 1 lists the sensitivities defining each selectivity ratio. Relative to BC, InneR responds 15% to 19% less to the distractors while its response to the task objects stays within 13% of BC’s. Nuclear’s effect is task-dependent: in close and stack it responds 2.6 to 3.4 times less than BC to the task objects and the distractors alike, and in push it responds 1.8 times more than BC to the distractors.
Figure 14. Selectivity of the three-distractor policies on states encountered during the real-world trials (higher is better). InneR is the most selective method on every task. Error bars show standard error over 400 states.
| BC | Nuclear | InneR | ||
| Close | Task objects | 2.193 | 0.636 | 2.072 |
| Distractors | 0.305 | 0.094 | 0.247 | |
| Stack | Task objects | 1.272 | 0.490 | 1.378 |
| Distractors | 0.249 | 0.094 | 0.212 | |
| Push | Task objects | 1.978 | 2.292 | 1.712 |
| Distractors | 0.259 | 0.479 | 0.211 |
Table 1. Sensitivity of the three-distractor policies
Conclusion
We have introduced InneR, a regularization algorithm that makes deterministic or stochastic imitation learning policies robust to local state perturbations. InneR builds on the insight that nearby states should have consistent first-order responses to the same perturbation. By clustering states whose inner products are aligned, averaging the state space inner product within each cluster, and then minimizing the smallest eigenvalues of that average, InneR shapes the learned policy to maintain a consistent null space without reducing its expressivity. Across our experiments, adding InneR to standard behavior cloning made the policy more robust to irrelevant state dimensions without requiring any additional data. Under distribution shift, InneR roughly halved the final state error in the single-integrator simulations and raised the landing success rate in the drone simulations by up to 17 percentage points. On hardware, InneR raised the average success rate of an imitation-learning policy with three distractors from 18% to 57%, while matching the performance of behavior cloning in tasks with no distractors.
Limitations. We highlight two related limitations. First, InneR increases the training time by roughly in our experiments, and this cost grows with state dimension because the decompositions in Algorithm 1 scale with the size of the inner product matrix. Second, due to this overhead, we have not explored InneR for policies that input image space observations or high-dimensional feature representations. Recent research has found efficient ways to estimate other Jacobian norms in high-dimensional spaces ([43]). In future work, we hope to extend these estimates to the eigenvalues of the averaged inner product so that inner product regularization can be used with high-dimensional inputs.
Funding
This work was supported in part by the National Science Foundation through grants #2535892, #2535276, and #2337884.
Supplemental Material
Code and videos of our experiments are available on our website: https://collab.me.vt.edu/inner.
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
References
- [1]Agarwal R, Machado MC, Castro PS and Bellemare MG (2021) Contrastive behavioral similarity embeddings for generalization in reinforcement learning. In: International Conference on Learning Representations.arxiv.org/abs/2101.05265
- [2]Agresti A (2002) Categorical Data Analysis. Wiley Series in Probability and Statistics, 2nd edition. Hoboken, NJ: Wiley-Interscience.
- [3]Arvanitidis G, González-Duque M, Pouplin A, Kalatzis D and Hauberg S (2022) Pulling back information geometry. In: International Conference on Artificial Intelligence and Statistics. pp. 4872–4894.arxiv.org/abs/2106.05367
- [4]Arvanitidis G, Hansen LK and Hauberg S (2018) Latent space oddity: On the curvature of deep generative models. In: International Conference on Learning Representations.arxiv.org/abs/1710.11379
- [5]Beik-Mohammadi H, Hauberg S, Arvanitidis G, Neumann G and Rozo L (2021) Learning Riemannian manifolds for geodesic motion skills. In: Robotics: Science and Systems.
- [6]Block A, Jadbabaie A, Pfommer D, Simchowitz M and Tedrake R (2023) Provable guarantees for generative behavior cloning: Bridging low-level stability and high-level behavior. In: Advances in Neural Information Processing Systems, volume 36.arxiv.org/abs/2307.14619
- [7]Carion N, Gustafson L, Hu YT, Debnath S, Hu R, Suris Coll-Vinent D, Ryali C, Alwala KV, Khedr H, Huang A et al. (2026) Sam 3: Segment anything with concepts. In: International conference on learning representations, volume 2026. pp. 138846–138923.
- [8]Chen Z, He X, Wang YJ, Liao Q, Ze Y, Li Z, Sastry SS, Wu J, Sreenath K, Gupta S and Peng XB (2025) Learning smooth humanoid locomotion through Lipschitz-constrained policies. In: IEEE/RSJ International Conference on Intelligent Robots and Systems. pp. 4743–4750.arxiv.org/abs/2410.11825
- [9]Chi C, Xu Z, Feng C, Cousineau E, Du Y, Burchfiel B, Tedrake R and Song S (2025) Diffusion policy: Visuomotor policy learning via action diffusion. The International Journal of Robotics Research 44(10-11): 1684–1704.
- [10]Chi C, Xu Z, Pan C, Cousineau E, Burchfiel B, Feng S, Tedrake R and Song S (2024) Universal manipulation interface: In-the-wild robot teaching without in-the-wild robots. Robotics: Science and Systems.arxiv.org/abs/2402.10329
- [11]Constantine PG, Dow E and Wang Q (2014) Active subspace methods in theory and practice: Applications to kriging surfaces. SIAM Journal on Scientific Computing 36(4): A1500–A1524.
- [12]Conway JH, Hardin RH and Sloane NJA (1996) Packing lines, planes, etc.: Packings in Grassmannian spaces. Experimental Mathematics 5(2): 139–159.DOI
- [13]de Haan P, Jayaraman D and Levine S (2019) Causal confusion in imitation learning. In: Advances in Neural Information Processing Systems, volume 32. pp. 11698–11709.arxiv.org/abs/1905.11979
- [14]Drucker H and LeCun Y (1992) Improving generalization performance using double backpropagation. IEEE Transactions on Neural Networks 3(6): 991–997.pubmed.ncbi.nlm.nih.gov/18276495
- [15]Ferns N, Panangaden P and Precup D (2004) Metrics for finite Markov decision processes. In: Conference on Uncertainty in Artificial Intelligence. pp. 162–169.
- [16]Giernacki W, Skwierczyński M, Witwicki W, Wroński P and Kozierski P (2017) Crazyflie 2.0 quadrotor as a platform for research and education in robotics and control engineering. In: 2017 22nd international conference on methods and models in automation and robotics (MMAR). IEEE, pp. 37–42.DOI
- [17]Gloeckler M, Deistler M and Macke JH (2023) Adversarial robustness of amortized Bayesian inference. In: International Conference on Machine Learning. pp. 11493–11524.arxiv.org/abs/2305.14984
- [18]Gouk H, Frank E, Pfahringer B and Cree MJ (2021) Regularisation of neural networks by enforcing Lipschitz continuity. Machine Learning 110(2): 393–416.arxiv.org/abs/1804.04368
- [19]Ho J, Jain A and Abbeel P (2020) Denoising diffusion probabilistic models. In: Advances in Neural Information Processing Systems, volume 33. pp. 6840–6851.arxiv.org/abs/2006.11239
- [20]Hoffman J, Roberts DA and Yaida S (2019) Robust learning with Jacobian regularization. arXiv preprint arXiv:1908.02729.arxiv.org/abs/1908.02729
- [21]Kelly M, Sidrane C, Driggs-Campbell K and Kochenderfer MJ (2019) HG-DAGGer: Interactive imitation learning with human experts. In: IEEE International Conference on Robotics and Automation. pp. 8077–8083.arxiv.org/abs/1810.02890
- [22]Kingma D and Ba J (2015) Adam: A method for stochastic optimization. In: 3rd International Conference on Learning Representations, ICLR 2015-Conference Track Proceedings.arxiv.org/abs/1412.6980
- [23]Kobayashi T (2022) L2C2: Locally Lipschitz continuous constraint towards stable and smooth reinforcement learning. In: IEEE/RSJ International Conference on Intelligent Robots and Systems. pp. 4032–4039.DOI
- [24]Krogh A and Hertz JA (1991) A simple weight decay can improve generalization. In: Advances in Neural Information Processing Systems, volume 4. pp. 950–957.
- [25]Laskey M, Lee J, Fox R, Dragan A and Goldberg K (2017) DART: Noise injection for robust imitation learning. In: Conference on Robot Learning. pp. 143–156.arxiv.org/abs/1703.09327
- [26]Lee SW, Kang X, Yang BY and Kuo YL (2025) CLASS: Contrastive learning via action sequence supervision for robot manipulation. In: Conference on Robot Learning. pp. 4743–4766.arxiv.org/abs/2508.01600
- [27]Li L, Walsh TJ and Littman ML (2006) Towards a unified theory of state abstraction for MDPs. In: International Symposium on Artificial Intelligence and Mathematics.
- [28]Liu Z, Li X, Kang B and Darrell T (2021) Regularization matters in policy optimization: An empirical study on continuous control. In: International Conference on Learning Representations.
- [29]Mehta SA, Ciftci YU, Ramachandran B, Bansal S and Losey DP (2025) Stable-BC: Controlling covariate shift with stable behavior cloning. IEEE Robotics and Automation Letters 10(2): 1952–1959.arxiv.org/abs/2408.06246
- [30]Miyato T, Maeda Si, Koyama M and Ishii S (2019) Virtual adversarial training: A regularization method for supervised and semi-supervised learning. IEEE Transactions on Pattern Analysis and Machine Intelligence 41(8): 1979–1993.DOI
- [31]Mysore S, Mabsout B, Mancuso R and Saenko K (2021) Regularizing action policies for smooth control with reinforcement learning. In: IEEE International Conference on Robotics and Automation. pp. 1810–1816.arxiv.org/abs/2012.06644
- [32]Panerati J, Zheng H, Zhou S, Xu J, Prorok A and Schoellig AP (2021) Learning to fly—a gym environment with pybullet physics for reinforcement learning of multi-agent quadcopter control. In: 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). pp. 7512–7519. DOI: 10.1109/IROS51168.2021.9635857.DOI
- [33]Parekh S, Culbertson P and Losey DP (2026) TransMASK: Masked state representation through learned transformation. arXiv preprint arXiv:2603.05670 .arxiv.org/abs/2603.05670
- [34]Pari J, Shafiullah NM, Arunachalam SP and Pinto L (2022) The surprising effectiveness of representation learning for visual imitation. In: Robotics: Science and Systems.arxiv.org/abs/2112.01511
- [35]Pfrommer D, Zhang T, Tu S and Matni N (2022) TaSIL: Taylor series imitation learning. In: Advances in Neural Information Processing Systems, volume 35. pp. 20162–20174.arxiv.org/abs/2205.14812
- [36]Qiu L, Ciftci YU and Bansal S (2026) Safety-aware imitation learning via MPC-guided disturbance injection. IEEE Robotics and Automation Letters 11(6): 7222–7229.
- [37]Radhakrishnan A, Beaglehole D, Pandit P and Belkin M (2024) Mechanism for feature learning in neural networks and backpropagation-free machine learning models. Science 383(6690): 1461–1467.DOI
- [38]Ratliff ND, Issac J, Kappler D, Birchfield S and Fox D (2018) Riemannian motion policies. arXiv preprint arXiv:1801.02854.arxiv.org/abs/1801.02854
- [39]Redmon J, Divvala S, Girshick R and Farhadi A (2016) You only look once: Unified, real-time object detection. In: IEEE Conference on Computer Vision and Pattern Recognition. pp. 779–788.arxiv.org/abs/1506.02640
- [40]Rifai S, Vincent P, Muller X, Glorot X and Bengio Y (2011) Contractive auto-encoders: Explicit invariance during feature extraction. In: International Conference on Machine Learning. pp. 833–840.
- [41]Ross AS, Hughes MC and Doshi-Velez F (2017) Right for the right reasons: Training differentiable models by constraining their explanations. In: International Joint Conference on Artificial Intelligence. pp. 2662–2670.arxiv.org/abs/1703.03717
- [42]Ross S, Gordon G and Bagnell D (2011) A reduction of imitation learning and structured prediction to no-regret online learning. In: International Conference on Artificial Intelligence and Statistics. pp. 627–635.arxiv.org/abs/1011.0686
- [43]Scarvelis C and Solomon J (2024) Nuclear norm regularization for deep learning. In: Advances in Neural Information Processing Systems, volume 37. pp. 116223–116253.arxiv.org/abs/2405.14544









