betaFeedback

WorksComputer Science

A Control-Theoretic Approximation to Predictive Coding Dynamics

Abstract

Backpropagation remains the dominant method for training deep neural networks, but its reliance on non-local computations has motivated the search for biologically plausible alternatives. Two prominent frameworks, predictive coding (PC) and deep feedback control (DFC), offer distinct approaches to local learning, yet their relationship remains unclear. Here, we show that DFC arises as a local approximation to predictive coding dynamics near fixed points, with exact equivalence in shallow linear networks and systematic deviations arising with depth, nonlinearity, and distance from equilibrium. We characterize how this approximation, which we call Approx PC, deviates from PC in its activity dynamics, and show that these deviations can be mitigated by relinearizing the feedback controller and approaching the output target through a sequence of intermediate nudges. In networks trained on MNIST and FashionMNIST, Approx PC reaches task performance comparable to PC trained with the same nudges, and weights learned with it transfer directly to PC inference. Our results unify energy-based and control-theoretic perspectives on learning and open the door to applying tools from control theory to the training of predictive coding networks.

Introduction

Backpropagation has become the dominant algorithm for training deep neural networks, yet it relies on computational primitives that are difficult to reconcile with biological neural circuits, including weight transport across layers [1], global error signals [17, 22], and phased learning stages [7]. The brain, on the other hand, relies on synaptic plasticity, which is widely believed to depend on local information available at each synapse, such as pre- and post-synaptic activity and modulatory feedback, and does not require distinct forward and backward passes [18]. This contrast has motivated a large body of work seeking biologically plausible alternatives for credit assignment in deep neural networks [2, 3, 10, 23, 20, 4, 6, 11, 9, 12]. Among these, two influential approaches have emerged from distinct perspectives: (1) predictive coding (PC) networks [21, 5, 24, 16], inspired by sensory processing in cortical circuits and typically formulated as energy-based models that minimize a global objective by reducing local mismatch between bottom-up and top-down signals, and (2) deep feedback control (DFC) [13], motivated by principles of motor control and rooted in control-theoretic frameworks, where credit assignment is achieved via dynamical error correction through a feedback controller.

While both frameworks implement local learning rules, their underlying neural dynamics appear markedly different. Although both rely on error signals, their origin and role differ: in PC, errors are computed locally at each layer, whereas in DFC they arise from a global target and are distributed through the network via the controller (see Figure 1). This apparent difference raises the question of whether these two classes of models implement fundamentally distinct mechanisms for credit assignment, or whether they can be understood within a common framework.

Illustrations of error signal sources in predictive coding and deep feedback control

Figure 1. Illustrations of error signal sources in predictive coding (left) and deep feedback control (right). PC errors are distributed while DFC is centralized. Pathways involved in calculating errors are denoted by solid lines, while general information pathways are denoted by dashed lines.

From a biological perspective, the connectivity motifs required by both frameworks are present in the cortex: hierarchical organization with local feedforward–feedback loops supports predictive coding, while long-range feedback from higher-order to early sensory areas enables global control signals. This suggests that the brain may support both local error correction mechanisms and more global control-driven dynamics, raising the possibility that the two frameworks reflect different regimes of a shared underlying computation.

Here, we show that DFC can be interpreted as a local approximation to PC dynamics in the vicinity of fixed points. This equivalence is exact for shallow linear networks and progressively breaks down with increasing network depth, with the introduction of nonlinearities, and as the system moves further from equilibrium. Building on this connection, we propose a practical scheme to approximate PC using DFC by decomposing the trajectory toward a target into a sequence of intermediate subgoals. Within each subgoal, DFC dynamics drive the network toward a local equilibrium before transitioning to the next stage. This perspective suggests that distributed, layer-wise error signals can be effectively replaced by a centralized feedback controller that sequentially guides the network through intermediate targets. Our contributions are organized in three parts:

Theory: We establish a formal connection between predictive coding and deep feedback control by showing that PC activity dynamics can be written as a distributed feedback controller, which reduces near fixed points to a DFC-style controller driven by the output error. We call the resulting approximate dynamics Approx PC. In linear networks, we show that the two are exactly equivalent with a single hidden layer, and that with two hidden layers they share the same fixed point and differ only by an additional damping term in PC.

Characterizing the approximation: Through controlled probes of activity dynamics on single inputs, we show how Approx PC deviates from PC as perturbation size, depth, and nonlinearity increase. We demonstrate that these deviations are mitigated by relinearizing the feedback controller or by approaching the output target through a sequence of intermediate nudges.

Validation under learning: We test whether the approximation error compounds during training and find that it does not substantially affect what the network learns. On MNIST [8] and FashionMNIST [25], networks trained with Approx PC reach task performance comparable to that of PC trained with the same nudges, their learned weights transfer directly to PC inference, and their hyperparameter sensitivity is similar to that of PC.2

Related work

Although not the focus of this work, many alternative biologically plausible learning rules use local signals and objectives to learn [23, 20, 4] or various centralized feedback pathways to drive learning [2, 3, 10, 20, 4, 6]. Many of these have theoretical relationships to both PC and DFC, but our objective is to understand PC through a specific family of control-based rules. In particular, while recent hierarchical variants of PC incorporate long-range feedback pathways, here we focus on the standard PC formulation. The work most closely related to ours is [15], where the authors briefly note that PC can be understood as a solution to a least-control problem. Yet the controller they propose is distributed rather than a DFC-style feedback controller driven by the final layer’s error. We discuss these distinctions in more detail in § 4.

Background

Consider a network comprising LL layers of neurons, with input x(t)∈Rd\mathbf{x}(t) \in\mathbb{R}^{d}. Let the membrane potential of neurons in layer ℓ\ell be vℓ(t)\mathbf{v}^{\ell}(t), and let their activity be rℓ(t)=ϕ(vℓ(t))\mathbf{r}^{\ell}(t) = \phi(\mathbf{v}^{\ell}(t)), where ϕ\phi is an activation function. Define v0(t):=x(t)\mathbf{v}^{0}(t) := \mathbf{x}(t) as the input. The parameters connecting layer ℓ−1\ell- 1 and ℓ\ell of the network are denoted Wℓ(t)\mathbf{W}^{\ell}(t). In general, we will seek to optimize both the potentials vℓ\mathbf{v}^{\ell} and the parameters Wℓ\mathbf{W}^{\ell} concurrently. For concision, variables with tt in the subscript denote time-varying functions, i.e., xt:=x(t)\mathbf{x}_{t} := \mathbf{x}(t).

Predictive coding

Given a network as defined above, predictive coding seeks to minimize the network’s total energy, defined as the sum of layer-wise energies. The energy of a given layer is defined as the difference between its potential and the input it receives from the previous layer 2^{2}:

Et=12∑ℓ∥vtℓ−Wtℓrtℓ−1⏟etℓ∥22(1)E_{t} = \frac{1}{2}\sum_{\ell}\left\|\underbrace{\mathbf{v}_{t}^{\ell} - \mathbf{W}_{t}^{\ell}\mathbf{r}_{t}^{\ell-1}}_{\mathbf{e}_{t}^{\ell}}\right\|_{2}^{2} \tag*{(1)}

To minimize the energy, the potential and weight dynamics at every layer follow the negative gradient,

τvdvtℓdt=−∇vtℓEt={−etℓ+ϕ′(vtℓ)⊙(Wtℓ+1)Tetℓ+1if ℓ<L−etLif ℓ=L(2)\tau_{v}\frac{\mathrm{d}\mathbf{v}_{t}^{\ell}}{\mathrm{d}t} = -\nabla_{\mathbf{v}_{t}^{\ell}} E_{t} = \begin{cases} -\mathbf{e}_{t}^{\ell} + \phi'(\mathbf{v}_{t}^{\ell}) \odot\left(\mathbf{W}_{t}^{\ell+1}\right)^{\mathrm{T}}\mathbf{e}_{t}^{\ell+1} & \text{if } \ell< L \\ -\mathbf{e}_{t}^{L} & \text{if } \ell= L \end{cases} \tag*{(2)}
τwdWtℓdt=−∇WtℓEt=etℓ(rtℓ−1)T(3)\tau_{w}\frac{\mathrm{d}\mathbf{W}_{t}^{\ell}}{\mathrm{d}t} = -\nabla_{\mathbf{W}_{t}^{\ell}} E_{t} = \mathbf{e}_{t}^{\ell}\left(\mathbf{r}_{t}^{\ell-1}\right)^{\mathrm{T}} \tag*{(3)}

where τv\tau_{v} is the neural time constant and τw\tau_{w} is the weight time constant. Thus, the network activity at each layer attempts to align top-down and bottom-up signals, yielding representations that minimize the system’s overall energy.

The network operates in two modes: inference and learning. In both modes, the input layer (i.e., vt0\mathbf{v}_{t}^{0}) is clamped, meaning it is fixed to an input xt\mathbf{x}_{t}. This, along with (2), drives changes in the network activity during inference, while the weights are fixed. In supervised learning, we additionally clamp the final layer’s activity to the labels yt\mathbf{y}_{t}. This, along with (2) and (3), drives both network activity and parameter updates during learning.

Note that we adopt a forward prediction convention, in which each layer predicts the activity of the next layer. While much of the PC literature uses the opposite (top-down) convention [21, 24], we choose the forward convention because it is more familiar to a general deep learning audience and matches the convention used for DFC [13].

Deep feedback control

Deep feedback control (DFC) [13, 14, 15] is an alternative framework, motivated by control theory, for adjusting the network activity and parameter dynamics to produce optimal predictions. DFC consists of a dynamical multi-layer network, where each layer integrates feedforward inputs with a feedback control signal 3^{3}:

τvdvtℓdt=−vtℓ+Wtℓrtℓ−1+utℓ(4)\tau_{v}\frac{\mathrm{d}\mathbf{v}_{t}^{\ell}}{\mathrm{d}t} = -\mathbf{v}_{t}^{\ell} + \mathbf{W}_{t}^{\ell}\mathbf{r}_{t}^{\ell-1} + \mathbf{u}_{t}^{\ell} \tag*{(4)}

Following control theory, we define a target, r∗\mathbf{r}^{*}, for the final layer activity. The feedback controller determines the control signal at each layer, utℓ\mathbf{u}_{t}^{\ell}, to minimize the error between final layer activity and the target, etL=rt∗−rtL\mathbf{e}_{t}^{L} = \mathbf{r}_{t}^{*} - \mathbf{r}_{t}^{L}. For proportional control, which we consider for this study, the overall control signal is ut=KpetL\mathbf{u}_{t} = K_{p}\mathbf{e}_{t}^{L}, which drives each layer through feedback weights, i.e., utℓ=Qℓut\mathbf{u}_{t}^{\ell} = \mathbf{Q}^{\ell}\mathbf{u}_{t}.

We choose r∗\mathbf{r}^{*} to minimize a loss function, L\mathcal{L},

r∗:=rssL−λdLdrL∣rL=rssL=rssL+δL(5)\mathbf{r}^{*} := \mathbf{r}_{\mathrm{ss}}^{L} - \lambda\left.\frac{\mathrm{d}\mathcal{L}}{\mathrm{d}\mathbf{r}^{L}}\right|_{\mathbf{r}^{L}=\mathbf{r}_{\mathrm{ss}}^{L}} = \mathbf{r}_{\mathrm{ss}}^{L} + \delta^{L} \tag*{(5)}

where rssLr_{\mathrm{ss}}^{L} is the steady state of rtLr_{t}^{L} in the absence of control feedback (i.e., utL=0u_{t}^{L}=0). The nudge size, λ>0\lambda>0, is chosen to be small to ensure that r∗r^{*} remains close to the steady-state activity.

While (4) ensures that the activity of each layer is modulated to bring the final layer close to the target, this might require a large control signal—∥ut∥≫0\lVert u_t\rVert\gg0. DFC follows the principle of least control and seeks to adjust Wℓ\mathbf{W}^{\ell} to reduce ∥ut∥\lVert u_t\rVert. This is achieved through the local learning dynamics,

τwdWtℓdt=(rtℓ−ϕ(Wtℓrtℓ−1))(rtℓ−1)T(6)\tau_w\frac{\mathrm{d}\mathbf{W}^{\ell}_t}{\mathrm{d}t}=\left(\mathbf{r}^{\ell}_t-\phi\left(\mathbf{W}^{\ell}_t\mathbf{r}^{\ell-1}_t\right)\right)\left(\mathbf{r}^{\ell-1}_t\right)^{\mathsf{T}} \tag*{(6)}

which seeks to minimize the mean squared error between a layer’s activity rtℓ\mathbf{r}^{\ell}_t and the input from the previous layer, ϕ(Wtℓrtℓ−1)\phi(\mathbf{W}^{\ell}_t\mathbf{r}^{\ell-1}_t). Note that when τw≫τv\tau_w\gg\tau_v, both (4) and (6) can be applied synchronously. Additionally, one could adjust the feedback weights, Qℓ\mathbf{Q}^{\ell}, to reduce ∥ut∥\lVert u_t\rVert, but in general a wide range of fixed feedback weights is sufficient.

Theoretical analysis of predictive coding as deep feedback control

Taken at face value, the activity dynamics of predictive coding (2) and deep feedback control (4) appear distinct. Our aim is to show that these two frameworks are closely related—specifically, that PC can be approximated by a specific DFC controller. For the remainder of this section, we focus on the activity dynamics, which occur much faster than the weight dynamics (τv≪τw\tau_v\ll\tau_w). In general, both PC and DFC follow equivalent weight dynamics (see (3) and (6), which differ only in activity vs. potential space). We use PC weight dynamics in our final proposed scheme, with the local error eℓe^\ell replaced by the control signal uℓu^\ell.

In PC, following (2), the potentials of layers ℓ<L\ell<L evolve according to (see Appendix § 9 for more):

τvdvtℓdt=−etℓ+(Jtℓ+1)Tetℓ+1=−vtℓ+Wtℓrtℓ−1+(Jtℓ+1)Tetℓ+1(7)\tau_v\frac{\mathrm{d}\mathbf{v}^{\ell}_t}{\mathrm{d}t}=-\mathbf{e}^{\ell}_t+\left(\mathbf{J}^{\ell+1}_t\right)^{\mathsf{T}}\mathbf{e}^{\ell+1}_t=-\mathbf{v}^{\ell}_t+\mathbf{W}^{\ell}_t\mathbf{r}^{\ell-1}_t+\left(\mathbf{J}^{\ell+1}_t\right)^{\mathsf{T}}\mathbf{e}^{\ell+1}_t \tag*{(7)}

Thus, we see that PC activity dynamics are very similar to DFC dynamics in (4), with the layer-wise control given by utℓ=(Jtℓ+1)Tetℓ+1u^\ell_t=\left(\mathbf{J}^{\ell+1}_t\right)^{\mathsf{T}}\mathbf{e}^{\ell+1}_t. Unlike DFC, this controller does not directly feed back the error signal from the output. Instead, it defines a recurrence where each layer’s error controls the activity of the preceding layer. So, while the dynamics as expressed in (7) are similar, PC dynamics follow an entirely different control strategy.

For the last layer, PC drives activity by clamping the output, while in DFC, the controller computes an error signal, r∗−rtL\mathbf{r}^{*}-\mathbf{r}^{L}_t. We reconcile these here by clamping the output to the target r∗\mathbf{r}^{*}. Then for PC, we have that etL=vtL−WtLrtL−1=ϕ−1(r∗)−WtLrtL−1\mathbf{e}^{L}_t=\mathbf{v}^{L}_t-\mathbf{W}^{L}_t\mathbf{r}^{L-1}_t=\phi^{-1}(\mathbf{r}^{*})-\mathbf{W}^{L}_t\mathbf{r}^{L-1}_t and ∇vtLEt=0\nabla_{\mathbf{v}^{L}_t}E_t=0 (since ϕ−1(r∗)\phi^{-1}(\mathbf{r}^{*}) is constant)5. Thus, the final layer control signal is utL=0u^{L}_t=0. During inference, rtL\mathbf{r}^{L}_t is not clamped and a non-zero error, etL=utL\mathbf{e}^{L}_t=u^{L}_t, drives the output dynamics.

This completes the control-theoretic perspective on PC, which captures its dynamics as a distributed, time-varying proportional feedback controller (see Figure 8). Indeed, [15] show that under a specific objective, PC is the optimal solution to a least-control problem. We, however, are concerned not with any control strategy but specifically with those that match DFC, in which the error computed at the output modulates hidden layers directly through feedback control. Next, we will make approximations to the distributed controller near fixed points to make PC and DFC completely equivalent.

Schematic of the “control system” view of predictive coding

Figure 8. A schematic of the “control system” view of predictive coding. The output is clamped to an arbitrary set point, r∗r^*, and each layer’s activity is modulated via a feedback controller driven by the subsequent layer’s error. Edges meet to indicate addition; edges with flat ends denote negative (inhibitory) signals, gray edges highlight control signals, and wavy edges denote clamping.

Equivalent control strategies for PC and DFC at fixed points

Once converged, the PC network is at a fixed point, so dvtℓdt=0\frac{\mathrm{d}\mathbf{v}^{\ell}_t}{\mathrm{d}t}=0. In this regime, (2) can be rewritten as etℓ=(Jtℓ+1)Tetℓ+1\mathbf{e}^{\ell}_t=\left(\mathbf{J}^{\ell+1}_t\right)^{\mathsf{T}}\mathbf{e}^{\ell+1}_t. This implies that for PC networks at a fixed point,

utℓ=(Jtℓ+1)Tetℓ+1=(∏k=ℓ+1LJtk)TetL(8)\mathbf{u}^{\ell}_t=\left(\mathbf{J}^{\ell+1}_t\right)^{\mathsf{T}}\mathbf{e}^{\ell+1}_t=\left(\prod_{k=\ell+1}^{L}\mathbf{J}^{k}_t\right)^{\mathsf{T}}\mathbf{e}^{L}_t \tag*{(8)}

so the utℓu_t^\ell inputs to each layer depend only on the error in the last layer. The control signal in (8) matches that of DFC, where the error is calculated at the final layer, and the feedback weights for each layer are given by:

QtℓKp:=(∏k=ℓ+1LJtk)T,(9)Q_t^\ell K_p := \left(\prod_{k=\ell+1}^{L} J_t^k\right)^{\mathsf{T}}, \tag*{(9)}

where Kp≥0K_p \ge0 is the proportional control constant. Thus, PC at fixed points can be exactly expressed using the feedback control strategy employed by DFC (see Appendix § 9). However, the feedback weights are time-dependent. So, we make two approximations: first, that (8) and (9) hold locally near fixed points; second, that the time-dependent Jacobian can be approximated by linearizing the network dynamics around the fixed point. The latter is only required for nonlinear networks, as the Jacobians for linear networks are constant with respect to time. Our final approach, which we refer to as “Approx PC”, is summarized by

τvℓdvtℓdt=−vtℓ+Wtℓrtℓ−1+utℓwhereutℓ=(∏k=ℓ+1LJ0k)TetL(10)\tau_v^\ell\frac{\mathrm{d}v_t^\ell}{\mathrm{d}t} = -v_t^\ell+ W_t^\ell r_t^{\ell-1} + u_t^\ell \quad\text{where} \quad u_t^\ell= \left(\prod_{k=\ell+1}^{L} J_0^k\right)^{\mathsf{T}} e_t^L \tag*{(10)}

where we assume the network is initially at a fixed point at t=0t = 0.

Approximation characterization in linear networks

Assume that the network is at steady state such that (9) holds exactly. We then clamp the target to r∗r^*, which is selected to be close to the current steady-state output. This perturbs the system away from the fixed point by a small amount. There are multiple sources of error: (8) may not hold, and for nonlinear networks, the Jacobians are only exact at the fixed point. To disentangle these sources of error, we begin by considering linear networks.

In a two-layer linear network (one hidden layer), we have activity dynamics for only one layer (the hidden layer), as the input and output layers are clamped. Since QL−1=(WL)TQ^{L-1} = (W^L)^{\mathsf{T}}, DFC and PC dynamics are exactly the same. However, in a three-layer linear network, the hidden layers’ activity dynamics are not identical for the PC and DFC cases. PC dynamics follow:

τv˙t1=−(vt1−Wt1x)+(Wt2)T[vt2−Wt2vt1]τv˙t2=−(vt2−Wt2vt1)+(Wt3)T[r∗−Wt3vt2](11)\begin{aligned} \tau\dot{v}_t^1 &= -\left(v_t^1 - W_t^1 x\right) + (W_t^2)^{\mathsf{T}}\left[v_t^2 - W_t^2 v_t^1\right] \\ \tau\dot{v}_t^2 &= -\left(v_t^2 - W_t^2 v_t^1\right) + (W_t^3)^{\mathsf{T}}\left[r^* - W_t^3 v_t^2\right] \tag*{(11)} \end{aligned}

In contrast, DFC dynamics evolve according to:

τv˙t1=−(vt1−Wt1x)+(Wt2)T(Wt3)T[r∗−Wt3vt2]τv˙t2=−(vt2−Wt2vt1)+(Wt3)T[r∗−Wt3vt2](12)\begin{aligned} \tau\dot{v}_t^1 &= -\left(v_t^1 - W_t^1 x\right) + (W_t^2)^{\mathsf{T}}(W_t^3)^{\mathsf{T}}\left[r^* - W_t^3 v_t^2\right] \\ \tau\dot{v}_t^2 &= -\left(v_t^2 - W_t^2 v_t^1\right) + (W_t^3)^{\mathsf{T}}\left[r^* - W_t^3 v_t^2\right] \tag*{(12)} \end{aligned}

We show in Appendix § 11 that the last layer’s dynamics (vt2v_t^2) can be described by second-order ordinary differential equations for both the DFC and PC cases, i.e.,

τ2v¨t2PC+τAPCv˙t2PC+Bvt2PC=C(13)\tau^2 \ddot{v}_t^{2\mathrm{PC}} + \tau A_{\mathrm{PC}}\dot{v}_t^{2\mathrm{PC}} + Bv_t^{2\mathrm{PC}} = C \tag*{(13)}
τ2v¨t2DFC+τADFCv˙t2DFC+Bvt2DFC=C(14)\tau^2 \ddot{v}_t^{2\mathrm{DFC}} + \tau A_{\mathrm{DFC}}\dot{v}_t^{2\mathrm{DFC}} + Bv_t^{2\mathrm{DFC}} = C \tag*{(14)}

where (13) follows from (11) and (14) from (12). Notably, the only coefficients that differ between these two systems are the damping coefficients, where APC−ADFC=Wt2(Wt2)TA_{\mathrm{PC}} - A_{\mathrm{DFC}} = W_t^2(W_t^2)^{\mathsf{T}}. This suggests that after being perturbed from their initial fixed point, both systems converge to the same steady state given by Bv2=CBv^2 = C. However, the transient dynamics will differ, with PC dynamics being more damped (as Wt2(Wt2)T⪰0W_t^2(W_t^2)^{\mathsf{T}} \succeq0).

We confirm these results empirically in Figure 2. Given a network and input, we ensure it is at a fixed point by running PC dynamics without output clamping until convergence. Then, from the converged fixed point, we perturb the system by clamping the output to a target a known distance away (reported as relative “perturbation size”). For a three-layer network, as seen in Figure 2 (left), both PC and our approximation converge to the same point, though they have different dynamics. As we increase the number of layers in the network to eight, the dynamics become oscillatory and converge to different values (Figure 2).

Differences in dynamics of PC and our approximation in their first two principal components

Figure 2. Differences in dynamics of PC and our approximation in their first two principal components. The clamped target for all plots is the same noise perturbation direction. See Appendix § 12 for details. (Left) The potential trajectories for individual layers across networks of increasing depth. The three-layer network converges to the same endpoint as PC, while the six- and eight-layer networks converge to different values with L2L_2-error on the order of 10−710^{-7} and 10−210^{-2}, respectively. (Right) L2L_2-norm of the difference in final potentials across layers and perturbation sizes, for the three networks on the left.

Looking across networks of different depth and perturbations of varying size, we observe that both perturbation magnitude and depth increase the worst-case error (Figure 3). Notably, the error is numerically zero for networks with two or three layers, as predicted by our theory. Empirically, while this appears to extend to four-layer networks, deeper networks exhibit persistent residual error between PC and Approx PC, suggesting that the approximation may not be reliable away from fixed points. In the following section, we examine this residual error more generally for nonlinear networks and discuss techniques to mitigate it, allowing our framework to remain applicable beyond local regimes.

Maximum L2-norm error across layers in networks of varying depth for different perturbation sizes

Figure 3. Maximum L2L_2-norm error across layers in networks of varying depth for different perturbation sizes. “X” indicates that the error is ≤10−10\le10^{-10}. See Appendix § 12 for details.

Approximation characterization in nonlinear networks

While linear networks allow us to characterize how violating the assumptions in (8) affects the approximate dynamics, we would also like to understand how the linearization of the network Jacobian produces errors away from the fixed point. To do this, we introduce the notion of relinearization time points—a set of time points, T\mathcal{T}, that denote when to recompute the Jacobian of the network. Specifically, we modify (10) to

τvdvtℓdt=−vtℓ+Wtℓrtℓ−1+utℓ\tau_v \frac{\mathrm{d}v_t^\ell}{\mathrm{d}t}=-v_t^\ell+\mathbf{W}_t^\ell r_t^{\ell-1}+u_t^\ell

where

utℓ=(∏k=ℓ+1LJtk)TetLandt′=arg min⁡t′′∈Tt′′<t  t−t′′(15)u_t^\ell=\left(\prod_{k=\ell+1}^{L}\mathbf{J}_t^k\right)^{\mathsf{T}}e_t^L \quad\text{and}\quad t'=\underset{\substack{t''\in\mathcal{T}\\t''<t}}{\operatorname{arg\,min}}\;t-t'' \tag*{(15)}

In practice, T\mathcal{T} is defined based on a relinearization frequency, i.e., the Jacobians are recomputed every NN steps. We examine the effect of relinearization frequency on the network dynamics for a nonlinear network in Figure 4. Without relinearization, the network fails to converge to the same final potentials as PC. With infrequent relinearization, Approx PC approaches the same endpoint as PC, but the dynamics overshoot, requiring more total integration steps and relinearizations (on the order of 10210^2) to converge. This persists until a relinearization frequency of 25 steps, at which point the dynamics are still underdamped compared to PC, but overshoot no longer occurs. As a result, the system converges faster and requires an order of magnitude fewer relinearizations. Increasing the relinearization frequency beyond this point does not fundamentally change the dynamics, and only increases the total number of relinearizations without meaningfully reducing the final endpoint difference between PC and Approx PC.

Differences in dynamics of PC and Approx PC in their first two principal components as the frequency of relinearization is adjusted

Figure 4. Differences in dynamics of PC and Approx PC in their first two principal components as the frequency of relinearization is adjusted. Never relinearizing corresponds to following the dynamics of (10). See Appendix § 12 for details. (Left) Trajectories of the potentials of the fourth layer in a six-layer network with tanh activation functions. (Right) L2L_2-norm error in the endpoints of PC and Approx PC across layers decreases as relinearization frequency increases. The number of relinearizations (gray line) stays ≈\approx constant until an abrupt change around a frequency of 25 steps.

We note that there is nothing fundamental about 25 steps. Instead, given the initial fixed point of the system, its nonlinear dynamics, and a particular perturbation, 25 steps represents the distance the system is allowed to move away from the fixed point without incurring significant error due to linearization. We measure this in Figure 5 (left) as the difference in flow w.r.t. the initial linearized flow as we move along the Approx PC trajectory without relinearization. Essentially, this measures at each timestep how much the dynamics would change if we relinearize. The peak at 25 steps indicates the moment at which we would achieve the largest error correction. By relinearizing at this point, we move more efficiently toward the PC endpoint and quickly benefit from the slow dynamics of the nonlinear system around the PC endpoint (Figure 5, right). Thus, we conclude that relinearization can reduce the residual error of Approx PC efficiently, provided that we do not move “too far” before reorienting the dynamics.

Change in flow w.r.t. initial linearization and percentage of early convergence

Figure 5. (Left) The L2L_2-norm of the change in flow w.r.t. the initial linearized flow moving along the Approx PC trajectory without relinearization in Figure 4. (Right) Percentage of segments between linearizations where the dynamics of Approx PC converge (i.e., the speed of dynamics ≤10−8\le10^{-8}) earlier than the relinearization frequency. See Appendix § 12 for details.

Resolving residual errors with nudging

In the previous section, we showed that Approx PC without relinearization exhibits residual errors compared to PC for all but the simplest linear models. While relinearization addresses this issue, its timing greatly affects dynamical efficiency, and this depends on the network, the current fixed point, and the target. In a realistic training scenario, it is not clear how these three factors interact.

To address this, we introduce a multi-stage algorithm (Algorithm 1) in which the trajectory toward a target is segmented into multiple smaller subgoals. Each subgoal is a “nudge,” i.e., a partial step toward the target, computed as a fraction of the linear path between the current output and the target (see step 2(a) of Algorithm 1). Importantly, after each nudge, the dynamics are allowed to relax to a new fixed point, after which a new subgoal is computed and the next nudge is initiated.

Figure 6 shows the resulting trajectories when we apply Algorithm 1. If the nudge scaling factor, γ\gamma, is small, the network follows a frequent relinearization-like trajectory (albeit with small oscillations to convergence between nudges). With large scaling factors, we observe a new behavior where the trajectory initially deviates from the PC endpoint significantly (similar to never relinearizing); however, since the network is allowed to relax to a fixed point, the next nudge reorients the trajectory toward the PC endpoint. This is in contrast to infrequent relinearization, which keeps overshooting the PC endpoint. As expected, the final difference between the endpoints of PC and Approx PC with nudges is small (of the same order as with relinearization). Notably, this applies to large scaling factors as well, where the total number of nudges is small. In this way, nudging preserves the low residual error of relinearization without consuming a significant number of additional integration steps.

Differences in dynamics of PC and Approx PC in their first two principal components for different nudge scalings

Figure 6. Differences in dynamics of PC and Approx PC in their first two principal components for different nudge scalings. A scaling factor of one corresponds to following the dynamics of (10). The perturbation size is one. See Appendix § 12 for details. (Left) Trajectories of the potentials of the fourth layer in a six-layer network with tanh activation functions for two different nudge scaling factors. Nudges are recomputed at relinearization points. (Right) L2L_2-norm error in the endpoints of PC and Approx PC across layers decreases with smaller scaling factors. The number of nudges (gray line) consistently decreases with larger scaling factors. The horizontal dashed line indicates the convergence threshold for nudging.

Algorithm 1 Approx PC with nudges

  1. Requires: a network of LL layers at fixed point v∗\mathbf{v}_*, input x\mathbf{x}, target y\mathbf{y}, nudge scaling factor γ∈(0,1]\gamma\in(0,1], nudge convergence threshold ϵ>0\epsilon> 0, dynamics convergence threshold δ>0\delta> 0

  2. while ∥v∗L−y∥>ϵ\lVert\mathbf{v}_*^L-\mathbf{y}\rVert> \epsilon do (a) compute nudge: r∗←v∗L+γ(y−v∗L)\mathbf{r}_* \leftarrow\mathbf{v}_*^L+\gamma(\mathbf{y}-\mathbf{v}_*^L) (b) clamp network: rL←r∗\mathbf{r}^L \leftarrow\mathbf{r}_* (c) compute Jacobian: Jℓ←\mathbf{J}^{\ell} \leftarrow layer-wise Jacobian at current timestep for input x\mathbf{x} for all ℓ\ell (d) run dynamics until convergence: do

    i. eL←vL−WLrL−1\mathbf{e}^L \leftarrow\mathbf{v}^L-\mathbf{W}^L\mathbf{r}^{L-1}

    ii. uℓ←(∏k=ℓ+1LJk)TeL\mathbf{u}^{\ell} \leftarrow\left(\prod_{k=\ell+1}^{L}\mathbf{J}^{k}\right)^{\mathsf{T}}\mathbf{e}^L for all ℓ\ell

    iii. vprevℓ←vℓ\mathbf{v}_{\mathrm{prev}}^{\ell} \leftarrow\mathbf{v}^{\ell}

    iv. vℓ←vℓ+τv−1(−vℓ+Wℓrℓ−1+uℓ)\mathbf{v}^{\ell} \leftarrow\mathbf{v}^{\ell}+\tau_{\mathbf{v}}^{-1}(-\mathbf{v}^{\ell}+\mathbf{W}^{\ell}\mathbf{r}^{\ell-1}+\mathbf{u}^{\ell}) for all ℓ\ell

    v. Wℓ←Wℓ+τw−1[uℓ(rℓ−1)T]\mathbf{W}^{\ell} \leftarrow\mathbf{W}^{\ell}+\tau_{\mathbf{w}}^{-1}[\mathbf{u}^{\ell}(\mathbf{r}^{\ell-1})^{\mathsf{T}}] for all $\ell`

    while ∥vℓ−vprevℓ∥>δ\lVert\mathbf{v}^{\ell}-\mathbf{v}_{\mathrm{prev}}^{\ell}\rVert> \delta

    (e) update fixed point: v∗←v\mathbf{v}_* \leftarrow\mathbf{v}

Approx PC with learning

To evaluate whether our Approx PC algorithm remains faithful to PC in practical settings with learning, we implemented and examined both approaches on standard supervised image classification tasks (see Appendix § 12 for details). We first tested whether networks trained with Approx PC converge to solutions that are compatible with predictive coding. To this end, we trained a network using Approx PC and then directly transplanted the learned weights into a PC model without further training for evaluation. Approx PC networks achieved a final test accuracy of 97.32 ±\pm 0.12% on MNIST across five seeds, and the corresponding PC networks achieved an evaluated test accuracy of 97.47 ±\pm 0.05% across the same seeds. This confirms that Approx PC learns solutions compatible with PC dynamics.

For both MNIST and FashionMNIST, Approx PC reaches accuracy similar to or higher than that of standard PC at their respective best learning rates (Figure 7). Interestingly, Approx PC also seems to learn faster per integration step (Figure 10). This could be due to the fact that Approx PC propagates a centralized error, allowing layers to converge concurrently, while PC uses distributed error signals that must propagate across the network one layer at a time. However, Approx PC also uses multiple nudges per sample, so the improvement may instead come from breaking each learning step into intermediate subgoals. To isolate this effect, we implemented PC with nudges, where the network is trained using intermediate nudges as clamped outputs but with standard PC dynamics. We found that adding nudges to PC recovers most of the learning performance and efficiency enjoyed by Approx PC (Figure 7 and Figure 10). This suggests that most of the gain over standard PC comes from nudging rather than from the centralized controller. However, for the same number of samples seen, Approx PC and PC with nudges require more total integration steps than standard PC. Finally, hyperparameter sweeps on FashionMNIST (Table 1) show that both methods degrade similarly with network depth, but differ in their sensitivity to other hyperparameters. In particular, Approx PC is less sensitive to the neural time constant and dynamics convergence tolerance, but has a slightly narrower range of effective learning rates.

MNIST and FashionMNIST test performance across learning rates

Figure 7. Final test accuracy on MNIST (left) and FashionMNIST (right) across learning rates for PC, Approx PC, and PC with nudges (mean and standard deviation across five seeds). Dashed lines show previously reported test accuracies for DFC [13], Strong-DFC [14], and standard backprop (via [14]). Some configurations did not learn and are excluded. “X” denotes configurations with no successful runs. Learning rate := τw−1\tau_w^{-1}.

LRTime constantDynamics conv. tol.Nudge conv. tol.# of layersPC Acc. (%)Approx PC Acc. (%)
(time constant)
1×10−81 \times10^{-8}51×10−21 \times10^{-2}1×10−21 \times10^{-2}216.38±2.2216.38 \pm2.2285.87±0.0585.87 \pm0.05
1×10−81 \times10^{-8}101×10−21 \times10^{-2}1×10−21 \times10^{-2}215.71±2.4515.71 \pm2.4585.92±0.0385.92 \pm0.03
1×10−81 \times10^{-8}201×10−21 \times10^{-2}1×10−21 \times10^{-2}214.64±2.3514.64 \pm2.3584.75±1.0384.75 \pm1.03
1×10−81 \times10^{-8}301×10−21 \times10^{-2}1×10−21 \times10^{-2}213.67±2.3213.67 \pm2.3285.70±0.0685.70 \pm0.06
1×10−81 \times10^{-8}401×10−21 \times10^{-2}1×10−21 \times10^{-2}213.02±2.2413.02 \pm2.2485.63±0.1485.63 \pm0.14
1×10−51 \times10^{-5}51×10−21 \times10^{-2}1×10−21 \times10^{-2}284.81±0.1184.81 \pm0.1167.73±2.3467.73 \pm2.34
1×10−51 \times10^{-5}101×10−21 \times10^{-2}1×10−21 \times10^{-2}283.67±0.1283.67 \pm0.1239.99±5.1539.99 \pm5.15
1×10−51 \times10^{-5}201×10−21 \times10^{-2}1×10−21 \times10^{-2}282.50±0.1482.50 \pm0.1423.99±3.2023.99 \pm3.20
1×10−51 \times10^{-5}301×10−21 \times10^{-2}1×10−21 \times10^{-2}281.41±0.1781.41 \pm0.1718.18±0.7118.18 \pm0.71
1×10−51 \times10^{-5}401×10−21 \times10^{-2}1×10−21 \times10^{-2}280.16±0.2380.16 \pm0.2315.80±0.7615.80 \pm0.76
(dyn. conv. tol.)
1×10−51 \times10^{-5}301×10−31 \times10^{-3}1×10−21 \times10^{-2}281.49±0.1781.49 \pm0.17—
1×10−81 \times10^{-8}301×10−31 \times10^{-3}1×10−21 \times10^{-2}2—85.54±0.3285.54 \pm0.32
1×10−51 \times10^{-5}301×10−21 \times10^{-2}1×10−21 \times10^{-2}281.46±0.2181.46 \pm0.21—
1×10−81 \times10^{-8}301×10−21 \times10^{-2}1×10−21 \times10^{-2}2—85.77±0.1285.77 \pm0.12
1×10−51 \times10^{-5}301×10−11 \times10^{-1}1×10−21 \times10^{-2}254.07±1.7754.07 \pm1.77—
1×10−81 \times10^{-8}301×10−11 \times10^{-1}1×10−21 \times10^{-2}2—78.57±0.6578.57 \pm0.65
(learning rate sweep)
1×10−91 \times10^{-9}301×10−21 \times10^{-2}1×10−21 \times10^{-2}213.03±2.7113.03 \pm2.7175.44±0.4575.44 \pm0.45
1×10−81 \times10^{-8}301×10−21 \times10^{-2}1×10−21 \times10^{-2}219.52±1.4019.52 \pm1.4085.73±0.0885.73 \pm0.08
1×10−71 \times10^{-7}301×10−21 \times10^{-2}1×10−21 \times10^{-2}260.45±0.2660.45 \pm0.2652.47±6.5952.47 \pm6.59
1×10−61 \times10^{-6}301×10−21 \times10^{-2}1×10−21 \times10^{-2}279.54±0.1479.54 \pm0.1415.94±0.3215.94 \pm0.32
1×10−51 \times10^{-5}301×10−21 \times10^{-2}1×10−21 \times10^{-2}283.27±0.1883.27 \pm0.1817.67±1.3617.67 \pm1.36
1×10−41 \times10^{-4}301×10−21 \times10^{-2}1×10−21 \times10^{-2}272.71±1.1672.71 \pm1.1614.58±1.2214.58 \pm1.22
1×10−31 \times10^{-3}301×10−21 \times10^{-2}1×10−21 \times10^{-2}2N/A11.84±1.8111.84 \pm1.81
(nudge conv. tol.)
1×10−51 \times10^{-5}301×10−21 \times10^{-2}1×10−31 \times10^{-3}241.45±16.4441.45 \pm16.44—
1×10−81 \times10^{-8}301×10−21 \times10^{-2}1×10−31 \times10^{-3}2—85.35±0.1485.35 \pm0.14
1×10−51 \times10^{-5}301×10−21 \times10^{-2}1×10−21 \times10^{-2}252.05±14.9652.05 \pm14.96—
1×10−81 \times10^{-8}301×10−21 \times10^{-2}1×10−21 \times10^{-2}2—81.25±3.1681.25 \pm3.16
1×10−51 \times10^{-5}301×10−21 \times10^{-2}1×10−11 \times10^{-1}279.48±2.2279.48 \pm2.22—
1×10−81 \times10^{-8}301×10−21 \times10^{-2}1×10−11 \times10^{-1}2—84.19±0.0684.19 \pm0.06
(# layers)
1×10−51 \times10^{-5}301×10−21 \times10^{-2}1×10−21 \times10^{-2}182.25±0.1382.25 \pm0.13—
1×10−81 \times10^{-8}301×10−21 \times10^{-2}1×10−21 \times10^{-2}1—84.49±0.4784.49 \pm0.47
1×10−51 \times10^{-5}301×10−21 \times10^{-2}1×10−21 \times10^{-2}281.28±0.2281.28 \pm0.22—
1×10−81 \times10^{-8}301×10−21 \times10^{-2}1×10−21 \times10^{-2}2—85.72±0.1385.72 \pm0.13
1×10−51 \times10^{-5}301×10−21 \times10^{-2}1×10−21 \times10^{-2}379.92±1.0679.92 \pm1.06—
1×10−81 \times10^{-8}301×10−21 \times10^{-2}1×10−21 \times10^{-2}3—82.49±0.9282.49 \pm0.92

Table 1. Hyperparameter sweep results for FashionMNIST. Unless a parameter is being swept, the default values are time constant = 30, dyn. conv. tol. = 1×10−21 \times10^{-2}, nudge conv. tol. = 1×10−21 \times10^{-2}, # of layers = 2, and learning rate = 1×10−51 \times10^{-5} (PC) or 1×10−81 \times10^{-8} (Approx PC). Values are mean and standard deviation over five random seeds.

Results on MNIST and FashionMNIST for PC, Approx PC, and PC + Nudges

Figure 10. Results on MNIST (left) and FashionMNIST (right) for PC, Approx PC, and PC with nudges across five seeds. The total number of training samples is fixed, but the number of integration steps per sample varies. Per integration step, Approx PC and PC with nudges improve in test accuracy more than PC. The first data point is recorded after the first sample (including multiple nudges). The x-axis is cropped to zoom in on the initial learning curve ramp-up where the effect is most apparent. Note that PC takes fewer total integration steps to process the same total number of samples, which is reflected in the MNIST results where the PC learning curve stops earlier than those of the nudged variants. Learning rate := τw−1=1×10−6\tau_w^{-1} = 1 \times10^{-6} (chosen so that the effect is easiest to visualize though this result holds across learning rates tested in the paper).

Discussion

We have shown that predictive coding and deep feedback control, two prominent frameworks for biologically plausible deep learning, can be understood within a common dynamical systems perspective. In particular, DFC provides a local approximation to PC dynamics near fixed points, revealing that these seemingly distinct approaches differ primarily in their transient dynamics rather than their steady-state behavior.

Implications for machine learning. We have shown that Approx PC reaches approximately the same fixed points as PC, while differing in its transient dynamics. Moreover, weights learned under Approx PC can be transplanted into PC networks for inference. These results suggest that part of the slow inference attributed to PC may stem from its particular dynamics rather than from the learned learning itself. In PC, errors reach early layers only by propagating one layer at a time. Separately, nudging accounted for most of the accuracy gain over standard PC in our experiments. This may partly reflect the extra weight updates that nudging entails. Another possibility is that clamping the network’s output directly to the label may cause it to reach a different equilibrium than would be reached by moving toward the label more gradually. Equilibrium propagation makes a similar argument for weak clamping, where small nudges keep the network’s new equilibrium close to its original one [23]. Whether moving the target gradually benefits energy-based learning more broadly remains to be tested.

Implications for biological interpretation. Control-theoretic and predictive coding frameworks have often been proposed as alternative accounts of credit assignment in the brain, distinguished by whether error signals are computed locally at each layer or generated centrally and broadcast through feedback pathways. Our results indicate that, near fixed points, PC dynamics can be expressed as an equivalent control strategy driven by output errors, making the two frameworks difficult to tell apart based on steady-state activity alone. Our theory instead suggests that distinguishing these mechanisms in the brain requires an analysis of transient activity. In particular, perturbations in one layer of a PC network immediately affect the preceding layer through local errors, whereas under DFC, preceding layers respond only after the perturbation has reached the output. We therefore suggest experiments that perturb one stage of a processing hierarchy and compare the timing of responses at earlier stages and at the output.

Limitations and future work. The primary purpose of our experiments was to validate predictions arising from the theoretical connection made between PC and DFC. Therefore, we focused on controlled experiments in fully connected networks, where making sense of activity and learning dynamics is more tractable. While we demonstrated the practical feasibility of Approx PC through MNIST and FashionMNIST experiments, future work should expand our approach to neural networks with more parameters and different architectures (e.g., convolutional networks), and assess whether Approx PC can be competitive with established training methods at scale. The appropriate nudge size and convergence thresholds also depend on the network and data, making them difficult to set in advance. In practice, we found the number of steps per nudge to be a useful indicator of convergence behavior, which future work could formalize into an adaptive nudging scheme. Beyond machine learning applications, an implementation of Approx PC in biology is unlikely to use the exact network Jacobian. Instead, one may consider slow-varying estimates of the Jacobian or arbitrary, structured feedback weights, whose exploration we leave to future work. Finally, we hope our work can inspire extensions that explore the use of control-theoretic improvements to PC training, such as PID control.

Acknowledgments and Disclosure of Funding

This work was supported by the Cold Spring Harbor Laboratory Fellows Program and Pershing Square Foundation. The authors declare no competing interests.

Rewriting PC activity dynamics

Starting from PC potential dynamics given by (2), we rewrite the dynamics in a form compatible with DFC. To begin, we will focus on layers ℓ<L\ell< L, where the inference and learning activity is governed by the same set of equations. Let us briefly review the derivation of (2). The gradient of the total energy is

∇vtℓEt=12∇vtℓ∥etℓ∥22+12∇vtℓ∥etℓ+1∥22(16)\nabla_{v_t^\ell} E_t = \frac{1}{2}\nabla_{v_t^\ell}\lVert e_t^\ell\rVert_2^2 + \frac{1}{2}\nabla_{v_t^\ell}\lVert e_t^{\ell+1}\rVert_2^2 \tag*{(16)}

The other terms in (1) do not depend on vℓv^\ell, so their derivatives are zero. Next, we can substitute in the layer-wise errors and proceed as follows

∇vtℓEt=12∇vtℓ∥vtℓ−Wtℓrtℓ−1∥22+12∇vtℓ∥vtℓ+1−Wtℓ+1rtℓ∥22=vtℓ−Wtℓrtℓ−1−ϕ′(vtℓ)⊙(Wtℓ+1)T(vtℓ+1−Wtℓ+1rtℓ)(17)\begin{aligned} \nabla_{v_t^\ell} E_t &= \frac{1}{2}\nabla_{v_t^\ell}\lVert v_t^\ell- W_t^\ell r_t^{\ell-1}\rVert_2^2 + \frac{1}{2}\nabla_{v_t^\ell}\lVert v_t^{\ell+1} - W_t^{\ell+1}r_t^\ell\rVert_2^2 \\ &= v_t^\ell- W_t^\ell r_t^{\ell-1} - \phi'(v_t^\ell) \odot(W_t^{\ell+1})^{\mathsf{T}}(v_t^{\ell+1} - W_t^{\ell+1}r_t^\ell) \tag*{(17)} \end{aligned}

Finally, the potential evolves according to the negative gradient of the energy, resulting in the following dynamics

τvdvtℓdt=−vtℓ+Wtℓrtℓ−1+(Jtℓ+1)Tetℓ+1(18)\tau_v\frac{\mathrm{d}v_t^\ell}{\mathrm{d}t} = -v_t^\ell+ W_t^\ell r_t^{\ell-1} + (J_t^{\ell+1})^{\mathsf{T}}e_t^{\ell+1} \tag*{(18)}

This matches the form of DFC dynamics in (4).

Schematics for PC and DFC style controllers

Schematic of DFC-style control

Figure 9. A schematic of DFC-style control. The output is clamped to an arbitrary set point, r∗r^*, and each layer’s activity is modulated via a feedback controller driven by the output error.

Derivation of PC and Approx PC exact dynamics in linear networks

In this appendix we derive the ordinary differential equations (ODEs) governing layer-wise representation for both PC and Approx PC. We consider a three-layer network (x,v1,v2,v3)(x, v^1, v^2, v^3), where the last layer v3v^3 is clamped to the training output, r∗r^*. We also consider linear activations (ϕ(vℓ)=vℓ,ϕ′(vℓ)=1)(\phi(v^\ell) = v^\ell, \phi'(v^\ell) = 1). For both PCN and DFC approximation dynamics, the goal is to yield a closed-form ODE purely in vℓv^\ell. We then define the residual between the two systems and compare them.

Throughout, we consider v1∈Rd1v^1 \in\mathbb{R}^{d_1}, v2∈Rd2v^2 \in\mathbb{R}^{d_2}, input x∈Rd0x \in\mathbb{R}^{d_0}, and target r∗∈Rd3r^* \in\mathbb{R}^{d_3}. Weight matrices are W1∈Rd1×d0\mathbf{W}^1 \in\mathbb{R}^{d_1 \times d_0}, W2∈Rd2×d1\mathbf{W}^2 \in\mathbb{R}^{d_2 \times d_1}, W3∈Rd3×d2\mathbf{W}^3 \in\mathbb{R}^{d_3 \times d_2}. We leave off the subscript tt for clarity. The time constant τv\tau_v is shared by all layers.

PC activity dynamics

Recall (2):

τv˙1=−(v1−W1x)+(W2)T[v2−W2v1](19)\tau\dot{v}^1 = -\left(v^1-\mathbf{W}^1x\right)+(\mathbf{W}^2)^T\left[v^2-\mathbf{W}^2v^1\right] \tag*{(19)}
τv˙2=−(v2−W2v1)+(W3)T[r∗−W3v2](20)\tau\dot{v}^2 = -\left(v^2-\mathbf{W}^2v^1\right)+(\mathbf{W}^3)^T\left[r^*-\mathbf{W}^3v^2\right] \tag*{(20)}

Expanding (19) and (20) and collecting terms in v1v_1 and v2v_2:

τv˙1=−(I+(W2)TW2)v1+(W2)Tv2+W1x(21)\tau\dot{v}^1=-\left(\mathbf{I}+(\mathbf{W}^2)^T\mathbf{W}^2\right)v^1+(\mathbf{W}^2)^Tv^2+\mathbf{W}^1x \tag*{(21)}
τv˙2=−(I+(W3)TW3)v2+W2v1+(W3)Tr∗(22)\tau\dot{v}^2=-\left(\mathbf{I}+(\mathbf{W}^3)^T\mathbf{W}^3\right)v^2+\mathbf{W}^2v^1+(\mathbf{W}^3)^Tr^* \tag*{(22)}

From (22) we isolate the v1v^1-dependent term:

W2v1=τv˙2+(I+(W3)TW3)v2−(W3)Tr∗(23)\mathbf{W}^2v^1=\tau\dot{v}^2+\left(\mathbf{I}+(\mathbf{W}^3)^T\mathbf{W}^3\right)v^2-(\mathbf{W}^3)^Tr^* \tag*{(23)}

Differentiating (22) with respect to time gives:

τv¨2=−(I+(W3)TW3)v˙2+W2v˙1(24)\tau\ddot{v}^2=-\left(\mathbf{I}+(\mathbf{W}^3)^T\mathbf{W}^3\right)\dot{v}^2+\mathbf{W}^2\dot{v}^1 \tag*{(24)}

Substituting v˙1\dot{v}^1 from (21) gives:

τ2v¨2=−τ(I+(W3)TW3)v˙2+W2[−(I+(W2)TW2)v1+(W2)Tv2+W1x](25)\tau^2\ddot{v}^2=-\tau\left(\mathbf{I}+(\mathbf{W}^3)^T\mathbf{W}^3\right)\dot{v}^2+\mathbf{W}^2\left[-\left(\mathbf{I}+(\mathbf{W}^2)^T\mathbf{W}^2\right)v^1+(\mathbf{W}^2)^Tv^2+\mathbf{W}^1x\right] \tag*{(25)}

Note that v1v^1 coefficient can be written as:

−W2(I+(W2)TW2)v1=−(I+W2(W2)T)(W2v1)(26)-\mathbf{W}^2\left(\mathbf{I}+(\mathbf{W}^2)^T\mathbf{W}^2\right)v^1=-\left(\mathbf{I}+\mathbf{W}^2(\mathbf{W}^2)^T\right)(\mathbf{W}^2v^1) \tag*{(26)}

and we can replace W2v1\mathbf{W}^2v^1 from (23), and write (25) as:

τ2v¨2=−τ(I+(W3)TW3)v˙2−(I+W2(W2)T)[τv˙2+(I+(W3)TW3)v2−(W3)Tr∗]+W2(W2)Tv2+W2W1x(27)\begin{aligned} \tau^2\ddot{v}^2={}&-\tau\left(\mathbf{I}+(\mathbf{W}^3)^T\mathbf{W}^3\right)\dot{v}^2\\ &-\left(\mathbf{I}+\mathbf{W}^2(\mathbf{W}^2)^T\right)\left[\tau\dot{v}^2+\left(\mathbf{I}+(\mathbf{W}^3)^T\mathbf{W}^3\right)v^2-(\mathbf{W}^3)^Tr^*\right]\\ &+\mathbf{W}^2(\mathbf{W}^2)^Tv^2+\mathbf{W}^2\mathbf{W}^1x \tag*{(27)} \end{aligned}

Grouping terms, the final PCN second-order ODE for v2v_2 can therefore be expressed as:

τ2v¨2+τ(2I+W2(W2)T+(W3)TW3)⏟APCNv˙2+(I+(W3)TW3+W2(W2)T(W3)TW3)⏟BPCNv2=W2W1x+(I+W2(W2)T)(W3)Tr∗⏟CPCN(28)\begin{aligned} \tau^2\ddot{v}^2 &+\tau\underbrace{\left(2\mathbf{I}+\mathbf{W}^2(\mathbf{W}^2)^T+(\mathbf{W}^3)^T\mathbf{W}^3\right)}_{A_{\mathrm{PCN}}}\dot{v}^2\\ &+\underbrace{\left(\mathbf{I}+(\mathbf{W}^3)^T\mathbf{W}^3+\mathbf{W}^2(\mathbf{W}^2)^T(\mathbf{W}^3)^T\mathbf{W}^3\right)}_{B_{\mathrm{PCN}}}v^2\\ &=\mathbf{W}^2\mathbf{W}^1x+\underbrace{\left(\mathbf{I}+\mathbf{W}^2(\mathbf{W}^2)^T\right)(\mathbf{W}^3)^Tr^*}_{C_{\mathrm{PCN}}} \tag*{(28)} \end{aligned}

Approx PC activity dynamics

In Approx PC, QℓQ^{\ell} is approximated by the product of the Jacobians of the following layers, that is:

τv˙1=−(v1−W1x)+(W2)T(W3)T[r∗−W3v2](29)\tau\dot{v}^{1} = -\left(v^{1} - W^{1}x\right) + \left(W^{2}\right)^{T}\left(W^{3}\right)^{T}\left[r^{*} - W^{3}v^{2}\right] \tag*{(29)}
τv˙2=−(v2−W2v1)+(W3)T[r∗−W3v2](30)\tau\dot{v}^{2} = -\left(v^{2} - W^{2}v^{1}\right) + \left(W^{3}\right)^{T}\left[r^{*} - W^{3}v^{2}\right] \tag*{(30)}

Expanding (29) and (30):

τv˙1=−v1+W1x+(W2)T(W3)Tr∗−(W2)T(W3)TW3v2(31)\tau\dot{v}^{1} = -v^{1} + W^{1}x + \left(W^{2}\right)^{T}\left(W^{3}\right)^{T}r^{*} - \left(W^{2}\right)^{T}\left(W^{3}\right)^{T}W^{3}v^{2} \tag*{(31)}
τv˙2=−(I+(W3)TW3)v2+W2v1+(W3)Tr∗(32)\tau\dot{v}^{2} = -\left(I + \left(W^{3}\right)^{T}W^{3}\right)v^{2} + W^{2}v^{1} + \left(W^{3}\right)^{T}r^{*} \tag*{(32)}

From (32) we isolate the v1v^{1}-dependent term:

W2v1=τv˙2+(I+(W3)TW3)v2−(W3)Tr∗(33)W^{2}v^{1} = \tau\dot{v}^{2} + \left(I + \left(W^{3}\right)^{T}W^{3}\right)v^{2} - \left(W^{3}\right)^{T}r^{*} \tag*{(33)}

Differentiating (32):

τ2v¨2=−τ(I+(W3)TW3)v˙2+W2(τv˙1)(34)\tau^{2}\ddot{v}^{2} = -\tau\left(I + \left(W^{3}\right)^{T}W^{3}\right)\dot{v}^{2} + W^{2}\left(\tau\dot{v}^{1}\right) \tag*{(34)}

Replacing v˙1\dot{v}^{1} from (31) gives:

τ2v¨2=−τ(I+(W3)TW3)v˙2+W2[−v1+W1x+(W2)T(W3)Tr∗−(W2)T(W3)TW3v2](35)\begin{aligned} \tau^{2}\ddot{v}^{2} ={}& -\tau\left(I + \left(W^{3}\right)^{T}W^{3}\right)\dot{v}^{2}\\ &+ W^{2}\left[-v^{1} + W^{1}x + \left(W^{2}\right)^{T}\left(W^{3}\right)^{T}r^{*} - \left(W^{2}\right)^{T}\left(W^{3}\right)^{T}W^{3}v^{2}\right] \tag*{(35)} \end{aligned}

And finally replacing W2v1W^{2}v^{1} from (33) and grouping terms gives us the final DFC second-order ODE:

τ2v¨2+τ(2I+(W3)TW3)⏟ADFCv˙2+(I+(W3)TW3+W2(W2)T(W3)TW3)⏟BDFCv2=W2W1x+(I+W2(W2)T)(W3)Tr∗⏟CDFC(36)\begin{aligned} \tau^{2}\ddot{v}^{2} &+ \underbrace{\tau\left(2I + \left(W^{3}\right)^{T}W^{3}\right)}_{A_{\mathrm{DFC}}}\dot{v}^{2}\\ &+ \underbrace{\left(I + \left(W^{3}\right)^{T}W^{3} + W^{2}\left(W^{2}\right)^{T}\left(W^{3}\right)^{T}W^{3}\right)}_{B_{\mathrm{DFC}}}v^{2}\\ &= W^{2}W^{1}x + \underbrace{\left(I + W^{2}\left(W^{2}\right)^{T}\right)\left(W^{3}\right)^{T}r^{*}}_{C_{\mathrm{DFC}}} \tag*{(36)} \end{aligned}

Residual between PCN and DFC

Comparing (28) and (36), the two second-order ODEs have identical stiffness (B) and forcing (C) terms, and differ only in their damping matrices:

APCN−ADFC=W2(W2)T.(37)A_{\mathrm{PCN}} - A_{\mathrm{DFC}} = W^{2}\left(W^{2}\right)^{T}. \tag*{(37)}

Hence any fixed point of one system is also a fixed point of the other, and the two differ only in their transient dynamics, with PC carrying an additional damping term.

Experimental details

Hyperparameters

In this section we specify the hyperparameters used to generate the results presented in this paper. For all experiments, the proportional control constant is Kp=1K_{p} = 1. For Figures 2 to 6, the width of hidden layers is eight, the input and output dimensions are one, and the neural dynamics time constant is τv=10\tau_{v} = 10. The perturbation direction is a single unit vector sampled from N(0,I)\mathcal{N}(0, I) and shared across comparisons; only its magnitude is varied where specified. For Figures 2 and 3, the dynamics convergence tolerance is 1×10−151 \times10^{-15}. For Figures 4 to 6, we used networks with five hidden layers and a dynamics convergence tolerance of 1×10−81 \times10^{-8}. For Figure 6, the nudge convergence tolerance is 1×10−101 \times10^{-10}. For the MNIST and FashionMNIST experiments, by default, two hidden layers of width 256 are used; the neural dynamics time constant is τv=30\tau_{v} = 30; the dynamics convergence tolerance is 1×10−21 \times10^{-2}; the nudge convergence tolerance is 1×10−21 \times10^{-2}; the nudge scaling factor is γ=0.1\gamma= 0.1; networks have a bias (all other experiments have no bias); and we use the Adam optimizer with weight decay of 1×10−41 \times10^{-4}. Figures 2 and 3 concern linear networks (identity activation functions), while all other experiments use tanh activation functions in the hidden layers.

Computational resources

For Figure 2 to 6, we used a MacBook Pro with M2 processor and 16GB of RAM. For Figure 7 and 10 and Table 1, we used a single H100 GPU with 96GB of VRAM per learning rate and seed configuration.

Additional experimental results

Broader Impact

Our work is focused on making a theoretical link between two biologically plausible learning frameworks, and therefore has limited societal implications. However, biologically plausible learning rules, in general, contribute to work enabling energy-efficient training of artificial intelligence (AI) models. On one hand, this can enable training of state-of-the-art models, such as large language models, with limited hardware resources, allowing more individuals to access this technology and guide its development. On the other hand, more accessible training for AI models could expand misuse and documented harms caused by this technology.

References

  1. [1]Mohamed Akrout, Collin Wilson, Peter C. Humphreys, Timothy Lillicrap, and Douglas Tweed. Deep learning without weight transport. In Advances in Neural Information Processing Systems, 2019.
  2. [2]David Balduzzi, Hastagiri Vanchinathan, and Joachim Buhmann. Kickback Cuts Backprop’s Red-Tape: Biologically Plausible Credit Assignment in Neural Networks. In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, 2015. URL https://www.aaai.org/ocs/index.php/AAAI/AAAI15/paper/view/9467/9284.arxiv.org/abs/1411.6191
  3. [3]Brian Crafton, Abhinav Parihar, Evan Gebhardt, and Arijit Raychowdhury. Direct Feedback Alignment With Sparse Connections for Local Learning. Frontiers in Neuroscience, 13:525, 2019. doi: 10.3389/fnins.2019.00525. URL https://www.frontiersin.org/article/10.3389/fnins.2019.00525/full.DOI
  4. [4]Kyle Daruwalla and Mikko Lipasti. Information bottleneck-based Hebbian learning rule naturally ties working memory and synaptic updates. Frontiers in Computational Neuroscience, 18:1240348, 2024. doi: 10.3389/fncom.2024.1240348. URL https://www.frontiersin.org/articles/10.3389/fncom.2024.1240348/full.DOI
  5. [5]Karl Friston. A theory of cortical responses. Philosophical Transactions of the Royal Society B: Biological Sciences, 360(1456):815–836, 2005.pubmed.ncbi.nlm.nih.gov/15937014
  6. [6]Nicolas Frémaux and Wulfram Gerstner. Neuromodulated Spike-Timing-Dependent Plasticity, and Theory of Three-Factor Learning Rules. Frontiers in Neural Circuits, 9:85, 2016. doi: 10.3389/fncir.2015.00085. URL http://journal.frontiersin.org/Article/10.3389/fncir.2015.00085/abstract.DOI
  7. [7]Max Jaderberg, Wojciech Marian Czarnecki, Simon Osindero, Oriol Vinyals, Alex Graves, David Silver, and Koray Kavukcuoglu. Decoupled neural interfaces using synthetic gradients. In Proceedings of the 34th International Conference on Machine Learning, 2017.arxiv.org/abs/1608.05343
  8. [8]Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998. doi: 10.1109/5.726791. 10.1109/5.726791.
  9. [9]Dong-Hyun Lee, Saizheng Zhang, Asja Fischer, and Yoshua Bengio. Difference target propagation. In Annalisa Appice, Pedro Pereira Rodrigues, Vítor Santos Costa, Carlos Soares, João Gama, and Alípio Jorge, editors, Machine Learning and Knowledge Discovery in Databases, pages 498–515, Cham, 2015. Springer International Publishing.
  10. [10]Timothy P. Lillicrap, Daniel Cownden, Douglas B. Tweed, and Colin J. Akerman. Random feedback weights support learning in deep neural networks. arXiv:1411.0247, 2014. URL http://arxiv.org/abs/1411.0247.
  11. [11]Timothy P. Lillicrap, Adam Santoro, Luke Marris, Colin J. Akerman, and Geoffrey Hinton. Back-propagation and the brain. Nature Reviews Neuroscience, 21(6):335–346, 2020. doi: 10.1038/s41583-020-0277-3. URL http://www.nature.com/articles/s41583-020-0277-3.
  12. [12]Alexander Meulemans, Francesco S. Carzaniga, Johan A.K. Suykens, João Sacramento, and Benjamin F. Grewe. A theoretical framework for target propagation. In Advances in Neural Information Processing Systems, 2020.
  13. [13]Alexander Meulemans, Matilde Tristany Farinha, Javier Garcia Ordonez, Pau Vilemelis Aceituno, João Sacramento, and Benjamin F. Grewe. Credit Assignment in Neural Networks through Deep Feedback Control. In Advances in Neural Information Processing Systems, volume 34, pages 4674–4687. Curran Associates, Inc., 2021.arxiv.org/abs/2106.07887
  14. [14]Alexander Meulemans, Matilde Tristany Farinha, Maria R. Cervera, João Sacramento, and Benjamin F. Grewe. Minimizing Control for Credit Assignment with Strong Feedback. In Proceedings of the 39th International Conference on Machine Learning, pages 15458–15483. PMLR, June 2022.arxiv.org/abs/2204.07249
  15. [15]Alexander Meulemans, Nicolas Zuchet, Seijin Kobayashi, Johannes von Oswald, and João Sacramento. The least-control principle for local learning at equilibrium. In Advances in Neural Information Processing Systems, volume 35, pages 36303–36317, December 2022.
  16. [16]Beren Millidge, Anil Seth, and Christopher L. Buckley. Predictive coding: A theoretical and experimental review. arXiv:2107.12979, 2022.arxiv.org/abs/2107.12979
  17. [17]Randall C. O’Reilly. Biologically plausible error-driven learning using local activation differences: The generalized recirculation algorithm. Neural Computation, 8(5):895–938, 1996.
  18. [18]Alexander G. Ororbia. Brain-Inspired Machine Intelligence: A Survey of Neurobiologically-Plausible Credit Assignment. arXiv:2312.09257, 2023. URL http://arxiv.org/abs/2312.09257.
  19. [19]Luca Pinchetti, Tommaso Salvatori, Yordan Yordanov, Beren Millidge, Yuhang Song, and Thomas Lukasiewicz. Predictive coding beyond Gaussian distributions. In Advances in Neural Information Processing Systems, 2022.arxiv.org/abs/2211.03481
  20. [20]Roman Pogodin and Peter E. Latham. Kernelized information bottleneck leads to biologically plausible 3-factor Hebbian learning in deep networks. In Advances in Neural Information Processing Systems, volume 33. Curran Associates, Inc., 2020. URL https://proceedings.neurips.cc/paper/2020/file/517f24c02e620d5a4dac1db388664a63-Paper.pdf.arxiv.org/abs/2006.07123
  21. [21]Rajesh P. N. Rao and Dana H. Ballard. Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nature Neuroscience, 2(1):79–87, 1999.pubmed.ncbi.nlm.nih.gov/10195184
  22. [22]João Sacramento, Rui Ponte Costa, Yoshua Bengio, and Walter Senn. Dendritic cortical microcircuits approximate the backpropagation algorithm. In Advances in Neural Information Processing Systems, 2018.arxiv.org/abs/1810.11393
  23. [23]Benjamin Scellier and Yoshua Bengio. Equilibrium Propagation: Bridging the Gap between Energy-Based Models and Backpropagation. Frontiers in Computational Neuroscience, 11:24, 2017. doi: 10.3389/fncom.2017.00024. URL http://journal.frontiersin.org/article/10.3389/fncom.2017.00024/full.DOI
  24. [24]James C. R. Whittington and Rafal Bogacz. An approximation of the error backpropagation algorithm in a predictive coding network with local Hebbian synaptic plasticity. Neural Computation, 29(5):1229–1262, 2017.DOI
  25. [25]Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-MNIST: A Novel Image Dataset for Benchmarking Machine Learning Algorithms. arXiv:1708.07747, 2017. URL http://arxiv.org/abs/1708.07747.

Paper details

vlnc:W_bmp2c9n7
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 9

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents