betaFeedback

WorksMathematics

Target-Clamped Contrastive Learning in Nonlinear Resistive Networks with Monotone Elements

Abstract

In the pursuit of energy-efficient hardware for machine learning, nonlinear resistive networks trained via local learning rules are emerging as promising alternatives to digital neural networks. Learning in these physical systems is often achieved with Equilibrium Propagation (EP), which compares a "free" state to a "nudged" target-driven state to perform gradient descent on an objective defined by an explicit cost function. In contrast, Target-Clamped Contrastive Learning (TCCL) "clamps" output nodes directly to exact target values, performing gradient descent on an implicit contrastive function - an objective whose properties have long remained unknown. Focusing on nonlinear networks composed of resistors with monotonically increasing current-voltage characteristics (such as Ohmic resistors and diodes), our contribution is twofold. First, we establish rigorous convergence guarantees for TCCL. We prove that the contrastive function is convex for nonlinear resistive networks without hidden nodes, and demonstrate that this property does not hold in general for the objective function optimized by EP. For these networks, TCCL converges to a global minimum of the contrastive function under suitable assumptions on the learning rate. In the presence of hidden nodes, the contrastive function is not convex, but we show that TCCL still converges to a stationary point on a subsequence. We also reveal cases where such stationary points are global minimizers of the contrastive function. Second, we characterize the representational capacity of these circuits. We establish a novel function approximation theorem demonstrating that any continuous, monotonically non-decreasing, and shift-equivariant function can be approximated to arbitrary precision, proving that these properties are the sole limitations on the network's computational expressivity.

Introduction

The escalating energy demands of modern machine learning have driven a resurgence of interest in analog computing paradigms and the development of training methods dedicated to such platforms [34]. By replacing the digital von Neumann architecture with analog compute-in-memory substrates, these systems promise substantial improvements in energy efficiency for both inference and training. Among these alternative platforms, resistive networks have attracted special interest.

A resistive network is an electrical circuit comprised of (linear or nonlinear) resistors, with nodes typically partitioned into input, hidden and output sets. In such a network where computation is driven by the circuit’s physics, the input node potentials are clamped to specific values, while the output node potentials serve as the network’s prediction. Conductances of variable resistors serve as the trainable parameters, and the objective is to find parameter configurations that implement a desired input-output mapping. It is well-known that the hidden and output potentials of a linear resistive network minimize the total dissipated power; in nonlinear resistive networks—those containing nonlinear one-port resistive elements—the hidden and output potentials extremize a more general notion known as the co-content [32], also referred to as “pseudo-power” [27]. This property allows resistive networks to function as energy-based models where the co-content acts as the energy function. For such systems, energy-based learning rules have been developed which adjust conductances based solely on local voltage measurements, allowing decentralized training [3, 17, 25, 27]. On the hardware side, small-scale prototypes have successfully demonstrated self-learning capabilities on toy tasks [16, 17]. On the simulation side, SPICE-class simulations of modest-size nonlinear resistive networks have been performed on MNIST [27, 51], while recent methods exploit the co-content minimization formulation to achieve speedups of several orders of magnitude [37, 40], paving the way for large-scale simulations.

To rival artificial neural networks, resistive networks must demonstrate representational capacity, and learning capabilities. Learning in resistive networks was initially formalized by [27] through Equilibrium Propagation (EP), an energy-based learning framework that enables gradient descent on an arbitrary cost function [42]. EP is part of a broader class of energy-based learning methods [43] that optimize parameters by comparing the network in two different states: the inference state (called free state), and a perturbed state where outputs are clamped or nudged towards a target. Other energy-based learning rules differ from EP in the way the perturbed state is defined. In particular, the Contrastive Learning (CL) algorithm of [25] (adapted from [35, 43]) clamps output nodes to exact target values in the perturbed state, effectively performing gradient descent on a nonnegative contrastive function whose minimum of zero is attained precisely when the network outputs match the target output. In [25], it was shown that the contrastive function is convex for linear networks without hidden nodes, establishing the convergence for a range of step sizes. These results were also recently extended to linear resistive-capacitive networks in [23]. On the representational capacity front, [44] showed that nonlinear resistive networks can approximate arbitrary continuous functions, provided they incorporate two-port elements like voltage-controlled voltage sources (VCVSs).

In this paper, we establish mathematical foundations for learning and representation in nonlinear resistive networks constructed exclusively from one-port elements. This deliberately precludes the VCVSs utilized by [44]. We assume all circuit components possess strictly monotonically increasing current-voltage characteristics, which encompasses Ohmic resistors and diodes, and yields a strictly convex network co-content. Within this framework, we answer two fundamental questions: under what conditions does learning reliably converge, and what is the network’s representational capacity?

While [25] proved the convergence of the CL algorithm with clamped outputs, their analysis was limited to linear networks without hidden nodes. Our first main contribution extends their framework to nonlinear networks. We prove that the contrastive function is convex for nonlinear networks without hidden nodes, guaranteeing convergence of the parameters to a global minimum. This is in contrast with the EP objective function, which is in general non-convex, even without hidden nodes. For the more expressive and general case that includes hidden nodes, the contrastive function is no longer convex, but in this case, we prove that the learning dynamics converge on a subsequence to a stationary point. Interestingly, any of such stationary points in the interior of the constraint set is shown to be a global minimum of the contrastive function.

Our second main contribution analyzes the representational expressivity of these circuits. We demonstrate that the input-output mappings of such networks are monotonically non-decreasing and shift-equivariant (translation-equivariant). Remarkably, we establish a novel approximation theorem proving that these two physical constraints are the only limitations on expressivity. Any continuous function satisfying monotonicity and shift equivariance can be approximated to arbitrary precision by a nonlinear resistive network featuring strictly convex co-content functions. To prove this, we introduce a novel circuit construction leveraging scalable, smooth approximations of min-max functions.

Finally, we clarify a terminological ambiguity present in the recent literature on physical learning algorithms. The term “contrastive learning” is used in two distinct ways: as the specific learning rule involving clamped targets [23, 25, 43], and as a broad umbrella term for the entire class of energy-based learning methods [17, 19, 34, 49]. This broader category includes other algorithms such as coupled learning [2, 17, 48], which employs a weaker form of clamping that fixes outputs to a convex combination of the free outputs and targets. To avoid confusion, we introduce the term Target-Clamped Contrastive Learning (TCCL) to explicitly refer to the specific learning rule that utilizes clamped targets. For brevity, we occasionally abbreviate this to Contrastive Learning (CL) when the context is unambiguous, in line with the terminology of [23, 25, 43].

We note a recent concurrent study by [30] on the computational expressivity of thermodynamic networks; however, their framework permits elements whose current depends on terminal voltages independently rather than on their difference, meaning the network non-equilibrium steady state cannot generally be characterized as the minimum of a co-content function, and energy-based learning algorithms such as TCCL or EP cannot be directly employed.

Another recent work [31] studied the convergence of energy-based learning algorithms for linear resistive circuits. This work is complementary to the current paper, in the sense that it deals with local convergence results for EP and coupled learning in linear circuits, whereas our study focuses on global convergence analysis for TCCL in nonlinear circuits.

The remainder of this paper is organized as follows. § 2 introduces nonlinear resistive circuits and co-content functions. Section III formalizes the learning problem. § 4 discusses the contrastive learning framework. § 5 analyzes the convexity of the contrastive function, and Section VI establishes convergence guarantees for the learning dynamics. Section VII defines the fundamental physical constraints of the circuits and proves the approximation theorem. Finally, conclusions are provided in Section VIII. The reader is also directed to Appendix A for relevant notation, and to Appendix B for preliminaries on convex, coercive and Lipschitz continuous functions.

Modeling of Nonlinear Resistive Circuits

We start by introducing a few graph-theoretic concepts used throughout the paper. For a general reference on graph theory, we refer the reader to [11]. We consider a connected undirected graph

G=(V,E),\mathcal{G} = (\mathcal{V},\mathcal{E}),

consisting of a set of nodes V={1,2,…,N}\mathcal{V} = \{1,2,\ldots,N\} and a set of branches (or edges) E\mathcal{E}, containing BB unordered pairs of nodes. The incidence matrix of G\mathcal{G} is defined as follows. First, we denote the edges in the graph by e1,e2,…,eBe_1,e_2,\ldots,e_B, and we assign an arbitrary orientation to each edge. Then, the incidence matrix D∈RN×BD \in\mathbb{R}^{N \times B} is defined element-wise by

Dij={−1if edge ej leaves node i,1if edge ej enters node i,0otherwise.D_{ij} = \begin{cases} -1 & \text{if edge } e_j \text{ leaves node } i,\\ 1 & \text{if edge } e_j \text{ enters node } i,\\ 0 & \text{otherwise.} \end{cases}

By definition of the incidence matrix, we have that im⁡1⊆ker⁡DT\operatorname{im}\mathbf{1} \subseteq\ker D^{\mathsf{T}}, where 1\mathbf{1} denotes the vector of ones. Moreover, since G\mathcal{G} is connected, it follows from [21 Thm. 8.3.1] that

ker⁡DT=im⁡1.(1)\ker D^{\mathsf{T}} = \operatorname{im}\mathbf{1}. \tag*{(1)}

Next, we consider the vectors p∈RNp \in\mathbb{R}^{N} of nodal voltage potentials, and j∈RNj \in\mathbb{R}^{N} of external currents injected into each of the nodes. The vectors v∈RBv \in\mathbb{R}^{B} and i∈RBi \in\mathbb{R}^{B} denote the voltages and currents over and through the edges, respectively. Obviously, due to the laws of Kirchhoff, we have

v=DTpandj=Di.(2)v = D^{\mathsf{T}}p \quad\text{and} \quad j = Di. \tag*{(2)}

Every branch k∈{1,2,…,B}k \in\{1,2,\ldots,B\} contains a nonlinear resistor modeled as

ik=dGkdvk(vk).(3)i_k = \frac{dG_k}{dv_k}(v_k). \tag*{(3)}

Here Gk:R→RG_k:\mathbb{R}\to\mathbb{R} is the so-called co-content function [32] of the kk-th resistor, which we assume to be differentiable, coercive and strictly convex. Note that the strict convexity of GkG_k is equivalent to dGkdvk\frac{dG_k}{dv_k} being strictly monotonically increasing. Thus, (3) defines a strictly monotone relation between the current and voltage at branch kk.

Remark 1. Some simple examples of co-content functions are as follows. The case Gk(vk)=12gkvk2G_k(v_k) = \frac{1}{2}g_kv_k^2, where gk>0g_k > 0 is a conductance, covers linear resistors described by ik=gkvki_k = g_kv_k. If Gk(vk)=evk−vkG_k(v_k) = e^{v_k} - v_k, then ik=evk−1i_k = e^{v_k} - 1 is an (idealized) model of a p-n diode, see e.g., [13]. Both of these examples of GkG_k are differentiable, coercive and strictly convex. An example of a co-content function that does not satisfy the strict convexity assumption is that of a tunnel diode [15 Ch. 2], whose current-voltage characteristic is not monotonically increasing.

Next, we define the “total” co-content function G:RB→RG:\mathbb{R}^{B}\to\mathbb{R} as

G(v):=∑k=1BGk(vk).G(v) := \sum_{k=1}^{B} G_k(v_k).

The function GG inherits properties of the functions GkG_k. In particular, GG is differentiable, coercive and strictly convex. We also observe that

∇G(v)=[dG1dv1(v1)dG2dv2(v2)⋮dGBdvB(vB)].\nabla G(v) = \begin{bmatrix} \dfrac{dG_1}{dv_1}(v_1)\\ \dfrac{dG_2}{dv_2}(v_2)\\ \vdots\\ \dfrac{dG_B}{dv_B}(v_B) \end{bmatrix}.

This implies that, by Kirchhoff’s laws (2) and the model of the resistors (3), we can succinctly write the relation between jj and pp as

j=D∇G(DTp).(4)j = D\nabla G(D^{\mathsf{T}}p). \tag*{(4)}

In the context of learning in electrical circuits, it will become useful to partition the set of nodes V\mathcal{V} into two or multiple sets (corresponding to sets of input, hidden, and output nodes). In what follows, we will derive some preliminary results to aid that discussion.

Let V=V1∪V2\mathcal{V} = \mathcal{V}_1 \cup\mathcal{V}_2 where V1\mathcal{V}_1 and V2\mathcal{V}_2 are disjoint. After possible relabeling of the nodes, this yields the partitioned vectors and matrix:

p=[p1p2],j=[j1j2],D=[D1D2],p = \begin{bmatrix} p_1\\ p_2 \end{bmatrix}, \quad j = \begin{bmatrix} j_1\\ j_2 \end{bmatrix}, \quad D = \begin{bmatrix} D_1\\ D_2 \end{bmatrix},

where p1,j1∈RN1p_1,j_1 \in\mathbb{R}^{N_1}, p2,j2∈RN2p_2,j_2 \in\mathbb{R}^{N_2}, and D1∈RN1×BD_1 \in\mathbb{R}^{N_1 \times B}, D2∈RN2×BD_2 \in\mathbb{R}^{N_2 \times B}. Here N1,N2∈NN_1,N_2 \in\mathbb{N} are positive integers such that N1+N2=NN_1 + N_2 = N.

We set the injected currents in the V2\mathcal{V}_2-nodes to zero, i.e., j2=0j_2 = 0. By (4), this results into the equations

\begin{bmatrix} j_1 \\ 0 \end{bmatrix} = \begin{bmatrix} D_1 \\ D_2 \end{bmatrix} \nabla G(D_1^{\mathsf{T}}p_1 + D_2^{\mathsf{T}}p_2). \tag*{(5)}\end{bmatrix}

Note that for given p1p_1 and p2p_2, the currents j1j_1 are uniquely defined by the first N1N_1 rows of (5). In addition, the last N2N_2 rows of (5) provide a relation between p1p_1 and p2p_2. Indeed,

0=D2∇G(D1Tp1+D2Tp2).(6)0 = D_2\nabla G(D_1^{\mathsf{T}}p_1 + D_2^{\mathsf{T}}p_2). \tag*{(6)}

We will view (6) as an input-output model with input p1p_1 and corresponding output p2p_2. The following lemma clarifies the existence and uniqueness of such an output p2p_2, given p1p_1.

Lemma 2. Assume that the function Gk:R→RG_k : \mathbb{R} \to\mathbb{R} is differentiable, coercive and strictly convex for all k=1,2,…,Bk = 1, 2, \ldots, B. Then, for every p1∈RN1p_1 \in\mathbb{R}^{N_1}, the function H:RN2→RH : \mathbb{R}^{N_2} \to\mathbb{R} defined as

H(p2)=G(D1Tp1+D2Tp2)H(p_2) = G(D_1^{\mathsf{T}}p_1 + D_2^{\mathsf{T}}p_2)

is differentiable, coercive and strictly convex. Moreover, for every p1∈RN1p_1 \in\mathbb{R}^{N_1}, there exists a unique p2∈RN2p_2 \in\mathbb{R}^{N_2} satisfying (6). This p2p_2 is the unique minimizer of HH.

References

  1. [1]D. H. Ackley, G. E. Hinton, and T. J. Sejnowski. A learning algorithm for Boltzmann machines. Cognitive Science, 9(1):147–169, 1985.
  2. [2]L. E. Altman, M. Stern, A. J. Liu, and D. J. Durian. Experimental demonstration of coupled learning in elastic networks. Physical Review Applied, 22:024053, Aug 2024.arxiv.org/abs/2311.00170
  3. [3]V. R. Anisetti, A. Kandala, B. Scellier, and J. M. Schwarz. Frequency propagation: Multimechanism learning in nonlinear physical networks. Neural Computation, 36(4):596–620, 2024.DOI
  4. [4]H. H. Bauschke and P. L. Combettes. The Baillon-Haddad theorem revisited. Journal of Convex Analysis, 17:781–787, 2010.arxiv.org/abs/0906.0807
  5. [5]H. H. Bauschke and P. L. Combettes. Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Springer, 2011.
  6. [6]A. Beck. First-Order Methods in Optimization. MOS-SIAM Series on Optimization. Society for Industrial and Applied Mathematics, 2017.DOI
  7. [7]A. Berman and R. J. Plemmons. Nonnegative Matrices in the Mathematical Sciences. Society for Industrial and Applied Mathematics, 1994.
  8. [8]D. P. Bertsekas. Convex Optimization Theory. Athena Scientific Optimization and Computation Series. Athena Scientific, 2009.
  9. [9]D. P. Bertsekas. Convex Optimization Algorithms. Athena Scientific, 2015.
  10. [10]D. P. Bertsekas. Nonlinear Programming. Athena Scientific, 3rd edition, 2016.
  11. [11]B. Bollobás. Modern Graph Theory. Graduate Texts in Mathematics 184. Springer-Verlag New York, 1st edition, 1998.DOI
  12. [12]L. Bottou, F. E. Curtis, and J. Nocedal. Optimization methods for large-scale machine learning. SIAM Review, 60(2):223–311, 2018.arxiv.org/abs/1606.04838
  13. [13]L. O. Chua, C. A. Desoer, and E. S. Kuh. Linear and Nonlinear Circuits. McGraw-Hill, New York, 1987.
  14. [14]G. Cybenko. Approximation by superpositions of a sigmoidal function. Mathematics of Control, Signals and Systems, 2:303–314, 1989.
  15. [15]C. A. Desoer and E. S. Kuh. Basic Circuit Theory. McGraw-Hill Book Company, 1969.
  16. [16]S. Dillavou, B. D. Beyer, M. Stern, A. J. Liu, M. Z. Miskin, and D. J. Durian. Machine learning without a processor: Emergent learning in a nonlinear analog network. Proceedings of the National Academy of Sciences, 121(28):e2319718121, 2024.DOI
  17. [17]S. Dillavou, M. Stern, A. J. Liu, and D. J. Durian. Demonstration of decentralized physics-driven learning. Physical Review Applied, 18(1):014040, 2022.DOI
  18. [18]D. E. Dobbs. On the smoothness condition in euler’s theorem on homogeneous functions. International Journal of Mathematical Education in Science and Technology, 49(8):1250–1259, 2018.DOI
  19. [19]Y. Du, R. van Mastrigt, J. Veenstra, and C. Coulais. Metamaterials that learn to change shape. Nature Physics, pages 1–7, 2026.DOI
  20. [20]W. H. Fleming. Functions of Several Variables. Springer-Verlag, 2nd edition, 1977.
  21. [21]C. Godsil and G. F. Royle. Algebraic Graph Theory. Springer, 2001.DOI
  22. [22]J. Harris, C. Koch, J. Luo, and J. Wyatt. Resistive Fuses: Analog Hardware for Detecting Discontinuities in Early Vision, pages 27–55. Springer US, Boston, MA, 1989.DOI
  23. [23]M. Heidema, B. Besselink, and H. J. van Waarde. Open-energy-based learning in resistor-capacitor circuits. In Proceedings of the European Control Conference, pages 356–361, 2026.
  24. [24]R. Høier, T. Kerjan, and B. Scellier. Training a convergent energy transformer with equilibrium propagation. In New Frontiers in Associative Memories-Workshop at ICLR 2026, 2026.
  25. [25]M. A. Huijzer, T. Chaffey, B. Besselink, and H. J. van Waarde. Convergence of energy-based learning in linear resistive networks. IEEE Transactions on Automatic Control, 2026.
  26. [26]C. Igel. Smooth min-max monotonic networks. arXiv:2306.01147, 2023.arxiv.org/abs/2306.01147
  27. [27]J. Kendall, R. Pantone, K. Manickavasagam, Y. Bengio, and B. Scellier. Training end-to-end analog neural networks with equilibrium propagation. arXiv:2006.01981, 2020.
  28. [28]T. Kerjan, R. Høier, and B. Scellier. Training a predictive coding network on imagenet using equilibrium propagation. arXiv:2606.03584, 2026.
  29. [29]H. K. Khalil. Nonlinear Systems. Pearson, 3rd edition, 2014.
  30. [30]P. Lipka-Bartosik, G. Blasi, J. L. Puértolas, G. Haack, M. Pernau-Llobet, and N. Brunner. Thermodynamic networks: Harnessing non-equilibrium steady states for computation. arXiv:2605.15985, 2026.
  31. [31]J. A. McGinnis, X. Li, and Y. Mori. Coercivity and local convergence of physical learning in linear circuits. arXiv:2606.15443, 2026.
  32. [32]W. J. Millar. Some general theorems for non-linear systems possessing resistance. Philosophical Magazine Series 1, 42:1150–1160, 1951.
  33. [33]A. Minin, M. Velikova, B. Lang, and H. Daniels. Comparison of universal approximators incorporating partial monotonicity by structure. Neural Networks, 23(4):471–475, 2010.
  34. [34]A. Momeni, B. Rahmani, B. Scellier, L. G. Wright, P. L. McMahon, C. C. Wanjura, Y. Li, A. Skalli, N. G. Berloff, T. Onodera, et al. Training of physical neural networks. Nature, 645(8079):53–61, 2025.arxiv.org/abs/2406.03372
  35. [35]J. R. Movellan. Contrastive Hebbian Learning in the Continuous Hopfield Model. In Connectionist Models, pages 10–17. Elsevier, 1991.
  36. [36]E. T. Nguyen. Energy-based learning in nonlinear resistive networks. Master’s thesis, University of Groningen, 2026.
  37. [37]F. Osana, J. Grollier, and D. Querioz. Fast simulation of nonlinear deep resistive networks for energy-based computation. arXiv:2609.07356, 2026.
  38. [38]W. Rudin. Principles of Mathematical Analysis. International series in pure and applied mathematics. McGraw-Hill, third edition, 1976.
  39. [39]E. K. Ryu and W. Yin. Large-Scale Convex Optimization: Algorithms and Analyses via Monotone Operators. Cambridge University Press, 2022.
  40. [40]B. Scellier. A fast algorithm to simulate nonlinear resistive networks. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 434477–43503. PMLR, 21–27 Jul 2024.arxiv.org/abs/2402.11674
  41. [41]B. Scellier. Quantum equilibrium propagation: Gradient-descent training of quantum systems. APL Quantum, 3(2):026117, 2026.arxiv.org/abs/2406.00879
  42. [42]B. Scellier and Y. Bengio. Equilibrium propagation: Bridging the gap between energy-based models and backpropagation. Frontiers in computational neuroscience, 11:24, 2017.
  43. [43]B. Scellier, M. Ernoult, J. Kendall, and S. Kumar. Energy-based learning algorithms for analog computing: a comparative study. In Advances in Neural Information Processing Systems, volume 36, pages 52705–52731. Curran Associates, Inc., 2023.
  44. [44]B. Scellier and S. Mishra. Universal approximation theorem for nonlinear resistive networks. Physical Review Applied, 23:044009, Apr 2025.
  45. [45]Benjamin Scellier. A deep learning theory for neural networks grounded in physics. PhD thesis, Université de Montréal, 2021.
  46. [46]J. Sill. Monotonic networks. In Advances in Neural Information Processing Systems (NeurIPS), volume 10, 1997.
  47. [47]M. Stern, S. Dillavou, D. Jayaraman, D. J. Durian, and A. J. Liu. Training self-learning circuits for power-efficient solutions. APL Machine Learning, 2(1):016114, 02 2024.
  48. [48]M. Stern, D. Hexner, J. W. Rocks, and A. J. Liu. Supervised learning in physical networks: From machine learning to learning machines. Physical Review X, 11:021045, May 2021.
  49. [49]M. Stern and A. Murugan. Learning without neurons in physical systems. Annual Review of Condensed Matter Physics, 14:417–441, 2023.
  50. [50]Q. Wang, C. C. Wanjura, and F. Marquardt. Training coupled phase oscillators as a neuromorphic platform using equilibrium propagation. Neuromorphic Computing and Engineering, 4(3):034014, Sep. 2024.
  51. [51]M. Watfa, A. Garcia-Ortiz, and G. Sassatelli. Energy-based analog neural network framework. Frontiers in Computational Neuroscience, 17:1114651, 2023.
  52. [52]Nicolas Zuchet and Joao Sacramento. Beyond backpropagation: bilevel optimization through implicit differentiation and equilibrium propagation. Neural Computation, 34(12):2309–2346, 2022.

Paper details

vlnc:W_x3hbde8q
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 3

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Mathematics

Not recorded for this paper

View statement
Looking for related papers…
Contents