WorksMathematics
On Hamilton-Jacobi equations and perceptron models
Abstract
The solution to a finite-dimensional Hamilton-Jacobi equation is classically known to admit a variational representation in two situations: when the nonlinearity is convex or concave, by the Hopf-Lax formula; and when the initial condition is convex or concave, by the Hopf formula. We identify a third situation: the space variable is split into two groups of coordinates, the nonlinearity is the scalar product of the two corresponding partial gradients, and the initial condition is the sum of a convex function of the first group of coordinates and of an arbitrary Lipschitz function of the second group. This structure is precisely that of the Hamilton-Jacobi equations describing the limit free energy of perceptron models. We illustrate this point by identifying the limit free energy of a class of perceptron models that can be rewritten as bipartite spin glasses, with Ising spins in one layer and bounded soft spins in the other. We also discuss how to extend the approach to several other classes of models, including general Ising and spherical perceptrons, possibly with labelled patterns.