Networks that compute by settling, and how to train them to generate
One of the most promising ideas for energy-efficient AI is networks that don’t compute with a forward pass at all.
Instead, every neuron updates at the same time, each one evolving downhill on a shared energy landscape, until the whole network settles into a stable state at a local energy minimum. The answer is read from the output neurons once everything has settled. Settling like this is one of the most common things physical systems do, and plausibly part of how the brain computes too. Many analogue and optical chips can do it natively, in a fraction of the time and energy it takes to simulate on a GPU.
The catch is training them. Backprop needs a completely separate backward pass through the network, which this kind of hardware can’t easily do and which is widely thought to be implausible in the brain. Equilibrium Propagation only uses the one thing these systems are already good at: settling. Let the network settle, then let it settle again with its outputs gently nudged towards the right answer, and each connection learns from how its own activity changed between the two. In effect, it reshapes the energy landscape so that its local minima move to places where the output neurons have the right values.
That’s enough for classification, where all that matters is the final output neuron values. But modern generative models, such as diffusion and flow matching, turn noise into data gradually. What they need to learn is the velocity of the visible neurons at every point, which is set by the slope of the energy everywhere, not just where its minima are.
In my ICONS 2026 paper, I introduce a method called GradEP, which finds a related energy landscape where moving the minima there corresponds to reshaping the slope of the original landscape. So EP’s one ability, moving local energy minima, becomes a way to learn velocities too, using purely local learning.
The paper won Best Paper at ICONS, and it’s what made my recent PotentialEP result possible: a 50M-parameter generative model trained without backprop, reaching 3.64 FID on CIFAR-10 against 3.54 with backprop.
Receiving the Best Paper Award at ICONS 2026.
- GradEP paper (FlowEqProp): published version (ACM) · free preprint (arXiv)
- PotentialEP: arxiv.org/abs/2610.14409