Equivalence of Equilibrium Propagation and Recurrent Backpropagation
share
Recurrent Backpropagation and Equilibrium Propagation are algorithms for fixed point recurrent neural networks which differ in their second phase. In the first phase, both algorithms converge to a fixed point which corresponds to the configuration where the prediction is made. In the second phase, Recurrent Backpropagation computes error derivatives whereas Equilibrium Propagation relaxes to another nearby fixed point. In this work we establish a close connection between these two algorithms. We show that, at every moment in the second phase, the temporal derivatives of the neural activities in Equilibrium Propagation are equal to the error derivatives computed iteratively in Recurrent Backpropagation. This work shows that it is not required to have a special network for the computation of error derivatives, and gives support to the hypothesis that, in biological neural networks, temporal derivatives of neural activities may code for error signals.
READ FULL TEXT