Equivariance Through Parameter-Sharing

02/27/2017
by   Siamak Ravanbakhsh, et al.
0

We propose to study equivariance in deep neural networks through parameter symmetries. In particular, given a group G that acts discretely on the input and output of a standard neural network layer ϕ_W: ^M→^N, we show that ϕ_W is equivariant with respect to G-action iff G explains the symmetries of the network parameters W. Inspired by this observation, we then propose two parameter-sharing schemes to induce the desirable symmetry on W. Our procedures for tying the parameters achieve G-equivariance and, under some conditions on the action of G, they guarantee sensitivity to all other permutation groups outside G.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset