Equivariant bifurcation, quadratic equivariants, and symmetry breaking for the standard representation of S_n

07/06/2021
by   Yossi Arjevani, et al.
0

Motivated by questions originating from the study of a class of shallow student-teacher neural networks, methods are developed for the analysis of spurious minima in classes of gradient equivariant dynamics related to neural nets. In the symmetric case, methods depend on the generic equivariant bifurcation theory of irreducible representations of the symmetric group on n symbols, S_n; in particular, the standard representation of S_n. It is shown that spurious minima do not arise from spontaneous symmetry breaking but rather through a complex deformation of the landscape geometry that can be encoded by a generic S_n-equivariant bifurcation. We describe minimal models for forced symmetry breaking that give a lower bound on the dynamic complexity involved in the creation of spurious minima when there is no symmetry. Results on generic bifurcation when there are quadratic equivariants are also proved; this work extends and clarifies results of Ihrig Golubitsky and Chossat, Lauterback Melbourne on the instability of solutions when there are quadratic equivariants.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/12/2022

Annihilation of Spurious Minima in Two-Layer ReLU Networks

We study the optimization problem associated with fitting two-layer ReLU...
research
03/10/2021

Symmetry Breaking in Symmetric Tensor Decomposition

In this note, we consider the optimization problem associated with compu...
research
03/23/2020

Symmetry critical points for a model shallow neural network

A detailed analysis is given of a family of critical points determining ...
research
05/17/2018

A spin glass model for reconstructing nonlinearly encrypted signals corrupted by noise

An encryption of a signal s∈R^N is a random mapping sy=(y_1,...,y_M)^T...
research
11/19/2020

On the geometry of symmetry breaking inequalities

Breaking symmetries is a popular way of speeding up the branch-and-bound...
research
12/14/2017

Data Structures for Representing Symmetry in Quadratically Constrained Quadratic Programs

Symmetry in mathematical programming may lead to a multiplicity of solut...
research
12/26/2019

Spurious Local Minima of Shallow ReLU Networks Conform with the Symmetry of the Target Model

We consider the optimization problem associated with fitting two-layer R...

Please sign up or login with your details

Forgot password? Click here to reset