Depth creates no more spurious local minima

01/28/2019

∙

by   Li Zhang, et al.

∙

0

∙

share

We show that for any convex differentiable loss function, a deep linear network has no spurious local minima as long as it is true for the two layer case. When applied to the quadratic loss, our result immediately implies the powerful result in [Kawaguchi 2016] that there is no spurious local minima in deep linear networks. Further, with the recent work [Zhou and Liang 2018], we can remove all the assumptions in [Kawaguchi 2016]. Our proof is short and elementary. It builds on the recent work of [Laurent and von Brecht 2018] and uses a new rank one perturbation argument.