
On Connected Sublevel Sets in Deep Learning
We study sublevel sets of the loss function in training deep neural netw...
read it

Evaluating representations by the complexity of learning lowloss predictors
We consider the problem of evaluating representations of data for use in...
read it

The Loss Surfaces of Multilayer Networks
We study the connection between the highly nonconvex loss function of a...
read it

Analytic Definition of Curves and Surfaces by Parabolic Blending
A procedure for interpolating between specified points of a curve or sur...
read it

Loss Surface Simplexes for Mode Connecting Volumes and Fast Ensembling
With a better understanding of the loss surfaces for multilayer networks...
read it

Optimizing Mode Connectivity via Neuron Alignment
The loss landscapes of deep neural networks are not well understood due ...
read it

Locating Transparent Objects to Millimetre Accuracy
Transparent surfaces, such as glass, transmit most of the visible light ...
read it
Lowloss connection of weight vectors: distributionbased approaches
Recent research shows that sublevel sets of the loss surfaces of overparameterized networks are connected, exactly or approximately. We describe and compare experimentally a panel of methods used to connect two lowloss points by a lowloss curve on this surface. Our methods vary in accuracy and complexity. Most of our methods are based on "macroscopic" distributional assumptions, and some are insensitive to the detailed properties of the points to be connected. Some methods require a prior training of a "global connection model" which can then be applied to any pair of points. The accuracy of the method generally correlates with its complexity and sensitivity to the endpoint detail.
READ FULL TEXT
Comments
There are no comments yet.