Learn2Hop: Learned Optimization on Rough Landscapes

07/20/2021
by   Amil Merchant, et al.
7

Optimization of non-convex loss surfaces containing many local minima remains a critical problem in a variety of domains, including operations research, informatics, and material design. Yet, current techniques either require extremely high iteration counts or a large number of random restarts for good performance. In this work, we propose adapting recent developments in meta-learning to these many-minima problems by learning the optimization algorithm for various loss landscapes. We focus on problems from atomic structural optimization–finding low energy configurations of many-atom systems–including widely studied models such as bimetallic clusters and disordered silicon. We find that our optimizer learns a 'hopping' behavior which enables efficient exploration and improves the rate of low energy minima discovery. Finally, our learned optimizers show promising generalization with efficiency gains on never before seen tasks (e.g. new elements or compositions). Code will be made available shortly.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/29/2023

StriderNET: A Graph Reinforcement Learning Approach to Optimize Atomic Structures on Rough Energy Landscapes

Optimization of atomic structures presents a challenging problem, due to...
research
02/27/2017

Depth Creates No Bad Local Minima

In deep learning, depth, as well as nonlinearity, create non-convex loss...
research
02/03/2023

Learning to Optimize for Reinforcement Learning

In recent years, by leveraging more data, computation, and diverse tasks...
research
09/12/2016

Non-square matrix sensing without spurious local minima via the Burer-Monteiro approach

We consider the non-square matrix sensing problem, under restricted isom...
research
11/19/2019

Information-Theoretic Local Minima Characterization and Regularization

Recent advances in deep learning theory have evoked the study of general...
research
06/12/2020

Complex Dynamics in Simple Neural Networks: Understanding Gradient Flow in Phase Retrieval

Despite the widespread use of gradient-based algorithms for optimizing h...
research
07/18/2023

Promoting Exploration in Memory-Augmented Adam using Critical Momenta

Adaptive gradient-based optimizers, particularly Adam, have left their m...

Please sign up or login with your details

Forgot password? Click here to reset