DeepAI
Log In Sign Up

Neural Complexity Measures

08/07/2020
by   Yoonho Lee, et al.
24

While various complexity measures for diverse model classes have been proposed, specifying an appropriate measure capable of predicting and explaining generalization in deep networks has proven to be challenging. We propose Neural Complexity (NC), an alternative data-driven approach that meta-learns a scalar complexity measure through interactions with a large number of heterogeneous tasks. The trained NC model can be added to the standard training loss to regularize any task learner under standard learning frameworks. We contrast NC's approach against existing manually-designed complexity measures and also against other meta-learning models, and validate NC's performance on multiple regression and classification tasks.

READ FULL TEXT

page 1

page 2

page 3

page 4

05/20/2018

Task-Agnostic Meta-Learning for Few-shot Learning

Meta-learning approaches have been proposed to tackle the few-shot learn...
05/12/2015

Permutational Rademacher Complexity: a New Complexity Measure for Transductive Learning

Transductive learning considers situations when a learner observes m lab...
06/09/2022

Learning to generate imaginary tasks for improving generalization in meta-learning

The success of meta-learning on existing benchmarks is predicated on the...
07/07/2020

Meta-Learning with Network Pruning

Meta-learning is a powerful paradigm for few-shot learning. Although wit...
12/04/2019

Fantastic Generalization Measures and Where to Find Them

Generalization of deep networks has been of great interest in recent yea...
08/10/2018

How Complex is your classification problem? A survey on measuring classification complexity

Extracting characteristics from the training datasets of classification ...
03/09/2022

SuperCone: Modeling Heterogeneous Experts with Concept Meta-learning for Unified Predictive Segments System

Understanding users through predicative segments play an essential role ...

Code Repositories

neural-complexity

Official repository, Neural Complexity Measures (NeurIPS 2020)


view repo