Neural Complexity Measures

by   Yoonho Lee, et al.

While various complexity measures for diverse model classes have been proposed, specifying an appropriate measure capable of predicting and explaining generalization in deep networks has proven to be challenging. We propose Neural Complexity (NC), an alternative data-driven approach that meta-learns a scalar complexity measure through interactions with a large number of heterogeneous tasks. The trained NC model can be added to the standard training loss to regularize any task learner under standard learning frameworks. We contrast NC's approach against existing manually-designed complexity measures and also against other meta-learning models, and validate NC's performance on multiple regression and classification tasks.


page 1

page 2

page 3

page 4


Task-Agnostic Meta-Learning for Few-shot Learning

Meta-learning approaches have been proposed to tackle the few-shot learn...

Learning to generate imaginary tasks for improving generalization in meta-learning

The success of meta-learning on existing benchmarks is predicated on the...

Permutational Rademacher Complexity: a New Complexity Measure for Transductive Learning

Transductive learning considers situations when a learner observes m lab...

Meta-Learning with Network Pruning

Meta-learning is a powerful paradigm for few-shot learning. Although wit...

Fantastic Generalization Measures and Where to Find Them

Generalization of deep networks has been of great interest in recent yea...

How Complex is your classification problem? A survey on measuring classification complexity

Extracting characteristics from the training datasets of classification ...

SuperCone: Modeling Heterogeneous Experts with Concept Meta-learning for Unified Predictive Segments System

Understanding users through predicative segments play an essential role ...

Code Repositories


Official repository, Neural Complexity Measures (NeurIPS 2020)

view repo

Please sign up or login with your details

Forgot password? Click here to reset