Meta-learning with differentiable closed-form solvers

05/21/2018
by   Luca Bertinetto, et al.
0

Adapting deep networks to new concepts from few examples is extremely challenging, due to the high computational and data requirements of standard fine-tuning procedures. Most works on meta-learning and few-shot learning have thus focused on simple learning techniques for adaptation, such as nearest neighbors or gradient descent. Nonetheless, the machine learning literature contains a wealth of methods that learn non-deep models very efficiently. In this work we propose to use these fast convergent methods as the main adaptation mechanism for few-shot learning. The main idea is to teach a deep network to use standard machine learning tools, such as logistic regression, as part of its own internal model, enabling it to quickly adapt to novel tasks. This requires back-propagating errors through the solver steps. While normally the matrix operations involved would be costly, the small number of examples works to our advantage, by making use of the Woodbury identity. We propose both iterative and closed-form solvers, based on logistic regression and ridge regression components. Our methods achieve excellent performance on three few-shot learning benchmarks, showing competitive performance on Omniglot and surpassing all state-of-the-art alternatives on miniImageNet and CIFAR-100.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/31/2022

HyperMAML: Few-Shot Adaptation of Deep Models with Hypernetworks

The aim of Few-Shot learning methods is to train models which can easily...
research
10/05/2019

Transductive Episodic-Wise Adaptive Metric for Few-Shot Learning

Few-shot learning, which aims at extracting new concepts rapidly from ex...
research
10/11/2021

A Closer Look at Prototype Classifier for Few-shot Image Classification

The prototypical network is a prototype classifier based on meta-learnin...
research
04/30/2020

Physarum Powered Differentiable Linear Programming Layers and Applications

Consider a learning algorithm, which involves an internal call to an opt...
research
05/05/2021

How Fine-Tuning Allows for Effective Meta-Learning

Representation learning has been widely studied in the context of meta-l...
research
06/25/2019

Learning Features with Differentiable Closed-Form Solver for Tracking

We present a novel and easy-to-implement training framework for visual t...
research
08/06/2023

Prototypes-oriented Transductive Few-shot Learning with Conditional Transport

Transductive Few-Shot Learning (TFSL) has recently attracted increasing ...

Please sign up or login with your details

Forgot password? Click here to reset