MetaNODE: Prototype Optimization as a Neural ODE for Few-Shot Learning

03/26/2021
by   Baoquan Zhang, et al.
13

Few-Shot Learning (FSL) is a challenging task, i.e., how to recognize novel classes with few examples? Pre-training based methods effectively tackle the problem by pre-training a feature extractor and then predict novel classes via a nearest neighbor classifier with mean-based prototypes. Nevertheless, due to the data scarcity, the mean-based prototypes are usually biased. In this paper, we diminish the bias by regarding it as a prototype optimization problem. Although the existing meta-optimizers can also be applied for the optimization, they all overlook a crucial gradient bias issue, i.e., the mean-based gradient estimation is also biased on scarce data. Consequently, we regard the gradient itself as meta-knowledge and then propose a novel prototype optimization-based meta-learning framework, called MetaNODE. Specifically, we first regard the mean-based prototypes as initial prototypes, and then model the process of prototype optimization as continuous-time dynamics specified by a Neural Ordinary Differential Equation (Neural ODE). A gradient flow inference network is carefully designed to learn to estimate the continuous gradients for prototype dynamics. Finally, the optimal prototypes can be obtained by solving the Neural ODE using the Runge-Kutta method. Extensive experiments demonstrate that our proposed method obtains superior performance over the previous state-of-the-art methods. Our code will be publicly available upon acceptance.

READ FULL TEXT

page 1

page 3

page 4

page 5

page 6

page 7

page 8

page 9

research
08/11/2021

Prototype Completion for Few-Shot Learning

Few-shot learning aims to recognize novel classes with few examples. Pre...
research
09/10/2020

Prototype Completion with Primitive Knowledge for Few-Shot Learning

Few-shot learning is a challenging task, which aims to learn a classifie...
research
03/03/2022

MetaDT: Meta Decision Tree for Interpretable Few-Shot Learning

Few-Shot Learning (FSL) is a challenging task, which aims to recognize n...
research
11/25/2019

Prototype Rectification for Few-Shot Learning

Few-shot learning is a challenging problem that requires a model to reco...
research
09/11/2019

Learning to Propagate for Graph Meta-Learning

Meta-learning extracts the common knowledge acquired from learning diffe...
research
06/11/2023

Compositional Prototypical Networks for Few-Shot Classification

It is assumed that pre-training provides the feature extractor with stro...

Please sign up or login with your details

Forgot password? Click here to reset