Attentional meta-learners are polythetic classifiers

by   Ben Day, et al.

Polythetic classifications, based on shared patterns of features that need neither be universal nor constant among members of a class, are common in the natural world and greatly outnumber monothetic classifications over a set of features. We show that threshold meta-learners require an embedding dimension that is exponential in the number of features to emulate these functions. In contrast, attentional classifiers are polythetic by default and able to solve these problems with a linear embedding dimension. However, we find that in the presence of task-irrelevant features, inherent to meta-learning problems, attentional models are susceptible to misclassification. To address this challenge, we further propose a self-attention feature-selection mechanism that adaptively dilutes non-discriminative features. We demonstrate the effectiveness of our approach in meta-learning Boolean functions, and synthetic and real-world few-shot learning tasks.


page 2

page 4

page 5

page 6

page 10

page 11

page 12

page 13


Meta-Meta-Classification for One-Shot Learning

We present a new approach, called meta-meta-classification, to learning ...

Learned Fine-Tuner for Incongruous Few-Shot Learning

Model-agnostic meta-learning (MAML) effectively meta-learns an initializ...

Shallow Bayesian Meta Learning for Real-World Few-Shot Recognition

Current state-of-the-art few-shot learners focus on developing effective...

Meta-Learning with Differentiable Convex Optimization

Many meta-learning approaches for few-shot learning rely on simple base ...

Representation based and Attention augmented Meta learning

Deep learning based computer vision fails to work when labeled images ar...

A Framework of Meta Functional Learning for Regularising Knowledge Transfer

Machine learning classifiers' capability is largely dependent on the sca...

Time series model selection with a meta-learning approach; evidence from a pool of forecasting algorithms

One of the challenging questions in time series forecasting is how to fi...