Attentional meta-learners are polythetic classifiers

06/09/2021
by   Ben Day, et al.
5

Polythetic classifications, based on shared patterns of features that need neither be universal nor constant among members of a class, are common in the natural world and greatly outnumber monothetic classifications over a set of features. We show that threshold meta-learners require an embedding dimension that is exponential in the number of features to emulate these functions. In contrast, attentional classifiers are polythetic by default and able to solve these problems with a linear embedding dimension. However, we find that in the presence of task-irrelevant features, inherent to meta-learning problems, attentional models are susceptible to misclassification. To address this challenge, we further propose a self-attention feature-selection mechanism that adaptively dilutes non-discriminative features. We demonstrate the effectiveness of our approach in meta-learning Boolean functions, and synthetic and real-world few-shot learning tasks.

READ FULL TEXT

page 2

page 4

page 5

page 6

page 10

page 11

page 12

page 13

04/17/2020

Meta-Meta-Classification for One-Shot Learning

We present a new approach, called meta-meta-classification, to learning ...
09/29/2020

Learned Fine-Tuner for Incongruous Few-Shot Learning

Model-agnostic meta-learning (MAML) effectively meta-learns an initializ...
01/08/2021

Shallow Bayesian Meta Learning for Real-World Few-Shot Recognition

Current state-of-the-art few-shot learners focus on developing effective...
04/07/2019

Meta-Learning with Differentiable Convex Optimization

Many meta-learning approaches for few-shot learning rely on simple base ...
11/19/2018

Representation based and Attention augmented Meta learning

Deep learning based computer vision fails to work when labeled images ar...
03/28/2022

A Framework of Meta Functional Learning for Regularising Knowledge Transfer

Machine learning classifiers' capability is largely dependent on the sca...
08/22/2019

Time series model selection with a meta-learning approach; evidence from a pool of forecasting algorithms

One of the challenging questions in time series forecasting is how to fi...