DeepAI AI Chat
Log In Sign Up

The Omniglot Challenge: A 3-Year Progress Report

02/09/2019
by   Brenden M. Lake, et al.
6

Three years ago, we released the Omniglot dataset for developing more human-like learning algorithms. Omniglot is a one-shot learning challenge, inspired by how people can learn a new concept from just one or a few examples. Along with the dataset, we proposed a suite of five challenge tasks and a computational model based on probabilistic program induction that addresses them. The computational model, although powerful, was not meant to be the final word on Omniglot; we hoped that the machine learning community would both build on our work and develop novel approaches to tackling the challenge. In the time since, we have been pleased to see the wide adoption of Omniglot and notable technical progress. There has been genuine progress on one-shot classification, but it has been difficult to measure since researchers have adopted different splits and training procedures that make the task easier. The other four tasks, while essential components of human conceptual understanding, have received considerably less attention. We review the progress so far and conclude that neural networks are still far from human-like concept learning on Omniglot, a challenge that requires performing all of the tasks with a single model. We also discuss new tasks to stimulate further progress.

READ FULL TEXT

page 1

page 2

page 3

page 4

07/19/2020

One-Shot Learning for Language Modelling

Humans can infer a great deal about the meaning of a word, using the syn...
05/24/2019

Human vs. Muppet: A Conservative Estimate of Human Performance on the GLUE Benchmark

The GLUE benchmark (Wang et al., 2019b) is a suite of language understan...
07/14/2020

Concept Learners for Generalizable Few-Shot Learning

Developing algorithms that are able to generalize to a novel task given ...
05/24/2019

Human vs. Muppet: A Conservative Estimate of HumanPerformance on the GLUE Benchmark

The GLUE benchmark (Wang et al., 2019b) is a suite of language understan...
09/17/2020

FewJoint: A Few-shot Learning Benchmark for Joint Language Understanding

Few-learn learning (FSL) is one of the key future steps in machine learn...
04/29/2020

Zero-Shot Learning and its Applications from Autonomous Vehicles to COVID-19 Diagnosis: A Review

The challenge of learning a new concept without receiving any examples b...
08/04/2018

Deep Reinforcement One-Shot Learning for Artificially Intelligent Classification Systems

In recent years there has been a sharp rise in networking applications, ...

Code Repositories

omniglot

Omniglot data set for one-shot learning


view repo

siamese-networks-tf

Implementation of Siamese-Networks for One Shot Learning in TensorFlow 2.0


view repo

matching-networks-tf

Implementation of Matching Networks for One Shot Learning in TensorFlow 2.0


view repo

oneshot-rs

A Rust implementation of Siamese Neural Networks for One-shot Image Recognition for benchmarking performance and results.


view repo