DeepAI
Log In Sign Up

Meta-Learning with Temporal Convolutions

07/11/2017
by   Nikhil Mishra, et al.
0

Deep neural networks excel in regimes with large amounts of data, but tend to struggle when data is scarce or when they need to adapt quickly to changes in the task. Recent work in meta-learning seeks to overcome this shortcoming by training a meta-learner on a distribution of similar tasks; the goal is for the meta-learner to generalize to novel but related tasks by learning a high-level strategy that captures the essence of the problem it is asked to solve. However, most recent approaches to meta-learning are extensively hand-designed, either using architectures that are specialized to a particular application, or hard-coding algorithmic components that tell the meta-learner how to solve the task. We propose a class of simple and generic meta-learner architectures, based on temporal convolutions, that is domain- agnostic and has no particular strategy or algorithm encoded into it. We validate our temporal-convolution-based meta-learner (TCML) through experiments pertaining to both supervised and reinforcement learning, and demonstrate that it outperforms state-of-the-art methods that are less general and more complex.

READ FULL TEXT

page 12

page 13

10/27/2021

Accelerating Gradient-based Meta Learner

Meta Learning has been in focus in recent years due to the meta-learner ...
06/05/2021

Signal Transformer: Complex-valued Attention and Meta-Learning for Signal Recognition

Deep neural networks have been shown as a class of useful tools for addr...
10/18/2021

Provable Hierarchy-Based Meta-Reinforcement Learning

Hierarchical reinforcement learning (HRL) has seen widespread interest a...
11/13/2020

Testing the Genomic Bottleneck Hypothesis in Hebbian Meta-Learning

Recent work has shown promising results using Hebbian meta-learning to s...
06/17/2019

Coupling Retrieval and Meta-Learning for Context-Dependent Semantic Parsing

In this paper, we present an approach to incorporate retrieved datapoint...
09/20/2018

Learning Quickly to Plan Quickly Using Modular Meta-Learning

Multi-object manipulation problems in continuous state and action spaces...
10/31/2021

Can we learn gradients by Hamiltonian Neural Networks?

In this work, we propose a meta-learner based on ODE neural networks tha...

Code Repositories

keita

My personal toolkit for PyTorch development.


view repo