Log In Sign Up

Meta-Learning with Temporal Convolutions

by   Nikhil Mishra, et al.

Deep neural networks excel in regimes with large amounts of data, but tend to struggle when data is scarce or when they need to adapt quickly to changes in the task. Recent work in meta-learning seeks to overcome this shortcoming by training a meta-learner on a distribution of similar tasks; the goal is for the meta-learner to generalize to novel but related tasks by learning a high-level strategy that captures the essence of the problem it is asked to solve. However, most recent approaches to meta-learning are extensively hand-designed, either using architectures that are specialized to a particular application, or hard-coding algorithmic components that tell the meta-learner how to solve the task. We propose a class of simple and generic meta-learner architectures, based on temporal convolutions, that is domain- agnostic and has no particular strategy or algorithm encoded into it. We validate our temporal-convolution-based meta-learner (TCML) through experiments pertaining to both supervised and reinforcement learning, and demonstrate that it outperforms state-of-the-art methods that are less general and more complex.


page 12

page 13


Accelerating Gradient-based Meta Learner

Meta Learning has been in focus in recent years due to the meta-learner ...

Signal Transformer: Complex-valued Attention and Meta-Learning for Signal Recognition

Deep neural networks have been shown as a class of useful tools for addr...

Provable Hierarchy-Based Meta-Reinforcement Learning

Hierarchical reinforcement learning (HRL) has seen widespread interest a...

Testing the Genomic Bottleneck Hypothesis in Hebbian Meta-Learning

Recent work has shown promising results using Hebbian meta-learning to s...

Coupling Retrieval and Meta-Learning for Context-Dependent Semantic Parsing

In this paper, we present an approach to incorporate retrieved datapoint...

Learning Quickly to Plan Quickly Using Modular Meta-Learning

Multi-object manipulation problems in continuous state and action spaces...

Can we learn gradients by Hamiltonian Neural Networks?

In this work, we propose a meta-learner based on ODE neural networks tha...

Code Repositories


My personal toolkit for PyTorch development.

view repo