DeepAI AI Chat
Log In Sign Up

Meta Learning in Decentralized Neural Networks: Towards More General AI

by   Yuwei Sun, et al.
The University of Tokyo

Meta-learning usually refers to a learning algorithm that learns from other learning algorithms. The problem of uncertainty in the predictions of neural networks shows that the world is only partially predictable and a learned neural network cannot generalize to its ever-changing surrounding environments. Therefore, the question is how a predictive model can represent multiple predictions simultaneously. We aim to provide a fundamental understanding of learning to learn in the contents of Decentralized Neural Networks (Decentralized NNs) and we believe this is one of the most important questions and prerequisites to building an autonomous intelligence machine. To this end, we shall demonstrate several pieces of evidence for tackling the problems above with Meta Learning in Decentralized NNs. In particular, we will present three different approaches to building such a decentralized learning system: (1) learning from many replica neural networks, (2) building the hierarchy of neural networks for different functions, and (3) leveraging different modality experts to learn cross-modal representations.


page 1

page 2


Eliminating Meta Optimization Through Self-Referential Meta Learning

Meta Learning automates the search for learning algorithms. At the same ...

Meta-Learning in Neural Networks: A Survey

The field of meta-learning, or learning-to-learn, has seen a dramatic ri...

A Brief Survey of Associations Between Meta-Learning and General AI

This paper briefly reviews the history of meta-learning and describes it...

Neural Routing in Meta Learning

Meta-learning often referred to as learning-to-learn is a promising noti...

Meta-Calibration: Meta-Learning of Model Calibration Using Differentiable Expected Calibration Error

Calibration of neural networks is a topical problem that is becoming inc...

Dif-MAML: Decentralized Multi-Agent Meta-Learning

The objective of meta-learning is to exploit the knowledge obtained from...

Warm Up Cold-start Advertisements: Improving CTR Predictions via Learning to Learn ID Embeddings

Click-through rate (CTR) prediction has been one of the most central pro...