Relational inductive biases, deep learning, and graph networks

06/04/2018
by   Peter W. Battaglia, et al.
0

Artificial intelligence (AI) has undergone a renaissance recently, making major progress in key domains such as vision, language, control, and decision-making. This has been due, in part, to cheap data and cheap compute resources, which have fit the natural strengths of deep learning. However, many defining characteristics of human intelligence, which developed under much different pressures, remain out of reach for current approaches. In particular, generalizing beyond one's experiences--a hallmark of human intelligence from infancy--remains a formidable challenge for modern AI. The following is part position paper, part review, and part unification. We argue that combinatorial generalization must be a top priority for AI to achieve human-like abilities, and that structured representations and computations are key to realizing this objective. Just as biology uses nature and nurture cooperatively, we reject the false choice between "hand-engineering" and "end-to-end" learning, and instead advocate for an approach which benefits from their complementary strengths. We explore how using relational inductive biases within deep learning architectures can facilitate learning about entities, relations, and rules for composing them. We present a new building block for the AI toolkit with a strong relational inductive bias--the graph network--which generalizes and extends various approaches for neural networks that operate on graphs, and provides a straightforward interface for manipulating structured knowledge and producing structured behaviors. We discuss how graph networks can support relational reasoning and combinatorial generalization, laying the foundation for more sophisticated, interpretable, and flexible patterns of reasoning.

READ FULL TEXT

page 4

page 5

page 8

page 11

page 15

page 20

research
06/09/2022

On Neural Architecture Inductive Biases for Relational Tasks

Current deep learning approaches have shown good in-distribution general...
research
08/24/2021

Interpretable deep-learning models to help achieve the Sustainable Development Goals

We discuss our insights into interpretable artificial-intelligence (AI) ...
research
11/30/2020

Inductive Biases for Deep Learning of Higher-Level Cognition

A fascinating hypothesis is that human and animal intelligence could be ...
research
09/15/2021

Target Languages (vs. Inductive Biases) for Learning to Act and Plan

Recent breakthroughs in AI have shown the remarkable power of deep learn...
research
06/04/2018

Relational inductive bias for physical construction in humans and machines

While current deep learning systems excel at tasks such as object classi...
research
12/09/2020

On the Binding Problem in Artificial Neural Networks

Contemporary neural networks still fall short of human-level generalizat...
research
09/17/2017

On Inductive Abilities of Latent Factor Models for Relational Learning

Latent factor models are increasingly popular for modeling multi-relatio...

Please sign up or login with your details

Forgot password? Click here to reset