DeepAI AI Chat
Log In Sign Up

Relational inductive biases, deep learning, and graph networks

by   Peter W. Battaglia, et al.

Artificial intelligence (AI) has undergone a renaissance recently, making major progress in key domains such as vision, language, control, and decision-making. This has been due, in part, to cheap data and cheap compute resources, which have fit the natural strengths of deep learning. However, many defining characteristics of human intelligence, which developed under much different pressures, remain out of reach for current approaches. In particular, generalizing beyond one's experiences--a hallmark of human intelligence from infancy--remains a formidable challenge for modern AI. The following is part position paper, part review, and part unification. We argue that combinatorial generalization must be a top priority for AI to achieve human-like abilities, and that structured representations and computations are key to realizing this objective. Just as biology uses nature and nurture cooperatively, we reject the false choice between "hand-engineering" and "end-to-end" learning, and instead advocate for an approach which benefits from their complementary strengths. We explore how using relational inductive biases within deep learning architectures can facilitate learning about entities, relations, and rules for composing them. We present a new building block for the AI toolkit with a strong relational inductive bias--the graph network--which generalizes and extends various approaches for neural networks that operate on graphs, and provides a straightforward interface for manipulating structured knowledge and producing structured behaviors. We discuss how graph networks can support relational reasoning and combinatorial generalization, laying the foundation for more sophisticated, interpretable, and flexible patterns of reasoning.


page 4

page 5

page 8

page 11

page 15

page 20


On Neural Architecture Inductive Biases for Relational Tasks

Current deep learning approaches have shown good in-distribution general...

Interpretable deep-learning models to help achieve the Sustainable Development Goals

We discuss our insights into interpretable artificial-intelligence (AI) ...

Inductive Biases for Deep Learning of Higher-Level Cognition

A fascinating hypothesis is that human and animal intelligence could be ...

Target Languages (vs. Inductive Biases) for Learning to Act and Plan

Recent breakthroughs in AI have shown the remarkable power of deep learn...

Relational inductive bias for physical construction in humans and machines

While current deep learning systems excel at tasks such as object classi...

On the Binding Problem in Artificial Neural Networks

Contemporary neural networks still fall short of human-level generalizat...

On Inductive Abilities of Latent Factor Models for Relational Learning

Latent factor models are increasingly popular for modeling multi-relatio...