Inductive Biases for Deep Learning of Higher-Level Cognition

by   Anirudh Goyal, et al.

A fascinating hypothesis is that human and animal intelligence could be explained by a few principles (rather than an encyclopedic list of heuristics). If that hypothesis was correct, we could more easily both understand our own intelligence and build intelligent machines. Just like in physics, the principles themselves would not be sufficient to predict the behavior of complex systems like brains, and substantial computation might be needed to simulate human-like intelligence. This hypothesis would suggest that studying the kind of inductive biases that humans and animals exploit could help both clarify these principles and provide inspiration for AI research and neuroscience theories. Deep learning already exploits several key inductive biases, and this work considers a larger list, focusing on those which concern mostly higher-level and sequential conscious processing. The objective of clarifying these particular principles is that they could potentially help us build AI systems benefiting from humans' abilities in terms of flexible out-of-distribution and systematic generalization, which is currently an area where a large gap exists between state-of-the-art machine learning and human intelligence.


page 1

page 2

page 3

page 4


Human-Level Intelligence or Animal-Like Abilities?

The vision systems of the eagle and the snake outperform everything that...

Relational inductive biases, deep learning, and graph networks

Artificial intelligence (AI) has undergone a renaissance recently, makin...

A Novel Approach for Semiconductor Etching Process with Inductive Biases

The etching process is one of the most important processes in semiconduc...

The Achilles Heel Hypothesis: Pitfalls for AI Systems via Decision Theoretic Adversaries

As progress in AI continues to advance at a rapid pace, it is crucial to...

Model-agnostic Fits for Understanding Information Seeking Patterns in Humans

In decision making tasks under uncertainty, humans display characteristi...

The Relativity of Induction

Lately there has been a lot of discussion about why deep learning algori...

Grand challenges in social physics: In pursuit of moral behavior

Methods of statistical physics have proven valuable for studying the evo...

Code Repositories