A Generalist Neural Algorithmic Learner

09/22/2022
by   Borja Ibarz, et al.
6

The cornerstone of neural algorithmic reasoning is the ability to solve algorithmic tasks, especially in a way that generalises out of distribution. While recent years have seen a surge in methodological improvements in this area, they mostly focused on building specialist models. Specialist models are capable of learning to neurally execute either only one algorithm or a collection of algorithms with identical control-flow backbone. Here, instead, we focus on constructing a generalist neural algorithmic learner – a single graph neural network processor capable of learning to execute a wide range of algorithms, such as sorting, searching, dynamic programming, path-finding and geometry. We leverage the CLRS benchmark to empirically show that, much like recent successes in the domain of perception, generalist algorithmic learners can be built by "incorporating" knowledge. That is, it is possible to effectively learn algorithms in a multi-task manner, so long as we can learn to execute them well in a single-task regime. Motivated by this, we present a series of improvements to the input representation, training regime and processor architecture over CLRS, improving average single-task performance by over 20 learners leveraging these improvements. Our results demonstrate a generalist learner that effectively incorporates knowledge captured by specialist models.

READ FULL TEXT

page 8

page 9

page 15

research
05/31/2022

The CLRS Algorithmic Reasoning Benchmark

Learning representations of algorithms is an emerging area of machine le...
research
07/18/2020

MTL2L: A Context Aware Neural Optimiser

Learning to learn (L2L) trains a meta-learner to assist the learning of ...
research
10/26/2021

How to transfer algorithmic reasoning knowledge to learn new algorithms?

Learning to execute algorithms is a fundamental problem that has been wi...
research
06/23/2023

Neural Algorithmic Reasoning Without Intermediate Supervision

Neural Algorithmic Reasoning is an emerging area of machine learning foc...
research
05/30/2019

What Can Neural Networks Reason About?

Neural networks have successfully been applied to solving reasoning task...
research
03/29/2022

Graph Neural Networks are Dynamic Programmers

Recent advances in neural algorithmic reasoning with graph neural networ...
research
02/20/2023

Neural Algorithmic Reasoning with Causal Regularisation

Recent work on neural algorithmic reasoning has investigated the reasoni...

Please sign up or login with your details

Forgot password? Click here to reset