Synthesis of Differentiable Functional Programs for Lifelong Learning

03/31/2018
by   Lazar Valkov, et al.
0

We present a neurosymbolic approach to the lifelong learning of algorithmic tasks that mix perception and procedural reasoning. Reusing highlevel concepts across domains and learning complex procedures are two key challenges in lifelong learning. We show that a combination of gradientbased learning and symbolic program synthesis can be a more effective response to these challenges than purely neural methods. Concretely, our approach, called HOUDINI, represents neural networks as strongly typed, end-to-end differentiable functional programs that use symbolic higher-order combinators to compose a library of neural functions. Our learning algorithm consists of: (1) a program synthesizer that performs a type-directed search over programs in this language, and decides on the library functions that should be reused and the architectures that should be used to combine them; and (2) a neural module that trains synthesized programs using stochastic gradient descent. We evaluate our approach on three algorithmic tasks. Our experiments show that our type-directed search technique is able to significantly prune the search space of programs, and that the overall approach transfers high-level concepts more effectively than monolithic neural networks as well as traditional transfer learning.

READ FULL TEXT
research
06/13/2022

From Perception to Programs: Regularize, Overparameterize, and Amortize

Toward combining inductive reasoning with perception abilities, we devel...
research
07/23/2020

Learning Differentiable Programs with Admissible Neural Heuristics

We study the problem of learning differentiable functions expressed as p...
research
06/15/2020

DreamCoder: Growing generalizable, interpretable knowledge with wake-sleep Bayesian program learning

Expert problem-solving is driven by powerful languages for thinking abou...
research
11/22/2020

Learning a Deep Generative Model like a Program: the Free Category Prior

Humans surpass the cognitive abilities of most other animals in our abil...
research
06/03/2023

LambdaBeam: Neural Program Search with Higher-Order Functions and Lambdas

Search is an important technique in program synthesis that allows for ad...
research
03/06/2023

Symbolic Synthesis of Neural Networks

Neural networks adapt very well to distributed and continuous representa...
research
03/30/2022

Type-Directed Program Synthesis for RESTful APIs

With the rise of software-as-a-service and microservice architectures, R...

Please sign up or login with your details

Forgot password? Click here to reset