Strong Generalization and Efficiency in Neural Programs

by   Yujia Li, et al.

We study the problem of learning efficient algorithms that strongly generalize in the framework of neural program induction. By carefully designing the input / output interfaces of the neural model and through imitation, we are able to learn models that produce correct results for arbitrary input sizes, achieving strong generalization. Moreover, by using reinforcement learning, we optimize for program efficiency metrics, and discover new algorithms that surpass the teacher used in imitation. With this, our approach can learn to outperform custom-written solutions for a variety of problems, as we tested it on sorting, searching in ordered lists and the NP-complete 0/1 knapsack problem, which sets a notable milestone in the field of Neural Program Induction. As highlights, our learned model can perform sorting perfectly on any input data size we tested on, with O(n log n) complexity, whilst outperforming hand-coded algorithms, including quick sort, in number of operations even for list sizes far beyond those seen during training.


Neural Program Meta-Induction

Most recently proposed methods for Neural Program Induction work under t...

Learning Neural Programs To Parse Programs

In this work, we study an important problem: learning programs from inpu...

Neural Execution Engines: Learning to Execute Subroutines

A significant effort has been made to train neural networks that replica...

Deriving sorting algorithms via abductive logic program transformation

Logic program transformation by the unfold/fold method ad- vocates the w...

Proving Properties of Sorting Programs: A Case Study in Horn Clause Verification

The proof of a program property can be reduced to the proof of satisfiab...

Representational efficiency outweighs action efficiency in human program induction

The importance of hierarchically structured representations for tractabl...

Divide and Conquer Networks

We consider the learning of algorithmic tasks by mere observation of inp...

Please sign up or login with your details

Forgot password? Click here to reset