Neuro-Symbolic Constraint Programming for Structured Prediction

by   Paolo Dragone, et al.

We propose Nester, a method for injecting neural networks into constrained structured predictors. The job of the neural network(s) is to compute an initial, raw prediction that is compatible with the input data but does not necessarily satisfy the constraints. The structured predictor then builds a structure using a constraint solver that assembles and corrects the raw predictions in accordance with hard and soft constraints. In doing so, Nester takes advantage of the features of its two components: the neural network learns complex representations from low-level data while the constraint programming component reasons about the high-level properties of the prediction task. The entire architecture can be trained in an end-to-end fashion. An empirical evaluation on handwritten equation recognition shows that Nester achieves better performance than both the neural network and the constrained structured predictor on their own, especially when training examples are scarce, while scaling to more complex problems than other neuro-programming approaches. Nester proves especially useful to reduce errors at the semantic level of the problem, which is particularly challenging for neural network architectures.Sub


page 3

page 5

page 8


Semantic Probabilistic Layers for Neuro-Symbolic Learning

We design a predictive layer for structured-output prediction (SOP) that...

Hello Edge: Keyword Spotting on Microcontrollers

Keyword spotting (KWS) is a critical component for enabling speech based...

Physical Constraint Embedded Neural Networks for inference and noise regulation

Neural networks often require large amounts of data to generalize and ca...

Constraint-based graph network simulator

In the rapidly advancing area of learned physical simulators, nearly all...

NSL: Hybrid Interpretable Learning From Noisy Raw Data

Inductive Logic Programming (ILP) systems learn generalised, interpretab...

Extending Answer Set Programs with Neural Networks

The integration of low-level perception with high-level reasoning is one...

Understanding Interlocking Dynamics of Cooperative Rationalization

Selective rationalization explains the prediction of complex neural netw...