Evaluating Relaxations of Logic for Neural Networks: A Comprehensive Study

07/28/2021
by   Mattia Medina Grespan, et al.
12

Symbolic knowledge can provide crucial inductive bias for training neural models, especially in low data regimes. A successful strategy for incorporating such knowledge involves relaxing logical statements into sub-differentiable losses for optimization. In this paper, we study the question of how best to relax logical expressions that represent labeled examples and knowledge about a problem; we focus on sub-differentiable t-norm relaxations of logic. We present theoretical and empirical criteria for characterizing which relaxation would perform best in various scenarios. In our theoretical study driven by the goal of preserving tautologies, the Lukasiewicz t-norm performs best. However, in our empirical analysis on the text chunking and digit recognition tasks, the product t-norm achieves best predictive performance. We analyze this apparent discrepancy, and conclude with a list of best practices for defining loss functions via logic.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/14/2022

Reduced Implication-bias Logic Loss for Neuro-Symbolic Learning

Integrating logical reasoning and machine learning by approximating logi...
research
06/14/2019

Augmenting Neural Networks with First-order Logic

Today, the dominant paradigm for training neural networks involves minim...
research
03/19/2023

Logic of Differentiable Logics: Towards a Uniform Semantics of DL

Differentiable logics (DL) have recently been proposed as a method of tr...
research
02/14/2020

Analyzing Differentiable Fuzzy Logic Operators

In recent years there has been a push to integrate symbolic AI and deep ...
research
10/06/2019

Semantic Interpretation of Deep Neural Networks Based on Continuous Logic

Combining deep neural networks with the concepts of continuous logic is ...
research
06/26/2023

logLTN: Differentiable Fuzzy Logic in the Logarithm Space

The AI community is increasingly focused on merging logic with deep lear...

Please sign up or login with your details

Forgot password? Click here to reset