DeepAI AI Chat
Log In Sign Up

Learning to Deceive Knowledge Graph Augmented Models via Targeted Perturbation

by   Mrigank Raman, et al.

Symbolic knowledge (e.g., entities, relations, and facts in a knowledge graph) has become an increasingly popular component of neural-symbolic models applied to machine learning tasks, such as question answering and recommender systems. Besides improving downstream performance, these symbolic structures (and their associated attention weights) are often used to help explain the model's predictions and provide "insights" to practitioners. In this paper, we question the faithfulness of such symbolic explanations. We demonstrate that, through a learned strategy (or even simple heuristics), one can produce deceptively perturbed symbolic structures which maintain the downstream performance of the original structure while significantly deviating from the original semantics. In particular, we train a reinforcement learning policy to manipulate relation types or edge connections in a knowledge graph, such that the resulting downstream performance is maximally preserved. Across multiple models and tasks, our approach drastically alters knowledge graphs with little to no drop in performance. These results raise doubts about the faithfulness of explanations provided by learned symbolic structures and the reliability of current neural-symbolic models in leveraging symbolic knowledge.


page 1

page 2

page 3

page 4


Neural-Symbolic Reasoning on Knowledge Graphs

Knowledge graph reasoning is the fundamental component to support machin...

Towards Data Poisoning Attack against Knowledge Graph Embedding

Knowledge graph embedding (KGE) is a technique for learning continuous e...

Computing Rule-Based Explanations of Machine Learning Classifiers using Knowledge Graphs

The use of symbolic knowledge representation and reasoning as a way to r...

Neural-Symbolic Reasoning over Knowledge Graph for Multi-stage Explainable Recommendation

Recent work on recommender systems has considered external knowledge gra...

Generalizable Neuro-symbolic Systems for Commonsense Question Answering

This chapter illustrates how suitable neuro-symbolic models for language...

Generalization to Novel Objects using Prior Relational Knowledge

To solve tasks in new environments involving objects unseen during train...

Knowledge Infused Learning (K-IL): Towards Deep Incorporation of Knowledge in Deep Learning

Learning the underlying patterns in the data goes beyond instance-based ...

Code Repositories


An implementation of the experiments on KG robustness

view repo