Extracting Grammars from a Neural Network Parser for Anomaly Detection in Unknown Formats

by   Alexander Grushin, et al.
Galois, Inc.

Reinforcement learning has recently shown promise as a technique for training an artificial neural network to parse sentences in some unknown format. A key aspect of this approach is that rather than explicitly inferring a grammar that describes the format, the neural network learns to perform various parsing actions (such as merging two tokens) over a corpus of sentences, with the goal of maximizing the total reward, which is roughly based on the estimated frequency of the resulting parse structures. This can allow the learning process to more easily explore different action choices, since a given choice may change the optimality of the parse (as expressed by the total reward), but will not result in the failure to parse a sentence. However, the approach also exhibits limitations: first, the neural network does not provide production rules for the grammar that it uses during parsing; second, because this neural network can successfully parse any sentence, it cannot be directly used to identify sentences that deviate from the format of the training sentences, i.e., that are anomalous. In this paper, we address these limitations by presenting procedures for extracting production rules from the neural network, and for using these rules to determine whether a given sentence is nominal or anomalous, when compared to structures observed within training data. In the latter case, an attempt is made to identify the location of the anomaly. Additionally, a two pass mechanism is presented for dealing with formats containing high-entropy information. We empirically evaluate the approach on artificial formats, demonstrating effectiveness, but also identifying limitations. By further improving parser learning, and leveraging rule extraction and anomaly detection, one might begin to understand common errors, either benign or malicious, in practical formats.


page 1

page 2

page 3

page 4


Interactive System-wise Anomaly Detection

Anomaly detection, where data instances are discovered containing featur...

Prosodic features improve sentence segmentation and parsing

Parsing spoken dialogue presents challenges that parsing text does not, ...

Learning grammar with a divide-and-concur neural network

We implement a divide-and-concur iterative projection approach to contex...

AMR Parsing with Action-Pointer Transformer

Abstract Meaning Representation parsing is a sentence-to-graph predictio...

Semi-Supervised Anomaly Detection - Towards Model-Independent Searches of New Physics

Most classification algorithms used in high energy physics fall under th...

Structured Training for Neural Network Transition-Based Parsing

We present structured perceptron training for neural network transition-...

Please sign up or login with your details

Forgot password? Click here to reset