KINN: Incorporating Expert Knowledge in Neural Networks

02/15/2019
by   Muhammad Ali Chattha, et al.
0

The promise of ANNs to automatically discover and extract useful features/patterns from data without dwelling on domain expertise although seems highly promising but comes at the cost of high reliance on large amount of accurately labeled data, which is often hard to acquire and formulate especially in time-series domains like anomaly detection, natural disaster management, predictive maintenance and healthcare. As these networks completely rely on data and ignore a very important modality i.e. expert, they are unable to harvest any benefit from the expert knowledge, which in many cases is very useful. In this paper, we try to bridge the gap between these data driven and expert knowledge based systems by introducing a novel framework for incorporating expert knowledge into the network (KINN). Integrating expert knowledge into the network has three key advantages: (a) Reduction in the amount of data needed to train the model, (b) provision of a lower bound on the performance of the resulting classifier by obtaining the best of both worlds, and (c) improved convergence of model parameters (model converges in smaller number of epochs). Although experts are extremely good in solving different tasks, there are some trends and patterns, which are usually hidden only in the data. Therefore, KINN employs a novel residual knowledge incorporation scheme, which can automatically determine the quality of the predictions made by the expert and rectify it accordingly by learning the trends/patterns from data. Specifically, the method tries to use information contained in one modality to complement information missed by the other. We evaluated KINN on a real world traffic flow prediction problem. KINN significantly superseded performance of both the expert and as well as the base network (LSTM in this case) when evaluated in isolation, highlighting its superiority for the task.

READ FULL TEXT
research
02/08/2022

KENN: Enhancing Deep Neural Networks by Leveraging Knowledge for Time Series Forecasting

End-to-end data-driven machine learning methods often have exuberant req...
research
08/31/2020

Anomaly Detection by Recombining Gated Unsupervised Experts

Inspired by mixture-of-experts models and the analysis of the hidden act...
research
05/06/2018

Incorporating Privileged Information to Unsupervised Anomaly Detection

We introduce a new unsupervised anomaly detection ensemble called SPI wh...
research
08/13/2020

A Survey on Knowledge integration techniques with Artificial Neural Networks for seq-2-seq/time series models

In recent years, with the advent of massive computational power and the ...
research
08/26/2021

Human readable network troubleshooting based on anomaly detection and feature scoring

Network troubleshooting is still a heavily human-intensive process. To r...
research
12/10/2016

Knowledge Elicitation via Sequential Probabilistic Inference for High-Dimensional Prediction

Prediction in a small-sized sample with a large number of covariates, th...
research
12/02/2019

An Attribute Oriented Induction based Methodology for Data Driven Predictive Maintenance

Attribute Oriented Induction (AOI) is a data mining algorithm used for e...

Please sign up or login with your details

Forgot password? Click here to reset