DeepAI AI Chat
Log In Sign Up

Mixture of Linear Models Co-supervised by Deep Neural Networks

by   Beomseok Seo, et al.

Deep neural network (DNN) models have achieved phenomenal success for applications in many domains, ranging from academic research in science and engineering to industry and business. The modeling power of DNN is believed to have come from the complexity and over-parameterization of the model, which on the other hand has been criticized for the lack of interpretation. Although certainly not true for every application, in some applications, especially in economics, social science, healthcare industry, and administrative decision making, scientists or practitioners are resistant to use predictions made by a black-box system for multiple reasons. One reason is that a major purpose of a study can be to make discoveries based upon the prediction function, e.g., to reveal the relationships between measurements. Another reason can be that the training dataset is not large enough to make researchers feel completely sure about a purely data-driven result. Being able to examine and interpret the prediction function will enable researchers to connect the result with existing knowledge or gain insights about new directions to explore. Although classic statistical models are much more explainable, their accuracy often falls considerably below DNN. In this paper, we propose an approach to fill the gap between relatively simple explainable models and DNN such that we can more flexibly tune the trade-off between interpretability and accuracy. Our main idea is a mixture of discriminative models that is trained with the guidance from a DNN. Although mixtures of discriminative models have been studied before, our way of generating the mixture is quite different.


page 1

page 2

page 3

page 4


Breaking the Paradox of Explainable Deep Learning

Deep Learning has achieved tremendous results by pushing the frontier of...

Towards Explanation of DNN-based Prediction with Guided Feature Inversion

While deep neural networks (DNN) have become an effective computational ...

ML Interpretability: Simple Isn't Easy

The interpretability of ML models is important, but it is not clear what...

A Survey on Solving and Discovering Differential Equations Using Deep Neural Networks

Ordinary and partial differential equations (DE) are used extensively in...

Over-Sampling in a Deep Neural Network

Deep neural networks (DNN) are the state of the art on many engineering ...

Modeling extra-deep EM logs using a deep neural network

Modern geosteering is heavily dependent on real-time interpretation of d...