Mixture of Linear Models Co-supervised by Deep Neural Networks

08/05/2021
by   Beomseok Seo, et al.
0

Deep neural network (DNN) models have achieved phenomenal success for applications in many domains, ranging from academic research in science and engineering to industry and business. The modeling power of DNN is believed to have come from the complexity and over-parameterization of the model, which on the other hand has been criticized for the lack of interpretation. Although certainly not true for every application, in some applications, especially in economics, social science, healthcare industry, and administrative decision making, scientists or practitioners are resistant to use predictions made by a black-box system for multiple reasons. One reason is that a major purpose of a study can be to make discoveries based upon the prediction function, e.g., to reveal the relationships between measurements. Another reason can be that the training dataset is not large enough to make researchers feel completely sure about a purely data-driven result. Being able to examine and interpret the prediction function will enable researchers to connect the result with existing knowledge or gain insights about new directions to explore. Although classic statistical models are much more explainable, their accuracy often falls considerably below DNN. In this paper, we propose an approach to fill the gap between relatively simple explainable models and DNN such that we can more flexibly tune the trade-off between interpretability and accuracy. Our main idea is a mixture of discriminative models that is trained with the guidance from a DNN. Although mixtures of discriminative models have been studied before, our way of generating the mixture is quite different.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/22/2023

Breaking the Paradox of Explainable Deep Learning

Deep Learning has achieved tremendous results by pushing the frontier of...
research
03/19/2018

Towards Explanation of DNN-based Prediction with Guided Feature Inversion

While deep neural networks (DNN) have become an effective computational ...
research
06/27/2022

Thermodynamics of Interpretation

Over the past few years, different types of data-driven Artificial Intel...
research
04/26/2023

A Survey on Solving and Discovering Differential Equations Using Deep Neural Networks

Ordinary and partial differential equations (DE) are used extensively in...
research
06/18/2022

Machine Learning in Sports: A Case Study on Using Explainable Models for Predicting Outcomes of Volleyball Matches

Machine Learning has become an integral part of engineering design and d...
research
09/16/2016

Rule Extraction Algorithm for Deep Neural Networks: A Review

Despite the highest classification accuracy in wide varieties of applica...
research
02/12/2015

Over-Sampling in a Deep Neural Network

Deep neural networks (DNN) are the state of the art on many engineering ...

Please sign up or login with your details

Forgot password? Click here to reset